Figure 4.6 Illustration of the soft margin for a linear support vector machine (SVM).
(4.47)
Figure 2.7 of Chapter 2 is modified to reflect better the definitions introduced in this section and presented as Figure 4.6. Often, the optimization problem, Eq. (4.46), can be solved more easily in its dual formulation. The dual formulation provides also the key for extending SV machine to nonlinear functions.
Lagrange function: The key idea here is to construct a Lagrange function from the objective function (primal objective function) and the corresponding constraints, by introducing a dual set of variables [95]. It can be shown that this function has a saddle point with respect to the primal and dual variables at the solution. So we have
Here, L is the Lagrangian, and ηi,
In deriving Eq. (4.49), we already eliminated the dual variables ηi,