Page 174 - Classification Parameter Estimation & State Estimation An Engg Approach Using MATLAB

NONPARAMETRIC LEARNING 163

Functions of this type are called linear discriminant functions. In fact,
these functions implement a linear machine. See also Section 2.1.2.
The notation can be simplified by the introduction of an augmented
measurement vector y, defined as:

z
y ¼ ð5:37Þ
1
With that, the discriminant functions become:

T
g k ðyÞ¼ w y ð5:38Þ
k
where the scalar w k in (5.36) has been embedded in the vector w k by
augmenting the latter with the extra element w k .
The augmentation can also be used for a generalization that allows for
nonlinear machines. For instance, a quadratic machine is obtained with:

2 2 2 T
yðzÞ¼ z 1 z 0 z 1 ... z N 1 z 0 z 1 z 0 z 2 ... z N 1 z N ð5:39Þ
T
The corresponding functions g k (y) ¼ w y(z) are called generalized linear
k
discriminant functions.
Discriminant functions depend on a set of parameters. In (5.38) these
parameters are the vectors w k . In essence, the learning process boils down
to a search for parameters such that with these parameters the decision
function in (5.35) correctly classifies all samples in the training set.
The basic approach to find the parameters is to define a performance
measure that depends on both the training set and the set of parameters.
Adjustment of the parameters such that the performance measure is
maximized gives the optimal decision function; see Figure 5.7.

decision performance Performance
Training set function measure

Parameters parameter
adjustment

Figure 5.7 Training by means of performance optimization