Page 161 -

5.1 LMS Adiusted Discriminants
149
order to obtain the best approximation to the target values, corresponding to
minimize E, we differentiate it with respect to the weights and equalize to zero:
We can write these so-called normal equations corresponding to the least-mean-
square or LMS solution, in a compact form as:
X'XW'= X'T, (5-2~)
where X is a nx(d+l) matrix with the augmented feature vectors, W is a cx(d+l)
matrix of the weights and T is a nxc matrix of the target values. Provided that the
square matrix X'X is non-singular, the weights can be immediately computed as:
The matrix X* = (x'x)-' X' is called the pseudo-inverse of X and satisfies the
property X*X=I.
In order to see how the LMS adjustment of discriminants works, let us consider
a very simple two-class one-dimensional problem with only two points, one from
each class, as shown in Figure 5.2a, where the target values are also indicated. For
adequate graphic inspection in the weight space, we will limit the number of
weights by restricting the analysis to the discriminant that corresponds to the
difference of the linear decision functions:
Let us compute the pseudo-inverse of X:
Since our goal now is to adjust one discriminant d, instead of dl and d2, matrix T
has only one column' with the respective target values, therefore:
I
Since we are using a single discriminant, c=l in this case.