Page 172 -

160 5 Neural Networks

want w'xi > 0 for patterns xi belonging to class W, and w7xi < 0 for patterns xi
belonging to class @. These two conditions can be written simply as w7xifi > 0,
using the target values. The fact that wlxiti must be positive for correct
classification, suggests the use of the following error function known as the
perceptron criterion:

E(w) =- Xw7xiti .
xi in error

Let us look more closely at this error function. Consider the situation with d = 2
depicted in Figure 5.11, where the unit length normal to the discriminant, n, points
in the positive direction (d(x) > 0), corresponding to class w,.
Let us determine the projection distance 1 - lo of a feature vector x onto the
positive normal n. The projection of d(x) referred to the origin, lo, is given by
-wdllnll '. The length I is the projection of x onto n. Since the components of the
normal vector n to the linear discriminant d(x) are precisely the weights of d(x), n
= (w,, w2) j, we can compute the projection distance l(x) of x onto the normal of
the linear discriminant. as:

This projection I(x) is positive for feature vectors x lying in the positive half
plane and negative otherwise.
Consider now that feature vector x represented in Figure 5.1 1 is wrongly
classified because it has target value -1. As the target value is negative, the
contribution of the pattern to the error (5-14) is a positive value. In the same way, a
feature vector lying in the negative half plane, therefore with I(x) < 0, will
contribute positively to the error if its target value is +1. In general, the
contributions of the wrongly classified patterns to the errors are the Euclidian
distances to the discriminant.
The perceptron compensates for these errors by applying the following learning
rule:

- Pattern correctly classified: do nothing.
- Pattern wrongly classified: add the pattern to the weight vector if the pattern is
from class CU~ (ti = + 1) and subtract it if it is from class w (ti = - 1).

Hence, the increment of the weight vector is:

Aw=tixi for wrong xi. (5-16)

See expressions (2-2d) relative to the distance of d(x) from the origin and the unitary
normal vector pointing into the positive direction.