Page 159 -

5 Neural Networks

In the previous chapters we learned how to design supervised and unsupervised
classifiers, which had in common the concept of class or cluster separability, based
on a distance measure. Artificial neural networks, or neural nets for short, afford a
means of classifying data and also use distance measures in a model-free approach,
but whereas, previously, class separability was the driving mechanism towards a
solution, we now apply another concept, that of minimizing errors between
obtained outputs and desired target values. Besides statistical considerations,
optimisation techniques play a fundamental role here.

5.1 LMS Adjusted Discriminants

Let us assume that we are searching for a solution in terms of linear discriminants
for a c-class classification problem. We are given n labelled patterns and wish to
use linear decision functions for their classification. Linear discriminants were
already presented in section 2.1; we rewrite here the respective expression 2-2:

Figure 5.1. Connectionist structure of a linear decision function. C is the
processing unit.

As we will, in principle, always use bias weights w0, we will simplify, for the
moment, the notation by calling x the augmented feature vector with an extra