Page 25 - Biosystems Engineering

6 Cha pte r O n e

In contrast to supervised learning, unsupervised learning discovers
patterns or features in the input data with no help from a teacher,
essentially performing a clustering of the input space. Typical unsu-
pervised learning rules include Hebbian learning, principal compo-
nent learning, and Kohonen learning rules. These rules are applied in
training self-organizing neural network architectures such as self-
organizing maps (SOM). Although neural networks developed for
both the supervised and unsupervised learning paradigms have per-
formed very well in their respective application fields, improvements
have been developed by combining the two paradigms. Examples
include a radial basis function (RBF) network and learning vector
quantization (LVQ) discussed as follows.
RBF networks are special feedforward networks that have a single
hidden layer. The activation functions of the neurons in the hidden
layer are radial basis functions, whereas the neurons in the output
layer have simple linear activation functions. Radial basis functions
are a set of predominantly nonlinear functions such as gaussian
functions that are built up into one function. Each gaussian function
responds only to a small region of the input space where the gauss-
ian is centered. Thus, while an MLP network uses hyperplanes
defined by weighted sums as arguments to sigmoidal functions, an
RBF network uses hyperellipsoids to partition the input space in
which measurements or observations are made. Thus, RBF networks
find the input-to-output map using local approximators. The key to
a successful implementation of RBF networks is to find suitable cen-
ters for the gaussian functions. Theoretically, the centers and width
of the gaussian functions can be determined with supervised learn-
ing, for example, the steepest gradient method. They can also be
determined through unsupervised learning by clustering the training
data points. Once the centers and width of the gaussian functions are
obtained, the weights for the connection between the hidden layer
and the output layer are easily and quickly determined using meth-
ods such as linear least squares, as the output neurons are simple
linear combiners.
LVQ selects training vectors with a known classification and pres-
ents them to the network to examine cases of misclassification. An
LVQ network has a first competitive layer and a second linear layer.
The competitive layer consists of reference vectors that learn to cate-
gorize input vectors. The linear layer transforms the competitive
layer’s classes into target classifications defined by the users. During
training, a class labeled “input vector” (e.g., expression pattern) is
picked at random and is compared with each reference vector. The
input vector is assigned to a class that the most similar reference vec-
tor possesses. The reference vector is moved closer to the input vector
if the assignment was correct; otherwise it is moved away. During
operation, the unknown expression profiles are classified based on
their similarity to the reference vectors.