Page 964 - The Mechatronics Handbook

0066_Frame_C32.fm Page 14 Wednesday, January 9, 2002 7:54 PM

For a neuron in the input layer to be reacting just for the stored pattern, the threshold value for this
neuron should be

) = n 1)
w n+1( ( – – (32.35)
If it is required that the neuron must also react for similar patterns, then the threshold should be set to
w n +1 = −[n − (1 + HD)], where HD is the Hamming distance defining the range of similarity. Since for
a given input pattern only one neuron in the first layer may have the value of 1 and remaining neurons
have 0 values, the weights in the output layer are equal to the required output pattern.
The network, with unipolar activation functions in the first layer, works as a lookup table. When the
linear activation function (or no activation function at all) is used in the second layer, then the network
also can be considered as an analog memory. For the address applied to the input as a binary vector, the
stored set of analog values, as weights in the second layer, can be accurately recovered. The feedforward
counterpropagation network may also use analog inputs, but in this case all input data should be
normalized,

w i = x ˆ i = -------- (32.36)
x i
x i
The counterpropagation network is very easy to design. The number of neurons in the hidden layer
is equal to the number of patterns (clusters). The weights in the input layer are equal to the input patterns,
and the weights in the output layer are equal to the output patterns. This simple network can be used
for rapid prototyping. The counterpropagation network usually has more hidden neurons than required.
However, such an excessive number of hidden neurons are also used in more sophisticated feedforward
networks such as the probabilistic neural network (PNN) Specht (1990) or the general regression neural
networks (GRNN) Specht (1992).

WTA Architecture

The winner take all (WTA) network was proposed by Kohonen (1988). This is basically a one-layer
network used in the unsupervised training algorithm to extract a statistical property of the input data,
Fig. 32.14(a). At the first step, all input data are normalized so that the length of each input vector is the
same and, usually, equal to unity, Eq. (32.36). The activation functions of neurons are unipolar and
continuous. The learning process starts with a weight initialization to small random values. During the
learning process the weights are changed only for the neuron with the highest value on the output—the
winner:

∆w w = c xw w ) (32.37)
(
–
where
w w = weights of the winning neuron,
x = input vector,
c = learning constant.

Usually, this single-layer network is arranged into a two-dimensional layer shape, as shown in Fig. 32.14(b).
The hexagonal shape is usually chosen to secure strong interaction between neurons. Also, the algorithm
is modified in such a way that not only the winning neuron but also neighboring neurons are allowed
for the weight change. At the same time, the learning constant c in Eq. (32.37) decreases with the distance
from the winning neuron. After such an unsupervised training procedure, the Kohonen layer is able to
organize data into clusters. Output of the Kohonen layer is then connected to the one- or two-layer
feedforward network with the error backpropagation algorithm. This initial data organization in the
WTA layer usually leads to rapid training of the following layer or layers.

©2002 CRC Press LLC