Page 52 - Rapid Learning in Robotics

38 Artificial Neural Networks

the available parameters are forced to decay to small values (weight
decay). These redundant terms are afterwards removed. The disad-
vantage of pruning (Hinton 1986; Hanson and Pratt 1989) or optimal
brain damage (Cun, Denker, and Solla 1990) methods is that both start
with rather large and therefore slower converging networks.

Growing Network Structures (additive model) follow the opposite direc-
tion. Usually, the learning algorithm monitors the network perfor-
mance and decides when and how to insert further network elements
(in form of data memory, neurons, or entire sub-nets) into the ex-
isting structure. This can be combined with outliers removing and
pruning techniques, which is particularly useful when the grow-
ing step is generous (one-shot learning and forgetting the unimpor-
tant things). Various unsupervised algorithms have been proposed:
additive models building local regression models (Breimann, Fried-
man, Olshen, and Stone 1984; Hastie and Tibshirani 1991), dynamic
memory based models (Atkeson 1992; Schaal and Atkeson 1994),
and RBF net (Platt 1991); the tiling algorithm (for binary outputs;
Mézard and Nadal 1989) has similarities to the recursive partition-
ing procedure (MARS) but allows also non-orthogonal hyper-planes.
The (binary output) upstart algorithm (Frean 1990) shares similarities
with the continuous valued cascade correlation algorithm (Fahlman
and Lebiere 1990; Littmann 1995). Adaptive topological models are
studied in (Jockusch 1990), (Fritzke 1991) and in combination with
the Neural-Gas in (Fritzke 1995).

3.7 Kohonen's Self-Organizing Map

Teuvo Kohonen formulated the (Self-Organizing Map) (SOM) algorithm as
a mathematical model of the self-organization of certain structures in the
brain, the topographic maps (e.g. Kohonen 1984).
In the cortex, neurons are often organized in two-dimensional sheets
with connections to other areas of the cortex or sensor or motor neurons
somewhere in the body. For example, the somatosensory cortex shows a
topographic map of the sensory skin of the body. Topographic map means
that neighboring areas on the skin find their neural connection and rep-
resentation to neighboring neurons in the cortex. Another example is the