Page 219 -

5.7 Approximation Methods in NN Training 207

A difficulty with the Levenberg-Marquardt algorithm is that the memory
required to compute and store the pseudo-inverse matrix is proportional to the
square of the number of weights in the network. This restricts its use to small
networks.
The Levenberg-Marquardt method was also applied to the foetal weight data set.
In several runs it tended to slightly over-fit the training set. Problems with local
minima occurred rarely. Figure 5.40 shows one solution with RMS errors of 266.1
g for the training set, 274.4 g for the verification set and 291.7 g for the test set.
The performance is only slightly worse than with conjugate gradient.

I
0 . .
Case 1 Case 101 Case 201 Case 301 Case 401
Case 51 Case 151 Case 251 Case 351
Figure 5.40. Predicted foetal weight (PR-FW) using an MLP3:6: 1 trained with the
Levenberg-Marquardt algorithm. The FW curve represents the true foetal weight
values.

5.8 Genetic Algorithms in NN Training

Genetic algorithms are a class of stochastic optimisation algorithms. They were
introduced by Holland (1975) and provide a way of stochastically training MLPs,
in addition to many other interesting applications in the Neural Networks field. The
key idea is the manipulation of the relevant information, such as neural weights,
using rules inspired by the evolution of living beings.
Let us imagine that each weight of a MLP is considered as a gene (attribute) of
the network device. The whole set of weights (genes) can then be considered as the
MLP chromosome. To give a concrete example, let us apply this concept to an
MLP2:2: 1, shown in Figure 5.41.