Page 229 - Compact Numerical Methods For Computers

Chapter 18

LEFT-OVERS

18.1. INTRODUCTION
This chapter is entitled ‘left-overs’ because each of the topics-approximation
of derivatives, constrained optimisation and comparison of minimisation
algorithms-has not so far been covered, though none is quite large enough in
the current treatment to stand as a chapter on its own. Certainly a lot more could
be said on each, and I am acutely aware that my knowledge (and particularly my
experience) is insufficient to allow me to say it. As far as I am aware, very little
work has been done on the development of compact methods for the mathemati-
cal programming problem, that is, constrained minimisation with many con-
straints. This is a line of research which surely has benefits for large machines, but
it is also one of the most difficult to pursue due to the nature of the problem. The
results of my own work comparing minimisation algorithms are to my knowledge
the only study of such methods which has been made on a small computer. With
the cautions I have given about results derived from experiments with a single
system, the conclusions made in §18.4 are undeniably frail, though they are for
the most part very similar to those of other workers who have used larger
computers.

18.2. NUMERICAL APPROXIMATION OF DERIVATIVES
In many minimisation problems, the analytic computation of partial derivatives is
impossible or extremely tedious. Furthermore, the program code to compute

(18.1)
in a general unconstrained minimisation problem or
(18.2)
in a nonlinear least-squares problem may be so long as to use up a significant
proportion of the working space of a small computer. Moreover, in my experience
9 cases out of 10 of ‘failure’ of a minimisation program are due to errors in the
code used to compute derivatives. The availability of numerical derivatives
facilitates a check of such possibilities as well as allowing programs which require
derivatives to be applied to problems for which analytic expressions for deriva-
tives are not practical to employ.
In the literature, a great many expressions exist for numerical differentiation of
functions by means of interpolation formulae (see, for instance, Ralston 1965).
However, in view of the large number of derivative calculations which must be

218