Page 235 - Compact Numerical Methods For Computers

Left-overs 223
subject to
(18.18b)
This can be solved by elimination. However, in order to perform the elimination it
is necessary to decide whether
(18.19)
or
(18.20)
-½
The first choice leads to the constrained minimum at b = b =2 . The second
2
1
-½
leads to the constrained maximum at b =b =-2 . This problem is quite easily
1 2
solved approximately by means of the penalty (18.14) and any of the uncon-
strained minimisation methods.
A somewhat different question concerning elimination arises in the following
problem due to Dr Z Hassan who wished to estimate demand equations for
commodities of which stocks are kept: minimise
(18.21)

subject to
b b = b b . (18.22)
3 6 4 5
The data for this problem are given in table 18.2. The decision that must now
be made is which variable is to be eliminated via (18.22); for instance, b 6 can be
found as
b = b b /b . (18.23)
6
3
4 5
The behaviour of the Marquardt algorithm 23 on each of the four unconstrained
minimisation problems which can result from elimination in this fashion is shown
in table 18.3. Numerical approximation of the Jacobian elements was used to save
some effort in running these comparisons. Note that in two cases the algorithm
has failed to reach the minimum. The starting point for the iterations was b =1,
j
j=l, 2, . . . , 6, in every case, and these failures are most likely due to the large
differences in scale between the variables. Certainly, this poor scaling is respons-
ible for the failure of the variable metric and conjugate gradients algorithms when
the problem is solved by eliminating b . (Analytic derivatives were used in these
6
cases.)
The penalty function approach avoids the necessity of choosing which parame-
ter is eliminated. The lower half of table 18.3 presents the results of computations
with the Marquardt-like algorithm 23. Similar results were obtained using the
Nelder-Mead and variable metric algorithms, but the conjugate gradients method
failed to converge to the true minimum. Note that as the penalty weighting w is
increased the minimum function value increases. This will always be the case if a
constraint is active, since enforcement of the constraint pushes the solution ‘up
the hill’.
Usually the penalty method will involve more computational effort than the
elimination approach (a) because there are more parameters in the resulting