Page 230 - Compact Numerical Methods For Computers

Left-overs 219
made during the minimisation of a function, these are not useful in the present
instance. Recall that
(18.3)

where e is the jth column of the unit matrix of order n (b is presumed to have n
j
elements). For explanatory purposes, the case n=1 will be used. In place of the
limit (18.3), it is possible to use the forward difference
D=[ S(b+h)-S(b)]/ h (18.4)

for some value of h.
Consider the possible sources of error in using D.
(i) For h small, the discrete nature of the representation of numbers in the
computer causes severe inaccuracies in the calculation of D. The function S is
continuous; its representation is not. In fact it will be a series of steps. Therefore,
h cannot be allowed to be small. Another way to think of this is that since most of
the digits of b are the same as those of (b+h), any function S which is not varying
rapidly will have similar values at b and (b+h), so that the expression (18.4)
implies a degree of digit cancellation causing D to be determined inaccurately.
(ii) For h large, the line joining the points (b , S(b)) and (b+h, S(b+h)) is no
longer tangential to the curve at the former. Thus expression (18.4) is in error due
to the nonlinearity of the function. Even worse, for some functions there may be
a discontinuity between b and (b+h). Checks for such situations are expensive of
both human and computer time. The penalty for ignoring them is unfortunately
more serious.
As a compromise between these extremes, I suggest letting

(18.5)
where e is the machine precision. The parameter has once more been given a
subscript to show that the step taken along each parameter axis will in general be
different. The value for h given by (18.5) has the pleasing property that it cannot
become smaller than the machine precision even if b is zero. Neither can it fail to
j
change at least the right-most half of the digits in b j since it is scaled by the
magnitude of the parameter. Table 18.1 shows some typical results for this
step-length choice compared with some other values.
Some points to note in table 18.1 are:
(i) The effect of the discontinuity in the tangent function in the computations for
b=1 and b=1·57 (near p/2). The less severely affected calculations for b=
-1·57 suggest that in some cases the backward difference
D=[S(b)-S(b-h)]/h (18.6)
may be preferred.
(ii) In approximating the derivative of exp(0·001) using h=1·93024E-6 as in
equation (18.5), the system used for the calculations printed identical values for
exp(b) and exp(b+h) even though the internal representations were different