Page 278 - MATLAB an introduction with applications

Optimization ——— 263

where (Jx * ) is the Jacobian matrix of second derivatives given by

∂ 2 Ux * ) ∂ 2 Ux * ) ..... ∂ 2 Ux * )
(
(
(
xx
x ∂ 2 ∂∂ ∂∂
xx
( Jx * ) = 1 1 2 1 n
∂ 2 Ux * ) ..... ∂ 2 Ux * )
(
(
∂∂ 1 x ∂ n 2
xx
n
A set of linear algebraic equations in the unknown x ’s are given by Eq.(5.4). If x is taken as x , the kth
*
i
k
point in the step by step search for the minimum, then solution of Eq. (5.4) given (k + 1)st approximation.
Writing Eq.(5.4) in the form
( gx k ) = ( J x k )(x k+ 1 − x k ) = 0 ...(5.5)
It is possible to obtain an iterative form for the solution as
−
1
x
x
J
x k+ 1 = x − [( )] g ( )
k
k
k
]
It is not actually necessary to obtain the inverse of the matrix [ Jx to solve for the new approximation
k
x . It is possible to write Eq.(5.5) in the form
k
()Jx δ= − g ()
x
k
k
k
where δ= x k+ 1 − x .
k
k
5.4 THE CONCEPT OF QUADRATIC CONVERGENCE
Conjugate Directions for a Quadratic Function
The gradient method is expressed as
x k+ 1 = x − λ k ( g x k ) ...(5.6)
k
or in the form
δ= −λ k gx k
()
k
During each iteration λ is selected to minimize Ux k+ 1 ) in the gradient direction. A gradient search tends
(
k
to zig-zig quite badly a particularly for quadratic functions. If it is possible to establish the best direction
to take for a quadratic function, it would likely also be better for most other non-linear functions. This is
called quadratic convergence and the approach is rather surprisingly successful. Equation (5.6) has the
general form, in this case, given by
x k+ 1 = x + λ k C (x k ) ...(5.7)
k
where ()Cx k defines the conjugate direction vector at each step. The general quadratic form is
1
T
() = x Ax + B x + d ...(5.8)
T
Ux
2
where A is a matrix and B is a vector.