Page 245 - Numerical Methods for Chemical Engineering

234 5 Numerical optimization

Lagrangian
L(x; λ) ≡ F(x) − λg(x) (5.69)
where at the constrained minimum,

= 0 (5.70)
∇L| x min = ∇F| x min − λ∇g| x min
λ is known as a Lagrange multiplier, and its value must be chosen so that at the minimum
of the Lagrangian, g(x) = 0.
The Lagrangian gives us a powerful technique to convert a constrained optimization
problem into a familiar unconstrained one, but there remains the problem of computing the
Lagrange multiplier used to define L(x; λ). In the augmented Lagrangian method,weadd
a quadratic penalty to the Lagrangian that serves to enforce g(x) = 0 whenever we use the
incorrect value of λ. Starting with an initial guess λ [0] of the multiplier, we choose some
µ [0] > 0 and define the augmented Lagrangian
1
[0] [0] [0] [0] 2
L x; λ ,µ ≡ F(x) − λ g(x) + [g(x)] (5.71)
A [0]
2µ
[0]
We then find a point x [0] minimizing L , where
A

[0]
g x
[0] [0]
∇L A [0] = 0 = ∇F| x − λ − ∇g| x [0] (5.72)
[0]
x
µ [0]
This yields the following update of the Lagrange multiplier estimate
[0]
g x
[1] [0]
λ = λ − (5.73)
µ [0]
With this updated multiplier, we define a new augmented Lagrangian and optionally make
[0]
the penalty stronger, 0 <µ [1] ≤ µ . We take x [0] as the initial guess in a minimization of
the new Lagrangian
1
[1] [1] [1] [1] 2
L x; λ ,µ ≡ F(x) − λ g(x) + [g(x)] (5.74)
A [1]
2µ
We iterate this process until convergence to the proper multiplier value is reached; i.e.,
until at the unconstrained minimum of the augmented Lagrangian, the equality constraint
is satisfied.
How do we treat multiple equality constraints? If x min satisfies all constraints, g i (x min ) =
0, i = 1,..., n e , then we can only move away from x min in a direction v that is tangent to
all constraints,
· v = 0 (5.75)
∇g i | x min ∀i = 1, 2,... , n e
If x min is a constrained minimum, we cannot decrease the cost function by moving in any
such tangent direction, so that for any v satisfying (5.75),
∇F| · v = 0 (5.76)
x min
plus some tangent
We can always write ∇F| x min as a linear combination of the ∇g i | x min
vector v satisfying (5.75),
= n e + v (5.77)
∇F| x min λ i ∇g i | x min
i=1