Page 402 - Numerical Methods for Chemical Engineering

Selecting a prior for single-response data 391

Although the posterior density above appears to be of a simple functional form, for a
general nonlinear model, S(θ) by (8.58) may depend in a very complicated manner upon θ.
Therefore, we approximate S(θ) in the local neighborhood of θ LS by a quadratic expansion
(for a linear model, this expansion is exact),
2
T
1
S(θ) − S(θ LS ) ≈ (θ − θ LS ) [∇ S(θ LS )](θ − θ LS ) (8.95)
2
2
∇ S(θ LS ) is the Hessian of S(θ), evaluated at θ LS . Following our numerical treatment of
nonlinear least squares, we define the matrix H(θ LS )as
2 1
H(θ LS ) =∇ S(θ LS ) (8.96)
2
with the elements
) *
N 2
T [k] [k] ∂ f
)
H ab (θ LS ) = (X X| θ LS ab − y − f x ; θ LS (8.97)

[k]
k=1 ∂θ a ∂θ b x ;θ LS
The linearized design matrix is again given by (8.80). It is consistent with a quadratic
expansion of S(θ) to neglect the second contribution to H ab (θ LS ), so that
T
X X| θ LS ≈ H(θ LS ) (8.98)
Again, for a linear model, this approximation is exact. The quadratic expansion for S(θ),
using 1 2 T , is then
2 ∇ S(θ LS ) = H(θ LS ) ≈ X X| θ LS
T T
S(θ) − S(θ LS ) ≈ (θ − θ LS ) [X X| θ LS ](θ − θ LS ) (8.99)
Thus, an approximate “conditional posterior” for θ given y and σ is
1
1 T T
π(θ|y,σ) ∝ exp − θ − θ LS [X X| θ LS ](θ − θ LS ) p(θ) (8.100)
2σ 2
The approximate “conditional likelihood function”
1

T
1 T
l approx (θ|y,σ) ∝ exp − θ − θ LS X X θ − θ LS (8.101)
2σ 2 θ LS
is of the form of a multivariate normal distribution
1 T −1

p(x|µ, ) ∝ exp − (x − µ) (x − µ) (8.102)
2
with
2 T −1
µ = θ LS = σ (X X| θ LS ) (8.103)
For a linear model, the design matrix X is completely specified at the time that we choose the
T
predictor values in each experiment, and thus so is X X. From one trial set of experiments
to the next, the only quantity that varies significantly is θ LS , and this quantity merely sets the
center of the distribution, but does not affect its shape. Therefore, the likelihood function
(8.101) is said to be translated by the data,or data-translated. For a linear model, where
X does not vary, this data-translation property is exact. For a nonlinear model, this data-
translation property is only approximate.