Page 395 - Numerical Methods for Chemical Engineering
P. 395
384 8 Bayesian statistics and parameter estimation
We also assume that the error value in each experiment is independent of the values in
the other experiments, i.e., they are uncorrelated, so that the covariances among errors
are
[k] [j] [k] [k] [j] [k] [k]
cov ε ,ε ≡ E ε − E ε ε − E ε = δ kj var ε (8.51)
That is, the covariance matrix of ε is diagonal:
σ
2
1
σ 2
2
2
σ = var ε [k] (8.52)
. . k
cov(ε) =
.
σ N 2
Finally, we assume that all variances are equal,
2 2
σ = σ k = 1, 2,..., N (8.53)
k
Combined, these are known as the Gauss–Markov conditions:
[k] [k] [ j] 2
E ε = 0cov ε ,ε = δ kj σ (8.54)
We make one additional assumption: that the error is distributed according to a Gaussian
(normal) distribution. This distribution is to be expected when the error in the measured
response is the net result of many independent sources of error being added together.
The probability of observing a particular value of the error in the kth experiment between
ε and ε + dε is then
2
[k]
1 ε
[k]
p ε |σ = √ exp − 2 (8.55)
σ 2π 2σ
We now make a subtle, though important, shift in our point of view about the “true” value
of θ. Previously, we have considered the random error to be added to the model prediction
[k]
with the “true” parameter vector θ (true) such that ε [k] = y [k] − ˆ y (θ (true) ). But, as we do
[k]
not know θ (true) , we cannot relate ε [k] to y . Therefore, let us drop all reference to the
[k]
“true” value completely, and assume that the relation ε [k] = y [k] − ˆ y (θ) holds for all trial
values of θ. That is, for any trial θ and σ, we propose that the probability of observing a
[k]
response y ,given θ and σ,is
2
(
[k]
[k]
1 y − ˆ y (θ)
[k] [k] [k] [k]
p y |θ,σ = p ε = y − ˆ y (θ)|σ = √ exp − 2
σ 2π 2σ
(8.56)
As we assume that the errors in each experiment are independent, the joint probability of
observing the complete vector of responses y is
N
p y |θ,σ
p(y|θ,σ) = 0 [k]
k=1
1 −N 1 [k] [k] 2
N N
= √ σ exp − 2 y − ˆ y (θ) (8.57)
2π 2σ
k=1
