Page 404 - Numerical Methods for Chemical Engineering

Selecting a prior for single-response data 393

A noninformative prior provides a reference standard that other researchers can repro-
duce and accept as not reflecting the analyst’s personal bias. It is not quite true to say
that the noninformative prior is objective, as it must be made proper, and this can be
done in many different ways. Also, if likelihood function has more than one symme-
try property available, it is then a subjective choice of which one to use to generate the
noninformative prior. Still, noninformative priors are perhaps “least subjective” or “most
reproducible,” and several results obtained using (8.68) agree with those of frequentist
statistics.
For the single-response model, using a quadratic expansion for S(θ), the likelihood func-
tion (8.101) is data-translated by the least-squares estimate θ LS . Thus the noninformative
prior is uniform in this same parameter θ, p(θ) ∼ c, at least in the region of appreciable
nonzero likelihood. Again, a uniform prior in θ is justified on the grounds of being non-
informative. The Bayesian most probable estimate θ M therefore agrees with the maximum
likelihood and least squares estimates,

θ M = θ MLE = θ LS (8.107)

Non informative prior for σ

We next consider the problem of estimating the error standard deviation, using the posterior
density for σ,
2 1
νs
−N
p(σ|s) ∝ l(σ|s)p(σ) = σ exp − p(σ) (8.108)
2σ 2
We see that the likelihood l(σ|s)is not data-translated in σ. Even though the likelihood
l(σ|s) is not simply translated by the data, there still may exist a one-to-one transformation
η = η(σ) such that l(η|s) is simply translated by the data. That is, the shape of l(η|s) does
not change from one data set to the next, but the location of its maximum does. Once we
find such a transformation, we should choose our prior to be noninformative in η(σ); i.e.,
such that the posterior density in the transformed coordinate, p(η|s), is also data-translated.
This is the case if the prior is locally uniform in η = η(σ), i.e., p(η) ∼ c, in the region of
appreciable nonzero likelihood.
While there exists a technique, due to Jeffreys, for generating noninformative priors
directly from the likelihood function, here we simply propose a transformation η(σ) = ln σ,
and then show that its likelihood is data-translated. For a discussion of automated procedures
for constructing noninformative priors, consult a general text on Bayesian statistics such as
Box & Tiao (1973).
η
Writing σ = e , we have the transformed likelihood
2 1 2 1
νs νs
η −N
l(η|s) ∝ (e ) exp − = exp −Nη − (8.109)
2(e ) 2e 2η
η 2
N −N
Substituting 1 = s s , we write
νs −N νs
2 1 2 1
N −N
l(η|s) ∝ s s exp −Nη − = s exp N(ln s − η) − (8.110)
2e 2η 2e 2η