Page 501 - Probability and Statistical Inference

478 10. Bayesian Methods

with what is known as a posterior distribution. All Bayesian inferences are
then guided by this posterior distribution. This approach which originated
from Bayes’s Theorem was due to Rev. Thomas Bayes (1783). A strong
theoretical foundation evolved due to many fundamental contributions of de
Finetti (1937), Savage (1954), and Jeffreys (1957), among others. The con-
tributions of L. J. Savage were beautifully synthesized in Lindley (1980).
From the very beginning, R. A. Fisher was vehemently opposed to any-
thing Bayesian. Illuminating accounts of Fisher’s philosophical arguments as
well as his interactions with some of the key Bayesian researchers of his time
can be found in the biography written by his daughter, Joan Fisher Box (1978).
Some interesting exchanges between R. A. Fisher and H. Jeffreys as well as
L. J. Savage are included in the edited volume of Bennett (1990). The articles
of Buehler (1980), Lane (1980) and Wallace (1980) gave important perspec-
tives on possible connections between Fisher’s fiducial inference and the Baye-
sian doctrine.
A branch of statistics referred to as “decision theory” provides a formal
structure to find optimal decision rules whenever possible. Abraham Wald
gave the foundation to this important area during the early to mid 1940’s.
Wald’s (1950) book, Statistical Decision Functions, is considered a classic in
this area. Berger (1985) treated modern decision theory with much emphasis
on Bayesian arguments. Ferguson (1967), on the other hand, gave a more
balanced view of the area. The titles on the cover of these two books clearly
emphasize this distinction.
In Section 10.2, we give a formal discussion of the prior and posterior
distributions. The Section 10.3 first introduces the concept of conjugate pri-
ors and then posterior distributions are derived in some standard cases when
has a conjugate prior. In this section, we also include an example where the
assumed prior for is not chosen from a conjugate family. In Section 10.4,
we develop the point estimation problems under the squared error loss func-
tion and introduce the Bayes estimator for . In the same vein, Section 10.5.1
develops interval estimation problems. These are customarily referred to as
credible interval estimators of v. Section 10.5.2 highlights important concep-
tual differences between a credible interval estimator and a confidence inter-
val estimator. Section 10.6 briefly touches upon the concept of a Bayes test of
hypotheses whereas Section 10.7 gives some examples of Bayes estimation
under non-conjugate priors.
We can not present a full-blown discussion of the Bayes theory at this
level. It is our hope that the readers will get a taste of the underlying basic
principles from a brief exposure to this otherwise vast area.