Page 323 - Probability and Statistical Inference

300 6. Sufficiency, Completeness, and Ancillarity

statistics within a rich class of statistical models, namely the exponential fam-
ily. It is a hard result to prove. One may refer to Lehmann (1983, pp. 43-44)
or Lehmann and Casella (1998) for some of the details.
Theorem 6.3.3 (Minimal Sufficiency in the Exponential Family) Sup-
pose that X , ..., X are iid with the common pmf or the pdf belonging to the
1
n
k-parameter exponential family defined by (3.8.4), namely

with some appropriate forms for g(x) ≥ 0, a(θθ θθ θ) ≥ 0, b (θθ θθ θ) and R (x), i = 1, ...,
i
i
k. Suppose that the regulatory conditions stated in (3.8.5) hold. Denote the
statistic . Then, the statistic T = (T , ..., T ) is
1
k
(jointly) minimal sufficient for θθ θθ θ.
The following result provides the nature of the distribution itself of a mini-
mal sufficient statistic when the common pmf or pdf comes from an expo-
nential family. Its proof is beyond the scope of this book. One may refer to
the Theorem 4.3 of Lehmann (1983) and Lemma 8 in Lehmann (1986). One
may also review Barankin and Maitra (1963), Brown (1964), Hipp (1974),
Barndorff-Nielsen (1978), and Lehmann and Casella (1998) to gain broader
perspectives.
Theorem 6.3.4 (Distribution of a Minimal Sufficient Statistic in the
Exponential Family) Under the conditions of the Theorem 6.3.3, the pmf or
the pdf of the minimal sufficient statistic (T , ..., T ) belongs to the k-param-
k
1
eter exponential family.
In each example, the data X was reduced enormously by the
minimal sufficient summary. There are situations where no
significant data reduction may be possible.
See the Exercise 6.3.19 for specific examples.

6.4 Information

Earlier we have remarked that we wish to work with a sufficient or mini-
mal sufficient statistic T because such a statistic will reduce the data and
preserve all the “information” about θθ θθ θ contained in the original data. Here,
θ θ θ θ θ may be real or vector valued. But, how much information do we have in
the original data which we are trying to preserve? Now our major concern
is to quantify the information content within some data. In order to keep
the deliberations simple, we discuss the one-parameter and two-parameter