Page 326 - Probability and Statistical Inference

6. Sufficiency, Completeness, and Ancillarity 303

Now, let us write

so that 0 = dx. Hence one
obtains

Next, combining (6.4.6)-(6.4.8), we conclude that I (θ) = nI (θ). !
X X1
Suppose that we have collected random samples X , ..., X from a popula-
1
n
tion and we have evaluated the information I (θ) contained in the data X =
X
(X , ..., X ). Next, suppose also that we have a statistic T = T(X) in mind for
1
n
which we have evaluated the information I (θ) contained in T. If it turns out
T
that I (θ), can we then claim that the statistic T is indeed sufficient for θ? The
T
answer is yes, we certainly can. We state the following result without supply-
ing its proof. One may refer to Rao (1973, result (iii), p. 330) for details. In a
recent exchange of personal communications, C. R. Rao has provided a simple
way to look at the Theorem 6.4.2. In the Exercise 6.4.15, we have given an
outline of Rao’s elegant proof of this result. In the Examples 6.4.3-6.4.4, we
find opportunities to apply this theorem.
Theorem 6.4.2 Suppose that X is the whole data and T = T(X) is some
statistic. Then, I (θ) ≥ I (θ) for all θ ∈ Θ. The two information measures
X
T
match with each other for all θ if and only if T is a sufficient statistic for θ.
Example 6.4.3 (Example 6.4.1 Continued) Suppose that X , ..., X are iid
1
n
Poisson(λ), where λ (> 0) is the unknown parameter. We already know that
is a minimal sufficient statistic for λ and T is distributed as
Poisson(nλ). But, let us now pursue T from the information point of view.
One can start with the pmf g(t; λ) of T and verify that

as follows: Let us write log{g(t; λ)} = –nλ+tlog(nλ) – log(t!) which implies
that log{g(t;λ)} = –n + tλ . So, I (λ) = E [{ log{g(T;λ)}} ] = E [(T -
-1
2
T λ λ
nλ) /λ ] = nλ since E [(T - nλ) ] = V(T) = nλ.
2
-1
2
2
λ
On the other hand, from (6.4.4) and Example 6.4.1, we can write I (λ)
X
= nI (λ) = nλ . That is, T preserves the available information from the
-1
X1
whole data X. The Theorem 6.4.2 implies that the statistic T is indeed suf-
ficient for λ. !