Page 292 - Fundamentals of Probability and Statistics for Engineers

Parameter Estimation 275
9.2.4 SUFFICIENCY

Let X 1 , X 2 ,..., X n be a sample of a population X the distribution of which

depends on unknown parameter . If Y ˆ h X 1 , X 2 , ... , X n )isa statisticsuch
that, for any other statistic
Z ˆ g X 1 ; X 2 ; ... ; X n †;

=
the conditional distribution of Z , given that Y y does not depend on , then
Y is called a sufficient statistic for . If also EfYgˆ , then Y is said to be a

sufficient estimator for .
In words, the definition for sufficiency states that, if Y is a sufficient statistic
for , all sample information concerning is contained in Y . A sufficient

statistic is thus of interest in that if it can be found for a parameter then an
estimator based on this statistic is able to make use of all the information that
the sample contains regarding the value of the unknown parameter. Moreover,
an important property of a sufficient estimator is that, starting with any
unbiased estimator of a parameter that is not a function of the sufficient
estimator, it is possible to find an unbiased estimator based on the sufficient
statistic that has a variance smaller than that of the initial estimator. Sufficient
estimators thus have variances that are smaller than any other unbiased esti-
mators that do not depend on sufficient statistics.
If a sufficient statistic for a parameter exists, Theorem 9.4, stated here
without proof, provides an easy way of finding it.

Theorem 9.4: Fisher–Neyman factorization criterion. Let

Y ˆ h X 1 , X 2 , ... , X n )
be a statistic based on a sample of size n. Then Y is a sufficient statistic for
if and only if the joint probability density function of X 1 , X 2 , . .., and
X n , f X x 1 ; ) f X x n ; ), can be factorized in the form
n
Y
f X x j ; †ˆ g 1 ‰h x 1 ; ... ; x n †; Šg 2 x 1 ; ... ; x n †: 9:49†
jˆ1
If X is discrete, we have
n
Y
p X x j ; †ˆ g 1 ‰h x 1 ; ... ; x n †; Šg 2 x 1 ; ... ; x n †: 9:50†
jˆ1
The sufficiency of the factorization criterion was first pointed out by Fisher
(1922). Neyman (1935) showed that it is also necessary.
TLFeBOOK