Page 381 - Probability and Statistical Inference

358 7. Point Estimation

7.4 Improved Unbiased Estimators via Sufficiency

It appears that one of the first examples of UMVUE was found by Aitken and
Silverstone (1942). The UMVUE’s as suitable and unique functions of suffi-
cient statistics were investigated by Halmos (1946), Kolmogorov (1950a),
and more generally by Rao (1947).
In a problem suppose that we can get hold of at least one unbiased estima-
tor for a real valued parametric function T(θθ θθ θ). Now the question is this: If we
start with some unbiased estimator T for T(θθ θθ θ), however trivial this estimator
may appear to be, can we improve upon T? That is, can we revise the initial
unbiased estimator in order to come up with another unbiased estimator T’ of
T(θθ θθ θ) such that V (T’) < V (T) for all θθ θθ θ ∈ Θ?
θ
θ
Let us illustrate. Suppose that X , ..., X are iid Bernoulli(p) where 0 < p <
n
1
1 is unknown. We wish to estimate the parameter p unbiasedly. Consider T =
X and obviously T is an unbiased estimator of p. But note that T can take one
1
of two possible values 0 or 1. Such an estimator, even though unbiased, may
appear naive and useless. T may be unreasonable as an estimator of the pa-
rameter p which lies between zero and one. But, this criticism against T should
not be too bothersome because T is not the estimator which would be recom-
mended for use in practice. The question is, even if we start with T, can we
hope to improve upon this initial estimator in the sense of reducing the vari-
ance? For all practical purposes, the answer is in the affirmative. The general
machinery comes next.

7.4.1 The Rao-Blackwell Theorem

The technique to improve upon an initial unbiased estimator of T(θθ θθ θ) is custom-
arily referred to as the Rao-Blackwellization in the statistical literature. C. R.
Rao and D. Blackwell independently published fundamental papers respec-
tively in 1945 and 1947 which included this path-breaking idea. Neither Rao
nor Blackwell knew about the other’s paper for quite some time because of
disruptions due to the war. We first state and prove this fundamental result.
Theorem 7.4.1 (Rao-Blackwell Theorem) Let T be an unbiased estima-
tor of a real valued parametric function T(θθ θθ θ) where the unknown parameter θθ θθ θ
∈ Θ ⊂ ℜ . Suppose that U is a jointly sufficient statistic for θθ θθ θ. Define g(u) =
k
E [T &pipe; U = u], for u belonging to U, the domain space of U. Then, the
θ
following results hold:
(i) Define W = g(U). Then, W is an unbiased estimator of T(θθ θθ θ);
(ii) V [W] ≤ V [T] for all θθ θθ θ ∈ Θ, with the equality holding if and only
θ
θ
if T is the same as W w. p. 1.