Page 389 - Probability and Statistical Inference

366 7. Point Estimation

about the other’s results for quite some time due to the war. We will first
prove this famous inequality.
Next, we discuss a fundamental result (Theorem 7.5.2) due to Lehmann
and Scheffé (1950) which leaps out of the Rao-Blackwell Theorem and it
helps us go very far in our search for the UMVUE. The Lehmann-Scheffé
(1950,1955,1956) series of papers is invaluable in this regard.

7.5.1 The Cramér-Rao Inequality and UMVUE
Lehmann (1983) referred to this inequality as the “information inequality,” a
name which was suggested by Savage (1954). Lehmann (1983, p. 145, Sec-
tion 9) wrote, “The first version of the information inequality appears to have
been given by Fréchet (1943). Early extensions and rediscoveries are due to
Darmois (1945), Rao (1945), and Cramér (1946b).” We will, however, con-
tinue to refer to this inequality by its commonly used name, the Cramér-Rao
inequality, for the ease of (i) locating cross-references and (ii) going for a
literature search among the available books and other sources.
This bound for the variance, customarily called the Cramér-Rao lower
bound (CRLB), for unbiased estimators of T(θ) is appreciated in many prob-
lems where one can (i) derive the expression of the CRLB, and (ii) easily
locate an unbiased estimator of T(θ) whose variance happens to coincide with
the CRLB. In situations like these, one has then precisely found the UMVUE
for T(θ).
Consider iid real valued and observable random variables X , ..., X from a
1
n
population with the common pmf or pdf f(x; θ) where the unknown param-
⊆
χ ⊆
eter θ ∈ Θ ℜ and x ∈ ℜ. Recall that we denote X = (X , ..., X ). Let us
n
1
pretend that we are working with the pdf and hence the expectations of func-
tions of random variables would be written as appropriate multiple integrals.
In the case of discrete random variables, one would replace the integrals by
the appropriate finite or infinite sums, as the case may be.
χ
Standing Assumptions: Let us assume that the support does not in-
volve θ and the first partial derivative of f(x; θ) with respect to θ and the
integrals with respect to X = (x , ..., x ) are interchangeable.
n
1
Theorem 7.5.1 (Cramér-Rao Inequality) Suppose that T = T(X) is an
unbiased estimator of a real valued parametric function T(θ), that is E (T) =
θ
T(θ) for all θ ∈ Θ. Assume also that T(θ), denoted by T’(?), exists and it is
finite for all θ ∈ Θ. Then, for all θ ∈ Θ, under the standing assumptions we
have: