Page 77 - Computational Statistics Handbook with MATLAB

Chapter 3: Sampling Concepts 63

To illustrate this concept, let’s use the sample mean as an example. We
know that the variance of the estimator is

1 2
VX() = ---σ ,
n
for large n. So, the standard error is given by

σ
SE X() = σ = ------- . (3.20)
X
n
σ
If the standard deviation for the underlying population is unknown, then
we can substitute an estimate for the parameter. In this case, we call it the esti-
mated standard error:

ˆ ˆ S
SE X() = σ X = ------- . (3.21)
n

Note that the estimate in Equation 3.21 is also a random variable and has a
probability distribution associated with it.
If the bias in an estimator is small, then the variance of the estimator is
approximately equal to the MSE, VT() ≈ MSE T() . Thus, we can also use the
square root of the MSE as an estimate of the standard error.

mum
Likelihood
n
Likelihood
mum
MaMa xi xxii imumLikelihoodEstimatioEstimation nn
Max
Ma
mumLikelihoodEstimatioEstimatio
A maximum likelihood estimator is that value of the parameter (or parame-
ters) that maximizes the likelihood function of the sample. The likelihood
function of a random sample of size n from density (mass) function f x θ;( ) is
the joint probability density (mass) function, denoted by
,
(
,
,
(
,
;
L θ x 1 … x n ) = fx 1 … x n θ) . (3.22)
;
Equation 3.22 provides the likelihood that the random variables take on a
, ,
particular value x 1 … x n . Note that the likelihood function L is a function of
θ
the unknown parameter θ, and that we allow to represent a vector of
parameters.
If we have a random sample (independent, identically distributed random
variables), then we can write the likelihood function as
(
(
,
(
,
;
L θ() = L θ x 1 … x n ) = fx 1 θ) × … × fx n θ) , (3.23)
;
;
© 2002 by Chapman & Hall/CRC