Page 75 - Computational Statistics Handbook with MATLAB

Chapter 3: Sampling Concepts 61

the sample data as a function of the random sample. An estimator is a func-
tion or mapping from the data space to the parameter space and is denoted as

T = t X … X,( , . ) (3.13)
1 n
Since an estimator is calculated using the sample alone, it is a statistic. Fur-
thermore, if we have a random sample, then an estimator is also a random
variable. This means that the value of the estimator varies from one sample
to another based on its sampling distribution. In order to assess the useful-
ness of our estimator, we need to have some criteria to measure the perfor-
mance. We discuss four criteria used to assess estimators: bias, mean squared
error, efficiency, and standard error. In this discussion, we only present the
definitional aspects of these criteria.

Bias BiasBias Bias
The bias in an estimator gives a measure of how much error we have, on aver-
age, in our estimate when we use T to estimate our parameter θ. The bias is
defined as

bias T() = ET[] – θ . (3.14)

If the estimator is unbiased, then the expected value of our estimator equals
the true parameter value, so ET[] = θ.
To determine the expected value in Equation 3.14, we must know the dis-
tribution of the statistic T. In these situations, the bias can be determined ana-
lytically. When the distribution of the statistic is not known, then we can use
methods such as the jackknife and the bootstrap (see Chapters 6 and 7) to esti-
mate the bias of T.

aann
rr oorr
quared
SSquaredErEr
Me
MeMe
Mea annS SquaredErquaredErr roor r
Let θ denote the parameter we are estimating and T denote our estimate, then
the mean squared error (MSE) of the estimator is defined as
( [
MSE T() = ET θ) 2 . ] (3.15)
–
Thus, the MSE is the expected value of the squared error. We can write this in
more useful quantities such as the bias and variance of T. (The reader will see
this again in Chapter 8 in the context of probability density estimation.) If we
expand the expected value on the right hand side of Equation 3.15, then we
have

© 2002 by Chapman & Hall/CRC