Page 74 - Computational Statistics Handbook with MATLAB

60 Computational Statistics Handbook with MATLAB

This information is important, because we can use it to determine how
µ
X
much error there is in using as an estimate of the population mean . We
can also perform statistical hypothesis tests using as a test statistic and can
X
µ
calculate confidence intervals for . In this book, we are mainly concerned
with computational (rather than theoretical) methods for finding sampling
distributions of statistics (e.g., Monte Carlo simulation or resampling). The
sampling distribution of X is used to illustrate the concepts covered in
remaining chapters.

3.4 Parameter Estimation

One of the first tasks a statistician or an engineer undertakes when faced with
data is to try to summarize or describe the data in some manner. Some of the
statistics (sample mean, sample variance, coefficient of skewness, etc.) we
covered in Section 3.2 can be used as descriptive measures for our sample. In
this section, we look at methods to derive and to evaluate estimates of popu-
lation parameters.
There are several methods available for obtaining parameter estimates.
These include the method of moments, maximum likelihood estimation,
Bayes estimators, minimax estimation, Pitman estimators, interval estimates,
robust estimation, and many others. In this book, we discuss the maximum
likelihood method and the method of moments for deriving estimates for
population parameters. These somewhat classical techniques are included as
illustrative examples only and are not meant to reflect the state of the art in
this area. Many useful (and computationally intensive!) methods are not cov-
ered here, but references are provided in Section 3.7. However, we do present
some alternative methods for calculating interval estimates using Monte
Carlo simulation and resampling methods (see Chapters 6 and 7).
Recall that a sample is drawn from a population that is distributed accord-
ing to some function whose characteristics are governed by certain parame-
ters. For example, our sample might come from a population that is normally
distributed with parameters and σ 2 . Or, it might be from a population that
µ
is exponentially distributed with parameter λ. The goal is to use the sample
to estimate the corresponding population parameters. If the sample is repre-
sentative of the population, then a function of the sample should provide a
useful estimate of the parameters.
Before we undertake our discussion of maximum likelihood, we need to
define what an estimator is. Typically, population parameters can take on val-
ues from a subset of the real line. For example, the population mean can be
any real number, ∞ <– µ < ∞ , and the population standard deviation can be
any positive real number, σ > 0 . The set of all possible values for a parameter
θ is called the parameter space. The data space is defined as the set of all pos-
sible values of the random sample of size n. The estimate is calculated from

© 2002 by Chapman & Hall/CRC