Page 276 - Fundamentals of Probability and Statistics for Engineers

9

Parameter Estimation

Suppose that a probabilistic model, represented by probability density function
(pdf) f (x), has been chosen for a physical or natural phenomenon for which
parameters 1 , 2 , . . . are to be estimated from independently observed data
x 1 , x 2 , .. ., x n . Let us consider for a moment a single parameter for simplicity

and write f (x; ) to mean a specified probability distribution where is the unknown
parameter to be estimated. The parameter estimation problem is then one of
determining an appropriate function of x 1 , x 2 , . . ., x n , say h(x 1 , x 2 , .. ., x n ), which

gives the ‘best’ estimate of . In order to develop systematic estimation procedures,
we need to make more precise the terms that were defined rather loosely in the
preceding chapter and introduce some new concepts needed for this development.

9.1 SAMPLES AND STATISTICS

Given an independent data set x 1 , x 2 ,..., x n , let
^
ˆ h x 1 ; x 2 ; ... ; x n † 9:1†

be an estimate of parameter . In order to ascertain its general properties, it is
recognized that, if the experiment that yielded the data set were to be repeated,
we would obtain different values for x 1 , x 2 , .. . , x n . The function h(x 1 , x 2 , .. . , x n )
^

when applied to the new data set would yield a different value for .We thus see
that estimate ^ is itself a random variable possessing a probability distribution,
which depends both on the functional form defined by h and on the distribution
^

of the underlying random variable X. The appropriate representation of is thus
^
ˆ h X 1 ; X 2 ; ... ; X n †; 9:2†
where X 1 , X 2 ,..., X n are random variables, representing a sample from random
variable X, which is referred to in this context as the population. In practically

Fundamentals of Probability and Statistics for Engineers T.T. Soong  2004 John Wiley & Sons, Ltd
ISBNs: 0-470-86813-9 (HB) 0-470-86814-7 (PB)

TLFeBOOK