Page 305 - Fundamentals of Probability and Statistics for Engineers

288 Fundamentals of Probability and Statistics for Engineers

Let f (x; ) be the density function of population X where, for simplicity, is

the only parameter to be estimated from a set of sample values x 1 , x 2 ,..., x n .
The joint density function of the corresponding sample X 1 , X 2 ,..., X n has the
form

f x 1 ; †f x 2 ; † f x n ; †:

We define the likelihood function L of a set of n sample values from the
population by

L x 1 ; x 2 ; ... ; x n ; †ˆ f x 1 ; †f x 2 ; † f x n ; †: 9:96†

In the case when X is discrete, we write

L x 1 ; x 2 ; .. . ; x n ; †ˆ p x 1 ; †p x 2 ; † p x n ; †: 9:97†

When the sample values are given, likelihood function L becomes a function
of a single variable . The estimation procedure for based on the method of

maximum likelihood consists of choosing, as an estimate of , the particular

value of that maximizes L. The maximum of L( ) occurs in most cases at the

value of where dL( )/d is zero. Hence, in a large number of cases, the
^

maximum likelihood estimate (MLE) of based on sample values x 1 , x 2 ,...,
and x n can be determined from
^
dL x 1 ; x 2 ; .. . ; x n ; †
ˆ 0: 9:98†
d ^
As we see from Equations (9.96) and (9.97), function L is in the form of a

product of many functions of . Since L is always nonnegative and attains its
^

maximum for the same value of as ln L, it is generally easier to obtain MLE
^ by solving
^
dln L x 1 ; ... ; x n ; †
ˆ 0; 9:99†
d ^
because ln L is in the form of a sum rather than a product.
Equation (9.99) is referred to as the likelihood equation. The desired solution
^

is one where root is a function of x j , j ˆ 1, 2, ..., n, if such a root exists. When
several roots of Equation (9.99) exist, the MLE is the root corresponding to the
global maximum of L or ln L.

TLFeBOOK