Page 300 - Computational Statistics Handbook with MATLAB

Chapter 8: Probability Density Estimation 289

Finite mixtures offer advantages in the area of the computational load put
on the system. Two issues to consider with many probability density estima-
tion methods are the computational burden in terms of the amount of infor-
mation we have to store and the computational effort needed to obtain the
probability density estimate at a point. We can illustrate these ideas using the
kernel density estimation method. To evaluate the estimate at a point x (in the
univariate case) we have to retain all of the data points, because the estimate
is a weighted sum of n kernels centered at each sample point. In addition, we
must calculate the value of the kernel n times. The situation for histograms
and frequency polygons is a little better. The amount of information we must
store to provide an estimate of the probability density is essentially driven by
the number of bins. Of course, the situation becomes worse when we move
to multivariate kernel estimates, histograms, and frequency polygons. With
the massive, high-dimensional data sets we often work with, the computa-
tional effort and the amount of information that must be stored to use the
density estimates is an important consideration. Finite mixtures is a tech-
nique for estimating probability density functions that can require relatively
little computer storage space or computations to evaluate the density esti-
mates.

tt
ee
rees
ee
MixtuMixtu
Un
UUnnii ivvar vvarar i ar ii iaat aatt teeFini Finit FiniFini teeMixtuMixtu rr eess s
r
Uni
The finite mixture method assumes the density f x() can be modeled as the
sum of c weighted densities, with c << n . The most general case for the
univariate finite mixture is
c
f x() = ∑ p gx θ ;( i , ) (8.31)
i
i = 1
represents the weight or mixing coefficient for the i-th term, and
where p i
(
gx θ i ) denotes a probability density, with parameters represented by the
;
vector θ i . To make sure that this is a bona fide density, we must impose the
condition that p 1 + … + p c = 1 and p i > 0. To evaluate f x() , we take our
point x, find the value of the component densities gx θ i ) at that point, and
(
;
take the weighted sum of these values.
Example 8.8
The following example shows how to evaluate a finite mixture model at a
given x. We construct the curve for a three term finite mixture model, where
the component densities are taken to be normal. The model is given by

,
,
(
(
(
,
f x() = 0.3 × φ x 31) + 0.3 × φ x 01) + 0.4 × φ x 20.5 , )
;
–
;
;
© 2002 by Chapman & Hall/CRC