Page 272 - Introduction to Statistical Pattern Recognition

Chapter 6

NONPARAMETRIC DENSITY ESTIMATION

So far we have been discussing the estimation of parameters. Thus, if
we can assume we have a density function that can be characterized by a set of
parameters, we can design a classifier using estimates of the parameters.
Unfortunately, we often cannot assume a parametric form for the density func-
tion, and in order to apply the likelihood ratio test we somehow have to esti-
mate the density functions using an unstructured approach. This type of
approach is called nonparametric estimation, while the former is called
parametric estimation. Since, in nonparametric approaches, the density func-
tion is estimated locally by a small number of neighboring samples, the esti-
mate is far less reliable with larger bias and variance than the parametric coun-
terpart.
There are two kinds of nonparametric estimation techniques available:
one is called the Par-zen density estimate and the other is the k-nearest neigh-
bor- densiry estimate. They are fundamentally very similar, but exhibit some
different statistical properties. Both are discussed in this chapter.
It is extremely difficult to obtain an accurate density estimate non-
parametrically, particularly in high-dimensional spaces. However, our goal
here is not to get an accurate estimate. Our goal is, by using these estimates,
to design a classifier and evaluate its performance. For this reason, the accu-
racy of the estimate is not necessarily a crucial issue. Classification and
performance evaluation will be discussed in Chapter 7. The intention of this

254