Page 365 - Introduction to Statistical Pattern Recognition

7 Nonparametric Classification and Error Estimation 347

the case in which the covariance determinants of the two classes are very dif-
ferent. Also, it is observed in Fig. 7-7(b) that the error curve increases very
quickly. With t ranging from -3 to -5, the error curve becomes more like the
one of Fig. 7-7(c), having a down-slope, a minimum at a higher I-, and a slower
up-slope. This means that smaller biases exist in a wider range of I' and that
(7.70) fits the actual errors more accurately.

WN Approach

So far, we have discussed error estimation based on the Parzen density
estimate. Similarly, we can develop the argument using the kNN density esti-
mation. These two approaches are closely related in all aspects of the error
estimation problem, and give similar results. In this section, the kNN approach
will be presented. However, in order to avoid lengthy duplication, our discus-
sion will be limited.

Bias of the kNN error: When the kNN density estimate
f;i(X) = (k-l)/Nv;(X) is used, E( pi(X)] and MSE { ii(X)] are available in
(6.91) and (6.94) respectively. Therefore, substituting E { Api(X) 1 = E { &(X) 1
-p;(X), and E{Ap? (X)) = E{[pi(X) -pi(X)I2) = MSE {pj(X)] into (7.46)
and (7.47),

(7.71)

2 k
E(Ah2(X)] G- + Ar2 - 2At (y2 - y,) (-)2'n
k N

(7.72)

where

1
.
y; (X) = - a;(X) r2 (v;p;)-2/" (7.73)
2
Substituting (7.71) and (7.72) into (7.45) and carring out the integration