Page 369 - Introduction to Statistical Pattern Recognition

7 Nonparametric Classification and Error Estimation 35 1

As far as the L type estimation of the kernel covariance matrix is con-
cerned, the same procedure used in the Parzen approach can be applied to the
kNN approach.

Experiment 15: Estimation of the kNN error, L and R
Same as Experiment 9, except
No. of neighbors: k = 10
Results: Table 7-5(a) [ 141
Experiment 16: Estimation of the kNN error, L and R
Same as Experiment 10, except
No. of neighbors: k = 10
Results: Table 7-5(b) [ 141

The conclusions from these experiments are similar to the ones from
Experiments 9 and 10.

7.5 Miscellaneous Topics in the kNN Approach
In this section, miscellaneous topics related to the kNN approach, which
were not discussed in the previous sections, will be studied. They are the error
control problem (the Neyman-Pearson and minimax tests), data display, pro-
cedures to save computational time, and the procedure to reduce the
classification error.

Two-Dimensional Data Display

Error control: So far, we have discussed the Bayes classifier for
minimum error by using the kNN approach. However, the discussion can be
easily extended to other hypothesis tests such as the Bayes classifier for
minimum cost, the Neyman-Pearson and minimax tests, as long as the
volumetric kNN is used. As (7.5) indicates, in the likelihood ratio test of the
kNN approach, two distances, d (Xil,)NN,X) and d2(XithN,X), are measured, and
their ratio is compared with a threshold as

where t must be determined according to which test is performed. For exam-
ple, t is selected to achieve = (E~: a preassigned value) for the Neyman-
Pearson test, and = E~ for the minimax test.