Page 303 - Introduction to Statistical Pattern Recognition

6 Nonparametric Density Estimation 285

x.

M
‘X 5NN
Pairwise classifier
Fig. 6-5 Distribution of class centers and classifiers.

neighbor class. If the distance to the closest neighbor, d(M,,MNN), is much
smaller than the distances to the other neighbors, the pairwise error between 0,
and o,, dominates the total error. However, (6.108) and (6.109) suggest that
d(M,MLNN) almost the same, regardless of k. The number of classes, N, and
is
the distribution of the M,’s (uniform, normal and so on) have very little effect.
Only the dimensionality, n, has a significant effect on d (M,MkNN). Since all
neighboring M,’s are equally distanced from MI, the error from each pair, &,,,
can be added up to produce a large total error, E,. Figure 6-6 shows expen-
mental results indicating the relationship between E, and oIE,E { dNN(X)} for
various values of n. Note that n is the intrinsic dimension of the distribution of
the M,’s.

Experiment 5: Error for N-class problem
Data: Mi - uniform with mean 0 and covariance I
x - N(M,,G*I) i = 1, ..., N
Dimensionality: n = 5, 10, 20

Sample size: N = 10n (10n classes)
1 On samples/class
No. of trials: T = 10
Classifier: Bisectors between the generated M, ’s.
Results: Fig. 6-6 [ 181
Although the results are not shown here, the experiment confirmed that these
curves are almost invariant for various values of N and distributions of M,’s.
The theoretical saturation error for (3-00 is (l-I/N)Zl (100%) for N-class
problem.