Page 187 - Introduction to Statistical Pattern Recognition

4 Parametric Classifiers 169

ferent from the ones used for test (given Xi's), the algorithm of (3.1 19)-(3.128)
must be used to calculate the theoretical error.

4.4 Other Classifiers

In this section, we will discuss subjects which were left out in the previ-
ous discussions. They are the piecewise classifiers and some of the properties
in binary inputs.

Piecewise Classifiers

If we limit our discussion to two-class problems, quadratic or linear
classifiers have wide applications. However, when we have to handle three or
more classes, a single quadratic or linear classifier cannot be adopted effec-
tively. Even in two-class problems, the same is true when each class consists
of several clusters. For these cases, a set of classifiers, which is called a piece-
wise classifier, gives increased flexibility.
Piecewise quadratic for multiclass problems: For multiclass problems,
the multihypothesis test in the Bayes sense gives the best classifier with regard
to minimizing the error. That is, from (3.44)

Pkpp(X) = max Pipi(X) + X E q . (4.147)
I
If the distributions of X for L classes are normal, (4.147) is replaced by

1 1
min[-(X - M,)'z;'(x - M,) + - ln I I - w,] , (4.148)
i 2 2

where max is changed to min because of the minus-log operation. Note that
the normalized distance of X from each class mean, Mi, must be adjusted by
two constant terms, (112)ln ICi I and In Pi. Equation (4.148) forms a piecewise
quadratic, boundary.

Piecewise quadratic for multicluster problems: For multicluster prob-
lems, the boundary is somewhat more complex. Assuming that L = 2, and that
each distribution consists of m, normal clusters with the cluster probability of
Pi, for the jth cluster, the Bayes classifier becomes