Page 202 - Classification Parameter Estimation & State Estimation An Engg Approach Using MATLAB

CRITERIA FOR SELECTION AND EXTRACTION 191

Example 6.1 Interclass and intraclass distance
Numerical calculations of the example in Figure 6.2 show that before
normalization J INTRA ¼ trace(S w ) ¼ 0:016 and J INTER ¼ trace(S b ) ¼
0:54. Hence, the ratio between these two is J INTER =J INTRA ¼ 33:8.
After normalization, J INTRA ¼ N ¼ 2, J INTER ¼ J INTER=INTRA ¼ 59:1,
and J INTER =J INTRA ¼ trace(S b )=trace(S w ) ¼ 29:6. In this example,
before normalization, the J INTER =J INTRA measure is too optimistic.
The normalization accounts for this phenomenon.

6.1.2 Chernoff–Bhattacharyya distance

The interclass and intraclass distances are based on the Euclidean
metric defined in the measurement space. Another possibility is to use
a metric based on probability densities. Examples in this category are the
Chernoff distance (Chernoff, 1952) and the Bhattacharyya distance
(Bhattacharyya, 1943). These distances are especially useful in the two-
class case.
The merit of the Bhattacharyya and Chernoff distance is that an
inequality exists with which the distances bound the minimum error
rate E min . The inequality is based on the following relationship:

p ffiffiffiffiffiffi
minfa; bg ab ð6:11Þ

The inequality holds true for any positive quantities a and b. We will use
it in the expression of the minimum error rate. Substitution of (2.15) in
(2.16) yields:

Z
E min ¼ ½1 maxfPð! 1 jzÞ; Pð! 2 jzÞgŠpðzÞdz
z
ð6:12Þ
Z
¼ minfpðzj! 1 ÞPð! 1 Þ; pðzj! 2 ÞPð! 2 Þgdz
z

Together with (6.11) we have the following inequality:

Z
p ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi p ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
E min Pð! 1 ÞPð! 2 Þ pðzj! 1 Þpðzj! 2 Þdz ð6:13Þ
z