Page 115 - Introduction to Statistical Pattern Recognition
P. 115
3 Hypothesis Testing 97
3.4 Upper Bounds on the Bayes Error
It is evident from the preceding discussion that the calculation of the
error probability is, in general, a difficult task. Even when observation vectors
have a normal distribution, we must resort to numerical techniques. However,
a closed-form expression for the error probability is the most desirable solution
for a number of reasons. Not only is the computational effort greatly reduced,
since we need only to evaluate a formula, but more importantly, the use of the
closed-form solution provides insight into the mechanisms causing the errors.
This information is useful later when we consider the problem of feature selec-
tion.
When we cannot obtain a closed-form expression for the error probabil-
ity, we may take some other approach. We may seek either an approximate
expression for the error probability, or an upper bound on the error probability.
In this section, we will discuss some upper. hounds of error pr.ohahility.
The Chernoff and Bhattacharyya Bounds
Chernoff bound: The Bayes error is given in (3.7) as
