Page 101 - Introduction to Statistical Pattern Recognition

3 Hypothesis Testing 83

suggest that appropriate statistics of r (X) can be used for model validity tests,
and that the error-reject curve is one option among many possible choices.
Three other possibilities are listed as follows [ 131.
(1) A rest bused on the mean of r(X): Since the Bayes error is E{r(X)},
the sample mean of r(X) from the data can be compared with the Bayes error
obtained from the model. This tests only one moment of the distribution of
r (X). Therefore, although simple, this does not provide sufficient information
to compare two models.
(2) Chi-square goodness-of-fiit test: The empirical distribution function,
A
P,.(r), is obtained from r(X,), . . . ,r(X,), and compared with 1-R(t) of the
model by the chi-square test. This procedure divides the space into a finite
number of bins according to the reject threshold values. The test is conducted
to compare the empirical probability in each bin with the predicted one.
(3) Kolmogorov-Smirnov test for- R(t): The empirical distribution func-
tion of r-(X) is compared with 1-R(t) by measuring the maximum difference
between them.
For details regarding the use, definition, and critical values of these tests,
the reader is refered to [lo].

Composite Hypothesis Tests

Sometimes p;(X) is not given directly, but is given by the combination of
I
p (X IO;) and p (0; mi), where p (X IO;) is the conditional density function of
mi)
X assuming a set of parameters or a parameter vector O;, and p (0; is the
I
conditional density function of €3; assuming class mi. In this case, we can cal-
culate p;(X) by
pj(X) = jp (X I 0;)p (0; mi) dOi . (3.90)
I

Once p,(X) is obtained, the likelihood ratio test can be carried out for
p I (X) and p2(X). as described in the previous sections. That is,

This is the composite hypothesis test.