Page 349 - Computational Statistics Handbook with MATLAB

338 Computational Statistics Handbook with MATLAB

bility of false alarm. Once we have the threshold, we can determine the
probability of correctly classifying the observations belonging to the target
class.
Before we go on to describe the receiver operating characteristic (ROC)
curve, we first describe some terminology. For any boundary we might set for
the decision regions, we are likely to make mistakes in classifying cases.
There will be some target patterns that we correctly classify as targets and
some we misclassify as non-targets. Similarly, there will be non-target pat-
terns that are correctly classified as non-targets and some that are misclassi-
fied as targets. This is summarized as follows:

• True Positives - TP: This is the fraction of patterns correctly classi-
fied as target cases.
• False Positives - FP: This is the fraction of non-target patterns
incorrectly classified as target cases.
• True Negatives - TN: This is the fraction of non-target cases cor-
rectly classified as non-target.
• False Negatives - FN: This is the fraction of target cases incorrectly
classified as non-target.

In our previous terminology, the false positives (FP) correspond to the false
alarms. Figure 9.8 shows these areas for a given decision boundary.
A ROC curve is a plot of the true positive rate against the false positive rate.
ROC curves are used primarily in signal detection and medical diagnosis
[Egan, 1975; Lusted, 1971; McNeil, et. al., 1975; Hanley and McNeil, 1983;
Hanley and Hajian-Tilaki, 1997]. In their terminology, the true positive rate is
also called the sensitivity. Sensitivity is the probability that a classifier will
classify a pattern as a target when it really is a target. Specificity is the prob-
ability that a classifier will correctly classify the true non-target cases. There-
fore, we see that a ROC curve is also a plot of sensitivity against 1 minus
specificity.
One of the purposes of a ROC curve is to measure the discriminating power
of the classifier. It is used in the medical community to evaluate the diagnos-
tic power of tests for diseases. By looking at a ROC curve, we can understand
the following about a classifier:

• It shows the trade-off between the probability of correctly classify-
ing the target class (sensitivity) and the false alarm rate (1 – spec-
ificity).
• The area under the ROC curve can be used to compare the perfor-
mance of classifiers.

We now show in more detail how to construct a ROC curve. Recall that the
likelihood ratio is given by

© 2002 by Chapman & Hall/CRC