Page 34 - Computational Statistics Handbook with MATLAB

20 Computational Statistics Handbook with MATLAB

Bayes’ Theorem can be derived from the definition of conditional probabil-
ity (Equation 2.5). Writing this in terms of our events, we are interested in the
following probability:

PM ∩( F)
PM F( ) = ---------------------------- , (2.9)
A
A PF()
where PM A F( ) represents the posterior probability that the part came from
manufacturer A, and F is the event that the piston ring failed. Using the Mul-
tiplication Rule (Equation 2.6), we can write the numerator of Equation 2.9 in
terms of event F and our prior probability that the part came from manufac-
turer A, as follows

PM A ∩( F) PM A )PF M A )
(
(
(
PM A F) = ---------------------------- = ----------------------------------------- . (2.10)
PF() PF()
The next step is to find PF() . The only way that a piston ring will fail is if:
1) it failed and it came from manufacturer A or 2) it failed and it came from
manufacturer B. Thus, using the third axiom of probability, we can write

(
PF() = PM A ∩( F) + PM B ∩ F) .
Applying the Multiplication Rule as before, we have

(
(
(
(
PF() = PM A )PF M A ) + PM B )PF M B . ) (2.11)
Substituting this for PF() in Equation 2.10, we write the posterior probability
as
PM )PF M( )
(
PM F( ) = --------------------------------------------------------------------------------------- . (2.12)
A
A
(
(
(
(
A
PM A )PF M A ) + PM B )PF M B )
(
Note that we need to find the probabilities PF M A ) and PF M B ) . These are
(
the probabilities that a piston ring will fail given it came from the correspond-
ing manufacturer. These must be estimated in some way using available
information (e.g., past failures). When we revisit Bayes’ Theorem in the con-
text of statistical pattern recognition (Chapter 9), these are the probabilities
that are estimated to construct a certain type of classifier.
Equation 2.12 is Bayes’ Theorem for a situation where only two outcomes
are possible. In general, Bayes’ Theorem can be written for any number of
, , , whose union makes up the entire sam-
mutually exclusive events, E 1 … E k
ple space. This is given below.

© 2002 by Chapman & Hall/CRC