Page 490 - Discrete Mathematics and Its Applications

7.3 Bayes’ Theorem 469

this disease would like to know the likelihood that they actually have the disease. In this section
we introduce a result that can be used to determine this probability, namely, the probability that
a person has the disease given that this person tests positive for it. To use this result, we will
need to know the percentage of people who do not have the disease but test positive for it and
the percentage of people who have the disease but test negative for it.
Similarly, suppose we know the percentage of incoming e-mail messages that are spam. We
will see that we can determine the likelihood that an incoming e-mail message is spam using
the occurrence of words in the message. To determine this likelihood, we need to know the
percentage of incoming messages that are spam, the percentage of spam messages in which
each of these words occurs, and the percentage of messages that are not spam in which each of
these words occurs.
The result that we can use to answer questions such as these is called Bayes’ theorem
and dates back to the eighteenth century. In the past two decades, Bayes’ theorem has been
extensively applied to estimate probabilities based on partial evidence in areas as diverse as
medicine, law, machine learning, engineering, and software development.

Bayes’ Theorem

We illustrate the idea behind Bayes’ theorem with an example that shows that when extra
information is available, we can derive a more realistic estimate that a particular event occurs.
That is, suppose we know p(F), the probability that an event F occurs, but we have knowledge
that an event E occurs. Then the conditional probability that F occurs given that E occurs,
p(F | E), is a more realistic estimate than p(F) that F occurs. In Example 1 we will see that
we can find p(F | E) when we know p(F), p(E | F), and p(E | F).

EXAMPLE 1 We have two boxes. The first contains two green balls and seven red balls; the second contains
four green balls and three red balls. Bob selects a ball by first choosing one of the two boxes at
random. He then selects one of the balls in this box at random. If Bob has selected a red ball,
what is the probability that he selected a ball from the first box?

Solution: Let E be the event that Bob has chosen a red ball; E is the event that Bob has chosen
a green ball. Let F be the event that Bob has chosen a ball from the first box; F is the event that
Bob has chosen a ball from the second box. We want to find p(F | E), the probability that the
ball Bob selected came from the first box, given that it is red. By the definition of conditional
probability, we have p(F | E) = p(F ∩ E)/p(E). Can we use the information provided to
determine both p(F ∩ E) and p(E) so that we can find p(F | E)?
First, note that because the first box contains seven red balls out of a total of nine balls,
we know that p(E | F) = 7/9. Similarly, because the second box contains three red balls
out of a total of seven balls, we know that p(E | F) = 3/7. We assumed that Bob selects a
box at random, so p(F) = p(F) = 1/2. Because p(E | F) = p(E ∩ F)/p(F), it follows that
7 1 7
p(E ∩ F) = p(E | F)p(F) = · = [as we remarked earlier, this is one of the quantities
9 2 18
we need to find to determine p(F | E)]. Similarly, because p(E | F) = p(E ∩ F)/p(F),it
3 1 3
follows that p(E ∩ F) = p(E | F)p(F) = 7 · 2 = 14 .
We can now find p(E). Note that E = (E ∩ F) ∪ (E ∩ F), where E ∩ F and E ∩ F are
disjoint sets. (If x belongs to both E ∩ F and E ∩ F, then x belongs to both F and F, which is
impossible.) It follows that
7 3 49 27 76 38
p(E) = p(E ∩ F) + p(E ∩ F) = + = + = = .
18 14 126 126 126 63
We have now found both p(F ∩ E) = 7/18 and p(E) = 38/63. We conclude that
p(F ∩ E) 7/18 49
p(F | E) = = = ≈ 0.645.
p(E) 38/63 76