16.3 Bayes’ Theorem
AHL 4.13
- how to “reverse” a conditional probability: find \(\mathrm{P}(A \mid B)\) when you know \(\mathrm{P}(B \mid A)\)
- what a partition of the sample space is, and the law of total probability
- Bayes’ theorem for two events (\(A\) and \(A'\)) and for three events (\(A_1\), \(A_2\), \(A_3\))
- how to use a tree diagram to organise a Bayes’ theorem problem
Reversing a conditional probability
Bayes’ theorem is named after Thomas Bayes (1701–1761), an English minister and mathematician. His result was published after his death, and the French mathematician Pierre-Simon Laplace developed it much further. Today it is the basis of Bayesian statistics, and it is used in medical testing, spam filters and machine learning.
Start from the definition of conditional probability in Section 16.2:
\[ \mathrm{P}(A \mid B) = \frac{\mathrm{P}(A \cap B)}{\mathrm{P}(B)} \qquad (1) \]
The same definition with \(A\) and \(B\) swapped gives
\[ \mathrm{P}(B \mid A) = \frac{\mathrm{P}(A \cap B)}{\mathrm{P}(A)} \quad\Longrightarrow\quad \mathrm{P}(A \cap B) = \mathrm{P}(A)\,\mathrm{P}(B \mid A) . \qquad (2) \]
Put (2) into (1):
Bayes’ theorem (simple form) Not in the formula booklet
\[ \mathrm{P}(A \mid B) = \frac{\mathrm{P}(A)\,\mathrm{P}(B \mid A)}{\mathrm{P}(B)} \]
- \(A\), \(B\): two events
- \(\mathrm{P}(A \mid B)\): the probability of \(A\) given that \(B\) has happened
- \(\mathrm{P}(B \mid A)\): the probability of \(B\) given that \(A\) has happened
So if we know \(\mathrm{P}(B \mid A)\), we can find the “reverse” probability \(\mathrm{P}(A \mid B)\).
Partitions of the sample space
An event \(A\) and its complement \(A'\) partition the sample space:
- \(A\) and \(A'\) are mutually exclusive: \(A \cap A' = \varnothing\);
- together they make the whole sample space: \(A \cup A' = U\).
Any other event \(B\) is split into the two pieces \(B \cap A\) and \(B \cap A'\) (Figure 1). The pieces do not overlap, so
\[ \mathrm{P}(B) = \mathrm{P}(B \cap A) + \mathrm{P}(B \cap A') . \]
Write each piece with (2):
Law of total probability Not in the formula booklet
\[ \mathrm{P}(B) = \mathrm{P}(A)\,\mathrm{P}(B \mid A) + \mathrm{P}(A')\,\mathrm{P}(B \mid A') \]
- \(A\), \(B\): two events
- \(A'\): the complement of \(A\) (“not \(A\)”); \(A\) and \(A'\) partition \(U\)
- \(\mathrm{P}(B \mid A)\): the probability of \(B\) given that \(A\) has happened
It appears in the booklet only as the denominator of Bayes’ theorem.
Putting this into the simple form gives the version we use most often.
Bayes’ theorem (two events) In the HL formula booklet
Booklet form (\(B\) and \(B'\) partition \(U\)):
\[ \mathrm{P}(B \mid A) = \frac{\mathrm{P}(B)\,\mathrm{P}(A \mid B)}{\mathrm{P}(B)\,\mathrm{P}(A \mid B) + \mathrm{P}(B')\,\mathrm{P}(A \mid B')} \]
With the letters used on this page (\(A\) and \(A'\) partition \(U\)):
\[ \mathrm{P}(A \mid B) = \frac{\mathrm{P}(A)\,\mathrm{P}(B \mid A)}{\mathrm{P}(A)\,\mathrm{P}(B \mid A) + \mathrm{P}(A')\,\mathrm{P}(B \mid A')} \]
- \(A\), \(B\): two events
- \(A'\), \(B'\): the complements of \(A\) and \(B\)
- \(U\): the sample space
- \(\mathrm{P}(B \mid A)\): the probability of \(B\) given that \(A\) has happened
More generally, three events that are mutually exclusive and together make up \(U\) also partition \(U\).
Bayes’ theorem (three events) Partly in the HL formula booklet
Booklet form (\(B_1, B_2, B_3\) partition \(U\)):
\[ \mathrm{P}(B_i \mid A) = \frac{\mathrm{P}(B_i)\,\mathrm{P}(A \mid B_i)}{\mathrm{P}(B_1)\,\mathrm{P}(A \mid B_1) + \mathrm{P}(B_2)\,\mathrm{P}(A \mid B_2) + \mathrm{P}(B_3)\,\mathrm{P}(A \mid B_3)} \]
In the HL booklet
With the letters used on this page (\(A_1, A_2, A_3\) partition \(U\)):
\[ \mathrm{P}(A_i \mid B) = \frac{\mathrm{P}(A_i)\,\mathrm{P}(B \mid A_i)}{\mathrm{P}(A_1)\,\mathrm{P}(B \mid A_1) + \mathrm{P}(A_2)\,\mathrm{P}(B \mid A_2) + \mathrm{P}(A_3)\,\mathrm{P}(B \mid A_3)} \]
- \(B_1, B_2, B_3\) (or \(A_1, A_2, A_3\)): three events that partition \(U\) (exactly one of them happens)
- \(U\): the sample space
- \(i\): \(1\), \(2\) or \(3\)
- \(A\) (or \(B\)): the event that we know has happened
- \(\mathrm{P}(B_i \mid A)\): the probability of \(B_i\) given that \(A\) has happened
The second form is the booklet form with the letters \(A\) and \(B\) swapped. The IB syllabus uses at most three events.
In words: the numerator is one branch path of a tree diagram, and the denominator is the sum of all the paths that end in \(B\).
Method for Bayes’ theorem questions
- Draw a tree: the first branches are the partition (\(A\), \(A'\) or \(A_1, A_2, A_3\)); the second branches are \(B\) and \(B'\).
- Multiply along each path to get \(\mathrm{P}(A_i \cap B)\).
- \(\mathrm{P}(B)\) = sum of the paths ending in \(B\).
- \(\mathrm{P}(A_i \mid B) = \dfrac{\text{the path through } A_i \text{ and } B}{\mathrm{P}(B)}\).
Example 1 Two printers, \(X\) and \(Y\), print all the labels in a factory. Printer \(X\) prints \(70\%\) of the labels and printer \(Y\) prints the rest. \(2\%\) of the labels from \(X\) are misprinted, and \(5\%\) of the labels from \(Y\) are misprinted.
(a) A label is chosen at random. Find the probability that it is misprinted.
(b) A label is chosen at random and found to be misprinted. Find the probability that it came from printer \(X\).
Let \(M\) = “misprinted”.
(a) Add the two paths that end in \(M\):
\[ \mathrm{P}(M) = 0.7 \times 0.02 + 0.3 \times 0.05 = 0.014 + 0.015 = 0.029 . \]
(b) By Bayes’ theorem,
\[ \mathrm{P}(X \mid M) = \frac{\mathrm{P}(X)\,\mathrm{P}(M \mid X)}{\mathrm{P}(M)} = \frac{0.014}{0.029} = \frac{14}{29} \approx 0.483 . \]
Notice: \(X\) prints \(70\%\) of all labels but only about \(48\%\) of the misprinted ones, because \(X\) is the more reliable printer.
Example 2 There are three bags.
- Bag 1 contains 3 green and 7 yellow marbles.
- Bag 2 contains 5 green and 5 yellow marbles.
- Bag 3 contains 2 green and 8 yellow marbles.
A fair die is rolled. If it shows 1, Bag 1 is chosen; if it shows 2 or 3, Bag 2 is chosen; otherwise Bag 3 is chosen. One marble is then taken at random from the chosen bag. Given that the marble is green, find the probability that it came from Bag 2.
Let \(A_1, A_2, A_3\) be the bags chosen and \(G\) = “green”. The bags partition the sample space:
\[ \begin{gathered} \mathrm{P}(A_1) = \frac{1}{6}, \quad \mathrm{P}(A_2) = \frac{2}{6}, \quad \mathrm{P}(A_3) = \frac{3}{6}; \\ \mathrm{P}(G \mid A_1) = \frac{3}{10}, \quad \mathrm{P}(G \mid A_2) = \frac{5}{10}, \quad \mathrm{P}(G \mid A_3) = \frac{2}{10} . \end{gathered} \]
Law of total probability:
\[ \mathrm{P}(G) = \frac{1}{6} \cdot \frac{3}{10} + \frac{2}{6} \cdot \frac{5}{10} + \frac{3}{6} \cdot \frac{2}{10} = \frac{3}{60} + \frac{10}{60} + \frac{6}{60} = \frac{19}{60} . \]
Bayes’ theorem:
\[ \mathrm{P}(A_2 \mid G) = \frac{\frac{10}{60}}{\frac{19}{60}} = \frac{10}{19} . \]
Common errors
| ✗ Wrong | ✓ Right | Why |
|---|---|---|
| \(\mathrm{P}(X \mid M) = \mathrm{P}(M \mid X) = 0.02\) | \(\mathrm{P}(X \mid M) = \frac{14}{29}\) | \(\mathrm{P}(A \mid B)\) and \(\mathrm{P}(B \mid A)\) are different. That is why we need Bayes. |
| Denominator \(= \mathrm{P}(M \mid X) + \mathrm{P}(M \mid Y)\) | \(\mathrm{P}(X)\mathrm{P}(M \mid X) + \mathrm{P}(Y)\mathrm{P}(M \mid Y)\) | Each conditional probability must be weighted by the probability of its branch. |
| Bags chosen with probability \(\frac{1}{3}\) each | Read how the bag is chosen | In Example 2 the die makes the bags not equally likely. |
| Using the booklet form with the letters mixed up | Write your own events first | The booklet uses \(\mathrm{P}(B \mid A)\); match the letters to your question. |
Exercises
1 \(\mathrm{P}(A) = 0.4\), \(\mathrm{P}(B \mid A) = 0.3\) and \(\mathrm{P}(B \mid A') = 0.6\). Find
(a) \(\mathrm{P}(B)\), \(\;\)
(b) \(\mathrm{P}(A \mid B)\).
(a) \(\mathrm{P}(B) = 0.4 \times 0.3 + 0.6 \times 0.6 = 0.12 + 0.36 = 0.48\).
(b) \(\mathrm{P}(A \mid B) = \dfrac{0.12}{0.48} = 0.25\).
2 Machine \(A\) makes \(60\%\) of the bolts in a factory and machine \(B\) makes the rest. \(3\%\) of the bolts from \(A\) and \(1\%\) of the bolts from \(B\) are defective. A bolt is chosen at random. Find the probability that
(a) it is defective,
(b) it was made by \(A\), given that it is defective.
(a) \(0.6 \times 0.03 + 0.4 \times 0.01 = 0.018 + 0.004 = 0.022\).
(b) \(\dfrac{0.018}{0.022} = \dfrac{9}{11} \approx 0.818\).
3 \(2\%\) of a population have a certain condition. A test gives a positive result for \(95\%\) of people who have the condition, and for \(4\%\) of people who do not have it. A person chosen at random tests positive. Find the probability that the person has the condition.
Let \(C\) = “has the condition” and \(T\) = “tests positive”.
\(\mathrm{P}(T) = 0.02 \times 0.95 + 0.98 \times 0.04 = 0.019 + 0.0392 = 0.0582\).
\(\mathrm{P}(C \mid T) = \dfrac{0.019}{0.0582} \approx 0.326\).
Only about a third of positive results are true positives, because the condition is rare.
4 The probability that it rains on a given day is \(0.3\). If it rains, the probability that Tomás is late for school is \(0.4\); if it does not rain, the probability is \(0.1\). Given that Tomás is late, find the probability that it rained.
\(\mathrm{P}(L) = 0.3 \times 0.4 + 0.7 \times 0.1 = 0.12 + 0.07 = 0.19\).
\(\mathrm{P}(R \mid L) = \dfrac{0.12}{0.19} = \dfrac{12}{19}\).
5 Bag \(A\) contains 3 red and 2 white balls; bag \(B\) contains 1 red and 4 white balls. A fair coin is tossed to choose a bag, and one ball is taken from that bag. Find
(a) \(\mathrm{P}(\text{red})\),
(b) \(\mathrm{P}(\text{bag } A \mid \text{red})\).
(a) \(\dfrac{1}{2} \times \dfrac{3}{5} + \dfrac{1}{2} \times \dfrac{1}{5} = \dfrac{3}{10} + \dfrac{1}{10} = \dfrac{2}{5}\).
(b) \(\dfrac{\frac{3}{10}}{\frac{2}{5}} = \dfrac{3}{4}\).
6 A shop buys a component from three suppliers. Supplier 1 provides \(50\%\), supplier 2 provides \(30\%\) and supplier 3 provides \(20\%\). The percentages of faulty components are \(1\%\), \(2\%\) and \(4\%\) respectively. A component is found to be faulty. Find the probability that it came from supplier 3.
\(\mathrm{P}(F) = 0.5 \times 0.01 + 0.3 \times 0.02 + 0.2 \times 0.04 = 0.005 + 0.006 + 0.008 = 0.019\).
\(\mathrm{P}(S_3 \mid F) = \dfrac{0.008}{0.019} = \dfrac{8}{19}\).
7 In a multiple-choice test, each question has 4 options. Priya knows the answer to a question with probability \(0.6\). If she does not know it, she guesses at random. Given that she answers a question correctly, find the probability that she knew the answer.
\(\mathrm{P}(\text{correct}) = 0.6 \times 1 + 0.4 \times \dfrac{1}{4} = 0.6 + 0.1 = 0.7\).
\(\mathrm{P}(\text{knew} \mid \text{correct}) = \dfrac{0.6}{0.7} = \dfrac{6}{7}\).
8 \(\mathrm{P}(A) = 0.5\), \(\mathrm{P}(B \mid A) = 0.8\) and \(\mathrm{P}(B) = 0.6\). Find
(a) \(\mathrm{P}(A \mid B)\), \(\;\)
(b) \(\mathrm{P}(B \mid A')\).
(a) \(\mathrm{P}(A \mid B) = \dfrac{0.5 \times 0.8}{0.6} = \dfrac{0.4}{0.6} = \dfrac{2}{3}\).
(b) \(\mathrm{P}(B \cap A') = \mathrm{P}(B) - \mathrm{P}(B \cap A) = 0.6 - 0.4 = 0.2\), so \(\mathrm{P}(B \mid A') = \dfrac{0.2}{0.5} = 0.4\).
9 Mia goes to school by bus with probability \(\frac{1}{2}\), by tram with probability \(\frac{1}{3}\), and walks otherwise. The probability that she is late is \(0.2\) by bus, \(0.1\) by tram and \(0.05\) when she walks. Given that she is late, find the probability that she took the tram.
Walking has probability \(1 - \frac{1}{2} - \frac{1}{3} = \frac{1}{6}\).
\(\mathrm{P}(L) = \dfrac{1}{2}(0.2) + \dfrac{1}{3}(0.1) + \dfrac{1}{6}(0.05) = \dfrac{12}{120} + \dfrac{4}{120} + \dfrac{1}{120} = \dfrac{17}{120}\).
\(\mathrm{P}(\text{tram} \mid L) = \dfrac{\frac{4}{120}}{\frac{17}{120}} = \dfrac{4}{17}\).
10 \(\mathrm{P}(A) = p\), \(\mathrm{P}(B \mid A) = 0.9\) and \(\mathrm{P}(B \mid A') = 0.2\). Given that \(\mathrm{P}(A \mid B) = 0.6\), find \(p\).
\[ \frac{0.9p}{0.9p + 0.2(1 - p)} = 0.6 \;\Longrightarrow\; 0.9p = 0.54p + 0.12 - 0.12p \;\Longrightarrow\; 0.48p = 0.12 \;\Longrightarrow\; p = 0.25 . \]