Bayes' Theorem

Bayes' theorem (or Bayes' Law and sometimes Bayes' Rule) is a direct application of conditional probabilities. The probability P(A|B) of "A assuming B" is given by the formula

which for our purpose is better written as

The left hand side P(A∩B) depends on A and B in a symmetric manner and would be the same if we started with P(B|A) instead:

This is actually what Bayes' theorem is about:

where B is an event complementary to B: B∪B = Ω, the universal event. (Of course also B∩B = Φ, an empty event.)

This is because

and, since A∩B and A∩B are mutually exclusive,

More generally, for a finite number of mutually exclusive and exhaustive events H_i (i = 1, ..., n), i.e. events that satisfy

Bayes' theorem states that

Example 1. Monty Hall Problem. [Havil, pp. 61-63]

Let A, B, C denote the events "the car is behind door A (or #1)", "the car is behind the door B (or #2)", "the car is behine the door C (or #3)." Let also M_A denote the event of Monty opening door A, etc.

You are called on stage and point to door A, say. Then

Since A, B, C are mutually exclusive and exhaustive,

Now you are given a chance to switch to another door, B or C (depending on which one remains closed.) If you stick with your original selection (A),

However, if you switch,

You'd be remiss not to switch.

Example 2. Sick Child and Doctor. [Falk, p. 48]

A doctor is called to see a sick child. The doctor has prior information that 90% of sick children in that neighborhood have the flu, while the other 10% are sick with measles. Let F stand for an event of a child being sick with flu and M stand for an event of a child being sick with measles. Assume for simplicity that F ∪ M = Ω, i.e., that there no other maladies in that neighborhood.

A well-known symptom of measles is a rash (the event of having which we denote R). P(R|M) = .95. However, occasionally children with flu also develop rash, so that P(R|F) = 0.08.

Upon examining the child, the doctor finds a rash. What is the probability that the child has measles?

Solution

Example 3. Incidence of Breast Cancer. [von Savant, pp. 103-104]

In a study, physicians were asked what the odds of breast cancer would be in a woman who was initially thought to have a 1% risk of cancer but who ended up with a positive mammogram result (a mammogram accurately classifies about 80% of cancerous tumors and 90% of benign tumors.) 95 out of a hundred physicians estimated the probability of cancer to be about 75%. Do you agree?

Solution

References

1. R. Falk, Understanding Probability and Statistics, A K Peters, 1993
2. J. Havil, Impossible?, Princeton University Press, 2008
3. M. vos Savant, The Power of Logical Thinking, St. Martin's Press, NY 1996

• What Is Probability?
• Probability Problems
• Sample Spaces and Random Variables
• Probabilities
  • Bernoulli Trials
  • Binomial Distribution
  • Proofreading Example
  • Continuous Sample Spaces
• Conditional Probability
  • Bayes' Theorem
• Dependent and Independent Events
  • Conditional Probability and Independent Events
  • Independent Events and Independent Experiments
• Algebra of Random Variables
• Expectation
• Distribution

Copyright © 1996-2008 Alexander Bogomolny

Sick Child and Doctor (Solution)

	P(M|R)	= P(R|M) P(M) / (P(R|M) P(M) + P(R|F) P(F))
		= .95 × .10 / (.95 × .10 + .08 × .90)
		≈ 0.57.

Which is nowhere close to 95% of P(R|M).

Copyright © 1996-2008 Alexander Bogomolny

Incidence of Breast Cancer (Solution)

Introduce the events:

	P	- mammogram result is positive,
	B	- tumor is benign,
	M	- tumor is malignant.

Bayes' formula in this case is

	P(M|P)	= P(P|M) P(M) / (P(P|M) P(M) + P(P|B) P(B))
		= .80 × .01 / (.80 × .01 + .10 × .99)
		≈ 0.075
		≈ 7.5%.

A far cry from a common estimate of 75%.

Copyright © 1996-2008 Alexander Bogomolny