Chapter 4
Bayes’ Rule
Learning Objectives
• Bayes’ Rule
• Tree Approach
• Matrix Approach
Bayes’ Rule
• Bayes’ rule is an extension to the law of conditional probability to
allow revision of original/prior probabilities with new
information.
• It was developed by and named for Thomas Bayes.
Thomas Bayes
According to Wikipedia: Thomas Bayes (1701 – 1761) was an English
statistician, philosopher and Presbyterian minister who is known for
formulating a specific case of the theorem that bears his name: Bayes'
theorem. Bayes never published what would become his most famous
accomplishment; his notes were edited and published after his death
by Richard Price.
Bayes’ Rule
• Bayesian applications
• Market segmentation
• HR, hiring choices
• Financial modeling
• Corporate development, sale vs. IPO
• Mergers and acquisitions
• Green light, new product decisions
• 10 Must Watch Movies on Data
Science and Machine Learning
Law of Conditional Probability
• Conditional probability for event A given event B is:
𝑃 𝐴|𝐵 =
𝑃(𝐴∩𝐵)
𝑃(𝐵)
• P(A | B) is the ratio of the relative size of A ∩ B to event B
From Week 2.4b PPT
Bayes’ Rule
• An extension of Conditional Probability：
• 𝑃 𝐴|𝐵 =
𝑃(𝐴∩𝐵)
𝑃(𝐴∩𝐵)
=
𝑃 𝐵 𝐴 ∗𝑃 𝐴 +𝑃 𝐵 𝐴’ ∗𝑃(𝐴’)
𝑃(𝐴∩𝐵)+𝑃(𝐴’∩𝐵)
• Used to revise prior probabilities with new information
Application
Prior
New
Probabilities
Information
of Bayes’
Rule
Revised
Probabilities
Bayes’ Rule
An Example Scenario: A particular formulation of an over-the-counter drug is produced by
only two companies, Prairie Pharmaceuticals and Badlands Generics. Suppose Prairie
produces 65% of the drug and Badlands produces 35%. 8% of the users of the drug
produced by Prairie show some side effects and 12% of the Badlands users show similar side
effects. A customer randomly picks up one of these drugs at the pharmacy. What is the
probability that Prairie produced the drug? What is the probability that Badlands produced
the drug?
The customer uses the product and develops side effects. Now what is the probability that
Prairie produced the drug? That Badlands produced the drug?
Bayes’ Rule
• Two techniques
• Matrix Approach
• Tree Approach
Bayes’ Rule - Example
• Suppose a survey reveals that
• The GMAT is generally required
to apply to MBA programs.
• Variety of prep courses designed
to help improve GMAT scores
(200-800)
• An applicant has determined
that he needs ≥650 to get into
an MBA program, but he feels
that his probability of getting
≥650 is 0.10 (10% chance).
• Among GMAT scorers of ≥650, 52% took
a prep course
• Whereas among GMAT scorers of < 650,
only 23% took a prep course
• He is considering taking a prep course
that costs $500.
• He is willing to do so only if his
probability of achieving ≥650 doubles.
• What should he do?
Bayes’ Rule - Example
• Suppose a survey reveals that
• The GMAT is generally required to
apply to MBA programs.
• Variety of prep courses designed to
help improve GMAT scores (200 –
800)
• An applicant has determined that
he needs ≥650 to get into an MBA
program, but he feels that his
probability of getting ≥650 is 0.10
(10% chance): P(≥650)=0.1.
• Among GMAT scorers of ≥650, 52% took a prep
course: P(prep | ≥650)=0.52
• Whereas among GMAT scorers of < 650, only
23% took a prep course: P(prep | <650)=0.23
• He is considering taking a prep course that
cost $500.
• He is willing to do so only if his probability of
achieving ≥650 doubles.
• What should he do?
• P(≥650 | prep) =?
Given:
Bayes’ Rule – Example: Matrix Approach
P(≥650)=0.1
P(Prep | ≥650)=0.52
P(Prep | <650)=0.23
Question: P(≥650 | prep) =?
Bayes’ Rule – Example: Tree approach
≥ 650
0.10
< 650
0.90
Prior Prob.
P(Ei)
Bayes’ Rule – Example: Tree approach
Prep – 0.52
≥ 650
0.10
NO Prep – 0.48
< 650
0.90
Prep – 0.23
NO Prep – 0.77
Prior Prob.
P(Ei)
Conditional Prob.
P(d | Ei)
Bayes’ Rule – Example: Tree approach
Prep – 0.52
0.052
NO Prep – 0.48
0.048
Prep – 0.23
0.207
NO Prep – 0.77
0.693
≥ 650
0.10
< 650
0.90
Prior Prob.
P(Ei)
Conditional Prob.
P(d | Ei)
Joint Prob.
P(Ei ∩ d)
Bayes’ Rule – Example: Tree approach
Prep – 0.52
≥ 650
0.10
< 650
0.90
Prior Prob.
P(Ei)
0.052
NO Prep – 0.48
0.048
Prep – 0.23
0.207
NO Prep – 0.77
0.693
Conditional Prob.
P(d | Ei)
Joint Prob.
P(Ei ∩ d)
= 0.052 + 0.207
= 0.259
Bayes’ Rule – Example: Tree approach
P(≥650 & prep)
Prep – 0.52
0.052
P(prep)
≥ 650
0.10
< 650
0.90
Prior Prob.
P(Ei)
= 0.052 + 0.207
= 0.259
NO Prep – 0.48
0.048
Prep – 0.23
0.207
Revised Prob.
= 0.052 / 0.259
= 0.201
NO Prep – 0.77
0.693
P(≥650 | prep)
Conditional Prob.
P(d | Ei)
Joint Prob.
P(Ei ∩ d)
Bayes’ Rule: Example 2
•
•
•
•
1% of the population have a certain disease. There is an established test for this disease.
80% of the tests detect the disease when it is there.
10% of the tests detect the disease when it’s not there.
A randomly selected person, John, gets a test and his test detects the disease. What is the
probability that John has the disease?