1/18/2024
[Last time]
Repeated Combination 𝐻𝑟𝑛 = 𝐶𝑟𝑛+𝑟−1
Conditional Probability
𝑃(𝐵|𝐴) is the probability of event B occurring, given that event A has already
occurred.
• The conditional probability of an event B given an event A, denoted as
𝑃(𝐴∩𝐵)
𝑃(𝐵 | 𝐴), is: 𝑃(𝐵 | 𝐴) =
𝑓𝑜𝑟 𝑃(𝐴) > 0.
𝑃(𝐴)
• From a relative frequency perspective of n equally likely outcomes:
– 𝑃(𝐴) = (number of outcomes in A) / n
– 𝑃(𝐴 ∩ 𝐵) = (number of outcomes in 𝐴 ∩ 𝐵) / n
– 𝑃(𝐵 | 𝐴) = number of outcomes in 𝐴 ∩ 𝐵 / number of outcomes in A
Example 2-22
400 parts are classified by surface flaws and as functionally defective. There
are 4 probabilities conditioned on flaws in the below table.
Parts Classified
Surface Flaws
Defective
Yes (F ) No (F' )
Yes (D )
10
18
No (D' )
30
342
Total
40
360
<sol>
Total
28
372
400
Multiplication Rule
• The conditional probability can be rewritten to generalize a multiplication
rule.
𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐵|𝐴) · 𝑃(𝐴) = 𝑃(𝐴|𝐵) · 𝑃(𝐵)
[Note] 𝑃(𝐴|𝐵) =
𝑃(𝐴∩𝐵)
𝑃(𝐴∩𝐵)
𝑃(𝐵)
𝑃(𝐴)
, 𝑃(𝐵 | 𝐴) =
Two Mutually Exclusive Subsets
• A and A are mutually exclusive.
• 𝐴 ∩ 𝐵 and 𝐴′ ∩ 𝐵 are mutually exclusive
• B = (𝐴 ∩ 𝐵) ∪ (𝐴′ ∩ 𝐵 )
Multiplication Rule
• The conditional probability can be rewritten to generalize a multiplication
rule.
𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐵|𝐴) · 𝑃(𝐴) = 𝑃(𝐴|𝐵) · 𝑃(𝐵)
𝑃(𝐴|𝐵) =
𝑃(𝐴∩𝐵)
𝑃(𝐴∩𝐵)
𝑃(𝐵)
𝑃(𝐴)
, 𝑃(𝐵 | 𝐴) =
Two Mutually Exclusive Subsets
• A and A are mutually exclusive.
• 𝐴 ∩ 𝐵 and 𝐴′ ∩ 𝐵 are mutually exclusive
• B = (𝐴 ∩ 𝐵) ∪ (𝐴′ ∩ 𝐵 )
Total Probability Rule
For any two events A and B
𝑃(𝐵) = 𝑃(𝐵 ∩ 𝐴) + 𝑃(𝐵 ∩ 𝐴′ ) = 𝑃(𝐵|𝐴) ⋅ 𝑃(𝐴) + 𝑃(𝐵|𝐴′ ) ⋅ 𝑃(𝐴′ )
𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐵|𝐴) · 𝑃(𝐴) = 𝑃(𝐴|𝐵) · 𝑃(𝐵)
Example 2-27: Semiconductor Contamination
Information about product failure based on chip manufacturing process
contamination is given below. Find the probability of failure.
Probability
Level of
Probability
of Failure Contamination of Level
0.1
0.005
High
Not High
0.2
0.8
<sol>
Let F denote the event that the product fails.
Let H denote the event that the chip is exposed to high contamination during
manufacture
Total Probability Rule (Multiple Events)
• A collection of sets E1, E2, … Ek such that 𝐸1 ∪ 𝐸2 ∪ … ∪ 𝐸𝑘 = 𝑆 is said to be
exhaustive.
• Assume E1, E2, … Ek are k mutually exclusive and exhaustive. Then
𝑃(𝐵) = 𝑃(𝐵 ∩ 𝐸1 ) + 𝑃(𝐵 ∩ 𝐸2 ) + ⋯ + 𝑃(𝐵 ∩ 𝐸𝑘 )
= 𝑃(𝐵|𝐸1 ) ⋅ 𝑃(𝐸1 ) + 𝑃(𝐵|𝐸2 ) ⋅ 𝑃(𝐸2 ) + ⋯ + 𝑃(𝐵|𝐸𝑘 ) ⋅ 𝑃(𝐸𝑘 )
Example 2-28: Semiconductor Failures
Probability
of Failure
0.100
0.010
0.001
Level of
Contamination
High
Medium
Low
Probability
of Level
0.2
0.3
0.5
Find 𝑃(𝐹)
<sol>
Event Independence
• Two events are independent if any one of the following equivalent
statements is true:
1. 𝑃(𝐴 | 𝐵) = 𝑃(𝐴)
2. 𝑃(𝐵 | 𝐴) = 𝑃(𝐵)
3. 𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴) · 𝑃(𝐵)
• This means that occurrence of one event has no impact on the probability of
occurrence of the other event.
Example 2-30: Flaws and Functions
Table 1 provides an example of 400 parts classified by surface flaws and as
(functionally) defective. Suppose that the situation is different and follows
Table 2. Let F denote the event that the part has surface flaws. Let D denote the
event that the part is defective. The data shows whether the events are
independent.
TABLE 2 Parts Classified (data
TABLE 1 Parts Classified
chg'd)
Surface Flaws
Surface Flaws
Yes
No
Yes
No
Defective (F)
(F') Total Defective (F)
(F') Total
Yes (D)
10
18
28
Yes (D)
2
18
20
No (D')
30
342
372 No (D')
38
342
380
Total
40
360
400
Total
40
360
400
<sol>
[Note] 𝑃(𝐹 |𝐷) =
2
20
= 0.1 = 𝑃(𝐹 ) =
40
400
= 0.1
Example: If we randomly draw two cards from a deck of 52 cards without
replacement. Let A: the first card is K and B: the second card is K. Are events A
and B independent?
<sol>
Example: If we randomly draw two cards from a deck of 52 cards with
replacement. Let A: the first card is K and B: the second card is K. Are events A
and B independent?
<sol>
[Note] Let A and B are two events and 𝑃(𝐴) ≠ 0, 𝑃(𝐵) ≠ 0
1. If A and B are independent, then A and B are not mutually exclusive.
2. If A and B are mutually exclusive, then A and B are not independent.
3. If A and B are independent, then 𝑨′ & 𝑩′ , 𝐴′ &𝐵, 𝐴&𝐵′ are all independent,
respectively.
<pf>
Bayes’ Theorem:
Let the event of interest B happen under event A with a known conditional
probability 𝑃(𝐵|𝐴). Assume the probability of A is known (prior probability).
Then the conditional (posterior) probability of the A given that event B
happened is:
𝑃(𝐴|𝐵 ) =
𝑃(𝐵|𝐴) ⋅ 𝑃(𝐴)
, 𝑓𝑜𝑟 𝑃(𝐵) > 0
𝑃(𝐵)
Example 2-36 (Example 2-27)
Probability
of Failure
Level of
Contamination
Probability
of Level
0.1
0.005
High
Not High
0.2
0.8
Find the probability of a High level of contamination given that a failure
occurred.
<sol>
Bayes’ Theorem with Total Probability
If 𝐸1 , 𝐸2 , . . . 𝐸𝑘 are k mutually exclusive and exhaustive events and B is any
event,
𝑃(𝐵|𝐸1 ) ⋅ 𝑃(𝐸1 )
𝑃 (𝐵|𝐸1 ) ⋅ 𝑃(𝐸1 )
𝑃(𝐸1 |𝐵) =
=
𝑃(𝐵)
𝑃(𝐵|𝐸1 ) ⋅ 𝑃(𝐸1 ) + ⋯ + 𝑃(𝐵 |𝐸𝑘 ) ⋅ 𝑃(𝐸𝑘 )
𝐹𝑜𝑟 𝑃(𝐵) > 0
[Note]
- Total probability expression of the denominator
- Numerator is always one term of the denominator
Example:
The following problem was posed by Casscells, Schoenberger, and Grayboys
(1978) to 60 students and staff at an elite medical school:
If a test to detect a disease whose prevalence is 1/1000 has a false positive rate
of 5%, what is the chance that a person found to have a positive result actually
has the disease, assuming you know nothing about the person’s symptoms or
signs? Assuming that the probability of a positive result given the disease is 1,
Let D denote that the event that you have the disease.
Let S denote that the event that your test is positive.
Casscells et al. found that only 18% of participants gave this answer. The most
frequent response was 95%
Before the test, your chance was 0.1%. After the positive result, your chance is
now 2%.
Example: Drug test for MLB players
Suppose 7% of all major league baseball players use an illegal steroid. A league
test is considered 96% effective in detecting the illegal steroid, but
unfortunately, it also incorrectly tests positive (indicating steroid use) for a
person who does not have the illegal steroid in their system 12% of the time.
1. What proportion of the league players will test positive for the illegal
steroid?
2. Given that someone tests positive for an illegal steroid, what is the
probability that they do not have an illegal steroid in their system?
3. Suppose four players have taken steroids. What is the probability all of them
are detected by the test?
<sol>
A: test positive
S: a player takes steroid
Naïve Bayes:
Let S: Spam email. A, B, C, D are just four key words from the email you try to
classify.
𝑃(𝐴, 𝐵, 𝐶, 𝐷|𝑆) ⋅ 𝑃(𝑆)
𝑃(𝐴, 𝐵, 𝐶, 𝐷)
𝑃(𝐴, 𝐵, 𝐶, 𝐷|𝑆) ⋅ 𝑃(𝑆)
=
𝑃(𝐴, 𝐵, 𝐶, 𝐷 |𝑆) ⋅ 𝑃(𝑆) + 𝑃(𝐴, 𝐵, 𝐶, 𝐷|𝑆 ′ ) ⋅ 𝑃(𝑆 ′ )
𝑃(𝑆|𝐴, 𝐵, 𝐶, 𝐷) =
Assume independence
𝑃(𝐴, 𝐵, 𝐶, 𝐷|𝑆) = 𝑃(𝐴|𝑆) ⋅ 𝑃(𝐵|𝑆) ⋅ 𝑃(𝐶 |𝑆) ⋅ 𝑃(𝐷|𝑆)
𝑃(𝐴, 𝐵, 𝐶, 𝐷|𝑆′) = 𝑃(𝐴|𝑆′) ⋅ 𝑃(𝐵|𝑆 ′ ) ⋅ 𝑃(𝐶 |𝑆 ′ ) ⋅ 𝑃(𝐷|𝑆′)
You should be able to calculate all the probabilities on the right through the
training data.
𝑃(𝑆′|𝐴, 𝐵, 𝐶, 𝐷 ) = 1 − 𝑃(𝑆|𝐴, 𝐵, 𝐶, 𝐷)
∴ 𝑖𝑓 𝑃(𝑆|𝐴, 𝐵, 𝐶, 𝐷) ≥ 0.5 → 𝐶𝑙𝑎𝑠𝑠𝑖𝑓𝑦 𝑡ℎ𝑒 𝑒𝑚𝑎𝑖𝑙 𝑎𝑠 𝑎 𝑆𝑝𝑎𝑚!
𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒, 𝑡ℎ𝑒 𝑒𝑚𝑎𝑖𝑙 𝑖𝑠 𝑛𝑜𝑡 𝑎 𝑆𝑝𝑎𝑚.
For example:
𝐼𝑓 𝑃(𝑆|𝐴, 𝐵, 𝐶, 𝐷) = 0.4 < 0.5 → 𝑁𝑜𝑡 𝑎 𝑠𝑝𝑎𝑚
𝐼𝑓 𝑃(𝑆|𝐴, 𝐵, 𝐶, 𝐷) = 0.6 ≥ 0.5 → 𝑆𝑝𝑎𝑚