Lecture Two
Probability
CH2010 AY2022 v5 PL
1
Probability and statistical inferences
Experimental design
Parameter of a
Data
Data Analysis
• Probability = if we know how the population works, we can predict what a sample will
look like.
• Statistical inference = knowing the sample, we can reconstruct what the population
looks like.
CH2010 AY2022 v5 PL
2
Intended learning objectives
• Random (probabilistic) process
• Relatively simple sample space
• A sequence of combination of outcomes
• Dependent or independent outcomes
Opportunity of an 𝐞𝐯𝐞𝐧𝐭 occuring
Probability of an event =
Sum of opportunity of all events in a 𝐬𝐚𝐦𝐩𝐥𝐞 𝐬𝐩𝐚𝐜𝐞 occuring
CH2010 AY2022 v5 PL
3
Other suggested topics of interest
• Hypothesis testing
• Lottery related problems
• Recurrence
• Roulette probability
• Apply probability in daily life
• The probability of choice?
CH2010 AY2022 v5 PL
4
2.1 Sample space
The collection of all possible outcomes is the sample space
of an operation (generating and recording outcomes).
• When tossing a fair coin, the sample space, S:
S = {H , T }
H and T are the elements, aka members or sample points in
the sample space.
CH2010 AY2022 v5 PL
5
2.1.1 Independent operations
In a combination of two independent operations, e.g. rolling a fair cubic
die while tossing a fair coin, then overall sample space contains a
combination of elements from the individual operation:
• Rolling a cubic die
S1 = {1, 2,3, 4,5, 6}
• Tossing a coin
S2 = {H , T }
• Overall
𝑆 = {1𝐻, 2𝐻, 3𝐻, 4𝐻, 5𝐻, 6𝐻, 1𝑇, 2𝑇, 3𝑇, 4𝑇, 5𝑇, 6𝑇}
CH2010 AY2022 v5 PL
6
2.1.2. Consecutive dependent operations
Example 2.1
• A coin is tossed once.
• If the outcome is H, toss the coin one more time.
• If the outcome is T, roll a cubic die once.
The sample space can be found by listing all sample points using a tree
diagram that traces all possible outcomes of consecutive operations.
N.B. The terms outcome and sample point are used interchangeably in this
course.
CH2010 AY2022 v5 PL
7
Example 2.1
N.B. The sampling points in the
sample space do not necessarily have
the same probability of occurring.
S = { HH , HT , T 1, T 2, T 3, T 4, T 5, T 6}
CH2010 AY2022 v5 PL
8
2.1.3 Consecutive independent operations
Example 2.2
• Three manufactured parts are
selected at random, inspected and
classified as defective, D, or nondefective, N.
• What are the possible outcomes?
S = { DDD, DDN , DND, DNN , NDD, NDN , NND, NNN }
CH2010 AY2022 v5 PL
9
2.1.4 Counting sample points
Multiplication rule: If an operation can be performed in n1 ways, and if
for each of these way a second operation can be performed in n2 ways,
then the two operations can be performed together in n1n2 ways.
• Rolling a cubic die (6 ways)
S1 = {1, 2,3, 4,5, 6}
• Tossing a coin (2 ways)
S2 = {H , T }
• Overall (6 × 2 = 12 ways)
𝑆 = {1𝐻, 2𝐻, 3𝐻, 4𝐻, 5𝐻, 6𝐻, 1𝑇, 2𝑇, 3𝑇, 4𝑇, 5𝑇, 6𝑇}
CH2010 AY2022 v5 PL
10
2.1.4.1 Multiplication rule
Generalised multiplication rule: if an operation can be performed in n1
ways, and if for each of these ways a second operation can be performed in n2
ways, and if for each of these ways a third operation can be performed in n3
ways, and so forth, then a sequence of k operations can be performed in
n1n2…nk ways.
• From a deck of 52 cards, how many ways are there to pick a hand of 5
cards?
• Answer:
52 ´ 51´ 50 ´ 49 ´ 48 = 311,875,200
• On your calculator, this can be computed by 52 nPr 5
CH2010 AY2022 v5 PL
11
2.1.4.2 Permutation
A permutation is an arrangement of all parts of a set of objects.
• How many ways to arrange three letters: a, b and c?
Answer: abc, acb, bac, bca, cab, cba → 6 ways
3 for the first letter, 2 for the second letter, 1 for the last letter, i.e.
3×2×1=6
CH2010 AY2022 v5 PL
12
2.1.4.2 Factorial expression
In general, n distinct object can be arranged in
n(n – 1)(n – 2)…(3)(2)(1) ways
= n! ways
n! = n factorial.
e.g. 1! = 1, 2! = 2, 3! = 6, 4! = 24, 5! = 120…
N.B. 0! = 1 ( watch https://www.youtube.com/watch?v=X32dce7_D48
to see why)
CH2010 AY2022 v5 PL
13
Example 2.3
How many ways are there to arrange 5 distinct playing cards?
5×4×3×2×1 = 120
or
5! = 120
How many ways can you draw any 5 cards from a deck of 52 cards?
52×51×50×49×48 = 311,875,200
or
52!/(52 − 5)! = 311,875,200
On your calculator, this can be computed by 52 nPr 5
CH2010 AY2022 v5 PL
14
Example 2.3 (continued)
3. How many 5-card combinations you could draw from a deck of 52
cards, if the order at which specific cards are drawn does not matter:
52×51×50×49×48
= 2,598,960
5×4×3×2×1
or
52!
= 2,598,960
52 − 5 ! 5!
On your calculator, this can be computed by 52 nCr 5
CH2010 AY2022 v5 PL
15
To generalise Example 2.3
To pick r distinct objects from a pool of n objects, where the order matters, the
number of ways to pick is
n Pr =
n!
(n - r )!
To pick r distinct objects from a pool of n, where the order doesn’t matter, the
number of combinations that can be picked is:
ænö
n!
n Cr = ç
ç r ÷÷ = (n - r )!r!
è ø
CH2010 AY2022 v5 PL
16
Pop quiz 1
(Let’s suppose) a cohort of CCEB freshers includes 180
CBE students, 120 BIE students and 80 CBC.
1.
We would like to form a committee of 6 students to
represent the freshers. How many possible ways are
there to form this committee?
2.
If the committee must consist of 2 students from
CBE, 2 students from BIE and 2 students from CBC.
How many possible ways are there to form this
committee?
CH2010 AY2022 v5 PL
Join at slido.com #2918 007
17
2.1.5 Sample statement
When there are too many sample points to be listed in a line, the
sample space is best described algebraically, by a sample statement
or rule method.
• The sample space includes all points on or inside a circle of radius
2 and centers at the origin:
{
x
2
}
S = ( x, y ) | x 2 + y 2 £ 4
-2
2
y
-2
CH2010 AY2022 v5 PL
18
2.2 Events
An event is a particular set of outcomes that are of interest.
It is a subset of the sample space.
From Example 2.2:
Event B corresponds to more than one sampled parts being defective
amongst the three inspected.
S = { DDD, DDN , DND, DNN , NDD, NDN , NND, NNN }
B = { DDN , DND, NDD, DDD}
BÍS
CH2010 AY2022 v5 PL
19
2.2.1 Complement and union
• The complement of B with respect to S is the subset of S that
are not in B, denoted B'.
S = { DDD, DDN , DND, DNN , NDD, NDN , NND, NNN }
B = { DDN , DND, NDD, DDD}
B' = {DNN , NDN , NND, NNN }
• The union of B and B' generates the full sample space
B'È B = S
CH2010 AY2022 v5 PL
20
2.2.2. Intersection
• Event A corresponds to at least one sample parts being defective amongst the
three inspected.
S = { DDD, DDN , DND, DNN , NDD, NDN , NND, NNN }
A = {DDD, DDN , DND, DNN , NDD, NDN , NND}
B = { DDN , DND, NDD, DDD}
• A and B are subsets of the same sample space and share some common elements.
• The shared elements are the intersection of A and B, denoted by:
A Ç B = {DDN , DND, NDD, DDD}
CH2010 AY2022 v5 PL
21
2.2.3 Mutual exclusivity
If two events have no elements in common, then their intersection is a
null set, ϕ.
Also the two events are said to be mutually exclusive, or disjoint.
• For example, B and B’ are mutually exclusive, i.e.
B'ÇB = f
CH2010 AY2022 v5 PL
22
2.2.4 Venn diagram
Example 2.4
S = {1,2,3,4,5,6,7}
A = {1,2,4,7}
B = {1,2,3,6}
C = {1,3,4,5}
CH2010 AY2022 v5 PL
23
2.2.4 Venn diagram
A È C = {1,2,3,4,5,7}
B'ÇA = {4,7}
A Ç B Ç C = {1}
( A È B ) Ç C ' = {2,6,7}
CH2010 AY2022 v5 PL
24
2.2.5 Useful identities
𝐴∩∅=𝜑
𝐴∪∅=𝐴
𝐴 ∩ 𝐴′ = 𝜑
𝐴 ∪ 𝐴′ = 𝑆
𝑆′ = 𝜑
∅′ = 𝑆
𝐴′ ′ = 𝐴
𝐴 ∩ 𝐵 ′ = 𝐴′ ∪ 𝐵′
𝐴 ∪ 𝐵 ′ = 𝐴′ ∩ 𝐵′
CH2010 AY2022 v5 PL
25
Summary
Sample space
• Tree diagram
• Permutation
• Combination
Counting events
Counting sample points
Algebraic analysis
Venn diagram
Complex events
• Union
• Intersection
• Complements
CH2010 AY2022 v5 PL
26
Lecture Two
Probability
CH2010 AY2022 v5 PL
27
2.3 Probability of an event
The likelihood of an event occurring is called probability,
between 0 and 1.
0 corresponds to the case when the event in no way can occur.
1 corresponds to the case when the event will definitely occur.
0 £ P ( A) £ 1
P(f ) = 0
CH2010 AY2022 v5 PL
P(S ) = 1
28
2.3.1 Counting sample points
For a sample space with equally probable sample points, the probability of an
event occurring equals the ratio of the sample points of the event to that of the
entire sample space.
Example 2.5
• What’s the probability of drawing a hand of five cards containing 2 A and 3 J
from a deck of 52 cards?
æ 4ö
4!
• Number of possible arrangements for 2 A = çç 2 ÷÷ = (4 - 2)!2! = 6
è ø
æ 4ö
4!
ç
÷
=
=4
• Number of possible arrangements for 3 J = ç ÷
è 3 ø (4 - 3)!3!
CH2010 AY2022 v5 PL
29
Example 2.5 continued
• Number of possible arrangements combining 2 A and 3 J:
æ 4 öæ 4 ö
çç ÷÷çç ÷÷ = 6 ´ 4 = 24
è 2 øè 3 ø
This is the number of sample points of the event
• The probability of drawing 2 A and 3 J:
Number of sample points of the event
=
24
= 0.9 ´10 -5
2,598,960
Probability
Number of sample points of the sample space
CH2010 AY2022 v5 PL
30
Chance of winning a lottery in Singapore (Toto)
Prize Group
Matches
Prize
1
6 numbers
38% of prize pool
2
5 numbers plus the 8% of prize pool
additional number
3
5 numbers
4
4 numbers plus the 3% of prize pool
additional number
5
4 numbers
6
3 numbers plus the $25 per winning
additional number combination
7
3 numbers
49
= 1/13,983,816
6
6 1
49
/
= 1/2,330,636
5 1
6
6 42
49
/
= 1/55,491
5 1
6
6 1 42
49
/
= 1/22,197
6
4 1 1
1/
5.5% of prize pool
6
4
$50 per winning
combination
42
49
/
= 1/1,083
2
6
6
3
$10 per winning
combination
6
3
CH2010 AY2022 v5 PL
1
1
49
42
/
= 1/22,197
6
2
42
49
/
= 1/61
3
6
31
2.3.2 Probability of union of mutually exclusive events
If A1, A2, A3, … is a sequence of mutually exclusive events,
then
P( A1 È A2 È A3 È !) = P( A1 ) + P( A2 ) + P( A3 ) + !
CH2010 AY2022 v5 PL
32
2.3.2 Probability of union of mutually exclusive events
It follows that:
CH2010 AY2022 v5 PL
33
Example 2.6
A fair coin is tossed twice, what is the probability to get at least one
head?
S = {HH , HT , TH , TT }
A = {HH , HT , TH }
P( A) = P(HH ) + P(HT ) + P(TH )
1 1 1 3
P ( A) = + + =
4 4 4 4
CH2010 AY2022 v5 PL
34
Example 2.6 (continued)
• If a coin is unevenly loaded so that H is twice as likely as T, what is the
probability to get at least one head?
S = {HH , HT , TH , TT }
A = {HH , HT , TH }
• We let the probabilities of H and T in each through to be 2w and w,
respectively
P(HH ) = 2 w ´ 2 w = 4 w2
P(HT ) = 2 w ´ w = 2 w2
P(HH ) + P(HT ) + P(TH )
P ( A)
=
P ( A) =
P(HH ) + P(HT ) + P(TH ) + P(TT )
P(S )
P(TH ) = w ´ 2 w = 2 w2
(4 + 2 + 2) w2
8
=
=
(4 + 2 + 2 + 1) w2 9
P(TT ) = w ´ w = w2
CH2010 AY2022 v5 PL
35
2.3.3 Probability of union
The additive rule, applies to unions of any two events:
P( A È B) = P( A) + P( B) - P( A Ç B)
CH2010 AY2022 v5 PL
36
Example 2.7
A fair cubic die is rolled. What is the probability of getting an odd number or a
prime number?
• Sample space: {1, 2, 3, 4, 5, 6}
• Let event A be getting an odd number: {1, 3, 5}
• Let event B be getting a prime number: {2, 3, 5}
• A∩B = {3, 5}
3 1
𝑃 𝐴 = =
6 2
3 1
𝑃 𝐵 = =
6 2
2 1
𝑃 𝐴∩𝐵 = =
6 3
1 1 1 2
𝑃 𝐴∪𝐵 =𝑃 𝐴 +𝑃 𝐵 −𝑃 𝐴∩𝐵 = + − =
2 2 3 3
CH2010 AY2022 v5 PL
37
2.3.3 Probability of union
For three events A, B and C,
P( A È B È C ) = P( A) + P( B) + P(C )
- P( A Ç B) - P( A Ç C ) - P( B Ç C ) + P( A Ç B Ç C )
S
A∩B
A
B
A∩B∩C
A∩C
The pattern in the formulation
can be followed to calculate
probabilities of unions of more
than three events.
B∩C
C
CH2010 AY2022 v5 PL
38
Pop quiz 2
1. If we toss 5 identical and balanced coin simultaneously. What is the
probability of getting 3 heads and 2 tails?
Method 1: counting sample space
5
3 = 10 = 0.3125
2A
32
Method 2: Probability of intersection
5
0.5B 0.5C = 0.3125
3
CH2010 AY2022 v5 PL
Join at slido.com #2918 007
39
Pop quiz 2 continued
2. If we toss 5 identical and unfairly balanced coin
simultaneously. Each coin is 0.6 likely to be
“head up” and 0.4 likely to be “tail up”. What is
the probability of getting 3 heads and 2 tails?
5
0.6B 0.4C = 0.3456
3
Join at slido.com #2918 007
CH2010 AY2022 v5 PL
40
2.6 Conditional probability
The probability of event B when it is known that event A has
occurred is a conditional probability denoted:
P( B | A)
It reads: “probability of B, given A.”
CH2010 AY2022 v5 PL
41
Example 2.9
When rolling a fair cubic die, what is the probability of getting more
than 3 given that we already knew that the outcome is a prime number?
What’s the probability of getting more than a 3 if we already know the
outcome is an odd number?
CH2010 AY2022 v5 PL
42
Example 2.9 continued
Let A be the event of getting a prime number, A = {2, 3, 5}
Let B be the event of getting an odd number, B = {1, 3, 5}
Let C be the event of getting >3, C = {4, 5, 6}
To satisfy both A and C, we compute: (A ∩ C) = {5}
To satisfy both B and C, we compute: (B ∩ C) = {5}
1
𝑃 𝐶𝐴 =
3
1
𝑃 𝐶𝐵 =
3
CH2010 AY2022 v5 PL
43
2.6 Conditional probability
In general, one can compute:
P( A Ç B)
P( B | A) =
P( A)
provided
P( A) > 0
If and only if two A and B are independent
P( B | A) = P( B)
P( A | B) = P( A)
Otherwise A and B are dependent.
CH2010 AY2022 v5 PL
44
2.7 Dependent and independent events
Example of independent events
• Rolling a cubic die multiple times, the probability of getting a certain
number is independent of preceding and succeeding results.
Examples of dependent event
• Serving cards from a deck of cards, the probability of getting a
specific card, changes with what the previously served cards are.
(Counting cards works based on this principle).
CH2010 AY2022 v5 PL
45
2.8 Product rule
An important result that follows the conditional probability
formula is the product rule, aka multiplication rule:
P( A Ç B)
P( B | A) =
P( A)
Þ P( A Ç B) = P( A) P( B | A)
Recall that, for independent events, P( B | A) = P( B)
Þ P( A Ç B) = P( A) P( B)
This is also the condition for A and B to be independent.
CH2010 AY2022 v5 PL
46
Example 2.10
• Consider the following circuit, with the reliabilities (probability of not failing) of
various components labelled.
Assume that the reliabilities of A, B, C and D are independent of each other (in
reality, not always the case)
a) Find the reliability of the entire system
b) Find the probability of C failing if the entire system still works
CH2010 AY2022 v5 PL
47
Example 2.10 continued
a) Find the reliability of the entire system
For the system to work, A must work, B must work, C or D must work.
P(system working) = P[A Ç B Ç (C È D )]
Events, A, B and (C ∪ D) are independent
P(system working) = P( A) P( B) P(C È D )
CH2010 AY2022 v5 PL
48
Example 2.10 continued
P(system working) = P( A) P( B) P(C È D )
P(C È D ) = 1 - P(C ) + P( D) - P(C Ç D)
= P(C ) + P( D) - P(C ) P( D)
= 0.8 + 0.8 - 0.8 ´ 0.8 = 0.96
P(system working) = 0.9 ´ 0.9 ´ 0.96 = 0.7776
CH2010 AY2022 v5 PL
49
Example 2.10 continued
b) Find the probability of C failing if the entire system still works
P=
P( the system works but C does not work )
P( the sytem works)
P( A Ç B Ç C 'Ç D) 0.9 ´ 0.9 ´ (1 - 0.8) ´ 0.8
P=
=
= 0.1667
P( the sytem works)
0.7776
CH2010 AY2022 v5 PL
50
Product rule for multiple events
The product rule can be extended to a collection of k events.
CH2010 AY2022 v5 PL
51
Pop quiz 3
0.96
0.9
0.96
A
B
C
0.85
D
What is the probability that the system still works if
component D has already failed?
𝑃=
𝑃(𝐴 ∩ 𝐵 ∩ 𝐶
𝑃(𝐷 ! )
∩ 𝐷!)
Join at slido.com #2918 007
= 𝑃 𝐴 𝑃 𝐵 𝑃 𝐶 = 0.82944
CH2010 AY2022 v5 PL
52
Summary
• Probability of union of events
• Additive rule
P( A È B ) = P ( A) + P ( B ) - P ( A Ç B )
• Mutually exclusive events
P( A È B) = P( A) + P( B)
• Conditional probability
• Product rule
P( B | A) =
P( A Ç B)
P( A)
P( A Ç B) = P( B | A) P( A)
• Independent events
P( A Ç B) = P( A) P( B)
CH2010 AY2022 v5 PL
53
Casino roulette game
American-style Roulette game with 36 numbered slots, a 0 and a 00. The chance
of the ball falling in each slot is supposed to be identical.
Roulette Bets & Payouts
1. Single number bet pays 35 to 1. Also called “straight up.”
2. Double number bet pays 17 to 1. Also called a “split.”
3. Three number bet pays 11 to 1. Also called a “street.”
4. Four number bet pays 8 to 1. Also called a “corner bet.”
5. Five number bet pays 6 to 1. Only one specific bet which includes the
following numbers: 0-00-1-2-3.
6. Six number bets pays 5 to 1. Example: 7, 8, 9, 10, 11, 12. Also called a “line.”
7. Twelve numbers or dozens (first, second, third dozen) pays 2 to 1.
8. Column bet (12 numbers in a row) pays 2 to 1.
9. 18 numbers (1-18) pays even money.
10. 18 numbers (19-36) pays even money.
11. Red or black pays even money.
12. Odd or even bets pay even money.
Source of image and rules: The Venetian Resort Las Vegas
CH2010 AY2022 v5 PL
54
Russian roulette
A six-shot revolver have only one bullet in it (in one of the chambers).
We do not know which chamber is loaded.
We start with two players.
The two players take turns to point the revolver at their own heads and
pull the trigger.
The player who gets shot in the head losses the game.
Then the game stops. The other player wins the game.
Do not try this in real life!
Which player is more likely to win the game, the one who shots first or
the one who shots second?
CH2010 AY2022 v5 PL
55