Chapter 2. Axioms of Probability
2
3
4
5
6
Sample Space
• Sample space: the set of all possible outcomes of an
experiment.
– Coin flipping S = {H, T}
– Flipping two coins
S = {(H, H), (H, T), (T, H), (T, T)}
– Order of finish in a race among the 7 horses having post
positions 1, 2, 3, 4, 5, 6, 7, then
S = {all 7! permutations of (1, 2, 3, 4, 5, 6, 7)}
– Measuring the lifetime of a light bulb
S = {x: 0 ≤ x < ∞}
7
Sample Space
• Weather forecast
– Sunny, cloudy, rainy, snowy, …
• Stock market
– Up, down, no change
– [0, +∞)
8
Events
• Event: any subset E of the sample space S.
– A set consisting some possible outcomes of the
experiment.
• For flipping coin, E = {H} is the event that a
head is seen.
• Flipping two coins, E = {(H, H), (H, T)} is the
event that a head appears on the first coin.
• Light bulb, E = {x: 0 ≤ x ≤ 100} is the event
that the light bulb lasts less than 100 hours.
9
Operations for events
• Union E  F
– Outcomes that are either in E or in F, or in both.
E  {H }, F  {T }, E  F  {H , T }
E  {( H , T ), ( H , H )}, F  {( H , T ), (T , T )},
E  F  {( H , T ), ( H , H ), (T , T )}
• Intersection of E and F, EF
– Outcomes that are both in E and F
– Also written as E  F
E  {( H , H ), ( H , T ), (T , H )}, F  {( H , T ), (T , H ), (T , T )}
EF  {( H , T ), (T , H )}
10
• If E and F have no outcomes in common, then
we say E and F are mutually exclusive
EF  
• Union and intersection of multiple events
n
For E1 , E2 , En , theunion of theseeventsis denotedby  Ei .
i 1
For E1 , E2 , En , theintersection of theseeventsis
n
denotedby  Ei .
i 1
11
• Complement Ec
– Outcomes in S that are not in E.
• E is contained in F, E  F
– All of the outcomes in E are also in F.
12
Venn diagram
• Venn diagram: a graphical representation for
illustrating logical relations among events.
– Union (AUB)
S
A
U
B
13
Venn diagram: intersection
S
A
A∩B
B
14
Venn diagram: compliment
S
Ac
A
15
Venn diagram: contained
S
B
A
BA
16
Some rules
• Commutative laws
EF  FE
EF  FE
• Associative laws
( E  F )  G  E  ( F  G)
( EF)G  E( FG)
• Distributive laws
( E  F )G  EG  FG
EF  G  ( E  G)(F  G)
• The rules can be verified using Venn diagram
17
( E  F )  G  E  ( F  G)
( E  F )G  EG  FG
( EF)G  E( FG)
EF  G  ( E  G)(F  G)
E
F
G
18
DeMorgan’s laws
c
n


c
  E i    E i
i 1
 i 1 
n
c
n


c
  E i    E i
i 1
 i 1 
n
19
Intersection of Greek, Latin and
Russian upper case letters
20
Looks cool, but…
21
Beef
Wheat
Hamburger
22
Axioms of probability
• Probability as a measure of frequency of occurrences.
n( E )
P( E )  lim
n 
n
• In coin flipping experiment, how do we know that the
proportion of heads obtained in the first n flips will
converge to some value as n gets large?
• How do we know that, if the experiment is repeatedly
performed a second time, we shall again obtain the
same limiting proportion of heads?
23
Probability as a measure of belief
• Sports
• Horse racing
• Investment decisions
24
Frequentist vs Bayesian
• Different ways of defining probability
• Different types of statistical approaches
– Frequentist’s and Bayesian methods
• Different philosophies for statistics and science
– Should we bring one’s belief into science?
• Many interesting articles
–
–
–
–
Link 1
Link 2
Link 3
Link 4
25
Axioms of probability
• How about a set of simpler and more selfevident axioms?
• We assume that for each event E in the sample
space S there exist a value P(E), referred to as
the probability of E.
26
Axioms of probability
• Axiom 1
0  P( E )  1
• Axiom 2
P(S) = 1
27
• Axiom 3
For any sequence of mutuallyexclusiveevents
E1 , E2 ,  (thatis, eventsfor whichEi E j  
when i  j ),
  
P  Ei    P( Ei )
 i 1  i 1
We refer toP(E)as theprobability of theevent E.
28
• Ex 3a
Tossing a coin.
If we assume that a head is as likely to appear
as a tail, then we have
P({H}) = P({T})=1/2
On the other hand, if the coin were biased and
we felt that a head were twice as likely to
appear as a tail, then we have
P({H}) = 2/3 and P({T}) = 1/3
29
• Ex 3b
Rolling a die.
If we suppose that all six sides are equally
likely to appear, then we would have 1/6 for
each possible number from 1 to 6.
The probability of rolling an even number
would be
P({2, 4, 6}) = P({2}) + P({4}) + P({6}) = 1/2
30
2.4 Some simple propositions
• Proposition 4.1 P(Ec) = 1-P(E)
• Proposition 4.2 If E  F , thenP( E)  P( F ).
F  E  EcF
P( F )  P( E )  P( E c F )
31
Inclusion-exclusion identity
• Proposition 4.3
P( E  F )  P( E )  P( F )  P( EF)
E  F can be written as the union of two disjoint events
E and E c F . From Axiom 3
P ( E  F )  P( E  ( E c F ))
 P( E )  P( E c F )
Since F  EF  E c F , from Axiom3
P( F )  P( EF )  P( E c F ) or
P( E c F )  P( F )  P( EF )
32
• Prove using Venn diagram
P( E  F )  P( E )  P( F )  P( EF)
S
E
E∩F
F
33
Movie recommendation system
• A movie recommendation system asks a user some
questions and then decides what movies it recommends
to the user. The goal is to make good recommendations
by asking as few questions as possible.
• After asking a few questions to a user, the system finds
from its database that people with similar
characteristics will watch movie A with probability .5,
watch movie B with probability .4 and watch both
movies with probability .3. What is the probability the
user will watch at least one movie? If the probability is
greater of equal than .7, the system will recommend the
two movies without asking more questions. Otherwise,
more questions will be asked.
34
Movie recommendation system
• MA: event that the user will watch movie A
• MB: event that the user will watch movie B
P(M A
P( M A
M B ) : Probability that at least one movie will be watched.
M B )  P( M A )  P( M B )  P( M A M B )
 .5  .4  .3  .6
35
• Ex 4a
J is taking two books along on her holiday
vacation. With probability .5 she will like the
first book; with probability .4 she will like the
second book; with probability .3 she will like
both book. What is the probability she like
neither book?
P( B1  B2 )  P( B1 )  P( B2 )  P( B1 B2 )
 .5  .4  .3  .6
P( B1c B2c )  P((B1  B2 )c )  1  P( B1  B2 )  .4
36
• Probability of any of the three events E or F or
G occurs.
P( E  F  G)  P[(E  F )  G]
• By proposition 4.3
P( E  F )  P(G)  P[(E  F )G]
• Distributive law
P( E  F  G )
 P( E )  P( F )  P( EF )  P(G)  P( EG  FG)
 P( E )  P( F )  P( EF )  P(G)  P( EG)  P( FG)  P( EGFG)
 P( E )  P( F )  P(G)  P( EF )  P( EG)  P( FG)  P( EFG)
37
38
• Proposition 4.4 Inclusion-exclusion identity
n
P( E1  E2    En )   P( Ei )   P( Ei1 Ei2 )  
i 1
i1 i2
 (1) r 1
 P( E E
i1 i2 ir
i1
i2
 Eir )
   (1) n 1 P( E1 E2  En )
T hesummat ion
 P( E E
i1 i2 ir
i1
i2
 Eir ) is takenover all of
n
the  possible subset s of size r of theset {1, 2, n}.
r
39
2.5 Sample space having equally likely
outcomes
• For S = {1, 2, …, N}
P({1}) = P({2}) = … = P({N})
• P({i})=1/N
• For any event E
P(E) = (number of outcomes in E) / (number
of outcomes in S)
Probability of any events equals the proportion
of outcomes in the sample space that are
contained in E.
40
• Ex 5a
• If two dice are rolled, what is the probability
that the sum of the upturned faces will equal
7?
• There are 6 possible outcomes, namely (1,6),
(2,5), (3,4), (4,3), (5,2), (6,1), that give the sum
of 7.
• Since there are totally 36 outcomes, the desired
probability is 1/6.
41
• Ex 5b
• If 3 balls are randomly drawn from a bowl
containing 6 white and 5 black balls, what is
the probability that one of the drawn balls is
white and the other two black?
6*5* 4  5*6* 4  5* 4*6 120  120  120 4


11*10*9
990
11
 6  5 
  
 1  2   4
11
11
 
3
42
• Ex 5d
• An urn contains n balls, of which one is
special. If k of these balls are withdrawn one at
a time, with each selection being equally likely
to be any of the balls that remain at the time,
what is the probability that the special ball is
chosen?
1 n  1
 

1 k  1  k
n
n
 
k 
43
• Ex 5e
• Suppose that n+m balls, of which n are red and
m are blue, are arranged in a linear order in
such a way that all (n+m)! possible orderings
are equally likely. If we record the result of
this experiment by only listing the colors of the
successive balls, show that all the possible
results remain equally likely.
n!m!
(n  m)!
44
• Ex 5f
• A poker hand consists of 5 cards. If the cards
have distinct consecutive values and are not all
of the same suit, we say that the hand is a
straight. For instance, a hand consisting of the
five of spades, six of spades, seven of spades,
eight of spades, and nine of hearts is a straight.
What is the probability that one is dealt a
straight?
10(45  4)
 0.0039
 52
 
5
45
• Ex 5g
• A 5-card poker hand is said to be a full house
if it consists of 3 cards of the same
denomination and 2 cards of the same
denomination. (That is, a full house is three of
a kind plus a pair) What is the probability that
one is dealt a full house?
 4  4 
1312    
 2  3   0.0014
 52
 
5
46
• Ex 5h
In the game of bridge the entire deck of 52
cards is dealt out to 4 players. What is the
probability that
(a) one of the players receive all 13 spades;
 39 

4
13
,
13
,
13

  6.3 1012
52




13
,
13
,
13
,
13


(b) each players receives 1 ace?
48



4!
12,12,12,12  0.105
52




13,13,13,13
47
• Ex 5i
If n people are present in a room, what is the
probability that no two of them celebrate their
birthday on the same day of the year? How large
need n be so that this probability is less than 1/2?
365 364 363(365 n  1) / 365n
When n ≥ 23, this probability is less than ½.
 23
   253
2
With 50 people, the probability that at least two
share the same birthday is approximately .97.
48
• Ex 5j
• A deck of 52 playing cards is shuffled and the
cards turned up one at a time until the first ace
appears. Is the next card – that is, the card
following the first ace – more likely to be the
ace of spades or the two of clubs?
The probabilities are the same:
51! 1

52! 52
49
• Ex 5l
• A total of 36 members of a club play tennis, 28
play squash, and 18 play badminton.
Furthermore, 22 of the members play both
tennis and squash, 12 play both tennis and
badminton, 9 play both squash and badminton,
and 4 play all three sports. How many
members of this club play at least one of these
sports?
P (T  S  B )
 P (T )  P ( S )  P ( B )  P (TS )  P (TB )  P ( SB)  P (TSB)
36  28  18  22  12  9  4

N
43

N
50
A lazy professor
• Consider the following problem. Suppose that
a certain professor is lazy and lets his students
grade their own quizzes. After he collects the
quizzes from his n students, he randomly
assigns the quizzes back to these n students for
grading. If a student is assigned his or her own
quiz, we say that it is a match. What is the
probability that none of the n students is a
match?
51
• Ei, i = 1, 2,..,N the event that the ith student picks
his/her own quiz.
• We first calculate the complementary probability
of at least one student’s picking his/her own quiz.
N  N
P  Ei    P( Ei )   P( Ei1 Ei2 ) 
i1 i2
 i 1  i 1
 (1) n 1

i1 i2

in
P( Ei1 Ei2
 (1) N 1 P( E1E2
Ein )
EN )

i1 i2
P( Ei1 Ei2
Ein ) 
( N  n)!
N!
P( Ei1 Ei2
Ein ) 
N !( N  n)!
1

( N  n)!n! N ! n!
in
N 
1 1
N 1 1
P  Ei   1     (1)
2! 3!
N!
 i 1 
The probability that none of the students picks their own quiz is
1 1
1
1  1     (1) N
 e1  .36788
2! 3!
N!
52
• Ex 5m The matching problem
• Suppose that each of N men at a party throws
his hat into the center of the room. The hats are
first mixed up, and then each man randomly
selects a hat. What is the probability that (a)
none of the men selects his own hat; (b)
exactly k of the men select their own hats?
53
Summary of Chapter 2
•
•
•
•
Sample space
Event
Union, intersection, complement of events
Axioms 1, 2, 3
• Propositions 4.1-4
54
Homework problems
• Chapter 2 Problems 3, 5, 7, 8, 9, 12, 14, 31, 37,
41.
55