Probability
Sample spaces, events, probabilities, conditional
probabilities, independence, Bayes’ formula
Chapter 1
1
Sample spaces and events
Envision an experiment for which the result is unknown.
• The collection of all possible outcomes is called the sample
space.
– Sample spaces can be discrete: {HH, HT, TH, TT} for two coins
– Or continuous, e.g., [0, ), 2
• A set of outcomes, or subset of the sample space, is called
an event. If E and F are events,
– EF is the event that either E or F (or both) occurs
– EF = EF is the event that both E and F occur. If EF =  then E
and F are called mutually exclusive.
c
– The complement of E is E  S \ E : the event that E does not occur
Chapter 1
2
Probability
• A probability space is a three-tuple (S ,, P) where S is a
sample space,  is a collection of events from the sample
space and P is a probability law that assigns a number to
each event in . P() must satisfy:
– P(S) = 1
– 0  P(A)  1
– For any collection
of mutually exclusive events E1, E2, …,


 
P  Ei    P  En 
 n 1  n 1
– If S is discrete, then  is the set of all subsets of S
– If S is continuous, then  can be defined in terms of “basic
events of interest,” e.g., if S = [0, 1] the basic events could be all (a,
b) with 0  a < b  1. Then  would be the set of intervals along
with all their countable unions and intersections.
Chapter 1
3
Probability
• If S consists of n equally likely outcomes, then the
probability of each is 1/n.
• Since E and Ec are mutually exclusive and E  Ec = S,
1  PS   PE
E c   P  E   P  E c  , so P  E c   1  P  E 
• P(E  F) = P(E) + P(F) – P(EF)
• P E  F  G) = P(E) + P(F) + P(G) – P(EF) – P(EG) –
P(FG) + P(EFG)
• Generalizes to any number of events.
• Use Venn diagrams!
Chapter 1
4
Conditional Probabilities
• If A and B are events with P(B)  0, the conditional
probability of A given B is
P  A  B
P  A B 
P  B
• This formula also tells how to find the probability of AB:
P  A  B   P  A B  P  B   P  A P  B A
• A and B are independent events if P  AB   P  A P  B 
or equivalently if P  A | B   P  A
If events are mutually exclusive, are they independent? Vice versa?
• A set of events E1, E2, …, En are independent if for every
subset E1’, E2’, …, Er’, P  E1 E2 Er   P  E1  P  E2  P  Er 
'
'
'
'
'
'
(pairwise independence is not enough – see Example 1.10)
Chapter 1
5
Bayes’ Formula
• The law of total probability says that if E and F are any
two events, then since E = EF  EFc,
P  E   P  EF  EF c   P  EF   P  EF c 
 P  E | F  P  F   P  E | F c  1  P  F  
• It can be generalized to any partition
of S: F1, F2, …, Fn
n
mutually exclusive events with Fi  S
P  E   i 1 P  E | Fi P  Fi 
n
i 1
• Bayes’ formula is relevant if we know that E occurred and
we want to know which of the F’s occurred.
P  Fj | E  
P  EFj 
PE

P  E | Fj  P  Fj 

n
i 1
P  E | Fi  P  Fi 
Chapter 1
6