1. EVENTS AND THEIR PROBABILITIES
1.1 Events
Example of an experiment: Toss a coin and observe up face.
Possible outcomes of this experiment: H (head), T (tail)
Sample space (set of all possible outcomes): {H,T}
Events (subsets of the sample space): {}, {H}, {T}, {H,T}
The event {H} occurs, if a head is observed.
The event {T} occurs, if a tail is observed.
The event {H,T} occurs, if a head or a tail is observed.
The event {} cannot occur.
The probability that {H} occurs is 12 .
The probability that {T} occurs is 12 .
The probability that {H,T} occurs is 1.
The probability that {} occurs is 0.
In abbreviated form we may write
P({H})= 12 , P({T})= 12 , P({H,T})=1, and P({})=0
Since the sample space (usually denoted by Ω) contains
all possible outcomes and therefore occurs with probability
1, it is also called the certain event.
Since the empty set (usually denoted by ∅) contains no
outcomes and therefore occurs with probability 0, it is also
called the impossible event.
1.1
1.2 Complementary events
The complement of an event A (denoted by Ac) contains all
possible outcomes that are not in A. Two events A and B are
called complementary if Ac=B.
The complements of the four events in the coin-tossing
experiment are given by
∅c=Ω, {H}c={T}, {T}c={H}, and Ωc=∅.
We observe that
and
P(∅)+P(∅c)=P(∅)+P(Ω)=0+1=1,
P({H})+P({H}c)=P({H})+P({T})= 12 + 12 =1,
P({T})+P({T}c)=P({T})+P({H})= 12 + 12 =1,
P(Ω)+P(Ωc)=P(Ω)+P(∅)=1+0=1.
In general, we have
P(A)+P(Ac)=1
or, equivalently,
P(Ac)=1-P(A).
• Exercise 1.a: Suppose that a die is tossed.
(i) List all subsets of Ω={1,2,3,4,5,6} that contain two
outcomes.
(ii) Find the complements of the events {4} and {1,3,6}.
(iii) What is the probability of {4}c given that P({4})= 16 ?
1.2
1.3 Unions and intersections of events
The union of two events A and B (denoted by A∪B) occurs
if either A occurs or B occurs or both occur.
The intersection of two events A and B (denoted by A∩B)
occurs if both A and B occur.
As an illustration, we form the unions and intersections of
some subsets of the probability space Ω={1,2,3,4,5,6}.
{1,2}∪{1,3,5}={1,2,3,5}
{1,3}∪{1,3,5}={1,3,5}
{2,4}∪{1,3,5}={1,2,3,4,5}
The union of {1,2} and {1,3,5} occurs if {1,2,3,5} occurs.
The union of {1,3} and {1,3,5} occurs if {1,3,5} occurs.
The union of {2,4} and {1,3,5} occurs if {1,2,3,4,5} occurs.
{1,2}∩{1,3,5}={1}
{1,3}∩{1,3,5}={1,3}
{2,4}∩{1,3,5}=∅
The intersection of {1,2} and {1,3,5} occurs if {1} occurs.
The intersection of {1,3} and {1,3,5} occurs if {1,3} occurs.
The intersection of {2,4} and {1,3,5} never occurs.
1.3
1.4 Mutually exclusive events
Two events A and B are called mutually exclusive
(or disjoint) if A∩B=∅.
In the coin-tossing experiment we have
and
P({H}∪{T})=P(Ω)=1
P({H})+P({T})= 12 + 12 =1,
but at the same time we have also
and
P({H}∪Ω)=P(Ω)=1
P({H})+P(Ω)= 12 +1= 32 .
In the latter case, the probabilty of the union is not equal to
the sum of the probabilities because the two events {H} and
Ω are not disjoint. The outcome H is contained in both
events and is therefore counted twice.
General rules:
P(A1∪A2)=P(A1)+P(A2) if A1∩A2=∅
P(A1∪A2∪A3)=P(A1)+P(A2)+P(A3)
if A1∩A2=A1∩A3=A2∩A3=∅
P(A1∪A2∪…)=P(A1)+P(A2)+… if Ai∩Aj=∅ ∀i≠j
1.4
1.5 Summing simple event propabilities
A simple event contains exactly one outcome.
In the die-tossing experiment the simple events are {1},{2},
{3},{4},{5}, and {6}. In this experiment, it is natural to
assume that all simple event probabilities are equal, i.e.,
P({1})=P({2})=P({3})=P({4})=P({5})=P({6})= 16 .
Of course, two different simple events are always disjoint.
To find the probability of the event A={2,5} we can
therefore simply add the probabilities P({2}) and P({5}) of
the simple events {2} and {5}, i.e.,
{2}∩{5}=∅ ⇒ P({2,5})=P({2})+P({5})= 16 + 16 = 62 = 13 .
• Exercise 1.b: What is the probability that an even number
is observed?
{2}∩{4}={2}∩{6}={4}∩{6}=∅
⇒ P({2,4,6})=P({2})+P({4})+P({6})= 16 + 16 + 16 = 36 = 12 .
• Exercise 1.c: What is the probability that an odd number
is observed?
{1}∩{3}={1}∩{5}={3}∩{5}=∅
⇒ P({1,3,5})=P({1})+P({3})+P({5})= 16 + 16 + 16 = 36 = 12 .
Alternatively, we can find the solution with the help of 1.b.
P({1,3,5})=P({2,4,6}c)=1-P({2,4,6})=1- 12 = 12 .
1.5
1.6 Additive rule of probability
Suppose that a die is tossed. What is the probability of
observing a prime number or an even number?
P({2,3,5}∪{2,4,6})=P({2,3,4,5,6})= 56
In contrast,
P({2,3,5})+P({2,4,6})= 36 + 36 =1,
because 2 is contained both in {2,3,5} and in {2,4,6} and is
therefore counted twice.
Equality can be obtained by subtracting P({2}) from the sum
of the probabilties P({2,3,5}) and P({2,4,6}), i.e.,
P({2,3,5}∪{2,4,6})=P({2,3,5})+P({2,4,6})-P({2}).
In general, the sum of the probabilities differs from the
probability of the union just by the probabilty of the
intersection.
Additive rule of probability:
P(A1∪A2)=P(A1)+P(A2)-P(A1∩A2)
1.6
1.7 Properties of conditional probability
Suppose that after having carried out an experiment we
know that event B has occurred. We can then conclude
firstly that the actual outcome of the experiment cannot be
contained in any event that is disjoint to B and secondly that
it must be contained in every superset of B.
Denoting the conditional probability of event A, given
that event B has occurred, by P(AB) we can write
and
(i) P(AB)=0 if A∩B=∅
(ii) P(AB)=1 if A⊇B.
Once we know that the actual outcome is contained in B,
the conditional probability of any other event A depends
exclusively on those of its outcomes that are contained in
A∩B. Clearly, none of the outcomes contained in A-B can
possibly be the actual outcome. Accordingly, we have
(iii) P(AB)=P(A∩BB).
Finally, suppose that A⊆B. Then B can be divided into
the two disjoint subsets A and B-A. Since the knowledge
that B has occurred cannot change the odds between A and
B-A, we also have
(iv) P(AB):P(B-AB)=P(A):P(B-A) if A⊆B.
1.7
1.8 Formal definition of conditional probability
Apart from the special properties
and
(i)
(ii)
(iii)
(iv)
P(AB)=0 if A∩B=∅,
P(AB)=1 if A⊇B,
P(AB)=P(A∩BB),
P(AB):P(B-AB)=P(A):P(B-A) if A⊆B,
the conditional probability must also meet the ordinary
standards of probability. In particular, since the sets A∩B,
B-A, A-B, and (A∪B)C are pairwise disjoint, we have
P(A∩BB)+P(B-AB)+P(A-BB)+P((A∪B)CB)
= P((A∩B)∪(B-A)∪(A-B)∪(A∪B)CB)=P(ΩB)=1.
It follows from (i) that P(A-BB)=0, P((A∪B)CB)=0, hence
P(A∩BB)+P(B-AB)=1.
Dividing both sides of this equation by P(A∩BB), using
B-A=B-A∩B, and applying (iv) gives
P ( A ∩ B B) P ( B − A B)
P(B− A ∩B B)
P(B− A ∩B)
1
=
+
=1+
=1+
P(A ∩B)
P ( A ∩ B B) P ( A ∩ B B) P ( A ∩ B B)
P(A∩ B B)
P ( A ∩B) P(B− A ∩B) P(A∩B)+ P(B−A ∩B)
P ( B)
=
+
=
=
.
P(A∩B)
P ( A ∩ B)
P(A ∩B)
P ( A ∩ B)
Finally, using (iii) we obtain a universal formula for
conditional probability.
P(A∩B)
P(AB)=P(A∩BB)= P(B)
1.8
1.9 Conditional versus unconditional probabilities
• Exercise 1.d: Suppose that a die is tossed. What is the
conditional probability of observing a prime number given
that an even number is observed?
P({2,3,5}{2,4,6})=P({2,3,5}∩{2,4,6})/P({2,4,6})
=P({2})/P({2,4,6})= 16 / 63 = 13
In contrast, the unconditional probability is given by
P({2,3,5})= 63 = 12 .
Obviously, the occurrence of {2,4,6} alters the probability
that {2,3,5} has occurred!
• Exercise 1.e: Suppose that a die is tossed. What is the
conditional probability of observing a number less than 3
given that an even number is observed?
P({1,2}{2,4,6})=P({1,2}∩{2,4,6})/P({2,4,6})
=P({2})/P({2,4,6})= 16 / 63 = 13
In this case, the conditional probability equals the
unconditional probability, because
P({1,2})= 62 = 13 .
1.9
1.10 Independent events
In general, the conditional probability P(AB) equals the
unconditional probability P(A) only if
P(A∩B)=P(A)P(B).
In this case, the probabilty P(B) is canceled out:
P(AB)=
P(A ∩ B) P(A)P(B)
=
=P(A)
P(B)
P(B)
Two events A and B are said to be independent if
P(A∩B)=P(A)P(B).
Note: In contrast to the definition of P(AB), where it is
required that P(B)≠0, the definition of independence is
always meaningful.
• Exercise 1.f: Suppose a die is tossed.
(i) Are the events {1,3,5} and {2,4,6} independent?
No, because
whereas
P({1,3,5}∩{2,4,6})=P(∅)=0,
P({1,3,5})P({2,4,6})= 12 12 = 14 .
(ii) Are the events {1,3,5} and {1,2} independent?
Yes, because
and also
P({1,3,5}∩{1,2})=P({1})= 16
P{1,3,5})P({1,2})= 12 13 = 16 .
1.10
1.11 Further exercises
• Exercise 1.g: Suppose a die is tossed.
(i) What is the probability of observing a six or
an odd number?
P({6}∪{1,3,5})=P({1,3,5,6}= 64 = 32
(ii) Find P({2,4,6}{1,3,5}), P({3,5}{1,3,5}),
P({2,3,5}{1,3,5}), and P({2,3,4,6}{1,3,5}).
P({2,4,6}{1,3,5})=P({2,4,6}∩{1,3,5})/P({1,3,5})
=P(∅)/P({1,3,5})=0/ 63 =0
P({3,5}{1,3,5})=P({3,5}∩{1,3,5})/P({1,3,5})
=P({3,5})/P({1,3,5})= 62 / 63 = 32
P({2,3,5}{1,3,5})=P({2,3,5}∩{1,3,5})/P({1,3,5})
=P({3,5})/P({1,3,5})= 62 / 63 = 32
P({1,2,3,4}{2,4})=P({1,2,3,4}∩{2,4})/P({2,4})
=P({2,4})/P({2,4})=1
(iii) Are the events {2,3,6} and {1,2,3,4} independent?
Yes, because
P({2,3,6}∩{1,2,3,4})=P({2,3})= 62 = 13
and
P({2,3,6})P({1,2,3,4})= 36 64 = 13 .
1.11
• Exercise 1.h: Suppose a coin is tossed twice and the
sequence of heads and tails is recorded.
(i) List the sample space.
Ω={(H,H),(H,T),(T,H),(T,T)}
(ii) Find all simple event probabilities.
To calculate P((H,H)) we assume that the events
H1={1st toss is a head}={(H,H),(H,T)}
and
H2={2nd toss is a head}={(H,H),(T,H)}
are independent and that P(H1)=P(H2)= 12 . Then
P({(H,H)})=P({(H,H),(H,T)}∩{(H,H),(T,H)})
=P(H1∩H2)
=P(H1)P(H2)
= 12 12 = 14 .
Analogously, P({(H,T)})=P({(T,H)})=P({(T,T)})= 14 .
(iii) Find P({(H,H),(H,T)}{(H,H),(T,T)}).
P({(H,H),(H,T)}{(H,H),(T,T)})
=P({(H,H),(H,T)}∩{(H,H),(T,T)})/P({(H,H),(T,T)})
=P({(H,H)})/P({(H,H),(T,T)})
= 14 / 24 = 12
1.12
• Exercise 1.i: Discuss the four properties
and
(i)
(ii)
(iii)
(iv)
P(AB)=0 if A∩B=∅,
P(AB)=1 if A⊇B,
P(AB)=P(A∩BB),
P(AB):P(B-AB)=P(A):P(B-A) if A⊆B
using the die-tossing experiment and A={4,6}, B={1,3,5}
for (i), A={1,2,3,4}, B={1,2} for (ii), A={2,3,4,5},
B={2,4,6} for (iii), and A={6}, B={2,4,6} for (iv).
1.13