H OMEWORK 1.2
I) If events A and B are independent, then events Ā and B are independent, and events
Ā and B̄ are also independent .
Proof. By definition,
A and B independent ⇔ P(A ∩ B) = P (A)P (B).
But B = (A ∩ B) + Ā ∪ B , so P (B) = P (A ∩ B) + P Ā ∪ B , which yields
P (Ā ∩ B) = P (B) − P (A ∩ B) = P (B) − P (A)P (B) = P (B) [1 − P (A)] = P (B)P (Ā).
Repeat the argument for the events Ā and B̄, this time starting from the statement that Ā
and B are independent and taking the complement of B.
II) If A ⊂ B, P (A) = 1/4, and P (B) = 1/3, find P (A |B ) and P (B |A ).
Proof. (solution) If A ⊂ B, then A ∩ B = A. Then, using the definition, P (A |B ) =
P (A ∩ B)
P (A)
3
P (A)
=
= . Then also P (B |A ) =
= 1.
P (B)
P (B)
4
P (A)
III) Three events Aj , j = 1, 2, 3 are called independent if
(1) the are independent in pairs
P (Aj ∩ Al ) = P (Aj )P (Al ) for any j 6= l
(2) and
P (A1 ∩ A2 ∩ A3 ) = P (A1 )P (A2 )P (A3 ).
(Caution: 3 events might be independent in pairs but not independent),
Show that
1: Any one event is independent of the intersection of the other two.
Proof. If events Aj , j = 1, 2, 3 are independent, by condition 2) in the definition above
P (A1 ∩ A2 ∩ A3 ) = P (A1 )P (A2 )P (A3 ).
Also, by condition 1) in the definition above, for example P (A1 )P (A2 ) = P (A1 ∩ A2 ) so
the right-hand side can be written as
P (A1 ∩ A2 ∩ A3 ) = P (A1 ∩ A2 )P (A3 ).
Denote B = (A1 ∩ A2 ). Then
P [(A1 ∩ A2 ) ∩ A3 ]) = P (B ∩ A3 ) = P (B)P (A3 ),
so B = (A1 ∩ A2 ) and A3 are independent.
2: Replacing one or more of these events with complements the resulting three events
are still independent.
1
Proof. Replace for example A3 7→ Ā3 . From Exercise 1, we know that the complement of
any of the events is independent of the rest of the two events. This takes care of Condition
1 in the 3-event mutual independence. We have to show that Condition 2 is also satisfied,
that is ,
P (A1 ∩ A2 ∩ Ā3 ) = P (A1 )P (A2 )P (Ā3 ).
Denote B = A1 ∩ A2 and partition it using A3 ,
B = (B ∩ A3 ) + (B ∩ Ā3 ),
Using the additivity of probability, thsi means that
P (B) = P (B ∩ A3 ) + P (B ∩ Ā3 ),
that is
P (B ∩ Ā3 ) = P (B) − P (B ∩ A3 ),
Replacing B = A1 ∩ A2 and with Aj , j = 1, 2, 3 are independent, yields
P (A1 ∩ A2 ∩ Ā3 ) = P (A1 ∩ A2 ) − P (A1 ∩ A2 ∩ A3 ) = P (A1 ∩ A2 ) − P (A1 ∩ A2 )P (A3 ),
or
P (A1 ∩ A2 ∩ Ā3 ) = P (A1 ∩ A2 ) − P (A1 ∩ A2 )P (A3 ) = P (A1 ∩ A2 ) [1 − P (A3 )] .
Hence
P (A1 ∩ A2 ∩ Ā3 ) = P (A1 ∩ A2 ) P (Ā3 ) = P (A1 )P (A2 )P (Ā3 ).
IV) (Total Probability) Show that if the events Aj , j = 1, 2, . . . , N are a partition of the
probability space S, i.e.
(1) they are mutually exclusive, Aj ∩ Al = ∅ for any j 6= l
S
(2) their union is the entire space, N
j=1 Aj = S,
then the probability of an event B may be written as
P (B) = P (B |A1 )P (A1 ) + P (B |A2 )P (A2 ) + . . . + P (B |AN )P (AN ).
2
Proof. We have seen that if A ⊂ S, the sets A and Ā form a partition of S, S = A + Ā, and
in addition, any set B can be partitioned into
B = (B ∩ A1 ) + (B ∩ Ā).
This can be generalized for any partition of S: if Aj , j = 1, 2, . . . , N are a partition of S,
B = (B ∩ A1 ) + (B ∩ A2 ) + . . . + (B ∩ AN ).
This is intuitively true: the sets Aj , j = 1, 2, . . . , N cover the entire space and B will be
expressed as a union of its intersections withAj , j = 1, 2, . . . , N . Since Aj , j = 1, 2, . . . , N
are disjoint, the intersections with B are als disjointy, hence the result.
Applying the additivity property of probability yields
P (B) = P (B ∩ A1 ) + P (B ∩ A2 ) + . . . + P (B ∩ AN ).
From the definition of conditional probability,
P (B |A1 ) =
P (B ∩ A1 )
⇒ P (B ∩ A1 ) = P (B |A1 ) P (A1 ) ,
P (A1 )
hence the result.
(A bit more convincing – not necessary for the homework), to show that if Aj , j =
1, 2, . . . , N are a partition of S,
B = (B ∩ A1 ) + (B ∩ A2 ) + . . . + (B ∩ AN ),
we need to show that
1)B ⊂ (B ∩ A1 ) + (B ∩ A2 ) + . . . + (B ∩ AN ), and
2) (B ∩ A1 ) + (B ∩ A2 ) + . . . + (B ∩ AN ) ⊂ B.
Obviously, B ∩ Aj , j = 1, 2, . . . , N are mutually exclusive (disjoint).
1. Assume that ζ ∈ B ⊂ S (ζ is an element of B, B is a subset of S). Then ζ ∈ S, but
sinceAj , j = 1, 2, . . . , N are a partition of S, ζ must be in one of them, i.e. there is a j such
that that ζ ∈ Aj . Hence there is a j such that ζ ∈ B ∩ Aj .
2. Reciprocally, assume that ζ ∈ (B ∩ A1 ) + (B ∩ A2 ) + . . . + (B ∩ AN ). Since B ∩ Aj ,
j = 1, 2, . . . , N are mutually exclusive (disjoint) there is a j such that ζ ∈ B ∩ Aj . This
means that ζ ∈ B.
V) Show that a set S with n elements has 2n subsets and
n!
n(n − 1) . . . (n − k + 1)
=
1 · 2 · ... · k
k!(n − k)!
subsets with k elements.
3
Proof. 1. The easiest way to show this is by induction.
Start with a set with n = 1 elements, S1 = {ζ1 }. The subsets are listed below and their
number is indeed 21 .
{∅} {ζ1 }.
Now let S2 = {ζ1 , ζ2 } have n = 2 elements. Its subsets are indeed 4, a shown below:
{∅} {ζ1 } {ζ2 } {ζ1 , ζ2 }.
Notice that the subsets are formed from the subsets of S = {ζ1 }, by “adding” the new
element. The subsets of S1 = {ζ1 }In the first row are , the second row are
{∅}
{ζ1 }
{ζ2 } = {∅, ζ2 } {ζ1 , ζ2 }
The number of subsets doubles. In general, Sn+1 = Sn ∪ {ζn+1 } , so the subsets of Sn+1
contain all the subsets of Sn plus all the “new” subsets, which contain also the element
ζn+1 , formed by the union of any of the subsets of Sn with the set {ζn+1 }. Hence the number
of subsets of Sn+1 is 2n + 2n = 2 · 2n = 2n+1 .
4