Independence
Independence
Example: A fair die is tossed and we want to guess the outcome.
The outcomes will be 1, 2, 3, 4, 5, 6 with equal probability 61 each.
If we are interested in getting the following results: A = {1, 3, 5},
B = {1, 2, 3} and C = {3, 4, 5, 6}, then we can calculate the
probability for each event:
P(A) = P(B) =
3
1
4
2
= , and P(C ) = = .
6
2
6
3
Independence
Example: A fair die is tossed and we want to guess the outcome.
The outcomes will be 1, 2, 3, 4, 5, 6 with equal probability 61 each.
If we are interested in getting the following results: A = {1, 3, 5},
B = {1, 2, 3} and C = {3, 4, 5, 6}, then we can calculate the
probability for each event:
P(A) = P(B) =
3
1
4
2
= , and P(C ) = = .
6
2
6
3
If someone tell you that after one toss, event C happened, i.e. the
outcome is one of {3, 4, 5, 6}, then what is the probability for
event A to happen and what for B?
P(A ∩ C )
P(A | C ) =
=
P(C )
1
3
2
3
1
P(B ∩ C )
= ; P(B | C ) =
=
2
P(C )
1
6
2
3
1
= .
4
Independence
Example: A fair die is tossed and we want to guess the outcome.
The outcomes will be 1, 2, 3, 4, 5, 6 with equal probability 61 each.
If we are interested in getting the following results: A = {1, 3, 5},
B = {1, 2, 3} and C = {3, 4, 5, 6}, then we can calculate the
probability for each event:
P(A) = P(B) =
3
1
4
2
= , and P(C ) = = .
6
2
6
3
If someone tell you that after one toss, event C happened, i.e. the
outcome is one of {3, 4, 5, 6}, then what is the probability for
event A to happen and what for B?
P(A ∩ C )
P(A | C ) =
=
P(C )
1
3
2
3
1
P(B ∩ C )
= ; P(B | C ) =
=
2
P(C )
P(A | C ) = P(A) while P(B | C ) 6= P(B)
1
6
2
3
1
= .
4
Independence
Independence
Definition
Two events A and B are independent if P(A | B) = P(A), and
are dependent otherwise.
Independence
Definition
Two events A and B are independent if P(A | B) = P(A), and
are dependent otherwise.
Remark:
Independence
Definition
Two events A and B are independent if P(A | B) = P(A), and
are dependent otherwise.
Remark:
1. P(A | B) = P(A) ⇒ P(B | A) = P(B). This is natural since the
definition for independent should be symmetric.
Independence
Definition
Two events A and B are independent if P(A | B) = P(A), and
are dependent otherwise.
Remark:
1. P(A | B) = P(A) ⇒ P(B | A) = P(B). This is natural since the
definition for independent should be symmetric.
P(B | A) =
P(A ∩ B)
P(A | B) · P(B)
=
P(A)
P(A)
Independence
Independence
Remark:
Independence
Remark:
2. If events A and B are mutually disjoint, then they can not be
independent. Intuitively, if we know event A happens, we then
know that B does not happen, since A ∩ B = ∅. Mathmatically,
P(A | B) =
P(A ∩ B)
P(∅)
=
= 0 6= P(A),
P(B)
P(B)
unless P(A) = 0 which is trivial.
Independence
Remark:
2. If events A and B are mutually disjoint, then they can not be
independent. Intuitively, if we know event A happens, we then
know that B does not happen, since A ∩ B = ∅. Mathmatically,
P(A | B) =
P(A ∩ B)
P(∅)
=
= 0 6= P(A),
P(B)
P(B)
unless P(A) = 0 which is trivial.
e.g. for the die tossing example, if A = {1, 3, 5} and B = {2, 4, 6},
then P(A ∩ B) = P(∅) = 0, therefore P(A | B) = 0. However,
P(A) = 0.5.
Independence
Independence
The Multiplication Rule for Independent Events
Independence
The Multiplication Rule for Independent Events
The general multiplication rule tells us
P(A ∩ B) = P(A | B) · P(B).
Independence
The Multiplication Rule for Independent Events
The general multiplication rule tells us
P(A ∩ B) = P(A | B) · P(B).
However, if A and B are independent, then the above equation
would be P(A ∩ B) = P(A) · P(B) since P(A | B) = P(A).
Independence
The Multiplication Rule for Independent Events
The general multiplication rule tells us
P(A ∩ B) = P(A | B) · P(B).
However, if A and B are independent, then the above equation
would be P(A ∩ B) = P(A) · P(B) since P(A | B) = P(A).
Furthermore, we have the following
Proposition
Events A and B are independent if and only if
P(A ∩ B) = P(A) · P(B)
Independence
The Multiplication Rule for Independent Events
The general multiplication rule tells us
P(A ∩ B) = P(A | B) · P(B).
However, if A and B are independent, then the above equation
would be P(A ∩ B) = P(A) · P(B) since P(A | B) = P(A).
Furthermore, we have the following
Proposition
Events A and B are independent if and only if
P(A ∩ B) = P(A) · P(B)
In words, events A and B are independent iff (if and only if) the
probability that the both occur (A ∩ B) is the product of the two
individual probabilities.
Independence
Independence
In real life, we often use this multiplication rule without noticing it.
Independence
In real life, we often use this multiplication rule without noticing it.
The probability for getting {HH} when you toss a fair coin twice is
1
1 1
4 , which is obtained by 2 · 2 ;
Independence
In real life, we often use this multiplication rule without noticing it.
The probability for getting {HH} when you toss a fair coin twice is
1
1 1
4 , which is obtained by 2 · 2 ;
The probability for getting {6,5,4,3,2,1} when you toss a fair die
six times is ( 61 )6 , which is simply obtained by 16 · 61 · 16 · 16 · 16 · 16 ;
Independence
In real life, we often use this multiplication rule without noticing it.
The probability for getting {HH} when you toss a fair coin twice is
1
1 1
4 , which is obtained by 2 · 2 ;
The probability for getting {6,5,4,3,2,1} when you toss a fair die
six times is ( 61 )6 , which is simply obtained by 16 · 61 · 16 · 16 · 16 · 16 ;
The probability for getting {♠♠♠} when you draw three cards
1
, which is
from a deck of well-shuffled cards with replacement is 64
1 1 1
simply obtained by 4 · 4 · 4 .
Independence
In real life, we often use this multiplication rule without noticing it.
The probability for getting {HH} when you toss a fair coin twice is
1
1 1
4 , which is obtained by 2 · 2 ;
The probability for getting {6,5,4,3,2,1} when you toss a fair die
six times is ( 61 )6 , which is simply obtained by 16 · 61 · 16 · 16 · 16 · 16 ;
The probability for getting {♠♠♠} when you draw three cards
1
, which is
from a deck of well-shuffled cards with replacement is 64
1 1 1
simply obtained by 4 · 4 · 4 .
However, if you draw the cards without replacement, then the
multiplication rule for independent events fails since the event {the
first card is ♠} is no longer independent of the event {the second
card is ♠}. In fact,
P({the second card is ♠ | the first card is ♠}) =
12
.
51
Independence
Independence
Example: Exercise 89
Suppose identical tags are placed on both the left ear and the right
ear of a fox. The fox is then let loose for a period of time.
Consider the two events C1 ={left ear tag is lost} and C2 = {right
ear tag is lost}. Let π = P(C1 ) = P(C2 ), and assume C1 and C2
are independent events. Derive an expression (involving π) for the
probability that exactly one tag is lost given that at most one is
lost.
Independence
Example: Exercise 89
Suppose identical tags are placed on both the left ear and the right
ear of a fox. The fox is then let loose for a period of time.
Consider the two events C1 ={left ear tag is lost} and C2 = {right
ear tag is lost}. Let π = P(C1 ) = P(C2 ), and assume C1 and C2
are independent events. Derive an expression (involving π) for the
probability that exactly one tag is lost given that at most one is
lost.
Independence
Independence
Remark:
Independence
Remark:
1. If events A and B are independent, then so are events A0 and B,
events A and B 0 as well as events A0 and B 0 .
P(B) − P(A ∩ B)
P(A ∩ B)
P(A0 ∩ B)
=
=1−
P(B)
P(B)
P(B)
0
= 1 − P(A | B) = 1 − P(A) = P(A )
P(A0 | B) =
Independence
Remark:
1. If events A and B are independent, then so are events A0 and B,
events A and B 0 as well as events A0 and B 0 .
P(B) − P(A ∩ B)
P(A ∩ B)
P(A0 ∩ B)
=
=1−
P(B)
P(B)
P(B)
0
= 1 − P(A | B) = 1 − P(A) = P(A )
P(A0 | B) =
2. We can use the condition P(A ∩ B) = P(A) · P(B) to define the
independence of the two events A and B.
Independence
Independence
Independence of More Than Two Events
Definition
Events A1 , A2 , . . . , An are mutually independent if for every k
(k = 2, 3, . . . , n) and every subset of indices i1 , i2 , . . . , ik ,
P(Ai1 ∩ Ai2 ∩ · · · ∩ Aik ) = P(Aii ) · P(Ai2 ) · ··· · P(Aik ).
Independence
Independence of More Than Two Events
Definition
Events A1 , A2 , . . . , An are mutually independent if for every k
(k = 2, 3, . . . , n) and every subset of indices i1 , i2 , . . . , ik ,
P(Ai1 ∩ Ai2 ∩ · · · ∩ Aik ) = P(Aii ) · P(Ai2 ) · ··· · P(Aik ).
In words, n events are mutually independent if the probability of
the intersection of any subset of the n events is equal to the
product of the individual probabilities.
Independence
Independence
An very interesting example: Exercise 113
A box contains the following four slips of paper, each having
exactly the same dimensions: (1) win prize 1; (2) win prize 2; (3)
win prize 3; and (4) win prize 1, 2 and 3. One slip will be randomly
selected. Let A1 = {win prize 1}, A2 = {win prize 2}, and A3 =
{win prize 3}. Are these three events mutually independent?
Independence
Independence
Example:
Consider a system of seven identical components connected as
following. For the system to work properly, the current must be
able to go through the system from the left end to the right end. If
components work independently of one another and P(component
works)=0.9, then what is the probability for the system to work?
Independence
Example:
Consider a system of seven identical components connected as
following. For the system to work properly, the current must be
able to go through the system from the left end to the right end. If
components work independently of one another and P(component
works)=0.9, then what is the probability for the system to work?
Let A = {the system works} and Ai = {component i works}. Then
A = (A1 ∪ A2 ) ∩ ((A3 ∩ A4 ) ∪ (A5 ∩ A6 )) ∩ A7 .