ECE 450 – Lecture 2
• Recall: Pr(A  B) = Pr(A) + Pr(B) – Pr(A  B) in general
= Pr(A) + Pr(B)
if A and B are m.e.
Lecture Overview
– Conditional Probability, Pr(A | B)
– Total Probability
– Bayes’ Theorem
– Independent Events
– Compound Experiments
– Binomial Distribution
ECE 450
D. van Alphen
1
Conditional Probability
• Defn: The conditional probability of event A, given that
event B has occurred is
Pr( A | B) 
Pr( A  B)
,
Pr(B)
P(B)  0
(1)
• Note (by symmetry of the definition):
Pr( A  B)
Pr(B | A ) 
Pr( A )
P( A )  0
(2)
• From (1) & (2), we have two new ways of writing Pr(A  B):
Pr(A  B) = Pr(A) Pr(B | A) = Pr(B) Pr(A | B)
ECE 450
D. van Alphen
2
Verification Example
(Is the definition reasonable?)
• Experiment: Toss a single die, and find Pr(2 | even)
Pr(2 | even) 
Pr(2  even)
Pr(2)
16


 1/ 3
Pr(even)
Pr(even) 3 6
• Another way to look at it - Let B be the event of getting an
even number:
B is called the restricted
S
B’
B
sample space; 2 is now
1
4
5
2
one of 3 equally likely
outcomes.
3
6
ECE 450
D. van Alphen
3
Another Example
• Experiment: Toss 2 coins
• Sample Space: S = {HH, HT, TH, TT}
• Find the conditional probability of obtaining two heads when
flipping a coin twice, given that at least one head was
obtained; i.e., find Pr(2 heads | at least 1 head)
• Def: A: event of obtaining 2 heads = {HH}
B: event of obtaining at least one head = {HH, HT, TH}
• Then
Pr( A  B) Pr( A ) 1 4
Pr( A | B) 


 1/ 3
Pr(B)
Pr(B) 3 4
ECE 450
D. van Alphen
4
Side Note & Definition
• Note: Conditional probabilities are themselves probabilities;
thus, they satisfy all the axioms for probabilities:
1. Pr(A | B)  0
2. Pr(B | B) = 1
3. Pr(A  C | B) = Pr(A|B) + Pr(C|B) if A and C are m.e.
• Another definition: consider a collection of subsets, {Ai},
(i = 1, …, n ), of S. The collection is
S A A
2
1
a partition of S if:
...
Ai  Aj = f, i  j
An
U Ai = S,
i = 1, …, n
ECE 450
D. van Alphen
5
Total Probability Theorem
• Let {Ai} be a partition of S, and let B be a subset of S:
S
A1 A2 A3
B
...
An
Strategy: To find Pr(B),
break apart B, into
mutually exclusive pieces
• Then Pr(B) = Pr[ (A1  B)  (A2  B)  . . . (An  B) ]
m.e
= Pr(A1  B) + Pr(A2  B) + … + Pr(An  B)
(see box,
bottom of p.2)
 Pr(B) = Pr(B|A1)Pr(A1) + Pr(B|A2)Pr(A2) + … + Pr(B|An)Pr(An)
ECE 450
D. van Alphen
6
Example Using “Total Probability”
Transistor types X , Y and Z make up 45%, 25%, and 30% of
the total number of transistors in a box, respectively. Let B be
the event that a transistor fails before 1000 hrs.
Given the reliability information:
Pr(B|X) = .15
Pr(B|Y) = .4
Pr(B|Z) = .25
Find the probability that a randomly chosen transistor from the
box fails before 1000 hours.
Pr(B) = ______ _____ + ________ ____+ _______ _____
= _____ ______ + ______ ____+
______ ______
= .243
ECE 450
D. van Alphen
7
Bayes’ Rule
• Recall from p. 2 (again, the box near the bottom)
Pr(A  B) = Pr(A|B) Pr(B) = Pr(B|A) Pr(A)
Focus here; solve for Pr(A|B)

Pr(B | A ) Pr( A )
Pr( A | B) 
,
Pr(B)
Pr(B)  0
(Bayes’ Rule)
Pr(B | A j ) Pr( A j )
 Pr( A j | B)  n
 Pr(B | A j ) Pr( A j )
i1
ECE 450
D. van Alphen
By __________
_____________
8
Bayes’ Rule: Note & Example
• Note: Use Bayes’ Rule when asked to find some Pr(A|B),
but it would be easier to find Pr(B|A). (“Backwards
conditional probability”)
Example: Say an observed transistor (from the previous
example) fails before 1000 hrs. Find the probability that it
was a Type Z transistor.
Let B: event that transistor fails before 1000 hrs.
We want: Pr(Z|B), non-trivial
We know Pr(B|Z) = .25 (the easier problem; given on p. 7)
Using Bayes’ Rule, next page:
ECE 450
D. van Alphen
9
Bayes’ Rule Example, continued
Pr(B | Z) Pr( Z)
Pr( Z | B) 

Pr(B)
 .30864
Answer from p. 7 example
In-Class Practice
• Two cards are drawn without replacement from a 52-card
deck.
• Find the probability that the 2nd is a queen, given that the 1st
is a queen.
• Find the probability that both the 1st and 2nd are queens.
• Unordered answers: 1/221, 3/51
ECE 450
D. van Alphen
10
Bayes’ Rule In-Class Example
A medical test for a particular type of cancer has the following
properties:
• It correctly detects the cancer (when present) with probability
95%;
• It incorrectly “detects” the cancer (when there is no cancer
present) with probability 20%.
Suppose that this particular type of cancer is present in only
1% of people of your age/sex/ethnicity.
Find the probability that you actually have this cancer, given
that your test is positive (i.e., cancer was detected).
Let c: event that you have cancer.
Let d: event that cancer is detected.
ECE 450
D. van Alphen
11
Pr(d|c) = .95; Pr(___|___) = .05 (complements)
Pr(d|c’) = .2; Pr(d’|c’) = _______;
Pr(c) = .01 (a priori)
Find Pr(c|d)
_______________________________________________
_______________________________________________
_______________________________________________
_______________________________________________
_______________________________________________
_______________________________________________
_______________________________________________
_______________________________________________
_______________________________________________
ECE 450
D. van Alphen
12
(Statistically) Independent Events
• Defn: 2 events A and B are statistically independent (__)
if and only if
Pr(A|B) = Pr(A)
(knowing B has occurred tells me nothing about whether or
not A has occurred)
• Equivalently:
Pr(B|A) = P(B)
and
Pr(A  B) = Pr(A) Pr(B)
(caution: only true for independent events)
ECE 450
D. van Alphen
13
Example Using Independence Definition
• Experiment: Toss 2 dice. Define the events
A: sum = 7
Question: Are A, B, 
B: 1 face = 6
• Pr(A | B) 
Pr
Pr(B)
Pr{(6,1)  (1, 6)}
2 / 36 2



Pr{(6,1)  (6, 2)  (6, 6)} 11/ 36 11
  A, B not _____
• Pr(A) = 6/36 = 1/6
 A, B dependent
ECE 450
D. van Alphen
14
Facts/Thoughts About Independence
• Fact 1: Complementary events are dependent events.
• Fact 2: Unions, intersections & complements of independent
events are independent.
• Consider the 2012 election (Obama/Biden vs.
Romney/Ryan):
Is there any pair of
– Event A: Obama wins as Pres.
events from this set
– Event B: Romney wins as Pres. that is an independent
– Event C: Biden wins as VP.
pair?
– Event D: Ryan wins as VP.
• Let A: it rains today; B: it rains tomorrow. Are A and B
independent ?
ECE 450
D. van Alphen
15
Example Using Independence
• Say the switches in the circuit below are closed ( -ly of
each other) at any time with probability 0.1. Find the
probability of a closed path from point A to point B.
B
A
• Labeling for the Solution:
top
right
B
A
bottom
ECE 450
D. van Alphen
Note: for a closed path to
exist, the “right” switch
must be closed; and,
either the “bottom” switch
must be closed or both of
the “top” switches must
be closed.
16
Example, continued
Pr(closed path) = Pr[‘right’ and (‘top’ or ‘bottom’)]
(both switches)
= Pr(right)  Pr(top or bottom), by Fact 2, p. 14
= (0.1)  [Pr(top) + Pr(bottom) – Pr(both top & bottom)]
by Corollary 4
(both switches: (.1)(.1) )
= .1 [(.01) + .1 – (.01)(.1)] = .l0109
top
right
Note: Don’t round off!
B
A
ECE 450
bottom
D. van Alphen
17
Compound Experiments
• Consider experiments: Ea, Eb
with sample spaces:
Sa, Sb
• Say Sa = {a1, …, an} and Sb = {b1, …, bm}.
• Define Sa x Sb (the Cartesian Cross Product) as the set of
ordered pairs with the 1st element from Sa and the second
element from Sb
• Do experiments Ea & Eb (jointly), and get pairs of outcomes
from Sa x Sb
• The joint performance of Ea & Eb is said to be a compound
experiment.
ECE 450
D. van Alphen
18
Cross Product Examples & Bernoulli Trials
Example 1: Toss a coin 2 times, with S1 = S2 = {H, T}.
S = S1 x S2 = { ____, ____, _____, _____}
Example 2: Toss a coin 3 times, with S1 = S2 = S3 = {H, T}.
S = S1 x S2 x S3 = { (HHH), (HHT), (HTH), (HTT), (THH),
(THT), (TTH), (TTT) }
Definition: A Bernoulli Trial is an experiment with only 2
possible outcomes, sometimes called “success” and
“failure”.
Success  1  yes,
ECE 450
Failure  0  no
D. van Alphen
19
Binomial Experiments
• Definition: A Binomial Experiment is an experiment
consisting of n independent Bernoulli Trials.
– Let Ak denote the event of getting k successes (and thus
n-k failures) in n trials.
– Let p be the probability of success on each trial.
– Let q = 1 – p be the probability of failure on each trial.
• Example: Consider a 5-trial binomial experiment, and say
we are interested in the probability of having 2 successes
(first), followed by 3 failures.
– Pr{1, 1, 0, 0, 0} = __
__
__
__
__ = ______
(O.K. to multiply since the events are ____)
ECE 450
D. van Alphen
20
Binomial Experiments, continued
• Refined example: Say we want to know the probability of
getting 2 successes, in 5 independent trials (order doesn’t
matter):
5 2 3
Pr{A2} =   p q
 2
(since there are 5C2 ways to decide where to
put the 2 successes in the string of 5 outcomes)
• In general, the probability of getting k successes in n
(independent) Bernoulli tials is
n k n – k
pn(k) = Pr{Ak} =   p q
k
ECE 450
D. van Alphen
(Binomial Experiments)
21
Binomial Experiment Examples
1. 10 missiles are fired at a tank; each missile has a (0.2)
probability of hitting the tank, independently of the other
missiles. Find the probability that exactly 3 missiles hit the
tank.



p10(3) = Pr{ A3} =   (__)3 (__)7  .2013
2. A certain football player can catch 2/3 of the passes thrown
to him. He needs to catch at least 3 more passes for his
team to win the game. Find the probability that his team
wins if the quarterback throws to him 5 more times.
Pr{win} = Pr{at least 3 catches}
= Pr{exactly 3 catches or exactly 4 catches or
exactly 5 catches}
ECE 450
D. van Alphen
22
Binomial Experiment Examples, continued
= Pr{A3  A4  A5}
= Pr{A3} + Pr{A4} + Pr{A5}
= ___ ___ ___ + ___ ___ ___ + ___ ___ ___
= .790123
** Note: we can add these probabilities, because the events
“catch exactly 1 pass”, “catch exactly 2 passes”, and
“catch exactly 3 passes” are:
_____________________________ events.
ECE 450
D. van Alphen
23
Binomial Experiment Examples
3. Using a normal deck of cards, say we cut the deck 5 times;
find the probability of getting an ace on at least 3 cuts.
Pr{at least 3 aces} = Pr{exactly 3 aces or exactly
4 aces or exactly 5 aces)
(Bernoulli Trials)
m.e.
= Pr{A3} + Pr{A4} + Pr{A5}
 5  1 3  12  2  5  1  4  12 1  5  1 5  12 0
=                 
 3  13   13   4  13   13   5  13   13 
1501
=
ECE 450
135
 .00404
D. van Alphen
24
Danger!!
1. Pr(A  B) = Pr(A) + Pr(B) – Pr(A  B)
= Pr(A) + Pr(B)
if A, B m.e.
2. Pr(A  B) = Pr(A)  Pr(B | A)
= Pr(A)  Pr(B)
ECE 450
if A, B independent
D. van Alphen
25
Review
• Pr(A|B) = ___________________ (defn)
= ___________________ (Bayes’ Rule)
• Total Probability: If {Ai} is a partition of sample space S, then
Pr(B) = ___________________________________________
• If A and B are independent, then Pr(A|B) = ______________
and Pr(A  B) = ____________
• (Binomial Experiments): The probability of getting k successes in n
independent Bernoulli trials is:
_____________________________________
ECE 450
D. van Alphen
26