Probability
An Experiment is any action that leads to an uncertain outcome.
The sample space , denoted S is the set of all possible outcomes.
Events are sets of outcomes or subsets of S and are denoted by capital letters such as A,
B, …
Simple events are events consisting of exactly 1 outcome.
Example1:
Flip a coin and pick a number (1, 2, or 3)
Draw Tree diagram to get S = {(H1, H2, H3, T1, T2, T3)}
Note that order of H1 ≠ 1H but order in S does not matter so
S = {( T1, T2, T3, H1, H2, H3)}
The probability of an event A is denoted P(A). P(A) is the measure of the likelihood that
A will occur when the experiment is run.
You can think of P(A) in three ways:
1. If the experiment was run many many times, what fraction of the time would A occur.
(Theoretical)
2. If you ran the experiment many times, the relative frequency of the times A occurred.
(Empirical)
3. If all the outcomes are equally likely to occur, then
P(A) =
number of ways that A occurs
Total number outcomes of the experiment
P(A)
=
number of outcomes in A
number of outcomes in S
Some important rules of probability
0 ≤ P(A) ≤ 1
P(S) = 1
{} = empty set, which is the set that contains no elements
P({}) = 0
0 ≠ {}
P(A) is the sum of the probabilities of the outcomes in A. You can just the sum the
outcomes to get the probability of A.
A' = compliment of A which means “not A” or the event that A did not occur.
P(A') = 1 – P(A)
P(A ∩ B) means that both event A and event B occur!
P(A U B) means that either A occurs or B occurs or they both occur.
Another helpful diagram to help calculate probabilities is a Venn Diagram.
Example 2:
Roll a fair die. Let E = {2, 4, 6} and F = {1, 2, 3, 4}
Draw Venn Diagram
S = {1,2,3,4,5,6}
P(E ) = 3/6 = ½ = 0.5
P(F) = 4/6 = 2/3 = 0.667
P(E') = 1 – ½ = ½ = 0.5
P(F') = 1 – 2/3 = 1/3 = 0.333
(E ∩ F ) = {2, 4} so P(E ∩ F) = 2/6 = 1/3 = 0.333
(E U F ) = {1, 2, 3, 4, 6} so P(E U F) = 5/6 = .833
Note that in the above example:
P(E U F) = P(E) + P(F) – P(E ∩ F) = ½ + 2/3 – 1/3 = 3/6 + 4/6 – 2/6 = 5/6 = .833
This is true in general for any events A and B:
P(A U B) = P(A) + P(B) – P(A ∩ B)
P( A∩B ) = 0 means A and B are Disjoint or Mutually Exclusive
Example #3
A couple plans to have two children. Each child is equally likely to be a boy or a girl, with
gender independent of each child.
a. Construct the sample space for the 2 children.
S = {GG, GB, BG, BB}
Note that depending on what we are interested in GB may not be the same as BG
b. Find the probability that both children are Girls.
Let E = {GG}
P(E) = ¼ = 0.25
c. What is the probability that the family has at least one boy?
The event {at least one boy} is the compliment of two girls {GG}, so
P(E') = 1 – ¼ = ¾ = 0.75, which is easier than calculating
P(At least one boy) = P(BB) + P(BG) + P(GB) = ¼ + ¼ + ¼ = ¾ = 0.75
Note to self: Draw tree diagram
e. If the probability of having a girl is actually 0.49 and not 0.5 how does this change the
P(GG) and the P(at least one boy)?
(Draw Tree diagram)
Let A = {GG}
P(A) = .49 * .49 = 0.2401 (slightly less than 0.25)
P(A’) = 1 - .2401 = .7599 (slightly more than .75)
Example #4
A single random digit is selected.
a. What is the sample space?
b. What is the probability that each digit is selected?
c. Justify that this is a valid probability model.
a. S = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9 }
b. The probability of each digit is 1/10 = 0.1
c. since 0 < 0.1 < 1, all the probabilities are between 0 and 1 and since there are 10 digits
the sum of the probabilities is 10 * 0.1 = 1.
Counting: essential to probability
Ex
How many ways can I order the letters A B C?
ABC, ACB, BAC, BCA, CAB, CBA: which is 6.
You can think of this as you have 3 choices for the first letter and 2 choices for the
second letter and 1 choice for the 3 letter. 3*2*1 =6
When you say “and” you multiply.
This is known as the multiplicative rule of counting.
If the first part of an experiment can be done in k ways and the second part can be done
in m ways, then the experiment can be done in k * m ways.
This can be generalized to an experiment with any number of parts.
Menu Example:
You own a restaurant and want to offer combination dinners. The patrons get a choice
of 1 of 3 appetizers, 1 of 5 entrees and 1 of 2 desserts. How many combination meals
are there?
3 * 5 * 2 = 30.
How many ways can I order the letters A B C D?
24 = 4 * 3 * 2 * 1
How many ways can I order n distinct letters (objects)?
n! = n factorial.
0! = 1
n! = n * (n – 1)! for any positive integer n
n! = n * (n-1) * (n-2) * …(2) * (1)
1! = 1
2! = 2
3! = 6
4! = 24
to find 5! On TI83/84, hit [5], [MATH], go over to PRB and the fourth option is 4:!(hit
[4]).
On screen you should see 5!
Hit [ENTER]
Answer is 120, which is 5 * 4! = 5 * 24
n! gets big really fast.
n! = the number of ways to order n distinct items.
Ex.
How many ways can I order A B C D?
n = 4, 4! = 24
A more complicated question:
Ex How many ways can I select and order 2 letters from
{A B C D E}?
AB AC AD AE BC BD BE CD CE DE
BA CA DA EA CB DB EB DC EC ED
20 ways = 5 * 4
Also this can be done using Permutations. When you have n distinct objects and you
select r of them and order is important, meaning AB ≠ BA, then you have n P r or P(n, r).
n Pr 
n!
(n  r )!
The above example: n = 5 and r = 2.
5 P 2 = 5!/3! = 20
To calculate 5 P 2 on the TI83/84:
hit [5], [MATH], go over to PRB and the
second option is nPr, hit [2]
On your screen you should see 5 nPr then hit [2]
and [ENTER]
You should get 20.
Ex. How many ways can I select 2 letters from {A B C D E}?
(here it assumed that order is not important)
AB AC AD AE BC BD BE CD CE DE = 10 ways.
This can also be looked at as a combination. When you have n distinct objects and you
select r of them and order is NOT important, meaning AB = BA, then you have n C r or
C(n, r).
Other Notation:
nCr 
n!
r!(n  r )!
In the above example:
n = 5, r = 2 so
n C r = 5 C 2 = 5! / (2! * 3!) = 120 / (2 * 6) = 10
To calculate 5 C 2 on the TI83/84:
hit [5], [MATH], go over to PRB and the
third option is nCr, hit [3]
On your screen you should see 5 nCr then hit [2]
and [ENTER]
You should get 10.
nCn = 1
nC1 = n
nC0 = 1
nC2 = n*(n-1)/2
Example:
You are given an urn that contains 10 marbles. There are 5 Red marbles, 3 White
marbles and 2 Blue marbles. You select 2 marbles without replacement. Find the
following probabilities:
(Hint: a tree diagram may be helpful.)
1. Both marbles are White.
2. Neither marble is White.
3. Both marbles are the same color.
P(R) = 5/10
R
R P(R) = 4/9
W P(W) = 3/9
B P(B) = 2/9
P(W) = 3/10
W
R P(R) = 5/9
W P(W) = 2/9
B P(B) = 2/9
P(B) = 2/10
B
R P(R) = 5/9
W P(W) = 3/9
B P(B) = 1/9
P(WW) = 3/10 * 2/9 = 6/90 = 1/15 = .067
P(NEITHER IS W) = P(RR) + P(RB) + P(BR) + P(BB)
= 5/10 * 4/9 + 5/10 * 2/9 + 2/10 * 5/9 + 2/10 * 1/9 = 42/90 = .467
P(BOTH SAME COLOR) = P(RR) + P(WW) + P(BB) =
= 5/10 * 4/9 + 3/10 * 2/9 + 2/10 * 1/9 = 28/90 = .311
Another way to answer question 1 is by using combinations.
P(WW) = number of ways to pick 2 white marbles
number of ways to pick 2 marbles
P(WW) = 3 C 2/ 10 C 2 = 3 / 45 = 6 /90
P(RR) = 5 C 2 / 10 C 2 = 10 / 45 = 20 / 90
P(BB) = 2 C 2 / 10 C 2 = 1 / 45 = 2 / 90
Example #4
One thousand people were asked to give their primary source of news. The results are
given in the following table. A random subject is selected.
Male
Female
Radio
275
200
TV
135
175
Paper
35
70
Other
60
50
a. Find the probability that the subject gets news from the Radio.
P (Radio) = (275 + 200)/ 1000 = 475 / 1000 = .475
b. Find the probability that the subject gets news from Paper and is Female.
P (Paper ∩ Female) = 70 / 1000 = .070
Remember that AND means both!
c. Find the probability that the subject gets news from TV or is Male
P (TV U Male) = P(TV) + P(Male) – P(TV ∩ Male) =
310/1000 + 505/1000 – 135/1000 = 680/1000 = 0.680
d. Find the probability that the subject gets news from Paper GIVEN she is Female.
This is a new type of probability called Conditional Probability. We are given additional
information here, that the subject is Female. So we are limiting our Sample Space.
P (Paper | Female) = 70 / 495 = .141
In general P (A | B) = P(A ∩ B)
P(B)
Provided that the P(B) ≠0
For the last example P(Paper ∩ Female) = 70/1000 = .07 and the
P(Female) = 495/1000 = .495
so the P(Paper | Female) = .07 / .495 = .141
e. Find the probability that the subject is Female given she gets news from Paper.
P(Female | Paper) = .07 / .105 = .667 = 70 / 105
In general P(A | B ) ≠ P(B | A) whereas
P(A ∩ B) = P(B ∩ A) and P( A U B ) = P( B U A)
P(A | B) is a probability so it is between 0 and 1, if you get a number bigger than 1 it
means you made a mistake! Fix it!
You may give answers as fractions or decimals but not both.
In part e of the last example, acceptable answers are:
P(Female | Paper) = .667 = 70 / 105 = 2/3
Unacceptable answers are
P(Female | Paper) = .07 / .105, this is not a proper fraction or a proper decimal.
Example:
P(A) = .45
P(B) = .35
P(A U B) = .60
Find the P(A | B) and P(B | A).
First we need to find the P(A ∩ B).
Recall that P(A U B) = P(A) + P(B) – P(A ∩ B). This implies that
P(A ∩ B) = P(A) + P(B) – P(A U B)
So
P(A ∩ B) = .45 + .35 - .60 = .20
Then
P(A | B) = P(A ∩ B) / P(B) = .20 / .35 = 4/7 = .571
P(B | A) = P(A ∩ B) / P(A) = .20 / .45 = 4/9 = .444
P(A ∩ B ) = P(B ∩ A) and P(A U B) = P(B U A) but
P(A | B) ≠ P(B | A)
Recall P(A′) = 1 – P(A)
P(A′ | B) = 1 – P(A | B) but P(A′ |B) ≠ 1 – P(A′ | B′)
Using the above example: Draw Venn diagram
P(A′) = .55
P(B′) = .65
P(A′ ∩ B′) = .40
P(A′ ∩ B) = .15 P(B′ ∩ A) = .25
P(A′ | B) = P(A′ ∩ B)/P(B) = .15 / .35 = 3/7 = .429
P(A′ | B′) = P(A′∩ B′) / P(B′) = .40 / .65 = 8/13 = .615
This is because for any sets E and F
P(E) = P(E ∩ F) + P(E ∩ F′) and
P(F) = P(F ∩ E) + P(F ∩ E′)
Draw Venn Diagram!
Another Example:
A standard Bridge or Poker deck of cards has 52 cards. 4 suits: 2 red: Hearts and
Diamond and 2 black: Clubs and Spades. Each suit has 13 cards labeled 2, 3, 4, 5, 6, 7, 8,
9, 10, Jack, Queen, King and Ace. The jack, queen and king are known as the picture or
face cards. You pick a card at random. Let E = {Jack}, F = {face card}, G = {black card}.
Find the following probabilities and describe in words what they mean:
P(E) = 4/52
P(F) = 12/52
P(G) = 26/52
P(E ∩ F) = 4/52
P(E ∩ G)= 2/52
P(F ∩ G)= 6/52
P(E U F)= 12/52
P(E U G)= 28/52
P(F U G)= 32/52
P(E | F) = 4/12
P(F | E)= 1
P(E | G)= 2/26
P(G | E)= 2/4
P(F | G)= 6/26
P(G | F)= 6/12
Multiplicative Rule of Probability
P(A ∩ B) = P(A | B) P(B)
If you knew P(A | B) = .42 and P(B) = .27 then
P(A ∩ B) = .42 * .27 = .1134
Two events (A and B) are independent when the fact that A occurs does not affect the
probability that B will occur and vice versa.
Equivalent definitions of Independent events A and B:
P( A ∩ B ) = P(A) P(B)
P(A | B) = P(A)
P(B | A ) = P(B)
So P(A ∩ B) = P(A) P(B) only if A and B are independent.
Ex. the experiment is to draw a single card from a deck of 52. Let A be the event that the
card is a Jack and let B be the event that the card is Black. Find the probability of A, B
and A ∩ B, A|B, B|A ? Are A and B independent?
P(A) = 4/52 = 1/13
P(B) = 26/52 = ½
P(A ∩ B) = 2/52 = 1/26
P(A | B) = 2/26 = 1/13
P(B | A) = 2/4 = ½
A and B are independent because:
P(A ∩ B) = 1/26 = 1/13 * ½ = P(A) P(B)
P(A | B) = 1/13 = P(A)
P(B |A) = ½ = P(B)
Ex. Roll a fair die. Let A = {1, 3, 5} C = {1, 2, 3}
Are A and C independent?
P(A) = ½
P(C) = ½
P(A ∩ C) = 2/6 ≠ ½ * ½ = ¼ so A and C are not independent.
P(A | C ) = 2/3 ≠ P(A)
P(C | A) = 2/3 ≠ P(C)
Ex. Two fair dice are rolled. (Draw Table!)
Let A = {Sum is odd}
B = {sum > 9}
C = {at least one die shows a 4}
Recall that:
P(A) = 18/36 = ½
P(B) = 6/36 = 1/6
P(C) = 11/36
P(A ∩ B) = 2/36 = 1/18
P(A ∩ C) = 6/36
P(B∩C) = 2/36 = 1/18
Now find P(A | B), P(B | A), P(A | C), P(C | A), P(B | C), P(C | B),
P(A|B) = (2/36) / (6/36) = 2/6 = 1/3
P(B|A) = (2/36) / (18/36) = 2/18 = 1/9
P(A|C) = (6/36) / (11/36) = 6/11
P(C|A) = (6/36) / (18/36) = 6/18 = 1/3
P(C|B) = (2/36) / (6/36) = 2/6 = 1/3
P(B|C) = (2/36) / (11/36) = 2/11
** DO not confuse Independent with Mutually Exclusive Events!
If A and B are independent then P(A ∩ B) = P(A) P(B)
If A and B are mutually exclusive then P(A ∩ B) = 0
Ex. An urn contain 8 balls: there are 4 white, 3 red and 1 black. 2 balls are picked
randomly.
If the balls are picked with replacement (meaning the balls are picked one at a time and
the first ball is replaced before the second is picked)
Find the following probabilities:
a.
P(2 white balls are picked) =
b.
P(no white balls are picked) =
c.
P(at least 1 white ball is picked) =
d.
P(2 black balls are picked) =
Here are 2 ways to solve these problems:
1. Draw a tree diagram ( note it should have 9 leaves)
(do this on board)
2. Note that picking with replacement, means that the picks are independent, so you
can just multiply probabilities.
On the first pick P(W) = 4/8 = ½ P(R) = 3/8 P(B) = 1/8
On the second pick P(W) = 4/8 = ½ P(R) = 3/8 P(B) = 1/8
P(WW) = ½ * ½ = ¼
P(No white) = P(W´W´) = ½ * ½ = ¼
P(at least 1 white) = 1 – ¼ = ¾
P(BB) = 1/8 * 1/8 = 1/64
Using the same urn structure, if the balls are picked without replacement (meaning the
balls are picked two at a time and the first ball is NOT replaced before the second is
picked)
Find the following probabilities:
a.
P(2 white balls are picked) =
b.
P(no white balls are picked) =
c.
P(at least 1 white ball is picked) =
d.
P(2 black balls are picked) =
Here are 2 ways to solve these problems:
1. Draw a tree diagram ( note it should have 9 leaves)
(Do this on board)
2. Note that picking without replacement, means that the picks are NOT independent,
so you CANNOT just multiply probabilities.
You can look at this as a combination problem.
P(WW) = 4 C 2 / 8 C 2 = 6 / 28
P(No white) = P(W´W´) = 4 C 2 / 8 C 2 = 6/ 28
P(at least 1 white) = 1 – 6/28 = 22/28
P(BB) = 1 C 2 / 8 C 2 = 0
Diagnostic testing
Let S = {condition or disease is present}
Then S′ = {condition or disease is NOT present}
Let T = { test for S is positive}
Then T′ = { test for S is negative}
Sensitivity = P(T | S )
Sensitivity = probability that the test is + given we know the subject has the condition
Specificity = P(T′ | S′)
Specificity = probability that the test is - given we know the subject does not have the
condition
P(False Positive) = P(T | S′)
P(False Negative) = P(T′ | S)
Note that
P(False Positive) = P(T | S′) = 1 – P(T′ | S′)
and
P(False Negative) = P(T′ | S) = 1 – P(T | S)
Prevalence of a condition or disease is the percent or probability of the population that
has it.
Prevalence = P(S)
What is usually of interest is the probability the subject actually has the condition given
the test is + or the probability the subject actually does not have the condition given the
test -.
Tree Diagram
---P(T | S)
--------- P(S)---|
|
---P(T′|S)
|
|
|
---P(T|S′)
--------- P(S′)---|
---P(T′|S′)
From the tree diagram we can calculate the AND probabilities.
---P(T | S)
--------- P(S)---|
|
---P(T′|S)
|
|
|
---P(T|S′)
--------- P(S′)---|
---P(T′|S′)
P(T ∩ S ) = P(T|S) * P(S)
P(T′ ∩ S ) = P(T′|S) * P(S)
P(T ∩ S′ ) = P(T|S′) * P(S′)
P(T′ ∩ S′) = P(T′|S′) * P(S′)
Note that P(S) = P(S ∩ T) + P(S ∩ T′) and
P(T) = P(T ∩ S) + P(T ∩ S′)
Draw Venn Diagram!
To find P(S | T) =
P(S ∩ T)
P(T)
Ex. It is known that about 1% of women who get a mammogram have breast cancer. The
sensitivity of a mammogram is 0.86 and the specificity is 0.88.
Real Questions:
Given that the mammogram is +, what is the probability the woman has breast cancer?
Given that the mammogram is -, what is the probability the woman does not have
breast cancer?
To get these answers we will make a tree diagram.
We will then find P(T), P(T′), P(S ∩ T) and P(S′ ∩ T′)
These will then be used to find P(S | T) and P(S′ | T′).
Note what we are given in the problem:
P(S) = .01, P(T|S) = .86 and P(T′|S′) = .88
which means: P(S′) = .99, P(T′|S) = .14 and P(T| S′) = .12
P(T|S)=.86
---P(S) = .01 |
|
P(T′|S)=.14
|
|
P(T ∩ S ) = .01 * .86 = .0086
P(T′ ∩ S ) = .01 * .14 = .0014
|
P(T|S′)=.12
--P(S′) = .99 |
P(T′|S′)=.88
P(T ∩ S′ ) = .99 * .12 = .1188
P(T′ ∩ S′) = .99 * .88 = .8712
P(T) = P(T ∩ S) + P(T ∩ S′)
P(T) = .0086 + .1188 = .1274
P(T′) = P(T′ ∩ S) + P(T′ ∩ S′)
P(T′) = .0014 + .8712 = .8726 = 1 – P(T)
To answer the questions:
P(S | T) = P(T ∩ S) / P(T) = .0086 / .1274 = .0675 = 6.75%
P(S′ | T′) = P(T′ ∩ S′) / P(T′) = .8712 / .8726 = .9984 = 99.84%
The fraction of women who take a pregnancy test that are pregnant is 75%. The test has
a sensitivity of 93% and a specificity of 96%.
Find the probability that:
a. a woman is pregnant if the test is positive.
b. a woman is not pregnant if the test is negative.
Note what we are given in the problem:
P(S) = .75, P(T|S) = .93 and P(T′|S′) = .96
which means: P(S′) = .25, P(T′|S) = .07 and P(T| S′) = .04
P(T|S)=.93
---P(S) = .75 |
|
P(T′|S)=.07
|
|
|
P(T|S′)=.04
--P(S′) = .25 |
P(T′|S′)=.96
P(T ∩ S ) = .75 * .93 = .6975
P(T′ ∩ S ) = .75 * .07 = .0525
P(T ∩ S′ ) = .25 * .04 = .01
P(T′ ∩ S′) = .25 * .96 = .24
P(T) = P(T ∩ S) + P(T ∩ S′)
P(T) = .6975 + .01= .7075
P(T′) = P(T′ ∩ S) + P(T′ ∩ S′)
P(T′) = .0525 + .24 = .2925 = 1 – P(T)
To answer the questions:
P(S | T) = P(T ∩ S) / P(T) = .6975 / .7075 = .9859
P(S′ | T′) = P(T′ ∩ S′) / P(T′) = .24/.2925 = .8205