Chapter 6
Probability
Assigning probabilities to Events
• Random experiment
– a random experiment is a process or course of
action, whose outcome is uncertain.
• Examples
Experiment
• Flip a coin
• Record a statistics test marks
• Measure the time to assemble
computer
Outcomes
Heads and Tails
Numbers between 0 and
100
numbers from zero to
infinite
Probabilities…
•
•
•
List the outcomes of a random experiment…
This list must be exhaustive, i.e. ALL possible
outcomes included.
Die roll {1,2,3,4,5}
Die roll {1,2,3,4,5,6}
The list must be mutually exclusive, i.e. no
two outcomes can occur at the same time:
Die roll {odd number or even number}
Die roll{ number less than 4 or even number}
Assigning probabilities to Events
• Performing the same random experiment
repeatedly, may result in different
outcomes, therefore, the best we can do is
consider the probability of occurrence of
a certain outcome.
• To determine the probabilities we need to
define and list the possible outcomes first.
Sample Space
• Determining the outcomes.
– Build an exhaustive list of all possible
outcomes.
– Make sure the listed outcomes are mutually
exclusive.
• A list of outcomes that meets the two
conditions above is called a sample
space.
Sample Space: S = {O1,
O2,…,Ok}
O1
O2
Sample Space
a sample space of a
random experiment
is a list of all possible
outcomes of the
experiment. The
outcomes must be
mutually exclusive and
exhaustive.
Simple Events
Event
An event is any collection
of one or more simple events
Our objective is to
determine P(A), the
probability that event A
will occur.
The individual
outcomes are called
simple events.
Simple events cannot
be further
decomposed
into constituent
outcomes.
Assigning Probabilities
– Given a sample space S={O1,O2,…,Ok}, the
following characteristics for the probability
P(Oi) of the simple event Oi must hold:
1.
0  POi   1 for each i
k
2.
 PO   1
i
i 1
– Probability of an event: The probability P(A)
of event A is the sum of the probabilities
assigned to the simple events contained in
A.
Approaches to Assigning
Probabilities…
•There are three ways to assign a probability,
P(Oi), to an outcome, Oi, namely:
•Classical approach: make certain
assumptions (such as equally likely,
independence) about situation.
•Relative frequency: assigning probabilities
based on experimentation or historical data.
•Subjective approach: Assigning
probabilities based on the assignor’s
judgment.
Example 1(Textbook 6.3)
•
A quiz contains multiple-choice questions with
five possible answers, only one of which is
correct. A student plans to guess the answers
because he knows absolutely nothing about
the subject.
a. Produce the sample space for each question.
b. Assign probabilities to the simple events in the
sample space you produced.
c. Which approach did you use to answer part b
d. Interpret the probabilities you assigned in part b
Solution
• a {a is correct, b is correct, c is correct, d
is correct, e is correct}
• b P(a is correct) = P(b is correct) = P(c is
correct) = P(d is correct) = P(e is correct)
= .2
• c Classical approach
• d In the long run all answers are equally
likely to be correct.
Joint, Marginal, and Conditional
Probability
• We study methods to determine probabilities of
events that result from combining other events in
various ways.
• There are several types of combinations and
relationships between events:
–
–
–
–
Intersection of events
Union of events
Dependent and independent events
Complement event
Intersection
• The intersection of event A and B is the event
that occurs when both A and B occur.
• The intersection of events A and B is denoted by
(A and B).
• The joint probability of A and B is the probability
of the intersection of A and B, which is denoted
by P(A and B)
Intersection
• Additional Example – 1
– The number of spots turning up when a six-side
die is tossed is observed. Consider the following
events.
– A: The number observed is at most 2.
– B: The number observed is an even number.
– Determine the probability of the intersection event
A and B.
1
3
A
2
A and B
B
P(A and B)
= P(2) = 1/6
6
4
5
Marginal Probability
• These probabilities are computed by
adding across rows and down columns
Mutual fund
outperforms
the market
(B1)
Mutual fund
doesn’t
outperform the
market (B2)
Marginal
Prob.
P(Ai)
Top 20 MBA program (A1)
P(A1 and B1)+ P(A1 and B2)
= P(A1)
Not top 20 MBA program (A2)
P(A2 and B1)+ P(A2 and B2)
= P(A2)
Marginal Probability P(Bj)
Marginal Probability
• These probabilities are computed by
adding across rows and down columns
Mutual fund
outperforms
the market
(B1)
Top 20 MBA program (A1)
Not top 20 MBA program
(A2)
Marginal Probability P(Bj)
P(A1 and B1)
+
P(A2 and B1
=
P(B1)
Mutual fund
doesn’t
outperform the
market (B2)
P(A1 and B2)
+
P(A2 and B2
=
P(B2)
Marginal
Prob.
P(Ai)
.40
.60
Conditional Probability
• Example 6.2 (Example 6.1 – continued)
– Find the conditional probability that a randomly
selected fund is managed by a “Top 20 MBA
Program graduate”, given that it did not outperform
the market.
• Solution
P(A1|B2) = P(A1 and B2) = .29 = .3949
P(B2)
.83
Independence
• Independent events
– Two events A and B are said to be
independent if
P(A|B) = P(A)
or
P(B|A) = P(B)
– That is, the probability of one event is not
affected by the occurrence of the other event.
Union
• The union event of A and B is the event
that occurs when either A or B or both
occur.
• It is denoted “A or B”.
Example 2
• Suppose we have the following joint probabilities
A1
A2
A3
B1
0.2
0.15
0.1
B2
0.25
0.25
0.05
• Compute the marginal probabilities
Solution
Example 3
•
•
The following tables lists the joint probabilities associated with smoking and
lung disease among 60 to 65 year-old-men.
He is a smoker
He is a nonsmoker
He has lung
disease
0.1
0.03
He does not have
lung disease
0.21
0.66
One 60 to 65 year old man is selected at random. What is the probability of
the following events
a. He is a smoker
b. He does not have lung disease
c. He has lung disease given that he is a smoker
d. He has lung disease given that he does not smoke
Solution
Probability Rules and Trees
• We present more methods to determine
the probability of the intersection and the
union of two events.
• Three rules assist us in determining the
probability of complex events from the
probability of simpler events.
Complement Rule
• The complement of event A (denoted by
AC) is the event that occurs when event A
does not occur.
• The probability of the complement event is
calculated by
A and AC consist of all the
simple events in the
sample space. Therefore,
P(A) + P(AC) = 1
P(AC) = 1 - P(A)
Multiplication Rule
• For any two events A and B
P(A and B) = P(A)P(B|A)
= P(B)P(A|B)
• When A and B are independent
P(A and B) = P(A)P(B)
Probability Trees
• This is a useful device to calculate
probabilities when using the probability
rules.
Example 4 (Textbook 6.52)
•
a.
b.
c.
d.
Approximately 10% of people are lefthanded. If two people are selected at
random, what is the probability of the
following events?
Both are right-handed
Both are left-handed
One is right-handed and the other is lefthanded
At least one is right-handed
Solution
Addition Rule
For any two events A and B
P(A or B) = P(A) + P(B) - P(A and B)
P(A) =6/13
+
A
P(B) =5/13
_
P(A and B) =3/13
P(A or B) = 8/13
B
A or B
Addition Rule
When A and B are mutually exclusive,
P(A or B) = P(A) + P(B)
A
B
B
P(A and B) = 0
決勝二十一點數學問題
• 當你上益智節目，主持人有三個門讓你選，其中
一個門打開有百萬名車帶回家，另外兩個門是兩
串蕉說掰掰，假如你隨便選了一個門是 A 門好了，
這時候主持人為了節目效果，打開了 C 門，後面
空無一物，現在只剩下 A , B 兩個門了，主持人再
問你要不要換選擇，這時候到底該選哪一個門？
• 要記住，主持人一開始就知道哪個門有百萬名車，
為了節目效果，不論你一開始選中或是沒中，他
都一定會讓你有再選一次的機會！這就是整個機
率關鍵的 Scenario！
【凡人都能懂的版】 Solution
• 選了A之後，A的機率是三分之一，把「B
和C」視為一體，加起來的機率是三分之二，
當主持人說出C不是之後，他可以完全肯定
如果獎在「B和C」這一組，就一定是B，
所以這三分之二的機率就完全算在B頭上了。
就這樣而已。
【有簡單機率背景的人的版】
Solution
• 條件機率問題
• 用數學來算，就要分為你(妳)選對了門，主持人
拿掉C的機率和你(妳)選錯了門時，主持人拿掉C
的機率，這兩種情況來算，最後的計算結果，還
是會跟上面一樣。（B是對的機率=「A是對的機
率」X「B才是對的機率」+「A是錯的機率」X
「B才是對的機率」=「1/3」X「0」+「2/3」X
「1」=「2/3」）
Bayes’ Law
• Conditional probability is used to find the
probability of an event given that one of its
possible causes has occurred.
• We use Bayes’ law to find the probability
of the possible cause given that an event
has occurred.
Bayes’ Law
• Example
– Medical tests can produce false-positive or falsenegative results.
– A particular test is found to perform as follows:
• Correctly diagnose “Positive” 94% of the time.
• Correctly diagnose “Negative” 98% of the time.
– It is known that 4% of men in the general population
suffer from the illness.
– What is the probability that a man is suffering from
the illness, if the test result were positive?
Bayes’ Law
• Solution
– Define the following events
•
•
•
•
D = Has a disease
DC = Does not have the disease
PT = Positive test results
NT = Negative test results
– Build a probability tree
Bayes’ Law
• Solution – Continued
– The probabilities provided are:
• P(D) = .04
• P(PT|D) = .94
• P(PT|DC) = .02
P(DC) = .96
P(NT|D)= .06
P(NT|DC) = .98
– The probability to be determined is
P(D | PT)
Bayes’ Law
P(D and PT)
=.0376
+
P(DC and PT)
=.0192
P(PT) =.0568
P(D | PT) 
P(D and PT)
P(PT)
.0376

 .6620
.0568
Bayes’ Law
Prior
probabilities
Likelihood
probabilities
Posterior probabilities
P(D | PT)  .0376  .6620
.0568
Bayes’ Theorem
• To find the posterior probability that
event Ai will occur given that event B
has occurred we apply Bayes’ theorem.
P ( Ai ) P ( B| Ai )
P ( Ai | B ) 
P ( A1 ) P ( B| A1 )  P ( A2 ) P ( B| A2 ) ...  P ( An ) P ( B| An )
• Bayes’ theorem is applicable when the
events for which we want to compute
posterior probabilities are mutually
exclusive and their union is the entire
sample space.
Bayesian Terminology…
• The probabilities P(A) and P(AC) are called
prior probabilities because they are
determined prior to the decision about
taking the preparatory course.
• The conditional probability P(A | B) is
called a posterior probability (or revised
probability), because the prior probability
is revised after the decision about taking
the preparatory course.