Probability (Part 1)
Random Experiments
Probability
Rules of Probability
Independent Events
Chapter
5
Random Experiments
 Sample Space
• A random experiment is an observational process
whose results cannot be known in advance.
• The set of all outcomes (S) is the sample space for
the experiment.
• A sample space with a countable number of
outcomes is discrete.
Random Experiments
 Sample Space
• For example, when CitiBank makes a consumer
loan, the sample space is:
S = {default, no default}
• The sample space describing a Wal-Mart
customer’s payment method is:
S = {cash, debit card, credit card, check}
Random Experiments
 Sample Space
• For a single roll of a die, the sample space is:
S = {1, 2, 3, 4, 5, 6}
• When two dice are rolled, the sample space is
the following pairs:
S = {(1,1), (1,2), (1,3), (1,4), (1,5), (1,6),
(2,1), (2,2), (2,3), (2,4), (2,5), (2,6),
(3,1), (3,2), (3,3), (3,4), (3,5), (3,6),
(4,1), (4,2), (4,3), (4,4), (4,5), (4,6),
(5,1), (5,2), (5,3), (5,4), (5,5), (5,6),
(6,1), (6,2), (6,3), (6,4), (6,5), (6,6)}
Random Experiments
 Sample Space
• Consider the sample space to describe a randomly
chosen United Airlines employee by
2 genders,
21 job classifications,
6 home bases (major hubs) and
4 education levels
There are: 2 x 21 x 6 x 4 = 1008 possible outcomes
• It would be impractical to enumerate this sample
space.
Random Experiments
 Sample Space
• If the outcome is a continuous measurement, the
sample space can be described by a rule.
• For example, the sample space for the length of a
randomly chosen cell phone call would be
S = {all X such that X > 0}
or written as S = {X | X > 0}
• The sample space to describe a randomly chosen
student’s GPA would be
S = {X | 0.00 < X < 4.00}
Random Experiments
 Events
• An event is any subset of outcomes in the sample
space.
• A simple event or elementary event, is a single
outcome.
• A discrete sample space S consists of all the
simple events (Ei):
S = {E1, E2, …, En}
Random Experiments
 Events
• Consider the random experiment of tossing a
balanced coin.
What is the sample space?
S = {H, T}
• What are the chances of observing a H or T?
• These two elementary events are equally likely.
• When you buy a lottery ticket, the sample space
S = {win, lose} has only two events.
• Are these two events equally likely to occur?
Random Experiments
 Events
• A compound event consists of two or more simple
events.
• For example, in a sample space of 6 simple
events, we could define the compound events
A = {E1, E2}
B = {E3, E5, E6}
• These are
displayed in a
Venn diagram:
Random Experiments
 Events
• Many different compound events could be defined.
• Compound events can be described by a rule.
• For example, the compound event
A = “rolling a seven” on a roll of two
dice consists of 6 simple events:
S = {(1,6), (2,5), (3,4), (4,3), (5,2), (6,1)}
Probability
 Definitions
• The probability of an event is a number that
measures the relative likelihood that the event will
occur.
• The probability of event A [denoted P(A)], must lie
within the interval from 0 to 1:
0 < P(A) < 1
If P(A) = 0, then the
event cannot occur.
If P(A) = 1, then the event
is certain to occur.
Probability
 Definitions
• In a discrete sample space, the probabilities of all
simple events must sum to unity:
P(S) = P(E1) + P(E2) + … + P(En) = 1
• For example, if the following number of purchases
were made by
credit card:
32%
debit card:
20%
cash:
35%
P(cash) = .35
check:
18%
P(check) = .18
Sum = 100%
Sum = 1.0
P(credit card) = .32
Probability
P(debit card) = .20
Probability
• Businesses want to be able to quantify the
uncertainty of future events.
• For example, what are the chances that next
month’s revenue will exceed last year’s average?
• How can we increase the chance of positive future
events and decrease the chance of negative future
events?
• The study of probability helps us understand and
quantify the uncertainty surrounding the future.
Probability
 What is Probability?
• Three approaches to probability:
Approach
Example
Empirical
There is a 2 percent chance
of twins in a randomlychosen birth.
There is a 50 % probability
of heads on a coin flip.
Classical
Subjective
There is a 75 % chance that England will
adopt the Euro currency by 2010.
Probability
 Empirical Approach
• Use the empirical or relative frequency approach
to assign probabilities by counting the frequency
(fi) of observed outcomes defined on the
experimental sample space.
• For example, to estimate the default rate on
student loans:
P(a student defaults) = f /n
=
number of defaults
number of loans
Probability
 Empirical Approach
• Necessary when there is no prior knowledge of
events.
• As the number of observations (n) increases or the
number of times the experiment is performed, the
estimate will become more accurate.
Probability
 Law of Large Numbers
• The law of large numbers is an important
probability theorem that states that a large sample
is preferred to a small one.
• Flip a coin 50 times. We would expect the
proportion of heads to be near .50.
• However, in a small finite sample, any ratio can be
obtained (e.g., 1/3, 7/13, 10/22, 28/50, etc.).
• A large n may be needed to get close to .50.
• Consider the results of 10, 20, 50, and 500 coin
flips.
Probability
Probability
 Practical Issues for Actuaries
• Actuarial science is a high-paying career that
involves estimating empirical probabilities.
• For example, actuaries
- calculate payout rates on life insurance,
pension plans, and health care plans
- create tables that guide IRA withdrawal
rates for individuals from age 70 to 99
Probability
 Practical Issues for Actuaries
• Challenges that actuaries face:
- Is n “large enough” to say that f/n has become a
good approximation to P(A)?
- Was the experiment repeated identically?
- Is the underlying process invariant over time?
- Do nonstatistical factors override data
collection?
- What if repeated trials are impossible?
Probability
 Classical Approach
• In this approach, we envision the entire sample
space as a collection of equally likely outcomes.
• Instead of performing the experiment, we can use
deduction to determine P(A).
• a priori refers to the process of assigning
probabilities before the event is observed.
• a priori probabilities are based on logic, not
experience.
Probability
 Classical Approach
• For example, the two dice experiment has 36
equally likely simple events. The P(7) is
P( A) 
number of outcomes with 7 dots
6

 0.1667
number of outcomes in sample space 36
• The probability is
obtained a priori
using the classical
approach as shown
in this Venn diagram
for 2 dice:
Probability
 Subjective Approach
• A subjective probability reflects someone’s
personal belief about the likelihood of an event.
• Used when there is no repeatable random
experiment.
• For example,
- What is the probability that a new truck
product program will show a return on
investment of at least 10 percent?
- What is the probability that the price of GM
stock will rise within the next 30 days?
Probability
 Subjective Approach
• These probabilities rely on personal judgment or
expert opinion.
• Judgment is based on experience with similar
events and knowledge of the underlying causal
processes.
Rules of Probability
 Complement of an Event
• The complement of an event A is denoted by
A′ and consists of everything in the sample space
S except event A.
Rules of Probability
 Complement of an Event
• Since A and A′ together comprise the entire
sample space,
P(A) + P(A′ ) = 1
• The probability of A′ is found by
P(A′ ) = 1 – P(A)
• For example, The Wall Street Journal reports that
about 33% of all new small businesses fail within
the first 2 years. The probability that a new small
business will survive is:
P(survival) = 1 – P(failure) = 1 – .33 = .67 or 67%
Rules of Probability
 Odds of an Event
• The odds in favor of event A occurring is
Odds =
P( A)
P( A)

P( A ') 1  P( A)
• Odds are used in sports and games of chance.
• For a pair of fair dice, P(7) = 6/36 (or 1/6).
What are the odds in favor of rolling a 7?
P(rolling seven)
1/ 6
1/ 6 1
Odds =



1  P(rolling seven) 1  1/ 6 5/ 6 5
Rules of Probability
 Odds of an Event
• On the average, for every time a 7 is rolled, there
will be 5 times that it is not rolled.
• In other words, the odds are 1 to 5 in favor of
rolling a 7.
• The odds are 5 to 1 against rolling a 7.
• In horse racing and other sports, odds are usually
quoted against winning.
Rules of Probability
 Odds of an Event
• If the odds against event A are quoted as b to a,
then the implied probability of event A is:
a
P(A) =
ab
• For example, if a race horse has a 4 to 1 odds
against winning, the P(win) is
a
1
1

  0.20 or 20%
P(win) =
a  b 4 1 5
Rules of Probability
 Union of Two Events
• The union of two events consists of all outcomes in
the sample space S that are contained either in
event A or in event B or both
(denoted A  B or “A or B”).
 may be read
as “or” since
one or the other
or both events
may occur.
Rules of Probability
 Union of Two Events
• For example, randomly choose a card from a deck
of 52 playing cards.
• If Q is the event that we draw a
queen and R is the event that we
draw a red card, what is Q  R?
• It is the possibility of drawing
either a queen (4 ways)
or a red card (26 ways)
or both (2 ways).
Rules of Probability
 Intersection of Two Events
• The intersection of two events A and B
(denoted A  B or “A and B”) is the event
consisting of all outcomes in the sample space S
that are contained in both event A and event B.
 may be read
as “and” since
both events
occur. This is a
joint probability.
Rules of Probability
 Intersection of Two Events
• For example, randomly choose a card from a deck
of 52 playing cards.
• If Q is the event that we draw a
queen and R is the event that we
draw a red card, what is
Q  R?
• It is the possibility of getting
both a queen and a red card
(2 ways).
Rules of Probability
 General Law of Addition
• The general law of addition states that the
probability of the union of two events A and B is:
P(A  B) = P(A) + P(B) – P(A  B)
When you add
So, you have
A and B
the P(A) and
to subtract
P(B) together,
P(A  B) to
you count the
avoid overA
B
P(A and B)
stating the
twice.
probability.
Rules of Probability
 General Law of Addition
• For the card example:
P(Q) = 4/52 (4 queens in a deck)
P(R) = 26/52 (26 red cards in a deck)
P(Q  R) = 2/52 (2 red queens in a deck)
P(Q  R) = P(Q) + P(R) – P(Q  Q)
Q and R = 2/52
= 4/52 + 26/52 – 2/52
= 28/52 = .5385 or 53.85%
Q
4/52
R
26/52
Rules of Probability
 Mutually Exclusive Events
• Events A and B are mutually exclusive (or disjoint)
if their intersection is the null set () that contains
no elements. If A  B = , then P(A  B) = 0
 Special Law of Addition
• In the case of mutually
exclusive events, the
addition law reduces
to:
P(A  B) = P(A) + P(B)
Rules of Probability
 Collectively Exhaustive Events
• Events are collectively exhaustive if their union is
the entire sample space S.
• Two mutually exclusive, collectively exhaustive
events are dichotomous (or binary) events.
Warranty
No
Warranty
For example, a car repair
is either covered by the
warranty (A) or not (B).
Rules of Probability
 Collectively Exhaustive Events
• More than two mutually exclusive, collectively
exhaustive events are polytomous events.
Cash
Debit
Card
Credit
Card
Check
For example, a Wal-Mart customer can pay by credit
card (A), debit card (B), cash (C) or check (D).
Rules of Probability
 Forced Dichotomy
• Polytomous events can be made dichotomous
(binary) by defining the second category as
everything not in the first category.
Polytomous Events
Binary (Dichotomous) Variable
Vehicle type (SUV, sedan, truck,
motorcycle)
X = 1 if SUV, 0 otherwise
A randomly-chosen NBA player’s
height
X = 1 if height exceeds 7 feet, 0
otherwise
Tax return type (single, married filing
jointly, married filing separately, head
of household, qualifying widower)
X = 1 if single, 0 otherwise
Rules of Probability
 Conditional Probability
• The probability of event A given that event B has
occurred.
• Denoted P(A | B).
The vertical line “ | ” is read as “given.”
P( A  B)
P( A | B) 
P( B)
for P(B) > 0 and
undefined otherwise
Rules of Probability
 Conditional Probability
• Consider the logic of this formula by looking at the
Venn diagram.
The sample space is
P( A  B)
restricted to B, an event
P( A | B) 
P( B)
that has occurred.
A  B is the part of B
that is also in A.
The ratio of the relative
size of A  B to B is
P(A | B).
Rules of Probability
 Example: High School Dropouts
• Of the population aged 16 – 21 and not in college:
Unemployed
13.5%
High school dropouts
29.05%
Unemployed high school dropouts
5.32%
• What is the conditional probability that a member
of this population is unemployed, given that the
person is a high school dropout?
Rules of Probability
 Example: High School Dropouts
• First define
U = the event that the person is unemployed
D = the event that the person is a high school
dropout
P(D) = .2905
P(UD) = .0532
P(U) = .1350
P(U  D) .0532
P(U | D) 

 .1831 or 18.31%
P( D)
.2905
• P(U | D) = .1831 > P(U) = .1350
• Therefore, being a high school dropout is related
to being unemployed.
Independent Events
• Event A is independent of event B if the conditional
probability P(A | B) is the same as the marginal
probability P(A).
• To check for independence, apply this test:
If P(A | B) = P(A) then event A is independent of B.
• Another way to check for independence:
If P(A  B) = P(A)P(B) then event A is
independent of event B since
P(A | B) = P(A  B) = P(A)P(B) = P(A)
P(B)
P(B)
Independent Events
 Example: Television Ads
• Out of a target audience of 2,000,000, ad A
reaches 500,000 viewers, B reaches 300,000
viewers and both ads reach 100,000 viewers.
300, 000
500, 000
P( B) 
 .15
P( A) 
 .25
2, 000, 000
2, 000, 000
100, 000
P( A  B) 
 .05
2, 000, 000
• What is P(A | B)?
P( A  B) .05
.3333 or 33%
P( A | B) 

 .30
P( B)
.15
Independent Events
 Example: Television Ads
• So, P(ad A) = .25
P(ad B) = .15
P(A  B) = .05
P(A | B) = .3333
• Are events A and B independent?
• P(A | B) = .3333 ≠ P(A) = .25
• P(A)P(B)=(.25)(.15)=.0375 ≠ P(A  B)=.05
Independent Events
 Dependent Events
• When P(A) ≠ P(A | B), then events A and B are
dependent.
• For dependent events, knowing that event B has
occurred will affect the probability that event A will
occur.
• Statistical dependence does not prove causality.
• For example, knowing a person’s age would affect
the probability that the individual uses text
messaging but causation would have to be proven
in other ways.
Independent Events
 Actuaries Again
• An actuary studies conditional probabilities
empirically, using
- accident statistics
- mortality tables
- insurance claims records
• Many businesses rely on actuarial services, so a
business student needs to understand the
concepts of conditional probability and statistical
independence.
Independent Events
 Multiplication Law for Independent Events
• The probability of n independent events occurring
simultaneously is:
P(A1  A2  ...  An) = P(A1) P(A2) ... P(An)
if the events are independent
• To illustrate system reliability, suppose a Web site
has 2 independent file servers. Each server has
99% reliability. What is the total system reliability?
Let,
F1 be the event that server 1 fails
F2 be the event that server 2 fails
Independent Events
 Multiplication Law for Independent Events
• Applying the rule of independence:
P(F1  F2 ) =
P(F1) P(F2) = (.01)(.01) = .0001
• So, the probability that both servers are down is
.0001.
• The probability that at least one server is “up” is:
1 - .0001 = .9999 or 99.99%
Independent Events
 Example: Space Shuttle
• Redundancy can increase system reliability even
when individual component reliability is low.
• NASA space shuttle has three independent flight
computers (triple redundancy).
• Each has an unacceptable .03 chance of failure
(3 failures in 100 missions).
• Let Fj = event that computer j fails.
Independent Events
 Example: Space Shuttle
• What is the probability that all three flight
computers will fail?
P(all 3 fail) = P(F1  F2  F3)
= P(F1) P(F2) P(F3)  presuming
that failures
are
= (0.03)(0.03)(0.03)
independent
= 0.000027
or 27 in 1,000,000 missions.
Independent Events
 The Five Nines Rule
• How high must reliability be?
• Public carrier-class telecommunications data links
are expected to be available 99.999% of the time.
• The five nines rule implies only 5 minutes of
downtime per year.
• This type of reliability is needed in many business
situations.
Independent Events
 The Five Nines Rule
• For example,
Independent Events
 How Much Redundancy is Needed?
•
Suppose a certain network Web server is up only 94% of the
time. What is the probability of it being down?
P(down) = 1 – P(up) = 1 – .94 = .06
•
How many independent servers are needed to ensure that
the system is up at least 99.99% of the time (or down only
1 - .9999 = .0001 or .01% of the time)?
Independent Events
 How Much Redundancy is Needed?
2 servers: P(F1  F2) = (0.06)(0.06) = 0.0036
3 servers: P(F1  F2  F3)
= (0.06)(0.06)(0.06) = 0.000216
4 servers: P(F1  F2  F3  F4)
= (0.06)(0.06)(0.06)(0.06)
=0.00001296
• So, to achieve a 99.99% up time, 4 redundant
servers will be needed.
Applied Statistics in
Business and Economics
End of Part 1 of Chapter 5