Chapter 5 – Probability
Defn: A random experiment is one for which the outcome cannot be predicted with certainty.
Defn: The set of all possible outcomes of a random experiment is called the sample space of the
experiment.
Defn: An event is a subset of the sample space. It may consist of one or more possible outcomes of
the experiment. If the result of performing the experiment is an outcome contained in an event A, we
say that the event A has occurred.
Example: Let our random experiment consist of tossing a coin twice. The sample space, S, consists of
the four possible outcomes. S = {HH, HT, TH, TT}. One event that we can define for this
experiment is the event that at least one Head results from tossing the coin twice. This event may be
denoted by A = {HH, HT, TH}. If we perform the experiment and the actual result is {HT}, then we
will say that the event A has occurred.
Defn: An event which contains a single outcome is called a simple event. An event which contains
more than one outcome is called a compound event.
In the example above, A = {HH, HT, TH} is a compound event;
B = {HT} is a simple event.
The probability of an event is a number which measures the likelihood that the event will happen when
we perform the random experiment. How do we assign numerical probabilities to events?
There are three approaches to assigning probabilities to events when examining the outcome of a
random experiment:
1) Classical method: In the classical approach, we assume that every outcome of the experiment is
equally likely. If there are N possible outcomes of the experiment, then the probability of occurrence
of any one of those outcomes is 1/N.
In general, if E is an event which contains n(E) of the possible outcomes of a random experiment, and
if the total number of possible outcomes is n(S), the size of the sample space, then we say that the
probability of occurrence of the event E is
P( E ) 
n( E )
n( S )
.
If we flip a fair coin twice, there are four possible outcomes, and we assign a probability of ¼ to each
of them. For the event A = {at least one head occurs} = {HH, HT, TH}, there are three possible
outcomes, and the probability that A occurs is P(A) = 0.75.
Note: The probability of the certain event, S, namely that some outcome occurs, is 1.
2) Empirical, or relative frequency, method: In this approach to assigning probabilities to events, we
look at the long-run proportion of occurrences of particular outcomes, when the random experiment is
performed many times. This long-run proportion tells us the approximate probability of occurrence of
each outcome.
Example: If we flip a coin once, what is the likelihood that the outcome is a head? Why? For a single
coin flip, we cannot say with certainty what the outcome will be. However, if we flip a coin
1,000,000 times, we are fairly sure that approximately one-half of the outcomes will be heads.
This approach is based on the Law of Large Numbers, which says, in particular, that the relative
frequency of occurrence of a particular outcome of a random experiment approaches a specific limiting
number between 0 and 1 if we perform the experiment a very large number of times.
Law of Large Numbers: As the number of repetitions of a random experiment increases, the fraction
of times that a particular event occurs gets closer and closer to a numerical limiting value. That value
is the probability of occurrence of the event.
Important Note on Interpretation of a Probability: In the example of flipping a fair coin twice, let event
A = {at least one head}. The probability that this event happens is 0.75. Interpret this probability: The
random experiment is to flip a fair coin twice. If we perform this experiment very many times, the
fraction of times that we get at least one head will get closer and closer to 0.75.
Important Note: If I randomly select a member of a population, the probability that I obtain a member
who has a certain characteristic is the fraction of the population who have that characteristic.
Example: p. 234, Exercises 25, 26, 27, 28
Example: Consider the random experiment of rolling two dice, a red one and a green one. The sample
space contains 36 possible outcomes, as listed in the table on the handout. If we assume that the dice
are fair, then each of the outcomes is equally likely and has probability 1/36. We can then find the
probability associated with various events, as shown on the handout, by counting the number of
outcomes in each event.
Example: p. 235, Exercise 40
3) Subjective method: Sometimes, neither of the above approaches will work for assigning numerical
values for the probabilities of occurrence of the possible outcomes of a random experiment. In such
situations, probability values are assigned based on educated guesses about the relative likelihoods of
the different possible outcomes. We will not use this approach in this course, as it is rather
complicated, and involves advanced concepts and principles, such as entropy.
Defn: Two events A and B are said to be mutually exclusive, or disjoint, if at most one of them can
occur when the random experiment is performed. I.e., A and B have no outcomes in common.
Example: Let our random experiment be to roll a pair of fair dice, one green and one red. Let the
event A = {sum of numbers showing on top face of dice is 7}; let the event B = {the number showing
on the green die is 6}; let the event C = {sum of the numbers showing on top face of dice is 11}. Are
events A and B mutually exclusive? Events B and C? Events A and C?
Example: p. 245, Exercises 5, 6
Defn: The complement of an event A is the set of outcomes in S which are not in A. We denote the
complement of A by A or by Ac.
Note: If A is any event, either A can occur, or Ac can occur, but they cannot both occur. Thus an
event and its complement are mutually exclusive.
Complement Rule: If A is an event, with probability P(A), then P(Ac) = 1 – P(A).
Example: Roll two fair dice, a green one and a red one. Let A = {the sum of the numbers showing on
the dice is 7} = {(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)}. Then P(A) = 1/6. Since Ac = {the sum of
the numbers showing on the dice is not 7}, then Ac contains the other 30 possible outcomes, and P(Ac)
= 30/36 = 5/6.
Defn: The union of two events A and B is an event that contains all outcomes that are either in A or in
B or in both. We denote this event by (A  B) or by (A OR B).
Defn: The intersection of two events A and B is that event which consists of those outcomes which
are in A and in B. Outcomes which are only in A or only in B are not in the intersection. We denote
this event by (A  B), or by (A AND B).
Note: Two events A and B are mutually exclusive if an only if
(A  B) contains no outcomes, and thus cannot occur.
Basic Laws of Probability:
1) For any event A, 0  P(A)  1.
2) P(S) = 1. In other words, the outcome of the random experiment is certain to be in the sample
space.
3) If two events A and B are mutually exclusive, then
P( A  B)  P( A)  P( B) .
Addition Rule for Non-Mutually Exclusive Events: For any two events A and B, we have
P(A  B) = P(A) + P(B) – P(A  B).
Example: p. 246, Exercises 13, 14, 15, 16.
Example: p. 247, Exercise 30.
Defn: If A and B are two events such that P(B) > 0, then the conditional probability that A occurs,
P A  B 
.
P B 
Note: The definition also says that P  A  B   P  A | B  P  B  .
given that B occurs, is given by P A | B  
Example: p. 262, Exercises 11, 19.
Example: Suppose that our random experiment consists of rolling a pair of fair dice, one green and
one red. Let A = {the sum of numbers showing on top face of dice is 7}; let D = {neither number
showing on the top faces is greater than 4}. What is P(A)? What is P(A|D)?
Defn: Two events A and B are independent if the occurrence or non-occurrence of one does not
change the probability that the other occurs. If the events are not independent, then they are called
dependent.
Note: If two events are mutually exclusive, then cannot be independent. If they are independent, they
cannot be mutually exclusive. If they are not mutually exclusive, then they may or may not be
independent; other information would be needed in order to decide whether they are independent.
Multiplication Rule: If the events A and B are independent, then P(A  B) = P(A)P(B).
Alternatively, the events are independent if
P  A | B   P  A and P  B | A  P  B  .
Example: p. 262, Exercises 11, 17
Combining Rules of Probability
Example: pp. 262-263, Exercises 19, 24