Introduction to Probability
Basic Concepts
A probability experiment is any experiment whose outcomes relies purely on chance (e.g. throwing a die). It has several possible outcomes, collectively called the sample space “S”.
An event is any possible subset of the sample space, e.g. the event
“throw the die and receive a six”, or “throw the die and receive an even number”. Such events are usually labelled using capital letters, e.g. A, B, E, ...
Simple Probability
If all possible outcomes in the sample space are equally likely (e.g. the die is “fair”) then we can compute the probability of any event E, written P(E), by
number of ways the event can occur n(E)
P(E) = ---------------------------------------------- = ------
number of all possible outcomes n(S)
Example:
The sample space S consists of six equally likely outcomes S=
{1,2,3,4,5,6}. Hence n(S)=6.
There is only one way to get a “six”, hence n(E)=1, and P(E)=1/6.
Important:
What is the probability of throwing a fair die and receiving a “six”?
• The above formula only applies if all outcomes are equally likely.
Suppose a die is weighted so that the “six” is three times as likely to occur than any other number. What is the probability of receiving a six on a single throw of this die?
• All probabilities are in between 0 and 1 (inclusive).
where P(E)=0 means the event is impossible.
i.e. E contains no elements of S, e.g. “receive a seven on a die throw” and P(E)=1 means the event is certain.
i.e. E contains all elements of S e.g. “receive a number between 1 and 6”
• The sum of outcome probabilities 3 P(E) must equal 1.
Page 1 of 24 Page 2 of 24

A Note on Playing Cards
Some examples will be based on playing card. The following is some basic information that you might need for such questions.
A standard 52-card deck (we will not consider “Jokers”) consists of four different suits: Clubs, Spades, Hearts and Diamonds. Clubs and
Spades are “black”, Hearts and Diamonds are “red”.
Each suit contains 13 different denominations:
Ace, 2, 3, 4, 5, 6, 7, 8, 9, 10, Jack, Queen, King
In some games the Ace is counted as a “1”, in other games it is counted as the highest card (above King).
Basic Probability Examples:
Find the probability of i) drawing a single card and receiving a face card.
ii) drawing a single card and receiving a spade or a 5.
iii) throwing two dice and receiving a sum of six.
iv) throwing two dice and receiving a sum of seven.
Page 3 of 24 Page 4 of 24

Empirical probability:
If we perform an experiment many times, we can get an idea of the probability of an event simply by counting favourable outcomes.
number of favourable outcomes f
P(E) = -------------------------------------------- = ----
number of trials n
Examples:
1. All 2500 employees in a company are surveyed. 1450 of them contribute to their RRSP. Hence, if we pick an employee at random, the probability that they contribute to their RRSP is
P(E)=1450/2500=0.58.
2. A slot machine is played 2000 times in a row. On 290 plays, the machine produces a payout (of any amount). Hence, empirically, we could state that the probability of a payout on any play for this machine is P(E)=0.145.
Note: Someone else might play the machine 2000 times and receive 305 payouts. The empirical probability is not guaranteed to be the same for all possible samples. This idea is explored in detail in the next unit.
Probability Rules and Compound Events
Suppose a new drug is tested on 500 patients, and 100 patients have negative side effects. Further, 200 patients are female, and 60 of the female patients experience side effects.
We can now discuss two separate events:
Let A=”patient has side effects.”
Let B= “patient is female.”
We can discuss the probability of different combinations of events. To help us, it can be useful to produce a two-way table (or contingency table) of all possible outcome combinations.
Side Effect No Side Effect Total
Female
Male
Total
Page 5 of 24 Page 6 of 24

1. Negation (“NOT”) of Events
What is the probability that a random patient is not female? This is written P(not B) or P(B’).
What is the probability of a random patient being female, ie P(B) =
Hence, P(B’) =
In general, for any negation P(B’) = 1 - P(B)
3. Union (“OR”) of Events
What is the probability that a random patient is female or has side effects ( or both )? This is written P(A or B) or P(A c B).
What is P(A) =
What is P(B) =
(“side effects”)
(“female”)
Can we add these probabilities to find P(A or B)? Why or why not?
2. Intersection (“AND”) of Events
What is the probability that a random patient is female and has side effects - written P(A and B) or P(A 1 B)?
From the table, P(A and B) =
Hence P(A or B) =
In general, we compute the probability of any “OR” event by using the “addition rule”:
P(A or B) =
(note: we will cover a more general version of the “and” rule later)
Page 7 of 24 Page 8 of 24

Special Case for “OR”:
If the events A and B do not have an overlap (we call such events mutually exclusive), then we can simply add their respective probabilities:
P(A or B) = P(A) + P(B) e.g. What is the probability that a randomly selected patient is a male with side effects or a female with side effect?
Page 9 of 24
4. Conditional Probability (“Given that”)
P(A given B) or P(A | B) is the probability of event A, given that we already know event B has occurred, i.e. the probability that a patient has side effects, given that we already know that the patient is female.
We deal with conditional probabilities by reducing the sample space.
In this case, knowing that event B has occured, we can reduce the sample space to just 200 women. Out of them, 60 had side effects.
Hence P(A | B) = eg) Find the probability that a patient is female, given that we know the patient experienced side effects.
P( B | A) = eg) Find the probability that a patient is male, given that we know the patient did not experience side effects.
P(not B | not A) =
Note that, in general, P(A|B) … P(B|A).
Page 10 of 24

Conditional Probability - Independence of Events
Compare these probabilities:
The probability that a random patient has side effects is
P(A) =
The probability that a patient has side effects, given that we know the patient is female is
P(A | B) =
In this case, P(A) … P(A|B), that is, knowing the patient’s gender has influenced our knowledge of the likelihood that the patient will experience side effects.
In other words, the occurrence of event B changed the probability of event A. Such events are called dependent.
Now consider throwing two dice, one red, the other green. Find the probability of getting a six on the red die (event A) given that we already know that the green die shows a six (event B).
Clearly, the number on the green die does not influence the outcome of the red die. These events are called independent, the probabilities are not changed:
P(A)=P(A | B) (and P(B)=P(B|A))
Another Example year students respectively. 90% of the first year students are also taking Engl 100, while only 50% of second and 72.5% of third year students do so. a) Find the probability that a random student is i) also taking Engl 100.
100" independent?
Page 11 of 24 Page 12 of 24

The General Multiplication Rule for “A and B”
Suppose we wish to calculate P(A and B) for two events.
If both events occur, then event A must occur first, followed by event
B (or vice versa). However, since A has already occurred by the time
B occurs, we must use the conditional probability P(B|A).
Hence, in general P(A and B) = P(A) A P(B|A)
(or = P(B) A P(A|B) )
This is sometimes called the “Multiplication Rule”
In the special case that events are independent,
P(A and B) = P(A) A P(B)
(since P(B|A)=P(B)) e.g. Two cards are drawn at random without replacement. What is the probability that both cards are spades?
In general, rewrite these problems (in your head) as a long & detailed sentence: “What is the probability that the “first card is a spade” AND the “second card is a spade
GIVEN THAT the first card was a spade”.
Answer:
A Side Note: Counting and Probability
(We will need a part of this for a later section, but you do not need to spend a lot of time on this.)
1. Full Arrangements
There are n! ways to arrange (or permute) n objects.
E.g. Arrange 20 kids in all possible lineups. How long does this take, if each lineup requires just one second?
2. Partial Arrangements (Permutations)
There are ways to arrange k objects chosen out of n objects.
(Note - Arrangement: Order Matters)
E.g. How many playlists consisting of five songs could you program into your player if you could choose from 30 different songs (without song repetition)?
Page 13 of 24 Page 14 of 24

3. Selections (Combinations)
If the order doesn’t matter, we can reduce our choices by a factor of k!.
There are ways to “choose” k objects out of n objects.
E.g. Find the probability of a Royal Flush in Poker.
How many possible Poker hands exist?
There are four possible royal flushes, hence P(E) =
E.g. Find the probability of winning Lotto 6-49.
How many ways can we choose 6 out of 49 numbers
(order doesn’t matter!)
Hence the probability of winning the jackpot is
P(E)=
Probability Models and Expected Values
So far, we have considered probabilities of particular events. To investigate probabilities of all possible outcomes, we assign a numerical variable (e.g. X) to the outcomes, and calculate the associated probabilities. Such a variable X is called a random variable, and the associated probabilities form a probability distribution.
e.g. Suppose a $2 instant lottery ticket has a 15% chance of winning $5, a 1% chance of winning $50, and a 0.01% chance of winning $1000.
We are interested in the random variable X, the amount won when a single ticket is purchased.
Since there are only four outcomes, we can summarize the results in a table. The probability distribution is hence: x
P(x)
As required: 1) 0 # P(x) # 1 and 2) 3 P(x)=1
Page 15 of 24 Page 16 of 24

We can now also answer questions on other events.
What’s the probability of winning at least $50?
P(X $ 50)=
What’s the probability of winning anything?
P(X>0) =
Expected Value
We can also bring our discussion on weighted means to probability.
To find the mean, or expected value, of a probability distribution, we calculate the mean using the probabilities as weights.
I.e. Mean/Expected Value of X is
: (X) = E(X) = 3 X P(X)
E.g.
If you bought one instant lottery ticket, how much can you expect to win?
E(X) =
Note: Of course you will never actually win $1.35 on any game.
However, if you were to purchase 100 tickets, you might expect to win $1.35 on average , i.e. a total of $135.
Examples
1. A concert organizer is watching the weather for an upcoming outdoor event. If the weather is nice, the organizer expects to make a profit of $15,000 on the event. In the event of light rain, the organizer expects fewer people to show up, resulting in a reduced profit of only
$9,000. If the event of heavy rain, the concert will need to be cancelled, leaving the organizer with a $8,000 loss.
The weather forecast indicates that there is a 30% chance of light rain, and a 9% chance of heavy rain (there is no chance of snow, hail, tornado, etc).
Construct the probability distribution table for the random variable X, where X is the concert profit/loss.
Then calculate the expected profit/loss for this event.
Page 17 of 24 Page 18 of 24

2. A piggy bank contains 3 loonies, 25 quarters, 5 dimes, 7 nickels, and 37 pennies. If a single coin is drawn at random from this piggy bank, what is its expected value?
3. In a game of chance, two fair dice are thrown. If you throw two sixes, you win $40. If you get any other pair (ones through fives), you win $20. If you don’t get a pair, you have to pay
$5. What are your expected “earnings” per game played. Is this game worth playing?
The Binomial Distribution - a discrete distribution
A binomial (or Bernouilli) experiment is one that is repeated multiple times, has only two outcomes (success or failure), and for which each repetition is independent of each other.
Example: We throw a fair die three times and want to count the number of sixes we get.
This experiment
- is repeated (think of the experiment as a single die throw, repeated three times)
- has only success (getting a six) or failure (getting any other number) as outcomes
- has each throw independent (die has no memory of previous throw)
Hence this qualifies as a binomial experiment.
Page 19 of 24 Page 20 of 24

a) What is the probability of getting exactly one six?
First, note that the probability of success for each trial
Next, the probability of failure must be q =
There are n= trials.
Let k be the number of successes, here k=
We want one six (success) and one “non-six” (failure) and another
“non-six” (failure). By the multiplication rule, the probability of this event is
P(E) =
However, we could also get the six on the second or third throw!
Since there are three possible “spots” for the six to appear, and we choose one of them, there are
C(n,k) = C(3,1) = 3 possible combinations.
The probability of having exactly one six is
P(X=1) = b) We now throw the die 6 times. What is the probability of getting exactly 2 sixes?
Now, k=2 successes followed by 4 failures has probability
P(E) =
However, again, the two successes could occur anywhere in the six throws. So we need to multiply this by the number of ways that we can distribute two successes among six throws.
Again C(n,k) = C(6,2) =
Hence the probability of getting exactly two “sixes” is
P(X=2) =
General Formula:
If X has a binomial distribution, then the probability of having exactly
“k” successes in “n” trials is given by
P(X=k) = C(n,k) p q n-k
Page 21 of 24 Page 22 of 24

e.g.
We now throw the die 6 times. What is the probability of getting at least one six?
This translates to “either one six or two sixes or three sixes”, and via the addition rule we get:
P(k $ 1) = P(k=1)+P(k=2)+P(k=3)+P(k=4)+P(k=5)+P(k=6)
Each of these would need to be worked out separately using the above formula!
A simpler approach is to use
Examples
1. Voter turnout in the 2008 Canadian federal election was the lowest ever. Only 59.1% of the electorate cast a ballot.
If we select a random sample of six individuals from the
Canadian electorate, what is the probability that a) exactly one of them cast a ballot?
b) at least one of the cast a ballot?
2. Suppose on any given day in November, the chance of snow is always 15%. a) How likely is it to get fewer than 3 days of snow in
November?
b) How many snowy days would you expect in November?
Note:
If X is a binomial variable, we can use a shortcut formula to calculate its mean and standard deviation:
Mean: µ = np Standard Deviation F = / npq
Page 23 of 24 Page 24 of 24