Econ 131 – Quantitative Economics
(a.k.a. Introductory Econometrics)
Review of probability concepts
Instructor: Anthony G. Sabarillo
University of the Philippines School of Economics
Not for distribution. For lecture purposes only.
1
Outline
• Basic probability concepts
• Theorems of probability
• Calculating probabilities and counting techniques
• Conditional probability
• Theorem of total probabilities, Bayes’ theorem, and the multiplication rule
• Statistical independence
Not for distribution. For lecture purposes only.
2
Basic probability concepts
• Probability theory
• the mathematics of randomness
• concerned with quantifying the exact or estimated chance that a random
event will occur
• Experiment
• a process whose outcome is not known in advance with certainty
• outcomes are “mutually exclusive potential results” of an experiment
(Stock and Watson 2020, p. 56)
• can be repeated, theoretically, an infinite number of times
• has a well-defined set of possible outcomes
• e.g., throwing a die
• Is measuring a person’s blood pressure an experiment?
Not for distribution. For lecture purposes only.
3
Basic probability concepts
• Sample space
• set of all possible outcomes of the experiment
• What is the sample space of the experiment of
• ….throwing a six-faced die?
• …tossing a coin?
• Event
• a subset of the sample space
• e.g., an odd outcome from throwing a die
Not for distribution. For lecture purposes only.
4
Basic probability concepts
• Let 𝐴𝐴 and 𝐵𝐵 be any two events that are subsets of the same sample space 𝑆𝑆
(i.e., 𝐴𝐴 ⊆ 𝑆𝑆 and 𝐵𝐵 ⊆ 𝑆𝑆).
• (digression: What does the ff. mean: 𝐴𝐴 ⊂ 𝑆𝑆?)
• Define the following events or sets:
• 𝐴𝐴 ∩ 𝐵𝐵
• 𝐴𝐴 ∪ 𝐵𝐵
• 𝐴𝐴𝐶𝐶
• What does it mean when 𝐴𝐴 ∩ 𝐵𝐵 = ∅?
• De Morgan’s Law
𝐴𝐴 ∪ 𝐵𝐵 𝐶𝐶 = 𝐴𝐴𝐶𝐶 ∩ 𝐵𝐵𝐶𝐶
𝑜𝑜𝑜𝑜
𝐴𝐴 ∩ 𝐵𝐵 𝐶𝐶 = 𝐴𝐴𝐶𝐶 ∪ 𝐵𝐵𝐶𝐶
Not for distribution. For lecture purposes only.
5
Basic probability concepts
• Definition of probability.
• “the proportion of the time that the outcome occurs in the long run”
(Stock and Watson 2020, p. 56)
• Formally: Let 𝐴𝐴 be an event in a sample space 𝑆𝑆. The probability of the event
𝐴𝐴, denoted by 𝑃𝑃(𝐴𝐴), is measured as the proportion of times that event 𝐴𝐴 will
occur in repeated trials of an experiment.
• Approximating probability using relative frequency:
• Suppose that in a total of 𝑛𝑛 possible equally likely outcomes of an experiment,
𝑚𝑚 of them are favorable to the occurrence of the event 𝐴𝐴;
• then ratio 𝑚𝑚/𝑛𝑛 is the relative frequency of 𝐴𝐴.
• For a large 𝑛𝑛, the relative frequency will provide a very good approximation of
the probability of 𝐴𝐴.
Not for distribution. For lecture purposes only.
6
Basic probability concepts
• Probability ≡ a nonnegative number assigned to every event
• Axioms of probability
• 𝑃𝑃 𝐴𝐴 ≥ 0 ∀ event 𝐴𝐴
• 𝑃𝑃 𝑆𝑆 = 1, 𝑆𝑆 ≡ sample space
Not for distribution. For lecture purposes only.
7
Basic probability concepts
• Classical probability
• If a sample space has a finite # of simple events with equal probabilities:
number of outcomes in 𝐴𝐴
𝑃𝑃 𝐴𝐴 =
number of outcomes in 𝑆𝑆
• Empirical Probability
• We can estimate the probability of an event A by repeating the same
experiment over and over (say n times) and observing the number of times
the event occurs.
• Other basic probability concepts
Not for distribution. For lecture purposes only.
8
Calculating probabilities
• When the sample space contains a finite number of simple events with equal
probabilities,
• and when a composite event is composed of mutually exclusive simple events:
• 𝑃𝑃(composite event)=∑ 𝑃𝑃(mutually exclusive simple events)
• Recall: If a sample space has a finite # of simple events with equal probabilities,
• 𝑃𝑃 𝐴𝐴 =
number of outcomes in 𝐴𝐴
number of outcomes in 𝑆𝑆
• Exercise:
• Consider the experiment of tossing a coin twice. What is the probability of getting
at least one tail?
Not for distribution. For lecture purposes only.
9
Counting methods
• Multiplication rule
• Permutation
• Combination
Not for distribution. For lecture purposes only.
10
The Multiplication Rule
• if a task is done in two stages and if the first stage can be done in 𝑚𝑚 ways
and for each way that the first stage can be done, the second stage can be
done in 𝑛𝑛 ways, then the two together can be done in 𝑚𝑚 × 𝑛𝑛 ways.
• Example 1: Suppose that a coin is tossed then a six-sided dice is rolled.
Since there are two possible outcomes in the first activity and six possible
outcomes in the second activity, the number of possible outcomes is
________.
• Example 2: Suppose you’re given the ff. menu:
• Meal: Chicken, pork, beef
• Drink: Soda, coffee, tea
• How many meal-drink combos can you order?
Not for distribution. For lecture purposes only.
11
Permutation
• The number of possible arrangements in a set when the order of the
arrangements matters.
• The number of permutations of 𝑛𝑛 elements taken 𝑟𝑟 at a time
is given by the ff. formula:
𝑃𝑃𝑟𝑟𝑛𝑛 =
𝑛𝑛!
𝑛𝑛 − 𝑟𝑟 !
• In other words: This is the number of ways we can take 𝑟𝑟 objects from 𝑛𝑛
choices (where the order matters).
• Example: In how many ways can you arrange 3 letters taken from the word
“ALMOST”?
Not for distribution. For lecture purposes only.
12
Combination
• number of ways to form a collection of unordered elements.
• The number of ways to form combinations of size 𝑟𝑟 from a set of 𝑛𝑛 distinct
objects, without repetition, is:
𝑛𝑛!
𝑛𝑛
𝑛𝑛
𝐶𝐶𝑟𝑟 =
=
𝑟𝑟
𝑛𝑛 − 𝑟𝑟 ! 𝑟𝑟!
• In other words: The combination of 𝑛𝑛 taken 𝑟𝑟 is the number of ways we
can take 𝑟𝑟 objects from 𝑛𝑛 choices, where the order does not matter.
• Example: (Larsen and Marx) There are nine students, five of whom are
men and four of whom are women, who interviewed for four summer
internships sponsored by a city newspaper. In how many ways can the
newspaper choose a set of four interns?
Not for distribution. For lecture purposes only.
13
Conditional Probability
• Given two events 𝐴𝐴 and 𝐵𝐵,
the probability of 𝐴𝐴 given that 𝐵𝐵 has occurred
is the conditional probability of 𝐴𝐴 given 𝐵𝐵.
• Notation: 𝑃𝑃(𝐴𝐴|𝐵𝐵)
• More formally:
• For events 𝐴𝐴 & 𝐵𝐵 where 𝑃𝑃(𝐵𝐵) > 0,
• 𝑃𝑃 𝐴𝐴 𝐵𝐵
𝑃𝑃(𝐴𝐴∩𝐵𝐵)
=
𝑃𝑃(𝐵𝐵)
Not for distribution. For lecture purposes only.
14
Conditional Probability
• Example:
• Experiment: tossing a fair coin twice
• Find: the probability of two heads occurring given that at least one head
has occurred
Not for distribution. For lecture purposes only.
15
Statistical independence
Def. Independence of events
Events 𝐴𝐴 and 𝐵𝐵 are independent
(or pairwise independent or statistically independent)
if and only if any one of the following conditions is satisfied:
1. 𝑃𝑃(𝐴𝐴 ∩ 𝐵𝐵) = 𝑃𝑃(𝐴𝐴)𝑃𝑃(𝐵𝐵)
2. 𝑃𝑃 𝐴𝐴 𝐵𝐵 = 𝑃𝑃(𝐴𝐴) if 𝑃𝑃(𝐵𝐵) > 0
3. 𝑃𝑃(𝐵𝐵|𝐴𝐴) = 𝑃𝑃(𝐵𝐵) if 𝑃𝑃(𝐴𝐴) > 0
Not for distribution. For lecture purposes only.
16
Statistical independence
• Example: Suppose you toss two dice. Let 𝐴𝐴 denote the event of a 1 on the
second die, 𝐵𝐵 the event of getting a 1 on the first die. Are 𝐴𝐴 and 𝐵𝐵
independent?
Not for distribution. For lecture purposes only.
17
Statistical independence
• Def. Events 𝐴𝐴, 𝐵𝐵, and 𝐶𝐶 are mutually independent if and only if
• 1) 𝑃𝑃 𝐴𝐴 ∩ 𝐵𝐵 = 𝑃𝑃 𝐴𝐴 𝑃𝑃(𝐵𝐵) and
• 2) 𝑃𝑃 𝐴𝐴 ∩ 𝐶𝐶 = 𝑃𝑃 𝐴𝐴 𝑃𝑃(𝐶𝐶) and
• 3) 𝑃𝑃 𝐵𝐵 ∩ 𝐶𝐶 = 𝑃𝑃 𝐵𝐵 𝑃𝑃(𝐶𝐶) and
• 4) 𝑃𝑃 𝐴𝐴 ∩ 𝐵𝐵 ∩ 𝐶𝐶 = 𝑃𝑃 𝐴𝐴 𝑃𝑃(𝐵𝐵)𝑃𝑃(𝐶𝐶).
Not for distribution. For lecture purposes only.
18
Statistical independence
• Example
• Experiment: toss a fair coin twice
• 𝐻𝐻𝐻𝐻, 𝐻𝐻𝐻𝐻, 𝑇𝑇𝑇𝑇, 𝑇𝑇𝑇𝑇
• Let 𝐴𝐴 ≡ 𝐻𝐻 on the first toss
• Let 𝐵𝐵 ≡ 𝐻𝐻 on the second toss
• Let 𝐶𝐶 ≡ 𝐻𝐻𝐻𝐻 or 𝑇𝑇𝑇𝑇
• Determine if 𝐴𝐴, 𝐵𝐵, and 𝐶𝐶 are mutually independent.
Not for distribution. For lecture purposes only.
19
Statistical independence
Def. Independence of several events
Events 𝐴𝐴1 , 𝐴𝐴2 , … , 𝐴𝐴𝑛𝑛 are mutually independent if and only if
• any proper subset of events are mutually independent, and
• 𝑃𝑃 𝐴𝐴1 ∩ 𝐴𝐴2 ∩ ⋯ ∩ 𝐴𝐴𝑛𝑛 = 𝑃𝑃 𝐴𝐴1 𝑃𝑃 𝐴𝐴2 … 𝑃𝑃 𝐴𝐴𝑛𝑛
Not for distribution. For lecture purposes only.
20
Appendix
Not for distribution. For lecture purposes only.
21
Basic probability concepts (back)
• 1. 𝑃𝑃 ∅ = 0
• 2. If 𝐴𝐴1 , 𝐴𝐴2 , … , 𝐴𝐴𝑛𝑛 are mutually exclusive events, then
𝑃𝑃 𝐴𝐴1 ∪ 𝐴𝐴2 ∪ ⋯ ∪ 𝐴𝐴𝑛𝑛 = 𝑃𝑃 𝐴𝐴1 + 𝑃𝑃 𝐴𝐴2 + ⋯ + 𝑃𝑃(𝐴𝐴𝑛𝑛 )
• 3. 𝑃𝑃 𝐴𝐴 = 1 − 𝑃𝑃 𝐴𝐴𝐶𝐶
• 4. For any two events 𝐴𝐴 and 𝐵𝐵,
• 𝑃𝑃 𝐴𝐴 = 𝑃𝑃 𝐴𝐴 ∩ 𝐵𝐵 + 𝑃𝑃 𝐴𝐴 ∩ 𝐵𝐵𝐶𝐶
• 𝑃𝑃 𝐴𝐴 − 𝐵𝐵 = 𝑃𝑃 𝐴𝐴 − 𝑃𝑃 𝐵𝐵
• 5. 𝑃𝑃 𝐴𝐴 ∪ 𝐵𝐵 = 𝑃𝑃 𝐴𝐴 + 𝑃𝑃 𝐵𝐵 − 𝑃𝑃(𝐴𝐴 ∩ 𝐵𝐵)
• 6. For any two events 𝐴𝐴 and 𝐵𝐵,
• if 𝐴𝐴 ⊂ 𝐵𝐵 then 𝑃𝑃 𝐴𝐴 < 𝑃𝑃(𝐵𝐵)
Not for distribution. For lecture purposes only.
22