STAT 153 : Statistical Methods I
Lecture 7 and 8:
Introduction to Probability
Carole F. Amoako-Yirenkyi
Department of Statistics and Actuarial Science
KNUST
March 8, 2024
1 / 55
Introduction to Probability
Basic Concepts
2 / 55
Probability
Probability, a general concept, may be defined as the chance
of an event occurring.
More formally, it is the likelihood or chance that a particular outcome or event from a random experiment will occur.
The basic concepts of probability include:
- probability or random experiments
- sample spaces
- addition and multiplication rules
- probabilities of complementary events.
3 / 55
Basic Concepts
A random experiment is a chance process that leads to well
defined results.
- The word ”chance” means that even when the experiment is
repeated in the same manner every time it may result in different
outcomes.
A trial is a single performance of an experiment (that is, a
repetition of an experiment.)
- A trial means rolling one die once, flipping a coin once, choosing
one participant from a group of people etc.
An outcome is the result of a single trial of an experiment.
4 / 55
Basic Concepts: Sample space
A sample space is the set of all possible outcomes of a
random experiment.
Examples:
- The single toss of a coin yield the possible outcomes in the
set, {H,T}
- A single roll of a die{1, 2, 3, 4, 5, 6}
- A play of a football match yields {win(W ), loss(L),
draw(D)}.
5 / 55
Tree diagrams
A sample space may be defined by using a tree diagram
Figure 1: Tree diagram to find sample space for gender of three
children in a family
6 / 55
Basic Concepts: Events
An event (E) is a subset of the sample space of a random
experiment.
- Events are therefore sets of outcomes.
- An event with one outcome is called a single event. Eg.
Getting a result of 3 when a die is rolled.
- The event of getting an odd number when a die is rolled is
called a compound event since it consists of three
outcomes or simple events.
- The event of observing a total score of 8 on two tosses of a
die.
B = {(2, 6), (3, 5), (4, 4), (5, 3), (6, 2)}
7 / 55
Types of Probability: Classical Probability
Probability Based on Equally-Likely Outcomes
Uses sample spaces to determine the numerical
probability that an event will happen.
Classical probability assumes that all outcomes in the
sample space are equally likely to occur.
Equally likely events are events that have the same
probability of occurring.
Example: When a die is rolled, each outcome has the
same probability of occurring.
The probability of any event E is
Number of outcomes in E
P(E) =
Total number of outcomes in the sample space
n(E)
=
n(S)
8 / 55
Interpreting Probability
Figure 2: Interpreting probability values
Example:
If a family has three children, find the probability that exactly two
of the three children are boys.
9 / 55
Types of Probability: Relative frequency probability
(Empirical probability)
While classical probability assumes that certain outcomes
are equally likely, relative frequency probability relies on
actual experience to calculate probabilities.
It based on how often an event A occurs if an experiment is
repeated a large number of times n.
In empirical probability, one might actually roll a die 6000
times, observe the various frequencies, and use these
frequencies to determine th probability of the outcome.
This employs the Law of Large Numbers: If an
experiment is repeated again and again, in the long-run
the proportion of times the event E actually occurs is the
theoretical probability.
10 / 55
Frequency interpretation in more detail
What do we mean by P(E) is the long-run proportion of
times E occurs in repeated experiments?
As more and more information (data!) are collected, we
can estimate the probability almost perfectly.
In some books the empirical probability is defined as
number of times A occurs out of n experiments
n→∞
n
n(A)
= lim
n→∞ n
P(A) =
lim
11 / 55
Types of Probability: Subjective probability
Subjective probability: prob assigned to an event based on
subjective judgment, experience, information and belief.
Example: “There is a 90% chance that I’ll pass my
driving test.”
12 / 55
Complement of an Event
Another important concept in probability is that of complementary events.
- The complement of an event is the set of outcomes in the
sample space that are not contained in the event.
′
- Denoted as E or E c or Ē.
13 / 55
Rules of Probability
Probability is a number that is assigned to each member of a
collection of events from a random experiment that satisfies the
following properties:
If S is the sample space and E is any event in the random experiment,
Rule 1: P(S) = 1
Rule 2: 0 ≤ P(E) ≤ 1
Rule 3: If an event E cannot occur, its probability is 0.
Rule 4: If an event E is certain to occur, its probability is 1.
14 / 55
Implication of rules
The rules of probability imply that
- P(ϕ) = 0
- P(E ′ ) = 1 – P(E) or
P(E) = 1 – P(E ′ ) or
P(E ′ ) + P(E) = 1
- If E1 is contained in E2 , then P(E1 ) ≤ P(E2 ).
15 / 55
Probability of any Event
Let all experimental outcomes be listed as simple events
E1 , E2 , E3 , · · · , Ek .
We can make new events from these, e.g A = {E2 , E3 }.
The probability of any event is the sum of the probabilities
of the experimental outcomes in the event.
X
P(A) =
P(Ei )
Ei in A
Computing probabilities involves a lot of counting and
summing.
16 / 55
Example: Blood Type
A random experiment for blood types of individuals has the
following as all possible outcomes: S = {A, B, AB, O}.
These outcomes are not equally-likely and suppose the
probabilities in Ghana are P(O) = 0.44, P(A) = 0.42,
P(B) = 0.10 and P(AB) = 0.04.
All of these are between 0 and 1.
P(O) + P(A) + P(B) + P(AB) = 1
The probability that a randomly selected individual does
not have type AB is
P(AB C ) = 1 − P(AB) = 1 − 0.04 = 0.96.
The probability of either A or AB is
P{A, AB} = P(A) + P(AB) = 0.42 + 0.04 = 0.46
17 / 55
Operations on Events
In general, a compound event is a combination of two or more
outcomes or simple events.
We can make a new event by taking a union (∪) or
intersection (∩) of two events E1 and E2 .
- The Union of two events consists of all outcomes that are
contained in one event or the other, denoted as E 1 ∪ E 2 .
- The Intersection of two events consists of all outcomes that
are contained in one event and the other, denoted as
E 1 ∩ E 2.
18 / 55
Venn Diagrams
Events A & B contain their respective outcomes. The shaded
regions indicate the event relation of each diagram.
Figure 3: (a) = Union of two sets; (b)= Intersection of two sets
19 / 55
Disjoint Events (Mutually Exclusive Events)
Two events A and B are said to be disjoint mutually exclusive
if they share no common outcomes. In other words, if they
cannot happen at the same time.
For a 6-sided die, E1 = {1, 3, 5} and E2 = {2, 4, 6} are disjoint.
If E1 and E2 are disjoint, what is their intersection?
20 / 55
Disjoint Events (Mutually Exclusive Events)
Two events A and B are said to be disjoint mutually exclusive
if they share no common outcomes. In other words, if they
cannot happen at the same time.
For a 6-sided die, E1 = {1, 3, 5} and E2 = {2, 4, 6} are disjoint.
If E1 and E2 are disjoint, what is their intersection?
- Symbolically,
A∩B =ϕ
20 / 55
Disjoint Events (Mutually Exclusive Events)
Two events A and B are said to be disjoint mutually exclusive
if they share no common outcomes. In other words, if they
cannot happen at the same time.
For a 6-sided die, E1 = {1, 3, 5} and E2 = {2, 4, 6} are disjoint.
If E1 and E2 are disjoint, what is their intersection?
- Symbolically,
A∩B =ϕ
A Venn diagram for disjoint events..
Figure 4: Venn diagram showing two disjoint events
20 / 55
Mutually Exclusive Events - Laws
1
Commutative law (event order is unimportant):
- A∩B =B∩A
2
and
A∪B =B∪A
Distributive law (like in algebra):
- (A ∪ B) ∩ C = (A ∩ C) ∪ (B ∩ C)
- (A ∩ B) ∪ C = (A ∪ C) ∩ (B ∪ C)
3
Associative law (like in algebra):
- (A ∪ B) ∪ C = A ∪ (B ∪ C)
- (A ∩ B) ∩ C = A ∩ (B ∩ C)
4
DeMorgan’s law:
′
′
′
′
- (A ∪ B) = A ∩ B ′ The complement of the union is the
intersection of the complements.
′
- (A ∩ B) = A ∪ B The complement of the intersection is
the union of the complements.
21 / 55
Addition Rules
22 / 55
Probability of a Union: Addition rules
Addition rules are used to compute probabilities for a union of
events (both disjoint and non-disjoint cases).
Addition Rule 1: For two mutually exclusive events, the
probability that A or B will occur is
P(A ∪ B) = P(A) + P(B)
since P(A ∩ B) = ϕ
Addition Rule 2: If A and B are not mutually exclusive
P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
23 / 55
Probability Rules
We therefore extend the rules or axioms of probability to include
the following:
Rule 5: For any two events E1 and E2 with E1 ∩ E2 = ϕ
P(E1 ∪ E2 ) = P(E1 ) + P(E2 )
Rule 6: For any two events E1 and E2 with E1 ∩ E2 non-empty,
P(E1 ∪ E2 ) = P(E1 ) + P(E2 ) − P(E1 ∩ E2 )
24 / 55
Addition Rules Extended
For a collection of events Ei that are pairwise mutually
exclusive, that is, Ei ∩ Ej = ϕ, for all i , j,
P(E1 ∪ E2 ∪ ... ∪ Ek ) =
k
X
P(Ei )
i=1
For three or more events,
P(A ∪ B ∪ C) = P(A) + P(B) + P(C) − P(A ∩ B)
P(A ∩ C) − P(B ∩ C) + P(A ∩ B ∩ C)
Note the alternating signs.
25 / 55
Example: Hair dye and Eye colour
The table below shows the relationship between hair dye and
eye colour for 1770 German men
Find the probability that a randomly chosen person from this
group has
1
black hair
2
black hair or red hair
3
blue eyes
4
black hair and blue eyes
5
black hair or blue eyes
26 / 55
Probability of an Intersection
Multiplication Rules
27 / 55
Independent events and Exhaustive events
Two events are said to be independent if the occurrence
of one has no effect on the occurrence of the other.
A collection of events E1 , E2 , · · · , Ek are said to be
exhaustive if
E1 ∪ E2 ∪ · · · ∪ Ek = S
28 / 55
Probability of an Intersection: Multiplication Rules
The multiplication rules can be used to find the probability of
two or more events that occur in sequence.
Recall the definition of independent events. Two events A and
B are independent if the fact that A occurs does not affect the
probability of B occurring.
Examples:
1
Rolling a die and getting a 6, and rolling a second die and
getting a 3.
2
Drawing a card from a deck and getting a queen, replacing
it, and drawing a second card and getting a queen.
29 / 55
Multiplication Rules
Multiplication Rule 1: When two events are independent,
the probability of both occurring is
P(A ∩ B) = P(A) · P(B)
Example:
A coin is flipped and a die is rolled. Find the probability of getting
a head on the coin and a 4 on the die.
Ans:
The sample space for a coin is S1 = {H, T }.
The sample space for a dis is S2 = {1, 2, 3, 4, 5, 6}
1
1 1
Then P(H and 4) = P(H ∩ 4) = · =
≈ 0.083.
2 6
12
30 / 55
Example: Blood Type
From the previous blood type example we stated that 44% of
the Ghanaian population has type O blood. Suppose it is also
true that 15% of the population is Rh negative and that this is
independent of blood group.
If this is the case, if someone is chosen at random, the probability that the person has type O, Rh negative blood is
P(O ∩ Rh) = P(O) × P(Rh)
= 0.44 × 0.15 = 0.066
31 / 55
Multiplication Rules
Sometimes we have information before we compute a
probability.
Events that are not independent are said to be dependent
events. In this case the outcome or occurrence of the first
event affects the outcome or occurrence of the second
event in such a way that the probability is changed.
When events are independent, we need to consider the
conditional probability of one event, given that the other
event has happened.
We use the notation P(E2 |E1 ) to represent the probability
of E2 happening when E1 has already occurred.
32 / 55
Conditional Probability
For example, we might know an individual from a group
has blue eyes.
Knowing this fact reduces the population to only those that
have blue eyes.
This can change the probability of having black hair.
33 / 55
The conditional probability of an event E2 given an event
E1 , denoted as P(E2 |E1 ), is:
P(E2 |E1 ) =
P(E1 ∩ E2 )
P(E1 )
for P(E1 ) > 0.
Example: Consider choosing a man at random from the group.
The probability of the man having blue eyes given that he has
black hair is
P(Blue eyes|Black hair) =
=
P(Black hair and blue eyes)
Black hair
200/1770
200
=
= 0.40
500/1770
500
34 / 55
Multiplication Rule 2
Multiplication Rule 2: When two events are dependent,
the probability of both occurring,
P(A ∩ B) = P(A) · P(B|A)
Example:
Consider choosing a man at random from the group, what is the
probability that the man will have red hair and brown eyes? Hair
color and eye color are dependent, so finding this probability
involves using a conditional probability.
P(R hair and Br eyes) = P(red hair) × P(brown eyes|red hair)
= 70/1770 × 20/70 =
20
1770
35 / 55
Sampling without replacement
Another situation that gives rise to conditional probabilites is
when we sample from a group withour replacing. Since the object is not replaced, the probability of the first event changes the
probability with which the second event occurs.
Example:
Three cards are drawn from an ordinary deck and not replaced.
Find the probability of these events.
a.
Getting 3 jacks
b.
Getting an ace, a king, and a queen in order
c.
Getting a club, a spade, and a heart in order
36 / 55
Independence with Multiple Events
The events E1 , E2 , · · · , Ek are independent if and only if, for any
subset of these events:
P(Ei1 ∩ Ei2 ∩ · · · , ∩Eik ) = P(Ei1 ) · P(Ei2 ) · · · · · P(Eik )
Example:
37 / 55
Figure 5
38 / 55
Total Probability Rule
Sometimes the probability of an event is given under each of
several conditions. With enough of these conditional
probabilities, the probability of the event can be recovered.
Recall the definition of mutually exclusive events, in this case
subsets:
disjoint; nothing in common; cannot occur at the same time.
Figure 6
′
A and A are mutually exclusive.
′
A ∩ B and A ∩ B are mutually exclusive
′
B = (A ∩ B) ∪ (A ∩ B)
39 / 55
Total Probability Rule
For any two events A and B
P(B)
′
= P(B ∩ A) + P(B ∩ A )
′
′
= P(B|A) · P(A) + P(B|A ) · P(A )
This concept can be extended to as many mutually exclusive
and exhaustive events as there are:
Then P(B)
= P(B ∩ E1 ) + P(B ∩ E2 + ... + P(B ∩ Ek
= P(B|E1 ) · P(E1 ) + P(B|E2 ) · P(E2 ) + ... + P(B|Ek ) · P(Ek )
40 / 55
Example: Picking a red ball
Box 1 contains 2 red balls and 1 yellow ball. Box 2 contains 3
yellow balls and 1 red ball.
A coin is tossed. If it falls heads up, box 1 is selected and a ball
is drawn. If it falls tails up, box 2 is selected and a ball is drawn.
Find the probability of selecting a red ball.
41 / 55
Example: Hand size
Consider choosing someone at random from a population that
is 60% female and 40% male.
1
Suppose a woman is chosen. What is the probability of
having a hand size smaller than 100 cm2 ?
2
For a man, what is the probability of having a hand size
smaller than 100 cm2 ?
3
What is the probability that the randomly chosen person
will have a hand size smaller than 100 cm2 ?
42 / 55
Solution
Let SH be the event of choosing someone with a small hand
size, W the event of selecting a woman and M the event of selecting a man.
186
186/1000
=
600/1000
600
= 0.31
P(SH|W ) =
32/1000
32
=
400/1000
400
= 0.08
P(SH|M) =
Therefore
P(SH) = P(W ) · P(SH|W ) × P(M) · P(SH|M)
= 0.6(0.31) + 0.4(0.08)
= 0.186 + 0.032 = 0.218
43 / 55
Bayes’ Theorem
Named after Rev. Thomas Bayes (1702-1761), an English
mathematician and Presbyterian minister.
He was one of the first to work on revising or estimating
probabilities when new information on a random sample is
obtained.
The conditional probability of E2 given E1 is defined to be,
P(E2 |E1 ) =
P(E2 ) · P(E1 |E2 )
for P(E1 ) > 0
P(E1 )
E1 and E2 are mutually exclusive and exhaustive.
44 / 55
Bayes’ Theorem
Their corresponding probabilities P(E1 ), P(E2 ), · · · P(Ek )
are called prior probabilities because they were obtained
before any new information was taken into account.
A probability that is revised based on new information is
called a posterior probabilities
It is the cornerstone of the statistical methods known as
‘Bayesian Statistics’
45 / 55
Example: Hand size
We found P(SH) = 0.218 using the total probability rule. Then if we
are interested in finding
P(W |SH),
P(W |SH)
=
=
P(W ) · P(SH|W )P(H)
P(SH)
0.60 · 0.31
= 0.8532
0.128
Interpretation: Of the people who have hands less than 100cm2 ,
approximately 83.3% are women.
Note: Note that this value can also be obtained straight from the
table.
Find P(M|SH).
46 / 55
Bayes Theorem with Total Probability
If E1 , E2 , ...Ek are k mutually exclusive and exhaustive events
and B is any event,
P(E1 |B) =
P(B|E1 )P(E1 )
P(B|E1 )P(E1 ) + P(B|E2 )P(E2 ) + ... + P(B|Ek )P(Ek )
where P(B)>0
Note : Numerator expression is always one of the terms in the
sum of the denominator.
47 / 55
In biological sciences
More relevant examples might be
- P(heart attack this year | cholestrol > 190)
- P(cavities | do not brush)
- P(low stress | exercise)
- P(pertussis | vaccinated)
48 / 55
Independent events
Two events are independent if knowing one has occurred
does not change the probability of the other. This is true for
any one of the following equivalent statements:
1
P(A|B) = P(A)
2
P(B|A) = P(B)
3
P(A ∩ B) = P(A) · P(B)
Recall that P(Black hair) = 0.28 and
P(Black hair|blue eyes) = 0.19. Are “black hair” and “blue
eyes” independent events?
49 / 55
Summary
From a relative frequency perspective of n equally likely outcomes:
Marginal Prob: P(A) = (number of outcomes in A) / n
Union Prob: P(A ∪ B) = (number of outcomes in A ∪ B) / n
Joint / Intersection Prob: P(A ∩ B) = (number of
outcomes in A ∩ B) / n
Conditional Prob: P(B|A) = number of outcomes in A ∩ B
/ number of outcomes in A
Bonus advice: Always identify which of these you have been
given in the question!
50 / 55
Using Probability Trees
If our event E is the result of two (or more) smaller
experiments, probability trees provide a convenient way to
find the probability of all possible outcomes.
The first experiment results in either A or B.
After the first experiment, a second experiment is carried
out resulting in either C or D.
The four possible events are AC , AD, BC , or BD and their
probabilities are obtained from a probability tree by
multiplying numbers going down the appropriate branch.
51 / 55
Example: Medical testing
Suppose that a particular test has a 95% chance of detecting the
disease if the person has it (this is called the sensitivity of the
test) and a 90% chance of correctly indicating that the disease
is absent if the person really does not have the disease (this is
called the specificity of the test).
Suppose 8% of the population has the disease.
What is the probability of testing positive?
52 / 55
53 / 55
A more relevant question...
Given that my test come out positive, what is the
probability that I have the disease?
This is the conditional probability
P(disease|test positive) =
=
P(disease and test positive)
P(test positive)
0.076
0.168
= 0.452
54 / 55
Thank you
55 / 55