Sample Spaces and Events
Basic Concepts in Probability:
Sample Spaces and Events
Basic Concepts in Probability:
I
Experiment: any action or process whose outcome is subject
to uncertainty
Sample Spaces and Events
Basic Concepts in Probability:
I
Experiment: any action or process whose outcome is subject
to uncertainty
e.g. tossing a coin 3 times, testing the pH value of some
reagent, counting the number of customers visiting a store in
one day, etc.
Sample Spaces and Events
Basic Concepts in Probability:
I
Experiment: any action or process whose outcome is subject
to uncertainty
e.g. tossing a coin 3 times, testing the pH value of some
reagent, counting the number of customers visiting a store in
one day, etc.
I
Sample Space: the set of all possible outcomes of an
experiment, usually denoted by S
Sample Spaces and Events
Basic Concepts in Probability:
I
Experiment: any action or process whose outcome is subject
to uncertainty
e.g. tossing a coin 3 times, testing the pH value of some
reagent, counting the number of customers visiting a store in
one day, etc.
I
Sample Space: the set of all possible outcomes of an
experiment, usually denoted by S
e.g. for the above 3 examples, the sample spaces are {TTT,
TTH, THH, THT, HHH, HHT, HTH, HTT}, [0,14] and {0,
1, 2, . . . , N, . . . }, respectively.
Sample Spaces and Events
Basic Concepts in Probability:
Sample Spaces and Events
Basic Concepts in Probability:
I
Event: any colletcion (subset) of outcomes contained in the
sample space S.
Sample Spaces and Events
Basic Concepts in Probability:
I
Event: any colletcion (subset) of outcomes contained in the
sample space S.
An event is simple if it consists of exactly one outcome and
compound if it consists of more than one outcome.
Sample Spaces and Events
Basic Concepts in Probability:
I
Event: any colletcion (subset) of outcomes contained in the
sample space S.
An event is simple if it consists of exactly one outcome and
compound if it consists of more than one outcome.
e.g. for the coin tossing example: {all the outcomes such that
the first result is Head}, i.e. {HHT, HTH, HTT, HHH}, is an
event and this is a compoud event;
Sample Spaces and Events
Basic Concepts in Probability:
I
Event: any colletcion (subset) of outcomes contained in the
sample space S.
An event is simple if it consists of exactly one outcome and
compound if it consists of more than one outcome.
e.g. for the coin tossing example: {all the outcomes such that
the first result is Head}, i.e. {HHT, HTH, HTT, HHH}, is an
event and this is a compoud event;
{all the outcomes which have 3 consecutive Head}, i.e.
{HHH}, is also an event, while this is a single event.
Sample Spaces and Events
Examples:
For the pH value testing example:
{pH value is less than 7.0}, i.e. [0, 7.0), is an event, and it is
compound;
Sample Spaces and Events
Examples:
For the pH value testing example:
{pH value is less than 7.0}, i.e. [0, 7.0), is an event, and it is
compound;
{pH value is between 2.0 and 3.0}, i.e. [2.0, 3.0], is another event,
and it is also compound.
Sample Spaces and Events
Examples:
For the pH value testing example:
{pH value is less than 7.0}, i.e. [0, 7.0), is an event, and it is
compound;
{pH value is between 2.0 and 3.0}, i.e. [2.0, 3.0], is another event,
and it is also compound.
For the customers’ visiting investigation example:
{the number of cumstomers visited in one day is less than 100},
i.e. {1, 2, 3, . . . , 98, 99}, is an event, and it is compound;
Sample Spaces and Events
Examples:
For the pH value testing example:
{pH value is less than 7.0}, i.e. [0, 7.0), is an event, and it is
compound;
{pH value is between 2.0 and 3.0}, i.e. [2.0, 3.0], is another event,
and it is also compound.
For the customers’ visiting investigation example:
{the number of cumstomers visited in one day is less than 100},
i.e. {1, 2, 3, . . . , 98, 99}, is an event, and it is compound;
{the number of cumstomers visited in one day is more than 200},
i.e. {201, 202, . . . } is also an event and it is compound.
Sample Spaces and Events
Another Coin Tossing Example:
This time our experiment is tossing a coin many times until we get
our first Head.
Then the sample space would be {1, 2, 3, 4, 5, . . . } and the
corresponding outcomes are {H, TH, TTH, TTTH, TTTTH, · · · }.
Here are some events:
{we tossed at most 3 times}, i.e. {1, 2, 3} or {H, TH, TTH}.
{we totally tossed an even number of times}, i.e. {2, 4, 6, . . . } or
{TH, TTTH, TTTTTH, · · · }.
Both events are compound.
Sample Spaces and Events
Basic Set Theory
I
Complement: the complement of an event A denoted by A’
is the set of all outcomes in S that are not contained in A.
Sample Spaces and Events
Basic Set Theory
I
Complement: the complement of an event A denoted by A’
is the set of all outcomes in S that are not contained in A.
e.g. for our first coin tossing example, if
A = {the first outcome is Head} = {HHH, HHT, HTH,
HTT}, then
A’ = {the first outcome is not Head, i.e. Tail} = {TTT,
TTH, THT, THH}
Sample Spaces and Events
Basic Set Theory
I
Complement: the complement of an event A denoted by A’
is the set of all outcomes in S that are not contained in A.
e.g. for our first coin tossing example, if
A = {the first outcome is Head} = {HHH, HHT, HTH,
HTT}, then
A’ = {the first outcome is not Head, i.e. Tail} = {TTT,
TTH, THT, THH}
for the pH value testing example, if
A = {the pH value of the reagent is below 7.0}, then
A’ = {the the pH value of the reagent is above 7.0}
Sample Spaces and Events
Basic Set Theory
I
Complement: the complement of an event A denoted by A’
is the set of all outcomes in S that are not contained in A.
e.g. for our first coin tossing example, if
A = {the first outcome is Head} = {HHH, HHT, HTH,
HTT}, then
A’ = {the first outcome is not Head, i.e. Tail} = {TTT,
TTH, THT, THH}
for the pH value testing example, if
A = {the pH value of the reagent is below 7.0}, then
A’ = {the the pH value of the reagent is above 7.0}
Sample Spaces and Events
Basic Set Theory
I
Union: the union of two events A and B, is the event
consisting of all outcomes that are eigther in A or in B or in
both events — that is, all outcomes in at least one of the
events, denoted by A∪B
Sample Spaces and Events
Basic Set Theory
I
Union: the union of two events A and B, is the event
consisting of all outcomes that are eigther in A or in B or in
both events — that is, all outcomes in at least one of the
events, denoted by A∪B
e.g. for the coin tossing example, if
A = {the first outcome is Head} = {HHH, HHT, HTH,
HTT}, and
B = {the last outcome is Head} = {HHH, TTH, HTH,
THH}, then
A ∪ B = {the first or the last outcomem is Head}
= {HHH, HHT , HTH, HTT , TTH, THH}
Sample Spaces and Events
Basic Set Theory
I
Union: the union of two events A and B, is the event
consisting of all outcomes that are eigther in A or in B or in
both events — that is, all outcomes in at least one of the
events, denoted by A∪B
e.g. for the coin tossing example, if
A = {the first outcome is Head} = {HHH, HHT, HTH,
HTT}, and
B = {the last outcome is Head} = {HHH, TTH, HTH,
THH}, then
A ∪ B = {the first or the last outcomem is Head}
= {HHH, HHT , HTH, HTT , TTH, THH}
Sample Spaces and Events
Basic Set Theory
I
Intersection: the intersection of two events A and B, is the
event consisting of all outcomes that are both in A and in B,
denoted by A∩B
Sample Spaces and Events
Basic Set Theory
I
Intersection: the intersection of two events A and B, is the
event consisting of all outcomes that are both in A and in B,
denoted by A∩B
e.g. for the coin tossing example, if
A = {the first outcome is Head} = {HHH, HHT, HTH,
HTT}, and
B = {the last outcome is Head} = {HHH, TTH, HTH,
THH}, then
A ∩ B = {the first and the last outcomem is Head}
= {HHH, HTH}
Sample Spaces and Events
Basic Set Theory
I
Intersection: the intersection of two events A and B, is the
event consisting of all outcomes that are both in A and in B,
denoted by A∩B
e.g. for the coin tossing example, if
A = {the first outcome is Head} = {HHH, HHT, HTH,
HTT}, and
B = {the last outcome is Head} = {HHH, TTH, HTH,
THH}, then
A ∩ B = {the first and the last outcomem is Head}
= {HHH, HTH}
Sample Spaces and Events
Basic Set Theory
I Null Event: the event consistion of no outcomes, denoted by
∅
Sample Spaces and Events
Basic Set Theory
I Null Event: the event consistion of no outcomes, denoted by
∅
e.g. the event {the first outcome is neither Head nor Tail} for
the coin tossing experiment is a null event.
Sample Spaces and Events
Basic Set Theory
I Null Event: the event consistion of no outcomes, denoted by
∅
e.g. the event {the first outcome is neither Head nor Tail} for
the coin tossing experiment is a null event.
I Mutually Exclusive: if two events A and B satisfy A∩B = ∅,
then A and B are said to be mutually exclusive or mutually
disjoint.
Sample Spaces and Events
Basic Set Theory
I Null Event: the event consistion of no outcomes, denoted by
∅
e.g. the event {the first outcome is neither Head nor Tail} for
the coin tossing experiment is a null event.
I Mutually Exclusive: if two events A and B satisfy A∩B = ∅,
then A and B are said to be mutually exclusive or mutually
disjoint.
e.g. for the coin tossing example, if
A = {the first outcome is Head} = {HHH, HHT, HTH,
HTT}, and
B = {the first outcome is Tail} = {THH, TTH, TTT, THT},
then
A ∩ B = {the first outcomem is Head and Tail}
=∅
So A and B are mutually disjoint.
Sample Spaces and Events
Remark:
1. The union and intersection operation can be extended to more
than two events.
Sample Spaces and Events
Remark:
1. The union and intersection operation can be extended to more
than two events.
e.g. for any three events A, B and C, the event A ∪ B ∪ C is the
set of all outcomes contained in at least one of the three events;
Similarly, A ∩ B ∩ C is the set of all outcomes contained in all
three events.
Sample Spaces and Events
Remark:
1. The union and intersection operation can be extended to more
than two events.
e.g. for any three events A, B and C, the event A ∪ B ∪ C is the
set of all outcomes contained in at least one of the three events;
Similarly, A ∩ B ∩ C is the set of all outcomes contained in all
three events.
2. Given n events A1 , A2 , . . . , An . They are said to be mutually
disjoint or pairwise disjoint, if any two events are mutually
disjoint.
Sample Spaces and Events
I
Venn Diagrams:
Sample Spaces and Events
I
Venn Diagrams:
e.g.
A∪B
A∩B
Sample Spaces and Events
I
Venn Diagrams:
e.g.
A∪B
A complement
A∩B
mutually disjoint
Axiomatic Probability
I
The objective of probability is to assign to each event A a
number P(A), called the probability of the event A, which will
give a precise measure of the chance thtat A will occur.
Axiomatic Probability
I
The objective of probability is to assign to each event A a
number P(A), called the probability of the event A, which will
give a precise measure of the chance thtat A will occur.
I
Probability Axioms:
Axiomatic Probability
I
The objective of probability is to assign to each event A a
number P(A), called the probability of the event A, which will
give a precise measure of the chance thtat A will occur.
I
Probability Axioms:
AXIOM 1 For any event A, P(A) ≥ 0.
Axiomatic Probability
I
The objective of probability is to assign to each event A a
number P(A), called the probability of the event A, which will
give a precise measure of the chance thtat A will occur.
I
Probability Axioms:
AXIOM 1 For any event A, P(A) ≥ 0.
AXIOM 2 P(S) = 1.
Axiomatic Probability
I
The objective of probability is to assign to each event A a
number P(A), called the probability of the event A, which will
give a precise measure of the chance thtat A will occur.
I
Probability Axioms:
AXIOM 1 For any event A, P(A) ≥ 0.
AXIOM 2 P(S) = 1.
AXIOM 3 If A1 , A2 , A3 , . . . is an infinite collection
of disjoint even
P
then P(A1 ∪ A2 ∪ A3 ∪ · · · ) = ∞
P(A
i)
i=1
Axiomatic Probability
Proposition
P(∅) = 0 where ∅ is the null event. This in turn implies that the
property contained in Axiom 3 is valid for finite collection of
events, i.e. if A1 , A2 , . . . , An is P
a finite collection of disjoint events,
then P(A1 ∪ A2 ∪ · · · ∪ A3 ) = ni=1 P(Ai )
Axiomatic Probability
Examples:
1. Consider the coin tossing experiment and we are only interested
in tossing the coin one time. Then S = {H, T}.
Axiomatic Probability
Examples:
1. Consider the coin tossing experiment and we are only interested
in tossing the coin one time. Then S = {H, T}.
Since P(S) = 1 (Axiom 1), and the event {H} and {T} are
mutually disjoint, by Axiom 3, we have
Axiomatic Probability
Examples:
1. Consider the coin tossing experiment and we are only interested
in tossing the coin one time. Then S = {H, T}.
Since P(S) = 1 (Axiom 1), and the event {H} and {T} are
mutually disjoint, by Axiom 3, we have
P({H}) + P({T }) = P({H} ∪ {T }) = P(S) = 1
Axiomatic Probability
Examples:
1. Consider the coin tossing experiment and we are only interested
in tossing the coin one time. Then S = {H, T}.
Since P(S) = 1 (Axiom 1), and the event {H} and {T} are
mutually disjoint, by Axiom 3, we have
P({H}) + P({T }) = P({H} ∪ {T }) = P(S) = 1
If the coin is fair, we should assign 0.5 to P({H}) and 0.5 to
P({T }).
Axiomatic Probability
Examples:
1. Consider the coin tossing experiment and we are only interested
in tossing the coin one time. Then S = {H, T}.
Since P(S) = 1 (Axiom 1), and the event {H} and {T} are
mutually disjoint, by Axiom 3, we have
P({H}) + P({T }) = P({H} ∪ {T }) = P(S) = 1
If the coin is fair, we should assign 0.5 to P({H}) and 0.5 to
P({T }).
If the coin is more likely to give a Head, then 0.8 for P({H}) and
0.2 for P({T }) may be suitable.
Axiomatic Probability
Examples:
1. Consider the coin tossing experiment and we are only interested
in tossing the coin one time. Then S = {H, T}.
Since P(S) = 1 (Axiom 1), and the event {H} and {T} are
mutually disjoint, by Axiom 3, we have
P({H}) + P({T }) = P({H} ∪ {T }) = P(S) = 1
If the coin is fair, we should assign 0.5 to P({H}) and 0.5 to
P({T }).
If the coin is more likely to give a Head, then 0.8 for P({H}) and
0.2 for P({T }) may be suitable.
In fact, if p is any fixed number between 0 and 1, then P({H}) = p
, and P({T }) = 1 − p is an assignment consistent with the axioms.
Axiomatic Probability
Examples:
2. Consider again the coin tossing example. However, this time we
are interested in getting a Head, i.e. we toss a coin many times
untill we get a Head. Then S = {H, TH, TTH, TTTH, TTTTH,
. . . }.
Axiomatic Probability
Examples:
2. Consider again the coin tossing example. However, this time we
are interested in getting a Head, i.e. we toss a coin many times
untill we get a Head. Then S = {H, TH, TTH, TTTH, TTTTH,
. . . }.
If P({H}) = 0.4 then P({T }) = 0.6, P({TH}) = (0.4)0.6,
P({TTH}) = (0.4)(0.6)2 , P({TTTH}) = (0.4)(0.6)3 , . . . .
Axiomatic Probability
Examples:
2. Consider again the coin tossing example. However, this time we
are interested in getting a Head, i.e. we toss a coin many times
untill we get a Head. Then S = {H, TH, TTH, TTTH, TTTTH,
. . . }.
If P({H}) = 0.4 then P({T }) = 0.6, P({TH}) = (0.4)0.6,
P({TTH}) = (0.4)(0.6)2 , P({TTTH}) = (0.4)(0.6)3 , . . . .
Since {H}, {TH}, {TTH}, {TTTH}, {TTTTH}, . . . are mutually
disjoint and
S = {H} ∪ {TH} ∪ {TTH} ∪ {TTTH} ∪ {TTTTH} ∪ . . . , we have
Axiomatic Probability
Examples:
2. Consider again the coin tossing example. However, this time we
are interested in getting a Head, i.e. we toss a coin many times
untill we get a Head. Then S = {H, TH, TTH, TTTH, TTTTH,
. . . }.
If P({H}) = 0.4 then P({T }) = 0.6, P({TH}) = (0.4)0.6,
P({TTH}) = (0.4)(0.6)2 , P({TTTH}) = (0.4)(0.6)3 , . . . .
Since {H}, {TH}, {TTH}, {TTTH}, {TTTTH}, . . . are mutually
disjoint and
S = {H} ∪ {TH} ∪ {TTH} ∪ {TTTH} ∪ {TTTTH} ∪ . . . , we have
1 = 0.4 + (0.4)(0.6) + (0.4)(0.6)2 + (0.4)(0.6)3 + · · ·
Axiomatic Probability
More Probability Properties
Axiomatic Probability
More Probability Properties
Proposition
For any event A, P(A) + P(A0 ) = 1, from which P(A) = 1 − P(A0 ).
Axiomatic Probability
More Probability Properties
Proposition
For any event A, P(A) + P(A0 ) = 1, from which P(A) = 1 − P(A0 ).
Example 2.13
Consider a system of five identical components connected in series,
as illustrated below.
Denote a component failure by F and success by S. Let A be the
event that the system fails. For A to occur, at least one of the
individual components must fail. If we know P({F }) = 0.1, then
what is P(A)?
Axiomatic Probability
Proposition
For any event A, P(A) ≤ 1 .
Axiomatic Probability
Proposition
For any event A, P(A) ≤ 1 .
Proposition
For any two events A and B,
P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
Axiomatic Probability
Proposition
For any event A, P(A) ≤ 1 .
Proposition
For any two events A and B,
P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
A Venn Diagram proof:
Axiomatic Probability
Proposition
For any event A, P(A) ≤ 1 .
Proposition
For any two events A and B,
P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
A Venn Diagram proof:
=
+
Axiomatic Probability
Example 2.14
In a certain residential suburb, 60% of all households subscribe to
the metropolitan newspaper published in a nearby city, 80%
subscribe to the local paper, and 50% of all households subscribe
to both papers. If a househlld is selected at random, what is the
probability that it subscribes to (1)at least one of the two
newspapers and (2) exactly one of the two newspapers?
Axiomatic Probability
Proposition
For any three events A, B, and C ,
P(A ∪ B ∪ C ) =P(A) + P(B) + P(C )
− P(A ∩ B) − P(B ∩ C ) − P(C ∩ A)
+ P(A ∩ B ∩ C )
A Venn Diagram interpretation:
Determining Probabilities
In Probability, our main focus is to determine the probabilities for
all possible events. However, some prior knowledge about the
sample space is available. (While in Statistics, the prior knowledge
is unavailable and we want to find it.)
Determining Probabilities
In Probability, our main focus is to determine the probabilities for
all possible events. However, some prior knowledge about the
sample space is available. (While in Statistics, the prior knowledge
is unavailable and we want to find it.)
For a sample space that is either finite or “countably infinite”
(meaning the outcomes can be listed in an infinite sequence), let
E1 , E2 , . . . denote all the simple events. If we know P(Ei ) for each
i, then for any event A,
X
P(A) =
P(Ei )
all Ei s that are in A
Determining Probabilities
In Probability, our main focus is to determine the probabilities for
all possible events. However, some prior knowledge about the
sample space is available. (While in Statistics, the prior knowledge
is unavailable and we want to find it.)
For a sample space that is either finite or “countably infinite”
(meaning the outcomes can be listed in an infinite sequence), let
E1 , E2 , . . . denote all the simple events. If we know P(Ei ) for each
i, then for any event A,
X
P(A) =
P(Ei )
all Ei s that are in A
Here the knowledge for P(Ei ) is given and we want to find P(A).
(In statistics, we want to find the knowledge about P(Ei ).)
Determining Probabilities
Example 2.15:
Determining Probabilities
Example 2.15:
During off-peak hours a commuter train has five cars. Suppose a
commuter is twice as likely to select the middle car(#3) as to
select either adjacent car(#2 or #4)(1) , and is twice as likely to
select either adjacent car as to select either end car(#1 or #5)(2) .
What is the probability for the commuter to select one of the three
middle cars?
Determining Probabilities
Example 2.15:
During off-peak hours a commuter train has five cars. Suppose a
commuter is twice as likely to select the middle car(#3) as to
select either adjacent car(#2 or #4)(1) , and is twice as likely to
select either adjacent car as to select either end car(#1 or #5)(2) .
What is the probability for the commuter to select one of the three
middle cars?
First we need to determine the probabilities for all the simple
events:
Determining Probabilities
Example 2.15:
During off-peak hours a commuter train has five cars. Suppose a
commuter is twice as likely to select the middle car(#3) as to
select either adjacent car(#2 or #4)(1) , and is twice as likely to
select either adjacent car as to select either end car(#1 or #5)(2) .
What is the probability for the commuter to select one of the three
middle cars?
First we need to determine the probabilities for all the simple
events:
Let pi = P({car i is selected}) = P(Ei ).
Determining Probabilities
Example 2.15:
During off-peak hours a commuter train has five cars. Suppose a
commuter is twice as likely to select the middle car(#3) as to
select either adjacent car(#2 or #4)(1) , and is twice as likely to
select either adjacent car as to select either end car(#1 or #5)(2) .
What is the probability for the commuter to select one of the three
middle cars?
First we need to determine the probabilities for all the simple
events:
Let pi = P({car i is selected}) = P(Ei ).
Then condition (1) tells us p3 = 2p2 = 2p4 ,
Determining Probabilities
Example 2.15:
During off-peak hours a commuter train has five cars. Suppose a
commuter is twice as likely to select the middle car(#3) as to
select either adjacent car(#2 or #4)(1) , and is twice as likely to
select either adjacent car as to select either end car(#1 or #5)(2) .
What is the probability for the commuter to select one of the three
middle cars?
First we need to determine the probabilities for all the simple
events:
Let pi = P({car i is selected}) = P(Ei ).
Then condition (1) tells us p3 = 2p2 = 2p4 , and condition (2) tells
us p2 = 2p1 = 2p5 = p4 .
Determining Probabilities
Example 2.15:
During off-peak hours a commuter train has five cars. Suppose a
commuter is twice as likely to select the middle car(#3) as to
select either adjacent car(#2 or #4)(1) , and is twice as likely to
select either adjacent car as to select either end car(#1 or #5)(2) .
What is the probability for the commuter to select one of the three
middle cars?
First we need to determine the probabilities for all the simple
events:
Let pi = P({car i is selected}) = P(Ei ).
Then condition (1) tells us p3 = 2p2 = 2p4 , and condition (2) tells
us p2 = 2p1 = 2p5 = p4 .
By Axiom 2 and 3,
1=
5
X
i=1
P(Ei ) = p1 + 2p1 + 4p1 + 2p1 + p1 = 10p1
Determining Probabilities
Example 2.15 (continued):
SincePp3 = 2p2 = 2p4 , p2 = 2p1 = 2p5 = p4 and
1 = 5i=1 P(Ei ) = p1 + 2p1 + 4p1 + 2p1 + p1 = 10p1 ,
Determining Probabilities
Example 2.15 (continued):
SincePp3 = 2p2 = 2p4 , p2 = 2p1 = 2p5 = p4 and
1 = 5i=1 P(Ei ) = p1 + 2p1 + 4p1 + 2p1 + p1 = 10p1 ,
we have p1 = 0.1, p2 = 0.2, p3 = 0.4, p = 0.2 and p5 = 0.1.
Determining Probabilities
Example 2.15 (continued):
SincePp3 = 2p2 = 2p4 , p2 = 2p1 = 2p5 = p4 and
1 = 5i=1 P(Ei ) = p1 + 2p1 + 4p1 + 2p1 + p1 = 10p1 ,
we have p1 = 0.1, p2 = 0.2, p3 = 0.4, p = 0.2 and p5 = 0.1.
Furthermore if A = {one of the three middle cars is selected}, then
P(A) = P(E2 ) + P(E3 ) + P(E4 ) = p2 + p3 + p4 = 0.8.