3 Components of Probability
Theory
• Sample space – space of all possible
outcomes of random experiment
• Events – subsets of sample space or
collections of possible outcomes.
• Probability measure – assigns a
numerical value between 0 and 1 to each
outcome representing likelihood of
outcome.
IE 3521: Probability, Statistics, and Reliability
Sample Space
• Sample space : all possible outcomes in
experiment of interest, denoted by S.
• Find the sample space for the following
experiments
• Rolling a six sided die
• Flipping a fair coin
• Drawing a card from a standard deck
of playing cards
IE 3521: Probability, Statistics, and Reliability
Events
• Collections of possible outcomes.
• For example, if the experiment is rolling a
die then S={1,2,3,4,5,6}.
• Possible events are
• A = {2,4,6}, B={1,3,5}, C = {1}, …
• Aside: How many possible events are
there?
IE 3521: Probability, Statistics, and Reliability
Disjointness
• Suppose we roll a conventional die;
consider two events
• Event A: {1,3,5}
• Event B: {2,4,6}
• Events A and B are disjoint as they have
no common elements
IE 3521: Probability, Statistics, and Reliability
Set operations
• The union of two sets A and B is written A È B;
consists of all elements in A, B, or both
S
A
IE 3521: Probability, Statistics, and Reliability
B
Set operations
• The intersection of two sets A and B is
written A Ç B; consists of the elements
common to A and B
S
A
IE 3521: Probability, Statistics, and Reliability
B
Set operations
• The complement of a set A in set S is
written Ac; consists of the elements in S
that are not in A
S
A
IE 3521: Probability, Statistics, and Reliability
Example
• Suppose we are rolling a six-sided die,
and hence S = {1,2,3,4,5,6}
• Let A = {1,2,3,4} and B = {1,6}
Then:
• A È B = {1,2,3,4,6}
• A Ç B = {1}
• Ac = {5,6}
• Bc = {2,3,4,5}
IE 3521: Probability, Statistics, and Reliability
Venn Diagram
1
6
IE 3521: Probability, Statistics, and Reliability
2
3
5
4
A couple of comments
• The sets A and B are disjoint if and only if
A Ç B = Æ (the empty set)
• By construction, A È Ac = S
IE 3521: Probability, Statistics, and Reliability
Probability of an event
• Probability: a quantity that characterizes how
frequently an event occurs, in the limit of
infinitely many trials
Number of experiments favoring A
n→∞
Total number of experiments
Pr(A) = lim
Here n is the number of experiments we
perform
IE 3521: Probability, Statistics, and Reliability
Probability of an event
The probability of an event satisfies three
axioms:
1. 0 ≤ Pr(A) ≤ 1
2. Pr(S) = 1 (any outcome must lie in the
sample space)
3. If A and B are mutually exclusive (disjoint)
events, then Pr(A È B) = Pr(A) + Pr(B)
3a) If A1 ,…, An are mutually exclusive
events, then
Pr(A1 È … È An) = Pr(A1) + ··· + Pr(An)
IE 3521: Probability, Statistics, and Reliability
Probability of an event
•
If event A = {O1, …, Ok }, a collection of
outcomes, then what is P(A)?
• Hint: Remember that outcomes are disjoint.
• Example: We roll a fair six-sided die and
A = {roll a 2 or 3}={2,3}, find P(A)?
IE 3521: Probability, Statistics, and Reliability
Probability of an event
• If event A = {O1, …, Ok }, then what is P(A)?
• Hint: Remember that outcomes are disjoint.
• Answer:
P(A) = P(O1 ) + … P(Ok )
• Example: We roll a fair six-sided die and
A = {roll a 2 or 3}={2,3}, find P(A)?
• P(A) = P({2}) + P({3}) = 1/6 + 1/6 = 2/6 = 1/3
• Why is P({2})=1/6???
IE 3521: Probability, Statistics, and Reliability
Probability and Compliments
•
What is P(A)+P(Ac)=?
IE 3521: Probability, Statistics, and Reliability
Probability of non-disjoint events
What if A and B are not mutually
exclusive? We don’t have
Pr(A È B) = Pr(A) + Pr(B)
S
A
IE 3521: Probability, Statistics, and Reliability
B
Probability of non-disjoint events
• Set C = A Ç B
• Create A*and B* such that A* , B* , and C
are all mutually exclusive and
A* È B* È C = A È B
S
A*
IE 3521: Probability, Statistics, and Reliability
C
B*
Probability of non-disjoint events
• We now have
Pr(A È B) = Pr(A* È B* È C)
= Pr(A* ) + Pr(B* ) + Pr(C)
S
A*
IE 3521: Probability, Statistics, and Reliability
C
B*
Probability of non-disjoint events
• Note that
Pr(A* ) + Pr(C ) = Pr(A)
Pr(B* ) + Pr(C ) = Pr(B)
• Hence,
Pr(A È B) = [Pr(A) – Pr(C)] + [Pr(B) – Pr(C)] + Pr(C)
Pr(A È B) = Pr(A) + Pr(B) – Pr(A Ç B)
S
A*
IE 3521: Probability, Statistics, and Reliability
C
B*
Probability of non-disjoint events
The preceding formula also holds in the
case that A and B are disjoint, in which
case A Ç B = Æ.
IE 3521: Probability, Statistics, and Reliability
Example
• Suppose we are rolling a six-sided die,
and hence S = {1,2,3,4,5,6}
• Let A = {1,2,3,4} and B = {1,6}
What is Pr(A È B)?
IE 3521: Probability, Statistics, and Reliability
Example
• Suppose we are rolling a six-sided die,
and hence S = {1,2,3,4,5,6}
• Let A = {1,2,3,4} and B = {1,6}
What is Pr(A È B)?
• By the preceding formula,
Pr(A È B) = Pr(A) + Pr(B) – Pr(A Ç B)
So,
Pr(A È B) = 4/6+ 2/6 – 1/6 = 5/6
IE 3521: Probability, Statistics, and Reliability
Example
• Suppose we are rolling two six-sided die,
and hence S =
(1,1)
(1,2)
(1,3)
(1,4)
(1,5)
(1,6)
(2,1)
(2,2)
(2,3)
(2,4)
(2,5)
(2,6)
(3,1)
(3,2)
(3,3)
(3,4)
(3,5)
(3,6)
(4,1)
(4,2)
(4,3)
(4,4)
(4,5)
(4,6)
(5,1)
(5,2)
(5,3)
(5,4)
(5,5)
(5,6)
(6,1)
(6,2)
(6,3)
(6,4)
(6,5)
(6,6)
(all are equally likely)
IE 3521: Probability, Statistics, and Reliability
Example
• What is the likelihood of rolling at least
one 2?
IE 3521: Probability, Statistics, and Reliability
Example
• What is the likelihood of rolling at least
one 2?
(1,1)
(1,2)
(1,3)
(1,4)
(1,5)
(1,6)
(2,1)
(2,2)
(2,3)
(2,4)
(2,5)
(2,6)
(3,1)
(3,2)
(3,3)
(3,4)
(3,5)
(3,6)
(4,1)
(4,2)
(4,3)
(4,4)
(4,5)
(4,6)
(5,1)
(5,2)
(5,3)
(5,4)
(5,5)
(5,6)
(6,1)
(6,2)
(6,3)
(6,4)
(6,5)
(6,6)
11 outcomes with a 2
= 11/ 36
36 possible outcomes
IE 3521: Probability, Statistics, and Reliability
A Useful Tool
• For any two events A and B we can
always write
P(A) = P(A  B) + P(A  B )
C
IE 3521: Probability, Statistics, and Reliability
Counting Techniques
• Sometimes all the elements of an event A are
equally likely to occur
• Randomly selecting a password of 8
characters, failure of 2 identical computers in
a network of 10, etc.
• We can determine probabilities of these events
by enumerating all outcomes.
• We will discuss three methods for counting.
IE 3521: Probability, Statistics, and Reliability