Lecture note 4
Probabilities
©
Topics to be covered
Basic set notations
Basic properties of probabilities
Bivariate probabilities
Conditional probabilities
Statistical independence
Sample Space
Suppose that you roll a die once. There will
be 6 possible outcomes; you may get either 1,
2, 3, 4, 5 or 6. These possible outcomes of
such a random experiment are called the
basic outcomes. The set of all basic outcomes
is called the sample space. The symbol S
will be used to denote the sample space.
Sample Space
- An Example -
What is the sample space for a roll
of a single six-sided die?
S = ｛1, 2, 3, 4, 5, 6｝
An event
A subset of a sample space is called an
event. We usually use a capital letter to
denote an event. Taking the rolling-of-adie as an example, A={1, 2} is an event.
The meaning of an event is important.
A={1, 2} means that this is the event that
“you get either 1 or 2”.
B={1,4,6} is the event that you get either 1,
4 or 6.
Some set notations
-Intersection Suppose you roll a die. Then the sample space is
S={1,2,3,4,5,6}.
 Consider the following two events A={1,3,5} and
B={1,3,6}.
Then , A∩B is defined as an event that consists of the
basic outcomes that are common to both A and B.
A∩B reads “A intersection B”.
Exercise: What is A∩B?
Some set notation
-UnionSuppose you roll a die. Then the sample space is
S={1,2,3,4,5,6}.
Consider the following two events A={1,3,5} and
B={1,3,6}.
A  B is defined as an event that consists of the
basic outcomes that are either in A or B.
A  B reads “A union B”.
Exercise: What is A  B?
Mutually Exclusive Events
If the events A and B have no common basic
outcomes, they are called mutually exclusive and
their intersection A  B is said to be the empty
set indicating that A  B cannot occur.
More generally, the K events E1, E2, . . . , EK
are said to be mutually exclusive if every pair of
them is a pair of mutually exclusive events.
Mutually exclusive events
-Example 1Suppose you roll a die. Then the sample
space is S={1, 2, 3, 4, 5, 6}. Now, consider
the following events. A={1, 2, 3} and B={4,
5, 6}
Then there is no common basic outcome
in event A and B. Therefore, A and B are
mutually exclusive events.
Mutually exclusive events
-Example 2Consider you roll a die. Then the sample
space is S={1, 2, 3, 4, 5, 6}. Now, consider
the following 3 events. E1={1, 2 }, E2={3, 4}
and E3={5,6}
Then there is no common basic outcome
in any pair of events. Therefore, E1, E2
and E3 are mutually exclusive sets.
Venn Diagrams
Venn Diagrams are drawings, usually
using geometric shapes, used to depict
basic concepts in set theory and the
outcomes of random experiments.
Intersection of Events A and B
S
S
A
AB
B
(a) AB is the striped area
A
B
(b) A and B are Mutually Exclusive
Union of events A and B
S
A
B
AB
A  B is the striped area.
Collectively Exhaustive Events
Given the K events E1, E2, . . ., EK in the
sample space S. If E1  E2  . . . EK = S,
these events are said to be collectively
exhaustive.
Collectively Exhaustive Events
-ExampleConsider rolling a die. Then the sample
space is S={1, 2, 3, 4, 5, 6}. Further consider
the following 3 events. E1={1, 2, 3 }, E2={2,
3, 4} and E3={4,5,6}
Then E1, E2 and E3 are collectively
exhaustive events since E1  E2  E3
={1,2,3,4,5,6}=S
Complement
Let A be an event in the sample space S.
The set of basic outcomes of a random
experiment belonging to S but not to A is
called the complement of A and is
denoted by A.
Venn Diagram for the
Complement of Event A
S
A
A
Unions, Intersections, and
Complements
(Example 4.3)
A die is rolled. Let A be the event “Number you get is even”
and B be the event “Number you get is at least 4.” Then
A = [2, 4, 6] and
B = [4, 5, 6]
Find A and B
Find A  B
Find A  B
Find A  A
Classical Probability
The classical definition of probability is the
proportion of times that an event will occur,
assuming that all outcomes in a sample space
are equally likely to occur. The probability of
an event is determined by counting the number
of outcomes in the sample space that satisfy the
event and dividing it by the number of
outcomes in the sample space.
Classical Probability
The probability of an event A is
NA
P(A) 
N
where NA is the number of outcomes that satisfy the
condition of event A and N is the total number of
outcomes in the sample space.
Classic Probability
-ExampleA die is rolled. Let A be the event
“Number you get is even”
Then A={2,4,6}. Therefore,
NA=3, and N=6. Therefore,
P(A)=3/6=0.5
Probability Postulates
Let S denote the sample space of a random experiment, Oi,
the basic outcomes, and A, an event. For each event A
of the sample space S, we assume that a number P(A)
is defined and we have the postulates
1. If A is any event in the sample space S
0  P( A)  1
2.
Let A be an event in S, and let Oi denote the basic
outcomes. Then
P( A)   P(Oi )
A
where the notation implies that the summation
extends over all the basic outcomes in A.
3. P(S) = 1
Probability Postulate
-Example for Postulate 2Consider a roll of a die. Let A be the event “Number
you get is even”. Then, A={2, 4, 6} and P(A)=0.5.
The notation in the postulate 2 means, {O1}={2},
{O2}={4} and {O3}={6}, and
1 1 1
P( A)   P(Oi )  P(O1 )  P(O2 )  P(O3 )     0.5
6 6 6
A
Bivariate Probabilities
Bivariate probabilities are the intersection
probabilities of two distinct sets of events.
Bivariate Probabilities
-ExampleConsider that you are an advisor for a
particular TV show. You may want to
know how often people watch the show,
and probably the income level of the
audience. You can consider the following
2 distinct sets of events about the
potential audiences.
Next Slide
Bivariate Probabilities
-Example- Contd
The first set of events is the following.
A1={Regular watcher}
A2={Occasional watcher}
A3={Never watched}
The second set of the events is
B1={High income}
B2={Middle income}
B3={Low income}
Bivariate Probabilities,
-Example- contd
Then, the Bivariate probabilities of the two sets
of events, A1, A2, A3 and B1, B2, B3 can be
represented by the following table.
B1
B2
B3
A1
P(A1B1)
P(A1B2)
P(A1B3)
A2
P(A2B1)
P(A2B2)
P(A2B3)
A3
P(A3B1)
P(A3B2)
P(A3B3)
Joint probabilities
In the context of bivariate probabilities, the
intersection probabilities P(Ai  Bj) are called joint
probabilities.
Joint Probabilities for the Television
Viewing and Income Example
Viewing
Frequency
High
Income
B1
Regular
A1
Occasional
A2
Never
A3
Middle
Income
B2
Low
Income
B3
0.04
0.13
0.04
0.10
0.11
0.06
0.13
0.17
0.22
Bivariate Probabilities
-TV viewer example, contdOften we also want to know the
probability that a person is a frequent
watcher of the program regardless of
his/her income, P(A1), or the probability
that a person has high income regardless
of his/her viewer status, P(B1).
Such probabilities are called the marginal
probabilities.
Marginal Probabilities
In the context of bivariate probabilities, the
probabilities for individual events P(Ai) and P(Bj)
are called marginal probabilities.
They can be computed by summing the
corresponding row or column.
Exercise: Compute the following
marginal probabilities
Viewing
Frequency
High
Income
B1
Middle
Income
Low
Income
B3
Marginal
Probabili
ty
Regular
0.04
0.13
0.04
0.10
0.11
0.06
P(A1)=
0.21
P(A2)=
0.27
Never
A3
0.13
0.17
0.22
P(A3)=
0.52
Marginal
Probabilitie
s
P(B1)=
P(B2)=
P(B3)=
Total=
A1
Occasional
A2
0.27
B2
0.41
0.32
1
Bivariate Probabilities and the
tree diagram
We have represented the Bivariate
Probabilities using a table.
Often the are represented by a tree
diagram.
Example is in the next slide
Tree Diagram
P(A1  B1) = .04
P(A1  B2) = .13
P(A1  B3) = .04
P(A2  B1) = .10
P(S) = 1
P(A2) = .27
P(A2  B2) = .11
P(A2  B3) = .06
P(A3 B1) = .13
P(A3  B2) = .17
P(A3  B3) = .22
Probability Rules
Conditional Probability:
Let A and B be two events. The conditional probability
of event A, given that event B has occurred, is denoted by
the symbol P(A|B) and is found to be:
P( A  B)
P( A | B) 
P( B)
provided that P(B) > 0.
Probability Rules
Conditional Probability:
Let A and B be two events. The conditional probability
of event B, given that event A has occurred, is denoted by
the symbol P(B|A) and is found to be:
P( A  B)
P( B | A) 
P( A)
provided that P(A) > 0.
Conditional Probability
-ExerciseContinue using the TV viewer example.
Suppose that a person is in the“high
income” range. Given this information,
what is the probability that this person is
a “occasional viewer” of the program?
Answer
This problem can be formulated in the following
way.
The event “the person has high income” is B1,
and the event “the person is an occasional
viewer of the problem” is A2.
The probability that that the person is an
occasional viewer given the information that the
person has “high income” can be written as
P(A2|B1)=P(A2  B1)/P(B1)=0.1/0.27=0.37
Probability Rules
The Multiplication Rule of Probabilities:
Let A and B be two events. The probability of
their intersection can be derived from the
conditional probability as
P( A  B)  P( A | B) P( B)
Also,
P( A  B)  P( B | A) P( A)
Multiplication rules of the
probability
-ExampleWhen we describe a situation that
involves sequential decision making,
multiplication rules become convenient.
Consider the following investment
problem. See next page.
Multiplication rules of the
probability
-Example, contdA company is considering to invest in a
project. Before investing in a full scale
project, the company will undertake a test
marketing. The probability that test
marketing turns out to be successful is 0.6.
Continues to the next slide
If the test marketing is successful, you
may go ahead with the full scale
investment. Given the successful test
marketing result, there will be 0.7
probability that the full scale investment
will generate \100 million, and 0.3
probability that full scale project will
generate \70 million.
Continue to the next slide
If the test marketing turns out to be
“unsuccessful”, you can still go ahead
with the full scale project. However, given
“unsuccessful” test marketing, there will
be only 0.15 probability that the full scale
project generates \70 million, and 0.85
probability that full scale project generates
only $40 million.
We would like to represent this
investment problem using a tree diagram.
First define the following:
A1={Test marketing successful}
A2={Test marketing unsuccessful}
B1={Full scale project generates \100
million}
B2={Full scale project generates \70 million}
B3={Full scale project generates \40 million}
Exercise
Using the tree diagram from the previous slide
find the following probabilities.
1. P(A1 B1)
2.
3.
4.
5.
6.
7.
P(A1 B2)
P(A2 B2)
P(A2 B3)
P(B1)
P(B2)
P(B3)
Statistical Independence
Let A and B be two events. These events are said to be
statistically independent if and only if
P( A  B)  P( A) P( B)
If A and B are statistically independent, from the
multiplication rule, it also follows that
P(A | B)  P(A)
(if P(B)  0)
P(B | A)  P(B)
(if P(A)  0)
More generally, the events E1, E2, . . ., Ek are mutually
statistically independent if and only if
P(E1  E2  EK )  P(E1 ) P(E 2 )P(E K )
Statistical Independence
-Example 1Consider tossing coins twice. Define the
following events.
A1={Get the heads for the first toss}
A2={Get the tails for the first toss}
B1={Get the heads for the second toss}
B2={Get the tails for the second toss}
The Bivariate Probabilities are given in the
table on the next slide.
Statistical Independence
-Example 1, contdBivariate
B1
probabilities
B2
A1
0.25
0.25
A2
0.25
0.25
Exercise
Show that A1 and B1 are statistically
independent.
Statistical Independence
-Example 2A survey was carried out for a
supermarket to classify the customers
according to whether their visits to the
store are frequent or infrequent, and to
whether they often, sometimes, or never
purchase generic products. The table in
the next slide gives the proportions of
people surveyed in each of the six joint
classifications.
Often
Sometimes
purchase purchase
Never
purchase
0.12
0.48
0.19
Visit
0.07
infrequently
0.06
0.08
Visit
frequently
Exercise: Check to see whether the event {visit
frequently} and the event {never purchase} are
statistically independent.
Probability Rules
Let A be an event and A its complement.
The complement rule is:
P( A )  1  P( A)
Probability Rules
The Addition Rule of Probabilities:
Let A and B be two events. The probability
of their union is
P( A  B)  P( A)  P( B)  P( A  B)
Probability Rules
Venn Diagram for Addition Rule
(Figure 4.8)
P( A  B)  P( A)  P( B)  P( A  B)
P(AB)
A
B
=
P(A)
A
P(B)
B
+
A
P(AB)
B
-
A
B
Set operation theorem
De Morgan’s law
A B  A B
and
A B  A B
An application of De Morgan’s law can be found in
Exercise 4.61 (d) of the textbook