Topic 2: Intro to
probability
CEE 11 Spring 2002
Dr. Amelia Regan
These notes draw liberally from the class text, Probability and Statistics for
Engineering and the Sciences by Jay L. Devore, Duxbury 1995 (4th edition)
This topic also draws from Probability Concepts in Engineering Planning and
Design: Volume I - Basic Principles by A. H-S. Ang and W. H.Tang, John
Wiley & Sons, 1975
definitions

The set of all outcomes in a probabilistic problem is collectively a
sample space.
Each of the individual possibilities is a sample point.

An event is defined as a subset of the sample space.

Sample space may be discrete or continuous.


A discrete sample space may be finite or countably infinite.
 Examples: {H,T}, {1,2,3,4,5,6}, {1/2,1/4,1/8,…1/n}
Class Exercise


Problem 3.6, slightly modified.
Starting at a fixed time, each car entering an intersection is
observed to see whether it turns left (L), right (R) or goes straight
ahead (S). The experiment terminates as soon as a car is
observed to turn left.
Let the random variable X be equal to the number of cars
entering the intersection before a car turns left.

What are the possible values for X?

List 5 outcomes and their associated X values.
definitions
If the occurrence of one event precludes the occurrence of
another event the two are said to be mutually exclusive. If two
events A and B are mutually exclusive then:

P(A  B)=0

Two or more events are said to be collectively exhaustive if the
union of all of these events constitute the underlying sample
space.
some basic properties
of probability



For any event A, P(A)  0. (no negative probabilities)
Let S represent the sample space, P(S) = 1
(probabilities sum to one)
if A1, A2, ...,An is a finite collection of mutually exclusive
events, then
n
P( A1  A2  ...  An )   P( Ai )
i 1
(the probability that at least one will occur is the sum of
the individual probabilities of each occurring)
some basic properties
of probability

For any event A
P(A) = 1-P(A)
Illustrate this with a Venn Diagram here
some basic properties
of probability

For any two events A and B
P(A  B) = P(A)+P(B)-P(A  B)
Illustrate this with a Venn Diagram here
some basic properties
of probability

Similarly
P(A  B  C) = P(A)+P(B)+P(C)
-P(A  B)-P(A  C)-P(B  C)
+P(A  B  C)
Illustrate this with a Venn Diagram here
some basic properties
of probability

For any events A and B
________
A  B = A  B de Morgan's Rule
________
P(A  B) = P(A  B)
Illustrate this with a Venn Diagram here
conditional
probability

For any two events A and B with P(B) > 0, the
conditional probability of A given that B has occurred is
defined by
P( A  B)
P( A | B) 
P( B)

Note that the multiplication rule for P(A intersection B)
follows directly
P( A  B)  P( A | B) P( B)
Example

In a game show a contestant is asked to pick from one
of three doors. Behind one of the doors is a new car
which he gets to keep if he selects that door.
1
P ( D1 )  P ( D2 )  P ( D3 ) 
3


The contestant picks door number 1.
One of the other two doors is selected at random and is
opened (say door 2). The car is not behind door 2. Now
what are the chances that the car is behind door number 1?
1
P( D1  D2 )  3  1
P( D1 | D2 ) 


P( D2 )
 2 2
 
 3
The “Monty Hall” Problem


This is actually a well known problem.
In a game show a contestant is asked to pick from one
of three doors. Behind one of the doors is a new car
which he gets to keep if he selects that door.
1
P ( D1 )  P ( D2 )  P ( D3 ) 
3



The contestant picks door number 1.
The host (Monty Hall), who knows where the prize is,
opens a door behind which there is no prize.
The contestant is offered a chance to switch his choice.
Should he switch?
Bayes’ Theorem

The multiplication rule
P( A  B)  P( A | B) P( B)
leads directly to Bayes’ Theorem which is used to
compute posterior probabilities from given prior
probabilities. First we need the law of total probability...
Bayes’ Theorem

Let A1, A2, ..., An be a collection of n mutually exclusive
and collectively exhaustive events. Then for any other
event B for which P(B) > 0, The law of total
probability states that
P( B)  P( B | A1 ) P( A1 )  P( B | A2 ) P( A2 )  ...  P( B | An ) P( An )
n
  P( B | Ai ) P( Ai )
i 1
Bayes’ Theorem
B
A1


A2
…
An
Remember the diagram associated with the law of total
probability
The rectangular region represents the universe which is
divided into non-overlapping areas which together
cover the whole region. The probability that B occurs
the sum of the probabilities that B occurs with any of
the mutually exclusive and collectively exhaustive
events.
Bayes’ Theorem

Let A1, A2, ..., An be a collection of n mutually exclusive
and collectively exhaustive events with P(Ai) > 0 for
i=1,...,n. Then for any other event B for which P(B) > 0
P( Ak  B)
P( B | Ak ) P( Ak )
P( Ak | B) 
 n
k  1, 2,...n
P( B)
 P( B | Ai ) P( Ai )
i 1

The second form follows directly from the first by using
the multiplication rule in the numerator and the law of
total probability in the denominator
Example (law of
total probability)



Assume that 1 in 10,000 adults is inflicted with a rare
disease.
A diagnostic test is developed – the test is 99%
accurate (1% false negatives and 1% false positives)
If a person tests positive – what is the probability she
has the disease? – Less than 10%?
P( D  P)
P( D  P)


P( P)
P( P | D) P( D)  P( P | D ) P( D )
(0.001)(0.99)
0.00099


(0.99)(0.001)  (0.01)(0.999) 0.00999 0.00999
0.00099
 0.09016
0.01098
P( D | P) 
independence

Two events A and B are independent if
P( A | B)  P( A)

They are dependent otherwise
independence
Remember that in general

P( A  B)  P( A | B) P( B)

However, if A and B are independent then
P( A  B)  P( A) P( B)

In fact, A and B are independent if and
only if the above is true
independence

Events A1, A2, ...., An are mutually
independent if for every k (k = 2,3,...,n)
and every subset of indices i1, i2, i3,...,in,
P( Aii  Ai2  ...  Ain )  P( Aii )P( Ai2 )P( Ain )
independence

Question: If two events are mutually
exclusive can they also be independent?
Class Exercise


A construction company has the opportunity to bid on
three jobs (A,B and C). From historical data the
company believes its changes of winning each job is
0.20, 0.50 and 0.10. Its chances of winning all three
jobs is 0.05. The probability that it will win both A and
B is 0.10. For A and C and B and C the probabilities
are 0.07 and 0.06.
Are the events that the construction company is
awarded jobs A, B and C independent? Why or why
not?
Class Exercise


A construction company has the opportunity to bid on
three jobs (A,B and C). From historical data the
company believes its changes of winning each job is
0.20, 0.50 and 0.10. Its chances of winning all three
jobs is 0.05. The probability that it will win both A and
B is 0.10. For A and C and B and C the probabilities
are 0.07 and 0.06.
Calculate the following probabilities:
The company wins none of the jobs
The company wins only job A
The company wins only one of the jobs
Hint -- draw and label a Venn Diagram
Class Exercise


A construction company has the opportunity to bid on
three jobs (A,B and C). From historical data the
company believes its changes of winning each job is
0.20, 0.50 and 0.10. Its chances of winning all three
jobs is 0.05. The probability that it will win both A and
B is 0.10. For A and C and B and C the probabilities
are 0.07 and 0.06.
Calculate the following conditional probabilities:
A given B
C given A
C given A and B