10.1
Equally Likely Outcomes Approach
Probability is the result of mapping an event into a number that in some sense
indicates how likely or often the event will occur. One way to do that is to use the
notion of equally likely outcomes.
Define the sample space for a chance experiment as the set of all possible distinct
outcomes. An event is a subset of the sample space. Let nA be the number ways
event A can occur in the sample space where n total equally likely outcomes are
possible. Define the probability of event A occurring as:
n
P ( A) =
n
Example:
Consider a toss of two dice where each side is equally likely to land facing up.
Illustrate the sample space for a single toss of 2 die. Compute the probability of the
following events: The sum of the two dice, Sd, equals 1,2,3,4,5,6,7,8,9,10,11,12, and
13.
Show: P(Sd=1) = 0, P(Sd=2) = 1/36, P(Sd=3) = 2/36, P(Sd=4) = 3/36, P(Sd=5) = 4/36,
P(Sd=6) = 5/36, P(Sd=7) = 6/36, P(Sd=8) = 5/36, P(Sd=9) = 4/36, P(Sd=10) = 3/36,
P(Sd=11) = 2/36, P(Sd=12) = 1/36, P(Sd=13) = 0
A
10.2
Relative Frequency Approach to Probability
Another way to create a probability is to use they notion of relative frequency.
Let nA be the number of times event A occurs out of n trials or experiments. Define
the probability of event A occurring as:
n
P ( A) = lim
n
Example:
Generate uniform random numbers between 0 and 1, and compute the probability
of the number being greater than .75 using relative frequency. Plot result as a
function of number of trials n.
A
n →∞
0.45
0.4
0.35
Relative Frequency
trials = 500;
for n=1:trials
x = rand(1,n);
c = find(x > .75);
na = length(c);
p(n) = na/n;
end
plot([1:trials],p,’w’)
xlabel(‘Number of trials’)
ylabel(‘Relative Frequency’)
0.3
0.25
0.2
0.15
0.1
0.05
0
0
100
200
300
N u m b e r o f t ria ls
400
10.3
Axioms of Probability
Consider the examples of events in a sample space S described with a Venn diagram
S
A
B
D
C
Events A and B are mutually exclusive,
since no single outcome can satisfy both
events. Event C and D are not mutually
exclusive, since an outcome exits that can
satisfy both events.
Events are mutually exclusive iff the occurrence of one event implies the other
events did not occur.
If the defined probabilities meet the conditions in the axioms below, then a selfconsistent theory of probability can be built.
For a given event A in sample space S:
1. P ( A) ≥ 0
2. P ( S ) = 1
3. If A and B are mutually exclusive events then P ( A ∪ B) = P ( A) + P ( B)
10.4
Example: Demonstrate that the equally likely outcomes approach satisfy the axioms
n
(i.e. the probability of event A occurring is P ( A) = ).
n
1. P ( A) ≥ 0
2. P ( S ) = 1
3. If A and B are mutually exclusive events then P ( A ∪ B) = P ( A) + P ( B)
A
For the two dice toss, describe two events that are mutually exclusive. What is the
probability the either event will occur? Describe two events that are not mutually
exclusive. What is the probability that either event will occur?
1
Die 2
Set Notation
1
2
3
4
5
6
Die 1
2
3
4
5
6
10.5
Sets are collections of objects referred to as elements denoted by:
S = {σ ,σ ,Lσ } , σ ∈ S (each element belongs to its set).
Let B = {σ ,σ } , then B ⊂ S (B is a subset of S).
1
2
n
1
i
2
Set operations:
A ∪ B (A union with B, in probability this reads A or B) A + B
A ∩ B (A intersection with B, in probability this reads A and B) AB
For sample space S (the universal set) and A ⊂ S , then Ac (or A ) is the complement
of A which is the set of all elements in S that are not in A.
A partition of S is a collection of mutually exclusive and collectively exhaustive
subsets.
A Cartesian product of two sets A × B is a new set C whose elements are all possible
pairs of elements in A and B.
De Morgan's laws described the following set relationships:
A ∪ B = A I B and A ∩ B = A ∪ B
Derived Probability Relationships
10.6
1. P ( A ) = 1 − P ( A)
2. P (∅ ) = 0
3. P ( A ∪ B ) = P ( A) + P ( B ) − P ( A ∩ B )
c
Consider a flip of 3 fair coins.
a) Compute the probability of at least 2 heads occurring
b) Compute the probability of at least 1 tail occurring
c) Compute the probability that at least 2 heads or at least 1 tail occurs
d) Compute the probability that at least 2 heads and at least 1 tail occurs
e) Compute the probability that at most 1 head occurs
f) Compute the probability that at least 2 tails occur
g) Compute the probability that at least 2 heads and 2 tails occur
Show a) P = .5, b) P=.875, c) P=1, d) P = .375, e) P=.5, f) P=.5, g) P=0
Conditional Probabilities
10.7
In some cases the occurrence of one event (or knowledge that one event has
occurred) may increase or decrease the probability that another event has or will
occur. This relationship between events is modeled with conditional probabilities.
The conditional probability for event A given event B has occurred, where P(A) and
P(B) > 0, is defined by:
P( A ∩ B)
P( A B) =
P( B)
and the conditional probability for event B given event A has occurred is defined by:
P( A ∩ B)
P ( B A) =
P( A)
The above relationships can be used to show:
P ( A ∩ B ) = P ( B A) P( A) = P( A B) P( B)
Use a Venn Diagram to illustrate the above relationships.
Statistical Independence
10.8
In some cases the occurrence of one event will have no influence on how likely
another event is to occur. This relationship is referred to statistical independence.
Events A and B are statistically independent iff P ( A ∩ B ) = P ( A) P ( B ) .
Example: Show that if events A and B are statistically independent, then
P ( A B ) = P ( A) and P ( B A) = P ( B ) .
Example: Let u1 and u2 denote two independently generated uniform random
numbers between 0 and 1. Given events:
A = {0 < u ≤.25} , B = {{.25 < u ≤.75} ∩ {.25 < u ≤.75}} , C = {.75 < u ≤ 1} , and
D = {u < u } .
1
1
1
2
2
2
Determine which events are statistically independent and Show:
P(A) = .25, P(B) = .25, P(C) = .25, P(D) = .5, P ( A C ) =.25 , P ( B C ) = 0 , P ( D C ) =.125 ,
P ( A D) =.0625, and P ( D B ) =.5
Total Probability
10.9
In some cases an event is influenced in varying degrees by several other mutually
exclusive events. For example the event that a voltage level measured from a radar
receiver (proportional to the energy in an electromagnetic echo) exceeds some
threshold, is dependent on whether a target is present (event 1) or no target is
present (event 2). To compute the total probability the measured voltage exceeds
the threshold, both events 1 and 2 must be considered.
Let events A1, A2, A3, … Am partition the sample space S. Consider an arbitrary
event B in S. Show (theorem of total probability)
P (B ) = ∑ P (B ∩ A ) = ∑ P (B A ) P ( A )
m
i =1
m
i
i =1
i
i
Example: Assume that a voltage measured at a radar receiver will cross a threshold
of 1 volt with probability 10-6 when no target is present. For 1 week radar
measurements were taken continually, while the target was present 1% of the time
and the received voltage crossed the threshold .004 % of the time. From these data,
estimate the probability that receiver voltage crosses the 1 volt threshold when the
target is present.
Show P(threshold crossed | Target Present) = .0039
Bayes' Theorem
10.10
In experiments and calibration studies the probability of the measurement event
(i.e. crossing a threshold) is estimated given the event of interest has occurred. In
the application of the system, however, it is desired to know what the probability of
the event of interest is, given the measurement. The latter can be computed from
the former via Bayes' theorem.
Let events A1, A2, A3, … Am partition the sample space S. Consider an arbitrary
event B in S. Show (Bayes' Theorem)
P( B A ) P( A )
P( B A ) P( A )
=
P( A B) =
P( B)
∑ P( B A ) P( A )
i
i
i
i
i
m
k =1
k
k
From the previous example compute the probability that a target is present given
the receiver voltage exceeded 1 volt. Assume now that the probability of a target
appearing in the actual application is .001.
Show P( target present | threshold crossed ) = .7961
Counting Techniques:
10.11
Filling slots: How many different ways can N slots be filled with M possible objects
for each slot?
Total = MN
Permutations: How many different ways can M objects be selected (where order of
the selection is considered) from N irreplaceable objects.
N!
Total =
( N − M )!
Combinations: How many different ways can M objects be selected (where order of
the selection is not considered) from N irreplaceable objects.
⎛ N⎞
N!
=⎜ ⎟
Total =
( N − M )! M ! ⎝ M ⎠
In a sequence of n independent experiments where only 2 outcomes are possible
(success, failure), and p = the probability of success, the probability of k successes is
⎛ n⎞
given by (the binomial distribution) ⎜ ⎟ p (1 − p)
⎝ k⎠
k
n− k