Counting & Basic probabilities Stat 430 Heike Hofmann
advertisement
Counting & Basic
probabilities
Stat 430
Heike Hofmann
Wednesday, August 24, 2011
1
Outline
• Combinatorics (Counting rules)
• Conditional probability
• Bayes’ rule
Wednesday, August 24, 2011
2
Combinatorics
Wednesday, August 24, 2011
3
{1, 1}, {1, 2}
{2, 2}
...
...
{1, 1}, {1, 2} ... {1, 6} .
Ω2
=
.
.
{2,
2}
...
{2,
6}
Alternative
Choices
n1 ways .
Ω2 =
.
Summation Principle
Start
n2 ways
..
..
{6, 6}
Stop
|A|
P|A|
(A) =
,
nk =
ways ,
|Ω|
P (A)
...
•
{1, 6}
{2, 6}
..
.
{6, 6}
|Ω|
If a complex action can be performed using k alternative methods
n1n, n2 , ..., nk
and each method can bendone
in
,
n
,
...,
1 2
ways, there are a totalk of N ways that the
complex action can be performed, with
N = n1N
+n
+1...++nn2k + ... + nk
=2 n
N = n1 · n2 · ... · nk
Wednesday, August 24, 2011
N = n1 · n2 · ... · nk
4
...
...
..
.
{1, 6}
{2, 6}
..
.
{6,6}6}
{1,
Multiplication
Principle
Ω =
2
•
{1, 1}, {1, 2}
{2, 2}
{1, 1}, {1, 2} ...
Sequence of Choices
{2, 2} ... {2, 6}
|A| .
Ω2 =
..
.
P
(A)
=
,
.
.
|Ω|
n
2
ways
n1 ways
k ways
... {6,n6}
Start
Stop
n1 , n2 , ...,|A|
nk
P (A) =
|Ω|
,
If a complex action
consists
series
of n
kkactions and each
N=
n1 +ofn2a +
... +
action can be done in n1 , n2 , ..., nk ways, there are a total of N
ways that the complex action can be performed, with
N ==nn1+· nn2 +· ...
·+nnk
N
...
1
2
k
Wednesday, August 24, 2011
|Ω1 |N==52
n · ·51
n ··50
... ·· n49 =
5
(iii)
for pairwise disjoint events A1 , A2 , A3 , ...
P (A1 ∪ A2 ∪ A3 ∪ ...) = P (A1 ) + P (A2 ) + P (A3 ) + .
Counting Examples
P (Ω) = 1
P (Ā) = 1 − P (A)
two∪dice
B) = P (A) + P (B) − P (A ∩ B)
• rollP (A
Ω1
(1, 1), (1, 2) ... (1, 6)
(2, 1), (2, 2) ... (2, 6)
=
..
..
..
.
.
.
.
.
.
(6, 1), (6, 2) ... (6, 6)
5
Wednesday, August 24, 2011
6
P (A ∪ B) = P (A) + P (B) − P (A ∩ B)
• e.g. roll two
dice
(1, 1),
Ω1 =
Ω2 =
Wednesday, August 24, 2011
(1, 2) ... (1, 6)
(2, 1), (2, 2) ... (2, 6)
..
..
..
..
.
.
.
.
(6, 1), (6, 2) ... (6, 6)
5
{1, 1}, {1, 2}
{2, 2}
|A|
P (A) =
,
|Ω|
... {1, 6}
... {2, 6}
..
..
.
.
{6, 6}
7
P (A ∪ B) = P (A) + P (B) − P (A ∩ B)
• e.g. roll two
dice
(1, 1),
Ω1 =
Ω2 =
Wednesday, August 24, 2011
(1, 2) ... (1, 6)
(2, 1), (2, 2) ... (2, 6)
..
..
..
36
..
.
.
.
.
elements
(6, 1), (6, 2) ... (6, 6)
5
{1, 1}, {1, 2}
{2, 2}
|A|
P (A) =
,
|Ω|
... {1, 6}
... {2, 6}
..
..
.
.
{6, 6}
7
P (A ∪ B) = P (A) + P (B) − P (A ∩ B)
• e.g. roll two
dice
(1, 1),
Ω1 =
Ω2 =
Wednesday, August 24, 2011
(1, 2) ... (1, 6)
(2, 1), (2, 2) ... (2, 6)
..
..
..
36
..
.
.
.
.
elements
(6, 1), (6, 2) ... (6, 6)
5
{1, 1}, {1, 2}
{2, 2}
|A|
P (A) =
,
|Ω|
... {1, 6}
... {2, 6}
21
..
..
.
.
elements
{6, 6}
7
P (A ∪ B) = P (A) + P (B) − P (A ∩ B)
• e.g. roll two
dice
(1, 1),
Ω1 =
order matters
Ω2 =
order does not
matter
Wednesday, August 24, 2011
(1, 2) ... (1, 6)
(2, 1), (2, 2) ... (2, 6)
..
..
..
36
..
.
.
.
.
elements
(6, 1), (6, 2) ... (6, 6)
5
{1, 1}, {1, 2}
{2, 2}
|A|
P (A) =
,
|Ω|
... {1, 6}
... {2, 6}
21
..
..
.
.
elements
{6, 6}
7
• e.g. roll three dice
order matters
order does not
matter
Wednesday, August 24, 2011
8
N = n1 · n2 �
· ... ·�nk
52
dice
|Ω2 | =
=
• e.g. roll three
4
order
matters
|Ω
|
=
52 · 51 · 50 · 49 =
1
|Ω1 | = 6 · 6 · 6 = 216
� �
elements
52
|Ω2 | =
=
4
|Ω | = ... = 55
2
= nk
|Ω1 |does
= 6 not
· 6 · 6 = 216
order
matter
n!
. . . · (n − k + 1) = (n−k)!
��
� 2 | = ... = 55 elements
|Ω
times
k
n
n!
−1
N=
· (k!)
(n − k)!
Wednesday, August 24, 2011
� �
n!
n
=
=
(n − k)!k!
k
8
• pick four cards from stack of 52
order matters
order does not
matter
Wednesday, August 24, 2011
9
N = n1 + n2 + ... + nk
n1 , n2 , ..., nk
cards from stack of 52
• pick four
N = n1 · n2 · ... · nk
order matters
N = n1 + n2 + ... + nk
|Ω1 | = 52 · 51 · 50 · 49 = 6497400
choices
� · ... · n
N = n�
·
n
1 52 2
k
|Ω2 | =
=
4
order|Ω
does
not
52 · 51 · 50 · 49 =
1| =
|Ω1 | = 6 · 6 · 6 = 216
matter
n� = nk
Wednesday, August 24, 2011
�
�
52
= = 55 = 270725
|Ω|Ω
=| ...
2| 2
4
elements
9
Difference between dice &
cards example
• die results are always in {1, 2, 3, 4, 5, 6}
(with replacement)
• once a card is drawn, it is out of the stack
(without replacement)
Wednesday, August 24, 2011
10
Counting Rules
• Goal: determine overall size of sample
space without listing all elements manually
• Side benefit: usually we can find a
mathematical description of the sample
space in the process.
Wednesday, August 24, 2011
11
Combinatorics
Wednesday, August 24, 2011
12
Counting rules
Urn model
n numbered objects
(balls) in a bag (urn)
1
3
2
Wednesday, August 24, 2011
13
Ordered samples with
replacement
|Ω
• Urn model:
pick ball from urn, write down number,
put ball back,repeat k times
•
|Ω1 | =
|Ω1 |
|Ω
Sequential setup fits Multiplication principle:
k
N =n
·
n
·
.
.
.
·
n
=
n
�
��
�
k times
Wednesday, August 24, 2011
N = n · (n − 1) · . . . · (n − k + 1) =
�
��
�
14
|Ω1 | = 52 · 51 · 50
� �
Ordered samples
52
|Ω | =
4
without replacement
2
|Ω1 | = 6 · 6 · 6 =
• Urn model:
pick ball from urn, write down number,
|Ω2 | = ... =
put ball not back,repeat k times
•
k
N =n
·
n
·
.
.
.
·
n
=
n
�
��
�
Sequential
setup fits Multiplication principle:
k times
n!
N = n · (n − 1) · . . . · (n − k + 1) = (n−k)!
�
��
�
k times
Wednesday, August 24, 2011
n!
N=
· (k!)−1 =
(n − k)!
(n
15
|Ω1 | = 52 · 51 · 50
� �
Ordered samples
52
|Ω | =
4
without replacement
2
|Ω1 | = 6 · 6 · 6 =
• Urn model:
pick ball from urn, write down number,
|Ω2 | = ... =
put ball not back,repeat k times
•
k
N =n
·
n
·
.
.
.
·
n
=
n
�
��
�
Sequential
setup fits Multiplication principle:
k times
n!
N = n · (n − 1) · . . . · (n − k + 1) = (n−k)!
�
��
�
k times
n!number −1
this is also called the permutation
N=
· (k!) =
(n − k)!
(n
Wednesday, August 24, 2011
15
� �
52
|Ω2 | =
=
4
Unordered samples
without|Ω replacement
| = 6 · 6 · 6 = 216
1
•
|Ω2 | = ... = 55
Urn model:
n ��
· . . . · n� =pick
nk k balls from urn at once
k times
•
n!
(n − 1) · . . .Trick:
· (n −pick
k +balls
1) =ordered
��
� (n−k)! w/o replacement, then
remove order:
k times
� �
n!
n!
n
−1
N=
· (k!) =
=
(n − k)!
(n − k)!k!
k
�
�
n+k−1
N=
Wednesday, August 24, 2011
16
4
Unordered
|Ω | = 6 · 6 samples
· 6 = 216
with replacement
1
model:
• Urn
k
|Ω2 | = ... = 55
�. . . · n� = npick ball from urn, write down number,
put ball back,repeat k times,
mes
n! #times each ball is sampled:
keep
record
of
− 1) · . . . · (n
− only
k + 1)
= (n−k)!
��
�
k times
1 1 2 4 4 4
7 7 � �
n!
n!
n
−1
Nk =
·
(k!)
=
=
balls,
(n(n-1)
− k)!lines, i.e. k+n-1(nobjects
− k)!k!
k
Trick: place k balls �
among k+n-1�places
n+k−1
N=
k
•
Wednesday, August 24, 2011
17
Example
• How many ways are there to uniquely
rearrange the letters of
MISSISSIPPI
Wednesday, August 24, 2011
18
Independence
Wednesday, August 24, 2011
19
|Ω2 | = ... = 55
n� = nk
Probability
concepts
· . . . · (n − k + 1) =
��
k times
�
n!
(n−k)!
� �
n!
n!
n
Independence/Dependency
−1
N=
· (k!) =
=
(n − k)!
(n − k)!k!
k
If occurrence�of event A changes
the
�
n + kB,−the
1 events are
probability of event
N=
dependent:
k
•
P (A ∩ B)
P (B | A) =
P (A)
•
if P(A) ≠ 0
P (A ∩ B) = P (A) · P (B)
Wednesday, August 24, 2011
20
��
�
k times
� �
n!
n!
n
−1
N=
· (k!) =
=
(n − k)!
(n − k)!k!
k
�
�
n+k−1
N=
k
Independence
•
P (A ∩ B)
P (B | A) =
P (A)
Events A and B are independent, if
P (A ∩ B) = P (A) · P (B)
6
Wednesday, August 24, 2011
21
System reliability
• Serial System:
system works, if all of its components are
working
•
•
• Parallel System:
system works if at least one component is
working
•
Wednesday, August 24, 2011
•
22
Bayes’ Rule
Wednesday, August 24, 2011
23
Bayes’ Rule
• Definition: cover
A set of events B1 , B2 , B3 , ... is a cover, if
the sets are pairwise disjoint and
et B1 , . . . , Bk is a cover of the sample space Ω, we can compute
exhaustive (i.e. they cover the sample
t A by (cf. fig.??):
space)
P (A) =
k
�
i=1
Wednesday, August 24, 2011
P (Bi ) · P (A|Bi ).
24
every possible case of the sample space, like pieces of a jig-saw puzzle e.g.
Compare with diagram 1.4.
Bayes’ Rule
The boxes from the last example, B1 , B2 , and B3 , are a cover of the sample sp
B1 , B2 , B3 , ...
Theorem 1.7.2 (Total Probability)
If the set B1 , . . . , Bk is a cover of the sample space Ω, we can compute the
(cf. fig.1.5):
a cover
of the sample space Ω, we can compu
probability
• isTotal
1 , . . . , Bk
Let B1 , B2 , B3 , ... be a cover of �
the
sample
by (cf. fig.??):
k
P (A) =
P
(Bi ) · P (A|Bi ).
space, then P(A) can be computed
as
i=1
is a cover of the sample space
Ω,
we
can
compute
the
probab
k
�
g.??):
P (A)
= P (BPi)·P(B
Pcould
(A|B
Note: Instead
of writing
(A|B
have
written P (A∩Bi ) - this
i )i )· we
i ).
probability cf. def. 1.5.1.
P (A) =
k
�
i=1
i=1
P (Bi ) · P (A|Bi ).
P (Bj ∩ A)
P (A|Bj ) · P (Bj )
=
= �k
P (A)
i=1 P (A|Bi ) · P (Bi )
for all j and
∩ A)
P (A|Bj ) · P (Bj )
for all j and ∅ =
� A ⊂ Ω.
= �k
24,Figure
2011
25
A) Wednesday, Augusti=1
1.5: The
P (A|B
· P (Bi )of event A is put together as sum of the probabilities
i ) probability
If the set B1 , . . . , Bk is a cover of the sample space Ω
for an event A by (cf. fig.??):
Bayes’ Rule
• Bayes Theorem
P (A) =
k
�
i=1
P (Bi ) · P (A|B
Let B1 , B2 , B3 , ... be a cover of the sample
space,
then
is a cover of the sample space Ω, we can compute the probab
P (A|Bj ) · P (Bj )
g.??): P (B |A) = P (Bj ∩ A) =
�k
j
P (A)
i=1 P (A|Bi ) · P (Bi )
P (A) =
k
�
i=1
P (Bi ) · P (A|Bi ).
∩ A)
P (A|Bj ) · P (Bj )
= �k
A) Wednesday, Augusti=1
24, 2011
P (A|Bi ) · P (Bi )
for all j and ∅ =
� A ⊂ Ω.
26
