Bayes’ Rule
Bayes’ Rule - Updating Probabilities
• Let A1,…,Ak be a set of events that partition a sample
space such that (mutually exclusive and exhaustive):
– each set has known P(Ai) > 0 (each event can occur)
– for any 2 sets Ai and Aj, P(Ai and Aj) = 0 (events are disjoint)
– P(A1) + … + P(Ak) = 1 (each outcome belongs to one of events)
• If C is an event such that
– 0 < P(C) < 1 (C can occur, but will not necessarily occur)
– We know the probability will occur given each event Ai: P(C|Ai)
• Then we can compute probability of Ai given C occurred:
P(C | Ai ) P( Ai )
P( Ai and C )
P( Ai | C ) 

P(C | A1 ) P( A1 )    P(C | Ak ) P( Ak )
P(C )
Example - OJ Simpson Trial
• Given Information on Blood Test (T+/T-)
– Sensitivity: P(T+|Guilty)=1
– Specificity: P(T-|Innocent)=.9957  P(T+|I)=.0043
• Suppose you have a prior belief of guilt: P(G)=p*
• What is “posterior” probability of guilt after seeing
evidence that blood matches: P(G|T+)?
P(T )  P(T  G )  P (T  I )  P(G ) P (T  | G )  P( I ) P(T  | I ) 
 p * (1)  (1  p*)(.0043)
P(T  G ) P(G ) P(T  | G )
p * (1)
p*
P(G | T ) 



P(T  )
P (T  )
p * (1)  (1  p*)(.0043) .9957 p * .0043

Source: B.Forst (1996). “Evidence, Probabilities and Legal Standards for Determination of Guilt: Beyond the OJ Trial”, in
Representing OJ: Murder, Criminal Justice, and the Mass Culture, ed. G. Barak pp. 22-28. Harrow and Heston, Guilderland, NY
OJ Simpson Posterior (to Positive Test) Probabilities
P(G)  .10 
Prior Probabilit y of Guilt :
.10(1)
.10
P(G | T ) 

 .9627
.10(1)  .90(.0043) .10387

P(G|T+) as function of P(G)
1
0.8
P(G|T+)
0.6
0.4
0.2
0
0
0.2
0.4
0.6
P(G)
0.8
1
1.2
Northern Army at Gettysburg
Regiment
I Corps
II Corps
III Corps
V Corps
VI Corps
XI Corps
XII Corps
Cav Corps
Arty Reserve
Sum
Label
A1
A2
A3
A4
A5
A6
A7
A8
A9
Initial #
10022
12884
11924
12509
15555
9839
8589
11501
2546
95369
Casualties
6059
4369
4211
2187
242
3801
1082
852
242
23045
P(Ai)
0.1051
0.1351
0.1250
0.1312
0.1631
0.1032
0.0901
0.1206
0.0267
1
P(C|Ai)
0.6046
0.3391
0.3532
0.1748
0.0156
0.3863
0.1260
0.0741
0.0951
P(C|Ai)*P(Ai)
0.0635
0.0458
0.0442
0.0229
0.0025
0.0399
0.0113
0.0089
0.0025
0.2416
P(C)
P(Ai|C)
0.2630
0.1896
0.1828
0.0949
0.0105
0.1650
0.0470
0.0370
0.0105
1.0002
• Regiments: partition of soldiers (A1,…,A9). Casualty: event C
• P(Ai) = (size of regiment) / (total soldiers) = (Column 3)/95369
• P(C|Ai) = (# casualties) / (regiment size) = (Col 4)/(Col 3)
• P(C|Ai) P(Ai) = P(Ai and C) = (Col 5)*(Col 6)
•P(C)=sum(Col 7)
• P(Ai|C) = P(Ai and C) / P(C) = (Col 7)/.2416
CRAPS
• Player rolls 2 Dice (“Come out roll”):
–
–
–
–
2,3,12 - Lose (Miss Out)
7,11 - Win (Pass)
4,5,6.8,9,10 - Makes point. Roll until point (Win) or 7 (Lose)
Probability Distribution for first (any) roll:
Roll
Probability
Outcome
2
1/36
Lose
3
2/36
Lose
4
3/36
Point
5
4/36
Point
6
5/36
Point
7
6/36
Win
8
5/36
Point
9
4/36
Point
10
3/36
Point
11
2/36
Win
After first roll:
•P(Win|2) = P(Win|3) = P(Win|12) = 0
•P(Win|7) = P(Win|11) = 1
•What about other conditional probabilities if make point?
12
1/36
Lose
CRAPS
• Suppose you make a point: (4,5,6,8,9,10)
–
–
–
–
You win if your point occurs before 7, lose otherwise and stop
Let P mean you make point on a roll
Let C mean you continue rolling (neither point nor 7)
You win for any of the mutually exclusive events:
• P, CP, CCP, …, CC…CP,…
• If your point is 4 or 10, P(P)=3/36, P(C)=27/36
• By independence, and multiplicative, and additive rules:
k
 27  3 
 27   3 
P (CP)  

C
P
)


 P (CC

 




36
36
 36  36 




k
Win  P  CP    CCC  CP  
3
P( P) 
36
k
 3   27  3 
 27   3 
 3    27 
P ( Win)  


  
 
   
 

 36   36  36 
 36   36 
 36  i 0  36 
1
1
 3 
 1 


(
4
)


 
3
 36  1  27 / 36  12 
i
CRAPS
• Similar Patterns arise for points 5,6,8, and 9:
– For 5 and 9: P(P) = 4/36 P(C) = 26/36
– For 6 and 8: P(P) = 5/36 P(C) = 25/36
i
1
 4    26   4 
  4  36  2
Points 5 and 9 : P( Win )        
     
 36  i 0  36   36  1  26 / 36   36  10  5
i
1
 5    25   5 
  5  36  5
Points 6 and 8 : P( Win )        
     
36
36
36
1

25
/
36
  i 0    
  36  11  11
Finally, we can obtain player’s probability of winning:
CRAPS - P(Winning)
Come Out Roll
2
3
4
5
6
7
8
9
10
11
12
Sum
P(Roll)
1/36
2/36
3/36
4/36
5/36
6/36
5/36
4/36
3/36
2/36
1/36
1
P(Win|Roll)
0
0
1/3
2/5
5/11
1
5/11
2/5
1/3
1
0
P(Roll&Win)
0
0
1/36
2/45
25/396
1/6
25/396
2/45
1/36
1/18
0
P(Roll&Win)
0
0
0.02777778
0.04444444
0.06313131
0.16666667
0.06313131
0.04444444
0.02777778
0.05555556
0
0.49292929
P(Win)
P(Roll|Win)
0
0
0.05635246
0.09016393
0.12807377
0.33811475
0.12807377
0.09016393
0.05635246
0.11270492
0
1
Note in the previous slides we derived P(Win|Roll), we multiply those
by P(Win to obtain P(Roll&Win) and sum those for P(Win). The last
column gives the probability of each come out roll given we won.