Independence and Bernoulli Trials
Independence
Events A and B are independent if
P ( AB )  P ( A) P ( B ).
 A, B independent implies:
Proof for independence of
are also independent.
:
B  ( A  A) B  AB  AB
AB  AB   ,
P( B )  P( AB  AB )  P( AB)  P( AB )  P( A) P( B )  P( AB )
P( AB )  P( B )  P( A) P( B )  (1  P( A)) P( B )  P( A) P( B ),
Sharif University of Technology
2
Application
Example
 let :
Ap  " the prime p divides the number N"
Aq  " the prime q divides the number N".
 Then
P{ Ap } 
1
,
p
P{ Aq } 
1
q
 Also
P{ Ap  Aq }  P{" pq divides N "} 
1
 P{ Ap } P{ Aq }
pq
 Hence it follows that Ap and Aq are independent events!
Sharif University of Technology
3
Some Properties of Independence
The event of zero probability is independent of
every other event
 Let P(A)=0,
 Then:
AB  A
P( AB)  P( A)  0  P( AB)  0,
 Thus: P(AB) = P(A) P(B) = 0
Other independent events cannot be ME
 Since:
P( A)  0, P( B )  0
P ( AB)  0.
Sharif University of Technology
4
Application
Remark
 A family of events {Ai} are said to be independent, if for every finite
sub collection Ai , Ai ,, Ai ,
1
2
n
 n
P  Aik
 k 1
n

   P( Aik ).
 k 1
Application
 If A  A1  A2  A3    An , and Ais are independent,
A  A1 A2  An
n
n
i 1
i 1
P ( A)  P ( A1 A2  An )   P ( Ai )   (1  P ( Ai )).
n
P( A)  1  P ( A)  1   (1  P( Ai )),
i 1
 Application in solving number theory problems.
Sharif University of Technology
5
Example
Three independent parallel switches
Each switch remains closed with
probability p.
(a) Find the probability of receiving an
input signal at the output.
Solution
 Let Ai = “Switch Si is closed” and R = “input signal is received at the
output”. Then
P( Ai )  p, i  1  3.
P( Ai Aj )  P( Ai ) P( Aj ); P( A1 A2 A3 )  P( A1 ) P( A2 ) P( A3 ).
R  A1  A2  A3.
P( R )  P( A1  A2  A3 )  1  (1  p )3  3 p  3 p 2  p 3.
Sharif University of Technology
6
Example – continued
Another Solution
 Since any event and its complement form a trivial partition,
P( R )  P( R | A1 ) P( A1 )  P( R | A1 ) P( A1 ).
P( R | A1 )  1,
P( R | A1 )  P( A2  A3 )  2 p  p2
 Thus:
P( R)  p  (2 p  p 2 )(1  p)  3 p  3 p 2  p 3 ,
Note:The events A1, A2, A3 do not form a partition, since they are not ME.
Moreover, P( A1 )  P( A2 )  P( A3 )  1.
Sharif University of Technology
7
Example – continued
(b) Find the probability that switch S1 is open given that
an input signal is received at the output.
Solution
 From Bayes’ theorem
P( R | A1 ) P( A1 ) (2 p  p 2 )(1  p)
2  3p  p2
P( A1 | R) 


.
2
3
2
P( R)
3p  3p  p
3  3p  p
 Also, because of the symmetry :
P( A1 | R )  P( A2 | R )  P( A3 | R ).
Sharif University of Technology
8
Repeated Trials
 A joint performance of the two experiments with probability models
(1, F1, P1) and (2, F2, P2).
 Two models’ elementary events: 1, 2
 How to define the combined trio (, F, P)?
  = 1 2
 Every elementary event  in  is of the form  = (, ).
 Events: Any subset A  B of  such that AF1 and B  F2
 F : all such subsets A  B together with their unions and
complements.
Sharif University of Technology
9
Repeated Trials
 In this model, The events A  2 and 1  B are such that
P( A  2 )  P1 ( A), P(1  B)  P2 ( B).
 Since
( A  2 )  (1  B)  A  B,
the events A  2 and 1  B are independent for any A  F1 and B  F2.
 So, for all A  F1 and B  F2
P( A  B)  P( A  2 )  P(1  B)  P1 ( A) P2 ( B)
 Now, we’re done with definition of the combined model.
Sharif University of Technology
10
Generalization
 Experiments 1 , 2 ,, n , with Fi and Pi , i  1  n,
 let
  1  2    n
 Elementary events
1 , 2 ,, n ,
 Events in are of the form A1  A2    An
and their unions and intersections.
 i  i .
Ai  Fi ,
 If Ai s are independent, and Pi ( Ai ) is the probability of the
event Ai in Fi then
P( A1  A2 
 An )  P1 ( A1 ) P2 ( A2 )
Sharif University of Technology
Pn ( An ).
11
Example
 A has probability p of occurring in a single trial.
 Find the probability that A occurs exactly k times, k  n in n
trials.
Solution
 The probability model for a single trial : (, F, P)
 Outcome of n experiments is :   1 ,  2 ,,  n  0 ,
where  i   and 0        
 A occurs at trial # i , if i  A.
 Suppose A occurs exactly k times in .
Sharif University of Technology
12
Example - continued
 If
i , i ,
1
, i k
2
belong to A,
P0 ( )  P({ i1 ,i2 ,,ik ,,in })  P({ i1 })P({ i2 }) P({ ik }) P({ in })
 P( A) P( A)  P( A) P( A) P( A)  P( A)  p k q n k .


 


n k
k
 Ignoring the order, this can happen in N disjoint equiprobable ways, so:
P(" A occurs exactly k times in n trials" )
N
  P0 (i )  NP0 ( )  Npk q n k ,
i 1
 Where
N
n
n(n  1)(n  k  1)
n!

  
k!
(n  k )! k!  k 
Sharif University of Technology
13
Bernoulli Trials
Pn (k )  P(" A occurs exactly k times in n trials" )
 n  k n k
   p q , k  0,1,2,, n,
k 
 Independent repeated experiments
 Outcome is either a “success” or a “failure”
 The probability of k successes in n trials is given by the above formula,
where p represents the probability of “success” in any one trial.
Sharif University of Technology
14
Example
Example
 Toss a coin n times. Obtain the probability of getting k heads in n trials.
 “head” is “success” (A) and let p  P (H ).
 Use the mentioned formula.
Example
 In rolling a fair dice for eight times, find the probability that either 3 or 4
shows up five times ?
"success"  A  { either 3 or 4 }   f3   f 4 .
1 1 1
P
(
A
)

P
(
f
)

P
(
f
)

 
3
4
 Thus
6 6 3
 n  k n k
8
Pn (k )    p q
P8 (5)    (1/ 3)5 (2 / 3)3  0.068282
k 
 5
Sharif University of Technology
15
Bernoulli Trials
 Consider a Bernoulli trial with success A, where P ( A)  p, P( A)  q.
 Let:
X k  " exactly k occurrence s in n trials" .
P( X 0  X 1    X n )  1.
 As X i , X j are mutually exclusive,
P( X 0  X 1    X n ) 

( a  b) 
n
 n  k n k


k 
a b ,
k 0 

n
 P( X k ) 
k 0
n
so,
 n  k n k


k 
p q .
k 0 

n
( p  q) n  1
Sharif University of Technology
16
Bernoulli Trials
 For a given n and p what is the most likely value of k ?
Pn (k  1)
n! p k 1q n k 1
(n  k )! k!
k
q


.
k n k
Pn (k )
(n  k  1)! (k  1)! n! p q
n  k 1 p
 Thus
Pn (k )  Pn (k  1), if k (1  p )  ( n  k  1) p or k  ( n  1) p.
 Thus Pn (k ) as a function of k increases until k  ( n  1) p
 The k max  [( n  1) p ], is
the most likely number of successes
Pn (k )
n  12, p  1 / 2.
in n trials.
k
Fig. 2.2
Sharif University of Technology
17
Effects Of p On Binomial Distribution
Sharif University of Technology
18
Effects Of p On Binomial Distribution
Sharif University of Technology
19
Effects Of n On Binomial Distribution
Sharif University of Technology
20
Example
 In a Bernoulli experiment with n trials, find the probability that the number
of occurrences of A is between k1 and k2.
Solution
P(" Occurrence s of A is between k1 and k2 " )
 n  k n k
 P( X k1  X k1 1    X k2 )   P( X k )     p q .
k k1
k k1  k 
k2
Sharif University of Technology
k2
21
Example
 Suppose 5,000 components are ordered. The probability that a part is
defective equals 0.1. What is the probability that the total number of
defective parts does not exceed 400 ?
Solution
Yk  " k parts are defective among 5,000 components ".
400
P(Y0  Y1    Y400 )   P(Yk )
k 0
 5000 
(0.1) k (0.9)5000 k .
  
k 
k 0 
400
Sharif University of Technology
22
 If kmax is the most likely number of successes in n trials,
(n  1) p  1  kmax  (n  1) p
 or
p
 So,
q k max
p

 p ,
n
n
n
km
 p.
n  n
lim
 This connects the results of an actual experiment to the axiomatic
definition of p.
Sharif University of Technology
23
Bernoulli’s Theorem
 A : an event whose probability of occurrence in a single trial is p.
 k : the number of occurrences of A in n independent trials
 Then ,
 k

pq

P   p    

.
2

n
n



k
 i.e. the frequency definition of probability of an event
and its
n
axiomatic definition ( p) can be made compatible to any degree of
accuracy.
Sharif University of Technology
24
Proof of Bernoulli’s Theorem
 Direct computation gives:
n 1
n
n
n!
n!
k n k
k
P
(
k
)

k
p
q

p k q n k



n
( n  k )! k!
k 0
k 1
k 1 ( n  k )! ( k  1)!
n 1
n!
( n  1)!
i 1 n i 1

p q
 np 
p i q n 1i
i 0 ( n  i  1)! i!
i 0 ( n  1  i )! i!
n 1
 np( p  q) n 1  np.
 Similarly,
n
n
n
n!
n!
k n k
k
P
(
k
)

k
p
q

p k q n k



n
( n  k )! ( k  1)!
k 0
k 1
k  2 ( n  k )! ( k  2)!
2
n
n!
p k q n k  n 2 p 2  npq.
k 1 ( n  k )! ( k  1)!

Sharif University of Technology
25
Proof of Bernoulli’s Theorem - continued

k
 p 
n
 Equivalently,
is equivalent to ( k  np ) 2  n 2 2 ,
n
 (k  np)
n
2
k 0
Pn ( k )   n 2 2 Pn ( k )  n 2 2 .
k 0
 Using equations derived in the last slide,
n
 (k  np)
k 0
n
2
n
Pn ( k )   k Pn ( k )  2np  k Pn ( k )  n 2 p 2
2
k 0
k 0
 n 2 p 2  npq  2np  np  n 2 p 2  npq.
n
 (k  np) P (k )   (k  np) P (k )   (k  np)
2
k 0
2
n
n
k  np  n

 (k  np) P (k ) 
2
k  np  n
n
 n 2 2 P k  np  n .
2
k  np  n
n 2 2
  P (k )
k  np  n
Sharif University of Technology
Pn ( k )
n
26
Proof of Bernoulli’s Theorem - continued
 Using last two equations,
 k

pq

P

p



.


2
 n

n



Conclusion
 For a given   0, pq / n 2 can be made arbitrarily small by letting n become
large.
 Relative frequency can be made arbitrarily close to the actual probability of
the event A in a single trial by making the number of experiments large
enough.
 As n  , the plots of Pn (k ) tends to concentrate more and more around kmax
Sharif University of Technology
27
Example: Day-trading strategy
 A box contains n randomly numbered balls (not necessarily 1 through n)
 m = np (p<1) of them are drawn one by one by replacement
 The drawing continues until a ball is drawn with a number larger than the
first m numbers.
 Determine the fraction p to be initially drawn, so as to maximize the
probability of drawing the largest among the n numbers using this
strategy.
Sharif University of Technology
28
Day-trading strategy: Solution
 Let
X k  (k  1) stdrawn ball has the largest number among:
all n balls,
the first k balls is in the group of first m balls, k > m.
 X k = A  B, where
A = “largest among the first k balls is in the group of first
m balls drawn”
B = (k+1)st ball has the largest number among all n balls”.
1 m 1 np p
P
(
X
)

P
(
A
)
P
(
B
)


 .
 Since A and B are independent ,
k
nk n k k
 So,
P (“selected ball has the largest number among all balls”)
n 1
n 1
n 1
1
  P( X k )  p   p  np  p ln k
k
k m
k m k
  p ln p.
Sharif University of Technology
n
np
29
Day-trading strategy: Solution
 To maximize with respect to p, let:
d
( p ln p)  (1  ln p)  0
dp
 So,
p = e-1 ≈ 0.3679
 This strategy can be used to “play the stock market”.
 Suppose one gets into the market and decides to stay up to 100 days.
 When to get out?
 According to the solution,
when the stock value exceeds the maximum among the first 37 days.
 In that case the probability of hitting the top value over 100 days for the stock is also about
37%.
Sharif University of Technology
30
Example: Game of Craps
 A pair of dice is rolled on every play,
- Win: sum of the first throw is 7 or 11
- Lose: sum of the first throw is 2, 3, 12
- Any other throw is a “carry-over”
 If the first throw is a carry-over, then the player throws the dice
repeatedly until:
- Win: throw the same carry-over again
- Lose: throw 7
Find the probability of winning the game.
Sharif University of Technology
31
Game of Craps - Solution
P1: the probability of a win on the first throw
Q1: the probability of loss on the first throw
P1  P(T  7)  P(T  11) 
6
2 2


36 36 9
Q1  P(T  2)  P(T  3)  P(T  12) 
1
2
1 1
 

36 36 36 9
B: winning the game by throwing the carry-over
C: throwing the carry-over
P2  P( B) 
10
10
 P( B | C  k )P(C  k )   P( B | C  k )P
k  4,k  7
k  4,k 7
Sharif University of Technology
k
32
Game of Craps - Solution
The probability of win by throwing the number of Plays that do not
count is:
rk  1  p k  1 / 6
The probability that the player throws the carry-over k on the j-th
throw is:
p k rk j 1 , j  1,2,3,..., 

a k  P ( B | C  k )  p k  rk j 1
j 1
k
ak
4
5
6
8
9
10
1 / 3 2 / 5 5 / 11 5 / 11 2 / 5 1 / 3
Sharif University of Technology
33
Game of Craps - Solution
P2 
10
 ak pk 
k  4,k  7
P1  P2 
1 3 2 4 5 5 5 5 2 4 1 3 134
          

3 36 5 36 11 36 11 36 5 36 3 36 495
2 134

 0.492929
9 495
the probability of winning the game of craps is 0.492929 for the player.
Thus the game is slightly advantageous to the house.
Sharif University of Technology
34
Game Of Craps Using Biased Dice
 Now assume that for each dice,
- Faces 1, 2 and 3 appear with probability 16  
- Faces 4, 5 and 6 appear with probability 16   ,
 0
 If T represents the combined total for the two dice, we get
p4  P{T  4}  P{(1,3),(2, 2),(1,3)}  3( 16   ) 2
1   2 )  2( 1   ) 2
p5  P{T  5}  P{(1, 4),(2,3),(3, 2),(4,1)}  2( 36
6
1   2 )  ( 1   )2
p6  P{T  6}  P{(1,5),(2, 4),(3,3),(4, 2),(5,1)}  4( 36
6
1 2)
p7  P{T  7}  P{(1,6),(2,5),(3, 4),(4,3),(5, 2),(6,1)}  6( 36
1   2 )  ( 1   )2
p8  P{T  8}  P{(2,6),(3,5),(4, 4),(5,3),(6, 2)}  4( 36
6
1   2 )  2( 1   ) 2
p9  P{T  9}  P{(3,6),(4,5),(5, 4),(6,3)}  2( 36
6
p10  P{T  10}  P{(4,6),(5,5),(6, 4)}  3( 16   ) 2
p11  P{T  11}  P{(5,6),(6,5)}  2( 16   ) 2 .
Sharif University of Technology
35
Game Of Craps Using Biased Dice
T=k
4
pk = P{T = k}
0.0706
5
6
0.1044
0.1353
7
0.1661
8
0.1419
9
0.1178
10
0.0936
11
0.0624
 This gives the probability of win on the first throw to be
P1  P(T  7)  P(T  11)  0.2285
 and the probability of win by throwing a carry-over to be
10
 Thus
P2  
k 4
k 7
pk2
 0.2717
pk  p7
 Although perfect dice gives rise to an unfavorable game,
P{winning the game}  P1  P2  0.5002
Sharif University of Technology
36
Game Of Craps Using Biased Dice
 Even if we let the two dice to have different loading factors 1 and
situation described above), similar conclusions do follow.
 For example,
 gives:
 2 (for the
1  0.01 and  2  0.005
P{winning the game}  0.5015.
Sharif University of Technology
37
Euler, Ramanujan and Bernoulli Numbers
Euler’s Identity
 S. Ramanujan in (J. of Indian Math Soc; V, 1913):
 if a2 , a3 , a5 , a7 , a11 ,
are numbers less than unity
 where the subscripts 2,3,5,7,11,
are the series of prime numbers,
then
1
1
1
1



 1  a2  a3  a2  a2  a5
1  a2 1  a3 1  a5 1  a7
 a 2  a3  a 7  a 2  a 2  a 2  a 3  a 3 
.
 The terms above are arranged so that the product obtained by multiplying the
subscripts are the series of all natural numbers 2,3, 4,5,6,7,8,9, .
 This follows by observing that the natural numbersare formed by multiplying
primes and their powers.
Sharif University of Technology
39
Euler’s Identity
 This is used to derive a variety of interesting identities including the Euler’s
identity.
s
s
s
a

1/
2
,
a

1/
3
,
a

1/
5
,
2
3
5
 By letting
 This gives the Euler identity

p prime

(1  p1s )  1/ n s .
1
n 1
 The right side can be related to the Bernoulli numbers (for s even).
 Bernoulli numbers are positive rational numbers defined through the power
series expansion of the even function 2x cot( x / 2).
 Thus if we write
 Then
x
x2
x4
x6

cot( x / 2)  1  B1
 B2
 B3

2
2!
4!
6!
1 ,B  1 ,B  1 ,B  1 , .
B1  16 , B2  30
3
42 4 30 5 66
Sharif University of Technology
40
Euler’s Identity
 Alse we can obtain :
x
x B1 x 2 B2 x 4 B3 x 6
 1 



x
e 1
2 2!
4!
6!
 This is another way to define Bernoulli numbers
 Further

Bn  4n  0

x 2 n 1
e2 x 1

dx   0 x 2 n1 ( e 2 x e 4 x 
2(2n )!  1
1
1
1




2n  2n
2n
2n
2n
(2 )  1
2
3
4
)dx



 which gives
S2 n 

 Thus

1/ k
k 1
2

1/ k
2n
k 1

2
6

;
(2 ) 2 n Bn

2(2n )!
1/ k
k 1
4

4
90
etc.
Sharif University of Technology
41