PROBABILITY AND STATISTICS
FOR ENGINEERING
Independence and Bernoulli Trials
Hossein Sameti
Department of Computer Engineering
Sharif University of Technology
Independence
Events A and B are independent if
P ( AB )  P ( A) P ( B ).
 A, B independent implies:
Proof for independence of
are also independent.
:
B  ( A  A) B  AB  AB
AB  AB   ,
P( B )  P( AB  AB )  P( AB)  P( AB )  P( A) P( B )  P( AB )
P( AB )  P( B )  P( A) P( B )  (1  P( A)) P( B )  P( A) P( B ),
Sharif University of Technology
2
Application
Example
 let :
Ap  " the prime p divides the number N"
Aq  " the prime q divides the number N".
 Then
P{ Ap } 
1
,
p
P{ Aq } 
1
q
 Also
P{ Ap  Aq }  P{" pq divides N "} 
1
 P{ Ap } P{ Aq }
pq
 Hence it follows that Ap and Aq are independent events!
Sharif University of Technology
3
Some Properties of Independence
The event of zero probability is independent of
every other event
 Let P(A)=0,
 Then:
AB  A
P( AB)  P( A)  0  P( AB)  0,
 Thus: P(AB) = P(A) P(B) = 0
Other independent events cannot be ME
 Since:
P( A)  0, P( B )  0
P ( AB)  0.
Sharif University of Technology
4
Application
Remark
 A family of events {Ai} are said to be independent, if for every finite
sub collection Ai , Ai ,, Ai ,
1
2
n
 n
P  Aik
 k 1
n

   P( Aik ).
 k 1
Application
 If A  A1  A2  A3    An , and Ais are independent,
A  A1 A2  An
n
n
i 1
i 1
P ( A)  P ( A1 A2  An )   P ( Ai )   (1  P ( Ai )).
n
P( A)  1  P ( A)  1   (1  P( Ai )),
i 1
 Application in solving number theory problems.
Sharif University of Technology
5
Example
Three independent parallel switches
Each switch remains closed with
probability p.
(a) Find the probability of receiving an
input signal at the output.
Solution
 Let Ai = “Switch Si is closed” and R = “input signal is received at the
output”. Then
P( Ai )  p, i  1  3.
P( Ai Aj )  P( Ai ) P( Aj ); P( A1 A2 A3 )  P( A1 ) P( A2 ) P( A3 ).
R  A1  A2  A3.
P( R )  P( A1  A2  A3 )  1  (1  p )3  3 p  3 p 2  p 3.
Sharif University of Technology
6
Example – continued
Another Solution
 Since any event and its complement form a trivial partition,
P( R )  P( R | A1 ) P( A1 )  P( R | A1 ) P( A1 ).
P( R | A1 )  1,
P( R | A1 )  P( A2  A3 )  2 p  p2
 Thus:
P( R)  p  (2 p  p 2 )(1  p)  3 p  3 p 2  p 3 ,
Note:The events A1, A2, A3 do not form a partition, since they are not ME.
Moreover, P( A1 )  P( A2 )  P( A3 )  1.
Sharif University of Technology
7
Example – continued
(b) Find the probability that switch S1 is open given that
an input signal is received at the output.
Solution
 From Bayes’ theorem
P( R | A1 ) P( A1 ) (2 p  p 2 )(1  p)
2  3p  p2
P( A1 | R) 


.
2
3
2
P( R)
3p  3p  p
3  3p  p
 Also, because of the symmetry :
P( A1 | R )  P( A2 | R )  P( A3 | R ).
Sharif University of Technology
8
Repeated Trials
 A joint performance of the two experiments with probability models
(1, F1, P1) and (2, F2, P2).
 Two models’ elementary events: 1, 2
 How to define the combined trio (, F, P)?
  = 1 2
 Every elementary event  in  is of the form  = (, ).
 Events: Any subset A  B of  such that AF1 and B  F2
 F : all such subsets A  B together with their unions and
complements.
Sharif University of Technology
9
Repeated Trials
 In this model, The events A  2 and 1  B are such that
P( A  2 )  P1 ( A), P(1  B)  P2 ( B).
 Since
( A  2 )  (1  B)  A  B,
the events A  2 and 1  B are independent for any A  F1 and B  F2.
 So, for all A  F1 and B  F2
P( A  B)  P( A  2 )  P(1  B)  P1 ( A) P2 ( B)
 Now, we’re done with definition of the combined model.
Sharif University of Technology
10
Generalization
 Experiments 1 , 2 ,, n , with Fi and Pi , i  1  n,
 let
  1  2    n
 Elementary events
1 , 2 ,, n ,
 Events in are of the form A1  A2    An
and their unions and intersections.
 i  i .
Ai  Fi ,
 If Ai s are independent, and Pi ( Ai ) is the probability of the
event Ai in Fi then
P( A1  A2 
 An )  P1 ( A1 ) P2 ( A2 )
Sharif University of Technology
Pn ( An ).
11
Example
 A has probability p of occurring in a single trial.
 Find the probability that A occurs exactly k times, k  n in n
trials.
Solution
 The probability model for a single trial : (, F, P)
 Outcome of n experiments is :   1 ,  2 ,,  n  0 ,
where  i   and 0        
 A occurs at trial # i , if i  A.
 Suppose A occurs exactly k times in .
Sharif University of Technology
12
Example - continued
 If
i , i ,
1
, i k
2
belong to A,
P0 ( )  P({ i1 ,i2 ,,ik ,,in })  P({ i1 })P({ i2 }) P({ ik }) P({ in })
 P( A) P( A)  P( A) P( A) P( A)  P( A)  p k q n k .


 


n k
k
 Ignoring the order, this can happen in N disjoint equiprobable ways, so:
P(" A occurs exactly k times in n trials" )
N
  P0 (i )  NP0 ( )  Npk q n k ,
i 1
 Where
N
n
n(n  1)(n  k  1)
n!

  
k!
(n  k )! k!  k 
Sharif University of Technology
13
Bernoulli Trials
Pn (k )  P(" A occurs exactly k times in n trials" )
 n  k n k
   p q , k  0,1,2,, n,
k 
 Independent repeated experiments
 Outcome is either a “success” or a “failure”
 The probability of k successes in n trials is given by the above formula,
where p represents the probability of “success” in any one trial.
Sharif University of Technology
14
Example
Example
 Toss a coin n times. Obtain the probability of getting k heads in n trials.
 “head” is “success” (A) and let p  P (H ).
 Use the mentioned formula.
Example
 In rolling a fair dice for eight times, find the probability that either 3 or 4
shows up five times ?
"success"  A  { either 3 or 4 }   f3   f 4 .
1 1 1
P
(
A
)

P
(
f
)

P
(
f
)

 
3
4
 Thus
6 6 3
 n  k n k
8
Pn (k )    p q
P8 (5)    (1/ 3)5 (2 / 3)3  0.068282
k 
 5
Sharif University of Technology
15
Bernoulli Trials
 Consider a Bernoulli trial with success A, where P ( A)  p, P( A)  q.
 Let:
X k  " exactly k occurrence s in n trials" .
P( X 0  X 1    X n )  1.
 As X i , X j are mutually exclusive,
P( X 0  X 1    X n ) 

( a  b) 
n
 n  k n k


k 
a b ,
k 0 

n
 P( X k ) 
k 0
n
so,
 n  k n k


k 
p q .
k 0 

n
( p  q) n  1
Sharif University of Technology
16
Bernoulli Trials
 For a given n and p what is the most likely value of k ?
Pn (k  1)
n! p k 1q n k 1
(n  k )! k!
k
q


.
k n k
Pn (k )
(n  k  1)! (k  1)! n! p q
n  k 1 p
 Thus
Pn (k )  Pn (k  1), if k (1  p )  ( n  k  1) p or k  ( n  1) p.
 Thus Pn (k ) as a function of k increases until k  ( n  1) p
 The k max  [( n  1) p ], is
the most likely number of successes
Pn (k )
n  12, p  1 / 2.
in n trials.
k
Fig. 2.2
Sharif University of Technology
17
Effects Of p On Binomial Distribution
Sharif University of Technology
18
Effects Of p On Binomial Distribution
Sharif University of Technology
19
Effects Of n On Binomial Distribution
Sharif University of Technology
20
Example
 In a Bernoulli experiment with n trials, find the probability that the number
of occurrences of A is between k1 and k2.
Solution
P(" Occurrence s of A is between k1 and k2 " )
 n  k n k
 P( X k1  X k1 1    X k2 )   P( X k )     p q .
k k1
k k1  k 
k2
Sharif University of Technology
k2
21
Example
 Suppose 5,000 components are ordered. The probability that a part is
defective equals 0.1. What is the probability that the total number of
defective parts does not exceed 400 ?
Solution
Yk  " k parts are defective among 5,000 components ".
400
P(Y0  Y1    Y400 )   P(Yk )
k 0
 5000 
(0.1) k (0.9)5000 k .
  
k 
k 0 
400
Sharif University of Technology
22
 If kmax is the most likely number of successes in n trials,
(n  1) p  1  kmax  (n  1) p
 or
p
 So,
q k max
p

 p ,
n
n
n
km
 p.
n  n
lim
 This connects the results of an actual experiment to the axiomatic
definition of p.
Sharif University of Technology
23
Bernoulli’s Theorem
 A : an event whose probability of occurrence in a single trial is p.
 k : the number of occurrences of A in n independent trials
 Then ,
 k

pq

P   p    

.
2

n
n



k
 i.e. the frequency definition of probability of an event
and its
n
axiomatic definition ( p) can be made compatible to any degree of
accuracy.
Sharif University of Technology
24
Proof of Bernoulli’s Theorem
 Direct computation gives:
n 1
n
n
n!
n!
k n k
k
P
(
k
)

k
p
q

p k q n k



n
( n  k )! k!
k 0
k 1
k 1 ( n  k )! ( k  1)!
n 1
n!
( n  1)!
i 1 n i 1

p q
 np 
p i q n 1i
i 0 ( n  i  1)! i!
i 0 ( n  1  i )! i!
n 1
 np( p  q) n 1  np.
 Similarly,
n
n
n
n!
n!
k n k
k
P
(
k
)

k
p
q

p k q n k



n
( n  k )! ( k  1)!
k 0
k 1
k  2 ( n  k )! ( k  2)!
2
n
n!
p k q n k  n 2 p 2  npq.
k 1 ( n  k )! ( k  1)!

Sharif University of Technology
25
Proof of Bernoulli’s Theorem - continued

k
 p 
n
 Equivalently,
is equivalent to ( k  np ) 2  n 2 2 ,
n
 (k  np)
n
2
k 0
Pn ( k )   n 2 2 Pn ( k )  n 2 2 .
k 0
 Using equations derived in the last slide,
n
 (k  np)
k 0
n
2
n
Pn ( k )   k Pn ( k )  2np  k Pn ( k )  n 2 p 2
2
k 0
k 0
 n 2 p 2  npq  2np  np  n 2 p 2  npq.
n
 (k  np) P (k )   (k  np) P (k )   (k  np)
2
k 0
2
n
n
k  np  n

 (k  np) P (k ) 
2
k  np  n
n
 n 2 2 P k  np  n .
2
k  np  n
n 2 2
  P (k )
k  np  n
Sharif University of Technology
Pn ( k )
n
26
Proof of Bernoulli’s Theorem - continued
 Using last two equations,
 k

pq

P

p



.


2
 n

n



Conclusion
 For a given   0, pq / n 2 can be made arbitrarily small by letting n become
large.
 Relative frequency can be made arbitrarily close to the actual probability of
the event A in a single trial by making the number of experiments large
enough.
 As n  , the plots of Pn (k ) tends to concentrate more and more around kmax
Sharif University of Technology
27
Example: Day-trading strategy
 A box contains n randomly numbered balls (not necessarily 1 through n)
 m = np (p<1) of them are drawn one by one by replacement
 The drawing continues until a ball is drawn with a number larger than the
first m numbers.
 Determine the fraction p to be initially drawn, so as to maximize the
probability of drawing the largest among the n numbers using this
strategy.
Sharif University of Technology
28
Day-trading strategy: Solution
 Let X k be the event of:
(k  1) st drawn ball has the largest number among all n balls,
and
the first k balls is in the group of first m balls, k > m.
 X k = A  B, where
A = “largest among the first k balls is in the group of first
m balls drawn”
B = “(k+1)st ball has the largest number among all n balls”.
m 1
np 1
p

 .
 Since A and B are independent , P (X k )  P (A )P (B ) 
k n
k n
k
 So,
P (“selected ball has the largest number among all balls”)
n 1
n 1
n 1
1
  P( X k )  p   p  np  p ln k
k
k m
k m k
  p ln p.
Sharif University of Technology
n
np
29
Day-trading strategy: Solution
 To maximize with respect to p, let:
d
( p ln p)  (1  ln p)  0
dp
 So,
p = e-1 ≈ 0.3679
 This strategy can be used to “play the stock market”.
 Suppose one gets into the market and decides to stay up to 100 days.
 When to get out?
 According to the solution,
when the stock value exceeds the maximum among the first 37 days.
 In that case the probability of hitting the top value over 100 days for the stock is also about
37%.
Sharif University of Technology
30
Example: Game of Craps
 A pair of dice is rolled on every play,
- Win: sum of the first throw is 7 or 11
- Lose: sum of the first throw is 2, 3, 12
- Any other throw is a “carry-over”
 If the first throw is a carry-over, then the player throws the dice
repeatedly until:
- Win: throw the same carry-over again
- Lose: throw 7
Find the probability of winning the game.
Sharif University of Technology
31
Game of Craps - Solution
P1: the probability of a win on the first throw
Q1: the probability of loss on the first throw
P1  P(T  7)  P(T  11) 
6
2 2


36 36 9
Q1  P(T  2)  P(T  3)  P(T  12) 
1
2
1 1
 

36 36 36 9
B: winning the game by throwing the carry-over
C: throwing the carry-over
P2  P( B) 
10
10
 P( B | C  k )P(C  k )   P( B | C  k ) p
k  4,k  7
k  4,k  7
Sharif University of Technology
k
32
Game of Craps - Solution
For winning by carry-over, the probability of each throw (k) that
does not count is:
rk  1  p k  1 / 6
The probability that the player throws the carry-over k on the j-th
throw is:
p k rk j 1 , j  1,2,3,..., 

a k  P ( B | C  k )  p k  rk j 1
j 1
k
ak
4
5
6
8
9
10
1 / 3 2 / 5 5 / 11 5 / 11 2 / 5 1 / 3
Sharif University of Technology
33
Game of Craps - Solution
P2 
10
 ak pk 
k  4,k  7
P1  P2 
1 3 2 4 5 5 5 5 2 4 1 3 134
          

3 36 5 36 11 36 11 36 5 36 3 36 495
2 134

 0.492929
9 495
the probability of winning the game of craps is 0.492929 for the player.
Thus the game is slightly advantageous to the house.
Sharif University of Technology
34