1.
2.
3.
Coin 1 comes up heads with probability 0.6 and coin 2 with probability 0.5. A
coin is continually flipped until it comes up tails, at which time that coin is put
aside and we start flipped the other one.
(a) What proportion of flips use coin 1?
(b) If we start the process with coin 1 what is the probability that coin 2 is
used on the fifth flip?
Trials are performed in sequence. If the last two trials were successes, then the
next trial is a success with probability 0.8; otherwise the next trial is a success
with probability 0.5. In the long run, what proportion of trials are successes?
For a series of dependent trials the probability of success on any trial is
(k+1)/(k+2) where k is equal to the number of successes on the previous two
trials. Compute lim n P{success on the nth trial}.
Solutions:
1. If the state at time n is the nth coin to be flipped then sequence of consecutive
state constitute a two state Markov chain with transition probabilities
P1,1=0.6=1-P1,2
P2,1=0.5=P2,2
(a) The stationary probabilities satisfy
 1  0.6 1  0.5 2
1   2  1
5
4
Solving yields that  1  ,  2  . So the proportion of flips that use
9
9
coin 1 is 5/9
(b) P1,42  0.4444
2. Let the state be 0 if the last two trials were both successes. 1 if the last trial was a
success and the one before it a failure. 2 if the last trial was a failure. The
transition probability matrix of this Markov chain is
| 0.8 0 0.2 |
P=| 0.5 0 0.5 |
| 0 0.5 0.5 |
5
2
4
1 
 2  . Consequently, the proportion of trials that
This gives  0 
11
11
11
are successes is 0.8  0  0.5(1   0 )  7 / 11.
[Method 2]
Let the state be 0 if the last two trials were both successes. 1 if the last trial was a
success and the one before it a failure. 2 if the last trial was a failure and the one
before it is a success. 3 if the last two trials were both failure. The transition
probability matrix of this Markov chain is
|
P=|
|
|
0.8
0.5
0
0
0 0.2 0 |
0 0.5 0 |
0.5 0 0.5 |
0.5 0 0.5|
5
2
2
2
1 
2 
 3  . Consequently, the proportion of
This gives  0 
11
11
11
11
trials that are successes is 0.8  0  0.5(1   0 )  7 / 11.
3. There are 4 states:
1=success on last 2 trials;
2= success on last, failure on next to last;
3=failure on last, success on next to last;
4=failure on last 2 trials.
Transition probabilities are:
P1,1=3/4, P1,3=1/4
P2,1=2/3, P2,3=1/3
P3,2=2/3, P3,4=1/3
P4,2=1/2, P4,4=1/2.
Limiting probabilities are given by
3
2
1  1   2
4
3
2
1
2  3  4
3
2
1
1
 3  1   2
4
3
1
1
4  3  4
3
2
1   2   3   4  1
1
3
1
3
3 
4 
And the solution  1   2 
2
16
16
8
11
Hence  1   2 
16