BU7527
Example sheet — 4
Mike Peardon — mjp@maths.tcd.ie
School of Mathematics, TCD
Thursday, 1st October
Try to answer these questions before tomorrow’s lectures. We will go through solutions
in class.
1. In three countries, the fractions of populations Fij resident in the next year in country
i given they are currently resident in country j are


97% 5% 1%


F =  2% 94% 0% 
1% 1% 99%
Find the long-term average fractional populations of the three countries.
2. A machine processes raw material. It can be in any one of three distinct states: idle,
working or broken each day. When it is idle, there is a probability of 25% new raw
material can be found and it will start working on the next day. When it is working
there is a probabilty of 50% it will complete processing and be idle the next day. There
is also the probability of 25% it breaks. When it is broken there is a probabilty of 10%
each day that it will be fixed and return to the idle state. Write a Markov matrix for
this system and find the steady-state probabilities of the three states {Idle, Working,
Broken} respectively.
3. Gambler’s ruin: For each starting amount a, winning goal c and win probability for
each round p, find the probability of reaching the winning goal. Each round of the
game the player wins e1 with probability p and loses e1 with probability 1-p.
1. p = 1/2, a = e5, c = e10
2. p = 0.499, a = e5, c = e10
3. p = 0.501, a = e5, c = e10
4. p = 1/2, a = e5, 000, c = e10, 000
5. p = 0.499, a = e5, 000, c = e10, 000
6. p = 0.501, a = e5, 000, c = e10, 000
1
4. For each turn of a game, an amount ex is gambled and the player either wins back
e2x or ex/2 with probability p and 1 − p respectively. What value of p makes this
game a martingale? A player starts with e8 and must stop playing when his fortune is
e1. The player decides they will stop playing when they have more than e100. What
is the probability they will meet this goal?
2