Markov Chains
Extra problems
1
Markov Chain Example -- Weather
raining today
40% rain tomorrow
60% no rain tomorrow
not raining today
20% rain tomorrow
80% no rain tomorrow
Construct a transition graph and a transition matrix.
2
Markov Chain Example – Stock Price
• At the end of a given day, the price of a stock
is recorded. If the stock has gone up, the
probability that it will go up tomorrow is 0.7.
If the stock has gone down, the probability
that it will go up tomorrow is only 0.5.
• Construct a transition graph and a transition
matrix.
3
Markov Chain Example -- Gambler’s Ruin
At time zero I have X0 = $2, and each day I make a $1 bet.
I win with probability p and lose with probability 1– p.
I’ll quit if I ever obtain $4 or if I lose all my money.
Xt = amount of money I have after the bet on day t.
So, X1 =
{
3 with probability p
1 with probability 1 – p
if Xt = 4 then Xt+1 = Xt+2 =
•••
=4
if Xt = 0 then Xt+1 = Xt+2 =
•••
= 0.
The possible values of Xt is S = { 0, 1, 2, 3, 4 }
Construct a transition graph and a transition matrix.
Markov Chain Example -- Game of Craps
The Game of Craps in Las Vegas plays as follows
The player rolls a pair of dice and sums the numbers showing.
A total of 7 or 11 on the first rolls wins for the player
Where a total of 2, 3, 12 loses
Any other number is called the point.
The player rolls the dice again.
If he/she rolls the point number, she wins
If he/she rolls number 7, she loses
Any other number requires another roll
The game continues until he/she wins or loses.
Construct a transition graph and a transition matrix.