Markov-opoly
Markov chains: an Applied Approach
By Daniel Huang and Mo Dwyer
What is a Markov Chain, Anyway?
• It is a sequence of states where every
future state is independent of the
preceding ones, except for the n-1 state.
The Process
• Create a “transition matrix.”
• This defines the probability of something
being in any given location in the state
you are interested in.
• Raise the transition matrix to the nth
power.
Confused? Have an example…
The state: a car rental agency has three
locations in LA:
• Downtown location (A)
• East end location (B)
• West end location (C).
The agency's statistician has
determined the following:
Of the calls to the Downtown location, 30% are delivered
in Downtown area, 30% are delivered in the East end, and
40% are delivered in the West end
Of the calls to the East end location, 40% are delivered in
Downtown area, 40% are delivered in the East end, and
20% are delivered in the West end
Of the calls to the West end location, 50% are delivered in
Downtown area, 30% are delivered in the East end, and
20% are delivered in the West end.
The Transition Matrix:
T=
T2=
T7=
Notice how it
starts to converge!
Tn=
You now have the probability distribution
of the drivers at time n!
So where will you be next?
• Take the transition matrix and multiply
by the current state.
• That is:
XO+1=TXO
So where will you be next?
• Take the transition matrix and multiply
by the current state.
• That is:
XO+1=TXO
• After time n approaching infinity, the
resulting vector is not dependent on Xo
P=TnX
Our Project:
Creating a Markov chain to predict
monopoly moves.
For Monopoly
• First, find the probability of rolling a
certain number, and thus landing on a
certain square.
• Add the prob of Chance or Community
Chest sending you somewhere.
Thank goodness for MATLAB
• You end up with a 40x40 matrix
because there are 40 squares to land on.
Go
Go
Meridian
CC
Baltic
Income
RR
Oriental
0.0121
0.0086
0.0069
0.0052
0.0035
0.0034
0.0035
0
0
0
0
0
0
0
Community Chest
0.0244
0
0
0
0
0
0
Baltic Avenue
0.0556
0.0278
0
0
0
0
0
Income Tax
0.0833
0.0642
0.0347
0.0052
0.0035
0.0017
0
Reading Railroad
0.1319
0.0919
0.0625
0.033
0.0035
0.0017
0
Oriental Ave
0.1389
0.1111
0.0833
0.0556
0.0278
0
0
Chance
0.0626
0.0529
0.0416
0.0313
0.0205
0.0108
0
Vermont Ave
0.1389
0.1666
0.1389
0.1111
0.0833
0.0556
0.0278
Connecticut Ave
0.1111
0.1389
0.1666
0.1389
0.1111
0.0833
0.0556
Just Visiting
0.0833
0.1111
0.1389
0.1666
0.1389
0.1111
0.0833
0.066
0.0919
0.1185
0.1441
0.1701
0.1406
0.1111
0.0382
0.0642
0.0902
0.1163
0.1425
0.1683
0.1389
States Ave
0
0.0278
0.0556
0.0833
0.1111
0.1389
0.1666
Virginia Ave
0
0
0.0278
0.0556
0.0833
0.1111
0.1389
0.0208
0.0172
0.0138
0.0382
0.0626
0.0867
0.1111
0
0
0
0
0.0278
0.0556
0.0833
Meridian Ave
St. Charles Pl
Electric Company
Pennsylvania Railroad
St. James Pl
Et c. …
Alternatively:
• You can also use eigenvectors to solve
steady-states!
Alternatively:
• You can also use eigenvectors to solve
steady-states!
• Take λ=1
• Tp=p
• p is the probability vector of
dimension mx1. Its elements add to
equal 1.
THE END!
Any questions?