Math 166 Week-in-Review - © S. Nite 11/17/2012
WIR #10
Page 1 of 2
Week in Review #10 (M1-M.2)
A Markov Process is a finite stochastic process such that 1) each experiment has
the same possible outcomes, and 2) the probabilities of the outcomes depend only
on the preceding experiment.
m-State Markov Processes
A Markov process with m states can be represented by a transition matrix where
each element pij is the conditional probability pij = p(next in state i | currently in
state j). If the column matrix Xn gives the state after the nth experiment, then Xn =
TnX0 when the column matrix X0 of dimension m is the initial state.
Stochastic (Transition) Matrix
A matrix T is a stochastic or transition matrix if (1) the matrix is square, (2) all
elements in the matrix are nonnegative, and (3) the sum of all the elements in any
column is 1.
A steady-state distribution for a Markov process is a limiting column matrix, i.e.,
one which Xn approaches.
The steady-state matrix is the limiting matrix that Tn approaches.
A regular Markov process is one for which the associated transition matrix is
such that no matter the initial probability distribution, the limiting distribution is
always the same.
A regular stochastic matrix, T, is one for which some power of T has all positive
entries.
Finding the Steady-State Distribution
For a steady-state matrix, T, Xn = TXn-1.
As n becomes large without bound, Xn approaches XL, where XL is the limiting
matrix. Thus, if T is a regular stochastic matrix, the steady-state distribution XL can
be found by solving the equation XL = TXL, together with the fact that the sum of
the entries in XL must be one.
Math 166 Week-in-Review - © S. Nite 11/17/2012
WIR #10
Page 2 of 2
1. At a university it is estimated that 50% of the commuter students drive to school and the other
50% take the bus. The university is upgrading the bus system and expects bus usage to
increase in the next 6 months. It is projected that 20% of the current car drivers will switch to
using the bus and 90% of the current bus users will continue to use the bus. What will the bus
and car usage look like in 6 months?
2. If the trend in #1 continues, what will the usage look like in 12 months?
24 months?
18 months?
3. A town has three pizza places, A, B, and C. If a customer ordered from A, then the next time
he/she orders a pizza there is a 50% chance he/she will order from A again, 30% from B, and
20% from C. If a customer ordered from B, then the next time he/she ordered a pizza there is a
40% chance he/she will order from A, 20% from B again, and 40% from c. If a customer
ordered from C, then the next time he/she orders a pizza there is a 25% change he/she will
order from A, 50% from B, and 25% from C again.
At the start of the year, 30% of the customers order from A, 50% from B, and 2% from C. If
customers order pizza once per week, find the distribution of customers among the three places
after 1, 2, and 6 weeks.
4. Find the steady-state value for # 1.
5. Find the steady-state value for # 3.