MATH 166 Spring 2016
c
Wen
Liu
M.2
M.2 Regular Markov Processes
Definition:
• A stochastic matrix T is said to be regular if some power of T has all positive entries.
• A Markov process is regular if the associated transition matrix is regular.
Finding the Steady-State Distribution: If T is a regular stochastic matrix, one can find the
steady-state distribution X by solving the equations
X = TX
together with the fact that the sum of the entries in X must be one.
Examples:
1. Classify the following matrices as:
(a) a stochastic matrix that is regular
(b) a stochastic matrix that is not regular
(c) not a stochastic matrix
(d) cannot be classified
(e) none of these
(i)
0.1 0.5
0.9 0.6
0.8 0.5
0.2 0.5
A=
(ii)
B=
(iii)
C=
0.3 0
0.7 1
Page 1 of 2
MATH 166 Spring 2016
M.2
c
Wen
Liu
2. It is known that a certain businessman always wears a white shirt (W) or a blue shirt (B) to
work and that 90% of the time when wearing a white shirt he changes color the next day, and
60% of the time when wearing a blue shirt he changes color the next day. What is the steady
state distribution?
3. A survey indicates that people in a certain area take their summer vacations either at the beach,
at the lake, or in the mountains. The survey finds that among people who have gone to the
beach, 50% go to the beach next summer, 25% go to the lake, and 25% go to the mountains.
The survey found also that among people who have gone to the lake, 20% go to the beach
next summer, 60% to the lake and 20% go to the mountains. Finally the survey finds that
among people who have gone to the mountains, 30% go to the beach next summer, and 70%
go to the lake. This year the lake suffered a fish kill and nobody went there. Half went to the
beach and half went to the mountains. Find the steady state distribution and use it to find the
probabilities of going to the beach, lake, and mountains.
Page 2 of 2