MATH 166 Spring 2016
M.1
c
Wen
Liu
Chapter M Markov Chains
M.1 Markov Processes
Definition:
• A finite stochastic process is called a Markov process if the following are true:
– Each experiment has the same possible outcomes.
– The probabilities of the outcomes depend only on the preceding experiment.
• A matrix T is a stochastic or transition matrix if
– the matrix T is square.
– all elements in the matrix T are nonnegative.
– the sum of all the elements in any column is 1.
Note:
• If is very useful to describe a Markov process by a transition matrix. The transition matrix
gives the probabilities of going from one state to another.
• Let T be a transition matrix and X0 be the initial probability matrix. If Xn is the distribution
in the nth generation, then
Xn = T n X0
Examples:
1. Given the transition matrix T and initial distribution matrix X0 below, find X1 and X5 .
0.7 1
0.4
T =
, X0 =
0.3 0
0.6
2. It is known that a certain businessman always wears a white shirt (W) or a blue shirt (B) to
work and that 90% of the time when wearing a white shirt he changes color the next day, and
60% of the time when wearing a blue shirt he changes color the next day.
(a) Write the transition matrix.
Page 1 of 3
MATH 166 Spring 2016
M.1
c
Wen
Liu
(b) He was observed wearing a blue shirt on Monday. What is the probability he will be
wearing a blue shirt again 4 business days later?
3. In a certain state, when a Democrat is governor 80% of the time a Democrat will be governor
next time, 10% of the time an Independent will be, and 10% of the time a Republican will
be. When an Independent is governor, 30% of the time an Independent will be governor next
time, 35% of the time a Democrat, and 35% of the time a Republican. When a Republican is
governor, 45% of the time a Republican will be governor next time, 20% of the time a Democrat,
and 35% of the time an Independent.
(a) Write the transition matrix.
(b) For this current election the experts are giving the Democrat a 30% chance, the Independent given a 45% chance, and the Republican a 25% chance of being elected governor.
What is the probability that an Democrat will be elected 6 elections from the current
election?
Page 2 of 3
MATH 166 Spring 2016
M.1
c
Wen
Liu
4. A survey indicates that people in a certain area take their summer vacations either at the beach,
at the lake, or in the mountains. The survey finds that among people who have gone to the
beach, 50% go to the beach next summer, 25% go to the lake, and 25% go to the mountains.
The survey found also that among people who have gone to the lake, 20% go to the beach next
summer, 60% to the lake and 20% go to the mountains. Finally the survey finds that among
people who have gone to the mountains, 30% go to the beach next summer, and 70% go to the
lake. This year the lake suffered a fish kill and nobody went there. Half went to the beach and
half went to the mountains.
(a) What is the probability that an individual will go to each of these places next summer?
(b) What is the probability that an individual will go to each of these places 5 summers from
now?
Page 3 of 3