Section M.1
MATH 166:503
April 21, 2015
1
1.1
MARKOV CHAINS
Markov Processes
ex. (from Exam 2) A small bowl of trail mix contains 10 peanuts, 8 M&Ms, and 4 raisins. You
randomly pick three items from the bowl without replacement. If the first item we pick is a peanut
and the second a raisin, what is the probability of picking an M&M on the third pick?
A finite stochastic process is a finite sequence of experiments where the outcome of an experiment depends on the outcomes of the previous experiments.
A Markov process is a finite stochastic process where each experiment has the same possible outcomes and the outcome of an experiment only depends on the outcome of the previous experiment.
1
ex. For a particular 6-sided die, the probability of rolling the face opposite to the face shown when
2
the die is picked up is 13 . The probability of rolling all other faces is 15
. You rolled a 3. Draw a
tree diagram that gives the probability of the outcomes on the second roll. Draw a tree digram
that gives the probabilty of the outcomes on the second roll if you rolled a 4.
Now assume that you recieve the die from a machine that gives you the die with each face being
equally likely. Compute the probability of rolling a 3 after recieving the die from the machine.
Of rolling a 4?
Of rolling a 1?
2
Compute these another way.
Notation for Markov processes:
Compute the probabilty of each of the outcomes occuring on roll 2.
In general,
3
The machine which releases the dice was reprogrammed and now gives a 1 20% of the time, a
2 6% of the time, a 3 11% of the time, a 4 3% of the time, a 5 46% of the time, and a 6 14% of
the time. What is the probability of getting a 5 on the 5th roll after recieving the die from this
machine?
ex. Sofia always goes to either Chipotle, Subway, or Antonio’s Pizza for lunch every day. If she
goes to Chipotle, she will return 2% of the time and will go to Antonio’s 15% of the time for lunch
tomorrow. If she goes to Subway, she will return 2% of the time and will go to Chipotle 36% of the
time for lunch tomorrow. If she goes to Antonio’s, she will return 73% of the time and will go to
Subway 10% of the time for lunch tomorrow. What is the probability that, if she went to Subway
today, she will go to Antonio’s next Monday?
4