MATH 5040/6810: Homework 1
Due on Thursday, Sep. 8, by the end of lecture.
Problem 1
a) Let A0 , A1 , . . . An be events of non-zero probability. Show that
P{A0 A1 A2 . . . An } = P{A0 }P{A1 |A0 } · · · P{An |An−1 . . . A0 }
Where are you using the assumption that the events have non-zero probability?
b) In a test paper the questions are arranged so that 2/5’s of the time a True answer is
followed by a True, while 3/5’s of the time a False answer is followed by a False. You are
confronted with a 100 question paper and the first question is so easy that you know it’s
true.
(1) What is the probability that the first 6 answers are T T F F T F ?
(2) Approximately what fraction of the answers will be True?
Problem 2
a) A fair coin is tossed repeatedly with results Y0 , Y1 , Y2 , · · · that are 0 and 1 with probability
1/2 each. For n ≥ 1, let Xn = Yn + Yn−1 be the number of 1’s in the (n − 1)th and nth
tosses. Is Xn a Markov chain?
b) Suppose that the probability it rains today is 0.3 if neither of the last two days was
rainy, but 0.6 if at least one of the last two days was rainy. Let the weather on day n , Wn ,
be R for rain, or S for sunny. Explain in plain English why Wn cannot be a Markov Chain.
Then show (using a computation) that Wn is not a Markov chain.
Problem 3
Let Nn be the number of heads observed in the first n flips of a coin and let Xn = Nn
mod 4, i.e., the remainder when we divide by 4. The coin is unfair: It favors heads with
probability 0.25.
(1) Show that Xn is a Markov chain, by writing down the transition matrix.
(2) (Computer required!) What is the probability that after 256 throws of the coin we
observed an odd number of heads?
(3) Use the Markov chain Xn to find the limit as n → ∞ of P{Nn is a multiple of 4}.
Problem 4
Three white balls and three black balls are distributed in two urns in such a way that
each urn contains 3 balls. At each step we draw one ball from each urn and exchange them.
Let Xn be the number of white balls in the left urn at time n.
a) Compute the transition probabilities for Xn .
b) Find the invariant distribution of the chain.
c) Assume we start with all white balls in the left urn. What is the probability that after
4 steps we have all white balls in the right urn?
Problem 5
A taxicab driver moves between the airport A and two hotels B and C according to the
following rules. If she is at the airport, she will be at one of the two hotels next with equal
1
2
probability. If at a hotel then she returns to the airport with probability 3/4 and goes to
the other hotel with probability 1/4.
(1) Find the transition matrix for the chain.
(2) Suppose the driver begins at the airport at time 0. Find the probability for each of
her three possible locations at time 2 and the probability she is at hotel B at time
3.
(3) Assuming that at time 3 she is at hotel B, what is the probability that she started
from hotel C if all starting positions were equally likely?
Problem 6
Suppose Xn and Yn are independent Markov chains on finite state spaces R and S
respectively. Show that Zn = (Xn , Yn ) is a Markov chain on R × S.
(Hint: Can you describe the transition probability of Zn in terms of those of Xn and Yn ?
Show your claim.)