MATH 5050/6815: Homework 1
Due on Tuesday, Jan. 31, by the end of lecture.
Problem 1
Consider a simple random walk with absorbing boundaries on {0, 1, . . . , 10}, where 0, 10
are the absorbing states. The payoff function is [0, 2, 4, 3, 10, 0, 6, 4, 3, 3, 0].
(1) Find the optimal strategy that optimizes the expected gain.
(2) Draw (as a piecewise linear graph) the optimal strategy (x, v(x)) and payoff function
(x, f (x)). What do you observe? Can you read the set S2 from your graph?
(3) Show that for any simple random walk (p = 1/2) on finite states with absorbing
boundaries, with a non - negative payoff function, the function v(x) is the smallest
concave function higher than f .
(4) (PC) How many iterations does the algorithm we discussed in class requires to be
close to the optimal v at three siginificant digits?
Problem 2
Consider a simple random walk with absorbing boundaries on {0, 1, . . . , 10}, where 0, 10
are the absorbing states. The payoff function is [0, 2, 4, 3, 10, 0, 6, 4, 3, 3, 0].
(1) Find the optimal strategy that optimizes the expected gain if there’s a discount
factor α = .9
(2) Find the optimal strategy that optimizes the expected gain if there’s a cost equal
to 2 for each move.
(3) Find the largest possible discount factor α such that the optimal strategy is to stop
at each move.
Problem 3
(1) Do Exercise 4.2, page 98 from your book.
(2) (PC) Do Exercise 4.4, page 98 from your book, using a computer.
Problem 4
Suppose you play this game with the instructor: You start by paying a ticket to the
instructor of either 3 or 6 dollars, and then roll one of two tetrahedra: If you paid 3 dollars,
the tetrahedron has sides marked up as 1, 2, D, D. If you paid 6, the tetrahedron has sides
1, 2, 0, D. At any step you roll the 4 sided-die and if it is 0, 1 or 2, that amount of money
is added in a jar by me and you either stop and collect the jar or continue and roll again
(no cost after the initial one). When you roll a D you lose all the money in the jar. How
much money do you pay for the ticket?
1