Macroeconomics TSE M1
Chapter 1 - Optimization in Discrete Time
Gaetano Gaballo and Franck Portier
Toulouse School of Economics
October 2012
Gaballo and Portier (TSE)
M1
10/12
1 / 24
Chapter 1 - Optimization in discrete time
Not a math class
We will review main concept with a “cake-eating problem"
More with Bénedicte Alziary (“Dynamic Optimization" course)
Gaballo and Portier (TSE)
M1
10/12
2 / 24
1. The problem
Finite horizon: t = 1, 2, ...T
Initial size of the cake W1
Size in t: Wt
Consumption ct ; ct
0
Flow utility u (ct )
u strict. increasing and strict. concave, di¤erentiable,
limc !0 u 0 (c ) = ∞
t 1
Intertemporal utility ∑T
u ( ct )
t =1 β
0 < β < 1: discount factor
Law of motion if the cake size: Wt +1 = Wt ct
Gaballo and Portier (TSE)
M1
10/12
3 / 24
1. The problem
Question
What is the optimal way of eating that cake?
Choice of a sequence of controls fc1 , c2 , . . . , cT g
Gaballo and Portier (TSE)
M1
10/12
4 / 24
2. First approach: using a Langrangian
T
max
fct gT1
subject to ct + Wt +1
Gaballo and Portier (TSE)
1
u ( ct )
t =1
Wt for each t
with W1 given and WT +1
which imply W1
∑ βt
∑Tt=1 ct
0
0.
M1
10/12
5 / 24
2. First approach: using a Langrangian
Form the Lagrangian:
T
L=
∑β
t 1
T
u ( ct ) + λ W 1
t =1
∑ ct
t =1
!
The objective is concave, continuous and di¤erentiable, the constraint
is linear so that the constraint set is compact (closed and bounded)
First Order Conditions (FOC) are therefore not only Necessary
Conditions (NC) but also Su¢ cient Conditions (SC)
Gaballo and Portier (TSE)
M1
10/12
6 / 24
2. First approach: using a Langrangian
Algebra
ON THE BLACKBOARD
Gaballo and Portier (TSE)
M1
10/12
7 / 24
2. First approach: using a Langrangian
Euler equation
We obtain the important Euler equation (which is a NC):
u 0 (ct ) = βu 0 (ct +1 )
Interpretation 1: reduce c by ε in t today and eat it in t + 1
Interpretation 2: reduce c by ε in t today and eat it in t + 2
Only a NC: what if WT +1 > 0?
Gaballo and Portier (TSE)
M1
10/12
8 / 24
2. First approach: using a Langrangian
Value function
VT (W1 ) = max ∑Tt=1 βt
subject to W1
∑Tt=1 ct
1
u ( ct )
0.
Analogy with an indirect utility function
Interpretation of the Lagrange multiplier λ:
VT0 (W1 ) = λ = βt 1 u 0 (ct ) for t = 1, 2, . . . , T
Gaballo and Portier (TSE)
M1
10/12
9 / 24
3. Finite Horizon Dynamic Programming Approach
Add a period 0
We change slightly the problem: add a period 0 and an initial size of
the cake W0
What is now the optimal plan?
Let’s take advantage of what we have already done
Highest utility that can be achieved from period 1 to T if the size of
the cake in period 1 is W1 : VT (W1 )
The intertemporal problem from period 0 to T can be written as
max u (c0 ) + βVT (W1 )
c0
where W1 = W0 c0 and with W0 given.
We just need to …nd c0
Principle of optimality: to choose c0 , we simply need to know that
decisions will be taken optimally in period 1 and onwards. What will
be exactly chosen for c1 , c2 , . . . , cT is irrelevant for deciding c0 .
Gaballo and Portier (TSE)
M1
10/12
10 / 24
3. Finite Horizon Dynamic Programming Approach
Working recursively
max u (c0 ) + βVT (W0
c0
c0 )
FOC is u 0 (c0 ) = βVT0 (W1 )
Prove that we have u 0 (c0 ) = βt u 0 (ct ) for t = 1, 2, . . . , T
1
This approach looks easy because we have assumed that we knew
VT ( W 1 )
What if we don’t?
We can construct it recursively:
compute V1 (W1 ) = u (W1 )
compute V2 (W1 ) = maxc1 u (c1 ) + βV1 (W2 )
compute V3 (W1 ) = maxW 2 u (W1 W2 ) + βV2 (W2 )
etc...
Gaballo and Portier (TSE)
M1
10/12
11 / 24
3. Finite Horizon Dynamic Programming Approach
Example
Take u (c ) = log(c )
Compute V1 (W1 ), V2 (W1 ), V3 (W1 ),
ON THE BLACKBOARD
Gaballo and Portier (TSE)
M1
10/12
12 / 24
4. In…nite Horizon Dynamic Programming Approach
Structure
max
fct g1∞
subject to Wt +1 = Wt
∞
∑ βt
1
u ( ct )
t =1
ct for t = 1, 2, . . .
Now consider V (W ) as the value of an in…nite horizon problem
starting with a cake of size W
The dynamic programming problem can be written
V (Wt ) =
max u (ct ) + βV (Wt
ct 2[0,W t ]
We will denote Wt +1 = Wt
Gaballo and Portier (TSE)
ct )
ct (prime for next period variables)
M1
10/12
13 / 24
4. In…nite Horizon Dynamic Programming Approach
Language
In this problem:
the state variable is the size of the cake Wt .
the state completely summarizes all information from the past that is
needed in the forward looking problem
the control variable is the variable currently chosen, consumption ct
the transition equation Wt +1 = Wt ct relates current state and
control to future state
It is possible to eliminate the control variable and write the problem as
V ( Wt ) =
max
W t +1 2[0,W t ]
u (Wt
Wt +1 ) + βV (Wt +1 ) (?)
(it happens to be sometime easier to use)
Gaballo and Portier (TSE)
M1
10/12
14 / 24
4. In…nite Horizon Dynamic Programming Approach
A functionnal equation
V (Wt ) =
max
W t +1 2[0,W t ]
u ( Wt
Wt +1 ) + βV (Wt +1 ) (?)
The unknown of this equation is not a number, but a function
The equation is also known as a Bellman equation
Time does not enter in the equation: stationarity
Assume for a moment that this equation has a solution V that is
di¤erentiable
Gaballo and Portier (TSE)
M1
10/12
15 / 24
4. In…nite Horizon Dynamic Programming Approach
Finding the optimal consumption plan
V (Wt ) =
max
W t +1 2[0,W t ]
u ( Wt
Wt +1 ) + βV (Wt +1 ) (?)
FOC is u 0 (ct ) = βV 0 (Wt +1 )
(?) implies that V 0 (Wt ) = u 0 (ct )
0
0
t +1
since V 0 (Wt ) = u 0 (ct ) + ∂W
∂W t ( u (ct ) + βV (Wt +1 ))
envelope theorem
We therefore have u 0 (c ) = βu 0 (c 0 )
Euler equation again
Gaballo and Portier (TSE)
M1
10/12
16 / 24
4. In…nite Horizon Dynamic Programming Approach
Policy function
Denote ct = φ(Wt ) the policy function
Note that optimal choice today depends only on the sate variable W
Then Wt +1 = Wt
φ(Wt ) = ψ(Wt )
so that the FOC is u 0 (φ(Wt )) = βu 0 (φ(Wt
φ(Wt ))) for all Wt
This is a functional equation whose unknown is the policy function
Gaballo and Portier (TSE)
M1
10/12
17 / 24
4. In…nite Horizon Dynamic Programming Approach
Analytical Example
Assume u (ct ) = log ct
Guess V (Wt ) = A + B log(Wt )
ON THE BLACKBOARD
Gaballo and Portier (TSE)
M1
10/12
18 / 24
5. Introducing Uncertainty
Taste shocks
Assume that the consumer is more or less hungry, so that utility is
εu (ct )
ε is a random variable that is observable today but whose future values
are unknown
ε takes 2 values εh > εl
It follows a …rst order Markov process (meaning that what happens
today depends only on what happened yesterday)
The evolution of ε is given by the transition matrix Π:
π ll
π hl
π lh
π hh
with for example π lh = Prob(εt +1 = εh jεt = εl )
π ih + π il = 1 for i = l, h
Gaballo and Portier (TSE)
M1
10/12
19 / 24
5. Introducing Uncertainty
The Bellman Equation
Now the agent cares about current and expected utility
The state is now both Wt and εt
The Bellman equation is, for all (Wt , εt )
V (Wt , εt ) = max εt u (Wt
Wt
Wt +1 ) + βEεt +1 jεt V (Wt +1 , εt +1 )
Eεt +1 jεt is computed using the transition matrix Π
Gaballo and Portier (TSE)
M1
10/12
20 / 24
5. Introducing Uncertainty
Solution
V (Wt , εt ) = max εt u (Wt
Wt
Wt +1 ) + βEεt +1 jεt V (Wt +1 , εt +1 )
FOC is
εt u 0 (Wt
Wt +1 ) = βEεt +1 jεt VW (Wt +1 , εt +1 )
This writes
εt u 0 (Wt
Wt +1 ) = βEεt +1 jεt εt +1 ut +1 (Wt +1
Wt + 2 )
Write the policy function as Wt +1 = ψ(Wt , εt )
The stochastic Euler equation becomes
ε t u 0 ( Wt
Wt +1 ) = βEεt +1 jεt εt +1 ut +1 (ψ(Wt , εt )
Gaballo and Portier (TSE)
M1
ψ(ψ(Wt , εt ), εt +1 ))
10/12
21 / 24
Discrete Choice
The problem
Assume that the cake grows if not eaten: W 0 = ρW
Assume that it has to be consumed in one period
(wine drinking or tree cutting more that cake eating)
Problem: shall I eat today a small cake or wait tomorrow for a bigger
one?
This is an optimal stopping model
Useful in economics (buying a car, a house, accepting a job;, stopping
university)
Gaballo and Portier (TSE)
M1
10/12
22 / 24
Discrete Choice
Assumptions
V E (W , ε) and V N (W , ε) are the values of Eating and waiting (Not
eating the cake)
V E (Wt , εt ) = εt u (Wt ) (the model stops)
V N (Wt , εt ) = βEεt +1 jεt V (ρWt , εt +1 ) where
V (Wt , εt ) = max V E (Wt , εt ), V N (Wt , εt ) for all (Wt , εt )
Gaballo and Portier (TSE)
M1
10/12
23 / 24
Discrete Choice
The tradeo¤
The cake grows
good to wait
Preference for the present β
Taste shocks ε
of waiting)
Gaballo and Portier (TSE)
good not to wait
good to wait if appetite is low today (option value
M1
10/12
24 / 24