Lecture Notes 8:
Calculus of Variations and Optimal Control
Part 1: Calculus of Variations
Peter J. Hammond
Autumn 2013, revised 2014
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
1 of 18
Outline
Introduction
Vickrey–Mirrlees Model
Typical Problem
Economic Application
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
2 of 18
Vickrey–Mirrlees Model
Problem: how much to pay workers of different skills.
Goal: achieve fairness while preserving incentives.
References: William S. Vickrey (1945)
“Measuring Marginal Utility by Reactions to Risk”
Econometrica 13: 319–333.
James A. Mirrlees (1971)
“An Exploration in the Theory of Optimal Income Taxation”
Review of Economic Studies 38: 175–208.
Let n ∈ R+ denote a person’s skill level, defined to mean
that there is a constant rate of marginal substitution
of n1 /n2 between hours of work supplied by workers
of the two skill levels n1 and n2 .
Thus, a worker’s productivity is proportional to n, personal skill.
Assume that the distribution of workers’ skills can be described
by a continuous density function R+ 3 n 7→ f (n) ∈R R+
∞
which, like a probability density function, satisfies 0 f (n)dn = 1.
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
3 of 18
Objective and Constraints
“Macro” model with a “representative consumer/worker”
whose preferences for consumption/labour supply pairs (c, `) ∈ R2+
are represented by the utility function u(c) − v (`),
where u 0 > 0, v 0 > 0, u 00 < 0 v 00 > 0.
The social objective
R ∞is to maximize
the utility integral 0 [u(c(n)) − v (`(n))]f (n)dn
w.r.t. the functions R+ 3 n 7→ (c(n), `(n)) ∈ R2+ .
The resource balance constraint takes the form C ≤ F (L) where
R∞
I C :=
0 c(n)f (n)dn is mean consumption;
R∞
I L :=
0 n`(n)f (n)dn is mean effective labour supply.
The aggregate production function R+ 3 L 7→ F (L) ∈ R+
is assumed to satisfy F 0 (L) > 0 and F 00 (L) ≤ 0 for all L ≥ 0.
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
4 of 18
Pseudo First-Order Conditions
Consider the Lagrangian
Z ∞
L(c(·), `(·)) :=
[u(c(n)) − v (`(n))]f (n)dn
0
Z ∞
Z ∞
−λ
c(n)f (n)dn − F
n`(n)f (n)dn
0
0
as a functional (rather than a mere function)
of the functions R+ 3 n 7→ (c(n), `(n)) ∈ R2+ .
We derive “pseudo” first-order conditions by pretending
∂L
∂L
that the derivatives
and
both exist, for all n ≥ 0.
∂c(n)
∂`(n)
This gives the pseudo first-order conditions
0 =
0 =
∂L
∂c(n)
∂L
∂`(n)
= [u 0 (c(n)) − λ]f (n)
= −v 0 (`(n))f (n) + λF 0 (L)nf (n)
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
5 of 18
Marxist First Best
For any skill level n such that f (n) > 0, these two equations
0 = [u 0 (c(n)) − λ]f (n)
and
− v 0 (`(n))f (n) + λF 0 (L)nf (n)
imply that:
I
I
u 0 (c(n)) = λ and so c(n) = c ∗ ,
where the constant c ∗ uniquely solves u 0 (c ∗ ) = λ
(“to each according to their need”);
d`
v 0 (`(n)) = λF 0 (L)n, implying that v 00 (`(n)) ·
= λF 0 > 0,
dn
d`
so
> 0 (“from each according to their ability”)
dn
Exercise
Use concavity arguments to prove
that this is the (essentially unique) solution.
What makes this solution practically infeasible?
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
6 of 18
Outline
Introduction
Vickrey–Mirrlees Model
Typical Problem
Economic Application
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
7 of 18
Problem Formulation
The calculus of variations is used to optimize a functional
that maps functions into real numbers.
A typical problem is to choose a function [t0 , t1 ] 3 t 7→ x(t) ∈ R,
often denoted simply by x,
in order to maximize the integral objective functional
Z t1
J(x) =
F (t, x(t), ẋ(t))dt
t0
subject to the fixed end point conditions x(t0 ) = x0 , x(t1 ) = x1 .
A variation involves moving away from the first path x
to the variant path x + u,
where u denotes the differentiable function [t0 , t1 ] 3 t 7→ u(t) ∈ R,
and ∈ R is a scalar.
To ensure that the end point conditions x(t0 ) + u(t0 ) = x0
and x(t1 ) + u(t1 ) = x1 remain satisfied by x + u,
one imposes the conditions u(t0 ) = u(t1 ) = 0.
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
8 of 18
Toward a Necessary First-Order Condition
A maximum is a path x∗ satisfying the end point conditions
such that J(x∗ ) ≥ J(x) for any alternative path x
that also the satisfies the end point conditions.
A necessary condition for x∗ to maximize J(x) w.r.t. x
is that J(x∗ ) ≥ J(x∗ + u) for all small .
Alternatively, the function
R 3 7→ fx∗ ,u () := J(x∗ + u)
must satisfy, for all small , the inequality
fx∗ ,u (0) = J(x∗ ) ≥ J(x∗ + u) = fx∗ ,u ()
In case the function 7→ fx∗ ,u () is differentiable at = 0,
a necessary first-order condition is therefore fx0∗ ,u (0) = 0.
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
9 of 18
Evaluating the Derivative
Our definitions of the functions J and fx∗ ,u imply that
∗
Z
t1
fx∗ ,u () = J(x + u) =
F (t, x ∗ (t) + u(t), ẋ ∗ (t) + u̇(t))dt
t0
Differentiating the integrand w.r.t. implies that
Z t1
[Fx0 (t)u(t) + Fẋ0 (t)u̇(t)]dt
fx0∗ ,u (0) =
t0
where for each t ∈ [t0 , t1 ], the partial derivatives Fx0 (t) and Fẋ0 (t)
of F (t, x, ẋ) are evaluated at the triple (t, x ∗ (t), ẋ ∗ (t)).
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
10 of 18
Integrating by Parts
The product rule for differentiation implies that
d 0
d 0
[F (t)u(t)] =
F (t) u(t) + Fẋ0 (t)u̇(t)
dt ẋ
dt ẋ
and so, integrating by parts, one has
Z t1
Z
t1 0
0
Fẋ (t)u̇(t)dt = |t0 Fẋ (t)u(t) −
t0
t1
t0
d 0
F (t) u(t)dt
dt ẋ
But the end point conditions imply that u(t0 ) = u(t1 ) = 0,
so the first term on the right-hand side vanishes.
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
11 of 18
The Euler Equation
Rt d 0 Rt
Substituting − t01 dt
Fẋ (t) u(t)dt for the term t01 Fẋ0 (t)u̇(t)dt
Rt
in the equation fx0∗ ,u (0) = t01 [Fx0 (t)u(t) + Fẋ0 (t)u̇(t)]dt,
then recognizing the common factor u(t), we finally obtain
Z t1 d 0
0
0
Fx (t) − Fẋ (t) u(t)dt
fx∗ ,u (0) =
dt
t0
The first-order condition is fx0∗ ,u (0) = 0
for every differentiable function t 7→ u(t)
satisfying the two end point conditions u(t0 ) = u(t1 ) = 0.
This condition holds
iff the integrand is zero for (almost) all t ∈ [t0 , t1 ],
d 0
which is true iff the Euler equation dt
Fẋ (t) = Fx0 (t)
holds for (almost) all t ∈ [t0 , t1 ].
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
12 of 18
Outline
Introduction
Vickrey–Mirrlees Model
Typical Problem
Economic Application
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
13 of 18
Are We Saving Too Little?
Kenneth Arrow, Gretchen Daily, Partha Dasgupta, Paul Ehrlich,
Lawrence Goulder, Geoffrey Heal, Simon Levin, Karl-Göran Mäler,
Stephen Schneider, David Starrett and Brian Walker (2004)
“Are We Consuming Too Much?”
Journal of Economic Perspectives 18: 147–172.
Macroeconomic variation of the Solow–Swan growth model.
Given a capital stock K , output Y is given by
the production function Y = f (K ), where f 0 > 0, and f 00 ≤ 0.
Net investment = gross investment, without depreciation.
So given capital K and consumption C , investment I is given by
I = K̇ = f (K ) − C
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
14 of 18
The Ramsey Problem and Beyond
The economy’s intertemporal objective is taken to be
Z
T
e −rt U(C (t))dt =
0
Z
T
e −rt U(f (K ) − K̇ )dt
0
Frank Ramsey (1928) assumed T = ∞ (infinite horizon)
and r = 0 (no discounting).
Nicholas Stern (of the Stern Report on Climate Change)
and others take:
I
T = ∞;
I
r as the hazard rate in a Poisson process
that determines when extinction occurs;
this implies that e −rt is the probability
that the human race has not become extinct
at or before time t.
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
15 of 18
Applying the Calculus of Variations
We apply the calculus
of variations
R T −rt
to the objective 0 e U(f (K ) − K̇ )dt
with the end conditions K (0) = K̄ , which is exogenous,
and K (T ) = 0 at the finite time horizon T .
Euler’s equation takes the form
d 0
dt FK̇ (t)
= FK0 (t)
where F (t, K , K̇ ) = e −rt U(f (K ) − K̇ ) = e −rt U(C ).
So Euler’s equation becomes
d
−rt U 0 (C )]
dt [−e
= e −rt U 0 (C )f 0 (K ).
Equivalently, after evaluating the time derivative,
−U 00 (C )Ċ e −rt + rU 0 (C )e −rt = e −rt U 0 (C )f 0 (K )
Cancelling the common factor e −rt and dividing by U 0 (C ) > 0,
then rearranging, one obtains
−
U 00 (C )
Ċ = f 0 (K ) − r
U 0 (C )
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
16 of 18
Further Interpretation
Define the (negative) elasticity of marginal utility as
η(C ) := −
U 00 (C )C
d ln U 0 (C )
=− 0
d ln C
U (C )
This is related to the curvature of the utility function,
and to how quickly marginal utility U 0 (C ) decreases as C increases.
Rearranging the equation −U 00 (C )Ċ /U 0 (C ) = f 0 (K ) − r yet again,
one obtains the equation
η(C )
Ċ
= f 0 (K ) − r
C
whose left hand side is the proportional rate of consumption growth
multiplied by the elasticity of marginal utility,
or the elasticity of an intertemporal marginal rate of substitution.
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
17 of 18
Final Recommendation
Morton I. Kamien and Nancy L. Schwartz (2012)
Dynamic Optimization, Second Edition:
The Calculus of Variations and Optimal Control
in Economics and Management (Dover Publications)
University of Warwick, EC9A0 Maths for Economists
Peter J. Hammond
18 of 18