Hindawi Publishing Corporation
Advances in Difference Equations
Volume 2008, Article ID 654267, 14 pages
doi:10.1155/2008/654267
Research Article
Infinite Horizon Discrete Time Control Problems
for Bounded Processes
Joël Blot1 and Naı̈la Hayek1, 2
1
Laboratoire Marin Mersenne, Université Paris I, Panthéon Sorbonne, Centre Pierre-Mendès-France,
90 rue de Tolbiac, 75013 Paris, France
2
Université de Franche-Comté, 45 Avenue de l’Observatoire, 25030 Besançon, France
Correspondence should be addressed to Naı̈la Hayek, hayek@univ-paris1.fr
Received 15 October 2008; Accepted 27 December 2008
Recommended by Leonid Shaikhet
We establish Pontryagin Maximum Principles in the strong form for infinite horizon optimal
control problems for bounded processes, for systems governed by difference equations. Results
due to Ioffe and Tihomirov are among the tools used to prove our theorems. We write necessary
conditions with weakened hypotheses of concavity and without invertibility, and we provide new
results on the adjoint variable. We show links between bounded problems and nonbounded ones.
We also give sufficient conditions of optimality.
Copyright q 2008 J. Blot and N. Hayek. This is an open access article distributed under the
Creative Commons Attribution License, which permits unrestricted use, distribution, and
reproduction in any medium, provided the original work is properly cited.
1. Introduction
The first works on infinite horizon optimal control problems are due to Pontryagin and
his school 1. They were followed by few others 2–6. We consider in this paper an
infinite horizon Optimal Control problem in the discrete time framework. Such problems are
fundamental in the macroeconomics growth theory 7–10 and see references of 11. Even
in the finite horizon case, the discrete time framework presents significant differences from
the continuous time one. Boltianski 12 shows that in the discrete time case, a convexity
condition is needed to guarantee a strong Pontryagin Principle while this last one can be
obtained without such condition in the continuous time setting. We study our problem in the
space of bounded sequences ∞ , which allows us to use Analysis in Banach spaces instead of
using reductions to finite horizon problems as in 5, 6. According to Chichlinisky 13, 14, the
space of bounded sequences was first used in economics by Debreu 15. It can also be found
in 7, 8, 16. We obtain Pontryagin Maximum Principles in the strong form using weaker
convexity hypotheses than the traditional ones and without invertibility 5. When we study
the problem in a general sequence space it turns out that the infinite series will not always
2
Advances in Difference Equations
converge. Therefore we present other notions of optimality that are currently used, notably
in the economic literature, see 3, 4, 9 and we show how our problem can be related to these
other problems. We end the paper by establishing sufficient conditions of optimality.
Now we briefly describe the contents of the paper. In Section 2 we introduce the
notations and the problem, then we state Theorems 2.1 and 2.2 which give necessary
conditions of optimality namely the existence of the adjoint variable in the space 1 satisfying
the adjoint equation and the strong Pontryagin maximum principle. In Section 3 we prove
these theorems through some lemmas and using results due to Ioffe-Tihomirov 17. In
Section 4 we introduce some other notions of optimality for problems in the nonbounded case
and we show links with our problem. For example, we show that when the objective function
is positive then a bounded solution is a solution among the unbounded processes. Finally we
give sufficient conditions of optimality for problems in the bounded and unbounded cases
adapting for each case the approprate transversality condition.
2. Pontryagin maximum principles for bounded processes
We first precise our notations. Let Ω be a nonempty open convex subset of Rn and U a
nonempty compact subset of Rm . Let Φ : Ω × U → R and, for all t ∈ N, ft : Ω × U → Rn . We
xt t , ut t .
set x, u
Define dom J {x, u ∈ ΩN × UN : t ∞0 βt Φxt , ut converges in R}.
For every x ∈ Rn define Cx as the closure of the set of terms of the sequence x. If
x ∈ ∞ N, Rn , Cx is compact. We set X {x xt t ∈ ∞ N, Rn , such that Cx ⊂ Ω}.X is
thus the set of the bounded sequences which are in the interior of Ω. Note that X is a convex
open subset of ∞ N, Rn since Ω is open and convex. We set U {u ut t ∈ UN }. Define
Admη
{x, u ∈ ΩN × U : xt 1 ft xt , ut , t ∈ N, and x0 η}; it is the set of admissible
processes with respect to the considered dynamical system.
Let β ∈ 0, 1 . We consider first the following problem P1 :
∞
βt Φ xt , ut
Maximize J x, u
t 0
xt 1 ft xt , ut , t ∈ N
P1
x0 η
x, u ∈ X × U
which can be written as follows.
P1 Maximize Jx, u when x, u ∈ Admη ∩ X × U .
be a solution of P1 . Assume the following.
Theorem 2.1. Let x,
u
i For all u ∈ U, the mapping x → Φx, u is of class C1 on Ω and for all t ∈ N, the mapping
x → ft x, u is Fréchet-differentiable on Ω.
ii For all t ∈ N, for all xt ∈ Ω, for all ut , ut ∈ U, for all α ∈ 0, 1, there exists ut ∈ U such
that
Φ xt , ut ≥ αΦ xt , ut
ft xt , ut
αft xt , ut
1 − α Φ xt , ut
1 − α ft xt , ut .
2.1
J. Blot and N. Hayek
3
iii For any compact set C ⊂ Ω, there exists a constant KC such that for all t ∈ N, for all
x, x ∈ C, for all u ∈ U, ft x, u ≤ KC and Dxt ft x, u − Dxt ft x , u ≤ KC x − x .
iv There exists r > 0 such that Bx,
r ⊂ X and for all xt , ut ∈ Bxt , r × U,
sup Dxt ft xt , ut < 1.
2.2
t≥0
Then there exists pt
a pt
pt
1
1 t
∈ 1 N, Rn such that
◦ Dxt ft xt , u
t
βt Dxt Φxt , u
t , for all t,
b βt Φxt , u
t
pt 1 , ft xt , u
t ≥ βt Φxt , ut
c limt →
0.
∞
pt
pt 1 , ft xt , ut , for all t, for all ut ∈ U,
Comments
For continuous time problems, one does not need conditions to obtain a strong Pontryagin
maximum principle, both in the finite horizon case see, e.g., 18 and in the infinite horizon
case see, e.g., 5 . But for discrete time problems, strong Pontryagin principles cannot hold
without an additional assumption namely a convexity condition, as Boltyanski shows in
12 for the finite horizon framework. Condition ii comes from the Ioffe and Tihomirov
book 17. It generalizes the usual convexity condition used to garantee a strong Pontryagin
maximum principle. The usual condition is: U convex subset, Φ concave with respect to u
and for every t, ft affine with respect to u. It implies condition ii . In iii the condition
ft x, u ≤ KC is satisfied when ft is continuous since U is compact and the condition
Dxt ft x, u − Dxt ft x , u ≤ KC x − x is satisfied when Dx2 t ft exists and is continuous.
Conclusion a is the adjoint equation, conclusion b is the strong Pontryagin
maximum principle and conclusion c is a transversality condition at infinity. In our case
c is immediately obtained since pt 1 t is in 1 N, Rn , but in general nonbounded cases it
is very delicate to obtain such a conclusion. 9
In the next theorem we consider the autonomous case. Thus the hypotheses are
simpler and easier to manipulate.
u
be a solution of P1 . Assume that the following
Theorem 2.2. Let ft f for all t ∈ N. Let x,
conditions are fulfilled.
i For all u ∈ U, the mappings x → Φx, u and x → fx, u are of class C1 on Ω.
ii For all t ∈ N, for all xt ∈ Ω, for all ut , ut ∈ U, for all α ∈ 0, 1, there exists ut ∈ U such
that
1 − α Φ xt , ut ,
Φ xt , ut ≥ αΦ xt , ut
αf xt , ut
1 − α f xt , ut .
f xt , ut
t
iii supt≥0 Dxt fxt , u
Then there exists pt
satisfied.
1 t
2.3
< 1.
∈ 1 N, Rn such that the assertions (a), (b), and (c) of Theorem 2.1 are
4
Advances in Difference Equations
3. Proofs of Theorems 2.1 and 2.2
Proof of Theorem 2.1
First part
The first part of the proof goes through several lemmas.
Lemma 3.1. J is well-defined and under hypothesis (i) of Theorem 2.1, for all u, the mapping x →
∞ t
Jx, u is of class C1 and one has, for all δx ∈ ∞ N, Rn , Dx Jx, u δx
t 0 β Dxt Φxt , ut δxt .
For the proof see 19.
ft xt , ut − xt
We set Fx, u
1 t≥0
for all x, u ∈ X × U.
Lemma 3.2. Assume that hypothesis (iii) of Theorem 2.1 holds. Then for x, u ∈ ∞ N, Ω × U,
one has Fx, u ∈ ∞ N, Rn . Moreover, if in addition hypotheses (i) and (iv) of Theorem 2.1 hold,
r in ∞ N, Rn and for all
then for all u, the mapping x → Fx, u is of class C1 on the ball Bx,
n
δx ∈ ∞ N, R , one has Dx Fx, u δx Dxt ft xt , ut δxt − δxt 1 t∈N .
Proof. Let x, u ∈ ∞ N, Ω × U. Let KC be the constant of iii with C
Cx . So for all
t ∈ N, ft xt , ut − xt 1 ≤ KC supt∈N xt . So one has Fx, u ∈ ∞ N, Rn .
Assume now that hypotheses i and iv of Theorem 2.1 hold. Let us show that x →
r . Take x0 , u ∈ Bx,
r × U. Let > 0 be given. Let x ∈ Bx,
r
Fx, u is of class C0 on Bx,
0
be such that x − x < min{2r, /2}. Then, for all t ∈ N, ft xt , ut − xt 1 ft xt0 , ut − xt0 1 ≤
xt 1 − xt0 1 ≤ supyt ∈x0 ,xt Dxt ft yt , ut · xt − xt0
xt 1 − xt0 1 <
ft xt0 , ut − ft xt , ut
t
/2 supt≥0 Dxt ft yt , ut
1 < under iv , which implies that Fx, u − Fx0 , u ∞ < .
r . Take
Let us now show that x → Fx, u is Fréchet-differentiable on Bx,
r × U. Let > 0 be given. Let x ∈ Bx,
r be such that x −
x0 , u ∈ Bx,
< min{2r, /2}. Then, for all t ∈ N, ft xt , ut − xt 1
ft xt0 , ut − xt0 1
x0
0
0
0
0
xt − xt0 ≤
Dxt ft xt , ut xt − xt − xt 1 − xt 1 ≤ supyt ∈x0 ,xt Dxt ft yt , ut − Dxt ft xt , ut
supyt ∈x0 ,xt Dxt ft yt , ut
t
Dxt ft xt0 , ut
t
xt − xt0
< under iii . But this implies
that Fx, u − Fx , u − Dxt ft xt0 , ut xt − xt0 −
is Fréchet-differentiable at x0 and Dx Fx0 , u δx
0
xt 1 − xt0 1 t∈N ∞ < . Thus x
Dxt ft xt0 , ut δxt − δxt 1 t∈N .
Dx Fx, u at x0 let KB be the constant of
→ Fx, u
hypothesis
To show the continuity of x →
r be such that x −
iii corresponding to Bx,
r . Let > 0 be given and let x ∈ Bx,
≤
x0 < min{2r, /KB }· Dx Fx, u − Dx Fx0 , u ∞ ≤ supt∈N Dxt ft xt0 , ut − Dxt ft xt , ut
0
1
supt∈N KB xt − xt < . So F is of class C .
Lemma 3.3. Under hypothesis (ii) of Theorem 2.1, for all x ∈ X, for all u , u ∈ U, for all α ∈ 0, 1,
there exists u ∈ U such that
J x, u ≥ αJ x, u
1 − α J x, u
F x, u
1 − α F x, u .
αF x, u
3.1
ut t ∈ U, u
ut t ∈ U and α ∈ 0, 1. Hypothesis ii of
Proof. Let x xt t ∈ X, u
Theorem 2.1 implies for all t ∈ N the existence of ut ∈ U such that
J. Blot and N. Hayek
5
Φ xt , ut ≥ αΦ xt , ut
1 − α Φ xt , ut
ft xt , ut
αft xt , ut
1 − α ft xt , ut .
3.2
∞
∞
βt Φ xt , ut ≥ α βt Φ xt , ut
βt Φ xt , ut
1 − α
t 0
t0
t 0
ft xt , ut − xt 1 t α ft xt , ut − xt 1 t 1 − α ft xt , ut − xt 1 t .
3.3
Therefore we obtain
∞
Set u
ut t , so u ∈ U and satisfies the required relations.
Lemma 3.4. Under hypotheses (i) and (iv) of Theorem 2.1, Im Dx Fx,
u
∞ N, Rn .
Proof. Since Dx Fx, u δx Dxt ft xt , ut δxt − δxt 1 t∈N , δx0 0, the problem is a problem of
bounded solutions of first-order linear difference equations.
Let Mt t≥0 ∈ ∞ N, Rn , Rn . Assume that supt≥1 Mt < 1. Then for all bt t≥0 ∈
∞ N, Rn there exists a unique ht t≥0 ∈ ∞ N, Rn such that for all t ≥ 0,
ht
− Mt ht
1
3.4
bt ,
where h0 0.
Consider the operator T : ∞ N∗ , Rn → ∞ N∗ , Rn such that for all h ∈ ∞ N∗ , Rn ,
Th
T
I
ht − Mt−1 ht−1
t≥1
3.5
T where
I identity of ∞ N∗ , Rn
T h : 0, −M1 h1 , −M2 h2 , . . . , −Mt ht , . . . .
3.6
∗
Recall that the · ∞ norm of z ∈ Rn N is defined by z ∞ supt≥1 zt and that the norm of
a linear operator S between normed spaces is defined by S L sup z ≤1 Sz .
So T h ∞ supt≥1 − Mt ht ≤ supt≥1 Mt h ∞ . So T L ≤ supt≥1 Mt < 1. Since
T I T and T L < 1, T is invertible so it is surjective.
Dxt ft xt , u
t . Then under iv one has supt≥1 Dxt ft xt , u
t
< 1. So T is
Set Mt
u
∞ N, Rn .
surjective that is Im Dx Fx,
∗
Recall that ∞
N, R
1 N, R ⊕ 1d N, R where 1d N, R consists of all singular
functionals, see Aliprantis and Border 20. In fact it consists up to scalar multiples of all
extensions of the “limit functional” to ∞ .
cN, R , θx
If θ ∈ 1d N, R , then there exists k ∈ R such that for all x ∈ c
k·limt → ∞ xt . c being the space of convergent sequences having a limit in R.
6
Advances in Difference Equations
∗
N, Rn
1 N, Rn ⊕ 1d N, Rn . If θ ∈ 1d N, Rn then θ
θ1 , θ2 , . . . , θn
Lemma 3.5 ∞
d
i
n
where θ ∈ 1 N, R for every i 1, . . . , n. So there exists k k1 , . . . , kn ∈ R such that for all
k, limt → ∞ xt .
x ∈ cN, Rn , θx
Second part
Our optimal control problem can be written as the following abstract static optimisation
problem in a Banach space:
Maximize J x, u
F x, u
0
x, u ∈ X × U
3.7
that satisfies all conditions of Theorem 4.3, Ioffe-Tihomirov 17. So we can apply this theorem
∗
N, Rn , not all zero, λ0 ≥ 0, such that:
and obtain the existence of λ0 ∈ R, P ∈ ∞
λ0 Dx J x,
u
P 0,
Dx F ∗ x,
u
P, F x,
u
≥ λ0 J
P, F x,
u ,
PMP
λ0 J
AE
3.8
∀u ∈ U.
AE denotes the adjoint equation of this problem and PMP the Pontryagin maximum
principle. They can be written, respectively:
λ0
∞
βt Dxt Φ xt , u
t δxt
t 0
λ0
∞
P, Dxt ft xt , u
t δxt − δxt 1 t≥0
∀δx ∈ ∞ N, Rn such that δx0
βt Φ xt , u
t
t 0
≥ λ0
∞
0,
3.9
t − xt 1 t≥0
P, ft xt , u
βt Φ x t , ut
t 0
Set P
0,
P, ft xt , ut − xt 1 t≥0 ,
∀u ∈ U
p θ where p ∈ 1 N, Rn and θ ∈ 1d N, Rn .
AE becomes:
λ0
∞
t 0
βt Dxt Φ xt , u
t δxt
∞
∞
t δxt −
pt 1 , Dxt ft xt , u
pt 1 , δxt
t 0
θ, Dxt ft xt , u
t δxt − δxt 1 t≥0
t 0
0,
∀δx ∈ ∞
1
N, Rn .
3.10
J. Blot and N. Hayek
7
So we get
∞
t
λ0 βt Dxt Φ xt , u
t 0
t pt
Dxt f ∗t xt , u
1
− pt , δxt
t δxt − δxt 1 t≥0 ,
− θ, Dxt ft xt , u
∀δx ∈ ∞ N, R
n
3.11
with δx0
0.
Let z be arbitrarily chosen in Rn and let t ≥ 1 be in N. Consider the sequence δxs
as follows:
δxs
⎧
⎨z,
if s
⎩0,
if s / t.
t
s
defined
3.12
So one has Dxs fs xs , u
s δxs − δxs 1 0 if s ≥ t 1, hence Dxs fs xs , u
s δxs − δxs 1 s ∈ c0 ⊂ c.
s δxs − δxs 1 s k, lims → ∞ Dxs fs xs , u
s δxs −
Thus, it holds that θ, Dxs fs xs , u
k, 0 0.
δxs 1 ∞
s
s , u
s
Dxs f ∗s xs , u
s ps 1 − ps , δxs λ0 βt Dxt Φxt , u
t
Now
s 1 λ0 β Dxs Φx
∗
t pt 1 − pt , z.
Dxt f t xt , u
Therefore, for all t ≥ 1 and for all z ∈ Rn one has
λ0 βt Dxt Φ xt , u
t
t pt
Dxt f ∗t xt , u
1
− pt , z
0
3.13
0,
∀t ≥ 1,
3.14
which implies
λ0 βt Dxt Φ xt , u
t
t pt
Dxt f ∗t xt , u
1
− pt
that is,
pt
pt
1
t ,
λ0 βt Dxt Φ xt , u
◦ Dxt ft xt , u
t
∀t ≥ 1,
3.15
PMP becomes:
λ0
∞
βt Φ xt , u
t − Φ xt , ut
t 0
t − ft xt , ut t≥0 ≥ 0,
P, ft xt , u
∀u ∈ U.
t −Φxt , ut
p, ft xt , u
t −ft xt , ut t≥0 θ, ft xt , u
t −ft xt , ut
So λ0 t ∞0 βt Φxt , u
for all u ∈ U.
Consider, for all ut ∈ U, the sequences ut s∈N defined as follows:
uts
⎧
⎨u
t ,
if s / t
⎩u ,
if s
t
t.
3.16
t≥0 ≥0
3.17
8
Advances in Difference Equations
Since the inequality holds for every u ∈ U, we obtain
λ0 βt Φ xt , u
t − Φ xt , ut
t − ft xt , ut
using θ, ft xt , u
t≥0 t − ft xt , ut ≥ 0,
pt 1 , ft xt , u
0 as ft xt , u
t − ft xt , ut
t≥0
∀ut ∈ U
3.18
is of finite support.
Lemma 3.6. (λ0 / 0).
∗
Proof. Recall we obtained the existence of λ0 ∈ R, P ∈ ∞
N, Rn , not all zero, λ0 ≥ 0, such
that:
λ0 Dx J x,
u
If λ0
P
Dx F ∗ x,
u
0.
3.19
0, then P 0 since Im Dx Fx,
u
∞ N, Rn .
Hence λ0 / 0. We can set it equal to one.
From Lemma 3.6 and the previous results, conclusions a and b are satisfied.
Conclusion c is a straightforward consequence of the belonging of pt 1 t to 1 N, Rn .
Lemma 3.7. (θ
0).
Proof. Indeed we obtained θ, Dxt ft xt , u
t δxt − δxt 1 t≥0 0, for all δx ∈ ∞ N, Rn . Using
n
u
∞ N, R one has θ, h 0, for all h ∈ ∞ N, Rn . Thus θ 0.
Im Dx Fx,
Proof of Theorem 2.2. Define F on X × U such that Fx, u
fxt , ut − xt 1 t≥0 . Under
hypothesis i of Theorem 2.2, for all u ∈ U, the mappings x → Jx, u and x → Fx, u
are of class C1 on X. The proof can be found in 19.
t . Then the proof goes
We consider the proof of Lemma 3.4 and we set Mt Dxt fxt , u
like that of Theorem 2.1.
4. Results for unbounded problems
We study now problems of maximization over admissible processes which are not necessarily
bounded when the optimal solution is bounded. So consider the following problems.
P2 Maximize Jx, u on dom J ∩ Admη .
∈ dom J ∩ Admη such that, for all x, u ∈ Admη ,
P3 Find x,
u
≥ lim supT → ∞ Tt 0 βt Φxt , ut .
Jx,
u
∈ Admη such that, for all x, u ∈ Admη ,
P4 Find x,
u
t − Tt 0 βt Φxt , ut ≥ 0.
lim infT → ∞ Tt 0 βt Φxt , u
∈ Admη such that, for all x, u ∈ Admη ,
P5 Find x,
u
t − Tt 0 βt Φxt , ut ≥ 0.
lim supT → ∞ Tt 0 βt Φxt , u
The optimality notion of P3 is called “the strong optimality,” that of P4 is called “the
overtaking optimality” and that of P5 the “weak overtaking optimality” in 3 in the
continuous-time framework . Many existence results of overtaking optimal solutions and
weakly overtaking optimal solutions are obtained in 3, 4. In 4 there are also results in
the discrete-time framework.
J. Blot and N. Hayek
9
is an optimal solution of P3 implies x,
u
is an optimal
Remark 4.1. Notice that x,
u
is an optimal solution of P5 .
solution of P4 which implies x,
u
is a bounded optimal solution of P4 then P3 and P4 reduce to
Moreover if x,
u
the same problem.
Lemma 4.2. The two following assertions hold.
is an optimal solution of problem (P2), (P3), (P4) or (P5) and x,
u
∈ X × U then
a If x,
u
is an optimal solution of problem P1 . Therefore Theorem 2.1 applies.
x,
u
is an optimal solution of problem (P3) or (P4) and
b Assume Φ ≥ 0 on Ω × U. If x,
u
∈ X × U then dom J Admη .
x,
u
Proof. a Since X × U ∩ Admη ⊂ dom J ∩ Admη ⊂ Admη , a bounded optimal solution
is a bounded optimal
of P2 or P3 is an optimal solution of P1 . Suppose now that x,
u
t − Tt 0 βt Φxt , ut ≥ 0, for all x, u ∈
solution of P4 that is lim infT → ∞ Tt 0 βt Φxt , u
∈ X × U this can be written Jx,
u
≥ lim supT → ∞ Tt 0 βt Φxt , ut , for all
Admη . Since x,
u
x, u ∈ Admη and so in particular for all x, u ∈ X × U.
In that case lim supT → ∞ Tt 0 βt Φxt , ut
Jx, u . The proof is analogous for P5 .
is an optimal solution of problem P3 and x,
u
∈ X × U, one has
b If x,
u
≥ lim supT → ∞ Tt 0 βt Φxt , ut , for all x, u ∈ Admη . Since Φ ≥ 0, the sequence
∞ > Jx,
u
Tt 0 βt Φxt , ut T is increasing and since it is also upper bounded it converges in R .
So lim supT → ∞ Tt 0 βt Φxt , ut
limT → ∞ Tt 0 βt Φxt , ut
Jx, u and x, u ∈ dom J.
Theorem 4.3. Let ft
f for all t ∈ N. One assumes the following conditions fulfilled:
i Φ ≥ 0 on Ω × U.
ii For all x ∈ Ω, there exists v ∈ U such that x
fx, v .
Then one has
a supx,u ∈dom J∩Admη Jx, u
supx,u ∈X×U ∩Admη Jx, u .
is an optimal solution of problem P1 , then it is an optimal solution of problems
b If x,
u
(P3), (P4), and (P5) which all reduce to the same problem.
Remark 4.4. b shows that under a nonnegativity assumption, solving the problem in the
space of bounded processes provides solutions for problems in spaces of admissible processes
which are not necessarily bounded. This type of results is in the spirit of Blot and Cartigny
21 where problems are studied in the continuous-time case.
Proof. a It is clear that the following inequality holds:
sup
x,u ∈dom J∩Admη
J x, u ≥
sup
x,u ∈X×U ∩Admη
J x, u .
4.1
10
Advances in Difference Equations
Let x, u ∈ dom J ∩ Admη . Let > 0 be given and let T
t
t>T β Φxt , ut ≤ . Set
xt
ut
if t ≤ T
⎩
⎧xT ,
⎨ut ,
if t > T,
if t ≤ T
4.2
⎩u , if t > T,
T
where uT is such that xT
fxT , uT .x
X × U ∩ Admη .
Since Φ ≥ 0 on Ω × U one has
∞
J x , x
⎧
⎨xt ,
T be such that 0 ≤
xt
t
ut
and u
t
are bounded and x , u ∈
βt Φ xt , ut
t 0
T
βt Φ xt , ut
t 0
≥
T
∞
βt Φ xT , uT
t T 1
4.3
βt Φ xt , ut
t 0
sup
x,u ∈X×U ∩Admη
T
J x, u ≥ J x , u ≥
βt Φ xt , ut
t 0
so we obtain
sup
x,u ∈X×U ∩Admη
J x, u ≥ J x, u .
4.4
Since this is true fo all > 0, letting → 0, we obtain
sup
x,u ∈X×U ∩Admη
J x, u ≥
sup
x,u ∈dom J∩Admη
J x, u .
4.5
b Since Φ ≥ 0, for all x, u ∈ Admη , the sequence Tt 0 βt Φxt , ut T is nonnegative and
nondecreasing so it converges in 0, ∞.
limT → ∞ Tt 0 βt Φxt , ut . Hence P3 , P4 reduce to
So lim supT → ∞ Tt 0 βt Φxt , ut
be an optimal solution of problem
the same problem. Similarly P5 reduces to it. Let x,
u
P1 and suppose it is not an optimal solution of problem P3 . So there exists x, u ∈ Admη
∞ that is
such that limT → ∞ Tt 0 βt Φxt , ut
∀R ∈ R,
∃TR ∈ N∗ ,
∀T ≥ TR ,
T
t 0
βt Φ xt , ut > R.
4.6
J. Blot and N. Hayek
11
Let R ∈ R and T
TR . Construct x xt t and u ut t as in a . Thus t ∞0 βt Φxt , ut
∞
T t
t
t 0 β Φxt , ut
t T 1 β ΦxT , uT ≥ R.
∞
Hence we obtain supx,u ∈X×U ∩Admη Jx, u ≥ R, so supx,u ∈X×U ∩Admη Jx, u
which contradicts the hypothesis so x,
u
is an optimal solution of problem P3 .
Following Michel, 22, for all t ∈ N and for all xt , xt 1 ∈ Ω × Ω, we define At xt , xt 1
as the set of the νt , ζt ∈ R × Rn for which there exists ut ∈ U satisfying νt ≤ βt Φxt , ut , ζt
fxt , ut − xt 1 . We also define Bt xt , xt 1 as the set of the νt , ζt ∈ R × Rn for which there
exists ut , αt ∈ U × Rn satisfying νt ≤ βt Φxt , ut , αk ζtk f k xt , ut − xtk 1 for all k 1, . . . , n.
Theorem 4.5. Let ft f for all t ∈ N. Let x,
u
be an optimal solution of problem P1 . One assumes
the following conditions fulfilled.
i Φ ≥ 0 on Ω × U.
ii For all x ∈ Ω, there exists v ∈ U such that x
fx, v .
t are Fréchet-differentiable at
iii For all t ∈ N, the mappings x → Φx, u
t and x → fx, u
xt .
iv For all t ∈ N, for all xt , xt
convex hull.
1
∈ Ω × Ω,coAt xt , xt
1
⊂ Bt xt , xt
1
where co denotes the
t is invertible.
v For all t ∈ N, Dxt fxt , u
Then there exists λ0 ∈ R , pt
a pt
pt
1
1 t
◦ Dxt fxt , u
t
t
b λ0 βt Φxt , u
∈ Rn
N
such that λ0 , p1 / 0 and
λ0 βt Dxt Φxt , u
t , for all t,
pt 1 , fxt , u
t ≥ λ0 βt Φxt , ut
pt 1 , fxt , ut , for all t, for all ut ∈ U.
Remark 4.6. Notice that condition iv is a convexity condition and that condition ii of
Theorem 2.2 implies this condition iv . Condition ii of Theorem 2.2 is equivalent to the
following condition: for all t, the set At xt , xt 1 is convex.
Proof. Use Theorem 4.3 of this paper and apply Theorem 3 in Blot-Chebbi 5.
5. Sufficient conditions of optimality
Let Ht xt , ut , pt
βt Φxt , ut
1
pt 1 , ft xt , ut , for all t ∈ N.
Theorem 5.1. Let x,
u
∈ X × U ∩ Admη where U is convex. One assumes that there exists
pt 1 t ∈ 1 N, Rn and that the following conditions are fulfilled.
i The mappings x, u → Φx, u and for all t ∈ N, x, u → ft x, u are of class C1 on
Ω × U.
ii pt
pt
1
◦ Dxt ft xt , u
t
t
iii βt Φxt , u
βt Dxt Φxt , u
t , for all t ≥ 1.
pt 1 , ft xt , u
t ≥ βt Φxt , ut
pt 1 , ft xt , ut , for all t, for all ut ∈ U.
iv The mapping Ht is concave with respect to xt , ut , for all t.
is an optimal solution of P1 .
Then x,
u
12
Advances in Difference Equations
t , pt 1
pt , for all t ≥ 1.
Proof. Notice that from ii , Dxt Ht xt , u
Let x, u ∈ X × U ∩ Admη . For all t, one has
t − βt Φ xt , ut
βt Φ xt , u
Ht xt , u
t , pt 1 − pt 1 , ft xt , ut − Ht xt , ut , pt 1
pt 1 , ft xt , ut
Ht xt , u
t , pt 1 − Ht xt , ut , pt 1 − Dxt 1 Ht 1 xt 1 , u
t 1 , pt 2 , xt 1 − xt
− Dut Ht xt , u
t , pt 1 , u
t , pt 1 , u
t − ut
Dut Ht xt , u
t − ut ,
5.1
1
therefore, we obtain
T
t − βt Φ xt , ut
βt Φ xt , u
t 0
T
t , pt 1 − Ht xt , ut , pt 1 − Dxt 1 Ht 1 xt 1 , u
t 1 , pt 2 , xt
Ht xt , u
t 0
t , pt 1 , u
− Dut Ht xt , u
t − ut
H0 x0 , u
0 , p1 − H0 x0 , u0 , p1
1
− xt
1
t , pt 1 , u
Dut Ht xt , u
t − ut
5.2
T
t , pt 1 − Ht xt , ut , pt 1
Ht xt , u
t 1
− Dxt Ht xt , u
t , pt 1 , xt − xt − Dut Ht xt , u
t , pt 1 , u
t − ut
− pT 1 , xT
1
− xT
T
1
t , pt 1 , u
Dut Ht xt , u
t − ut
t 1
Since for all t ∈ N, Ht is concave with respect to xt and ut , one has Ht xt , u
t , pt 1 −
t , pt 1 , xt − xt − Dut Ht xt , u
t , pt 1 , u
t − ut ≥ 0. Using hypothesis
Ht xt , ut , pt 1 − Dxt Ht xt , u
0 , p1 −H0 x0 , u0 , p1 ≥ 0 and using hypothesis iii , the first order
iii with t 0 gives H0 x0 , u
t , pt 1 , u
t − ut ≥ 0. Thus one has
necessary condition for the optimality of u
t is Dut Ht xt , u
T
t
t , u
t − βt Φxt , ut ≥ pT 1 , xT 1 − xT 1 .
t 0 β Φx
The hypothesis pt 1 t ∈ 1 N, Rn implies limt → ∞ pt 0 and since x and x belong to
x < ∞. Hence we obtain limT → ∞ pT 1 , xT 1 − xT 1 0
∞ N, Rn one has x − x ≤ x
t − βt Φxt , ut ≥ 0. That is Jx,
u
− Jx, u ≥ 0.
so limT → ∞ Tt 0 βt Φxt , u
Corollary 5.2. Let x,
u
∈ dom J ∩ Admη (resp., Admη ). If the hypotheses of the previous
theorem are satisfied except that pt 1 t ∈ 1 N, Rn is replaced by pt 1 t ∈ Rn N and if the following
hypothesis is also satisfied:
v lim infT →
∞
pT 1 , xT
1
− xT 1 0,
then x,
u
is a solution of (P3) (resp., (P4)).
Notice that if x,
u
∈ Admη with lim supT →
is a solution of P5 .
x,
u
∞
pT 1 , xT
1
− xT 1 0 we obtain that
J. Blot and N. Hayek
13
One can weaken the hypothesis of concavity of Ht with respect to xt and ut and replace
it by the concavity of Ht∗ with respect to xt as the following theorem shows. See 23 for a
quick survey of sufficient conditions.
maxut ∈U Ht xt , ut , pt 1 .
Let Ht∗ xt , pt 1
The maximum is attained since U is compact.
∈ X × U ∩ Admη . One assumes that there exists pt
Theorem 5.3. Let x,
u
that the following hypotheses are fulfilled.
1 t
∈ 1 N, Rn and
i For all u ∈ U, the mappings x → Φx, u and for all t ∈ N, x → ft x, u are of class C1 on
Ω.
ii Also iii of the previous theorem.
iv The mapping Ht∗ is concave with respect to xt , for all t.
Then x,
u
is an optimal solution of P1 .
Proof. Let x,
u
∈ X × U ∩ Admη and let x, u ∈ X × U ∩ Admη . For all t, one has
βt Φ xt , u
t − βt Φ xt , ut
t , pt 1 − Ht xt , ut , pt 1 − pt 1 , ft xt , ut − ft xt , ut
Ht xt , u
≥ Ht∗ xt , pt 1 − Ht∗ xt , pt 1 − pt 1 , xt 1 − xt 1
5.3
by the definition of Ht∗ and noticing that Ht xt , u
t , pt
1
Ht∗ xt , pt
1
. So we obtain
T
t − βt Φ xt , ut
βt Φ xt , u
t 0
≥ H0∗ x0 , p1 − H0∗ x0 , p1
Ht∗ xt , pt 1 − Ht∗ xt , pt 1 − pt , xt − xt
t 1
− pT 1 , xT
T
T
1
− xT
1
t , pt 1 , xt − xt − pT 1 , xT
Ht∗ xt , pt 1 − Ht∗ xt , pt 1 − Dxt Ht xt , u
t 1
1
− xT
1
.
5.4
Notice that H0∗ x0 , p1
H0∗ x0 , p1 . Now using Dxt Ht∗ xt , pt 1
Dxt Ht xt , u
t , pt 1 see
t −βt Φxt , ut ≥ H0∗ x0 , p1 −
Seierstad and Sydsaeter 24, page 390 we obtain Tt 0 βt Φxt , u
T
∗
t , pt 1 −Ht∗ xt , pt 1 − Dxt Ht∗ xt , pt 1 , xt −xt pT 1 , xT 1 − xT 1 . The
H0∗ x0 , p1
t 1 Ht x
T
∗
t
t
t − β Φxt , ut ≥ pT 1 , xT 1 − xT 1 .
concavity of Ht with respect to xt gives t 0 β Φxt , u
− Jx, u ≥ 0 follows as in the proof of the previous theorem.
Finally Jx,
u
Corollary 5.4. Let x,
u
∈ dom J ∩ Admη (resp., Admη ). If the hypotheses of the previous
n
theorem are satisfied except that pt 1 t ∈ 1 N, Rn is replaced by pt 1 t ∈ RN and if the following
hypothesis is also satisfied:
v lim infT →
∞
pT 1 , xT
1
− xT 1 then x,
u
is a solution of (P3) (resp., (P4)).
0,
14
∈ Admη with lim supT →
Notice that if x,
u
is a solution of P5 .
x,
u
Advances in Difference Equations
∞
pT 1 , xT
1
− xT 1 0 we obtain that
References
1 L. Pontryagin, V. Boltianski, R. Gramkrelidze, and E. Mitchenko, Théorie Mathématique des Processus
Optimaux, Mir, Moscow, Russia, 1974.
2 I. Ekeland and J. A. Scheinkman, “Transversality conditions for some infinite horizon discrete time
optimization problems,” Mathematics of Operations Research, vol. 11, no. 2, pp. 216–229, 1986.
3 D. A. Carlson, A. B. Haurie, and A. Leizarowitz, Infinite Horizon Optimal Control. Deterministic and
Stochastic Systems, Springer, Berlin, Germany, 2nd edition, 1991.
4 A. J. Zaslavski, Turnpike Properties in the Calculus of Variations and Optimal Control, vol. 80 of Nonconvex
Optimization and Its Applications, Springer, New York, NY, USA, 2006.
5 J. Blot and H. Chebbi, “Discrete time Pontryagin principles with infinite horizon,” Journal of
Mathematical Analysis and Applications, vol. 246, no. 1, pp. 265–279, 2000.
6 J. Blot and N. Hayek, “Infinite-horizon Pontryagin principles with constraints,” in Communications of
the Laufen Colloquium on Science, Laufen, Austria, A. Ruffing, A. Suhrer, and J. Suhrer, Eds., Berichte aus
der Mathematik, 2, pp. 1–14, Shaker, Aachen, Germany, 2007.
7 A. Araujo and J. A. Scheinkman, “Smoothness, comparative dynamics, and the turnpike property,”
Econometrica, vol. 45, no. 3, pp. 601–620, 1977.
8 A. Araujo and J. A. Scheinkman, “Notes on comparative dynamics,” in General Equilibrium, Growth,
and Trade, J. R. Green and J. A. Scheinkman, Eds., pp. 217–226, Academic Press, New York, NY, USA,
1979.
9 P. Michel, “Some clarifications on the transversality condition,” Econometrica, vol. 58, no. 3, pp. 705–
723, 1990.
10 N. L. Stokey, R. E. Lucas Jr., and E. C. Prescott, Recursive Methods in Economic Dynamics, Harvard
University Press, Cambridge, Mass, USA, 1989.
11 R. Grinold, “Convex infinite horizon programs,” Mathematical Programming, vol. 25, no. 1, pp. 64–82,
1983.
12 V. Boltianski, Commande Optimale des Systèmes Discrets. Traduit du Russe par A. Sossinsk, Mir, Moscow,
Russia, 1976.
13 G. Chichilnisky, “An axiomatic approach to sustainable development,” Social Choice and Welfare, vol.
13, no. 2, pp. 231–257, 1996.
14 G. Chichlinisky, “What is sustainable development?” Land Economics, vol. 73, no. 4, p. 46791, 1997.
15 G. Debreu, “Valuation equilibrium and Pareto optimum,” Proceedings of the National Academy of
Sciences of the United States of America, vol. 40, no. 7, pp. 588–592, 1954.
16 B. Peleg and H. Ryder, “On optimal consumption plans in a multi-sector economy,” Review of
Economics Studies, vol. 39, no. 2, pp. 159–169, 1972.
17 A. D. Ioffe and V. M. Tihomirov, Theory of Extremal Problems, vol. 6 of Studies in Mathematics and Its
Applications, North-Holland, Amsterdam, The Netherlands, 1979.
18 V. Alexéev, V. Tikhomirov, and S. Fomine, Commande Optimale, Mir, Moscow, Russia, 1982.
19 J. Blot and B. Crettez, “On the smoothness of optimal paths,” Decisions in Economics and Finance, vol.
27, no. 1, pp. 1–34, 2004.
20 C. D. Aliprantis and K. C. Border, Infinite-Dimensional Analysis, Springer, Berlin, Germany, 2nd edition,
1999.
21 J. Blot and P. Cartigny, “Optimality in infinite-horizon variational problems under sign conditions,”
Journal of Optimization Theory and Applications, vol. 106, no. 2, pp. 411–419, 2000.
22 P. Michel, “Programmes mathématiques mixtes. Application au principe du maximum en temps
discret dans le cas déterministe et dans le cas stochastique,” RAIRO Recherche Opérationnelle, vol. 14,
no. 1, pp. 1–19, 1980.
23 J. Blot and N. Hayek, “Sufficient conditions for infinite-horizon calculus of variations problems,”
ESAIM: Control, Optimisation and Calculus of Variations, vol. 5, pp. 279–292, 2000.
24 A. Seierstad and K. Sydsaeter, “Sufficient conditions in optimal control theory,” International Economic
Review, vol. 18, no. 2, pp. 367–391, 1977.