Hindawi Publishing Corporation
International Journal of Stochastic Analysis
Volume 2014, Article ID 201491, 17 pages
http://dx.doi.org/10.1155/2014/201491
Research Article
The Relationship between the Stochastic
Maximum Principle and the Dynamic Programming in
Singular Control of Jump Diffusions
Farid Chighoub and Brahim Mezerdi
Laboratory of Applied Mathematics, University Mohamed Khider, P.O. Box 145, 07000 Biskra, Algeria
Correspondence should be addressed to Brahim Mezerdi; bmezerdi@yahoo.fr
Received 7 September 2013; Revised 28 November 2013; Accepted 3 December 2013; Published 9 January 2014
Academic Editor: Agnès Sulem
Copyright © 2014 F. Chighoub and B. Mezerdi. This is an open access article distributed under the Creative Commons Attribution
License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly
cited.
The main objective of this paper is to explore the relationship between the stochastic maximum principle (SMP in short) and
dynamic programming principle (DPP in short), for singular control problems of jump diffusions. First, we establish necessary
as well as sufficient conditions for optimality by using the stochastic calculus of jump diffusions and some properties of singular
controls. Then, we give, under smoothness conditions, a useful verification theorem and we show that the solution of the adjoint
equation coincides with the spatial gradient of the value function, evaluated along the optimal trajectory of the state equation.
Finally, using these theoretical results, we solve explicitly an example, on optimal harvesting strategy, for a geometric Brownian
motion with jumps.
1. Introduction
In this paper, we consider a mixed classical-singular control
problem, in which the state evolves according to a stochastic
differential equation, driven by a Poisson random measure
and an independent multidimensional Brownian motion, of
the following form:
𝑑𝑥𝑡 = 𝑏 (𝑡, 𝑥𝑡 , 𝑢𝑡 ) 𝑑𝑡 + 𝜎 (𝑡, 𝑥𝑡 , 𝑢𝑡 ) 𝑑𝐵𝑡
̃ (𝑑𝑡, 𝑑𝑒) + 𝐺𝑡 𝑑𝜉𝑡 ,
+ ∫ 𝛾 (𝑡, 𝑥𝑡− , 𝑢𝑡 , 𝑒) 𝑁
𝐸
(1)
𝑥0 = 𝑥,
where 𝑏, 𝜎, 𝛾, and 𝐺 are given deterministic functions and 𝑥
is the initial state. The control variable is a suitable process
(𝑢, 𝜉), where 𝑢 : [0, 𝑇] × Ω → 𝐴 1 ⊂ R𝑑 is the usual
classical absolutely continuous control and 𝜉 : [0, 𝑇] × Ω →
𝐴 2 = ([0, ∞))𝑚 is the singular control, which is an increasing
process, continuous on the right with limits on the left, with
𝜉0− = 0. The performance functional has the form
𝑇
𝑇
0
0
𝐽 (𝑢, 𝜉) = 𝐸 [∫ 𝑓 (𝑡, 𝑥𝑡 , 𝑢𝑡 ) 𝑑𝑡 + ∫ 𝑘 (𝑡) 𝑑𝜉𝑡 + 𝑔 (𝑥𝑇 )] .
(2)
The objective of the controller is to choose a couple
(𝑢⋆ , 𝜉⋆ ) of adapted processes, in order to maximize the
performance functional.
In the first part of our present work, we investigate
the question of necessary as well as sufficient optimality
conditions, in the form of a Pontryagin stochastic maximum
principle. In the second part, we give under regularity
assumptions, a useful verification theorem. Then, we show
that the adjoint process coincides with the spatial gradient of
the value function, evaluated along the optimal trajectory of
the state equation. Finally, using these theoretical results, we
solve explicitly an example, on optimal harvesting strategy
for a geometric Brownian motion, with jumps. Note that
our results improve those in [1, 2] to the jump diffusion
setting. Moreover we generalize results in [3, 4], by allowing
2
both classical and singular controls, at least in the complete
information setting. Note that in our control problem, there
are two types of jumps for the state process, the inaccessible
ones which come from the Poisson martingale part and
the predictable ones which come from the singular control
part. The inclusion of these jump terms introduces a major
difference with respect to the case without singular control.
Stochastic control problems of singular type have received
considerable attention, due to their wide applicability in
a number of different areas; see [4–8]. In most cases,
the optimal singular control problem was studied through
dynamic programming principle; see [9], where it was shown
in particular that the value function is continuous and is the
unique viscosity solution of the HJB variational inequality.
The one-dimensional problems of the singular type,
without the classical control, have been studied by many
authors. It was shown that the value function satisfies a
variational inequality, which gives rise to a free boundary
problem, and the optimal state process is a diffusion reflected
at the free boundary. Bather and Chernoff [10] were the first
to formulate such a problem. Beneš et al. [11] explicitly solved
a one-dimensional example by observing that the value
function in their example is twice continuously differentiable.
This regularity property is called the principle of smooth fit.
The optimal control can be constructed by using the reflected
Brownian motion; see Lions and Sznitman [12] for more
details. Applications to irreversible investment, industry
equilibrium, and portfolio optimization under transaction
costs can be found in [13]. A problem of optimal harvesting
from a population in a stochastic crowded environment is
proposed in [14] to represent the size of the population at
time 𝑡 as the solution of the stochastic logistic differential
equation. The two-dimensional problem that arises in portfolio selection models, under proportional transaction costs,
is of singular type and has been considered by Davis and
Norman [15]. The case of diffusions with jumps is studied
by Øksendal and Sulem [8]. For further contributions on
singular control problems and their relationship with optimal
stopping problems, the reader is referred to [4, 5, 7, 16, 17].
The stochastic maximum principle is another powerful tool for solving stochastic control problems. The first
result that covers singular control problems was obtained
by Cadenillas and Haussmann [18], in which they consider
linear dynamics, convex cost criterion, and convex state
constraints. A first-order weak stochastic maximum principle
was developed via convex perturbations method for both
absolutely continuous and singular components by Bahlali
and Chala [1]. The second-order stochastic maximum principle for nonlinear SDEs with a controlled diffusion matrix
was obtained by Bahlali and Mezerdi [19], extending the
Peng maximum principle [20] to singular control problems.
A similar approach has been used by Bahlali et al. in [21], to
study the stochastic maximum principle in relaxed-singular
optimal control in the case of uncontrolled diffusion. Bahlali
et al. in [22] discuss the stochastic maximum principle in
singular optimal control in the case where the coefficients
are Lipschitz continuous in 𝑥, provided that the classical
derivatives are replaced by the generalized ones. See also the
recent paper by Øksendal and Sulem [4], where Malliavin
International Journal of Stochastic Analysis
calculus techniques have been used to define the adjoint
process.
Stochastic control problems in which the system is
governed by a stochastic differential equation with jumps,
without the singular part, have been also studied, both by
the dynamic programming approach and by the Pontryagin
maximum principle. The HJB equation associated with this
problems is a nonlinear second-order parabolic integrodifferential equation. Pham [23] studied a mixed optimal
stopping and stochastic control of jump diffusion processes
by using the viscosity solutions approach. Some verification
theorems of various types of problems for systems governed
by this kind of SDEs are discussed by Øksendal and Sulem
[8]. Some results that cover the stochastic maximum principle
for controlled jump diffusion processes are discussed in [3,
24, 25]. In [3] the sufficient maximum principle and the
link with the dynamic programming principle are given
by assuming the smoothness of the value function. Let us
mention that in [24] the verification theorem is established
in the framework of viscosity solutions and the relationship between the adjoint processes and some generalized
gradients of the value function are obtained. Note that Shi
and Wu [24] extend the results of [26] to jump diffusions.
See also [27] for systematic study of the continuous case.
The second-order stochastic maximum principle for optimal
controls of nonlinear dynamics, with jumps and convex state
constraints, was developed via spike variation method, by
Tang and Li [25]. These conditions are described in terms of
two adjoint processes, which are linear backward SDEs. Such
equations have important applications in hedging problems
[28]. Existence and uniqueness for solutions to BSDEs with
jumps and nonlinear coefficients have been treated by Tang
and Li [25] and Barles et al. [29]. The link with integral-partial
differential equations is studied in [29].
The plan of the paper is as follows. In Section 2, we
give some preliminary results and notations. The purpose of
Section 3 is to derive necessary as well as sufficient optimality
conditions. In Section 4, we give, under-regularity assumptions, a verification theorem for the value function. Then, we
prove that the adjoint process is equal to the derivative of the
value function evaluated at the optimal trajectory, extending
in particular [2, 3]. An example has been solved explicitly, by
using the theoretical results.
2. Assumptions and Problem Formulation
The purpose of this section is to introduce some notations,
which will be needed in the subsequent sections. In all what
follows, we are given a probability space (Ω, F, (F𝑡 )𝑡≤𝑇 , P),
such that F0 contains the P-null sets, F𝑇 = F for an
arbitrarily fixed time horizon 𝑇, and (F𝑡 )𝑡≤𝑇 satisfies the
usual conditions. We assume that (F𝑡 )𝑡≤𝑇 is generated by a
𝑑-dimensional standard Brownian motion 𝐵 and an independent jump measure 𝑁 of a Lévy process 𝜂, on [0, 𝑇] × 𝐸,
where 𝐸 ⊂ R𝑚 \ {0} for some 𝑚 ≥ 1. We denote by (F𝐵𝑡 )𝑡≤𝑇
(resp., (F𝑁
𝑡 )𝑡≤𝑇 ) the P-augmentation of the natural filtration
of 𝐵 (resp., 𝑁). We assume that the compensator of 𝑁 has the
form 𝜇(𝑑𝑡, 𝑑𝑒) = ](𝑑𝑒)𝑑𝑡, for some 𝜎-finite Lévy measure ]
on 𝐸, endowed with its Borel 𝜎-field B(𝐸). We suppose that
International Journal of Stochastic Analysis
3
̃
∫𝐸 1 ∧ |𝑒|2 ](𝑑𝑒) < ∞ and set 𝑁(𝑑𝑡,
𝑑𝑒) = 𝑁(𝑑𝑡, 𝑑𝑒) − ](𝑑𝑒)𝑑𝑡,
for the compensated jump martingale random measure of 𝑁.
Obviously, we have
F𝑡 = 𝜎 [∫ ∫
𝐴×(0,𝑠]
𝑁 (𝑑𝑟, 𝑑𝑒) ; 𝑠 ≤ 𝑡, 𝐴 ∈ B (𝐸)]
(3)
∨ 𝜎 [𝐵𝑠 ; 𝑠 ≤ 𝑡] ∨ N,
We denote by U = U1 × U2 the set of all admissible
controls. Here U1 (resp., U2 ) represents the set of the
admissible controls 𝑢 (resp., 𝜉).
Assume that, for (𝑢, 𝜉) ∈ U, 𝑡 ∈ [0, 𝑇], the state 𝑥𝑡 of our
system is given by
𝑑𝑥𝑡 = 𝑏 (𝑡, 𝑥𝑡 , 𝑢𝑡 ) 𝑑𝑡 + 𝜎 (𝑡, 𝑥𝑡 , 𝑢𝑡 ) 𝑑𝐵𝑡
where N denotes the totality of ]-null sets and 𝜎1 ∨ 𝜎2 denotes
the 𝜎-field generated by 𝜎1 ∪ 𝜎2 .
Notation. Any element 𝑥 ∈ R𝑛 will be identified with a
column vector with 𝑛 components, and its norm is |𝑥| =
|𝑥1 | + ⋅ ⋅ ⋅ + |𝑥𝑛 |. The scalar product of any two vectors 𝑥 and
𝑦 on R𝑛 is denoted by 𝑥𝑦 or ∑𝑛𝑖=1 𝑥𝑖 𝑦𝑖 . For a function ℎ, we
denote by ℎ𝑥 (resp., ℎ𝑥𝑥 ) the gradient or Jacobian (resp., the
Hessian) of ℎ with respect to the variable 𝑥.
Given 𝑠 < 𝑡, let us introduce the following spaces.
̃ (𝑑𝑡, 𝑑𝑒) + 𝐺𝑡 𝑑𝜉𝑡 ,
+ ∫ 𝛾 (𝑡, 𝑥𝑡− , 𝑢𝑡 , 𝑒) 𝑁
𝐸
𝑥0 = 𝑥,
where 𝑥 ∈ R𝑛 is given, representing the initial state.
Let
𝑏 : [0, 𝑇] × R𝑛 × 𝐴 1 󳨀→ R𝑛 ,
𝜎 : [0, 𝑇] × R𝑛 × 𝐴 1 󳨀→ R𝑛×𝑑 ,
(i) L2],(𝐸;R𝑛 ) or L2] is the set of square integrable functions
l(⋅) : 𝐸 → R𝑛 such that
‖l (𝑒)‖2L2 𝑛
],(𝐸;R )
2
:= ∫ |l (𝑒)| ] (𝑑𝑒) < ∞.
(4)
𝐸
(ii) S2([𝑠,𝑡];R𝑛 ) is the set of R𝑛 -valued adapted cadlag
processes 𝑃 such that
‖𝑃‖S2
([𝑠,𝑡];R𝑛 )
󵄨 󵄨2
:= E[ sup 󵄨󵄨󵄨𝑃𝑟 󵄨󵄨󵄨 ]
< ∞.
(5)
(iii) M2([𝑠,𝑡];R𝑛 ) is the set of progressively measurable R𝑛 valued processes 𝑄 such that
𝑡
‖𝑄‖M2
([𝑠,𝑡];R𝑛 )
(iv)
󵄨 󵄨2
:= E[∫ 󵄨󵄨󵄨𝑄𝑟 󵄨󵄨󵄨 𝑑𝑟]
𝑠
< ∞.
(6)
is the set of B([0, 𝑇] × Ω) ⊗ B(𝐸)
measurable maps 𝑅 : [0, 𝑇] × Ω × 𝐸 → R𝑛 such
that
‖𝑅‖L2
],([𝑠,𝑡];R𝑛 )
󵄨2
󵄨
:= E[∫ ∫ 󵄨󵄨󵄨𝑅𝑟 (𝑒)󵄨󵄨󵄨 ] (𝑑𝑒) 𝑑𝑟]
𝑠 𝐸
1/2
< ∞.
(7)
To avoid heavy notations, we omit the subscript
([𝑠, 𝑡]; R𝑛 ) in these notations when (𝑠, 𝑡) = (0, 𝑇).
Let 𝑇 be a fixed strictly positive real number; 𝐴 1 is a
closed convex subset of R𝑛 and 𝐴 2 = ([0, ∞)𝑚 ). Let us define
the class of admissible control processes (𝑢, 𝜉).
Definition 1. An admissible control is a pair of measurable,
adapted processes 𝑢 : [0, 𝑇] × Ω → 𝐴 1 , and 𝜉 : [0, 𝑇] × Ω →
𝐴 2 , such that
(1) 𝑢 is a predictable process, 𝜉 is of bounded variation,
nondecreasing, right continuous with left-hand limits, and 𝜉0− = 0,
2
2
(2) E[sup𝑡∈[0,𝑇] |𝑢𝑡 | + |𝜉𝑇 | ] < ∞.
(9)
𝐺 : [0, 𝑇] 󳨀→ R𝑛×𝑚
be measurable functions.
Notice that the jump of a singular control 𝜉 ∈ U2 at any
jumping time 𝜏 is defined by Δ𝜉𝜏 = 𝜉𝜏 − 𝜉𝜏− , and we let
𝜉𝑡𝑐 = 𝜉𝑡 − ∑ Δ𝜉𝜏 ,
(10)
0<𝜏≤𝑡
be the continuous part of 𝜉.
We distinguish between the jumps of 𝑥𝜏 caused by the
jump of 𝑁(𝜏, 𝑒), defined by
Δ 𝑁𝑥𝜏 := ∫ 𝛾 (𝜏, 𝑥𝜏− , 𝑢𝜏 , 𝑒) 𝑁 ({𝜏} , 𝑑𝑒)
1/2
L2],([𝑠,𝑡];R𝑛 )
𝑡
𝛾 : [0, 𝑇] × R𝑛 × 𝐴 1 × 𝐸 󳨀→ R𝑛 ,
1/2
𝑟∈[𝑠,𝑡]
(8)
𝐸
𝛾 (𝜏, 𝑥𝜏− , 𝑢𝜏 , 𝑒)
:= {
0
if 𝜂 has a jump of size 𝑒 at 𝜏,
otherwise,
(11)
and the jump of 𝑥𝜏 caused by the singular control 𝜉, denoted
by Δ 𝜉 𝑥𝜏 := 𝐺𝜏 Δ𝜉𝜏 . In the above, 𝑁({𝜏}, ⋅) represents the
jump in the Poisson random measure, occurring at time 𝜏. In
particular, the general jump of the state process at 𝜏 is given
by Δ𝑥𝜏 = 𝑥𝜏 − 𝑥𝜏− = Δ 𝜉 𝑥𝜏 + Δ 𝑁𝑥𝜏 .
If 𝜑 is a continuous real function, we let
Δ 𝜉 𝜑 (𝑥𝜏 ) := 𝜑 (𝑥𝜏 ) − 𝜑 (𝑥𝜏− + Δ 𝑁𝑥𝜏 ) .
(12)
The expression (12) defines the jump in the value of
𝜑(𝑥𝜏 ) caused by the jump of 𝑥 at 𝜏. We emphasize that the
possible jumps in 𝑥𝜏 coming from the Poisson measure are
not included in Δ 𝜉 𝜑(𝑥𝜏 ).
Suppose that the performance functional has the form
𝑇
𝑇
0
𝑠
𝐽 (𝑢, 𝜉) = E [∫ 𝑓 (𝑡, 𝑥𝑡 , 𝑢𝑡 ) 𝑑𝑡 + 𝑔 (𝑥𝑇 ) + ∫ 𝑘𝑡 𝑑𝜉𝑡 ] ,
for (𝑢, 𝜉) ∈ U,
(13)
4
International Journal of Stochastic Analysis
where 𝑓 : [0, 𝑇] × R𝑛 × 𝐴 1 → R, 𝑔 : R𝑛 → R, and 𝑘 :
𝑙
𝑙
[0, 𝑇] → ([0, ∞))𝑚 , with 𝑘𝑡 𝑑𝜉𝑡 = ∑𝑚
𝑙=1 𝑘𝑡 𝑑𝜉𝑡 .
⋆ ⋆
An admissible control (𝑢 , 𝜉 ) is optimal if
𝐽 (𝑢⋆ , 𝜉⋆ ) = sup 𝐽 (𝑢, 𝜉) .
(14)
(𝑢,𝜉)∈U
Let us assume the following.
(H1 ) The maps 𝑏, 𝜎, 𝛾, and 𝑓 are continuously differentiable
with respect to (𝑥, 𝑢) and 𝑔 is continuously differentiable in 𝑥.
(H2 ) The derivatives 𝑏𝑥 , 𝑏𝑢 , 𝜎𝑥 , 𝜎𝑢 , 𝛾𝑥 , 𝛾𝑢 , 𝑓𝑥 , 𝑓𝑢 , and 𝑔𝑥 are
continuous in (𝑥, 𝑢) and uniformly bounded.
(H3 ) 𝑏, 𝜎, 𝛾, and 𝑓 are bounded by 𝐾1 (1 + |𝑥| + |𝑢|), and 𝑔
is bounded by 𝐾1 (1 + |𝑥|), for some 𝐾1 > 0.
(H4 ) For all (𝑢, 𝑒) ∈ 𝐴 1 × 𝐸, the map
(i) For all V ∈ 𝐴 1
𝐻𝑢 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑝𝑡 , 𝑞𝑡 , 𝑟𝑡 (⋅)) (V𝑡 − 𝑢𝑡⋆ ) ≤ 0,
𝑑𝑡—𝑎.𝑒., P—𝑎.𝑠.
𝑘𝑡𝑖 + 𝐺𝑡𝑖 𝑝𝑡 ≤ 0,
(15)
:= 𝜁T (𝛾𝑥 (𝑡, 𝑥, 𝑢, 𝑒) + 𝐼𝑑 ) 𝜁
for 𝑖 = 1, . . . , 𝑚,
(19)
(20)
𝑚
∑1{𝑘𝑡𝑖 +𝐺𝑡𝑖 𝑝𝑡 ≤0} 𝑑𝜉𝑡⋆𝑐𝑖 = 0,
(21)
𝑖=1
𝑛
satisfies uniformly in (𝑥, 𝜁) ∈ R × R ,
󵄨 󵄨2
𝑎 (𝑡, 𝑥, 𝑢, 𝜁; 𝑒) ≥ 󵄨󵄨󵄨𝜁󵄨󵄨󵄨 𝐾2−1 ,
Theorem 2 (necessary conditions of optimality). Let (𝑢⋆ , 𝜉⋆ )
be an optimal control maximizing the functional 𝐽 over U, and
let 𝑥⋆ be the corresponding optimal trajectory. Then there exists
an adapted process (𝑝, 𝑞, 𝑟(⋅)) ∈ S2 × M2 × L2] , which is
the unique solution of the BSDE (18), such that the following
conditions hold.
(ii) For all 𝑡 ∈ [0, 𝑇], with probability 1
(𝑥, 𝜁) ∈ R𝑛 × R𝑛 󳨀→ 𝑎 (𝑡, 𝑥, 𝑢, 𝜁; 𝑒)
𝑛
3.1. Necessary Conditions of Optimality. The purpose of this
section is to derive optimality necessary conditions, satisfied
by an optimal control, assuming that the solution exists. The
proof is based on convex perturbations for both absolutely
continuous and singular components of the optimal control
and on some estimates of the state processes. Note that our
results generalize [1, 2, 21] for systems with jumps.
𝑘𝑡𝑖 + 𝐺𝑡𝑖 (𝑝𝑡− + Δ 𝑁𝑝𝑡 ) ≤ 0,
for some 𝐾2 > 0.
(16)
for 𝑖 = 1, . . . , 𝑚,
(22)
𝑚
∑1{𝑘𝑡𝑖 +𝐺𝑡𝑖 (𝑝𝑡− +Δ 𝑁 𝑝𝑡 )≤0} Δ𝜉𝑡⋆𝑖 = 0,
(23)
𝑖=1
(H5 ) 𝐺, 𝑘 are continuous and bounded.
where Δ 𝑁𝑝𝑡 = ∫𝐸 𝑟𝑡 (𝑒)𝑁({𝑡}, 𝑑𝑒).
3. The Stochastic Maximum Principle
Let us first define the usual Hamiltonian associated to the
control problem by
𝐻 (𝑡, 𝑥, 𝑢, 𝑝, 𝑞, X (⋅)) = 𝑓 (𝑡, 𝑥, 𝑢) + 𝑝𝑏 (𝑡, 𝑥, 𝑢)
𝑛
+ ∑ 𝑞𝑗 𝜎𝑗 (𝑡, 𝑥, 𝑢)
(17)
𝑗=1
+ ∫ X (𝑒) 𝛾 (𝑡, 𝑥, 𝑢, 𝑒) ] (𝑑𝑒) ,
𝐸
In order to prove Theorem 2, we present some auxiliary
results.
3.1.1. Variational Equation. Let (V, 𝜉) ∈ U be such that (𝑢⋆ +
V, 𝜉⋆ + 𝜉) ∈ U. The convexity condition of the control domain
ensures that for 𝜀 ∈ (0, 1) the control (𝑢⋆ +𝜀V, 𝜉⋆ +𝜀𝜉) is also in
U. We denote by 𝑥𝜀 the solution of the SDE (8) corresponding
to the control (𝑢⋆ + 𝜀V, 𝜉⋆ + 𝜀𝜉). Then by standard arguments
from stochastic calculus, it is easy to check the following
estimate.
Lemma 3. Under assumptions (H1 )–(H5 ), one has
𝑛
𝑛
𝑛×𝑛
×L2] . 𝑞𝑗
where (𝑡, 𝑥, 𝑢, 𝑝, 𝑞, X(⋅)) ∈ [0, 𝑇]×R ×𝐴 1 ×R ×R
and 𝜎𝑗 for 𝑗 = 1, . . . , 𝑛, denote the 𝑗th column of the matrices
𝑞 and 𝜎, respectively.
Let (𝑢⋆ , 𝜉⋆ ) be an optimal control and let 𝑥⋆ be the
corresponding optimal trajectory. Then, we consider a triple
(𝑝, 𝑞, 𝑟(⋅)) of square integrable adapted processes associated
with (𝑢⋆ , 𝑥⋆ ), with values in R𝑛 × R𝑛×𝑑 × R𝑛 such that
𝑑𝑝𝑡 = −𝐻𝑥 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑝𝑡 , 𝑞𝑡 , 𝑟𝑡 (⋅)) 𝑑𝑡
̃ (𝑑𝑡, 𝑑𝑒) ,
+ 𝑞𝑡 𝑑𝐵𝑡 + ∫ 𝑟𝑡 (𝑒) 𝑁
𝐸
𝑝𝑇 = 𝑔𝑥 (𝑥𝑇⋆ ) .
󵄨
󵄨2
lim E [ sup 󵄨󵄨󵄨𝑥𝑡𝜀 − 𝑥𝑡⋆ 󵄨󵄨󵄨 ] = 0.
𝜀→0
𝑡∈[0,𝑇]
(24)
Proof. From assumptions (H1 )–(H5 ), we get by using the
Burkholder-Davis-Gundy inequality
󵄨
󵄨2
E [ sup 󵄨󵄨󵄨𝑥𝑡𝜀 − 𝑥𝑡⋆ 󵄨󵄨󵄨 ]
𝑡∈[0,𝑇]
𝑇
󵄨
󵄨2
≤ 𝐾 ∫ E [ sup 󵄨󵄨󵄨𝑥𝜏𝜀 − 𝑥𝜏⋆ 󵄨󵄨󵄨 ] 𝑑𝑠
0
(18)
𝜏∈[0,𝑠]
𝑇
󵄨 󵄨2
󵄨 󵄨2
+𝐾𝜀2 (∫ E [ sup 󵄨󵄨󵄨V𝜏 󵄨󵄨󵄨 ] 𝑑𝑠 + E󵄨󵄨󵄨𝜉𝑇 󵄨󵄨󵄨 ) .
0
𝜏∈[0,𝑠]
(25)
International Journal of Stochastic Analysis
5
󵄨󵄨2
󵄨󵄨 1
𝑡
󵄨
󵄨
+ 𝐾E ∫ ∫ 󵄨󵄨󵄨󵄨∫ 𝛾𝑥 (𝑠, 𝑥𝑠𝜇,𝜀 , 𝑢𝑠𝜇,𝜀 , 𝑒) Γ𝑠𝜀 𝑑𝜇󵄨󵄨󵄨󵄨 ] (𝑑𝑒) 𝑑𝑠
󵄨󵄨
0 𝐸 󵄨󵄨 0
󵄨 󵄨2
+ 𝐾E󵄨󵄨󵄨𝜌𝑡𝜀 󵄨󵄨󵄨 ,
From Definition 1 and Gronwall’s lemma, the result follows immediately by letting 𝜀 go to zero.
We define the process 𝑧𝑡 = 𝑧𝑡𝑢
𝑑𝑧𝑡 =
{𝑏𝑥 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) 𝑧𝑡
+
⋆
,V,𝜉
by
𝑏𝑢 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) V𝑡 } 𝑑𝑡
𝑑
(30)
where
𝑗
+ ∑ {𝜎𝑥𝑗 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) 𝑧𝑡 + 𝜎𝑢𝑗 (𝑡, 𝑥𝑡⋆ , 𝑢t⋆ ) V𝑡 } 𝑑𝐵𝑡
𝑗=1
⋆
⋆
+ ∫ {𝛾𝑥 (𝑡, 𝑥𝑡−
, 𝑢𝑡⋆ , 𝑒) 𝑧𝑡− + 𝛾𝑢 (𝑡, 𝑥𝑡−
, 𝑢𝑡⋆ , 𝑒) V𝑡 }
(26)
𝑡
𝑡
0
0
𝜌𝑡𝜀 = − ∫ 𝑏𝑥 (𝑠, 𝑥𝑠⋆ , 𝑢𝑠⋆ ) 𝑧𝑠 𝑑𝑠 − ∫ 𝜎𝑥 (𝑠, 𝑥𝑠⋆ , 𝑢𝑠⋆ ) 𝑧𝑠 𝑑𝐵𝑠
⋆
̃ (𝑑𝑠, 𝑑𝑒)
− ∫ ∫ 𝛾𝑥 (𝑠, 𝑥𝑠−
, 𝑢𝑠⋆ , 𝑒) 𝑧𝑠− 𝑁
̃ (𝑑𝑡, 𝑑𝑒) + 𝐺𝑡 𝑑𝜉𝑡 ,
×𝑁
0
𝐸
𝑡
𝑡
0
0
− ∫ 𝑏V (𝑠, 𝑥𝑠⋆ , 𝑢𝑠⋆ ) V𝑠 𝑑𝑠 − ∫ 𝜎V (𝑠, 𝑥𝑠⋆ , 𝑢𝑠⋆ ) V𝑠 𝑑𝐵𝑠
𝑧0 = 0.
From (H2 ) and Definition 1, one can find a unique
solution 𝑧 which solves the variational equation (26), and the
following estimate holds.
Lemma 4. Under assumptions (H1 )–(H5 ), it holds that
󵄨󵄨 𝑥𝜀 − 𝑥⋆
󵄨󵄨2
󵄨
󵄨
𝑡
(27)
lim E󵄨󵄨󵄨󵄨 𝑡
− 𝑧𝑡 󵄨󵄨󵄨󵄨 = 0.
𝜀→0 󵄨
𝜀
󵄨󵄨
󵄨
Proof. Let
𝑥𝜀 − 𝑥𝑡⋆
= 𝑡
− 𝑧𝑡 .
𝜀
𝜇,𝜀
is given by
𝑡
𝐸
Γ𝑡𝜀
𝜌𝑡𝜀
(28)
𝜇,𝜀
We denote 𝑥𝑡 = 𝑥𝑡⋆ + 𝜇𝜀(Γ𝑡𝜀 + 𝑧𝑡 ) and 𝑢𝑡 = 𝑢𝑡⋆ + 𝜇𝜀V𝑡 ,
for notational convenience. Then we have immediately that
Γ0𝜀 = 0 and Γ𝑡𝜀 satisfies the following SDE:
1
𝜇,𝜀 𝜇,𝜀
𝑑Γ𝑡𝜀 = { (𝑏 (𝑡, 𝑥𝑡 , 𝑢𝑡 ) − 𝑏 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ))
𝜀
𝑡
⋆
̃ (𝑑𝑠, 𝑑𝑒)
− ∫ ∫ 𝛾V (𝑠, 𝑥𝑠−
, 𝑢𝑠⋆ , 𝑒) V𝑠 𝑁
0
𝐸
𝑡
1
0
0
𝑡
1
0
0
+ ∫ ∫ 𝑏𝑥 (𝑠, 𝑥𝑠𝜇,𝜀 , 𝑢𝑠𝜇,𝜀 ) 𝑧𝑠 𝑑𝜇 𝑑𝑠
+∫ ∫
(31)
𝜎𝑥 (𝑠, 𝑥𝑠𝜇,𝜀 , 𝑢𝑠𝜇,𝜀 ) 𝑧𝑠 𝑑𝜇 𝑑𝐵𝑠
𝑡
1
0
𝐸 0
𝑡
1
0
0
𝑡
1
0
0
𝜇,𝜀 𝜇,𝜀
̃ (𝑑𝑠, 𝑑𝑒)
+ ∫ ∫ ∫ 𝛾𝑥 (𝑠, 𝑥𝑠−
, 𝑢𝑠 , 𝑒) 𝑧𝑠− 𝑑𝜇𝑁
+ ∫ ∫ 𝑏V (𝑠, 𝑥𝑠𝜇,𝜀 , 𝑢𝑠𝜇,𝜀 ) V𝑠 𝑑𝜇 𝑑𝑠
+ ∫ ∫ 𝜎V (𝑠, 𝑥s𝜇,𝜀 , 𝑢𝑠𝜇,𝜀 ) V𝑠 𝑑𝜇 𝑑𝐵𝑠
𝑡
1
0
𝐸 0
𝜇,𝜀 𝜇,𝜀
̃ (𝑑𝑠, 𝑑𝑒) .
+ ∫ ∫ ∫ 𝛾V (𝑠, 𝑥𝑠−
, 𝑢𝑠 , 𝑒) V𝑠 𝑑𝜇𝑁
− (𝑏𝑥 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) 𝑧𝑡 + 𝑏𝑢 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) V𝑡 ) } 𝑑𝑡
Since 𝑏𝑥 , 𝜎𝑥 , and 𝛾𝑥 are bounded, then
1
𝜇,𝜀 𝜇,𝜀
+ { (𝜎 (𝑡, 𝑥𝑡 , 𝑢𝑡 ) − 𝜎 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ))
𝜀
𝑡
󵄨 󵄨2
󵄨 󵄨2
󵄨 󵄨2
E󵄨󵄨󵄨Γ𝑡𝜀 󵄨󵄨󵄨 ≤ 𝑀E ∫ 󵄨󵄨󵄨Γ𝑠𝜀 󵄨󵄨󵄨 𝑑𝑠 + 𝑀E󵄨󵄨󵄨𝜌𝑡𝜀 󵄨󵄨󵄨 ,
0
− (𝜎𝑥 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) 𝑧𝑡 + 𝜎𝑢 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) V𝑡 ) } 𝑑𝐵𝑡
1
𝜇,𝜀 𝜇,𝜀
⋆
+ ∫ { (𝛾 (𝑡, 𝑥𝑡− , 𝑢𝑡 , 𝑒) − 𝛾 (𝑡, 𝑥𝑡−
, 𝑢𝑡⋆ , 𝑒))
𝐸 𝜀
⋆
⋆
− (𝛾𝑥 (𝑡, 𝑥𝑡−
, 𝑢𝑡⋆ , 𝑒) 𝑧𝑡− + 𝛾𝑢 (𝑡, 𝑥𝑡−
, 𝑢𝑡⋆ , 𝑒) V𝑡 ) }
(32)
where 𝑀 is a generic constant depending on the constants 𝐾,
](𝐸), and 𝑇. We conclude from Lemma 3 and the dominated
convergence theorem, that lim𝜀 → 0 𝜌𝑡𝜀 = 0. Hence (27)
follows from Gronwall’s lemma and by letting 𝜀 go to 0. This
completes the proof.
3.1.2. Variational Inequality. Let Φ be the solution of the
linear matrix equation, for 0 ≤ 𝑠 < 𝑡 ≤ 𝑇
̃ (𝑑𝑡, 𝑑𝑒) .
×𝑁
(29)
Since the derivatives of the coefficients are bounded, and
from Definition 1, it is easy to verify by Gronwall’s inequality
that Γ𝜀 ∈ S2 and
󵄨󵄨2
𝑡 󵄨󵄨 1
󵄨
󵄨
󵄨 󵄨2
E󵄨󵄨󵄨Γ𝑡𝜀 󵄨󵄨󵄨 ≤ 𝐾E ∫ 󵄨󵄨󵄨󵄨∫ 𝑏𝑥 (𝑠, 𝑥𝑠𝜇,𝜀 , 𝑢𝑠𝜇,𝜀 ) Γ𝑠𝜀 𝑑𝜇󵄨󵄨󵄨󵄨 𝑑𝑠
󵄨󵄨
0 󵄨󵄨 0
󵄨󵄨2
𝑡 󵄨󵄨 1
󵄨
󵄨
+ 𝐾E ∫ 󵄨󵄨󵄨󵄨∫ 𝜎𝑥 (𝑠, 𝑥𝑠𝜇,𝜀 , 𝑢𝑠𝜇,𝜀 ) Γ𝑠𝜀 𝑑𝜇󵄨󵄨󵄨󵄨 𝑑𝑠
󵄨󵄨
0 󵄨󵄨 0
𝑑
𝑗
𝑑Φ𝑠,𝑡 = 𝑏𝑥 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) Φ𝑠,𝑡 𝑑𝑡 + ∑ 𝜎𝑥𝑗 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) Φ𝑠,𝑡 𝑑𝐵𝑡
𝑗=1
⋆
̃ (𝑑𝑡, 𝑑𝑒) ,
+ ∫ 𝛾𝑥 (𝑡, 𝑥𝑡−
, 𝑢𝑡⋆ , 𝑒) Φ𝑠,𝑡− 𝑁
(33)
𝐸
Φ𝑠,𝑠 = 𝐼𝑑 ,
where 𝐼𝑑 is the 𝑛 × 𝑛 identity matrix. This equation is linear,
with bounded coefficients, then it admits a unique strong
6
International Journal of Stochastic Analysis
solution. Moreover, the condition (H4 ) ensures that the
tangent process Φ is invertible, with an inverse Ψ satisfying
suitable integrability conditions.
From Itô’s formula, we can easily check that 𝑑(Φ𝑠,𝑡 Ψ𝑠,𝑡 ) =
0, and Φ𝑠,𝑠 Ψ𝑠,𝑠 = 𝐼𝑑 , where Ψ is the solution of the following
equation
𝑑
{
𝑑Ψ𝑠,𝑡 = −Ψ𝑠,𝑡 {𝑏𝑥 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) − ∑ 𝜎𝑥𝑗 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) 𝜎𝑥𝑗 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ )
𝑗=1
{
−∫
𝐸
−
𝑑
∑Ψ𝑠,𝑡 𝜎𝑥𝑗
𝑗=1
}
𝛾𝑥 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑒) ] (𝑑𝑒) } 𝑑𝑡
The rest of this subsection is devoted to the computation
of the above limit. We will see that the expression (37) leads to
a precise description of the optimal control (𝑢⋆ , 𝜉⋆ ) in terms
of the adjoint process. First, it is easy to prove the following
lemma.
Lemma 5. Under assumptions (H1 )–(H5 ), one has
𝐼 = lim 𝜀−1 (𝐽 (𝑢𝜀 , 𝜉𝜀 ) − 𝐽 (𝑢⋆ , 𝜉⋆ ))
𝜀→0
𝑇
= E [∫ {𝑓𝑥 (𝑠, 𝑥𝑠⋆ , 𝑢𝑠⋆ ) 𝑧𝑠 + 𝑓𝑢 (𝑠, 𝑥𝑠⋆ , 𝑢𝑠⋆ ) V𝑠 } 𝑑𝑠
0
}
𝑇
+ 𝑔𝑥 (𝑥𝑇⋆ ) 𝑧𝑇 + ∫ 𝑘𝑡 𝑑𝜉𝑡 ] .
𝑗
(𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) 𝑑𝐵𝑡
0
−1
⋆
⋆
− Ψ𝑠,𝑡− ∫ (𝛾𝑥 (𝑡, 𝑥𝑡−
, 𝑢𝑡⋆ , 𝑒) + 𝐼𝑑 ) 𝛾𝑥 (𝑡, 𝑥𝑡−
, 𝑢𝑡⋆ , 𝑒)
𝐸
Proof. We use the same notations as in the proof of Lemma 4.
First, we have
𝜀−1 (𝐽 (𝑢𝜀 , 𝜉𝜀 ) − 𝐽 (𝑢⋆ , 𝜉⋆ ))
× 𝑁 (𝑑𝑡, 𝑑𝑒) ,
Ψ𝑠,𝑠 = 𝐼𝑑 ,
(34)
so Ψ = Φ . If 𝑠 = 0 we simply write Φ0,𝑡 = Φ𝑡 and Ψ0,𝑡 = Ψ𝑡 .
By the integration by parts formula ([8, Lemma 3.6]), we can
see that the solution of (26) is given by 𝑧𝑡 = Φ𝑡 𝜂𝑡 , where 𝜂𝑡 is
the solution of the stochastic differential equation
𝑑
{
𝑑𝜂𝑡 = Ψ𝑡 {𝑏𝑢 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) V𝑡 − ∑ 𝜎𝑥𝑗 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) 𝜎𝑢𝑗 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) V𝑡
𝑗=1
{
𝐸
}
𝛾𝑢 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑧) V𝑡 ] (𝑑𝑒) } 𝑑𝑡
1
0
0
1
𝑇
𝜇,𝜀
+ ∫ 𝑔𝑥 (𝑥𝑇 ) 𝑧𝑇 𝑑𝜇 + ∫ 𝑘𝑡 𝑑𝜉𝑡 ] + 𝛽𝑡𝜀 ,
0
0
(39)
where
𝑇
1
1
0
0
0
𝜇,𝜀
𝛽𝑡𝜀 = E [∫ ∫ 𝑓𝑥 (𝑠, 𝑥𝑠𝜇,𝜀 , 𝑢𝑠𝜇,𝜀 ) Γ𝑠𝜀 𝑑𝜇 𝑑𝑠 + ∫ 𝑔𝑥 (𝑥𝑇 ) Γ𝑇𝜀 𝑑𝜇] .
(40)
By using Lemma 4, and since the derivatives 𝑓𝑥 , 𝑓𝑢 , and
𝑔𝑥 are bounded, we have lim𝜀 → 0 𝛽𝑡𝜀 = 0. Then, the result
follows by letting 𝜀 go to 0 in the above equality.
}
𝑑
𝑇
= E [∫ ∫ {𝑓𝑥 (𝑠, 𝑥𝑠𝜇,𝜀 , 𝑢𝑠𝜇,𝜀 ) 𝑧𝑠 + 𝑓𝑢 (𝑠, 𝑥𝑠𝜇,𝜀 , 𝑢𝑠𝜇,𝜀 ) V𝑠 } 𝑑𝜇 𝑑𝑠
−1
−∫
(38)
𝑗
+ ∑Ψ𝑡 𝜎𝑢𝑗 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) V𝑡 𝑑𝐵𝑡
Substituting by 𝑧𝑡 = Φ𝑡 𝜂𝑡 in (38) leads to
𝑗=1
𝑇
+ Ψ𝑡− ∫
𝐸
⋆
(𝛾𝑥 (𝑡, 𝑥𝑡−
, 𝑢𝑡⋆ , 𝑒)
𝐼 = E [∫ {𝑓𝑥 (𝑠, 𝑥𝑠⋆ , 𝑢𝑠⋆ ) Φ𝑠 𝜂𝑠 + 𝑓𝑢 (𝑠, 𝑥𝑠⋆ , 𝑢𝑠⋆ ) V𝑠 } 𝑑𝑠
−1
+ 𝐼𝑑 )
0
⋆
× 𝛾𝑢 (𝑡, 𝑥𝑡−
, 𝑢𝑡⋆ , 𝑒) V𝑡 𝑁 (𝑑𝑡, 𝑑𝑒)
+ Ψ𝑡 𝐺𝑡 𝑑𝜉𝑡 − Ψ𝑡 ∫
𝐸
(𝛾𝑥 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑒)
+𝑔𝑥 (𝑥𝑇⋆ ) Φ𝑇𝜂𝑇
−1
+ 𝐼𝑑 )
× 𝛾𝑥 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑒) 𝑁 ({𝑡} , 𝑑𝑒) 𝐺𝑡 Δ𝜉𝑡 ,
𝜂0 = 0.
(35)
Let us introduce the following convex perturbation of the
optimal control (𝑢⋆ , 𝜉⋆ ) defined by
(𝑢⋆,𝜀 , 𝜉⋆,𝜀 ) = (𝑢⋆ + 𝜀V, 𝜉⋆ + 𝜀𝜉) ,
(36)
for some (V, 𝜉) ∈ U and 𝜀 ∈ (0, 1). Since (𝑢⋆ , 𝜉⋆ ) is an optimal
control, then 𝜀−1 (𝐽(𝑢𝜀 , 𝜉𝜀 ) − 𝐽(𝑢⋆ , 𝜉⋆ )) ≤ 0. Thus a necessary
condition for optimality is that
lim 𝜀−1 (𝐽 (𝑢𝜀 , 𝜉𝜀 ) − 𝐽 (𝑢⋆ , 𝜉⋆ )) ≤ 0.
𝜀→0
(37)
(41)
𝑇
+ ∫ 𝑘𝑡 𝑑𝜉𝑡 ] .
0
Consider the right continuous version of the square
integrable martingale
𝑇
𝑀𝑡 := E [∫ 𝑓𝑥 (𝑠, 𝑥𝑠⋆ , 𝑢𝑠⋆ ) Φ𝑠 𝑑𝑠 + 𝑔𝑥 (𝑥𝑇⋆ ) Φ𝑇 | F𝑡 ] . (42)
0
By the Itô representation theorem [30], there exist two
processes 𝑄 = (𝑄1 , . . . , 𝑄𝑑 ) where 𝑄𝑗 ∈ M2 , for 𝑗 = 1, . . . , 𝑑,
and 𝑈(⋅) ∈ L2] , satisfying
𝑇
𝑀𝑡 = E [∫ 𝑓𝑥 (𝑠, 𝑥𝑠⋆ , 𝑢𝑠⋆ ) Φ𝑠 𝑑𝑠 + 𝑔𝑥 (𝑥𝑇⋆ ) Φ𝑇 ]
0
𝑑
𝑡
+∑∫
𝑗=1 0
𝑄𝑠𝑗 𝑑𝐵𝑠𝑗
(43)
𝑡
̃ (𝑑𝑠, 𝑑𝑒) .
+ ∫ ∫ 𝑈𝑠 (𝑒) 𝑁
0
𝐸
International Journal of Stochastic Analysis
7
𝑡
Let us denote 𝑦𝑡⋆ = 𝑀𝑡 − ∫0 𝑓𝑥 (𝑠, 𝑥𝑠⋆ , 𝑢𝑠⋆ )Φ𝑠 𝑑𝑠. The adjoint
variable is the process defined by
𝑝𝑡 = 𝑦𝑡⋆ Ψ𝑡 ,
𝑗
𝑞𝑡
=
𝑗
𝑄𝑡 Ψ𝑡
−
𝑝𝑡 𝜎𝑥𝑗
(𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) ,
for 𝑗 = 1, . . . , 𝑑,
(44)
−1
𝑟𝑡 (𝑒) = 𝑈𝑡 (𝑒) Ψ𝑡 (𝛾𝑥 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑒) + 𝐼𝑑 )
𝑝𝑡 ((𝛾𝑥 (𝑠, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑒)
+
−1
+ 𝐼𝑑 )
3.1.3. Adjoint Equation and Maximum Principle. Since (37)
is true for all (V, 𝜉) ∈ U and 𝐼 ≤ 0, we can easily deduce the
following result.
Theorem 7. Let (𝑢⋆ , 𝜉⋆ ) be the optimal control of the problem
(14) and denote by 𝑥⋆ the corresponding optimal trajectory,
then the following inequality holds:
𝑇
E [ ∫ 𝐻V (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑝𝑡 , 𝑞𝑡 , 𝑟𝑡 (⋅)) (V𝑡 − 𝑢𝑡⋆ ) 𝑑𝑡
− 𝐼𝑑 ) .
0
𝑇
Theorem 6. Under assumptions (H1 )–(H5 ), one has
𝑇
+ ∑ {𝑘𝑡 + 𝐺𝑡 (𝑝𝑡− + Δ 𝑁𝑝𝑡 )} Δ(𝜉 − 𝜉⋆ )𝑡 ] ≤ 0,
0
+
0<𝑡≤𝑇
𝑑
∑ 𝑞𝑠𝑗 𝜎𝑢𝑗
𝑗=1
where the Hamiltonian 𝐻 is defined by (17), and the adjoint
variable (𝑝, 𝑞𝑗 , 𝑟(⋅)) for 𝑗 = 1, . . . , 𝑑, is given by (44).
(𝑠, 𝑥𝑠⋆ , 𝑢𝑠⋆ )
+ ∫ 𝑟𝑠 (𝑧) 𝛾𝑢 (𝑠, 𝑥𝑠⋆ , 𝑢𝑠⋆ , 𝑒) ] (𝑑𝑒)} V𝑠 𝑑𝑠
(45)
𝐸
𝑚
Now, we are ready to give the proof of Theorem 2.
Proof of Theorem 2. (i) Let us assume that (𝑢⋆ , 𝜉⋆ ) is an
optimal control for the problem (14), so that inequality (48)
is valid for every (V, 𝜉). If we choose 𝜉 = 𝜉⋆ in inequality
(48), we see that for every measurable, F𝑡 -adapted process
V : [0, 𝑇] × Ω → 𝐴 1
𝑇
+ ∑ ∫ {𝑘𝑠𝑖 + 𝐺𝑠𝑖 𝑝𝑠 } 𝑑𝜉𝑠𝑐𝑖
𝑖=1 0
𝑚
+ ∑ ∑ {𝑘𝑠𝑖 + 𝐺𝑠𝑖 (𝑝𝑠− + Δ 𝑁𝑝𝑠 )} Δ𝜉𝑠𝑖 ] .
𝑇
𝑖=1 0<𝑠≤𝑇
E [∫ 𝐻V (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑝𝑡 , 𝑞𝑡 , 𝑟𝑡 (⋅)) (V𝑡 − 𝑢𝑡⋆ ) 𝑑𝑡] ≤ 0.
Proof. From the integration by parts formula ([8, Lemma
𝑗
3.5]), and by using the definition of 𝑝𝑡 , 𝑞𝑡 for 𝑗 = 1, . . . , 𝑑,
and 𝑟𝑡 (⋅), we can easily check that
0
For V ∈ U1 define
𝐴 = { (𝑡, 𝜔) ∈ [0, 𝑇] × Ω
such that 𝐻V (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑝𝑡 , 𝑞𝑡 , 𝑟𝑡 (⋅)) (V𝑡 − 𝑢𝑡⋆ ) > 0} .
(50)
𝑑
𝑇{
= E [ ∫ {𝑝𝑡 𝑏𝑢 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) + ∑𝑞𝑠𝑗 𝜎𝑢𝑗 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ )
0
𝑗=1
[
{
+∫
𝐸
𝑇
Obviously 𝐴V𝑡 ∈ F𝑡 , for each 𝑡 ∈ [0, 𝑇]. Let us define
̃V ∈ U1 by
V,
̃V𝑡 (𝜔) = { ⋆
𝑢𝑡 ,
}
𝑟𝑡 (𝑒) 𝛾𝑢 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑒) ] (𝑑𝑒) } V𝑡 𝑑𝑡
}
𝑇
𝑖=1
0
+ ∑ (∫
𝐺𝑡𝑖 𝑝𝑡 𝑑𝜉𝑡𝑐𝑖
+
∑ 𝐺𝑡𝑖
0<𝑡≤𝑇
if (𝑡, 𝜔) ∈ 𝐴V𝑡 ,
.
otherwise
(51)
If 𝜆 ⊗ P(𝐴V ) > 0, where 𝜆 denotes the Lebesgue measure,
then
𝑓𝑥 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) 𝜂𝑡 Φ𝑡 𝑑𝑡
𝑚
(49)
V
𝐸 [𝑦𝑇 𝜂𝑇 ]
0
(48)
0
𝐼 = E [ ∫ {𝑓𝑢 (𝑠, 𝑥𝑠⋆ , 𝑢𝑠⋆ ) + 𝑝𝑠 𝑏𝑢 (𝑠, 𝑥𝑠⋆ , 𝑢𝑠⋆ )
−∫
𝑐
+ ∫ {𝑘𝑡 + 𝐺𝑡 𝑝𝑡 } 𝑑(𝜉 − 𝜉⋆ )𝑡
𝑇
(𝑝𝑡− +
Δ 𝑁𝑝𝑡 ) Δ𝜉𝑡𝑖 )] .
]
(46)
Also we have
𝑇
𝐼 = E [𝑦𝑇 𝜂𝑇 + ∫ 𝑓𝑥 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) Φ𝑡 𝜂𝑡 𝑑𝑡
0
𝑇
𝑇
0
0
E [∫ 𝐻V (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑝𝑡 , 𝑞𝑡 , 𝑟𝑡 (⋅)) (̃V𝑡 − 𝑢𝑡⋆ ) 𝑑𝑡] > 0,
0
which contradicts (49), unless 𝜆 ⊗ P(𝐴V ) = 0. Hence the
conclusion follows.
(ii) If instead we choose V = 𝑢⋆ in inequality (48), we
obtain that for every measurable, F𝑡 -adapted process 𝜉 :
[0, 𝑇] × Ω → 𝐴 2 , the following inequality holds:
𝑇
(47)
𝑐
E [ ∫ {𝑘𝑡 + 𝐺𝑡 𝑝𝑡 } 𝑑(𝜉 − 𝜉⋆ )𝑡
0
+ ∫ 𝑓𝑢 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) V𝑡 𝑑𝑡 + ∫ 𝑘𝑡 𝑑𝜉𝑡 ] ,
substituting (46) in (47), the result follows.
(52)
(53)
⋆
+ ∑ {𝑘𝑡 + 𝐺𝑡 (𝑝𝑡− + Δ 𝑁𝑝𝑡 )} Δ(𝜉 − 𝜉 )𝑡 ] ≤ 0.
0<𝑡≤𝑇
8
International Journal of Stochastic Analysis
In particular, for 𝑖 = 1, . . . , 𝑚, we put 𝜉𝑡𝑖 = 𝜉𝑡⋆𝑖 +
1{𝑘𝑡𝑖 +𝐺𝑡𝑖 𝑝𝑡 >0} 𝜆(𝑡). Since the Lebesgue measure is regular then
which implies that
𝑚
𝑑
the purely discontinuous part (𝜉𝑡𝑖 − 𝜉𝑡⋆𝑖 ) = 0. Obviously, the
relation (53) can be written as
𝑚
𝑇
𝑖=1
0
∑E [∫
{𝑘𝑡𝑖
𝐺𝑡𝑖 𝑝𝑡 } 𝑑(𝜉𝑖
+
−
𝑑
0
𝑖=1
0
= ∑E [∫
{𝑘𝑡𝑖
+
𝐺𝑡𝑖 𝑝𝑡 } 1{𝑘𝑡𝑖 +𝐺𝑡𝑖 𝑝𝑡 >0} 𝑑𝜆 (𝑡)]
(54)
𝑚
> 0.
𝑐
𝑖=1
0
−∑E [∫ {𝑘𝑡𝑖 + 𝐺𝑡𝑖 𝑝𝑡 } 1{𝑘𝑡𝑖 +𝐺𝑡𝑖 𝑝𝑡 ≤0} 𝑑𝜉𝑡⋆𝑐𝑖 ] > 0.
(55)
By comparing with (53) we get
𝑚
𝑇
𝑖=1
0
∑E [∫ 1{𝑘𝑡𝑖 +𝐺𝑡𝑖 𝑝𝑡 ≤0} 𝑑𝜉𝑡⋆𝑐𝑖 ] = 0,
(56)
then we conclude that
𝑚
𝑇
∑ ∫ {𝑘𝑡𝑖 + 𝐺𝑡𝑖 𝑝𝑡 } 1{𝑘𝑡𝑖 +𝐺𝑡𝑖 𝑝𝑡 ≤0} 𝑑𝜉𝑡𝑐𝑖 = 0.
𝑖=1 0
(57)
Expressions (22) and (23) are proved by using the same
techniques. First, for each 𝑖 ∈ {1, . . . , 𝑚} and 𝑡 ∈ [0, 𝑇]
fixed, we define 𝜉𝑠𝑖 = 𝜉𝑠𝑖 + 𝛿𝑡 (𝑠)1{𝑘𝑡𝑖 +𝐺𝑡𝑖 (𝑝𝑡− +Δ 𝑁 𝑝𝑡 )>0} , where 𝛿𝑡
denotes the Dirac unit mass at 𝑡. 𝛿𝑡 is a discrete measure, then
𝑐
𝑑
(𝜉𝑠𝑖 − 𝜉𝑠𝑖 ) = 0 and (𝜉𝑠𝑖 − 𝜉𝑠𝑖 ) = 𝛿𝑡 (𝑠)1{𝑘𝑡𝑖 +𝐺𝑡𝑖 (𝑝𝑡− +Δ 𝑁 𝑝𝑡 )>0} . Hence
𝑚
E [∑ {𝑘𝑡𝑖 + 𝐺𝑡𝑖 (𝑝𝑡− + Δ 𝑁𝑝𝑡 )} 1{𝑘𝑡𝑖 +𝐺𝑡𝑖 (𝑝𝑡− +Δ 𝑁 𝑝𝑡 )>0} ] > 0 (58)
𝑖=1
which contradicts (53), unless for every 𝑖 ∈ {1, . . . , 𝑚} and
𝑡 ∈ [0, 𝑇], we have
P {𝑘𝑡𝑖
+
𝐺𝑡𝑖
(𝑝𝑡− + Δ 𝑁𝑝𝑡 ) > 0} = 0.
(59)
Next, let 𝜉 be defined by
𝑑𝜉𝑡𝑖
=1
{𝑘𝑡𝑖 +𝐺𝑡𝑖 (𝑝𝑡− +Δ 𝑁 𝑝𝑡 )≥0}
𝑑𝜉𝑡⋆𝑖
+ 1{𝑘𝑡𝑖 +𝐺𝑡𝑖 (𝑝𝑡− +Δ 𝑁 𝑝𝑡 )<0} 𝑑𝜉𝑡⋆𝑐𝑖 .
3.2. Sufficient Conditions of Optimality. It is well known
that in the classical cases (without the singular part of the
control), the sufficient condition of optimality is of significant
importance in the stochastic maximum principle, in the
sense that it allows to compute optimal controls. This result
states that, under some concavity conditions, maximizing the
Hamiltonian leads to an optimal control.
In this section, we focus on proving the sufficient maximum principle for mixed classical-singular stochastic control
problems, where the state of the system is governed by a
stochastic differential equation with jumps, allowing both
classical control and singular control.
Theorem 8 (sufficient condition of optimality in integral
form). Let (𝑢⋆ , 𝜉⋆ ) be an admissible control and denote 𝑥⋆
the associated controlled state process. Let (𝑝, 𝑞, 𝑟(⋅)) be the
unique solution of 𝐵𝑆𝐷𝐸 (18). Let one assume that (𝑥, 𝑢) →
𝐻(𝑡, 𝑥, 𝑢, 𝑝𝑡 , 𝑞𝑡 , 𝑟𝑡 (⋅)) and 𝑥 → 𝑔(𝑥) are concave functions.
Moreover suppose that for all 𝑡 ∈ [0, 𝑇], V ∈ 𝐴 1 , and 𝜉 ∈ U2
𝑇
E [ ∫ 𝐻V (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑝𝑡 , 𝑞𝑡 , 𝑟𝑡 (⋅)) (V𝑡 − 𝑢𝑡⋆ ) 𝑑𝑡
0
𝑇
𝑐
+ ∫ {𝑘𝑡 + 𝐺𝑡 𝑝𝑡 } 𝑑(𝜉 − 𝜉⋆ )𝑡
0
(64)
Then (𝑢⋆ , 𝜉⋆ ) is an optimal control.
Proof. For convenience, we will use the following notations
throughout the proof:
∑E [∑ − {𝑘𝑡𝑖 + 𝐺𝑡𝑖 (𝑝𝑡− + Δ 𝑁𝑝𝑡 )}
(61)
× 1{𝑘𝑡𝑖 +𝐺𝑡𝑖 (𝑝𝑡− +Δ 𝑁 𝑝𝑡 )<0} Δ𝜉𝑡⋆𝑖 ] > 0,
Now, by applying Itô’s formula to 𝑦𝑡⋆ Ψ𝑡 , it is easy to check
that the processes defined by relation (44) satisfy BSDE (18)
called the adjoint equation.
0<𝑡≤𝑇
(60)
𝑚
0<𝑡≤𝑇
Thus (23) holds. The proof is complete.
+ ∑ {𝑘𝑡 + 𝐺𝑡 (𝑝𝑡− + Δ 𝑁𝑝𝑡 )} Δ(𝜉 − 𝜉⋆ )𝑡 ] ≤ 0.
Then, the relation (53) can be written as
𝑖=1
(63)
𝑖=1
1{𝑘𝑡𝑖 +𝐺𝑡𝑖 𝑝𝑡− ≤0} 𝑑𝜉𝑡⋆𝑑𝑖 , for 𝑖 = 1, . . . , 𝑚, then we have 𝑑(𝜉𝑖 − 𝜉⋆𝑖 )𝑡 =
−1{𝑘𝑡𝑖 +𝐺𝑡𝑖 𝑝𝑡 ≤0} 𝑑𝜉𝑡⋆𝑐𝑖 , and 𝑑𝜉𝑡𝑑𝑖 = 𝑑𝜉𝑡⋆𝑑𝑖 . Hence, we can rewrite
(53) as follows:
𝑇
By the fact that 𝑘𝑡𝑖 + 𝐺𝑡𝑖 (𝑝𝑡− + Δ 𝑁𝑝𝑡 ) < 0, and Δ𝜉𝑡𝑖 ≥ 0, we get
∑1{𝑘𝑡𝑖 +𝐺𝑡𝑖 (𝑝𝑡− +Δ 𝑁 𝑝𝑡 )<0} Δ𝜉𝑡⋆𝑖 = 0.
This contradicts (53) unless for every 𝑖 ∈ {1, . . . , 𝑚}, 𝜆 ⊗
P{𝑘𝑡𝑖 + 𝐺𝑡𝑖 𝑝𝑡 > 0} = 0. This proves (20).
Let us prove (21). Define 𝑑𝜉𝑡𝑖 = 1{𝑘𝑡𝑖 +𝐺𝑡𝑖 𝑝𝑡− >0} 𝑑𝜉𝑡⋆𝑖 +
𝑚
(62)
× 1{𝑘𝑡𝑖 +𝐺𝑡𝑖 (𝑝𝑡− +Δ 𝑁 𝑝𝑡 )<0} Δ𝜉𝑡⋆𝑖 ] = 0.
𝑇
𝑇
𝑖=1
𝑐
𝜉⋆𝑖 )𝑡
+ ∫ {𝑘𝑡𝑖 + 𝐺𝑡𝑖 (𝑝𝑡− + Δ 𝑁𝑝𝑡 )} 𝑑(𝜉𝑖 − 𝜉⋆𝑖 )𝑡 ]
𝑚
E [∑ {𝑘𝑡𝑖 + 𝐺𝑡𝑖 (𝑝𝑡− + Δ 𝑁𝑝𝑡 )}
Θ⋆ (𝑡) = Θ (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑝𝑡 , 𝑞𝑡 , 𝑟𝑡 (⋅)) ,
Θ (𝑡) = Θ (𝑡, 𝑥𝑡 , 𝑢𝑡 , 𝑝𝑡 , 𝑞𝑡 , 𝑟𝑡 (⋅)) ,
for Θ = 𝐻, 𝐻𝑥 , 𝐻𝑢
International Journal of Stochastic Analysis
9
𝛿𝜙 (𝑡) = 𝜙 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) − 𝜙 (𝑡, 𝑥𝑡 , 𝑢𝑡 ) ,
for 𝜙 = 𝑏, 𝜎, 𝜎𝑗 , 𝑗 = 1, . . . , 𝑛, 𝑓
𝛿𝛾 (𝑡, 𝑒) = 𝛾 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑒) − 𝛾 (𝑡, 𝑥𝑡 , 𝑢𝑡 , 𝑒) ,
𝑇
0
(65)
+ ∑ 𝐺𝑡 (𝑝𝑡− + Δ 𝑁𝑝𝑡 ) Δ(𝜉 − 𝜉⋆ )𝑡 ] .
0<𝑡≤𝑇
⋆
, 𝑢𝑡⋆ , 𝑒) − 𝛾 (𝑡, 𝑥𝑡− , 𝑢𝑡 , 𝑒) .
𝛿𝛾− (𝑡, 𝑒) = 𝛾 (𝑡, 𝑥𝑡−
(68)
Let (𝑢, 𝜉) be an arbitrary admissible pair, and consider the
difference
𝐽 (𝑢⋆ , 𝜉⋆ ) − 𝐽 (𝑢, 𝜉)
𝑇
𝑇
0
0
= E [∫ 𝛿𝑓 (𝑡) 𝑑𝑡 + ∫ 𝑘𝑡 𝑑(𝜉⋆ − 𝜉)𝑡 ]
(66)
+ E [𝑔 (𝑥𝑇⋆ ) − 𝑔 (𝑥𝑇 )] .
By the fact that (𝑝, 𝑞𝑗 , 𝑟(⋅)) ∈ S2 × M2 × L2] for 𝑗 =
1, . . . , 𝑛, we deduce that the stochastic integrals with respect
to the local martingales have zero expectation. Due to the
concavity of the Hamiltonian 𝐻, the following holds
E [𝑔 (𝑥𝑇⋆ ) − 𝑔 (𝑥𝑇 )]
𝑇
≥ E [∫ {− (𝐻⋆ (𝑡) − 𝐻 (𝑡)) + 𝐻𝑢⋆ (𝑡) (𝑢𝑡⋆ − 𝑢𝑡 )} 𝑑𝑡]
We first note that, by concavity of 𝑔, we conclude that
E [𝑔 (𝑥𝑇⋆ ) − 𝑔 (𝑥𝑇 )]
0
𝑛
𝑇{
𝑗
+ E [∫ {𝑝𝑡 (𝛿𝑏 (𝑡)) + ∑ (𝛿𝜎𝑗 (𝑡)) 𝑞𝑡
0
𝑗=1
[ {
}
+ ∫ (𝛿𝛾 (𝑡, 𝑒)) 𝑟𝑡 (𝑒) ] (𝑑𝑒) } 𝑑𝑡]
𝐸
} ]
≥ E [(𝑥𝑇⋆ − 𝑥𝑇 ) 𝑔𝑥 (𝑥𝑇⋆ )] = E [(𝑥𝑇⋆ − 𝑥𝑇 ) 𝑝𝑇 ]
𝑇
𝑇
0
0
⋆
− 𝑥𝑡− ) 𝑑𝑝𝑡 + ∫ 𝑝𝑡− 𝑑 (𝑥𝑡⋆ − 𝑥𝑡 )]
= E [∫ (𝑥𝑡−
𝑇 𝑛
𝑐
+ E [ ∫ 𝐺𝑡 𝑝𝑡 𝑑(𝜉 − 𝜉⋆ )𝑡
𝑇
𝑐
+ E [ ∫ 𝐺𝑡𝑇 𝑝𝑡 𝑑(𝜉 − 𝜉⋆ )𝑡
0
+ E [∫ ∑ (𝛿𝜎
0
[ 𝑗=1
𝑗
𝑗
(𝑡)) 𝑞𝑡 𝑑𝑡
(67)
+ ∑ 𝐺𝑡T (𝑝𝑡− + Δ 𝑁𝑝𝑡 ) Δ(𝜉 − 𝜉⋆ )𝑡 ] .
0<𝑡≤𝑇
𝑇
+ ∫ ∫ (𝛿𝛾− (𝑡, 𝑒)) 𝑟𝑡 (𝑒) 𝑁 (𝑑𝑡, 𝑑𝑒) ]
0 𝐸
]
+ E [ ∑ 𝐺𝑡 (Δ 𝑁𝑝𝑡 ) Δ(𝜉 − 𝜉⋆ )𝑡 ] ,
(69)
The definition of the Hamiltonian 𝐻 and (64) leads to
𝐽(𝑢⋆ , 𝜉⋆ )−𝐽(𝑢, 𝜉) ≥ 0, which means that (𝑢⋆ , 𝜉⋆ ) is an optimal
control for the problem (14).
0<𝑡≤𝑇
which implies that
𝐸 [𝑔 (𝑥𝑇⋆ ) − 𝑔 (𝑥𝑇 )]
𝑇
≥ E [∫ (𝑥𝑡⋆ − 𝑥𝑡 ) (−𝐻𝑥⋆ (𝑡)) 𝑑𝑡]
0
𝑛
𝑇{
𝑗}
+ E [∫ {𝑝𝑡 (𝛿𝑏 (𝑡)) + ∑ (𝛿𝜎𝑗 (𝑡)) 𝑞𝑡 } 𝑑𝑡]
0
𝑗=1
[ {
} ]
𝑇
+ E [∫ ∫ (𝛿𝛾− (𝑡, 𝑒)) 𝑟𝑡 (𝑒) 𝑁 (𝑑𝑡, 𝑑𝑒)]
0
𝐸
𝑇
+ E [∫ {(𝑥𝑡⋆ − 𝑥𝑡 ) 𝑞𝑡 + (𝛿𝜎 (𝑡)) 𝑝𝑡 } 𝑑𝐵𝑡 ]
0
𝑇
+ E [∫ ∫
0
𝐸
⋆
{(𝑥𝑡−
− 𝑥𝑡− ) 𝑟𝑡 (𝑒) + 𝑝𝑡− (𝛿𝛾− (𝑡, 𝑒))}
̃ (𝑑𝑡, 𝑑𝑒) ]
×𝑁
The expression (64) is a sufficient condition of optimality
in integral form. We want to rewrite this inequality in a
suitable form for applications. This is the objective of the
following theorem which could be seen as a natural extension
of [2, Theorem 2.2] to the jump setting and [3, Theorem 2.1]
to mixed regular-singular control problems.
Theorem 9 (sufficient conditions of optimality). Let (𝑢⋆ , 𝜉⋆ )
be an admissible control and 𝑥⋆ the associated controlled state
process. Let (𝑝, 𝑞, 𝑟(⋅)) be the unique solution of 𝐵𝑆𝐷𝐸 (18). Let
one assume that (𝑥, 𝑢) → 𝐻(𝑡, 𝑥, 𝑢, 𝑝𝑡 , 𝑞𝑡 , 𝑟𝑡 (⋅)) and 𝑥 →
𝑔(𝑥) are concave functions. If in addition one assumes that
(i) for all 𝑡 ∈ [0, 𝑇], V ∈ 𝐴 1
𝐻 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑝𝑡 , 𝑞𝑡 , 𝑟𝑡 (⋅)) = sup 𝐻 (𝑡, 𝑥𝑡⋆ , V, 𝑝𝑡 , 𝑞𝑡 , 𝑟𝑡 (⋅)) ,
V∈𝐴 1
𝑑𝑡—𝑎.𝑒., P—𝑎.𝑠;
(70)
10
International Journal of Stochastic Analysis
(ii) for all 𝑡 ∈ [0, 𝑇], with probability 1
𝑘𝑡𝑖 + 𝐺𝑡𝑖 𝑝𝑡 ≤ 0,
for 𝑖 = 1, . . . , 𝑚,
𝑚
∑1{𝑘𝑡𝑖 +𝐺𝑡𝑖 𝑝𝑡 ≤0} 𝑑𝜉𝑡⋆𝑐𝑖
𝑖=1
𝑘𝑡𝑖 + 𝐺𝑡𝑖 (𝑝𝑡− + Δ 𝑁𝑝𝑡 ) ≤ 0,
= 0,
for 𝑖 = 1, . . . , 𝑚,
Since our objective is to maximize this functional, the
value function of the singular control problem becomes
(71)
(𝑢,𝜉)∈U
(72)
(73)
𝑚
∑1{𝑘𝑡𝑖 +𝐺𝑡𝑖 (𝑝𝑡− +Δ 𝑁 𝑝𝑡 )≤0} Δ𝜉𝑡⋆𝑖 = 0.
𝑉 (𝑡, 𝑥) = sup 𝐽(𝑢,𝜉) (𝑡, 𝑥) .
If we do not apply any singular control, then the infinitesimal generator A𝑢 , associated with (8), acting on functions 𝜑, coincides on 𝐶𝑏2 (R𝑛 ; R) with the parabolic integrodifferential operator A𝑢 given by
(74)
𝑖=1
𝑛
A𝑢 𝜑 (𝑡, 𝑥) = ∑𝑏𝑖 (𝑡, 𝑥, 𝑢)
𝑖=1
Then (𝑢⋆ , 𝜉⋆ ) is an optimal control.
+
Proof. Using (71) and (72) yields
𝑇
𝑚
0
𝑖=1 0
𝜕𝜑
(𝑡, 𝑥)
𝜕𝑥𝑖
𝜕2 𝜑
1 𝑛 𝑖𝑗
∑ 𝑎 (𝑡, 𝑥, 𝑢) 𝑖 𝑗 (𝑡, 𝑥)
2 𝑖,𝑗=1
𝜕𝑥 𝜕𝑥
𝑇
E [∫ {𝑘𝑡 + 𝐺𝑡 𝑝𝑡 } 𝑑𝜉𝑡⋆𝑐 ] = E [∑ ∫ {𝑘𝑡𝑖 + 𝐺𝑡𝑖 𝑝𝑡 } 𝑑𝜉𝑡⋆𝑐𝑖 ] = 0.
+ ∫ {𝜑 (𝑡, 𝑥 + 𝛾 (𝑡, 𝑥, 𝑢, 𝑒)) − 𝜑 (𝑡, 𝑥)
𝐸
(75)
𝑛
−∑𝛾𝑖 (𝑡, 𝑥, 𝑢, 𝑒)
The same computations applied to (73) and (74) imply
E [ ∑ {𝑘𝑡 + 𝐺𝑡 (𝑝𝑡− + Δ 𝑁𝑝𝑡 )} Δ𝜉𝑡⋆ ] = 0.
(76)
0<𝑡≤𝑇
Hence, from Definition 1, we have the following inequality:
𝑇
(79)
𝑖=1
𝜕𝜑
(𝑡, 𝑥)} ] (𝑑𝑒) ,
𝜕𝑥𝑖
(80)
where 𝑎𝑖𝑗 = ∑𝑑ℎ=1 (𝜎𝑖ℎ 𝜎𝑗ℎ ) denotes the generic term of the
symmetric matrix 𝜎𝜎𝑇 . The variational inequality associated
to the singular control problem is
max {sup𝐻1 (𝑡, 𝑥, (𝑊, 𝜕𝑡 𝑊, 𝑊𝑥 , 𝑊𝑥𝑥 ) (𝑡, 𝑥) , 𝑢) ,
𝑐
𝑢
E [ ∫ {𝑘𝑡 + 𝐺𝑡 𝑝𝑡 } 𝑑(𝜉 − 𝜉⋆ )𝑡
0
(77)
+ ∑ {𝑘𝑡 + 𝐺𝑡 (𝑝𝑡− + Δ 𝑁𝑝𝑡 )} Δ(𝜉 − 𝜉⋆ )𝑡 ] ≤ 0.
𝐻2𝑙
(81)
(𝑡, 𝑥, 𝑊𝑥 (𝑡, 𝑥)) , 𝑙 = 1, . . . , 𝑚} = 0,
for (𝑡, 𝑥) ∈ [0, 𝑇] × 𝑂,
0<𝑡≤𝑇
𝑊 (𝑇, 𝑥) = 𝑔 (𝑥) ,
The desired result follows from Theorem 8.
∀𝑥 ∈ 𝑂.
(82)
𝐻1 and 𝐻2𝑙 , for 𝑙 = 1, . . . , 𝑚, are given by
4. Relation to Dynamic Programming
In this section, we come back to the control problem studied
in the previous section. We recall a verification theorem,
which is useful to compute optimal controls. Then we show
that the adjoint process defined in Section 3, as the unique
solution to the BSDE (18), can be expressed as the gradient
of the value function, which solves the HJB variational
inequality.
4.1. A Verification Theorem. Let 𝑥𝑠𝑡,𝑥 be the solution of the
controlled SDE (8), for 𝑠 ≥ 𝑡, with initial value 𝑥𝑡 = 𝑥. To put
the problem in a Markovian framework, so that we can apply
dynamic programming, we define the performance criterion
𝐻1 (𝑡, 𝑥, (𝑊, 𝜕𝑡 𝑊, 𝑊𝑥 , 𝑊𝑥𝑥 ) (𝑡, 𝑥) , 𝑢)
=
𝜕𝑊
(𝑡, 𝑥) + A𝑢 𝑊 (𝑡, 𝑥) + 𝑓 (𝑡, 𝑥, 𝑢) ,
𝜕𝑡
(83)
𝑛
𝜕𝑊
(𝑡, 𝑥) 𝐺𝑡𝑖𝑙 + 𝑘𝑡𝑙 .
𝑖
𝜕𝑥
𝑖=1
𝐻2𝑙 (𝑡, 𝑥, 𝑊𝑥 (𝑡, 𝑥)) = ∑
We start with the definition of classical solutions of the
variational inequality (81).
Definition 10. Let one consider a function 𝑊 ∈ 𝐶1,2 ([0, 𝑇] ×
𝑂), and define the nonintervention region by
𝐶 (𝑊) = { (𝑡, 𝑥) ∈ [0, 𝑇] × 𝑂,
𝐽(𝑢,𝜉) (𝑡, 𝑥)
𝑇
𝑇
= E [∫ 𝑓 (𝑠, 𝑥𝑠 , 𝑢𝑠 ) 𝑑𝑠 + ∫ 𝑘𝑠 𝑑𝜉𝑠 + 𝑔 (𝑥𝑇 ) | 𝑥𝑡 = 𝑥] .
𝑡
𝑡
(78)
𝑛
𝜕𝑊
max ∑ { 𝑖 (𝑡, 𝑥) 𝐺𝑡𝑖𝑙 + 𝑘𝑡𝑙 } < 0} .
1≤𝑙≤𝑚
𝑖=1 𝜕𝑥
(84)
International Journal of Stochastic Analysis
11
We say that 𝑊 is a classical solution of (81) if
𝜕𝑊
(𝑡, 𝑥) + sup {A𝑢 𝑊 (𝑡, 𝑥) + 𝑓 (𝑡, 𝑥, 𝑢)} = 0,
𝜕𝑡
𝑢
(85)
∀ (𝑡, 𝑥) ∈ 𝐶 (𝑊) ,
𝑛
𝜕𝑊
(𝑡, 𝑥) 𝐺𝑡𝑖𝑙 + 𝑘𝑡𝑙 ≤ 0,
𝑖
𝜕𝑥
𝑖=1
∑
Here 𝜉𝑡 is the total number of individuals harvested up
to time 𝑡. If we define the price per unit harvested at time
𝑡 by 𝑘(𝑡) = 𝑒−𝜘𝑡 and the utility rate obtained when the size
𝛾
of the population at 𝑡 is 𝑋𝑡 by 𝑒−𝜘𝑡 𝑋𝑡 . Then the objective is
to maximize the expected total time-discounted value of the
harvested individuals starting with a population of size 𝑥; that
is,
𝑇
(86)
0
∀ (𝑡, 𝑥) ∈ [0, 𝑇] × 𝑂, for 𝑙 = 1, . . . , 𝑚,
𝜕𝑊
(𝑡, 𝑥) + A𝑢 𝑊 (𝑡, 𝑥) + 𝑓 (𝑡, 𝑥, 𝑢) ≤ 0,
𝜕𝑡
(87)
for every (𝑡, 𝑥, 𝑢) ∈ [0, 𝑇] × 𝑂 × 𝐴 1 .
The following verification theorem is very useful to
compute explicitly the value function and the optimal control,
at least in the case where the value function is sufficiently
smooth.
Theorem 11. Let 𝑊 be a classical solution of (81) with the
terminal condition (82), such that for some constants 𝑐1 ≥
1, 𝑐2 ∈ (0, ∞), |𝑊(𝑡, 𝑥)| ≤ 𝑐2 (1 + |𝑥|𝑐1 ). Then, for all (𝑡, 𝑥) ∈
[0, 𝑇] × 𝑂, and (𝑢, 𝜉) ∈ U
𝑊 (𝑡, 𝑥) ≥ 𝐽(𝑢,𝜉) (𝑡, 𝑥) .
(88)
Furthermore, if there exists (𝑢⋆ , 𝜉⋆ ) ∈ U such that with
probability 1
(𝑡, 𝑥𝑡⋆ ) ∈ 𝐶 (𝑊) ,
𝐿𝑒𝑏𝑒𝑠𝑔𝑢𝑒 𝑎𝑙𝑚𝑜𝑠𝑡 𝑒𝑣𝑒𝑟𝑦 𝑡 ≤ 𝑇,
𝑢𝑡⋆ ∈ arg max {A𝑢 𝑊 (𝑡, 𝑥𝑡⋆ ) + 𝑓 (𝑡, 𝑥𝑡⋆ , 𝑢)} ,
𝑢
𝑚
𝑙=1
(89)
(90)
𝑛
𝜕𝑊
(𝑡, 𝑥𝑡⋆ ) 𝐺𝑡𝑖𝑙 = 𝑘𝑡𝑙 } 𝑑𝜉𝑡⋆𝑐𝑙 = 0,
𝑖
𝜕𝑥
𝑖−1
∑ {∑
[0,𝑇)
𝑒−𝜘𝑡 𝑑𝜉𝑡 ] ,
𝜙 (𝑡, 𝑥) = sup 𝐽𝜉 (𝑡, 𝑥) .
(95)
𝜉∈Π(𝑡,𝑥)
Note that the definition of Π(𝑡, 𝑥) is similar to Π(𝑥), except
that the starting time is 𝑡, and the state at 𝑡 is 𝑥.
If we guess the nonintervention region 𝐶 has the form 𝐶 =
{(𝑡, 𝑥) : 0 < 𝑥 < 𝑏} for some barrier point 𝑏 > 0, then (84)
gets the form,
0=
𝜕2 Φ
𝜕Φ
1
𝜕Φ
(𝑡, 𝑥) + 𝜇𝑥
(𝑡, 𝑥) + 𝜎2 𝑥2 2 (𝑡, 𝑥)
𝜕𝑡
𝜕𝑥
2
𝜕𝑥
+ ∫ {Φ (𝑡, 𝑥 (1 + 𝜃𝑒)) − Φ (𝑡, 𝑥) − 𝜃𝑥𝑒
R+
𝜕Φ
(𝑡, 𝑥)} ] (𝑑𝑒)
𝜕𝑥
+ 𝑥𝛾 exp (−𝜘𝑡) ,
(96)
(91)
for 0 < 𝑥 < 𝑏. We try a solution Φ of the form
(92)
𝑙=1
for all jumping times 𝑡 of 𝜉𝑡⋆ , then it follows that 𝑊(𝑡, 𝑥) =
⋆ ⋆
𝐽(𝑢 ,𝜉 ) (𝑡, 𝑥).
Φ (𝑡, 𝑥) = Ψ (𝑥) exp (−𝜘𝑡) ;
(97)
AΦ (𝑡, 𝑥) = exp (−𝜘𝑡) A0 Ψ (𝑥) ,
(98)
hence
where Ψ is the fundamental solution of the ordinary integrodifferential equation
Proof. See [8, Theorem 5.2].
In the following, we present an example on optimal
harvesting from a geometric Brownian motion with jumps
see, for example, [5, 8].
Example 12. Consider a population having a size 𝑋 = {𝑋𝑡 :
𝑡 ≥ 0} which evolves according to the geometric Lévy process;
that is
1
− 𝜘Ψ (𝑥) + 𝜇𝑥Ψ󸀠 (𝑥) + 𝜎2 𝑥2 Ψ󸀠󸀠 (𝑥)
2
+ ∫ {Ψ (𝑥 (1 + 𝜃𝑒)) − Ψ (𝑥) − 𝜃𝑥𝑒Ψ󸀠 (𝑥)} ] (𝑑𝑒)
R+
+ 𝑥𝛾 = 0.
𝑑𝑋𝑡 = 𝜇𝑋𝑡 𝑑𝑡 + 𝜎𝑋𝑡 𝑑𝐵𝑡
̃ (𝑑𝑡, 𝑑𝑒) − 𝑑𝜉𝑡 ,
+ 𝜃𝑋𝑡− ∫ 𝑒𝑁
R+
𝑋0− = 𝑥 > 0.
(94)
where 𝑇 := inf{𝑡 ≥ 0 : 𝑋𝑡 = 0} is the time of complete
depletion, 𝛾 ∈ (0, 1) and 𝜇, 𝜎, 𝜘, 𝜃 are positive constants with
𝜎2 /2 + 𝜃 ∫R 𝑒](𝑑𝑒) ≤ 𝜇 < 𝜘. The harvesting admissible
+
strategy 𝜉𝑡 is assumed to be nonnegative, nondecreasing
continuous on the right, satisfying 𝐸|𝜉𝑇 |2 < ∞ with 𝜉0− = 0,
and such that 𝑋𝑡 > 0. We denote by Π(𝑥) the class of such
strategies. For any 𝜉 define
𝑚
Δ 𝜉 𝑊 (𝑡, 𝑥𝑡⋆ ) + ∑𝑘𝑡𝑙 Δ𝜉𝑡⋆𝑙 = 0,
𝛾
𝐽 (𝜉) = E [∫ 𝑒−𝜘𝑡 𝑋𝑡 𝑑𝑡 + ∫
(99)
for 𝑡 ∈ [0, 𝑇] , (93)
We notice that Ψ(𝑥) = 𝐴𝑥𝜌 +𝐾𝑥𝛾 , for some arbitrary constant
𝐴; we get
AΦ (𝑡, 𝑥) = 𝑥𝛾 (𝐴ℎ1 (𝜌) + ℎ2 (𝛾)) exp (−𝜘𝑡) ,
(100)
12
International Journal of Stochastic Analysis
where
1
1
ℎ1 (𝜌) = 𝜎2 𝜌2 + (𝜇 − 𝜎2 ) 𝜌
2
2
+ ∫ {(1 + 𝜃𝑒)𝜌 − 1 − 𝜃𝑒𝜌} ] (𝑑𝑒) − 𝜘,
R+
1
1
ℎ2 (𝛾) = 𝐾 ( 𝜎2 𝛾2 + (𝜇 − 𝜎2 ) 𝛾
2
2
+ ∫ {(1 + 𝜃𝑒)𝛾 − 1 − 𝜃𝑒𝛾} ] (𝑑𝑒) − 𝜘) +1.
R+
(101)
Note that ℎ1 (1) = 𝜇 − 𝜘 < 0 and lim𝑟 → ∞ ℎ1 (𝜌) = ∞; then
there exists 𝜌 > 1 such that ℎ1 (𝜌) = 0. The constant 𝐾 is given
by
1
1
𝐾 = − ( 𝜎2 𝛾2 + (𝜇 − 𝜎2 ) 𝛾
2
2
−1
𝛾
(102)
+ ∫ {(1 + 𝜃𝑒) − 1 − 𝜃𝑒𝛾} ] (𝑑𝑒) − 𝜘) .
R+
⋆𝜌
Outside 𝐶 we require that Ψ(𝑥) = 𝑥 + 𝐵, where 𝐵 is a
constant to be determined. This suggests that the value must
be of the form
Φ (𝑡, 𝑥) = {
Arguing as in [7], we can adapt Theorem 15 in [16] to obtain
an identification of the optimal harvesting strategy as a local
time of a reflected jump diffusion process. Then, the system
(106)–(109) defines the so-called Skorokhod problem, whose
solution is a pair (𝑋𝑡⋆ , 𝜉𝑡⋆ ), where 𝑋𝑡⋆ is a jump diffusion
process reflected at 𝑏.
The conditions (89)–(92) ensure the existence of an
increasing process 𝜉𝑡⋆ such that 𝑋𝑡⋆ stays in 𝐶 for all times
𝑡. If the initial size 𝑥 ≤ 𝑏, then 𝜉𝑡⋆ is nondecreasing and his
continuous part 𝜉𝑡⋆𝑐 increases only when 𝑋𝑡⋆ = 𝑏 so as to
ensure that 𝑋𝑡⋆ ≤ 𝑏.
On the other hand, we only have Δ𝜉𝑡⋆ > 0 if the initial
⋆
= 𝑥 − 𝑏, or if 𝑋𝑡⋆ jumps out of the
size 𝑥 > 𝑏 then 𝜉0−
nonintervention region by the random measure 𝑁; that is,
⋆
+ Δ 𝑁𝑋𝑡⋆ > 𝑏. In these cases we get Δ𝜉𝑡⋆ > 0 immediately
𝑋𝑡−
to bring 𝑋𝑡⋆ to 𝑏.
It is easy to verify that, if (𝑋⋆ , 𝜉⋆ ) is a solution of the
Skorokhod problem (106)–(109), then (𝑋⋆ , 𝜉⋆ ) is an optimal
solution of the problem (93) and (94).
By the construction of 𝜉⋆ and Φ, all the conditions of the
verification Theorem 11 are satisfied. More precisely, the value
function along the optimal state reads as
(𝐴𝑥𝜌 + 𝐾𝑥𝛾 ) exp (−𝜘𝑡)
(𝑥 + 𝐵) exp (−𝜘𝑡)
for 0 < 𝑥 < 𝑏,
(103)
for 𝑥 ≥ 𝑏.
Assuming smooth fit principle at point 𝑏, then the reflection threshold is
𝑏=(
𝐾𝛾 (1 − 𝛾)
)
𝐴𝜌 (𝜌 − 1)
1/(𝜌−𝛾)
,
(104)
where
𝐴=
1 − 𝐾𝛾𝑏𝛾−1
,
𝜌𝑏𝜌−1
(105)
⋆𝛾
Φ (𝑡, 𝑋𝑡⋆ ) = (𝐴𝑋𝑡 + 𝐾𝑋𝑡 ) exp (−𝜘𝑡) ,
for all 𝑡 ∈ [0, 𝑇] .
4.2. Link between the SMP and DPP. Compared with the
stochastic maximum principle, one would expect that the
solution (𝑝, 𝑞, 𝑟(⋅)) of BSDE (18) to correspond to the derivatives of the classical solution of the variational inequalities
(81)-(82). This is given by the following theorem, which
extends [3, Theorem 3.1] to control problems with a singular
component and [2, Theorem 3.3] to diffusions with jumps.
Theorem 13. Let 𝑊 be a classical solution of (81), with the
terminal condition (82). Assume that 𝑊 ∈ 𝐶1,3 ([0, 𝑇] × 𝑂),
with 𝑂 = R𝑛 , and there exists (𝑢⋆ , 𝜉⋆ ) ∈ U such that the
conditions (89)–(92) are satisfied. Then the solution of the
BSDE (18) is given by
𝐵 = 𝐴𝑏 + 𝐾𝑏 − 𝑏.
𝑝𝑡 = 𝑊𝑥 (𝑡, 𝑥𝑡⋆ ) ,
Since 𝛾 < 1 and 𝜌 > 1, we deduce that 𝑏 > 0.
To construct the optimal control 𝜉⋆ , we consider the
stochastic differential equation
𝑞𝑡 = 𝑊𝑥𝑥 (𝑡, 𝑥𝑡⋆ ) 𝜎 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) ,
𝜌
𝛾
̃ (𝑑𝑡, 𝑑𝑒) − 𝑑𝜉⋆ ,
𝑑𝑋𝑡⋆ = 𝜇𝑋𝑡⋆ 𝑑𝑡 + 𝜎𝑋𝑡⋆ 𝑑𝐵𝑡 + ∫ 𝜃𝑋𝑡⋆ 𝑒𝑁
𝑡
R+
(106)
𝑋𝑡⋆ ≤ 𝑏,
𝑡 ≥ 0,
(107)
1{𝑋𝑡⋆ <𝑏} 𝑑𝜉𝑡⋆𝑐 = 0,
(108)
1{𝑋𝑡−⋆ +Δ 𝑁 𝑋𝑡⋆ ≤𝑏} Δ𝜉𝑡⋆ = 0,
(109)
and if this is the case, then
⋆
Δ𝜉𝑡⋆ = min {𝑙 > 0 : 𝑋𝑡−
+ Δ 𝑁𝑋𝑡⋆ − 𝑙 = 𝑏} .
(110)
(111)
𝑟𝑡 (⋅) =
𝑊𝑥 (𝑡, 𝑥𝑡⋆
+
𝛾 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑒))
−
(112)
𝑊𝑥 (𝑡, 𝑥𝑡⋆ ) .
Proof. Throughout the proof, we will use the following
abbreviations: for 𝑖, 𝑗 = 1, . . . , 𝑛, and ℎ = 1, . . . , 𝑑,
𝜙1 (𝑡) = 𝜙1 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ ) ,
for 𝜙1 = 𝑏𝑖 , 𝜎𝑖 , 𝜎𝑖ℎ , 𝜎, 𝑎𝑖𝑗 ,
𝜙2 (𝑡, 𝑒) = 𝜙2 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑒) ,
⋆
, u⋆𝑡 , 𝑒) ,
𝛾− (𝑡, 𝑒) = 𝛾 (𝑡, 𝑥𝑡−
𝜕𝑏𝑖 𝜕𝑏 𝜕𝑎𝑖𝑗 𝜕𝜎𝑖ℎ 𝜕𝑓
,
,
,
,
,
𝜕𝑥𝑘 𝜕𝑥𝑘 𝜕𝑥𝑘 𝜕𝑥𝑘 𝜕𝑥𝑘
for 𝜙2 = 𝛾, 𝛾𝑖 ,
𝜕𝛾𝑖 𝜕𝛾
,
,
𝜕𝑥𝑘 𝜕𝑥𝑘
⋆
𝛾−𝑖 (𝑡, 𝑒) = 𝛾𝑖 (𝑡, 𝑥𝑡−
, 𝑢𝑡⋆ , 𝑒) .
(113)
International Journal of Stochastic Analysis
13
𝜏𝑅⋆ 𝑛
𝜕2 𝑊
(𝑠, 𝑥𝑠⋆ ) 𝜎𝑖 (𝑠) 𝑑𝐵𝑠
𝑘 𝜕𝑥𝑖
𝜕𝑥
𝑖=1
From Itô’s rule applied to the semimartingale (𝜕𝑊/
𝜕𝑥𝑘 )(𝑡, 𝑥𝑡⋆ ), one has
+∫ ∑
𝜕𝑊 ⋆ ⋆
(𝜏 , 𝑥 ⋆ )
𝜕𝑥𝑘 𝑅 𝜏𝑅
+∫ ∫ {
𝑡
𝜏𝑅⋆
𝑡
𝜕𝑊
⋆
(𝑠, 𝑥𝑠−
+ 𝛾− (𝑠, 𝑒))
𝜕𝑥𝑘
𝐸
⋆
=
𝜏𝑅
𝜕2 𝑊
𝜕𝑊
(𝑡, 𝑥𝑡⋆ ) + ∫
(𝑠, 𝑥𝑠⋆ ) 𝑑𝑠
𝑘
𝑘
𝜕𝑥
𝑡 𝜕𝑠𝜕𝑥
−
𝜏𝑅⋆ 𝑛
𝜏𝑅⋆ 𝑛
𝜕2 𝑊
⋆
) 𝑑𝑥𝑠⋆𝑖
+ ∫ ∑ 𝑘 𝑖 (𝑠, 𝑥𝑠−
𝜕𝑥
𝜕𝑥
𝑡 𝑖=1
𝜏𝑅⋆
1
∫
2 𝑡
+
𝑛
∑ 𝑎𝑖𝑗 (𝑠)
𝑖,𝑗=1
𝜏𝑅⋆
+∫ ∫ {
𝑡
𝐸
𝑡
𝜕3 𝑊
(𝑠, 𝑥𝑠⋆ ) 𝑑𝑠
𝜕𝑥𝑘 𝜕𝑥𝑖 𝜕𝑥𝑗
+ ∑ Δ𝜉
𝑡<𝑠≤𝜏𝑅⋆
𝜕𝑊
(𝑠, 𝑥𝑠⋆ ) .
𝜕𝑥𝑘
(116)
𝑛
𝜕2 𝑊
⋆
(𝑠, 𝑥𝑠−
) 𝛾−𝑖 (𝑠, 𝑒)} 𝑁 (𝑑𝑠, 𝑑𝑒)
𝑘
𝑖
𝑖=1 𝜕𝑥 𝜕𝑥
−∑
𝑡<𝑠≤𝜏𝑅⋆
𝑚
𝜕2 𝑊
⋆
(𝑠,
𝑥
)
𝐺𝑠𝑖𝑙 𝑑𝜉𝑠⋆𝑐𝑙
∑
𝑠
𝑘 𝜕𝑥𝑖
𝜕𝑥
𝑖=1
𝑙=1
+∫ ∑
𝜕𝑊
𝜕𝑊
⋆
⋆
(𝑠, 𝑥𝑠−
+ 𝛾− (𝑠, 𝑒)) − 𝑘 (𝑡, 𝑥𝑠−
)
𝜕𝑥𝑘
𝜕𝑥
+ ∑ {Δ 𝜉
𝜕𝑊
⋆
̃ (𝑑𝑠, 𝑑𝑒)
(𝑠, 𝑥𝑠−
)} 𝑁
𝜕𝑥𝑘
Let 𝜉𝑠⋆𝑐 denotes the continuous part of 𝜉𝑠⋆ ; that is, 𝜉𝑠⋆𝑐 = 𝜉𝑠⋆ −
∑𝑡<𝑠≤𝜏𝑅⋆ Δ𝜉𝑠⋆𝑙 . Then, we can easily show that
𝜏𝑅⋆ 𝑛
𝜕2 𝑊
(𝑠, 𝑥𝑠⋆ ) 𝐺𝑠𝑖𝑙 𝑑𝜉𝑠⋆𝑐𝑙
𝑘 𝜕𝑥𝑖
𝜕𝑥
𝑖=1
∫ ∑
𝑡
𝜕𝑊
(𝑠, 𝑥𝑠⋆ )
𝜕𝑥𝑘
𝜏𝑅⋆ 𝑛
𝜕2 𝑊
(𝑠, 𝑥𝑠⋆ ) 𝐺𝑠𝑖𝑙 1{(𝑠,𝑥𝑠⋆ )∈𝐷𝑙 } 𝑑𝜉𝑠⋆𝑐𝑙
𝑘 𝜕𝑥𝑖
𝜕𝑥
𝑖=1
=∫ ∑
𝑡
𝑛
𝜕2 𝑊
⋆
−∑ 𝑘 𝑖 (𝑠, 𝑥𝑠−
) Δ 𝜉 𝑥𝑠⋆𝑖 } ,
𝜕𝑥
𝜕𝑥
𝑖=1
𝜏𝑅⋆ 𝑛
where 𝜏⋆ is defined as in Theorem 11, and the sum is taken
over all jumping times 𝑠 of 𝜉⋆ . Note that
𝑡
For every (𝑡, 𝑥) ∈ 𝐷𝑙 , using (88) we have
𝑛
𝑛
𝜕2 𝑊
𝜕𝑊
𝜕
𝑖𝑙
=
{
𝑥)
𝐺
∑
(𝑡,
(𝑡, 𝑥) 𝐺𝑡𝑖𝑙 + 𝑘𝑠𝑙 } = 0,
𝑡
𝑖
𝑘 𝜕𝑥𝑖
𝑘
𝜕𝑥
𝜕𝑥
𝜕𝑥
𝑖=1
𝑖=1
∑
𝑚
𝑙=1
𝜕2 𝑊
(𝑠, 𝑥𝑠⋆ ) 𝐺𝑠𝑖𝑙 1{(𝑠,𝑥𝑠⋆ )∈𝐶𝑙 } 𝑑𝜉𝑠⋆𝑐𝑙 .
𝑘 𝜕𝑥𝑖
𝜕𝑥
𝑖=1
+∫ ∑
(114)
⋆𝑖
+ Δ 𝑁𝑥𝑠⋆𝑖 ) = ∑𝐺𝑠𝑖𝑙 Δ𝜉𝑠⋆𝑙 ,
Δ 𝜉 𝑥𝑠⋆𝑖 = 𝑥𝑠⋆𝑖 − (𝑥𝑠−
(115)
for 𝑙 = 1, . . . , 𝑚.
(118)
for 𝑖 = 1, . . . , 𝑛,
⋆𝑙
where Δ𝜉𝑠⋆𝑙 = 𝜉𝑠⋆𝑙 − 𝜉𝑠−
is a pure jump process. Then, we can
rewrite (114) as follows:
+∫ {
𝑡
𝑛
𝜕2 𝑊
𝜕2 𝑊
⋆
𝑖
(𝑠,
𝑥
)
+
𝑏
(𝑠, 𝑥𝑠⋆ )
∑
(𝑠)
𝑠
𝑘 𝜕𝑥𝑖
𝜕𝑠𝜕𝑥𝑘
𝜕𝑥
𝑖=1
+
𝜏𝑅⋆ 𝑛 𝑚
𝑡
𝜕𝑊
(𝑡, 𝑥𝑡⋆ )
𝜕𝑥𝑘
𝜏𝑅⋆
This proves
𝜕2 𝑊
(𝑠, 𝑥𝑠⋆ ) 𝐺𝑠𝑖𝑙 1{(𝑠,𝑥𝑠⋆ )∈𝐷𝑙 } 𝑑𝜉𝑠⋆𝑐𝑙 = 0.
𝑘 𝜕𝑥𝑖
𝜕𝑥
𝑖=1 𝑙=1
∫ ∑∑
𝜕𝑊 ⋆ ⋆
(𝜏 , 𝑥 ⋆ )
𝜕𝑥𝑘 𝑅 𝜏𝑅
=
𝜕3 𝑊
1 𝑛 𝑖𝑗
∑ 𝑎 (𝑠) 𝑘 𝑖 𝑗 (𝑠, 𝑥𝑠⋆ )
2 𝑖,𝑗=1
𝜕𝑥 𝜕𝑥 𝜕𝑥
𝜕𝑊
𝜕𝑊
⋆
+ ∫ ( 𝑘 (𝑠, 𝑥𝑠⋆ + 𝛾 (𝑠, 𝑒)) − 𝑘 (𝑠, 𝑥𝑠−
)
𝜕𝑥
𝜕𝑥
𝐸
𝑛
2
𝜕𝑊
(𝑠, 𝑥𝑠⋆ ) 𝛾𝑖 (𝑠, 𝑒)) ] (𝑑𝑒)} 𝑑𝑠
𝑘
𝑖
𝑖=1 𝜕𝑥 𝜕𝑥
−∑
(117)
(119)
Furthermore, for every (𝑡, 𝑥) ∈ 𝐶𝑙 and 𝑙 = 1, . . . , 𝑚, we have
∑𝑛𝑖=1 (𝜕𝑊/𝜕𝑥𝑘 𝜕𝑥𝑖 )(𝑡, 𝑥)𝐺𝑡𝑖𝑙 < 0.
⋆𝑐𝑙
But (91) implies that ∑𝑚
𝑙=1 1{(𝑠,𝑥𝑠⋆ )∈𝐶𝑙 } 𝑑𝜉𝑠 = 0; thus
𝜏𝑅⋆ 𝑛 𝑚
𝜕2 𝑊
(𝑠, 𝑥𝑠⋆ ) 𝐺𝑠𝑖𝑙 1{(𝑠,𝑥𝑠⋆ )∈𝐶𝑙 } 𝑑𝜉𝑠⋆𝑐𝑙 = 0.
𝑘 𝜕𝑥𝑖
𝜕𝑥
𝑖=1 𝑙=1
∫ ∑∑
𝑡
(120)
The mean value theorem yields
Δ𝜉
𝜕𝑊
𝜕𝑊
(𝑠, 𝑥𝑠⋆ ) = ( 𝑘 ) (𝑠, 𝑦 (𝑠)) Δ 𝜉 𝑥𝑠⋆ ,
𝑘
𝜕𝑥
𝜕𝑥 𝑥
(121)
⋆
where 𝑦(𝑠) is some point on the straight line between 𝑥𝑠−
+
⋆
⋆
𝑘
Δ 𝑁𝑥𝑠 and 𝑥𝑠 , and (𝜕𝑊/𝜕𝑥 )𝑥 represents the gradient matrix
of 𝜕𝑊/𝜕𝑥𝑘 . To prove that the right-hand side of the above
14
International Journal of Stochastic Analysis
𝑛
𝜕𝑏𝑖
𝜕𝑊
(𝑡) 𝑖 (𝑡, 𝑥𝑡⋆ )
𝑘
𝜕𝑥
𝜕𝑥
𝑖=1
equality vanishes, it is enough to check that if Δ𝜉𝑠⋆𝑙 > 0 then
∑𝑛𝑖=1 (𝜕2 𝑊/𝜕𝑥𝑘 𝜕𝑥𝑖 )(𝑠, 𝑦(𝑠))𝐺𝑠𝑖𝑙 = 0, for 𝑙 = 1, . . . , 𝑚. It is clear
by (92) that
= −∑
𝑚
0 = Δ 𝜉 𝑊 (𝑠, 𝑥𝑠⋆ ) + ∑𝑘𝑠𝑙 Δ𝜉𝑠⋆𝑙
𝑙=1
𝑚
𝜕𝑓
1 𝑛 𝜕𝑎𝑖𝑗
𝜕2 𝑊
∑ 𝑘 (𝑡, 𝑥𝑡⋆ ) 𝑖 𝑗 (𝑡, 𝑥𝑡⋆ ) − 𝑘 (𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ )
2 𝑖,𝑗=1 𝜕𝑥
𝜕𝑥 𝜕𝑥
𝜕𝑥
−
𝑛
𝜕𝛾𝑖
(𝑡, 𝑒)
𝑘
𝐸 𝑖=1 𝜕𝑥
(122)
𝑛
𝜕𝑊
(𝑠, 𝑦 (𝑠)) 𝐺𝑠𝑖𝑙 + 𝑘𝑠𝑙 } Δ𝜉𝑠⋆𝑙 .
𝑖
𝜕𝑥
𝑖=1
−∫ ∑
= ∑ {∑
𝑙=1
×{
Δ𝜉𝑠⋆𝑙
> 0, then (𝑠, 𝑦(𝑠)) ∈ 𝐷𝑙 , for 𝑙 = 1, . . . , 𝑚.
Since
According to (88), we obtain
(126)
Finally, substituting (119), (120), (124), and (126) into (116)
yields
𝑛
𝜕2 𝑊
(𝑠, 𝑦 (𝑠)) 𝐺𝑠𝑖𝑙
𝑘 𝜕𝑥𝑖
𝜕𝑥
𝑖=1
∑
(123)
𝑛
𝜕𝑊
𝜕
= 𝑘 {∑ 𝑖 (𝑠, 𝑦 (𝑠)) 𝐺𝑠𝑖𝑙 + 𝑘𝑠𝑙 } = 0.
𝜕𝑥 𝑖=1 𝜕𝑥
𝑑(
𝜕𝑊
(𝑡, 𝑥𝑡⋆ ))
𝜕𝑥𝑘
𝑛
𝜕𝑊
𝜕𝑏𝑖
(𝑡) 𝑖 (𝑡, 𝑥𝑡⋆ )
𝑘
𝜕𝑥
𝑖=1 𝜕𝑥
= − {∑
This shows that
∑ Δ𝜉
𝑡<𝑠≤𝜏𝑅⋆
𝜕𝑊
(𝑠, 𝑥𝑠⋆ ) = 0.
𝜕𝑥𝑘
𝜕𝑓
𝜕2 𝑊
1 𝑛 𝜕𝑎𝑖𝑗
∑ 𝑘 (𝑡) 𝑖 𝑗 (𝑡, 𝑥𝑡⋆ ) + 𝑘 (𝑡)
2 𝑖,𝑗=1 𝜕𝑥
𝜕𝑥 𝜕𝑥
𝜕𝑥
+
(124)
𝑛
𝜕𝛾𝑖
(𝑡, 𝑒)
𝑘
𝐸 𝑖=1 𝜕𝑥
On the other hand, define
𝐴 (𝑡, 𝑥, 𝑢) =
𝜕𝑊
𝜕𝑊
(𝑡, 𝑥𝑡⋆ + 𝛾 (𝑡, 𝑒)) − 𝑖 (𝑡, 𝑥𝑡⋆ )} ] (𝑑𝑒) .
𝜕𝑥𝑖
𝜕𝑥
+∫ ∑
𝑛
𝜕𝑊
𝜕𝑊
(𝑡, 𝑥) + ∑𝑏𝑖 (𝑡, 𝑥, 𝑢) 𝑖 (𝑡, 𝑥)
𝜕𝑡
𝜕𝑥
𝑖=1
+
×(
𝜕2 𝑊
1 𝑛 𝑖𝑗
∑ 𝑎 (𝑡, 𝑥, 𝑢) 𝑖 𝑗 (𝑡, 𝑥) + 𝑓 (𝑡, 𝑥, 𝑢)
2 𝑖,𝑗=1
𝜕𝑥 𝜕𝑥
+ ∫ {𝑊 (𝑡, 𝑥 + 𝛾 (𝑡, 𝑥, 𝑢, 𝑒)) − 𝑊 (𝑡, 𝑥)
𝑛
𝜕2 𝑊
(𝑡, 𝑥𝑡⋆ ) 𝜎𝑖 (𝑡) 𝑑𝐵𝑡
𝑘 𝜕𝑥𝑖
𝜕𝑥
𝑖=1
+∑
+∫ {
𝐸
𝐸
𝑛
𝜕𝑊
−∑𝛾𝑖 (𝑡, 𝑥, 𝑢, 𝑒) 𝑖 (𝑡, 𝑥)} ] (𝑑𝑒) .
𝜕𝑥
𝑖=1
(125)
𝑘
If we differentiate 𝐴(𝑡, 𝑥, 𝑢) with respect to 𝑥 and
evaluate the result at (𝑥, 𝑢) = (𝑥𝑡⋆ , 𝑢𝑡⋆ ), we deduce easily from
(84), (89), and (90) that
𝑛
𝜕2 𝑊
𝜕2 𝑊
⋆
𝑖
(𝑡,
𝑥
)
+
𝑏
(𝑡, 𝑥𝑡⋆ )
∑
(𝑡)
𝑡
𝑘 𝜕𝑥𝑖
𝜕𝑡𝜕𝑥𝑘
𝜕𝑥
𝑖=1
1 𝑛
𝜕3 𝑊
+ ∑ 𝑎𝑖𝑗 (𝑡) 𝑘 𝑖 𝑗 (𝑡, 𝑥𝑡⋆ )
2 𝑖,𝑗=1
𝜕𝑥 𝜕𝑥 𝜕𝑥
𝜕𝑊
𝜕𝑊
+ ∫ { 𝑘 (𝑡, 𝑥𝑡⋆ + 𝛾 (𝑡, 𝑒)) − 𝑘 (𝑡, 𝑥𝑡⋆ )
𝜕𝑥
𝐸 𝜕𝑥
𝑛
𝜕2 𝑊
−∑𝛾𝑖 (𝑠, 𝑒) 𝑘 𝑖 (𝑡, 𝑥𝑡⋆ )} ] (𝑑𝑒)
𝜕𝑥 𝜕𝑥
𝑖=1
𝜕𝑊
𝜕𝑊
(𝑡, 𝑥𝑡⋆ + 𝛾 (𝑡, 𝑒)) − 𝑖 (𝑡, 𝑥𝑡⋆ ))] (𝑑𝑒)}𝑑𝑡
𝑖
𝜕𝑥
𝜕𝑥
𝜕𝑊
𝜕𝑊
⋆
⋆
̃ (𝑑𝑡, 𝑑𝑒).
(𝑡, 𝑥𝑡−
+ 𝛾− (𝑡, 𝑒))− 𝑘 (𝑡, 𝑥𝑡−
)} 𝑁
𝜕𝑥𝑘
𝜕𝑥
(127)
The continuity of 𝜕𝑊/𝜕𝑥𝑘 leads to
lim
𝜕𝑊
𝑅 → ∞ 𝜕𝑥𝑘
(𝜏𝑅⋆ , 𝑥𝜏⋆⋆ ) =
𝑅
=
𝜕𝑊
(𝑇, 𝑥𝑇⋆ )
𝜕𝑥𝑘
𝜕𝑔 ⋆
(𝑥 ) ,
𝜕𝑥𝑘 𝑇
for each 𝑘 = 1, . . . , 𝑛.
(128)
Clearly,
1 𝑛 𝜕𝑎𝑖𝑗
𝜕2 𝑊
∑ 𝑘 (𝑡) 𝑖 𝑗 (𝑡, 𝑥𝑡⋆ )
2 𝑖,𝑗=1 𝜕𝑥
𝜕𝑥 𝜕𝑥
=
𝑑
𝜕2 𝑊
1 𝑛 𝜕
∑ 𝑘 ( ∑ 𝜎𝑖ℎ (𝑡) 𝜎𝑗ℎ (𝑡)) 𝑖 𝑗 (𝑡, 𝑥𝑡⋆ ) (129)
2 𝑖,𝑗=1 𝜕𝑥 ℎ=1
𝜕𝑥 𝜕𝑥
𝑛
𝑑
𝑛
= ∑ ∑ (∑𝜎𝑖ℎ (𝑡)
𝑗=1ℎ=1
𝑖=1
𝜕2 𝑊
𝜕𝜎𝑖ℎ
(𝑡, 𝑥t⋆ )) 𝑘 (𝑡) .
𝑖
𝑗
𝜕𝑥 𝜕𝑥
𝜕𝑥
International Journal of Stochastic Analysis
15
(𝑝⋆ , 𝑞⋆ , 𝑟⋆ (⋅)) of the following adjoint equation, for all 𝑡 ∈
[0, 𝑇)
Now, from (17) we have
𝜕𝐻
(𝑡, 𝑥, 𝑢, 𝑝, 𝑞, 𝑟 (⋅))
𝜕𝑥𝑘
𝑑𝑝𝑡⋆ = − (𝜇𝑝𝑡⋆ + 𝜎𝑞𝑡⋆ + 𝜃 ∫ 𝑒𝑟𝑡⋆ (𝑒) ] (𝑑𝑒)
R+
𝑛
𝜕𝑏𝑖
= ∑ 𝑘 (𝑡, 𝑥, 𝑢) 𝑝𝑖
𝑖=1 𝜕𝑥
⋆𝛾−1
+𝛾𝑋𝑡
𝑑 𝑛
𝜕𝑓
𝜕𝜎𝑖ℎ
+ ∑ ∑ 𝑘 (𝑡, 𝑥, 𝑢) 𝑞𝑖ℎ + 𝑘 (𝑡, 𝑥, 𝑢)
𝜕𝑥
ℎ=1 𝑖=1 𝜕𝑥
(130)
R+
𝑛
𝜕𝛾𝑖
(𝑡, 𝑥, 𝑢, 𝑒) 𝑟𝑖 (𝑒) ] (𝑑𝑒) .
𝑘
𝐸 𝑖=1 𝜕𝑥
The 𝑘th coordinate 𝑝𝑡𝑘 of the adjoint process 𝑝𝑡 satisfies
𝜕𝐻
(𝑡, 𝑥𝑡⋆ , 𝑢𝑡⋆ , 𝑝𝑡 , 𝑞𝑡 , 𝑟𝑡 (⋅)) 𝑑𝑡
𝜕𝑥𝑘
𝑘
̃ (𝑑𝑡, 𝑑𝑒) ,
+ 𝑞𝑡𝑘 𝑑𝐵𝑡 + ∫ 𝑟𝑡−
(𝑒) 𝑁
𝐸
𝑝𝑇𝑘
+ exp (−𝜘𝑡) ≤ 0,
∀𝑡,
(135)
1{−𝑝𝑡⋆ +exp(−𝜘𝑡)<0} 𝑑𝜉𝑡⋆𝑐 = 0,
(136)
⋆
+ Δ 𝑁𝑝𝑡⋆ ) + exp (−𝜘𝑡) ≤ 0,
− (𝑝𝑡−
(137)
1{−(𝑝𝑡−⋆ +Δ 𝑁 𝑝𝑡⋆ )+exp(−𝜘𝑡)<0} Δ𝜉𝑡⋆ = 0.
(138)
Since 𝑔 = 0, we assume the transversality condition
for 𝑡 ∈ [0, 𝑇] ,
E [𝑝𝑇⋆ (𝑋𝑇⋆ − 𝑋𝑇 )] ≤ 0.
(139)
⋆
+ Δ 𝑁𝑝𝑡⋆ = 𝑝𝑡⋆ , and
We remark that Δ 𝜉 𝑝𝑡⋆ = 0; then 𝑝𝑡−
the condition (138) reduces to
𝜕𝑔
= 𝑘 (𝑥𝑇⋆ ) ,
𝜕𝑥
(131)
with 𝑞𝑡𝑘 𝑑𝐵𝑡 = ∑𝑑ℎ=1 𝑞𝑡𝑘ℎ 𝑑𝐵𝑡ℎ . Hence, the uniqueness of the
solution of (131) and relation (128) allows us to get
𝑝𝑡𝑘 =
(134)
⋆
̃ (𝑑𝑡, 𝑑𝑒) ,
+ 𝑞𝑡⋆ 𝑑𝐵𝑡 + ∫ 𝑟𝑡−
(𝑒) 𝑁
−𝑝𝑡⋆
+∫ ∑
𝑑𝑝𝑡𝑘 = −
exp (−𝜘𝑠) ) 𝑑𝑡
𝜕𝑊
(𝑡, 𝑥𝑡⋆ ) ,
𝜕𝑥𝑘
1{−𝑝𝑡⋆ +exp(−𝜘𝑡)<0} Δ𝜉𝑡⋆ = 0.
We use the relation between the value function and the
solution (𝑝⋆ , 𝑞⋆ , 𝑟⋆ (𝑒)) of the adjoint equation along the
optimal state. We prove that the solution of the adjoint
equation is represented as
⋆𝜌−1
𝑝𝑡⋆ = (𝐴𝜌𝑋𝑡
𝑛
𝜕2 𝑊
(𝑡, 𝑥𝑡⋆ ) 𝜎𝑖ℎ (𝑡) ,
𝑘 𝜕𝑥𝑖
𝜕𝑥
𝑖=1
𝑞𝑡𝑘ℎ = ∑
(132)
(140)
⋆𝛾−1
+ 𝐾𝛾𝑋𝑡
⋆𝜌−1
𝑞𝑡⋆ = 𝜎 (𝐴𝜌 (𝜌 − 1) 𝑋𝑡
) exp (−𝜘𝑡) ,
⋆𝛾−1
+ 𝐾𝛾 (𝛾 − 1) 𝑋𝑡
) exp (−𝜘𝑡) ,
⋆𝜌−1
𝑟𝑡⋆ (𝑒) = (𝐴𝜌 ((1 + 𝜃𝑒)𝜌−1 − 1) 𝑋𝑡
𝜕𝑊
𝜕𝑊
𝑘
⋆
⋆
𝑟𝑡−
+ 𝛾 (𝑡, 𝑒)) − 𝑘 (𝑡, 𝑥𝑡−
),
(⋅) = 𝑘 (𝑡, 𝑥𝑡−
𝜕𝑥
𝜕𝑥
⋆𝛾−1
+𝐾𝛾 ((1 + 𝜃𝑒)𝛾−1 − 1) 𝑋𝑡
) exp (−𝜘𝑡)
(141)
where 𝑞𝑡𝑘ℎ
is the generic element of the matrix 𝑞𝑡 and 𝑥𝑡⋆
is the
optimal solution of the controlled SDE (8).
Example 14. We return to the same example in the previous
section.
Now, we illustrate a verification result for the maximum
principle. We suppose that 𝑇 is a fixed time. In this case the
Hamiltonian gets the form
𝛾
𝐻 (𝑡, 𝑋𝑡 , 𝑝𝑡 , 𝑞𝑡 , 𝑟𝑡 (⋅)) = 𝜇𝑋𝑡 𝑝𝑡 + 𝜎𝑋𝑡 𝑞𝑡 + 𝑋𝑡 (−𝜘𝑡)
+ 𝜃𝑋𝑡− ∫ 𝑒𝑟𝑡 (𝑒) ] (𝑑𝑒) .
for all 𝑡 ∈ [0, 𝑇).
⋆𝜌−1
+
To see this, we differentiate the process (𝐴𝜌𝑋𝑡
⋆𝛾−1
𝐾𝛾𝑋𝑡 ) exp(−𝜘𝑡) using Itô’s rule for semimartingales and
by using the same procedure as in the proof of Theorem 13.
Then, the conclusion follows readily from the verification of
(135), (136), and (139). First, an explicit formula for 𝑋𝑡 is given
in [4] by
𝑋𝑡 = 𝑒𝜇𝑡 𝑀𝑡 {𝑥 − ( ∫
(133)
R+
Let 𝜉⋆ be a candidate for an optimal control, and let 𝑋⋆ be
the corresponding state process with corresponding solution
[0,𝑡)
𝑀𝑠−1 exp (−𝜇𝑠) 𝑑𝜉𝑠
+ ∑ 𝑀𝑠−1 𝛽𝑠 exp (−𝜇𝑠) Δ𝜉𝑠 )} ,
0<𝑠≤𝑡
for 𝑡 ∈ [0, 𝑇] ,
(142)
16
International Journal of Stochastic Analysis
−1
where 𝛽𝑡 = (∫R 𝜃𝑒𝑁({𝑡}, 𝑑𝑒))(1 + ∫R 𝜃𝑒𝑁({𝑡}, 𝑑𝑒)) , and
+
+
𝑀𝑡 is a geometric Lévy process defined by
1
𝑀𝑡 = exp {(− 𝜎2 + ∫ {ln (1 + 𝜃𝑒) − 𝜃𝑒} ] (𝑑𝑒)) 𝑡
2
R+
𝑡
(143)
̃ (𝑑𝑡, 𝑑𝑒) } .
+ 𝜎𝐵𝑡 + ∫ ∫ ln (1 + 𝜃𝑒) 𝑁
0
References
R+
⋆
≤
From the representation (142) and by the fact that 𝑋𝑇∧𝑡
𝑥𝑀𝑇∧𝑡 exp(𝜇(𝑇 ∧ 𝑡)), we get
1−
𝑋𝑇∧𝑡 1
≤ (∫
𝑀𝑠−1 exp (−𝜇𝑠) 𝑑𝜉𝑠
⋆
𝑋𝑇∧𝑡
𝑥
[0,𝑇∧𝑡)
(144)
+ ∑ 𝑀𝑠−1 𝛽𝑠 exp (−𝜇𝑠) Δ𝜉𝑠 ) < ∞,
0<𝑠≤𝑡
hence
⋆
⋆
E [𝑝𝑇∧𝑡
(𝑋𝑇∧𝑡
− 𝑋𝑇∧𝑡 )]
⋆𝜌
2 1/2
⋆𝛾
≤ E[((𝐴𝜌𝑋𝑇∧𝑡 + 𝐾𝛾𝑋𝑇∧𝑡 ) exp (−𝜘 (𝑇 ∧ 𝑡))) ]
1
× E[( ∫
𝑀−1 exp (−𝜇𝑠) 𝑑𝜉𝑠
𝑥 [0,𝑇∧𝑡) 𝑠
[
2
+
∑
0<𝑠≤𝑇∧𝑡
𝑀𝑠−1 𝛽𝑠
exp (−𝜇𝑠) Δ𝜉𝑠 ) ]
]
improvement of the paper. This work has been partially
supported by the Direction Générale de la Recherche Scientifique et du Développement Technologique (Algeria),
under Contract no. 051/PNR/UMKB/2011, and by the French
Algerian Cooperation Program, Tassili 13 MDU 887.
1/2
.
(145)
By the dominated convergence theorem, we obtain (139)
by sending 𝑡 to infinity in (145).
A simple computation shows that the conditions (135)–
(138) are consequences of (107)–(109). This shows in particular that the pair (𝑋𝑡⋆ , 𝜉𝑡⋆ ) satisfies the optimality sufficient
conditions and then it is optimal. This completes the proof
of the following result.
Theorem 15. One supposes that 𝜎2 /2 + 𝜃 ∫R 𝑒](𝑑𝑒) ≤ 𝜇 < 𝜘,
+
and 𝑒 ≥ 0 𝑑] − 𝑎.𝑒. If the strategy 𝜉⋆ is chosen such that the
corresponding solution of the adjoint process is given by (141),
then this choice is optimal.
Remark 16. In this example, it is shown in particular that the
relationship between the stochastic maximum principle and
dynamic programming could be very useful to solve explicitly
constrained backward stochastic differential equations with
transversality condition.
Conflict of Interests
The authors declare that there is no conflict of interests
regarding the publication of this paper.
Acknowledgments
The authors would like to thank the referees and the associate
editor for valuable suggestions that led to a substancial
[1] S. Bahlali and A. Chala, “The stochastic maximum principle
in optimal control of singular diffusions with non linear
coefficients,” Random Operators and Stochastic Equations, vol.
13, no. 1, pp. 1–10, 2005.
[2] K. Bahlali, F. Chighoub, and B. Mezerdi, “On the relationship
between the stochastic maximum principle and dynamic programming in singular stochastic control,” Stochastics, vol. 84,
no. 2-3, pp. 233–249, 2012.
[3] N. C. Framstad, B. Øksendal, and A. Sulem, “Sufficient stochastic maximum principle for the optimal control of jump diffusions and applications to finance,” Journal of Optimization
Theory and Applications, vol. 121, pp. 77–98, 2004, Erratum in
Journal of Optimization Theory and Applications, vol. 124, no. 2,
pp. 511–512, 2005.
[4] B. Øksendal and A. Sulem, “Singular stochastic control and
optimal stopping with partial information of Itô-Lévy processes,” SIAM Journal on Control and Optimization, vol. 50, no.
4, pp. 2254–2287, 2012.
[5] L. H. R. Alvarez and T. A. Rakkolainen, “On singular stochastic
control and optimal stopping of spectrally negative jump
diffusions,” Stochastics, vol. 81, no. 1, pp. 55–78, 2009.
[6] W. H. Fleming and H. M. Soner, Controlled Markov Processes
and Viscosity Solutions, vol. 25, Springer, New York, NY, USA,
1993.
[7] N. C. Framstad, B. Øksendal, and A. Sulem, “Optimal consumption and portfolio in a jump diffusion market with proportional
transaction costs,” Journal of Mathematical Economics, vol. 35,
no. 2, pp. 233–257, 2001, Arbitrage and control problems in
finance.
[8] B. Øksendal and A. Sulem, Applied Stochastic Control of Jump
Diffusions, Springer, Berlin, Germany, 2005.
[9] U. G. Haussmann and W. Suo, “Singular optimal stochastic
controls. II. Dynamic programming,” SIAM Journal on Control
and Optimization, vol. 33, no. 3, pp. 937–959, 1995.
[10] J. A. Bather and H. Chernoff, “Sequential decision in the control
of a spaceship (finite fuel),” Journal of Applied Probability, vol.
49, pp. 584–604, 1967.
[11] V. E. Beneš, L. A. Shepp, and H. S. Witsenhausen, “Some
solvable stochastic control problems,” Stochastics, vol. 4, no. 1,
pp. 39–83, 1980/81.
[12] P.-L. Lions and A.-S. Sznitman, “Stochastic differential equations with reflecting boundary conditions,” Communications on
Pure and Applied Mathematics, vol. 37, no. 4, pp. 511–537, 1984.
[13] F. M. Baldursson and I. Karatzas, “Irreversible investment and
industry equilibrium,” Finance and Stochastics, vol. 1, pp. 69–89,
1997.
[14] E. M. Lungu and B. Øksendal, “Optimal harvesting from a
population in a stochastic crowded environment,” Mathematical
Biosciences, vol. 145, no. 1, pp. 47–75, 1997.
[15] M. H. A. Davis and A. R. Norman, “Portfolio selection with
transaction costs,” Mathematics of Operations Research, vol. 15,
no. 4, pp. 676–713, 1990.
International Journal of Stochastic Analysis
[16] M. Chaleyat-Maurel, N. El Karoui, and B. Marchal, “Réflexion
discontinue et systèmes stochastiques,” The Annals of Probability, vol. 8, no. 6, pp. 1049–1067, 1980.
[17] P. L. Chow, J.-L. Menaldi, and M. Robin, “Additive control of
stochastic linear systems with finite horizon,” SIAM Journal on
Control and Optimization, vol. 23, no. 6, pp. 858–899, 1985.
[18] A. Cadenillas and U. G. Haussmann, “The stochastic maximum principle for a singular control problem,” Stochastics and
Stochastics Reports, vol. 49, no. 3-4, pp. 211–237, 1994.
[19] S. Bahlali and B. Mezerdi, “A general stochastic maximum
principle for singular control problems,” Electronic Journal of
Probability, vol. 10, pp. 988–1004, 2005.
[20] S. G. Peng, “A general stochastic maximum principle for optimal
control problems,” SIAM Journal on Control and Optimization,
vol. 28, no. 4, pp. 966–979, 1990.
[21] S. Bahlali, B. Djehiche, and B. Mezerdi, “The relaxed stochastic
maximum principle in singular optimal control of diffusions,”
SIAM Journal on Control and Optimization, vol. 46, no. 2, pp.
427–444, 2007.
[22] K. Bahlali, F. Chighoub, B. Djehiche, and B. Mezerdi, “Optimality necessary conditions in singular stochastic control problems
with nonsmooth data,” Journal of Mathematical Analysis and
Applications, vol. 355, no. 2, pp. 479–494, 2009.
[23] H. Pham, “Optimal stopping of controlled jump diffusion processes: a viscosity solution approach,” Journal of Mathematical
Systems, Estimation, and Control, vol. 8, pp. 1–27, 1998.
[24] J.-T. Shi and Z. Wu, “Relationship between MP and DPP for the
stochastic optimal control problem of jump diffusions,” Applied
Mathematics and Optimization, vol. 63, no. 2, pp. 151–189, 2011.
[25] S. J. Tang and X. J. Li, “Necessary conditions for optimal control
of stochastic systems with random jumps,” SIAM Journal on
Control and Optimization, vol. 32, no. 5, pp. 1447–1475, 1994.
[26] X. Y. Zhou, “A unified treatment of maximum principle and
dynamic programming in stochastic controls,” Stochastics and
Stochastics Reports, vol. 36, no. 3-4, pp. 137–161, 1991.
[27] J. Yong and X. Y. Zhou, Stochastic Controls, Hamiltonian Systems
and HJB Equations, vol. 43, Springer, New York, NY, USA, 1999.
[28] A. Eyraud-Loisel, “Backward stochastic differential equations
with enlarged filtration: option hedging of an insider trader in
a financial market with jumps,” Stochastic Processes and Their
Applications, vol. 115, no. 11, pp. 1745–1763, 2005.
[29] G. Barles, R. Buckdahn, and E. Pardoux, “BSDEs and
integral-partial differential equations,” Stochastics and Stochastics Reports, vol. 60, no. 1-2, pp. 57–83, 1997.
[30] N. Ikeda and S. Watanabe, Stochastic Differential Equations and
Diffusion Processes, vol. 24, North-Holland, Amsterdam, The
Netherlands, 2nd edition, 1989.
17
Advances in
Operations Research
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Advances in
Decision Sciences
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Mathematical Problems
in Engineering
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Journal of
Algebra
Hindawi Publishing Corporation
http://www.hindawi.com
Probability and Statistics
Volume 2014
The Scientific
World Journal
Hindawi Publishing Corporation
http://www.hindawi.com
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
International Journal of
Differential Equations
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Volume 2014
Submit your manuscripts at
http://www.hindawi.com
International Journal of
Advances in
Combinatorics
Hindawi Publishing Corporation
http://www.hindawi.com
Mathematical Physics
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Journal of
Complex Analysis
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
International
Journal of
Mathematics and
Mathematical
Sciences
Journal of
Hindawi Publishing Corporation
http://www.hindawi.com
Stochastic Analysis
Abstract and
Applied Analysis
Hindawi Publishing Corporation
http://www.hindawi.com
Hindawi Publishing Corporation
http://www.hindawi.com
International Journal of
Mathematics
Volume 2014
Volume 2014
Discrete Dynamics in
Nature and Society
Volume 2014
Volume 2014
Journal of
Journal of
Discrete Mathematics
Journal of
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Applied Mathematics
Journal of
Function Spaces
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Optimization
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014