Optimal control of SDEs associated with
general Lévy generators
Jiang-Lun Wu
Department of Mathematics, Swansea University, UK
EPSRC Workshop on Topics on Control
Institute of Mathematics, Warwick University, UK
30th Nov. – 2nd Dec., 2011
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
Based on joint works with Jonathan Bennett
[1] Explicit construction of SDEs associated with
polar-decomposed Lévy measures and application to
stochastic optimization, Frontiers of Mathematics in China 2
(2007), 539–558.
[2] An optimal control problem associated with SDEs driven by
Lévy-type processes, Stochastic Analysis and Applications, 26
(2008), 471–494.
[3] Stochastic control of SDEs associated with Lévy generators
and application to financial optimization, Frontiers of
Mathematics in China 5 (2010), 89–102.
and [4] JLW, Harry Zheng, On an optimal portfolio-comsumption
problem associated with Lévy-type generators, in progress.
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
A fairly large class of Markov processes on Rd are governed by
Lévy generator, either via martingale problem (cf e.g. D W
Stroock, “Markov Processes from K. Itô’s Perspectives”,
Princeton Univ Press 2003 or V.N. Kolokoltsov, “Markov
Processes, Semigroups and Generators”, de Gruyter, 2011) or
via Dirichlet form (cf e.g. N Jacob,“Pseudo-Differential
Operators and Markov Processes III” Imperial College Press,
2005)
Lf (t, x) :=
1 i,j
a (t, x)∂i ∂j f (t, x) + bi (t, x)∂i f (t, x)
2 Z
{f (t, x + z) − f (t, x)
+
Rd \{0}
−
z1{|z|<1} · ∇f (t, x)
}ν(t, x, dz)
1 + |z|2
where a(t, x) = (ai,j (t, x))d×d is non-negative definite
symmetric and ν(t, x, dz) is a Lévy kernel, i.e.,
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
∀(t, x) ∈ [0, ∞) × Rd , ν(t, x, ·) is a σ-finite measure on
(Rd \ {0}, B(Rd \ {0}) such that
Z
Rd \{0}
|z|2
ν(t, x, dz) < ∞.
1 + |z|2
For such L, in order to get rid of variable dependence on ν, N El
Karoui and J P Lepeltier (Z. Wahr. verw. Geb. 39 (1977))
construct a bimeasurable bijection
c : [0, ∞) × Rd × U → Rd \ {0}
such that
Z
Z
1A (c(t, x, y ))λ(dy ) =
U
Rd \{0}
1A (z)ν(t, x, dz),
∀(t, x)
for A ∈ B(Rd \ {0}). Where (U, B(U)) is a Lusin space and λ is
a σ-finite measure on it. Actually, we can construct c explicitly
in case ν has a polar decomposition (with the stable-like case
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
as a concrete example). It is well-known (cf e.g. Theorem I.8.1
in N Ikeda and S Watanabe’s book): ∃ a Poisson random
measure
N : B([0, ∞)) × B(U) × Ω → N ∪ {0} ∩ {∞}
on any given probability set-up (Ω, F, P; {Ft }t≥0 ) with
E(N(dt, dy , ·)) = dtλ(dy), and
Ñ(dt, dy , ω) := N(dt, dy , ω) − dtλ(dy )
being the associated compensating {Ft }t≥0 -martingale
measure.
We then can formulate a jump SDE associated with L
Z
dSt = b(t, St )dt + σ(t, St )dWt +
c(t, St− , y )Ñ(dt, dy )
U
where σ(t, x) is a d × m-matrix such that
σ(t, x)σ T (t, x) = a(t, x)
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
and {Wt }t∈[0,∞) is an m-dimensional {Ft }t≥0 -Brownian motion.
We shall consider such equation in the following general
formulation
dSt
= b(t, St )dt + σ(t, St )dWt
Z
+
c1 (t, St− , z)Ñ(dt, dz)
U\U0
Z
+
c2 (t, St− , z)N(dt, dz)
U0
where U0 ∈ B(U) with λ(U0 ) < ∞ is arbitrarily fixed.
Sufficient Maximum Principle
Framstad,Øksendal, Sulem (J Optim Theory Appl 121 (2004))
Øksendal, Sulem (“Applied Stochastic Control of
Jump-Diffusions”, Springer, 2005); Math Finance 19 (2009);
SIAM J Control Optim 2010; Commun Stoch Anal 4 (2010)
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
Start with a controlled jump Markov process
(u)
St = St
,
t ∈ [0, T ]
for any arbitrarily fixed T ∈ (0, ∞), by the following
dSt
= b(t, St , ut )dt + σ(t, St , ut )dWt
Z
+
c1 (t, St− , ut− , z)Ñ(dt, dz)
U\U0
Z
c2 (t, St− , ut− , z)N(dt, dz)
+
(1)
U0
where the control process ut = u(t, ω), taking values in a given
Borel set U ∈ B(Rd ), is assumed to be {Ft }-predictable and
cádlág.
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
The performance criterion is
Z
J(u) := E
!
T
f (t, St , ut )dt + g(ST ) ,
u∈A
0
for A the totality of all admissible controls, and for
f : [0, T ] × Rd × U → R
being continuous, and for g : Rd → R being concave. The
objective is to achieve the following
J(û) = sup J(u)
u∈A
referring û being the optimal control of the system.
(û)
Moreover, if Ŝt = St is the solution to the jump type SDE (1)
corresponding to û, then the pair (Ŝ, û) is called the optimal
pair.
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
Now the Hamiltonian is defined
H : [0, T ] × Rd × U × Rd × Rd⊗m × R → R
via
H(t, r , u, p, q, n(1) , n(2) )
1
= f (t, r , u) + µ(t, r , u)p + σ T (t, r , u)q
2
Z
(1)
+
n (t, z)c1 (t, r , u, z)λ(dz)
U\U0
Z
+
[n(2) (t, z)c2 (t, r , u, z) + c2 (t, r , u, z)p]λ(dz)
U0
where R is the collection of all Rd⊗d -valued processes
n : [0, ∞) × Ω → Rd⊗d such that the two integrals in the above
formulation converge absolutely.
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
It is known that the adjoint equation corresponding to an
admissible pair (S, u) is the BSDE
dp(t) = −∇r H(t, St , ut , p(t), q(t), n(1) (t, ·), n(2) (t, ·))dt
Z
+q(t)dWt +
n(1) (t−, z)Ñ(dt, dz)
U\U0
Z
+
n(2) (t−, z)N(dt, dz)
U0
with terminal condition
p(T ) = 5g(ST ) .
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
Theorem ([3])
Given an admissible pair (Ŝ, û). Suppose ∃ an {Ft }-adapted
solution (p̂(t), q̂(t), n̂(t, z)) to the BSDE s.t. for u ∈ A
T
Z
E[
(u)
(Ŝt − St )T {q̂(t)q̂(t)T
0
Z
+
[tr (n̂(t, z)n̂(t, z)T )λ(dz)]}
U0
×(Ŝt − S (u) (t))dt] < ∞ ,
Z
E[
0
T
T
Z
[tr (c(t, St− , ut , z)c T (t, St− , ut , z))λ(dz)]
p̂ (t){
U0
+σ(t, St , ut )σ T (t, St , ut )}p̂(t)dt] < ∞,
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
Theorem (cont’d)
and ∀t ∈ [0, T ]
H(t, Ŝt , ût , p̂(t), q̂(t), n̂(t, .)) = sup H(t, Ŝt , ut , p̂(t), q̂(t), n̂(t, .)).
u∈A
(2)
If Ĥ(r ) := maxu∈A H(t, r , u, p̂(t), q̂(t), n̂(t, ·)) exists and is a
concave function of r , then (Ŝ, û) is an optimal pair.
Remark For (2), it suffices that the function
(r , u) → H(t, r , u, p̂(t), q̂(t), n̂(t, ·))
is concave, ∀t ∈ [0, T ].
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
Optimal control problem
Benth, Karlsen, Reikvam (Finance Stoch 5 (2001); Stochastics
Stochastics Rep 74(2002))
Ishikawa (Appl Math Optim 50 (2004))
Jakobsen, Karlsen (JDE 212 (2005); NoDEA 13 (2006))
Start with a Lévy type process
Z
Zt
t
Z tZ
= µt +
c1 (z)Ñ(ds, dz)
θ(s)dWs +
0
0
U\U0
Z tZ
c2 (z)N(ds, dz)
+
0
U0
where µ is a constant, θ : [0, T ] → R and c1 , c2 : U → R are
measurable. Here assume that
Z
(ec2 (z) − 1)λ(dz) < ∞.
U0
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
We are concerned with the following 1-dimensional linear SDE
dSt
1
= b(t)St dt + σ(t)2 St dt + σ(t)St dWt
2
Z
+St (ec1 (z) − 1 − c1 (z)1{U\U0 } (z))λ(dz)dt
U
Z
+St− (ec1 (z) − 1)Ñ(dt, dz).
U
Based on the driving processes Zt and St , we construct two
processes Xt and Yt with X0 = x, Y0 = y, via
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
Z
t
Xt = x − Gt +
σ(s)πs Xs dWs + Lt
0
Z t
Z
1
2
+ (r + ([b(s) + σ(s) +
(ec1 (z)
2
0
U\U0
Z
t
+
−1 − c1 (z))λ(dz)] − r )πs )Xs ds
Z
πs− Xs−
(ec1 (z) − 1)Ñ(ds, dz)
0
Z
U\U0
t
Z
πs− Xs−
+
0
(ec2 (z) − 1)N(ds, dz)
U0
and
Yt = ye−βt + β
t
Z
e−β(t−s) dGs
0
respectively, where
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
Z
Gt :=
t
gs ds
0
with (gt )t≥0 being a nondecreasing {Ft }-adapted cádlág
process of finite variation such that 0 ≤ supt≥0 gt < ∞, Lt is a
nondecreasing, nonnegative, and {Ft }-adapted cádlág
process, and πt ∈ [0, 1] is {Ft }-adapted cádlág. The triple
(Gt , Lt , πt ) is referred as the parameter process.
Remark The background for Xt being the self-financing
investment policy according to the portfolio πt :
dBt
dSt
dXt
= (1 − πt )
+ πt
Xt−
Bt
St−
with Bt standing for the riskless bond dBt = rBt dt.
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
By Itô formula, the generator A to (Xt , Yt ) is
Av (x, y) = −αv − βyvy + σ(t)πxvxx
1
+{(r + π([b(t) + σ(t)2
2
Z
(ec1 (z) − 1 − c1 (z))λ(dz)] − r ))xvx
+
U\U0
Z
(v (x + πx(ec1 (z) − 1), y)
+
U\U0
−v (x, y ) − πxvx (ec1 (z) − 1))λ(dz)
Z
+
(v (x + πx(ec2 (z) − 1), y ) − v (x, y ))λ(dz)}
U0
+u(g) − g(vx − βvy )
for any v ∈ C 2,2 (R × R) and for π ∈ [0, 1], g ∈ [0, M1 ].
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
Now we define the value function
v (x, y) :=
(X (π.,g.,L.) ,Y (π.,g.,L.) )
E
sup
(π.,g.,L.)∈A
Z
[
∞
e−αs u(gs )ds]
0
where the supremum is taken over all admissible controls and u
is a utility function, i.e., u is strictly increasing, differential, and
concave on [0, ∞) such that
u(0) = u 0 (∞) = 0,
Jiang-Lun Wu
u(∞) = u 0 (0) = ∞.
HJB equation associated with Lévy-type generators
We also denote that
1
k(γ, ρ) := max γ(r + π([b(t) + σ(t)2
π
2
Z
+
(ec1 (z) − 1 − c1 (z))λ(dz)] − r ))
U\U0
Z
+σ(t)πρ +
[(1 + π(ec1 (z) − 1))γ
U\U0
−1 − γπ(ec1 (z) − 1))
Z
(1 + π(e
+
c2 (z)
− 1)) − 1]λ(dz) .
γ
U0
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
Theorem ([2])
[i](Existence) v is well-defined, i.e., there exists an optimal
control (π ∗ , g ∗ , L∗ ) ∈ A such that
Z ∞
∗ ∗ ∗
∗ ∗ ∗
(X (π ,g ,L ) ,Y (π ,g ,L ) )
v (x, y) = E
[
e−αs u(gs∗ )ds].
0
Furthermore, v is a constrained viscosity solution to the
following Hamilton-Jacobi-Bellman integro-variational inequality
(
)
max vx 1{x≤0} , sup {Av }, (βvy − vx )1{x≥0}
=0
(π,g)∈A
in Dβ := {(x, y) : y > 0, y + βx > 0}, and
v =0
outside of
Jiang-Lun Wu
Dβ .
HJB equation associated with Lévy-type generators
Theorem (cont’d)
[ii] (Uniqueness) For γ > 0 and each ρ ≥ 0 choose α > 0 s.t.
k (γ, ρ) < α. Then the HJB integro-variational inequality admits
at most one constrained viscosity solution.
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
Theorem (JLW and H. Zheng [4])
Assume the following dynamic programming principle hold for
the value function v : ∀t ≥ 0 and for any stopping time τ
Z t∧τ
e−αs u(gs∗ )ds
v (x, y) = sup(π,g,L)∈A E
0
i
(π,g,L)
(π,g,L)
−α(t∧τ )
+e
v (Xt∧τ , Yt∧τ ) .
Then, v is the unique, constrained (subject to a gradient
constraint) viscosity solution of the following integro-differential
HJB equation
(
)
max βvy − vx , sup {Av }
=0
(π,g)∈A
in D := {(x, y) : x > 0, y > 0}.
Further discussion on the properties of v is under way.
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
The case of polar-decomposed Lévy measures
Recall the Lévy generator
Lf (t, x) :=
1 i,j
a (t, x)∂i ∂j f (t, x) + bi (t, x)∂i f (t, x)
2 Z
{f (t, x + z) − f (t, x)
+
Rd \{0}
−
z1{|z|<1} · ∇f (t, x)
}ν(t, x, dz)
1 + |z|2
and the associated SDE
Z
dSt = b(t, St )dt + σ(t, St )dWt +
c(t, St− , y )Ñ(dt, dy )
U
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
Here we consider a special case: ν admits a
polar-decomposition.
(U, B(U), λ) = (Sd−1 × (0, ∞), λ)
where λ is σ-finite. Now let
m: a finite Borel measure on Sd−1
z : Rd × Sd−1 × (0, ∞) → Rd \ {0} bimeasurable bijection
g : Rd × Sd−1 × B((0, ∞)) → (0, ∞) is a positive kernel
Our ν is then taken the form
Z
Z ∞
ν(x, dz) =
1dz (z(x, θ, r )g(x, θ, dr )
Sd−1
0
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
M Tsuchiya, Stoch Stoch Reports 38 (1992)
V. Kolkoltsov, Proc London Maths Soc 80 (2000)
V. Kolkoltsov, Nonlinear Markov Processes and
Kinetic Equations. (CUP, 2010)
Example (Bass, PTRF (1988); Kolkoltsov) Take
z(x, θ, r ) = r θ
and g(x, θ, dr ) =
dr
r 1+α(x)
then
ν(x, dz) =
Jiang-Lun Wu
dr
r 1+α(x)
m(dθ)
HJB equation associated with Lévy-type generators
Theorem ([1])
(i) For d ≥ 2, i.e., for the case that the given σ-finite measure
space
U = (S d−1 × (0, ∞))
the coefficient of the jump term in the SDE associated to
ν(x, dz) is given by c(t, x, (r , θ)) = r θ ;
(ii) For the case when d = 1, namely, for the case that the given
σ-finite measure space
(U, B(U), λ) = ((0, ∞), B((0, ∞)), λ)
the coefficient of the jump term in the SDE associated to
ν(x, dz) defined by
ν(x, dz) =
dr
r 1+α(x)
α(x) ∈ (0, 2),
x ∈R
is given by |c(t, x, (r , θ))| = r .
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
As an application, we consider a consumption-portfolio
optimzation problem. The wealth process is modelled via
dS(t) = {ρt S(t) + (b(t) − ρt )u(t) − w(t)}dt
+σ(t)w(t)dW (t)
Z
+w(t−)
r θÑ(dt, drdθ)
0<|r |<1
Z
+w(t−)
r θN(dt, drdθ).
|r |≥1
Our objective is to solve the following consumption-portfolio
optimization problem:
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
Z
sup E[
(w,u)∈A
T
t
Z
exp(−
0
δ(s)ds)[
0
w(t)γ
]dt]
γ
(3)
subject to the terminal wealth constraint
S(T ) ≥ 0
a.s.
where A is the set of predictable consumption-portfolio pairs
(w, u) with the control u being tame and the consumption w
being nonnegative, such that the above SDE has a strong
solution over [0, T ].
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
Theorem ([1])
An optimal control (u ∗ , w ∗ ) is given by
Z t
1
δ(s)
∗
u (t, x) = exp
ds f (t) γ−1 x
0 γ−1
and
w ∗ (t, x) = π̂x
with f (t) and π̂ being explicitly constructed.
Jiang-Lun Wu
HJB equation associated with Lévy-type generators
Thank You!
Jiang-Lun Wu
HJB equation associated with Lévy-type generators