Hindawi Publishing Corporation
International Journal of Stochastic Analysis
Volume 2010, Article ID 329185, 25 pages
doi:10.1155/2010/329185
Research Article
Optimal Control with Partial Information for
Stochastic Volterra Equations
Bernt Øksendal1, 2 and Tusheng Zhang3
1
CMA and Department of Mathematics, University of Oslo, P.O. Box 1053 Blindern, 0316 Oslo, Norway
Norwegian School of Economics and Business Administration (NHH), Helleveien 30,
5045 Bergen, Norway
3
School of Mathematics, University of Manchester, Oxford Road, Manchester M13 9PL, UK
2
Correspondence should be addressed to Bernt Øksendal, oksendal@math.uio.no
Received 26 October 2009; Revised 26 February 2010; Accepted 9 March 2010
Academic Editor: Agnès Sulem
Copyright q 2010 B. Øksendal and T. Zhang. This is an open access article distributed under
the Creative Commons Attribution License, which permits unrestricted use, distribution, and
reproduction in any medium, provided the original work is properly cited.
In the first part of the paper we obtain existence and characterizations of an optimal control for a
linear quadratic control problem of linear stochastic Volterra equations. In the second part, using
the Malliavin calculus approach, we deduce a general maximum principle for optimal control of
general stochastic Volterra equations. The result is applied to solve some stochastic control problem
for some stochastic delay equations.
1. Introduction
Let Ω, F, Ft , P be a filtered probability space and Bt, t ≥ 0 a Ft -real valued Brownian
motion. Let R0 R \ {0} and νdz a σ-finite measure on R0 , BR0 . Let Ndt, dz denote
a stationary Poisson random measure on R × R0 with intensity measure dtνdz. Denote by
Ndt,
dz Ndt, dz − dtνdz the compensated Poisson measure. Suppose that we have a
cash flow where the amount Xt at time t is modelled by a stochastic delay equation of the
form:
dXt A1 tXt A2 tXt − h C1 tdBt t
A0 t, sXsds dt
t−h
C2 t, zNdt,
dz;
t ≥ 0,
R0
Xt ηt;
t ∈ −h, 0.
1.1
2
International Journal of Stochastic Analysis
Here h > 0 is a fixed delay and A1 t, A2 t, A0 t, s, C1 t, C2 t, z, and η are given bounded
deterministic functions.
Suppose that we consume at the rate ut at time t from this wealth Xt, and that
this consumption rate influences the growth rate of Xt both through its value ut at time
t and through its former value ut − h, because of some delay mechanisms in the system
determining the dynamics of Xt.
With such a consumption rate ut the dynamics of the corresponding cash flow X u t
is given by
dX t A1 tX t A2 tX t − h u
u
u
t
A0 t, sX u sds
t−h
B1 tut B2 tut − h dt C1 tdBt
C2 t, zNdt,
dz;
1.2
t ∈ −h, 0,
R0
X u t ηt;
t ≤ 0,
where B1 t and B2 t are deterministic bounded functions.
Suppose that the consumer wants to maximize the combined utility of the consumption up to the terminal time T and the terminal wealth. Then the problem is to find u· such
that
T
Ju : E
U1 t, utdt U2 X T u
1.3
0
is maximal. Here Ut, · and U2 · are given utility functions, possibly stochastic. See
Section 4.
This is an example of a stochastic control problem with delay. Such problems have been
studied by many authors. See, for example, 1–5 and the references therein. The methods
used in these papers, however, do not apply to the cases studied here. Moreover, these papers
do not consider partial information control see below.
It was shown in 6 that the system 1.2 is equivalent to the following controlled
stochastic Volterra equation:
X u t t
Kt, susds t
0
0
Φt, 0η0 Φt, sCsdBs 0 h
−h
0
0
−h
t 0
Φt, s hA2 s hηsds
Φt, τA0 τ, sdτ ηsds,
Φt, sC2 s, zNds,
dz
R0
1.4
International Journal of Stochastic Analysis
3
where
Kt, s Φt, sB1 s Φt, s hB2 s h,
1.5
and Φ is the transition function satisfying
∂Φ
A1 tΦt, s A2 tΦt − h, s ∂t
Φs, s I;
t
A0 t, τΦτ, sdτ,
1.6
t−h
Φt, s 0 for t < s.
So the control of the system 1.2 reduces to the control of the system 1.4. For more
information about stochastic control of delay equations we refer to 6 and the references
therein.
Stochastic Volterra equations are interesting on their own right, also for applications,
for example, to economics or population dynamics. See, for example, Example 1.1 in 7 and
the references therein.
In the first part of this paper, we study a linear quadratic control problem for the
following controlled stochastic Volterra equation:
X u t ξt t
t 0
0
K4 t, s, zX u sNds,
dz t
t
D2 t, sX u sds
0
R0
t 0
K1 t, sX u s D1 t, sus K2 t, sdBs
D3 t, s, zusNds,
dz 0
R0
1.7
t K5 t, s, zNds,
dz
R0
K3 t, susds,
0
where ut is our control process and ξt is a given predictable process with Eξ2 t < ∞ for
all t ≥ 0, while Ki , Di are bounded deterministic functions. In reality one often does not have
the complete information when performing a control to a system. This means that the control
processes is required to be predictable with respect to a subfiltration {Gt } with Gt ⊂ Ft . So the
space of controls will be
U
us; us is Gt -predictable and such that E
T
2
|us| ds < ∞ .
1.8
0
U is a Hilbert space equipped with the inner product
u1 , u2 E
T
0
u1 su2 sds .
1.9
4
International Journal of Stochastic Analysis
|| · || will denote the norm in U. Let AG be a closed, convex subset of U, which will be the
space of admissible controls. Consider the linear quadratic cost functional
Ju E
T
Q1 su sds 2
0
T
T
2
Q2 sX s ds u
T
0
Q3 susds
0
1.10
2
Q4 sX sds a1 X T a2 X T u
u
u
0
and the value function
J inf Ju.
1.11
u∈AG
In Section 2, we prove the existence of an optimal control and provide some characterizations
for the control.
In the second part of the paper from Section 3, we consider the following general
controlled stochastic Volterra equation:
X t ξt u
t
t 0
bt, s, X s, us, ωds u
0
t
σt, s, X u s, us, ωdBs
0
1.12
θt, s, X s, us, z, ωNds,
dz,
u
R0
where ξt is a given predictable process with Eξ2 t < ∞ for all t ≥ 0. The performance
functional is of the following form:
Ju E
T
ft, X t, ut, ωdt gX T , ω ,
u
u
1.13
0
where b : 0, T × 0, T × R × R × Ω → R, σ : 0, T × 0, T × R × R × Ω → R, θ : 0, T × 0, T ×
R × R × R0 × Ω → R and f : 0, T × R × R × Ω → R are Ft -predictable and g : R × Ω → R is
FT measurable and such that
T
E
ft, X u t, ut
dt gX u T < ∞,
1.14
0
for any u ∈ AG , the space of admissible controls. The problem is to find u
∈ AG such that
u.
Φ : sup Ju J
u∈AG
1.15
Using the Malliavin calculus, inspired by the method in 8, we will deduce a general
maximum principle for the above control problem.
International Journal of Stochastic Analysis
5
Remark 1.1. Note that we are off the Markovian setting because the solution of the Volterra
equation is not Markovian. Therefore the classical method of dynamic programming and the
Hamilton-Jacobi-Bellman equation cannot be used here.
Remark 1.2. We emphasize that partial information is different from partial observation,
where the control is based on noisy observations of the current state. For example, our
discussion includes the case Gt Ft−δ δ > 0 constant, which corresponds to delayed
information flow. This case is not covered by partial observation models. For a comprehensive
presentation of the linear quadratic control problem in the classical case with partial
observation, see 9, with partial information see 10.
2. Linear Quadratic Control
Consider the controlled stochastic Volterra equation 1.7 and the control problem 1.10,
1.11. We have the following Theorem.
Theorem 2.1. Suppose that R0 K42 t, s, zνdz is bounded and Q2 s ≥ 0, a1 ≥ 0 and Q1 s ≥ δ
for some δ > 0. Then there exists a unique element u ∈ AG such that
J Ju inf Jv.
2.1
v∈AG
Proof. For simplicity, we assume D3 t, s, z 0 and K5 t, s, z 0 in this proof because these
terms can be similarly estimated as the corresponding terms for Brownian motion B·. By
1.7 we have
⎡
⎡
2 ⎤
2 ⎤
t
t
2
2
E X u t ≤ 7E ξt 7E⎣
K1 t, sX u sdBs ⎦ 7E⎣
D1 t, susdBs ⎦
0
0
⎡
⎡
2 ⎤
2 ⎤
t
t
7E⎣
K2 t, sdBs ⎦ 7E⎣
K3 t, susds ⎦
0
0
⎡
⎡
2 ⎤
t
t 7E⎣
D2 t, sX u sds ⎦ 7E⎣
0
0
≤ 7E ξt2 7E
t
0
7
t
0
7E
K22 t, sds
t 0
R0
K12 t, sX u s2 ds
7
t
0
K32 t, sdsE
7E
2 ⎤
K4 t, s, zX u sNds,
dz ⎦
R0
t
0
t
D12 t, sus2 ds
u sds 7tE
2
0
t
0
D22 t, sX u s2 ds
K42 t, s, zνdz X u s2 ds .
2.2
6
International Journal of Stochastic Analysis
Applying Gronwall’s inequality, there exists a constant C1 such that
E X u t2 ≤
t
C1 E
2.3
u sds C1 eC1 T .
2
0
Similar arguments also lead to
E X u1 t − X u2 t2
⎛ ⎡
2 ⎤
t
≤ C2 eC2 T ⎝E⎣
K3 t, su2 s − u1 sds ⎦
0
E
t
2.4
D1 t, s2 u2 s − u1 s2 ds
0
for some constant C2 . Now, let un ∈ AG be a minimizing sequence for the value function, that
is, limn → ∞ Jun J. From the estimate 2.3 we see that there exists a constant c such that
T
E
0
Q3 susds T
Q4 sX sds a2 X T ≤ c
u
c.
u
u
2.5
0
Thus, by virtue of the assumption on Q1 , we have, for some constant M,
M ≥ Jun ≥ δ
un 2 − c
un − c.
2.6
This implies that {un } is bounded in U, hence weakly compact. Let unk , k ≥ 1 be a
subsequence that converges weakly to some element u0 in U. Since AG is closed and convex,
the Banach-Sack Theorem implies u0 ∈ AG . From 2.4 we see that un → u in U implies that
X un t → X u t in L2 Ω for every t ≥ 0 and X un · → X u · in U. The same conclusion holds
also for Zu t : X u t − X 0 t. Since Zu is linear in u, we conclude that equipped with the
weak topology both on U and L2 Ω, Zu t : U → L2 Ω is continuous for every t ≥ 0 and
Zu · : U → U is continuous. Thus,
X u t : U −→ L2 Ω,
X u · : U −→ U
2.7
are continuous with respect to the weak topology of U and L2 Ω. Since the functionals of X u
involved in the definition of Ju in 1.10 are lower semicontinuous with respect to the weak
International Journal of Stochastic Analysis
7
topology, it follows that
lim Junk lim E
k→∞
T
k→∞
0
Q1 su2nk sds
T
T
Q2 sX
unk
2
s ds T
0
Q3 sunk sds
0
Q4 sX
unk
sds a1 X
unk
2
T a2 X
unk
T 0
≥E
T
0
Q1 su20 sds T
T
Q2 sX u0 s2 ds T
0
2.8
Q3 su0 sds
0
2
Q4 sX sds a1 X T a2 X T u0
u0
u0
0
Ju0 ,
which implies that u0 is an optimal control.
The uniqueness is a consequence of the fact that Ju is strictly convex in u which is
due to the fact that X u is affine in u and x2 is a strictly convex function. The proof is complete.
To characterize the optimal control, we assume D1 t, s 0 and D3 t, s, z 0; that is,
consider the controlled system:
X t ξt u
t
t 0
0
K4 t, s, zX u sNds,
dz t
t
K3 t, susds
0
D2 t, sX u sds
2.9
0
R0
t 0
K1 t, sX s K2 t, sdBs u
K5 t, s, zNds,
dz
R0
Set
dFt, s : ds Ft, s
K4 t, s, zNds,
dz D2 t, sds.
K1 t, sdBs 2.10
R0
For a predictable process hs, we have
t
t
hsdFt, s :
0
0
K1 t, shsdBs t 0
K4 t, s, zhsNds,
dz R0
t
0
D2 t, shsds.
2.11
8
International Journal of Stochastic Analysis
Introduce
M1 t ξt ∞ t
n1
sn−1
···
dFt, s1 s1
0
dFs1 , s2 0
ξsn dFsn−1 , sn ,
0
M2 t t
K2 t, s1 dBs1 0
···
∞ t
sn−2
dFsn−2 , sn−1 M3 t 0
···
K2 sn−1 , sn dBsn ,
0
1 , dz K5 t, s1 , zdNds
R0
sn−2
dFsn−2 , sn−1 sn−1
dFt, s1 0
s1
2.12
dFs1 , s2 0
n , dz,
K5 sn−1 , sn , zdNds
0
Lt, s K3 t, s ∞ t
n1
sn−1
∞ t
n1
0
···
dFs1 , s2 0
sn−1
0
t dFt, s1 0
n1
s1
dFt, s1 s
s1
dFs1 , s2 s
K3 sn , sdFsn−1 , sn .
s
Lemma 2.2. Under our assumptions, the above series converges at least in L1 Ω. Thus Mi , i 1, 2, 3
and L are well-defined.
Proof. We first note that
⎡
2 ⎤
t t
t
2
2
hsdFt, s ⎦ E
K1 t, sh sds E
E⎣
0
0
0
R0
K42 t, s, zh2 sνdzds
⎡
2 ⎤
t
t
2
⎦
⎣
≤ CT E
D2 t, shsds
gt, sh sds
E
0
0
2.13
for t ≤ T , where
gt, s K12 t, s R0
K42 t, s, zνdz D22 t, s
2.14
International Journal of Stochastic Analysis
9
is a bounded deterministic function. Because of the similarity, let us prove only that M1 is
well-defined. Repeatedly using 2.13, we have
⎡
2 ⎤
s1
sn−1
t
dFt, s1 dFs1 , s2 · · ·
ξsn dFsn−1 , sn ⎦
E⎣
0
0
≤ CT
t
ds1 gt, s1 E
s
1
0
≤ ···
0
dFs1 , s2 · · ·
0
sn−1
2 ξsn dFsn−1 , sn 0
2.15
t
s1
sn−1
ds1 gt, s1 ds2 gs1 , s2 · · ·
dsn gsn−1 , sn E ξ2 sn 0
0 T
0
n−1
t
≤ Rn−1
ξ2 sds
T E
− 1!
n
0
≤ CTn−1
for some constant RT . This implies that
t
s1
sn−1
dFs1 , s2 · · ·
ξsn dFsn−1 , sn E dFt, s1 0
0
0
T
1/2
tn−1/2
n−1/2
2
≤ RT
E
ξ sds
.
n − 1!
0
2.16
Thus, we have
E|M1 t| ≤ E|ξt| ∞
n−1/2
RT
n1
T
1/2
tn−1/2
2
E
ξ sds
< ∞.
n − 1!
0
2.17
The following theorem is a characterization of the optimal control.
Theorem 2.3. Assume that R0 K42 t, s, zνdz and R0 K52 t, s, zνdz are bounded and
T
E 0 ξ2 sds < ∞. Suppose AG U. Let u be the unique optimal control given in Theorem 2.1.
Then u is determined by the following equation:
2Q1 sus 2E
2a1 E
2E
T
T
T
T
ut
0
s∨t
Q2 lLl, tLl, sdl dt | Gs
utLT, tLT, sdt | Gs Q3 s E
0
T
Q4 lLl, sdl | Gs
s
Q2 lM1 l M2 l M3 lLl, sdl | Gs a2 ELT, s | Gs s
2a1 EM1 T M2 T M3 T LT, s | Gs 0,
almost everywhere with respect to mds, dω : ds × P dω.
2.18
10
International Journal of Stochastic Analysis
Proof. For any w ∈ U, since u is the optimal control, we have
J uw d
Ju εw
0.
dε
ε0
2.19
This leads to
T
T
d uεw u
E 2
Q1 suswsds 2
Q2 sX s X
s
ds
dε
0
0
ε0
T
T
d
Q3 swsds Q4 s X uεw s
ds
dε
0
0
2.20
ε0
d uεw
d uεw
0
2a1 X T X
T a2 X
T dε
dε
ε0
ε0
u
for all w ∈ U. By virtue of 2.9, it is easy to see that
d uεw Y t :
X
t
dε
ε0
2.21
w
satisfies the following equation:
Y t w
t
0
K1 t, sY sdBs w
K3 t, swsds
0
t 0
t
K4 t, s, zY sNds,
dz w
t
2.22
D2 t, sY sds.
w
0
E
Remark that Y w is independent of u. Next we will find an explicit expression for X u . Let
dFt, s be defined as in 2.10. Repeatedly using 2.9 we have
X u t ξt t
t 0
0
1 , dz K4 t, s1 , zX s1 Nds
u
1 , dz
K5 t, s1 , zNds
R0
t
0
R0
t 0
K1 t, s1 X u s1 K2 t, s1 dBs1 t
K3 t, s1 us1 ds1
0
D2 t, s1 X u s1 ds
International Journal of Stochastic Analysis
t
s
ξt s1 0
0
s1
R0
2 , dz K4 s1 , s2 , zX s2 Nds
D2 s1 , s2 X s2 ds2 u
0
D2 t, s1 , z ξs1 0 R0
s1 0
t
D2 s1 , s2 X s2 ds2 K2 t, s1 dBs1 t
0
ξt ···
···
t
0
t
0
0
s1
s1
2 , dz ds1
K5 s1 , s2 , zNds
R0
K3 t, s1 us1 ds1 t 0
dFs1 , s2 · · ·
sn−1
1 , dz
K5 t, s1 , zNds
R0
ξsn dFsn−1 , sn 0
dFs1 , s2 sn−1
K2 sn−1 , sn dBsn 0
dFs1 , s2 0
dFsn−2 , sn−1 s1
K3 s1 , s2 us2 ds2
0
s1 0
dFsn−2 , sn−1 dFt, s1 0
sn−2
s1
s1
0
dFt, s1 0
sn−2
0
∞ t
n1
0
sn−2
0
∞ t
n1
dFt, s1 0
dFt, s1 ···
∞ t
n1
∞ t
n1
K1 s1 , s2 X u s2 K2 s1 , s2 dBs2 0
···
R0
2 , dz K4 s1 , s2 , zX s2 Nds
u
K3 s1 , s2 us2 ds2
1 , dz
K5 s1 , s2 , zNds2 , dz Nds
u
0
s1
s1
0
0
R0
s1
K1 s1 , s2 X u s2 K2 s1 , s2 dBs2 s1 t R0
2 , dz K4 s1 , s2 , zX u s2 Nds
R0
s1
s1
0
0
0
K4 t, s1 , z ξs1 0 R0
s1 0
s1 t s1
K3 s1 , s2 us2 ds2
2 , dz dBs1 K5 s1 , s2 , zNds
u
D2 s1 , s2 X u s2 ds2 0
K1 s1 , s2 X u s2 K2 s1 , s2 dBs2 0
0
1
K1 t, s1 ξs1 11
sn−1
K3 sn−1 , sn usn dsn
0
dFs1 , s2 0
dFsn−2 , sn−1 0
K2 t, s1 dBs1 sn−1 t
0
n , dz
K5 sn−1 , sn , zNds
R0
K3 t, s1 us1 ds1
0
1 , dz.
K5 t, s1 , zNds
R0
2.23
12
International Journal of Stochastic Analysis
Similarly, we have the following expansion for Y w :
Y t w
t
∞ t
K3 t, swsds 0
···
sn−2
dFsn−2 , sn−1 dFt, s1 0
n1
dFs1 , s2 0
sn−1
0
s1
2.24
K3 sn−1 , sn wsn dsn .
0
Interchanging the order of integration,
Y t w
t
ws K3 t, s 0
∞ t
n1
dFt, s1 s
s1
dFs1 , s2 · · ·
s
sn−1
K3 sn , sdFsn−1 , sn ds
s
t
2.25
Lt, swsds.
0
Now substituting Y w into 2.20 we obtain that
T
T
s
u
Q1 suswsds 2
Q2 sX s
Ls, lwldl ds
E 2
0
0
E
T
Q3 swsds 0
2a1 E
T
0
Q4 s
s
0
T
Ls, lwldl ds
2.26
0
X T LT, swsds a2
u
0
T
LT, swsds 0
0
for all w ∈ U. Interchanging the order of integration and conditioning on Gs we see that 2.26
is equivalent to
T
T
T
u
E 2
Q1 suswsds 2
wsE
Q2 lX lLl, sdl | Gs ds
0
E
T
Q3 swsds 0
2a1 E
0
s
T
T
wsE
0
T
Q4 lLl, sdl | Gs ds
s
EX u T LT, s | Gs wsds
0
a2 E
T
0
ELT, s | Gs wsds 0.
2.27
International Journal of Stochastic Analysis
13
Since this holds for all w ∈ U, we conclude that
2Q1 sus 2E
T
Q2 lX lLl, sdl | Gs Q3 s E
u
T
s
Q4 lLl, sdl | Gs
2.28
s
2a1 EX u T LT, s | Gs a2 ELT, s | Gs 0,
m-a.e. Note that X u t can be written as
X u t M1 t M2 t M3 t t
2.29
usLt, sds.
0
Substituting X u t into 2.28, we get 2.18, completing the proof.
Example 2.4. Consider the controlled system
X u t ξt t
K2 t, sdBs t
0
2.30
K3 t, susds
0
and the performance functional
Ju E
T
Q1 su sds 2
0
T
Q3 susds 0
T
2
Q4 sX sds a1 X T a2 X T .
u
u
u
0
2.31
Suppose Gt {Ω, ∅}, meaning that the control is deterministic. In this case, we can find
the unique optimal control explicitly. Noting that the conditional expectation reduces to
expectation, the 2.18 for the optimal control u becomes
2Q1 sus 2a1
T
utK3 T, tdt K3 T, s
0
Q3 s T
2.32
Q4 lK3 l, sdl a2 K3 T, s 2a1 gT K3 T, s 0 ds-a.e.,
s
where we have used the fact that EM2 t 0, M1 t ξt, Lt, s K3 t, s in this special
case. Put
b
T
2.33
utK3 T, tdt.
0
Then 2.33 yields
us −a1 b
K3 T, s
hs,
Q1 s
ds-a.e.,
2.34
14
International Journal of Stochastic Analysis
where
hs −
Q3 s T
Q4 lK3 l, sdl
s
2Q1 s
−
a2 K3 T, s 2a1 gT K3 T, s
.
2Q1 s
2.35
htK3 T, tdt b.
2.36
Substitute the expression of u into 2.34 to get
−a1 b
T
K3 T, t2
dt Q1 t
0
T
0
Consequently,
b
1 a1
1
T 0
K3 T, t2 /Q1 t dt
Together with 2.35 we arrive at
⎛
⎜
us −a1 ⎝
1 a1
T 0
1
2
K3 T, t /Q1 t dt
T
0
T
0
htK3 T, tdt.
2.37
⎞
⎟ K3 T, s
hs,
htK3 T, tdt⎠
Q1 s
2.38
ds-a.e.
3. A General Maximum Principle
In this section, we consider the following general controlled stochastic Volterra equation:
t
t
u
u
X t ξt bt, s, X s, us, ωds σt, s, X u s, us, ωdBs
0
0
3.1
t u
θt, s, X s, us, z, ωNds, dz,
0
R0
where ut is our control process taking values in R and ξt is as in 1.7. More precisely,
u ∈ AG , where AG is a family of Gt -predictable controls. Here Gt ⊂ Ft is a given subfiltration
and b : 0, T × 0, T × R × R × Ω → R, σ : 0, T × 0, T × R × R × Ω → R and θ : 0, T × 0, T ×
R × R × R0 × Ω → R are given measurable, Ft -predictable functions. Consider a performance
functional of the following form:
T
ft, X u t, ut, ωdt gX u T , ω ,
Ju E
3.2
0
where f : 0, T × R × D × Ω → R is Ft predictable and g : R × Ω → R is FT measurable and
such that
T
u
u
ft, X t, ut, ω dt gX T , ω < ∞, ∀u ∈ AG .
3.3
E
0
International Journal of Stochastic Analysis
15
The purpose of this section is to give a characterization for the critical point of Ju. First, in
the following two subsections we recall briefly some basic properties of Malliavin calculus
· which will be used in the sequel. For more information we refer to 11
for B· and N·,
and 12.
3.1. Integration by Parts Formula for B·
In this subsection, FT σBs, 0 ≤ s ≤ T . Recall that the Wiener-Ito chaos expansion
theorem states that any F ∈ L2 FT , P admits the representation
F
∞
In fn
3.4
n0
for a unique sequence of symmetric deterministic function fn ∈ L2 0, T ×n and
In fn n!
T tn
0
···
t2
0
fn t1 , . . . , tn dBt1 dBt2 · · · dBtn .
3.5
0
Moreover, the following isometry holds:
∞
! !2
E F2 n!!fn !L2 0,T ×n .
3.6
n0
Let D1,2 be the space of all F ∈ L2 FT , P such that its chaos expansion 3.4 satisfies
F
2D1,2 :
∞
! !2
nn!!fn !L2 0,T ×n < ∞.
3.7
n0
For F ∈ D1,2 and t ∈ 0, T , the Malliavin derivative of F, Dt F, is defined by
Dt F ∞
nIn−1 fn ·, t ,
3.8
n0
where In−1 fn ·, t is the n − 1 times iterated integral to the first n − 1 variables of fn keeping
the last variable tn t as a parameter. We need the following result.
Theorem A Integration by parts formula duality formula for B·. Suppose that ht is Ft T
adapted with E 0 h2 tdt < ∞ and let F ∈ D1,2 . Then
T
T
E F
htdBt E
htDt F dt .
0
0
3.9
16
International Journal of Stochastic Analysis
3.2. Integration by Parts Formula for N
s In this section FT σηs, 0 ≤ s ≤ T , where ηs 0 R0 zNdr,
dz. Recall that the WienerIto chaos expansion theorem states that any F ∈ L2 FT , P admits the representation
F
∞
In fn
3.10
n0
2 dt × νn , where L
2 dt × νn is the space of
for a unique sequence of functions fn ∈ L
functions fn t1 , z1 , . . . , tn , zn ; ti ∈ 0, T , zi ∈ R0 such that fn ∈ L2 dt × νn and fn is
symmetric with respect to the pairs of variables t1 , z1 , t2 , z2 , . . . , tn , zn . Here In fn is the
iterated integral:
In fn n!
T 0
tn R0
0
···
R0
t2 0
1 , dz1 · · · Ndt
n , dzn .
fn t1 , z1 , . . . , tn , zn Ndt
3.11
R0
Moreover, the following isometry holds:
∞
! !2
n!!fn !L2 dt×νn .
E F2 3.12
n0
" 1,2 be the space of all F ∈ L2 FT , P such that its chaos expansion 3.18 satisfies
Let D
F
2D"
1,2
:
∞
! !2
nn!!fn !L2 dt×νn < ∞.
3.13
n0
" 1,2 and t ∈ 0, T , the Malliavin derivative of F, Dt,z F, is defined by
For F ∈ D
Dt,z F ∞
nIn−1 fn ·, t, z ,
3.14
n0
where In−1 fn ·, t, z is the n − 1 times iterated integral with respect to the first n − 1 pairs
of variables of fn keeping the last pair tn , zn t, z as a parameter. We need the following
result
Suppose ht, z is Ft Theorem B Integration by parts formula duality formula for N.
T 2
"
predictable with E 0 R0 h t, zdtνdz < ∞ and let F ∈ D1,2 . Then
T ht, zNdt, dz E
T E F
0
R0
0
ht, zDt,z F dtνdz .
R0
3.15
International Journal of Stochastic Analysis
17
3.3. Maximum Principles
Consider 3.1. We will make the following assumptions throughout this subsection.
H.1 The functions b : 0, T ×0, T ×R×R×Ω → R, σ : 0, T ×0, T ×R×R×Ω → R,
θ : 0, T × 0, T × R × R × R0 × Ω → R, f : 0, T × R × R × Ω → R, and g : R × Ω → R
are continuously differentiable with respect to x ∈ R and u ∈ R.
H.2 For all t ∈ 0, T and all bounded Gt -measurable random variables α the
control
βα s αχt,T s
3.16
belongs to AG .
H.3 For all u, β ∈ AG with β bounded, there exists δ > 0 such that
u yβ ∈ AG
∀y ∈ −δ, δ.
3.17
H.4 For all u, β ∈ AG with β bounded, the process Y β t d/dyX uyβ t|y0
exists and satisfies the following equation:
t
∂b
∂b
u
β
Y t t, s, X s, usY sds t, s, X u s, usβsds
0 ∂x
0 ∂u
t
t
∂σ
∂σ
u
β
t, s, X s, usY sdBs t, s, X u s, usβsdBs
0 ∂x
0 ∂u
t ∂θ
dz
t, s, X u s, us, zY β sNds,
0 R0 ∂x
t ∂θ
dz.
t, s, X u s, us, zβsNds,
0 R0 ∂u
β
t
3.18
H.5 For all u ∈ AG , the Malliavin derivatives Dt g X u T and Dt,z g X u T exist.
In the sequel, we omit the random parameter ω for simplicity. Let Ju be defined as
in 3.2.
H.6 The functions ∂b/∂ut,
s, x, u2 , ∂b/∂xt,s, x, u2 , ∂σ/∂ut, s, x, u2 ,
2
∂σ/∂xt, s, x, u , and R0 ∂θ/∂ut, s, x, u, z2 νdz, R0 ∂θ/∂xt, s, x, u, z2 νdz
are bounded on 0, T × 0, T × R × R × Ω.
18
International Journal of Stochastic Analysis
Theorem 3.1 Maximum principle I for optimal control of stochastic Volterra equations. (1)
Suppose that u
is a critical point for Ju in the sense that d/dyJ
u yβ|y0 0 for all bounded
β ∈ AG . Then
T
E
t
T
∂f s, Xs, u
s Λs, tds ∂x
T
t
∂f s, Xs, u
s ds
∂u
∂b ds
T, s, Xs,
u
s Λs, tg XT
t ∂x
T
∂b ds
T, s, Xs,
u
s g XT
t ∂u
T
∂σ ds
T, s, Xs,
u
s Λs, tDs g XT
t ∂x
T
∂σ ds
T, s, Xs,
u
s Ds g XT
t ∂u
T ∂θ T, s, Xs, u
s, z Λs, tDs,z g XT νdz ds
t
R0 ∂x
T ∂θ νdz ds | Gt 0,
T, s, Xs,
u
s, z Ds,z g XT
t
R0 ∂u
3.19
X u .
where Λs, t is defined in 3.29 below and X
is a critical point for J·.
(2) Conversely, suppose u
∈ AG such that 3.19 holds. Then u
X u .
Proof. 1 Suppose that u
is a critical point for Ju. Let β ∈ AG be bounded. Write X
Then
T #
$
∂f ∂f d β
β
J u
yβ E
t, Xt, u
t Y t
t, Xt, u
t βt dtg XT Y T ,
0
dy
∂u
0 ∂x
y0
3.20
where
d uyβ X
t
dy
y0
t
t
∂b
∂b β
t, s, Xs, u
s Y sds t, s, Xs,
u
s βsds
0 ∂x
0 ∂u
t
t
∂σ ∂σ β
t, s, Xs, u
s Y sdBs t, s, Xs,
u
s βsdBs
0 ∂x
0 ∂u
t ∂θ t, s, Xs,
u
s, z Y β sNds,
dz
0 R0 ∂x
t ∂θ t, s, Xs,
u
s, z βsNds,
dz.
3.21
0 R0 ∂u
Y β t International Journal of Stochastic Analysis
19
By the duality formulae 3.9 and 3.15, we have
T
∂b β
β
T, s, Xs, u
s Y sg XT ds
E g XT Y T E
0 ∂x
E
T
0
E
∂b ds
T, s, Xs,
u
s βsg XT
∂u
T
0
E
T
0
E
∂σ β
T, s, Xs,
u
s Y sdBs g XT
∂x
∂σ T, s, Xs, u
s βsdBs g XT ∂u
R0
∂θ β
T, s, Xs,
u
s, z Y sNds,
dz g XT
∂x
R0
∂θ T, s, Xs,
u
s, z βsNds,
dz g XT
∂u
T 0
E
T 0
E
T
0
E
∂b β
T, s, Xs, u
s Y sg XT ds
∂x
T
0
E
T
0
E
T
0
E
∂b ds
T, s, Xs,
u
s βsg XT
∂u
∂σ β
ds
T, s, Xs,
u
s Y sDs g XT
∂x
∂σ T, s, Xs, u
s βsDs g XT ds
∂u
0
E
R0
∂θ β
νdzds
T, s, Xs,
u
s, z Y sDs,z g XT
∂x
R0
∂θ νdzds .
T, s, Xs,
u
s, z βsDs,z g XT
∂u
T T 0
3.22
20
International Journal of Stochastic Analysis
Let α be bounded, Gt measurable. Choose βα s αχt,T s and substitute 3.22 into 3.20
to obtain
T E
t
∂f s, Xs, u
s Y βα sds α
∂x
E
T
t
T
t
∂f s, Xs, u
s ds
∂u
∂b βα
T, s, Xs, u
s Y sg XT ds
∂x
T
∂b ds
T, s, Xs,
u
s g XT
E α
t ∂u
E
T
t
∂σ β
α
ds
T, s, Xs,
u
s Y sDs g XT
∂x
T
∂σ ds
T, s, Xs,
u
s Ds g XT
E α
t ∂u
E
R0
∂θ β
T, s, Xs, u
s, z Y sDs,z g XT νdzds
∂x
R0
∂θ T, s, Xs, u
s, z Ds,z g XT νdzds 0,
∂u
T t
T E α
t
3.23
where Y βα l 0 for l ≤ t, and for l ≥ t,
Y βα l l
t
∂b l, s, Xs,
u
s Y βα sds
∂x
α
l
∂b l, s, Xs,
u
s ds
∂u
t
l
t
α
∂σ l, s, Xs,
u
s Y βα sdBs
∂x
l
∂σ l, s, Xs,
u
s dBs
∂u
t
l t
α
R0
∂θ l, s, Xs,
u
s, z Y β sNds,
dz
∂x
l t
R0
∂θ l, s, Xs,
u
s, z Nds,
dz.
∂u
3.24
International Journal of Stochastic Analysis
21
For l ≥ s, put
dΓl, s : ds Γl, s
∂σ ∂b l, s, Xs,
u
s ds l, s, Xs,
u
s dBs
∂x
∂x
∂θ l, s, Xs,
u
s, z Nds,
dz.
R0 ∂x
3.25
This means that for a predictable process hs, we have
l
hsdΓl, s t
l
t
∂b l, s, Xs,
u
s hsds ∂x
l t
R0
l
t
∂σ l, s, Xs,
u
s hsdBs
∂x
∂θ l, s, Xs,
u
s, z hsNds,
dz.
∂x
3.26
Set
Dl, t l
t
∂b l, s, Xs,
u
s ds
∂u
l
t
∂σ l, s, Xs,
u
s dBs
∂u
l t
R0
3.27
∂θ l, s, Xs,
u
s, z Nds,
dz.
∂u
Repeatedly using the linear equation 3.24, as in the proof of 2.23, we obtain
3.28
Y βα l αΛl, t,
where
Λl, t Dl, t ∞ l
k1
···
sk−1
t
t
dΓl, s1 s1
dΓs1 , s2 t
Dsk , tdΓsk−1 , sk .
3.29
22
International Journal of Stochastic Analysis
As in the proof of Lemma 2.2, we can check that the above series converges in L1 Ω under
the assumption H.6. We substitute 3.28 into 3.23 to get
T
T
∂f ∂f s, Xs, u
s Λs, tds s, Xs, u
s ds
E α
t ∂x
t ∂u
T
t
T
∂b ds
T, s, Xs,
u
s Λs, tg XT
∂x
t
∂b ds
T, s, Xs,
u
s g XT
∂u
t
∂σ ds
T, s, Xs,
u
s Λs, tDs g XT
∂x
T
3.30
T
∂σ ds
T, s, Xs,
u
s Ds g XT
t ∂u
T ∂θ νdzds
T, s, Xs,
u
s, z Λs, tDs,z g XT
t
R0 ∂x
T t
R0
∂θ 0.
T, s, Xs, u
s, z Ds,z g XT νdzds
∂u
Since α is arbitrary, it follows that
T
E
t
∂f s, Xs, u
s Λs, tds ∂x
T
T
t
∂f s, Xs, u
s ds
∂u
t
∂b ds
T, s, Xs,
u
s Λs, tg XT
∂x
t
∂b ds
T, s, Xs,
u
s g XT
∂u
t
∂σ ds
T, s, Xs,
u
s Λs, tDs g XT
∂x
T
T
T
∂σ ds
T, s, Xs,
u
s Ds g XT
t ∂u
T ∂θ νdzds
T, s, Xs,
u
s, z Λs, tDs,z g XT
t
R0 ∂x
T ∂θ T, s, Xs, u
s, z Ds,z g XT νdzds | Gt 0,
t
R0 ∂u
3.31
completing the proof of 1.
International Journal of Stochastic Analysis
23
2 Suppose that 3.19 holds for some u
∈ AG . Running the arguments in the proof of
1 backwards, we see that 3.20 holds for all bounded β ∈ AG of the form βs αχt,T s.
This is sufficient because the set of linear combinations of such β is dense in AG .
Next we consider the case where the coefficients are independent of x. The maximum
principle will be simplified significantly. Fix a control u
∈ AG with corresponding state
process Xt.
Define the associated Hamiltonian process Ht, u by
σT, t, uDt g XT
Ht, u ft, u bT, t, ug XT
νdz; t ∈ 0, T , u ∈ R.
θT, t, u, zDt,z g XT
3.32
R0
Theorem 3.2 Maximum principle II for optimal control of stochastic Volterra equations.
Suppose that f, b, σ, θ are all independent of x. Then the followings are equivalent.
i u
is a critical point for Ju.
ii For each t ∈ 0, T , u u
t is a critical point for u → EHt, u | Gt , in the sense that
∂
EHt, u | Gt uut 0.
∂u
Proof. Suppose that f, b, σ, θ are all independent of x. Then 3.19 reduces to
T
∂f
E
sds
s, u
v ∂u
T
∂b
ds
sg XT
T, s, u
v ∂u
T
∂σ
ds
sDs g XT
T, s, u
v ∂u
T ∂θ
s, zDs,z g XT νdzds | Gv 0 ∀v ∈ 0, T .
T, s, u
v R0 ∂u
By inserting Gt we deduce that for all v ≥ t,
T
∂f
E
sds
s, u
v ∂u
T
∂b
ds
sg XT
T, s, u
v ∂u
T
∂σ
ds
sDs g XT
T, s, u
v ∂u
T ∂θ
s, zDs,z g XT νdzds | Gt 0.
T, s, u
v R0 ∂u
Taking the right derivative with respect to v at the point t we obtain 3.33.
3.33
3.34
3.35
24
International Journal of Stochastic Analysis
4. Applications to Stochastic Delay Control
We now apply the general maximum principle for optimal control of Volterra equations to
the stochastic delay problem 1.2-1.3 in the Introduction, by using the equivalence between
1.2 and 1.4. Note that in this case we have see 3.1, 3.2 and compare with 1.2, 1.3
ft, x, u U1 t, u, gx U2 x, bt, s, x, u Kt, su,
σt, s, x, u Φt, sCs, Θt, s, x, u, z Φt, sC2 s, z,
0
0 h
ξt Φt, 0η0 −h
Φt, s hA2 s hηsds −h
Φt, τA0 τ, sdτ ηsds.
0
4.1
Hence the system 1.4 satisfies the conditions of Theorem 3.2. By 3.32 we get the
Hamiltonian
ΦT, tCtDt g XT
Ht, u U1 t, u KT, tuU2 XT
R0
νdz.
ΦT, tC2 t, zDt,z U2 XT
4.2
Therefore by Theorem 3.2 we get the following condition for an optimal harvesting rate u
t:
, ω | Gt 0,
E U1 t, u
t, ω KT, tU2 XT
4.3
X u T and U ∂/∂xUi ; i 1, 2.
where XT
i
Now suppose that U1 and U2 are stochastic utilities of the form
" 1 t, u,
U1 t, u, ω γt ωU
" 2 x,
U2 x, ω ζωU
ω ∈ Ω,
ω ∈ Ω,
4.4
4.5
" 1, U
" 2 are concave, C1 -functions
where γt ω > 0 is Ft -adapted, ζω is FT -measurable, and U
on 0, ∞ and R, respectively. The 4.3 simplifies to
%
&
" XT
Gt .
" t, u
tE γt Gt −KT, tE ζU
U
2
1
4.6
This gives a relation between the optimal control u
t and the corresponding optimal terminal
. In particular, if
wealth XT
" 2 x x,
U
4.7
International Journal of Stochastic Analysis
25
we get
KT, tEζ | Gt " t, u
t −
U
%
& .
1
E γt | Gt
4.8
We have proved the following.
Corollary 4.1. The optimal consumption rate u
t for the stochastic delay system 1.2, 4.4, 4.5,
4.7 and the performance functional
Ju E
T
u
"
γt ωU1 t, utdt ζωX T 4.9
0
with partial information Gt is given by 4.8.
Acknowledgment
The research leading to these results has received funding from the European Research
Council under the European Community’s Seventh Framework Programme FP7/20072013/ERC grant agreement no. 228087.
References
1 I. Elsanosi, B. ∅ksendal, and A. Sulem, “Some solvable stochastic control problems with delay,”
Stochastics and Stochastics Reports, vol. 71, pp. 69–89, 2000.
2 B. ∅ksendal and A. Sulem, “A maximum principle for optimal control of stochastic systems with
delay, with applications to finance,” in Optimal Control and Partial Differential Equations-Innovations
and Applications, J. M. Menaldi, E. Rofman, and A. Sulem, Eds., pp. 64–79, IOS Press, Amsterdam, The
Netherlands, 2000.
3 V. B. Kolmanovskii and L. E. Shaikhet, Control of Systems with Aftereffect, vol. 157 of Translation of
Mathematical Monographs, American Mathematical Society, Providence, RI, USA, 1996.
4 B. Larssen, “Dynamic programming in stochastic control of systems with delay,” Stochastics and
Stochastics Reports, vol. 74, pp. 651–673, 2002.
5 B. Larssen and N. H. Risebro, “When are HJB-equations in stochastic control of delay systems finite
dimensional?” Stochastic Analysis and Applications, vol. 21, no. 3, pp. 643–671, 2003.
6 A. Lindquist, “On feedback control of linear stochastic systems,” SIAM Journal Control, vol. 11, no. 2,
pp. 323–343, 1973.
7 B. ∅ksendal and T. Zhang, “The stochastic Volterra equation,” in Barcelona Seminar on Stochastic
Analysis, D. Nualart and M. S. Solé, Eds., pp. 168–202, Birkhäauser, Boston, Mass, USA, 1993.
8 T. Meyer-Brandis, B. ∅ksendal, and X. Y. Zhou, “A Mean field stochastic maximum principle via
Malliavin calculus,” preprint, 2008.
9 A. Bensoussan, Stochastic Control of Partially Observable Systems, Cambridge University Press,
Cambridge, UK, 1992.
10 Y. Hu and B. ∅ksendal, “Partial information linear quadratic control for jump diffusions,” SIAM
Journal on Control and Optimization, vol. 47, no. 4, pp. 1744–1761, 2008.
11 G. Di Nunno, B. ∅ksendal, and F. Proske, Malliavin Calculus for Léevy Processes and Applications to
Finance, Springer, London, UK, 2009.
12 G. Di Nunno, T. Meyer-Brandis, B. ∅ksendal, and F. Proske, “Malliavin calculus and anticipating Ito
formulae for Lévy processes,” Infinite Dimensional Analysis, Quantum Probability and Related Topics, vol.
8, no. 2, pp. 235–258, 2005.