Optimal impulse control on an unbounded
domain with nonlinear cost functions
Stefano Baccarin∗ and Simona Sanfelici†
Keywords: impulse control, stochastic cash management, quasi-variational inequalities, finite element approximation.
Mathematics Subject Classification (2000): 49J40, 60G40, 65N30.
Abstract
In this paper we consider the optimal impulse control of a system which
evolves randomly in accordance with a homogeneous diffusion process in
R1 . Whenever the system is controlled a cost is incurred which has a
fixed component and a component which increases with the magnitude
of the control applied. In addition to these controlling costs there are
holding or carrying costs which are a positive function of the state of the
system. Our objective is to minimize the expected discounted value of
all costs over an infinite planning horizon. Under general assumptions on
the cost functions we show that the value function is a weak solution of a
quasi-variational inequality and we deduce from this solution the existence
of an optimal impulse policy. The computation of the value function is
performed by means of the Finite Element Method on suitable truncated
domains, whose convergence is discussed.
1
Introduction
We consider a fund x(t) whose dynamics in absence of control is described by a
homogeneous diffusion process in <1 . Holding or carrying costs of the fund are
continuously incurred at a rate f (x(t)) which is a positive function of x(t). To
reduce these costs the controller can at any time increase or decrease the fund by
a quantity ξ ∈ <, but each time the system is controlled a cost C(ξ) is incurred.
We will assume for these controlling costs the form C(ξ) = k + c(ξ) where k > 0
is a fixed component and c(ξ) is a component which increases with the size of
the intervention. Given a discount rate γ > 0 our objective is to minimize the
expected discounted value of all these costs over an infinite planning horizon.
∗ Department of Applied Mathematics, University of Torino, Piazza Arbarello 8,
I-10122 Torino, Italy. E-mail: stefano.baccarin@unito.it
† Department of Economics, University of Parma, Via J.F. Kennedy 6, I-43100
Parma, Italy. E-mail: simona.sanfelici@unipr.it
1
For a concrete financial application of this model one may consider the socalled “stochastic cash management problem”. In this case x(t) is a cash fund
which fluctuates randomly because it collects a great number of different receipts
and payments. Whenever the cash level is positive an opportunity (holding)
cost must be accounted for and whenever cash falls below zero penalty costs are
incurred. Therefore it would be convenient for the treasurer to maintain at zero
the cash fund by continuously buying (when x(t) > 0) or selling (when x(t) < 0)
short-term securities, but each time he converts cash into securities he must bear
transaction costs. The presence of a fixed component in the transaction costs
makes a continuous control unprofitable and the financial manager intervenes
only at isolated points of time and for discrete operations.
From a theoretical point of view our problem is one of impulse control where
a policy is made up of a sequence ti of stopping times and corresponding random jumps ξi enforced upon the system. It is well known that applying the
optimality principle of Dynamic Programming we can associate an impulse control problem with a quasi-variational inequality1 . If one succeeds in finding
a regular solution of this inequality it can be shown that this solution is the
value function and from this solution it is possible to derive the optimal policy
(see, for example, Richard (1977), Harrison, Sellke and Taylor (1983), Eastham
and Hastings (1988), Korn(1997,1998)). In this paper using a variational approach and the functional analysis techniques developed in the monographs of
Bensoussan e Lions (1982,1984) we go beyond “verification theorems”. Under
general assumptions we show that the value function of our problem is always a
weak solution of the associated quasi-variational inequality in a suitable Sobolev
space. From this solution we can obtain the optimal control whose existence is
therefore always ensured.
The analytical solution in the form of a control band policy can be obtained
only in some special cases involving, for instance, linear or quadratic holdingpenalty costs (see Harrison, Sellke and Taylor (1983), Baccarin (2002)). For
more general specifications, a numerical approach to the problem is compulsory.
We perform a Finite Element (FE) approximation by means of continuous,
piecewise linear functions defined on a suitable truncated domain [−r, r] and
vanishing on the boundary. The convergence of the solution ur of the truncated
problem to the value function as r → +∞ can be shown. The FE approximation
urh is obtained by means of the “Bensoussan-Lions discrete iterative process”,
whose convergence analysis on bounded domains can be found in Loinger (1980),
Cortey-Dumont (1980) and Boulbrachene (1998).
The paper is organized as follows. In §2 we give a precise formulation of our
impulse control problem and we recall some results of Bensoussan and Lions
(1984) concerning a variational inequality in a weighted Sobolev space. In §3
we prove some properties of the non local operator associated with the transaction cost structure. Furthermore we consider a weak formulation of the quasi
variational inequality in a weighted Sobolev space and we show the existence of
1 A formal argument which shows how to derive the quasi-variational inequality from Belmann’s optimality principle can be found, for the cash management problem, in Constantinides
and Richard (1978) or in Baccarin (2002).
2
a minimum solution for this inequality. §4 is devoted to the characterization of
the value function of the problem as the minimum solution of the quasi variational inequality. Moreover using the value function we prove the existence of an
optimal impulse policy dividing < in two regions: a continuation set where the
system evolves freely and a transaction region where the system is controlled.
Finally, in §5 we show the FE approach for computing the value function and
some numerical simulations, with different data specifications, are performed
and analyzed in §6.
2
Problem formulation and preliminary results
We consider a filtered probability space S = (Ω, F, P, Ft ) and a one-dimensional
standard Wiener process Wt adapted to Ft . We are given the following functions
(
σ(x) > λ > 0 , σ(x) ∈ W 1,∞ (R)
(1)
b(x) ∈ W 1,∞ (R) .
The dynamics of the cash fund x(t), in absence of control, is described by the
following Itô stochastic differential equation
(
dx(t) = b(x)dt + σ(x)dWt
x(0) = x .
An impulse policy V = (t1 , ξ1 ; t2 , ξ2 ; ...; ti , ξi ; .....) is an increasing sequence
(i ≥ 1) of stopping times 0 ≤ t1 ... ≤ ti ≤ ti+1 .....(with respect to Ft ) and
corresponding random variables ξi , which represent the jumps enforced upon
the system. An impulse control is said to be admissible if it satisfies the two
feasibility conditions
ti → ∞ a.s. when i → ∞
(2)
ξi is Fti measurable ∀i ≥ 1
where Fti is the minimum σ-algebra of events prior to ti 2 . We will denote by
A the set of admissible policies. When policy V is used, the controlled process
y(t) is generated by the following set of stochastic differential equations with
random initial conditions3

when ti ≤ t < ti+1 (i ≥ 0)
 dy(t) = b(y)dt + σ(y)dWt
(3)
y(ti ) = y(t−
)
+
ξ
∀i
≥1
i
i

y(t0 ) = x
2 t → ∞ implies that only a finite number of actions can be taken in any bounded interval;
i
ξi is Fti measurable means that the ξi decision depends only on the information available at
ti .
3 Since σ(x) and b(x) are Lipschitz continuous this system has a strong solution: the probability space S = (Ω, F, P, Ft ) is fixed and it will not depend on the control V .
3
−
4
where t0 ≡ 0 and y(t−
i ) = lim y(t) if ti−1 < ti , or y(ti ) = y(ti−1 ) if ti−1 = ti .
t↑ti
By yx (t) we will denote the controlled process y(t) starting in x = y(0).
We make the following assumptions on the holding/penalty costs f (x) and on
the variable part c(ξ) of the transaction costs C(ξ) :
|
s
(4)
c(ξ) → ∞ if | ξ |→ ∞,
c(ξ) is sub-linear c(ξ1 + ξ2 ) ≤ c(ξ1 ) + c(ξ2 )
(5)
f (x) is measurable; 0 ≤ f (x) ≤ f0 (1 + |x| ), s > 0
c(ξ) : < → <+ is continuous, c(0) = 0,
∀ξ1 , ξ2 ∈ < .
Thus we are imposing polynomial growth conditions on f (x) and we suppose
c(ξ) continuous, sublinear and unbounded from above when | ξ |→ ∞. The
sublinearity condition on c(ξ) is largely verified in applications, because implementing the same transfer of ξ = ξ1 + ξ2 is usually cheaper if we make it with
one transaction.
To each feasible control V and initial condition x = y(0) it is associated the
cost
)
(∞
Z ∞
X
−γs
−γti
e
f (yx (s))ds ,
Jx (V ) = Ex
χti <∞ +
C(ξi ) e
0
i=1
where γ > 0 is the discount rate and
1
χti <∞ (ω) =
0
if ti (ω) < ∞
elsewhere .
Our objective is to characterize the value function U (x) = inf Jx (V ) and
V ∈A
to show that, for every x ∈ <, there exists an optimal control V̂ such that
inf Jx (V ) = Jx (V̂ ) .
V ∈A
We define the following functions a2 (x) =
From (1) it results
a2 (x) ∈ W 1,∞ (<), a2 (x) >
a1 (x) ∈ L∞ .
σ 2 (x)
2 ,
λ2
2
a1 (x) = −b(x) + a02 (x).
≡α>0
(6)
Since the cost functions of our problem are unbounded in < it is convenient
to characterize U (x) as a solution of a variational problem not in W 1,∞ (<)
but in a weighted Sobolev space, in such a√way to allow for the limit condition
2
U (x) → +∞ as |x| → ∞. Let wµ (x) = e−µ 1+x be the weight function (µ > 0);
we define Lp,µ (<) functions as follows
Z
p
Lp,µ (<) = f | wµp (x) | f | dx < ∞ .
4 More
than one control action is allowed at the same time instant.
4
We consider weighted Sobolev spaces that, in the sense of distributions, can be
defined as
W 1,p,µ (<)
W 2,p,µ (<)
=
=
{f ∈ Lp,µ | f 0 (x) ∈ Lp,µ }
{f ∈ Lp,µ | f 0 (x), f 00 (x) ∈ Lp,µ }
and in particular H 1,µ = W 1,2,µ (R), H 2,µ = W 2,2,µ (R) . We denote by
Z
Z
Z
2
2
(u, v)µ =
u v wµ dx,
(u, v)H 1,µ =
u v wµ dx +
u0 v 0 wµ2 dx
R
R
R
the inner products respectively in L2,µ and H 1,µ , which are Hilbert spaces, and
by |u|µ ||u||µ the corresponding norms.
Let us consider the continuous bilinear form in H 1,µ
Z
Z
2a2 xµ 0
(7)
aµ (u, v) =
)u v wµ2 dx.
a2 u0 v 0 wµ2 dx + (a1 − √
1 + x2
R
R
For fixed µ > 0 it can be easily shown that there exist ρ, β > 0 such that it
results
2
2
aµ (u, u) + ρ |u|µ ≥ β ||u||µ
(8)
that is to say the bilinear form Aρ,µ (u, v) = aµ (u, v)+ρ (u, v)µ becomes coercive
on H 1,µ .
We set
Z ∞
u0 (x) = E
e−γt f (x(t))dt
(9)
0
which is the cost function corresponding to no action (i.e. t1 = +∞). From (4)
it follows that u0 (x) is finite and continuous in <.
In the sequel we will be interested in the following variational inequality

 aµ (u, v − u) + γ (u, v − u)µ ≥ (f, v − u)µ
1,µ
(10)
such that v ≤ ψ
 ∀v ∈ H1,µ
0
u ∈ H , u ≤ ψ, 0 ≤ u(x) ≤ u (x)
where we are making the underlying assumptions on the obstacle ψ
ψ continuous, ψ ∈ L2,µ , ψ ≥ 0.
(11)
In Bensoussan and Lions (1984) it is proved that there exists one and only one
solution uψ (x) of (10).
We consider now the process y(t) controlled by an admissible policy V. Bensoussan and Lions (1984) show that the only solution5 uψ (x) verifies the next
essential theorem
5 In
the sequel we will always take the continuous representative of uψ (x).
5
Theorem 1 Let ti ≤ ti+1 be two consecutive stopping times of V and y(t) the
corresponding controlled process. The solution uψ (x) of (10) verifies
Z ti+1
f (y(s)e−γs ds
(12)
e−γti uψ (y(ti ))χti <∞ ≤ E χti <∞
ti
.
+e−γti+1 uψ (y(t−
i+1 ))χti+1 <∞ |Fti
Furthermore if ti+1 = t = inf {t ≥ ti |uψ (y(t− )) = ψ(y(t− )) } then the equality
holds
Z t
−γti
uψ (y(ti ))χti <∞ = E χti <∞
e
f (y(s)e−γs ds
(13)
ti
+e−γt uψ (y(t))χt<∞ |Fti .
This a fundamental result because the regularity properties of uψ ∈ H 1,µ
are much weaker than those usually required to apply Itô’s Lemma. Moreover
the solution of (10) ensures the existence and uniqueness of uψ .
3
The quasi-variational inequality associated with
the impulse control problem
In order to consider the controlling costs C(ξ) we introduce the non local operator
M u(x) = inf [C(ξ) + u(x + ξ)]
ξ∈<
which from (5) is well defined for all u(x) : < → < bounded from below.
The next theorem gives some properties of M u.
Theorem 2 Given u(x) bounded from below and assumptions (5) the operator
M satisfies the following properties.
1. If u(x) ≥ z(x) ≥ 0 then M u(x) ≥ M z(x) ≥ 0 .
2. If u(x) is continuous then there exists a Borel measurable function ξˆu (x) :
< → < which verifies
M u(x) = C(ξˆu (x)) + u(x + ξˆu (x))
∀x ∈ < .
(14)
3. If u(x) is continuous then M u(x) is continuous .
4. Assume 0 ≤ u0 (x) ≤ ... ≤ un (x) ≤ ... ≤ u(x) and kun − ukC 0 (I¯r ) → 0, when
n → ∞, ∀r > 0 fixed. Then it follows kM un − M ukC 0 (I¯r ) → 0, ∀r > 0 fixed.
Proof. Property 1 is evident from the definition of M u.
To deduce property 2 we observe that, for fixed x, the function
Gu,x (ξ)
≡
C(ξ) + u(x + ξ) is continuous and we have
lim Gu,x (ξ) = +∞ because lim C(ξ) = +∞ and u(x) is bounded from be|ξ|→∞
|ξ|→∞
low. Therefore Gu,x (ξ) achieves its greatest lower bound and for every x ∈ <
6
we can choose a point of global minimum ξˆu (x) of Gu,x (ξ) in order to define a
Borel measurable function ξˆu (x) : < → < which satisfies (14).
We prove property 3 by showing that for any sequence xn → x, we have
M u(xn ) → M u(x), ∀x ∈ <. For ξ ∈ < fixed it holds
M u(xn ) ≤ C(ξ) + u(xn + ξ)
and from xn → x and the continuity of u it follows
lim sup M u(xn ) ≤ C(ξ) + u(x + ξ) .
n→∞
Since we can choose ξ arbitrarily, we obtain for any sequence xn → x
lim sup M u(xn ) ≤ M u(x) .
n→∞
Let xn → x be a sequence such that the corresponding M u(xn ) converges. From
(14) it holds
M u(xn ) = C(ξˆu (xn )) + u(x + ξˆu (xn ))
and from the continuity of u we obtain, for r > 0 and n large enough
C(ξˆu (xn )) + u(x + ξˆu (xn )) ≤ C(0) + u(x) + r .
Since lim C(ξ) = +∞ the sequence ξˆu (xn ) stays bounded and we can find
|ξ|→∞
a subsequence (xn , ξˆu (xn )) converging to (x, ξ ∗ ). Since C(ξ) is continuous it
follows
lim M u(xn ) = C(ξ ∗ ) + u(x + ξ ∗ ) ≥ M u(x).
n→∞
This result is true for any sequence xn → x such that M u(xn ) converges. Therefore we can deduce, for any sequence xn → x
lim sup M u(xn ) ≤ M u(x) ≤ lim inf M u(xn )
n→∞
n→∞
and consequently M u is continuous.
In order to prove property 4 we first show that M un (x) converges pointwise
to M u(x), ∀x ∈ <. From un (x) ≤ u(x) it follows
M un (x) = C(ξˆun (x)) + un (x + ξˆun (x)) ≤ C(0) + u(x) .
Since C(ξˆun (x)) is bounded and
ˆ
ξun (x) ≤ L, ∀n ∈ N.
Setting r = |x| + L we obtain
M un (x)
≥
≥
lim C(ξ) = +∞, for fixed x it must be
|ξ|→∞
C(ξˆun (x)) + u(x + ξˆun (x)) − ku − un kC 0 (I¯r )
M u(x) − ku − un kC 0 (I¯r ) .
7
From property 1, M un (x) ≤ M u(x) and therefore it holds
M u(x) − ku − un kC 0 (I¯r ) ≤ M un (x) ≤ M u(x)
∀n ∈ N.
By assumption ku − un kC 0 (I¯r ) → 0, ∀r > 0 fixed, and this implies
lim M un (x) = M u(x). We can repeat the preceding argument ∀x ∈ < obtain-
n→∞
ing that M un (x) converges pointwise to M u(x) in <. Property 4 then follows
from properties 1, 3 and Dini’s theorem.
We consider now the set of functions
K = z ∈ H 1,µ | 0 ≤ z ≤ u0
and we define an operator T : K → K, which relates z ∈ K to the continuous solution η ∈ K of (10) corresponding to the obstacle ψ = M z, where
M z is continuous from property 3 of M. The operator T has the fundamental
monotonicity property
T z1 ≤ T z 2
if z1 ≤ z2
(15)
which follows from property 1 of M and the fact that the solution of (10)
increases when the obstacle ψ is increasing (see Bensoussan, Lions 1978).
We will look for the value function U (x) of our problem among the solutions
of the underlying quasi-variational inequality6

 aµ (u, v − u) + γ (u, v − u)µ ≥ (f, v − u)µ
(16)
∀v ∈ H 1,µ such that v ≤ M u

u ∈ H 1,µ , u ≤ M u, 0 ≤ u ≤ u0 .
It is immediate to observe that u is a solution of (16) if and only if u is a fixed
point of the operator T. It should be noted also that if a regularity result of
the type u ∈ H 2,µ holds true, than one can easily show the equivalence between
(16) and the strong formulation

Au ≤ f



u ≤ Mu
(17)
(Au
− f )(u − M u) = 0



0 ≤ u ≤ u0
where A is the second order differential operator
Au = −
du
du
d
(a2
) + a1
+ γu .
dx
dx
dx
The strong formulation (17) of the quasi-variational inequality is the one commonly used to obtain verification theorems for stationary (infinite horizon) impulse control problems.
Inequality (16) may have many solutions, as it is usual with unbounded domains,
but the most relevant solution comes out to be the minimum solution.
6 The attribute quasi-variational points out that we have an implicit obstacle ψ = M u
defined by means of the unknown function u.
8
Theorem 3 Under assumptions (4), (5), (6) the quasi variational inequality
(16) has a minimum solution umin . Moreover, the function umin verifies
umin (x) ≤ inf Jx (V )
V ∈A
∀x ∈ < .
(18)
Proof. We consider the sequence un = T un−1 starting from u0 = 0. By
u1 = T u0 ≥ 0 = u0 and the monotonicity property (15) of T we obtain recursively un+1 = T un ≥ un = T un−1 and the sequence un ∈ C 0 (<) is increasing.
Therefore un converges pointwise to a function umin , with 0 ≤ umin ≤ u0 . From
un+1 ≤ M un it follows
un+1 (x) ≤ C(ξ) + un (x + ξ)
∀ξ ∈ <
and when n → ∞, as ξ is arbitrary, we deduce immediately
umin (x) ≤ M umin (x), ∀x ∈ <.
The sequence un converges strongly in L2,µ to umin because 0 ≤ un ≤ u0
and u0 ∈ L2,µ . Setting v = 0 into (10), which is always admissible, we obtain
aµ (un , un ) + γ (un , un )µ ≤ (f, un )µ
∀n ∈ N.
Using (8) we can deduce, for ρ large enough
2
β ||un ||µ
2
aµ (un , un ) + γ |un |µ + (ρ − γ) |un |µ
≤
(f, un )µ + (ρ − γ) |un |µ
|un |µ (|f |µ + (ρ − γ) u0 µ )
≤
and then it follows
2
≤
2
β ||un ||µ ≤ |f |µ + (ρ − γ) u0 µ
∀n ∈ N .
The norms ||un ||µ stay bounded in H 1,µ , thus we can find a subsequence um
which converges weakly to a function u∗ ∈ H 1,µ . Since the injection of H 1,µ in
L2,µ is compact we obtain also that um converges strongly to u∗ in L2,µ and
consequently u∗ = umin almost everywhere. If we consider any fixed interval
Ir , the norms ||um ||H 1 (Ir ) of um in H 1 (Ir ), remain uniformly bounded by a
constant Mr , which depends only on r, ∀m ∈ N . Then we can apply Morrey’s
theorem on Sobolev spaces (see, for example, Brezis 1983) and we obtain
1
|um (x) − um (y)| ≤ Cr |x − y| 2
∀m, ∀x, y ∈ I¯r
that is the functions um (x) ∈ C 0 (I¯r ) are uniformly Hölder continuous on I¯r ,
∀r > 0 fixed. This implies umin ∈ C 0 (I¯r ) and7
kum − umin kC 0 (I¯r ) → 0
7 The
∀r fixed, when m → ∞.
(19)
function umin , which we defined pointwise, is the continuous representative of u∗ .
9
Let v ∈ H 1,µ satisfy v ≤ M umin . If we choose a function g ∈ Cc∞ (<) such that
1 if x ∈ I1
g(x) =
0 if x ∈ </I2
and we define gr (x) = g( xr ), r > 0, one can easily show that, when r → ∞,
vgr → v in H 1,µ . For ε > 0 we have
(v − ε) gr
≤
=
≤
(M umin − ε) gr
M um gr + (M umin − M um )gr − εgr
M um gr + (kM umin − M um kC 0 (I¯2r ) − ε)gr .
From Property 4 of M and (19) it follows kM umin − M um kC 0 (I¯2r ) → 0, when
m → ∞, and for fixed r, ε we can find N = N (ε, r) such that
(v − ε) gr ≤ M um gr ≤ M um
∀m > N (ε, r) .
The function (v − ε) gr is therefore an admissible test function for every problem
um+1 , with m > N, and substituting (v − ε) gr into (10) we obtain
aµ (um+1 , (v − ε) gr − um+1 ) + γ (um+1 , (v − ε) gr − um+1 )µ ≥
≥ (f, (v − ε) gr − um+1 )µ
∀m > N
Since aµ (u, u) + γ(u, u)µ is weakly lower semi-continuous in H 1,µ , and um converges weakly to umin , we can deduce
aµ (umin , (v − ε) gr − umin ) + γ (umin , (v − ε) gr − umin )µ ≥
≥ (f, (v − ε) gr − umin )µ
∀v ∈ H 1,µ , v ≤ M umin , ∀ε, r > 0 .
Making ε → 0 and afterwards r → ∞ it follows
aµ (umin , v − umin ) + γ (umin , v − umin )µ ≥ (f, v − umin )µ
∀v ∈ H 1,µ such that
v ≤ M umin
and we have proved that umin is a solution of (16). It is immediate to show
that umin is the minimum solution. Any solution u of (16) is a fixed point
of the operator T and from
u0 = 0 ≤ u
we obtain by recurrence that
um = T um−1 ≤ u = T m u. When m → ∞ this inequality implies umin ≤ u.
In order to prove the second part of the theorem we observe that the function
un (x) is the solution of (10), corresponding to the obstacle ψ = M un−1 (x). Thus
we can apply (12) to tj , tj+1 , and un−j , with 0 ≤ j ≤ n − 1. It follows that
e−γtj un−j (yx (tj ))χtj <∞ ≤
(20)
i
h
R tj+1
≤ E χtj <∞ tj f (yx (s)e−γs ds + e−γtj+1 un−j (yx (t−
j+1 ))χtj+1 <∞ Ftj .
10
As un−j ≤ M un−j−1 we can deduce
un−j (yx (t−
j+1 ))
≤
=
C(ξj+1 ) + un−j−1 (yx (t−
j+1 ) + ξj+1 )
(21)
C(ξj+1 ) + un−j−1 (yx (tj+1 )).
Substituting (21) into (20) and taking the mathematical expectation we have
j
Ee−γt
h un−jR(yx (tj ))χtj <∞ ≤
t
≤ E χtj <∞ tjj+1 f (yx (s)e−γs ds
+ e−γtj+1 (C(ξj+1 ) + un−j−1 (yx (tj+1 )))χtj+1 <∞ .
If we recall that u0 ≡ 0, t0 ≡ 0, yx (t0 ) = x and we sum up all these inequalities
for j varying from 0 to n − 1 we obtain


Z tn
n
X
un (x) ≤ E 
e−γtj C(ξj )χtj <∞  .
f (yx (s)e−γs ds +
0
j=1
When n → ∞ this inequality implies umin (x) ≤ Jx (V )
4
∀x ∈ <, ∀V ∈ A.
Value function and optimal policy
We describe now the optimal policy V̂ . For this purpose we define in < two
regions: a continuation region F where the system evolves freely, which is the
open set
F = {x ∈ < : umin (x) < M umin (x)}
(22)
and the complementary intervention region G = F C , where the system is controlled. The first optimal stopping time is defined to be the first exit time of
the uncontrolled process from F
t̂1 = inf {t ≥ 0 | yx (t) ∈
/ F} .
In t̂1 it is enforced the optimal jump
ˆ x (t̂− ))
ξ(y
1
ˆ
ξ1 =
arbitrary
if t̂1 < ∞
if t̂1 = +∞
the chosenfunction which verifies (14) corresponding to
where ξˆ = ξˆumin (x) is umin . The subsequent t̂i+1 , ξˆi+1 are defined recursively by (i ≥ 1)
t̂i+1 = inf t ≥ t̂i | yx (t− ) ∈
/F
ˆ x (t̂− ))
ξ(y
if t̂i+1 < ∞
i+1
ξˆi+1 =
arbitrary
if t̂i+1 = +∞
11
.
(23)
It is interesting to note that, if the optimal stopping time t̂i is finite, ξˆi puts
back the system inside F. This implies t̂i+1 > t̂i , that is to say the optimal
stopping times are separated whenever they are finite. In fact we have
umin (yx (t̂i ))
=
ˆ
umin (yx (t̂−
i ) + ξi )
=
ˆ
M umin (yx (t̂−
i )) − C(ξi )
ˆ
ˆ
C(ξˆi + ξ) + umin (yx (t̂−
i ) + ξi + ξ) − C(ξi )
≤
∀ξ ∈ <
and from the sublinearity of c(ξ) we obtain
umin (yx (t̂i ))
≤
=
ˆ
umin (yx (t̂−
i ) + ξi + ξ) + C(ξ) − C(0)
umin (yx (t̂i ) + ξ) + C(ξ) − C(0)
∀ξ ∈ < .
As ξ is arbitrary we can deduce (t̂i < ∞)
umin (yx (t̂i )) ≤ M umin (yx (t̂i )) − C(0) < M umin (yx (t̂i ))
and therefore yx (t̂i ) ∈ F and consequently t̂i+1 > t̂i .
For policy V̂ to be admissible it has still to be shown that t̂i → ∞ a.s. when
i → ∞. Next theorem shows that V̂ is admissible and optimal.
o
n
Theorem 4 Under assumptions (1), (4), (5), the policy V̂ = t̂i , ξˆi defined
in (23) is admissible and optimal. Furthermore the minimum solution umin of
(16) is the value function, that is to say
umin (x) = inf Jx (V ) = Jx (V̂ ) .
(24)
V ∈A
Proof. We look at umin (x) as the solution of (10) with ψ = M umin . Since
it is always
−
umin (yx (t̂−
i )) = M umin (yx (t̂i )) = ψ
we can apply (13) between t̂i and t̂i+1 . It follows
(25)
e−γ t̂i umin (yx (t̂i ))χt̂i <∞ =
i
h
R t̂i+1
F
.
f (yx (s)e−γs ds + e−γ t̂i+1 umin (yx (t̂−
))χ
= E χt̂i <∞ t̂
t̂i+1 <∞
t̂i
i+1
i
But there holds
ˆ
umin (yx (t̂−
i+1 ))χt̂i+1 <∞ = C(ξi+1 )χt̂i+1 <∞ + umin (yx (t̂i+1 ))χt̂i+1 <∞
and substituting (26) into (25) and taking expectation we obtain
"
Z t̂i+1
−γ t̂i
f (yx (s)e−γs ds
umin (yx (t̂i ))χt̂i <∞ = E χt̂i <∞
Ee
t̂i
+ χt̂i+1 <∞ C(ξˆi+1 ) e−γ t̂i+1
i
+ E e−γ t̂i+1 umin (yx (t̂i+1 ))χt̂i+1 <∞ .
12
(26)
(27)
Summing up (26) for i varying from 0 to n − 1 it follows
"Z
t̂n
umin (x)
=
E
f (yx (s)e−γs ds
(28)
0
+
n
X
χt̂i <∞ C(ξˆi ) e
−γ t̂i
i=1
#
+E e−γ t̂n umin (yx (t̂n ))χt̂n <∞ .
As umin ≥ 0, f ≥ 0 this equality implies
C(0)
∞
X
i=1
χt̂i <∞ e−γ t̂i < ∞
a.s.
whence we obtain immediately that V̂ is admissible
t̂i → ∞
a.s.
Furthermore from (28) it follows
"Z
t̂n
umin ≥ E
when i → ∞ .
f (yx (s)e−γs ds +
0
n
X
#
χt̂i <∞ C(ξˆi ) e−γ t̂i ,
i=1
∀n
and when n → ∞ we can deduce
umin (x) ≥ Jx (V̂ )
and consequently from (18) we obtain (24).
5
Computation of the value function
The numerical solution to problem (16) can be accomplished in two steps. First,
we consider a sequence of approximated problems over suitable truncated domains Ir = [−r, r], r → +∞, with homogeneous Dirichlet boundary conditions.
The convergence in L∞
loc (<) of the solutions of the truncated problems to u min
as r → +∞ is shown in Theorem 5. Second, thanks to this convergence result,
we perform a Finite Element (FE) approximation to (16) by means of continuous, piecewise linear functions defined on a convenient truncated interval I r and
vanishing on the boundary. This procedure yields an accurate approximation to
umin on any fixed compact domain, if we choose r sufficiently large with respect
to the domain of interest.
For r > 0 fixed, we consider the following quasi variational inequality in
H01 (Ir )

 ar (u, v − u) + γ (u, v − u)r ≥ (f, v − u)r
∀v ∈ H01 (Ir ) such that v ≤ Mr u
(29)

u ∈ H01 (Ir ), u ≤ Mr u, 0 ≤ u ≤ u0 .
13
where
ar (u, v) =
Z
a2 u0 v 0 dx +
Ir
Z
Mr u(x) =
a1 u0 v dx,
(u, v)r =
Ir
inf
ξ∈<
x+ξ∈I¯r
Z
u v dx,
Ir
C(ξ) + u(x + ξ)
and u0 ∈ H01 (Ir ) is the solution of the equation
∀v ∈ H01 (Ir ) .
ar (u, v) + γ (u, v)r = (f, v)r
(30)
Under assumptions (4), (5), (6), there is one and only one solution u0 ≥ 0 of
(30) and one and only one solution ur of (29) (see Bensoussan and Lions (1984)).
The solution ur (x) has the probabilistic interpretation
ur (x) = umin (x) = M inJxr (V )
V ∈A
where
Jxr (V
)=E
(∞
X
C(ξi ) e
−γti
χti <∞ +
i=1
Z
τr
e
−γs
f (yx (s))ds
0
)
and
τr = inf t ≥ 0 | yx (t− ) or yx (t) ∈
/ Ir
is the first exit time from the interior of Ir . We denote by ũr the function ur
continued by 0 outside Ir . In the next theorem we show the above mentioned
convergence result of ũr to umin when r → ∞.
Theorem 5 Under assumptions (4), (5), (6), when r → ∞ we have
kũr − umin kC 0 (K)
kũr − umin kL2,µ
→
→
0
∀K ⊂ <, compact
0,
and there exists a subsequence um such that kũm − umin k → 0 weakly in H 1,µ .
Proof. By restricting the integration to Ir , we define
Z
Z
2a2 xµ 0
)u v wµ2 dx
(a1 − √
a2 u0 v 0 wµ2 dx +
aµ,r (u, v) =
2
1
+
x
Ir
I
Zr
2
u v wµ dx .
(u, v)µ,r =
Ir
Let w ∈ H01 (Ir ) satisfy w ≤ Mr ur . As (wµ2 ) = − √2µx
w2 , setting u = ur and
1+x2 µ
2
v = ur + wµ (w − ur ) in the first inequality of (29), we obtain
ar (ur , wµ2 (w − ur )) + γ ur , wµ2 (w − ur ) r =
= aµ,r (ũr , w̃ − ũr ) + γ (ũr , w̃ − ũr )µ,r ≥
≥ (f, wµ2 (w − ur ))r = (f, w̃ − ũr )µ,r .
0
14
Therefore for every w ∈ H01 (Ir ) such that w ≤ Mr ur we have
aµ (ũr , w̃ − ũr ) + γ (ũr , w̃ − ũr )µ ≥ (f, w̃ − ũr )µ .
(31)
We consider now the sequence ũr , with r ∈ N. From the probabilistic interpretation of ur it follows ũ1 ≤ ũ1 ≤ ... ≤ ũr ≤ .. ≤ umin and ũr converges pointwise
to a function u∗ . Setting w̃ = 0 into (31), ∀r ∈ N, and using (8) we obtain in
the same way as in Theorem 3
β ||ũr ||µ ≤ |f |µ + (ρ − γ) |umin |µ
∀r ∈ N .
(32)
Therefore there exists a subsequence ũm which converges pointwise to u∗ , strongly
in L2,µ , weakly in H 1,µ . From Morrey’s theorem the functions ũm are uniformly
Hölder continuous on < and consequently u∗ is continuous and ũm converges
uniformly to u∗ , on every compact subset of <. It has still to be shown that
u∗ = umin . Since u∗ ≤ umin it is sufficient to show that u∗ is a solution of (16).
From ur ≤ Mr ur it follows ũr ≤ Mr ũr and we obtain also that u∗ ≤ M u∗ .
It remains to prove that for every w ∈ H 1,µ such that w ≤ M u∗ it holds
aµ (u∗ , w − u∗ ) + γ (u∗ , w − u∗ )µ ≥ (f, w − u∗ )µ . This is shown in the same way
as in the proof of Theorem 3.
This convergence result allows us to compute umin by solving the truncated
problem (29), for a conveniently chosen interval Ir including the domain of
interest.
The FE method performs a decomposition of Ir into some finite number Nh
of subintervals −r = x0 < x1 < . . . < xNh = r. Let h > 0 be the maximum
length of the intervals [xi , xi+1 ] of the decomposition and Vh (as h → 0) be the
space of all continuous, piecewise linear functions defined on Ir and vanishing
on the boundary. We remark that we shall only consider the piecewise linear
element case due to the limited higher order regularity for the solution ur .
The discrete problem reads
ar (urh , vh − urh ) + γ(urh , vh − urh )r ≥ (f, vh − urh )r
(33)
urh ∈ Vh , urh ≤ πh M urh , ∀vh ∈ Vh , vh ≤ πh M urh ,
where πh is the interpolation operator from C 0 (Ir ) onto Vh . The convergence
analysis of the FE approximation to models of the form (29) can be found in
Loinger (1980), Cortey-Dumont (1980) for the coercive case and Boulbrachene
(1998) for the noncoercive case. In particular, under the additional regularity
assumptions
ur ∈ W 2,p (Ir ), 2 ≤ p < ∞,
Aur ∈ L∞ (Ir )
(34)
the following optimal error estimate holds
kur − urh kC 0 (Ir ) ≤ Ch2 | log h|3 .
(35)
Finally, combining (35) with the convergence result of Theorem 5, we get
the following convergence result to the original problem (16)
15
Corollary 6 For any fixed compact set K, we get
lim ( lim kumin − ũrh kC 0 (K) ) = 0.
r→∞ h→0
Proof. It follows immediately from
kumin − ũrh kC 0 (K) ≤ kumin − ũr kC 0 (K) + kũr − ũrh kC 0 (K) → 0,
(36)
as r → +∞ and h → 0.
Analogously to the continuous case, the FE approximation urh is obtained
by means of the following discrete iterative process starting from urh,0 = 0
ar (urh,n , vh − urh,n ) + γ(urh,n , vh − urh,n )r ≥ (f, vh − urh,n )r
urh,n ∈ Vh , urh,n ≤ πh M urh,n−1 , ∀vh ∈ Vh , vh ≤ πh M urh,n−1 ,
(37)
for n = 1, 2, . . . . The sequence {urh,n } defined by (37) is pointwise non dePNh −1 n
ui ϕi (x), where
creasing and converges to urh in Vh . If we set urh,n (x) = i=1
the ϕi ’s are the so called hat functions generating Vh , we get the following finite
system of linear algebraic inequalities
uTn AT (v − un ) ≥ bT (v − un )
un ≤ ψn−1 ,
v ≤ ψn−1 ,
(38)
where A = (Ai,j ) is the stiffness matrix with Ai,j := ar (ϕj , ϕi ) + γ(ϕj , ϕi )r ,
un = (un1 , un2 , . . . , unNh −1 )T , b = (b1 , b2 , . . . , bNh −1 )T is the vector with bj :=
(f, ϕj )r and ψn−1 is the vector of the nodal values of M urh,n−1 . System (38) is
equivalent to the following constrained system
Aun ≤ b,
un ≤ ψn−1 ,
(ψn−1 − un )T (Aun − b) = 0.
(39)
For the solution of (39), the projected SOR method has been used.
6
Numerical experiments
In this section, we show some numerical results concerning different data specifications8 .
The first set of simulations refers to a case with quadratic holding-penalty
costs. Such kind of problems can be solved analytically as in Baccarin (2002)
and the optimal control takes the form of a control band policy with barriers
d < D < U < up. In particular, d and up are the free boundaries separating
the continuation region {u < M u} from the intervention region {u = M u}.
In Table 1, we list the norms of the approximation error against the analytical
solution for decreasing h in the L2 , H 1 and L∞ -norms. The parameter values
are a2 (x) = 0.5, a1 (x) = 0, k = 1, γ = 0.5, f (x) = x2 , c(x) = |x|. Each
line i (i = 1, 2, 3, 4) of the table refers to a different approximation, where
8 All
the simulations are run in Matlab on a Pentium IV processor.
16
i
1
2
3
4
ri
30
35
40
45
h
0.4
0.2
0.1
0.05
kumin − urhi kL2
0.060604 (0.00)
0.015399 (3.94)
0.002095 (7.35)
0.005108 (0.41)
kumin − urhi kH 1
0.384473 (0.00)
0.185515 (2.07)
0.093810 (1.98)
0.048068 (1.95)
kumin − urhi kL∞
0.038770 (0.00)
0.008477 (4.57)
0.001784 (4.75)
0.002772 (0.64)
Table 1: The columns list the norms in the spaces L2 (−3, 3), H 1 (−3, 3) and the
L∞ norm of the approximation error umin − urhi for decreasing h and increasing
ri . The ratios of error norms on successive grids are in parentheses.
i
1
2
3
4
ri
h
30 0.4
35 0.2
40 0.1
45 0.05
analytical
d
-2.2
-2.0
-2.0
-2.05
-2.0349
D
-0.6
-0.4
-0.5
-0.5
-0.4943
U
0.6
0.4
0.5
0.5
0.4943
up
2.2
2.0
2.0
2.05
2.0349
Table 2: Numerical and analytical barriers of the control band policy.
the computational domain Iri = [−30 − 5(i − 1), 30 + 5(i − 1)] is discretized
by a nonuniform mesh having smaller elements of length h = 0.4/2i−1 in the
core region [−3 − (i − 1), 3 + (i − 1)]. In parentheses, we record the ratios
i
k of errors of the numerical approximations sequence,
kumin − urhi k/kumin − urh/2
obtained by halving the mesh size h on successive grids. The integrals in the
discrete formulation (39) are computed by means of the fourth order GaussLegendre quadrature rule and the error estimates are confined to the inner
region K = [−3, 3]. In order to reduce approximation errors in the computation
i
of the nonlinear obstacle, the nodal values of M urh,n−1
are computed using the
ri
interpolation values of uh,n−1 on five intermediate points for each finite element.
As it can be seen from the table, the convergence rate is affected by the presence
of the term kumin − ũr kC 0 (K) in (36). The evaluation of this truncation error and
the question of how large the computational domain Ir should be conveniently
chosen in terms of a given error tolerance will be the object of future research.
The numerical barriers of the control band policy are compared with the
exact ones in Table 2. The barriers are always mesh points and there is strong
evidence that the computed barriers converge to the corresponding analytical
values. In all our simulations a mesh size h smaller than a given tolerance ε
gives an approximation error smaller than ε for the barriers. Figure 1 shows the
i
} and corresponding obstacles for i = 3.
approximating sequence {urh,n
In the following simulation, we try to mimic a more complex and verisimilar
study case where we assume a variable diffusion coefficient and more realistic
17
8
7
6
5
4
3
2
1
0
−5
−4
−3
−2
−1
0
1
2
3
4
5
i
} (solid line) and corresponding obstaFigure 1: Approximating sequence {urh,n
cles (dotted line) for i = 3 in the quadratic case.
cost functions are considered9 . Hence, we assume
b(x) ≡ 0.05,
2
C(ξ) = 0.01 + c(ξ),
σ(x) = 0.3 − 0.2 e−x ,
2
0.05x − 0.01x e−x , x ≥ 0
f (x) =
2
−0.1x + 0.04x ,
x<0
γ = 0.03,
where
c(ξ) = 0.03|ξ| − 0.01e−|ξ| + 0.01.
The results are displayed in Figure 2. The computed barrier values are d =
−0.22, D = −0.04, U = 0.02, up = 0.28 for a discretization step h = 0.02. Since
there is a small interval near the origin where the transaction costs are larger
than the carrying costs, we observe that there exists a small continuation region
around the cash level x = 0. However, due to the presence of a positive drift, we
notice that this region is slightly asymmetric, although this effect is somehow
compensated by the higher penalty cost rate.
9 For instance, in the cash management problem the carrying cost rate is certainly higher
for negative than for positive cash levels.
18
0.34
0.335
0.33
0.325
u
0.32
0.315
0.31
0.305
0.3
0.295
d = −0.22; D = −0.04; U = 0.02; up = 0.28
0.29
−0.4
−0.3
−0.2
−0.1
0
x
0.1
0.2
0.3
Figure 2: Numerical solution uh (solid line) versus the obstacle M uh (dotted
line) in the non quadratic case.
References
• Baccarin, S. (2002): Optimal impulse control for cash management with
quadratic holding-penalty costs. Decisions in Economics and Finance 25
19-32.
• Bensoussan, A. and Lions, J.L. (1982): Applications of variational inequalities in stohastic control. North Holland.
• Bensoussan, A. and Lions, J.L. (1984): Impulse Control and QuasiVariational Inequalities. Gauthiers-Villars, Paris.
• Boulbrachene (1998): The noncoercive quasi-variational inequalities related to impulse control problems, Comput. Math. Appl., 35 , no. 12,
101–108.
• Brezis, H. (1983): Analyse fonctionnelle. Théorie et applications. Masson
Editeur, Paris.
• Cortey-Dumont, P. (1980): Approximation numerique d’une I.Q.V. liée a
des problemesde gestion de stock. RAIRO, Num. Anal., 14 335-346.
• Eastham, J. and Hastings K. (1988): Optimal impulse control of portfolios. Math. Oper. Res. 4 588-605.
19
• Harrison, J.M., Sellke, T. and Taylor, A. (1983): Impulse control of a
Brownian motion. Math. Oper. Res. 8 454-466.
• Korn, R. (1997): Optimal impulse control when control actions have random consequences. Math. Oper. Res. 22 639-667.
• Korn, R. (1998): Portfolio optimisation with strictly positive transaction
costs and impulse control. Finance and Stochastics 2 85-114.
• Loinger, E. (1980): A finite element approach to a quasivariational inequality. Calcolo, 17 , no. 3, 197–209.
• Richard, S.F. (1977): Optimal impulse control of a diffusion process with
both fixed and proportional costs of control. SIAM J. Control Optimization 15 79-91.
20