Differential games with continuous, switching and
impulse controls
A.J. Shaijua,1 , Sheetal Dharmattib,∗,2
a TIFR Centre, IISc Campus, Bangalore 560012, India
b Department of Mathematics, Indian Institute of Science, Bangalore 560 012, India
Abstract
A two-person zero-sum differential game of infinite duration with discounted payoff involving
hybrid controls is studied. The minimizing player is allowed to take continuous, switching, and
impulse controls whereas the maximizing player is allowed to take continuous and switching controls.
By taking strategies in the sense of Elliott–Kalton, we prove the existence of value and characterize
it as the unique viscosity solution of the associated system of quasi-variational inequalities.
Keywords: Differential games; Viscosity solutions; Strategy; Mixed controls; Value
1. Introduction and preliminaries
The study of differential games with Elliott–Kalton strategies in the viscosity solution
framework is initiated by Evans and Souganidis [2] where both players are allowed to
take continuous controls. Differential games where both players use switching controls are
studied by Yong [3,4]. In [5], differential games involving impulse controls are considered;
one player is using continuous controls whereas the other uses impulse control. In the
final section of [5], the author mentions that by applying the ideas and techniques used in
the previous sections one can study differential games where one player uses continuous,
∗ Corresponding author. Tel.: +91 080 2293 2265; fax: +91 080 2360 0851.
E-mail addresses: shaiju@math.tifrbng.res.in (A.J. Shaiju), sheetal@math.iisc.ernet.in (S. Dharmatti).
1 The financial support to A.J. Shaiju from NBHM is gratefully acknowledged.
2 The financial support to S. Dharmatti from UGC is gratefully acknowledged.
switching, and impulse controls and the other player uses continuous and switching controls.
The uniqueness result for the associated system of quasi-variational inequalities (SQVI for
short) with bilateral constraints is established under suitable nonzero loop switching-cost
condition and cheaper switching condition.
In this paper, we study a two-person zero-sum differential game where the minimizer
(Player 2) uses three types of controls: continuous, switching and impulse. The maximizer
(Player 1) uses continuous and switching controls. We first prove dynamic programming
principle (DPP for short) for this problem. Using this, we prove that the lower and upper
value functions satisfy the associated SQVI in the viscosity sense. Finally, we establish
the existence of the value by proving a uniqueness theorem for SQVI. We obtain our results without any assumption like nonzero loop switching-cost condition and/or cheaper
switching-cost condition on the cost functions. This will be further explained in the concluding section. Hence, we prove the existence of the value under Issacs’-type condition
alone.
The paper is organized as follows. We set up necessary notations and assumptions in the
rest of this section. The statement of the main result is also given at the end of this introductory section. The DPP is proved in Section 2. Here, we also show that the lower/upper
value function is a viscosity solution of SQVI. Section 3 is devoted to the proof of the main
uniqueness result for SQVI and the existence of value. The paper ends in Section 4 with
some concluding remarks.
We first describe the notations and basic assumptions. The state space is a Euclidean space
Rd . The continuous control set for Player i, i = 1, 2, is U i , a compact metric space. The set
i }; i = 1, 2, is the switching control set for Player i. The impulse control
D i = {d1i , . . . , dm
i
set for the Player 2 is K, a compact subset of the state space Rd . The space of all U i -valued
measurable maps on [0, ∞) is continuous control space for Player i and is denoted by Ui .
By Ui [0, t] we mean the space of all U i valued measurable maps on [0, t]. The switching
control space Di and the impulse control space K are defined as follows:


dji −1 [i ,i ) (·) : dji ∈ D i , (ij ) ⊂ [0, ∞], i0 = 0, (ij ) ↑ ∞,
Di = d i (·) =
j −1 j

j 1


dji −1 = dji if ij < ∞ ,



K = (·) =

j 0


j [j ,∞] (·) : j ∈ K, (j ) ⊂ [0, ∞], (j ) ↑ ∞ .

An impulse control (·) = j 0 j [j ,∞] (·), consists of the impulse times j ’s and
impulse vectors j ’s. We use the notation ()1,j to denote j and ()2,j to denote j .
Similarly for switching controls d 1 (·) and d 2 (·) we write,
(d 1 )1,j = 1j
and (d 1 )2,j = dj1 ,
(d 2 )1,j = 2j
and (d 2 )2,j = dj2 .
Now we describe the dynamics and cost functions involved in the game. To this end, let
C1 = U1 × D1 and C2 = U2 × D2 × K. For (u1 (·), d 1 (·)) ∈ C1 and (u2 (·), d 2 (·), (·)) ∈ C2 ,
the corresponding state yx (·) is governed by the following hybrid controlled system:
ẏx (t) = f (yx (t), u1 (t), d 1 (t), u2 (t), d 2 (t)) + ˙ (t),
yx (0−) = x,
(1.1)
where f : Rd × U 1 × D 1 × U 2 × D 2 → Rd .
We assume that
(A1) The function f is bounded, continuous and for all x, y ∈ Rd , d i ∈ D i , ui ∈ U i ,
f (x, u1 , d 1 , u2 , d 2 ) − f (y, u1 , d 1 , u2 , d 2 ) Lx − y.
Note that under assumption (A1), for each x ∈ Rd , d i (·) ∈ Di , ui (·) ∈ Ui , and (·) ∈ K
there is a unique solution yx (·) of (1.1).
Let k : Rd × U 1 × D 1 × U 2 × D 2 → R be the running cost function, ci : D i × D i → R
the switching cost functions, and l : K → R the impulse cost function.
We assume that
(A2) the cost functions k, ci , l are nonnegative, bounded, continuous, and for all x, y ∈ Rd ,
i
d ∈ D i , ui ∈ U i , 0 , 1 ∈ K,
|k(x, u1 , d 1 , u2 , d 2 ) − k(y, u1 , d 1 , u2 , d 2 )| Lx − y,
l(0 + 1 ) < l(0 ) + l(1 ) ∀0 , 1 ∈ K,
inf ci (d1i , d2i ) =: c0i > 0.
d1i =d2i
Let > 0 be the discount parameter. The total discounted cost functional Jx : C1 × C2 →
R is given by
Jx [u1 (·), d 1 (·), u2 (·), d 2 (·), (·)]
=
∞
e−t k(yx (t), u1 (t), d 1 (t), u2 (t), d 2 (t)) dt
1
2
−
e−j c1 (dj1−1 , dj1 ) +
e−j c2 (dj2−1 , dj2 ) +
e−j l(j ).
0
j 0
j 0
j 1
We next define the strategies and value functions for Players 1 and 2 in the Elliott–Kalton
framework. The strategy set for Player 1 is the collection of all nonanticipating maps from C2 to C1 . The strategy set for Player 2 is the collection of all nonanticipating maps
from C1 to C2 .
For a strategy of Player 2 if (u1 (·), d 1 (·)) = (u2 (·), d 2 (·), (·)), then we write 1
(u1 (·), d 1 (·)) = u2 (·), 2 (u1 (·), d 1 (·)) = d 2 (·) and 3 (u1 (·), d 1 (·)) = (·). That is,
i is the projection on the ith component of the map . Similar notations we use for
(u2 (·), d 2 (·), (·)) as well.
i
Let Di,d denote the set of all switching controls for Player i starting at d i . Then we define
1
1
2
2
sets C1,d = U1 × D1,d and C2,d = U2 × D2,d × K. The collection of all ∈ such that
2
d
2
1
2 (0−) = d is denoted by and the collection of all ∈ such that 2 (0−) = d
is denoted by d . Let d0 denote the set of all ∈ d with (
1
1
1
2 )1,1
= ∞, and d0 the
2
set of all ∈ d with ( 2 )1,1 = ( 3 )1,1 = ∞. An impulse control (·) with ()1,1 = ∞
is denoted by ∞ (·).
The upper and lower value functions are defined, respectively, as follows:
2
V+d
1 ,d 2
inf Jxd
(x) = sup
∈
1 2
V−d ,d (x) =
C
d1
inf sup Jxd
∈
1 ,d 2
2,d 2
d2
1 ,d 2
[(u2 (·), d 2 (·), (·)), u2 (·), d 2 (·), (·)],
[u1 (·), d 1 (·), (u1 (·), d 1 (·))],
1,d 1
C
where Jxd ,d is the restriction of the cost functional Jx to C1,d × C2,d .
1 2
1 2
Let V+ = {V+d ,d : (d 1 , d 2 ) ∈ D 1 × D 2 } and V− = {V−d ,d : (d 1 , d 2 ) ∈ D 1 × D 2 }. If
V+ ≡ V− ≡ V , then we say that the differential game has a value and V is referred to as
the value function.
Since all cost functions involved are bounded, value functions are also bounded. In view
of (A1) and (A2), the proof for uniform continuity of V+ and V− is standard. Hence, both
V+ and V− belong to BU C(Rd ; Rm1 ×m2 ).
For x, p ∈ Rd , let
1
2
H−d
1 ,d 2
1
2
(x, p) = max min [−p, f (x, u1 , d 1 , u2 , d 2 ) − k(x, u1 , d 1 , u2 , d 2 )],
u2 ∈U 2 u1 ∈U 1
1 2
H+d ,d (x, p) =
min max [−p, f (x, u1 , d 1 , u2 , d 2 ) − k(x, u1 , d 1 , u2 , d 2 )]
u1 ∈U 1 u2 ∈U 2
and for V ∈ C(Rd ; Rm1 ×m2 ), let
d ,d
[V ](x) = min [V d
M−
1 ,d̄ 2
(x) + c2 (d 2 , d̄ 2 )],
d ,d
[V ](x) = max [V d̄
M+
1 ,d 2
(x) − c1 (d 1 , d̄ 1 )],
1
2
d̄ 2 =d 2
1
2
d̄ 1 =d 1
N[V d
1 ,d 2
1 ,d 2
](x) = min [V d
∈K
(x + ) + l()].
The HJI upper systems of equations associated to the hybrid differential game are as
follows:
min {max(V d
Vd
1 ,d 2
− N [V d
1 ,d 2
1 ,d 2
1 ,d 2
]), V d
1 ,d 2
(x, DV d
1
(x, DV d
d ,d
− M−
[V ], V d
1 ,d 2
− N[V d
1
2
), V d
1 ,d 2
d ,d
− M−
[V ],
1
2
2
1 ,d 2
1 ,d 2
1 ,d 2
d ,d
− M+
[V ]} = 0; (d 1 , d 2 ) ∈ D 1 × D 2 ,
+ H+d
max{min(V d
Vd
+ H+d
1 ,d 2
1 ,d 2
), V d
1 ,d 2
1 ,d 2
(HJI1+)
d ,d
− M+
[V ]),
1
2
]} = 0; (d 1 , d 2 ) ∈ D 1 × D 2 .
(HJI2+)
If we replace H+d ,d in the above system of equations by H−d ,d , then we obtain the HJI
lower system of equations denoted, respectively, by (HJI1−) and (HJI2−). If V satisfies
both (HJI1+) and (HJI2+), then we say that V satisfies (HJI+) and similarly if it satisfies
1
2
1
2
both (HJI1−) and (HJI2−), we say that V satisfies (HJI−). In the next section, we show
that V− satisfies (HJI+) and V+ satisfies (HJI−) in the viscosity sense.
We say that the Isaacs min–max condition holds if H− = H+ , Under this condition, the
equations (HJI1+) and (HJI2+), respectively, coincide with (HJI1−) and (HJI2−). We
now state the main result of this paper; the proof will be worked out in subsequent sections.
Theorem 1.1. Assume (A1), (A2) and the Isaacs min–max condition. Then V− = V+ is the
unique viscosity solution of (HJI+) (or, (HJI−)) in BU C(Rd , Rm1 ×m2 ).
2. Dynamic programming principle
In this section, we first prove the DPP for differential games with hybrid controls. We
first state the results and then the proofs will be given. Throughout this section we assume
(A1) and (A2).
Lemma 2.1. For (x, d 1 , d 2 ) ∈ Rd × D 1 × D 2 and t > 0,
V−d
1 ,d 2
t
(x) = inf sup
∈
d2
0
1,d 1
C
1
e−s k(yx (s), u1 (s), d 1 (s),
(u , d 1 )(s)) ds −
2
(
2
(
3
e−j c1 (dj1−1 , dj1 )
1
e−(
2
)1,j 2
e−(
3
)1,j
c ((
2
)2,j −1 , (
2
)2,j )
)1,j <t
+
(u1 , d 1 )(s),
1j <t
+
1
l((
3
)2,j )
)1,j <t
d 1 (t),
+ e−t V−
2
(t)
(yx (t)) .
(2.1)
Lemma 2.2. For (x, d 1 , d 2 ) ∈ Rd × D 1 × D 2 and t > 0,

V+d
1 ,d 2
(x) = sup
d1
∈
−
(
+

inf 
2
C2,d
t
0
e−(
e−s k(yx (s), (u2 , d 2 , )(s), u2 (s), d 2 (s), (s)) ds
2 )1,j
c1 ((
2 )1,j −1 , (
2 )1, j )
2 )1,j <t
2j <t
e
−2j

c2 (dj2−1 , dj2 )+
j <t
e−j l(j )+e−t V+ 2
(t),d 2 (t)

(yx (t)) .
d ,d
Lemma 2.3. (i) M+
[V− ](x) V−d
1
2
1 ,d 2
(x).
1 2
1 2
d 1 ,d 2
[V− ](x), N[V−d ,d ](x)}.
(ii) V−d ,d (x) min{M−
(iii) Let (x, d 1 , d 2 ) be such that strict inequality holds
2
in (i). Let ¯ ∈ d0 . Then there
exists t0 > 0 such that the following holds:
For each 0 t t0 , there exists u1,t (·) ∈ U1 [0, t] such that
V−d
1 ,d 2
t
(x) − t 2 0
e−s k(yx (s), u1,t (s), d 1 , ¯ (u1,t (·), d 1 )(s) ds
+ e−t V−d
1 ,d 2
(yx (t)).
(iv) Let (x, d 1 , d 2 ) be such that strict inequality holds in (ii). Let ū1 ∈ U 1 . Then there
exists t0 > 0 such that the following holds:
2
For each 0 t t0 , there exists t ∈ d with ( 2 t (ū1 , d 1 ))1,1 , ( 3 t (ū1 , d 1 ))1,1 > t0
such that
V−d
1 ,d 2
t
(x) + t 2 0
e−s k(yx (s), ū1 , d 1 ,
d ,d
Lemma 2.4. (i) M+
[V+ ](x) V+d
1
2
1 ,d 2
t
(ū1 , d 1 )(s)) ds + e−t V−d
1 ,d 2
(yx (t)).
(x).
1 2
1 2
d 1 ,d 2
(ii) V+d ,d (x) min{M−
[V+ ](x), N[V+d ,d ](x)}.
(iii) Let (x, d 1 , d 2 ) be such that strict inequality holds
in (i). Let ¯ ∈ d0 . Then there
1
exists t0 > 0 such that the following holds:
For each 0 t t0 , there exists u2,t (·) ∈ U2 [0, t] such that
V+d
1 ,d 2
t
(x) + t 2 0
e−s k(yx (s), u2,t (s), d 2 , ¯ (u2,t , d 2 , ∞ )(s)) ds
+ e−t V+d
1 ,d 2
(yx (t)).
(iv) Let (x, d 1 , d 2 ) be such that strict inequality holds in (ii). Let ū2 ∈ U 2 . Then there
exists t0 > 0 such that the following holds:
1
For each 0 t t0 , there exists t ∈ d with ( 2 t (ū2 , d 2 , ∞ ))1,1 > t0 such that
V+d
1 ,d 2
t
(x)−t 2 0
e−s k(yx (s), ū2 , d 2 , t (ū2 , d 2 , ∞ )(s)) ds+e−t V+d
1 ,d 2
(yx (t)).
We prove Lemmas 2.1 and 2.3 and the proofs of Lemmas 2.2 and 2.4 are analogous.
Proof of Lemma 2.1. Let (x, d 1 , d 2 ) ∈ Rd × D 1 × D 2 and t > 0. Let us denote the RHS
of (2.1) by W (x). Fix > 0.
2
Let ¯ ∈ d be such that

t

W (x) sup 
0
1,d 1
C
−
e−s k(yx (s), u1 (s), d 1 (s),
1
1j <t
(
+
(
3
e−j c1 (dj1−1 , dj1 )+
e−(
3
¯)
1,j
l((
2
3
1
¯ (u1 , d 1 )(s),
e−(
2
¯)
1,j
c2 ((
2
2
¯ (u1 , d 1 )(s)) ds
¯ )2,j −1 , (
¯)
1,j <t
¯ )2,j ) + e−t V
¯)
2
−
¯ )(t)

(yx (t)) − .
1,j <t
1
d 1 (t),(
2
¯ )(t)
d 1 (t),((
(yx (t)) Jyx (t)
2
¯ )(t)
[u1 (·), d 1 (·),
u1 (·),d 1 (·)
u1 (·),d 1 (·) (u
1
∈ (
2
(u (·), d (·))(s) =
1
1
¯ (u1 (·), d 1 (·))(s),
s t,
1 (· + t), d 1 (· + t))(s − t), s > t.
(u
u1 (·),d 1 (·)
By change of variables, we get
d 1 (t),(
Jyx (t)
=
2
¯ )(t)
[u1 (· + t), d 1 (· + t),
u1 (·),d 1 (·) (u
∞
e− k(yx (), u1 (), d 1 (), (
1
−
e−j c1 (dj1−1 , dj1 )
t
1
1
(· + t), d 1 (· + t))]
)(u1 , d 1 )(), (
2
1j >t
+
(
2
(
3
2
)1,j 2
e−(
3
)1,j
c ((
2
)2,j −1 , (
2
)2,j )
)1,j >t
+
e−(
l((
3
)2,j ).
)1,j >t
Therefore
d 1 (t),(
V−
2
¯ )(t)
(yx (t)) (∗) − ,
where (*) denotes the RHS of the above equation. This implies that
W (x) Jxd
1 ,d 2
[u1 (·), d 1 (·), (u1 , d 1 )(·)] − 2 .
2
¯ )(t)
(·), d 1 (·))] − .
∈ d by
Define
¯ )2,j )

d 1 (t),(
By the definition of V− , for each (u1 (·), d 1 (·)) ∈ C1,d , there exists
such that
V−
2
)(u1 , d 1 )()) d
1
This holds for all (u1 (·), d 1 (·)) ∈ C1,d and hence
1 2
W (x) V−d ,d (x) − 2 .
Since > 0 is arbitrary, we get W (x) V−d
1 ,d 2
(x).
∈ d and > 0. Choose (ū1 (·), d̄ 1 (·)) ∈
2
We now prove the other way inequality. Fix
1
C1,d such that
W (x) t
e−s k(yx (s), ū1 (s), d̄ 1 (s), (ū1 , d̄ 1 )(s)) ds
0
1
−
e−j c1 (dj1−1 , dj1 )
1j <t
+
(
2
(
3
2
)1,j 2
e−(
3
)1,j
c ((
2
)2,j −1 , (
2
)2,j )
)1,j <t
+
e−(
l((
)2,j )
3
)1,j <t
d̄ 1 (t),(
+ e−t V−
)(t)
2
(yx (t)) + .
(2.2)
Now for each u1 (·), define ũ1 (·) by
1
s t,
ū (s),
ũ1 (s) =
u1 (s − t), s > t.
Similarly, for each d 1 (·), we define d̃ 1 (·). Let
ˆ (u1 (·), d 1 (·))(s) = (ũ1 (·), d̃ 1 (·))(s + t).
By the definition of V− , we can choose (u1 (·), d 1 (·)) ∈ C1,d̄
d̄ 1 (t),(
V−
)(t)
2
d̄ 1 (t),(
Jyx (t)
1 (t)
such that
(yx (t))
2
)(t)
[u1 (· + t), d 1 (· + t), ˆ (u1 , d 1 )(· + t))] + et .
(2.3)
Now, combining (2.2) and (2.3), we get
t
W (x) 0
(
2
e−j c1 (dj1−1 , dj1 )
1
1j <t
+
+
e−s k(yx (s), ū1 (s), d̄ 1 (s), (ū1 , d̄ 1 )(s)) ds −
e−
2 1,j
c2 ((
2
)2,j −1 , (
2
)2,j )
)1,j <t
e−(
3
3 1,j <t
)1,j
l((
3
d̄ 1 (t),(
)2,j )+e−t Jyx (t)
2
)(t)
By change of variables, it follows that
W (x) Jxd
1 ,d 2
[ũ1 (·), d̃ 1 (·), (ũ1 , d̃ 1 )(·))] + 2 .
[u1 (·), d 1 (·), ˆ (u1 (·), d 1 (·))]+2 .
∈ d and hence
2
This holds for any
W (x) V−d
1 ,d 2
(x) + 2 .
The proof is now complete, since is arbitrary.
Proof of Lemma 2.3. We first prove (i) and (ii). By the definition of V− , for any d̄ 2 = d 2
V−d
1 ,d 2
(x) V−d
1 ,d̄ 2
(x) + c2 (d 2 , d̄ 2 ).
From this we get
V−d
1 ,d 2
d ,d
[V− ](x).
(x) M−
1
2
The inequality
V−d
1 ,d 2
d ,d
(x) M+
[V− ](x)
2
1
can be proved in a similar fashion.
2
Clearly for any ∈ K and ∈ d
V−d
1 ,d 2
d ,d
1
1
1 1
(x) sup Jx+
[u (·), d (·), (u , d )(·)] + l().
1
C
∈ d and then over ∈ K, we obtain
2
First take infimum over
V−d
2
1,d 1
1 ,d 2
(x) N[V−d
1 ,d 2
](x).
We now turn to the proof of (iii). By Lemma 2.1, for each t 0, there exists (u1,t (·), d 1,t
1
(·)) ∈ C1,d such that
V−d
1 ,d 2
t
(x) − t 2 0
−
e−s k(yx (s), u1,t (s), d 1,t (s), ¯ (u1,t , d 1,t )(s)) ds
1,t
1,t
−t
e−j c1 (dj1,t
V−
−1 , dj ) + e
d 1,t (t),(
2
¯ )(t)
(yx (t)).
1,t
j <t
It is enough to show that, for some t0 > 0, 1,t
1 t for all 0 t t0 . If this does not happen,
n
then there would exist a sequence tn ↓ 0 such that 1,t
1 < tn for all n. This would imply that
1 2
n 2
V−d ,d (x) − (1,t
1 ) n
1,t
1
0
e−s k(yx (s), u1,tn (s), d 1,tn (s), ¯ (u1,tn , d 1,tn )(s)) ds
1,tn
1,tn
d 1,tn ,d 2
− e−1 c1 (d 1 , d11,tn ) + e−1 V−1
n
(yx (1,t
1 )).
We may assume that for all n, d11,tn = d̄ 1 = d 1 . Now by letting n → ∞ in the above
inequality, we get
V−d
1 ,d 2
(x) − c1 (d 1 , d̄ 1 ) + V−d̄
1 2
M+d ,d [V− ](x).
1 ,d 2
(x)
This contradicts the hypothesis that strict inequality holds in (i) and the proof of (iii) is
now complete.
2
We next prove (iv). By Lemma 2.1, for each t > 0, there exists t ∈ d such that
V−d
1 ,d 2
t
(x) + t 2 e−s k(yx (s), ū1 , d 1 ,
0
d 1 ,(
+ e−t V−
+
(
2
+
(
3
t
e
)(t)
(ū1 , d 1 )(s)) ds
(yx (t))
−2,t
j
2,t
c2 (dj2,t
−1 , dj )
)1,j <t
t
t
2
t
t
e−j l(tj ).
)1,j <t
t
It is enough to show that, for some t0 > 0, min(2,t
1 , 1 ) t for all 0 t t0 . If this
were not true, then (without any loss of generality) there would be a sequence tn ↓ 0
tn
n
and two cases to consider. In the first case, 2,t
1 min(tn , 1 ) whereas in the second case
n
t1n min(tn , 2,t
1 ). By dropping to a subsequence if necessary and proceeding as in the
d ,d
proof of (iii), we get V−d ,d (x) M−
[V− ](x) and V−d ,d (x) N [V−d ,d ](x) in case 1
and case 2 respectively. This contradicts our hypothesis that strict inequality holds in (ii)
and the proof is now complete. 1
2
1
2
1
2
1
2
Remark 2.5. From the proofs it is clear that, instead of the term t 2 in the statement of
Lemmas 2.3 and 2.4, we can take any modulus (t).
By properly modifying the arguments in the proof of Lemma 1.11, Chapter VIII in [1],
we can prove the next lemma. We omit the details.
Lemma 2.6. Let > 0, ∈ C 1 (Rd ) and (x, d 1 , d 2 ) ∈ Rd × D 1 × D 2 be such that
(x) + H+d
1 ,d 2
(x, D (x)) = .
1
2
Then there exists ¯ ∈ d0 such that for all (u1 (·), d 1 (·)) ∈ C1,d and t small enough,
t
0
e−s [(yx (s)) − D (yx (s)), f (yx (s), u1 (s), d 1 (s), ¯ (u1 , d 1 )(s))
− k(yx (s), u1 (s), d 1 (s), ¯ (u1 , d 1 )(s))] ds t.
2
We are now ready to prove the fact that V− (resp., V+ ) is a viscosity solution of (HJI+)
(resp., (HJI−)).
Theorem 2.7. The lower value function V− is a viscosity solution of (HJI+) and the upper
value function V+ is a viscosity solution of (HJI−).
Proof. We prove that V− is a viscosity solution of (HJI+). The other part can be proved in
an analogous manner.
We first prove that V− is a subsolution of (HJI1+). Let (x, d 1 , d 2 ) ∈ Rd × D 1 × D 2
1 2
and ∈ C 1 (Rd ) be such that V−d ,d − has a local maximum at x. Without any loss of
generality, we may assume that V−d
are done. Assume that V−d
(x) + H+d
1 ,d 2
1 ,d 2
1 ,d 2
(x) = (x). If V−d
1 ,d 2
d ,d
(x) = M+
[V− ](x), then we
1
2
d ,d
(x) > M+
[V− ](x). It suffices to show that
1
2
(x, D (x)) =: r 0.
If possible, let r > 0. By Lemma 2.6, there exists ¯ ∈ d0 such that for all (u1 (·), d 1 (·)) ∈
1
C1,d and t small enough
2
r
t
2
t
0
e−s [(yx (s)) − D (yx (s)), f (yx (s), u1 (s), d 1 (s), ¯ (u1 , d 1 )(s))
− k(yx (s), u1 (s), d 1 (s), ¯ (u1 , d 1 )(s))] ds
= (x) − e−t (yx (t)) −
V−d
1 ,d 2
t
−
0
(x) − e−t V−d
1 ,d 2
t
0
e−s k(yx (s), u1 (s), d 1 (s), ¯ (u1 , d 1 )(s))] ds
(yx (t))
e−s k(yx (s), u1 (s), d 1 (s), ¯ (u1 , d 1 )(s))] ds.
By Lemma 2.3(iii), for t small enough, there exits u1,t (·) ∈ U1 [0, t] with
V−d
1 ,d 2
t
(x) − t 2 0
e−s k(yx (s), u1,t (s), d 1 , ¯ (u1,t (·), d 1 )(s) ds+e−t V−d
1 ,d 2
(yx (t)).
Therefore, for t small enough, we obtain 2r t t 2 . This contradiction proves the fact that
V− is a subsolution of (HJI1+).
1 2
To prove that V− is a supersolution of (HJI1+), let x be a local minimum of V−d ,d − .
Without any loss of generality, we may assume that V−d
d 1 ,d 2
d 1 ,d 2
1 ,d 2
d 1 ,d 2
(x) = (x). If V−d
[V− ](x) or V− (x)=N [V− ](x), then we are done.Assume that V−
1 2
d 1 ,d 2
(M− [V− ](x), N[V−d ,d ](x)). In this case, we need to show that
M−
(x) + H+d
1 ,d 2
1 ,d 2
d 1 ,d 2
(x) =
(x) < min
(x, D (x)) =: r̂ 0.
If possible, let r̂ < 0. Then
(x) − D (x), f (x, ū1 , d 1 , u2 , d 2 ) − k((x, ū1 , d 1 , u2 , d 2 ) for some ū1 ∈ U 1 and all u2 ∈ U 2 . This implies that, for all
r̂
2
∈ d0 and s small enough
(yx (s)) − D (yx (s)), f (yx (s), ū1 , d 1 , (ū1 , d 1 )(s))
r̂
− k(yx (s), ū1 , d 1 , (ū1 , d 1 )(s)) .
4
2
Multiplying throughout by e−s and integrating from 0 to t, we get
r̂
t (x) − e−t (yx (t)) −
4
t
0
e−s k(yx (s), ū1 , d 1 , (ū1 , d 1 )(s)) ds
1 2
1 2
V−d ,d (x) − e−t V−d ,d (yx (t)) −
t
0
e−s k(yx (s), ū1 , d 1 , (ū1 , d 1 )(s)) ds.
Now by Lemma 2.3(iv), for t small enough, there exists
V−d
1 ,d 2
t
(x) + t 2 0
e−s k(yx (s), ū1 , d 1 ,
t
t
∈ d0 such that
2
(ū1 , d 1 )(s)) ds + e−t V−d
1 ,d 2
(yx (t)).
Therefore, for t small, we obtain −t 2 4r̂ t. This is a contradiction and proves the fact that
V− is a supersolution of (HJI1+). In a similar fashion, we can show that V− is a viscosity
solution of (HJI2+). Hence V− is a viscosity solution of (HJI+). 3. Proof of the main result
In this section we prove the main theorem of the paper, namely uniqueness. First, we
state and prove two lemmas needed in the proof of uniqueness. The proof of first lemma is
same as in [5], but we give the proof for the sake of completeness.
Lemma 3.1. Assume (A2). Let w be uniformly continuous.
wd
If
1 ,d 2
(y0 ) = N[w d
1 ,d 2
](y0 ) = w d
1 ,d 2
(y0 + 0 ) + l(0 ),
then there exists > 0 (which depends only on w) such that for all y ∈ B̄(y0 + 0 , ),
wd
1 ,d 2
(y) < N [w d
1 ,d 2
](y).
Proof. Let
wd
1 ,d 2
(y0 ) = N [w d
1 ,d 2
](y0 ) = w d
1 ,d 2
(y0 + 0 ) + l(0 ).
Then, for every 1 ∈ K
wd
1 ,d 2
(y0 + 0 + 1 ) + l(1 ) − w d
d 1 ,d 2
1 ,d 2
(y0 + 0 )
(y0 + 0 + 1 ) + l(1 ) − w d
=w
− l(0 + 1 ) + l(0 ) + l(1 ).
1 ,d 2
(y0 ) + l(0 )
Hence
N [wd
1 ,d 2
(y0 + 0 )] − w d
1 ,d 2
(y0 + 0 ) inf [l(0 ) + l(1 ) − l(0 + 1 )]
1
= l¯ > 0.
By using uniform continuity of w d
B̄(y0 + 0 , ),
wd
1 ,d 2
(y) < N [w d
1 ,d 2
1 ,d 2
and N [w d
1 ,d 2
] we get a such that for all y ∈
](y).
Lemma 3.2. Assume (A1) and (A2).
(i) Any supersolution w of (HJI1+) satisfies w d
(ii) Any subsolution w of (HJI2+) satisfies wd
d 1, d 2.
1 ,d 2
1 ,d 2
M+d ,d [w] for all d 1 , d 2 .
1
2
min(M−d ,d [w], N [wd
1
2
1 ,d 2
]) for all
Proof. Let w be a supersolution of (HJI1+). If possible, let
wd
1 ,d 2
d ,d
(x0 ) < M+
[w](x0 ).
1
2
By continuity, the above holds for all x in an open ball B around x0 . By Lemma 1.8(d),
1 2
p. 30 in [1], there exists y0 ∈ B and a smooth map such that wd ,d − has local minimum
at y0 . Since w is a supersolution of (HJI1+), this will lead to a contradiction
wd
1 ,d 2
d ,d
(y0 ) M+
[w](y0 ).
1
2
This proves (i). The proof of (ii) is similar.
Next is the proof of uniqueness theorem.
Theorem 3.3. Assume (A1) and (A2). Let v and w ∈ BU C(Rd ; Rm1 ×m2 ) be viscosity
solutions of (HJI+) (or (HJI−)). Then, v = w.
Proof. We prove the uniqueness for (HJI+). The result for (HJI−) is similar.
1 2
1 2
Let v and w be viscosity solutions of (HJI+). We prove v d ,d w d ,d for all d 1 , d 2 .
1
2
1
2
In a similar fashion we can prove that wd ,d v d ,d for all d 1 , d 2 .
1 2
For (d 1 , d 2 ) ∈ D 1 × D 2 , define d ,d : Rd × Rd → R by
d
1 ,d 2
(x, y) = v d
1 ,d 2
(x) − w d
1 ,d 2
(y) −
|x − y|2
− [xm̄ + ym̄ ],
2
where m̄ ∈ (0, 1)∩(0, f∞ ) is fixed, , ∈ (0, 1) are parameters, and xm̄ =(1+x2 )m̄/2 .
Note that x → xm̄ is Lipschitz continuous with Lipschitz constant 1.
Let M0 = sup∈K ||. We first fix > 0. Let (x , y ) and (d 1 , d 2 ) be such that
d
1 ,d 2
(x , y ) = sup max d
x,y
d 1 ,d 2
1 ,d 2
(x, y).
By Lemma 2.3, we have
wd
1 ,d 2
(y ) N [w d
1 ,d 2
](y ),
(3.1)
wd
vd
d 1 ,d 2
1 ,d 2
1 ,d 2
(y ) M−
d 1 ,d 2
(x ) M+
[w](y ),
(3.2)
[v](x ).
(3.3)
If we have strict inequality in all the above three equations (that is, 3.1–3.3), then by the
definition of viscosity sub and super solution we will have
x −y
d 1 ,d 2
d 1 ,d 2
m̄−2
x,
v
(x ) + H+
+ m̄x x 0,
(3.4)
w
d 1 ,d 2
(y
d 1 ,d 2
) + H+
y,
x −y
− m̄y m̄−2
0.
y
(3.5)
In this case we can proceed by the usual comparison principle method. Thus we first show
that for some auxiliary function at the maximum point strict inequality occurs in (3.1), (3.2)
and (3.3). We deal these cases one by one. For, let there be equality in (3.1). i.e.,
wd
1 ,d 2
(y ) = N [w d
1 ,d 2
](y ).
By the definition of N, since varies in the compact set, let be such that
wd
1 ,d 2
(y ) = N [w d
1 ,d 2
](y ) = w d
1 ,d 2
(y + ) + l( ).
Then,
d
1 ,d 2
(x + , y + ) = v d
1 ,d 2
(x + ) − w d
1 ,d 2
(y + ) −
− [x + m̄ + y + m̄ ]
|x − y |2
2
|x − y |2
2
− [x m̄ + y m̄ ] − 2| |
vd
1 ,d 2
= d
(x ) − w d
1 ,d 2
(y ) −
1 ,d 2
(x , y ) − 2| |
d 1 ,d 2
(x , y ) − 2M0 .
Hence we have
d
1 ,d 2
(x , y ) − d
1 ,d 2
(x + , y + ) 2M0 .
(3.6)
We will be using this difference to define the new auxiliary function.
Observe that |x |, |y |, | | are bounded hence without any loss of generality we may
assume that
(x , y ) → (x0 , y0 )
and
→ 0 .
We define the new auxiliary function by
d 1 ,d 2
d 1 ,d 2
(x, y) = x − x0 − 0 y − y0 − 0
(x, y) + 2M0 ,
,
where = (w) is the constant coming from Lemma 3.1 and : Rd × Rd → R is a smooth
function with the following properties:
supp() ⊂ B((0, 0), 1),
0 1,
(0, 0) = 1 and < 1 if (x, y) = (0, 0),
|D | 1.
1.
2.
3.
4.
Now by the definition of d
1 ,d 2
(x0 + 0 , y0 + 0 ) = d
d
1 ,d 2
(x, y) = d
1 ,d 2
(x0 + 0 , y0 + 0 ) + 2M0 d
1 ,d 2
(x0 , y0 )
and
1 ,d 2
(x, y) if |x − x0 |2 + |y − y0 |2 2 .
Hence d ,d attains its maximum in the ball around (x0 + 0 , y0 + 0 ) at say (x̂ , ŷ ).
By Lemma 3.1 we now know that
1
wd
1 ,d 2
2
(ŷ ) < N[w d
1 ,d 2
](ŷ ).
If strict inequality holds in (3.2) at ŷ and in (3.3) at x̂ then wd ,d and v d
(3.5) and (3.4), respectively. In this case we can proceed by usual method.
If not let there be equality in (3.2). That is
1
wd
1 ,d 2
d 1 ,d 2
(ŷ ) = M−
d 1 ,d 2
1 ,d 2
(ŷ ) = w d
1 ,d 2
1
1 ,d 2
(x̂ ) v d
1 ,d 2
1
(ŷ ) + c2 (d 2 , d 21 ).
(3.7)
(x̂ ) + c2 (d 2 , d 21 ).
Hence
d
1 ,d 2
(x̂ , ŷ ) v d
1 ,d 2
1
(x̂ ) + c2 (d 2 , d 21 ) − w d
1 ,d 2
1
(ŷ ) − c2 (d 2 , d 21 )
|x̂ − ŷ |2
− [x̂ m̄ + ŷ m̄ ] + 2M0 (x̂ , ŷ )
2
1 2
d ,d 1 (x̂ , ŷ ).
−
But by the definition of (d 1 , d 2 ),
d
1 ,d 2
(x̂ , ŷ ) d
1 ,d 2
1
1 ,d 2
(x̂ , ŷ ) = d
(x̂ , ŷ ).
Hence
d
satisfy
, there exists d 21 ∈ D 2 such that
We know that
vd
1 ,d 2
[w](ŷ ).
By the definition of M−
wd
2
1 ,d 2
1
(x̂ , ŷ ).
d 1 ,d 2
Now if the strict inequality holds in w d ,d 1 (ŷ ) M− 1 [w](ŷ ) we proceed to check
if strict inequality holds in (3.3) else we repeat the above argument and get d 22 ∈ D 2 such
that,
1
wd
1 ,d 2
1
wd
(ŷ ) = w d
1 ,d 2
2
1 ,d 2
2
(ŷ ) + c2 (d 21 , d 22 ),
1 ,d 2
1
(ŷ ) − c2 (d 21 , d 22 )
(ŷ ) = w d
w
2
d 1 ,d 2
1
wd
1 ,d 2
(ŷ ) − c02
(ŷ ) − 2c02
proceeding in similar fashion after finitely many steps, boundedness of w will be contradicted and thus for some d 2k ∈ D 2 strict inequality will hold at ŷ in (3.2).
Now if strict inequality holds at x̂ in (3.3), we can proceed by usual comparison method
else let us assume that equality holds in (3.3).
As before then we have that
v
d 1 ,d 2
k
d 1 ,d 2
(x̂ ) = M+
k
[v](x̂ )
and by the definition of M+ , there exists d 11 such that
v
d 1 ,d 2
k
(x̂ ) = v
d 1 ,d 2
(x̂ ) − c1 (d 11 , d 1 ).
k
1
(3.8)
This case can be handled exactly similarly to the case
wd
1 ,d 2
d 1 ,d 2
(ŷ ) = M−
[w](ŷ .).
Thus for some d 1 j ∈ D 1 strict inequality will hold at x̂ in (3.3).
By Lemma 2.3 we have
v
d 1 ,d 2
k
(ŷ ) v
d 1 ,d 2
(ŷ ) − c1 (d 11 , d 1 ).
k
1
Hence
d
1 ,d 2
k
(x̂ , ŷ ) v
d 1 ,d 2
k
1
(x̂ ) − c1 (d 11 , d 1 ) − w
d 1 ,d 2
1
k
(ŷ ) + c1 (d 11 , d 1 )
|x̂ − ŷ |2
− [x̂ m̄ + ŷ m̄ ] + 2M0 (x̂ , ŷ )
2
1
2
d 1 ,d k (x̂ , ŷ ).
−
Hence
d
1 ,d 2
k
d 1 ,d 2
(x̂ , ŷ ) = 1
k
(x̂ , ŷ ).
d 1 ,d 2
d 1 ,d 2
Now if the strict inequality holds in, v 1 k (x̂ ) M+ 1 k [v](x̂ ) then we are done. If
there is equality, then we repeat the above argument by replacing index by 1 and get an
d 12 ∈ D 1 such that,
v
d 1 ,d 2
1
k
(x̂ ) = v
d 1 ,d 2
2
k
(x̂ ) + c1 (d 11 , d 12 ).
Since c1 (d 1 , d 11 ), c1 (d 11 , d 12 ) > 0, the possibility 1 =
some d 1j ∈ D 1 strict inequality will hold at x̂ in (3.3).
is ruled out by (3.8). Thus, for
d 1 ,d 2
Thus we have at (x̂ , ŷ ), the maximizer of the auxiliary function holds in all the three inequalities (3.1)–(3.3).
Now we define test functions 1 and 2 as follows:
j
k
strict inequality
|x − ŷ |2
(ŷ ) +
+ [xm̄ + ŷ m̄ ]
2
x − x0 − 0 ŷ − y0 − 0
,
,
− 2M0 1 (x) = w
d 1 ,d 2
j
k
|x̂ − y|2
(x̂ ) −
− [x̂ m̄ + ym̄ ]
2
x̂ − x0 − 0 y − y0 − 0
.
,
+ 2M0 2 (y) = v
d 1 ,d 2
j
k
Observe that
D 1 (x̂ ) =
D 2 (ŷ ) =
Note that v
at ŷ . Hence
[v
x̂ − ŷ
d 1 ,d 2
j
d 1 ,d 2
j
x̂ − ŷ
k
k
+ m̄x̂ m−2 x̂ −
x̂ − x0 − 0 ŷ − y0 − 0
2 M 0
,
D
,
− m̄ŷ m−2 ŷ +
x̂ − x0 − 0 ŷ − y0 − 0
2 M 0
.
D
,
− 1 attains its maximum at x̂ and w
(x̂ ) − w
d 1 ,d 2
j
k
d 1 ,d 2
(ŷ )] H+ j
k
d 1 ,d 2
j
k
− 2 attains its minimum
d 1 ,d 2
(ŷ , D 2 (ŷ )) − H+ j
x̂ − ŷ L|x̂ − ŷ | 1 + k
(x̂ , D 1 (x̂ ))
+ f ∞ m̄(x̂ m̄−1 + ŷ m̄−1 ) +
4M0
.
Note that we have used |ŷ | (1 + |ŷ |2 )1/2 = ŷ to get the above inequality. Now as
m̄ − 1 < 0 it follows that
v
d 1 ,d 2
j
k
(x̂ ) − w
d 1 ,d 2
j
k
L
L |x̂ − ŷ |2
2f ∞ m̄ 4f ∞ M0
|x̂ − ŷ | +
+
+
2f ∞ m̄ 4f ∞ M0
+
+ o(1); as ↓ 0.
(ŷ ) For any x ∈ Rd and (d 1 , d 2 ) ∈ D 1 × D 2 ,
vd
1 ,d 2
(x) − w d
1 ,d 2
(x) − 2xm̄ d
1 ,d 2
(x, x)
d 1 ,d 2
j
k
(x̂ , ŷ )
d 1 ,d 2
j
k
d 1 ,d 2
v
(x̂ ) − w j k (ŷ ) + 2M0
2f ∞ m̄ 4f ∞ M0
+
+ 2M0 + o(1); as ↓ 0.
By letting ↓ 0 and then ↓ 0, we get
vd
1 ,d 2
(x) − w d
1 ,d 2
(x) 0.
This completes the proof of uniqueness for (HJI+).
Under the Isaacs min–max condition, (HJI−) and (HJI+) coincide. Hence, the main
result (Theorem 1.1) follows from Theorems 2.7 and 3.3.
4. Conclusions
We have studied two-person zero-sum differential games with hybrid controls. The minimizing player uses continuous, switching, and impulse controls whereas the maximizing
player uses continuous and switching controls. The DPP for lower and upper value functions is proved and using this we have established the existence and uniqueness of the value
under Isaacs min–max condition.
Similar result had been obtained by Yong [5] under two additional assumptions:
(Y1) cheaper switching cost condition
min c2 (d 2 , d̄ 2 ) =: c02 < l0 = inf l(),
∈K
d̄ 2 =d 2
(Y2) nonzero loop switching cost condition.
j
For any loop {(di1 , di2 )}i=1 ⊂ D 1 × D 2 , with the property that
j m 1 m2 ,
dj1+1 = d11 ,
1
either di+1
= di1 ,
or
dj2+1 = d12 ;
2
di+1
= di2
∀1 i j ,
it holds that
j
i=1
1
c1 (di1 , di+1
)−
j
i=1
2
c2 (di2 , di+1
) = 0.
Hence, we prove the existence and uniqueness of value without the above two conditions
(Y1) and (Y2). Moreover, we state and prove the explicit formulation and proof of DPP for
hybrid differential games which is not done in [5].
Acknowledgements
The authors wish to thank M.K. Ghosh for suggesting the problem and for several useful
discussions. They also thank Mythily Ramaswamy for carefully reading the manuscript and
suggesting several modifications.
References
[1] M. Bardi, I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solutions of Hamilton–Jacobi–Bellman
Equations, Birkhauser, Basel, 1997.
[2] L.C. Evans, P.E. Souganidis, Differential games and representation formulas for Hamilton–Jacobi equations,
Indiana Univ. Math. J. 33 (1984) 773–797.
[3] J. Yong, Differential games with switching strategies, J. Math. Anal. Appl. 145 (1990) 455–469.
[4] J. Yong, A zero-sum differential game in a finite duration with switching strategies, SIAM J. Control Optim.
28 (1990) 1234–1250.
[5] J. Yong, Zero-sum differential games involving impulse controls, Appl. Math. Optim. 29 (1994) 243–261.