Electron. Commun. Probab. 17 (2012), no. 59, 1–8.
DOI: 10.1214/ECP.v17-2318
ISSN: 1083-589X
ELECTRONIC
COMMUNICATIONS
in PROBABILITY
Large deviation exponential inequalities
for supermartingales
Xiequan Fan∗
Ion Grama∗
Quansheng Liu∗
Abstract
Pk
Let (Xi , Fi )i≥1 be a sequence of supermartingale differences and let Sk =
i=1 Xi .
We give an exponential moment condition under which P(max1≤k≤n Sk ≥ n) =
O(exp{−C1 nα }), n → ∞, where α ∈ (0, 1) is given and C1 > 0 is a constant. We
also show that the power α is optimal under the given moment condition.
Keywords: Large deviation; martingales; exponential inequality; Bernstein type inequality.
AMS MSC 2010: 60F10; 60G42; 60E15.
Submitted to ECP on September 17, 2012, final version accepted on December 9, 2012.
1
Introduction
Pk
Let (Xi , Fi )i≥1 be a sequence of martingale differences and let Sk = i=1 Xi , k ≥ 1.
Under the Cramér condition supi Ee|Xi | < ∞, Lesigne and Volný [9] proved that
P(Sn ≥ n)
1
= O(exp{−C1 n 3 }),
n → ∞,
(1.1)
for some constant C1 > 0. Here and throughout the paper, for two functions f and g ,
we write f (n) = O(g(n)) if there exists a constant C > 0 such that |f (n)| ≤ C|g(n)| for
all n ≥ 1. Lesigne and Volný [9] also showed that the power 13 in (1.1) is optimal even
for stationary and ergodic sequence of martingale differences, in the sense that there
exists a stationary and ergodic sequence of martingale differences (Xi , Fi )i≥1 such that
1
Ee|X1 | < ∞ and P(Sn ≥ n) ≥ exp{−C2 n 3 } for some constant C2 > 0 and infinitely many
n’s. Liu and Watbled [10] proved that the power 31 in (1.1) can be improved to 1 under
the conditional Cramér condition supi E(e|Xi | |Fi−1 ) ≤ C3 , for some constant C3 . It is
natural to ask under what condition
P(Sn ≥ n)
= O(exp{−C1 nα }),
n → ∞,
(1.2)
where α ∈ (0, 1) is given and C1 > 0 is a constant. In this paper, we give some sufficient
conditions in order that (1.2) holds for supermartingales (Sk , Fk )k≥1 .
The paper is organized as follows. In Section 2, we present the main results. In
Sections 3-5, we give the proofs of the main results.
∗ Université de Bretagne-Sud, LMBA, UMR CNRS 6205, Vannes, France.
E-mail: fanxiequan@hotmail.com E-mail: ion.grama,quansheng.liu@univ-ubs.fr
Exponential inequalities for supermartingales
2
Main Results
Our first result is an extension of the bound (1.1) of Lesigne and Volný [9].
Theorem 2.1. Let α ∈ (0, 1). Assume that (Xi , Fi )i≥1 is a sequence of supermartingale
2α
differences satisfying supi E exp{|Xi | 1−α } ≤ C1 for some constant C1 ∈ (0, ∞). Then, for
all x > 0,
max Sk ≥ nx
P
1≤k≤n
x 2α α
n
,
≤ C(α, x) exp −
4
(2.1)
where
C(α, x) = 2 + 35C1
1
1
+ 2
x2α 161−α
x
3(1 − α)
2α
!
1−α
α
does not depend on n. In particular, with x = 1, it holds
P
max Sk ≥ n
1≤k≤n
1
= O exp{− nα } ,
16
n → ∞.
(2.2)
Moreover, the power α in (2.2) is optimal in the class of martingale differences: for
each α ∈ (0, 1), there exists a sequence of martingale differences (Xi , Fi )i≥1 satisfying
2α
supi E exp{|Xi | 1−α } < ∞ and
P max Sk ≥ n
1≤k≤n
≥ exp{−3nα },
(2.3)
for all n large enough.
In fact, we shall prove that the power α in (2.2) is optimal even for stationary martingale difference sequences.
It is clear that when α = 13 , the bound (2.2) implies the bound (1.1) of Lesigne and
Volný.
2α
Our second result shows that the moment condition supi E exp{|Xi | 1−α } < ∞ in
α
+
+ 1−α
} < ∞, where Xi = max{Xi , 0}, if
Theorem 2.1 can be relaxed to supi E exp{(Xi )
we add a constraint on the sum of conditional variances
hSik =
k
X
E(Xi2 |Fi−1 ).
i=1
Theorem 2.2. Let α ∈ (0, 1). Assume that (Xi , Fi )i≥1 is a sequence of supermartingale
α
differences satisfying supi E exp{(Xi+ ) 1−α } ≤ C1 for some constant C1 ∈ (0, ∞). Then,
for all x, v > 0,
P Sk ≥ x and hSik ≤ v 2 for some k ∈ [1, n]
x2
≤ exp −
+ nC1 exp{−xα }.
2(v 2 + 13 x2−α )
(2.4)
For bounded random variables, some inequalities closely related to (2.4) can be
found in Freedman [5], Dedecker [1], Dzhaparidze and van Zanten [3], Merlevède,
Peligrad and Rio [11] and Delyon [2].
Adding a hypothesis on hSin to Theorem 2.2, we can easily obtain the following
Bernstein type inequality which is similar to an inequality of Merlevède, Peligrad and
Rio [12] for weakly dependent sequences.
ECP 17 (2012), paper 59.
ecp.ejpecp.org
Page 2/8
Exponential inequalities for supermartingales
Corollary 2.3. Let α ∈ (0, 1). Assume that (Xi , Fi )i≥1 is a sequence of supermartingale
α
differences satisfying supi E exp{(Xi+ ) 1−α } ≤ C1 and E exp{(
constants C1 , C2 ∈ (0, ∞). Then, for all x > 0,
(
x1+α
nα
≤ exp −
2 1 + 13 x
max Sk ≥ nx
P
1≤k≤n
α
hSin 1−α
}
n )
≤ C2 for some
)
+ (nC1 + C2 ) exp{−xα nα }. (2.5)
In particular, with x = 1, it holds
max Sk ≥ n
P
=
1≤k≤n
O (exp{−C nα }) ,
n → ∞,
(2.6)
where C > 0 is an absolute constant. Moreover, the power α in (2.6) is optimal for the
class of martingale differences: for each α ∈ (0, 1), there exists a sequence of martingale
α
α
hSi
differences (Xi , Fi )i≥1 satisfying supi E exp{(Xi+ ) 1−α } < ∞, supn E exp{( n n ) 1−α } < ∞
and
P
max Sk ≥ n
1≤k≤n
≥
exp{−3nα }
(2.7)
for all n large enough.
Actually, just as (2.2), the power α in (2.6) is optimal even for stationary martingale
difference sequences.
In the i.i.d. case, the conditions of Corollary 2.3 can be weakened considerably,
see Lanzinger and Stadtmüller [8] where it is shown that if E exp{(X1+ )α } < ∞ with
α ∈ (0, 1), then
P
3
max Sk ≥ n
1≤k≤n
=
O (exp{−Cα nα }) ,
n → ∞.
(2.8)
Proof of Theorem 2.1
We shall need the following refined version of the Azuma-Hoeffding inequality.
Lemma 3.1. Assume that (Xi , Fi )i≥1 is a sequence of martingale differences satisfying
|Xi | ≤ 1 for all i ≥ 1. Then, for all x ≥ 0,
x2
P max Sk ≥ x
≤ exp −
.
(3.1)
1≤k≤n
2n
A proof can be found in Laib [7].
For the proof of Theorem 2.1, we use a truncating argument as in Lesigne and Volný
[9]. Let (Xi , Fi )i≥1 be a sequence of supermartingale differences. Given u > 0, define
Xi0
=
Xi00
= Xi 1{|Xi |>u} − E(Xi 1{|Xi |>u} |Fi−1 ),
Sk0
=
Xi 1{|Xi |≤u} − E(Xi 1{|Xi |≤u} |Fi−1 ),
k
X
Xi0 ,
Sk00 =
i=1
k
X
Xi00 ,
Sk000 =
i=1
k
X
E(Xi |Fi−1 ).
i=1
Then (Xi0 , Fi )i≥1 and (Xi00 , Fi )i≥1 are two martingale difference sequences and Sk =
Sk0 + Sk00 + Sk000 . Let t ∈ (0, 1). Since Sk000 ≤ 0, for any x > 0,
P
max Sk ≥ x
1≤k≤n
max Sk0 + Sk000 ≥ xt + P max Sk00 ≥ x(1 − t)
1≤k≤n
1≤k≤n
≤ P max Sk0 ≥ xt + P max Sk00 ≥ x(1 − t) .
(3.2)
≤
P
1≤k≤n
1≤k≤n
ECP 17 (2012), paper 59.
ecp.ejpecp.org
Page 3/8
Exponential inequalities for supermartingales
Using Lemma 3.1 and the fact that |Xi0 | ≤ 2u, we have
P
max
1≤k≤n
Sk0
x2 t 2
exp −
8nu2
≥ xt
≤
.
(3.3)
2α
Let Fi (x) = P(|Xi | ≥ x), x ≥ 0. Since E exp{|Xi | 1−α } ≤ C1 , we obtain, for all x ≥ 0,
2α
2α
2α
Fi (x) ≤ exp{−x 1−α }E exp{|Xi | 1−α } ≤ C1 exp{−x 1−α }.
Using the martingale maximal inequality (cf. e.g. p. 14 in [6]), we get
P
max
1≤k≤n
≥ x(1 − t)
Sk00
≤
n
X
1
EXi00 2 .
x2 (1 − t)2 i=1
(3.4)
It is easy to see that
Z
EXi00 2
∞
t2 dFi (t)
Z ∞
2
u Fi (u) +
2tFi (t)dt
−
=
u
=
u
Z
2α
2
∞
2α
C1 u exp{−u 1−α } + 2C1
≤
t exp{−t 1−α }dt.
(3.5)
u
2α
Notice that the function g(t) = t3 exp{−t 1−α } is decreasing in [β, +∞) and is increasing
in [0, β], where β =
∞
Z
t exp{−t
3(1−α)
2α
2α
1−α
1−α
2α
. If 0 < u < β , we have
Z
}dt
β
≤
t exp{−t
u
2α
1−α
u
β
Z
2α
∞
t exp{−u 1−α }dt +
≤
2α
t−2 t3 exp{−t 1−α }dt
}dt +
β
Z
2α
t−2 β 3 exp{−β 1−α }dt
β
u
≤
∞
Z
2α
3 2
β exp{−u 1−α }.
2
(3.6)
If β ≤ u, we have
Z
∞
Z
2α
∞
t exp{−t 1−α }dt =
u
Zu∞
≤
=
2α
t−2 t3 exp{−t 1−α }dt
2α
t−2 u3 exp{−u 1−α }dt
u
2
2α
u exp{−u 1−α }.
(3.7)
By (3.5), (3.6) and (3.7), we get
EXi00 2
2α
≤ 3C1 (u2 + β 2 ) exp{−u 1−α }.
(3.8)
From (3.4), it follows that
P
max
1≤k≤n
Sk00
≥ x(1 − t)
≤
2α
3nC1
(u2 + β 2 ) exp{−u 1−α }.
x2 (1 − t)2
(3.9)
Combining (3.2), (3.3) and (3.9), we obtain
P
max Sk ≥ x
1≤k≤n
≤
2
2α
x2 t 2
3nC1
u
β2
2 exp −
+
+
exp{−u 1−α }.
2
2
2
2
8nu
(1 − t)
x
x
ECP 17 (2012), paper 59.
ecp.ejpecp.org
Page 4/8
Exponential inequalities for supermartingales
Taking t =
√1
2
and u =
P
x
√
4 n
1−α
, we get, for all x > 0,
max Sk ≥ x
1≤k≤n
2 α x
≤ Cn (α, x) exp −
,
16n
where
Cn (α, x) = 2 + 35nC1
β2
1
+ 2
2α
1−α
x (16n)
x
.
Hence, for all x > 0,
P
max Sk ≥ nx
1≤k≤n
x 2α α
≤ C(α, x) exp −
n
,
4
where
C(α, x) = 2 + 35C1
1
1
+ 2
2α
1−α
x 16
x
3(1 − α)
2α
!
1−α
α
.
This completes the proof of the first assertion of Theorem 2.1.
Next, we prove that the power α in (2.2) is optimal by giving a stationary sequence
of martingale differences satisfying (2.3). We proceed as in Lesigne and Volný ([9], p.
150). Take a positive random variable X such that
P (X > x) =
for all x > 1.
exp{t
2α
1−α
2e
1+x
n
o
2α
exp −x 1−α
1+α
1−α
Using the formula Ef (X) = f (1) +
R∞
1
(3.10)
f 0 (t)P(X > t)dt for f (t) =
}, t ≥ 1, we obtain
E exp{X
2α
1−α
Z
4e α
}=e+
1−α
3α−1
∞
t 1−α
1+α
dt < ∞.
1 + t 1−α
1
Assume that (ξi )i≥1 are Rademacher random variables independent of X , i.e. P(ξi =
1) = P(ξi = −1) = 21 . Set Xi = Xξi , F0 = σ(X) and Fi = σ(X, (ξk )k=1,...,i ). Then,
(Xi , Fi )i≥1 is a stationary sequence of martingale differences satisfying
2α
2α
sup E exp{|Xi | 1−α } = E exp{X 1−α } < ∞.
i
For β ∈ (0, 1), it is easy to see that
P
max Si ≥ n
≥ P (Sn ≥ n) ≥ P
1≤k≤n
n
X
!
ξi ≥ nβ
P X ≥ n1−β .
i=1
Since, for n large enough,
P
n
X
!
ξi ≥ nβ
≥ exp −n2β−1 ,
i=1
(cf. Corollary 3.5 in Lesigne and Volný [9]), we get, for n large enough,
P
max Si ≥ n
1≤k≤n
2e
≥
1 + (n1−β )
1+α
1−α
n
o
2α
exp −n2β−1 − (n1−β ) 1−α .
(3.11)
Setting 2β − 1 = α, we obtain, for n large enough,
P
max Si ≥ n
1≤k≤n
≥
2e
1+n
1+α
2
exp {−2nα } ≥ exp {−3nα } ,
which proves (2.3). This ends the proof of Theorem 2.1.
ECP 17 (2012), paper 59.
ecp.ejpecp.org
Page 5/8
Exponential inequalities for supermartingales
4
Proof of Theorem 2.2
To prove Theorem 2.2, we need the following inequality.
Lemma 4.1 ([4], Remark 2.1). Assume that (Xi , Fi )i≥1 are supermartingale differences
satisfying Xi ≤ 1 for all i ≥ 1. Then, for all x, v > 0,
x2
P Sk ≥ x and hSik ≤ v for some k ∈ [1, n] ≤ exp −
2(v 2 + 31 x)
2
.
(4.1)
Assume that (Xi , Fi )i≥1 are supermartingale differences. Given u > 0, set
Xi0 = Xi 1{Xi ≤u} , Xi00 = Xi 1{Xi >u} , Sk0 =
k
X
Xi0 and Sk00 =
i=1
k
X
Xi00 .
i=1
Then, (Xi0 , Fi )i≥1 is also a sequence of supermartingale differences and Sk = Sk0 + Sk00 .
Since hS 0 ik ≤ hSik , we deduce, for all x, u, v > 0,
P Sk ≥ x and hSik ≤ v 2 for some k ∈ [1, n]
≤ P Sk0 ≥ x and hSik ≤ v 2 for some k ∈ [1, n]
+P Sk00 ≥ 0 and hSik ≤ v 2 for some k ∈ [1, n]
≤ P Sk0 ≥ x and hS 0 ik ≤ v 2 for some k ∈ [1, n] + P max Sk00 ≥ 0 .
1≤k≤n
(4.2)
Applying Lemma 4.1 to the supermartingale differences (Xi0 /u, Fi )i≥1 , we have, for all
x, u, v > 0,
P(Sk0
x2
≥ x and hS ik ≤ v for some k ∈ [1, n]) ≤ exp −
2(v 2 + 13 xu)
0
2
.
(4.3)
α
Using the exponential Markov’s inequality and the condition E exp{(Xi+ ) 1−α } ≤ C1 , we
get
P
max
1≤k≤n
Sk00
≥0
≤
≤
n
X
i=1
n
X
P(Xi > u)
α
α
E exp{(Xi+ ) 1−α − u 1−α }
i=1
≤
α
nC1 exp{−u 1−α }.
(4.4)
Combining the inequalities (4.2), (4.3) and (4.4) together, we obtain, for all x, u, v > 0,
P(Sk ≥ x and hSik ≤ v 2 for some k ∈ [1, n])
α
x2
+ nC1 exp{−u 1−α }.
≤ exp −
2(v 2 + 13 xu)
(4.5)
Taking u = x1−α , we get, for all x, v > 0,
P(Sk ≥ x and hSik ≤ v 2 for some k ∈ [1, n])
x2
≤ exp −
+ nC1 exp{−xα }.
2(v 2 + 31 x2−α )
(4.6)
This completes the proof of Theorem 2.2.
ECP 17 (2012), paper 59.
ecp.ejpecp.org
Page 6/8
Exponential inequalities for supermartingales
5
Proof of Corollary 2.3.
To prove Corollary 2.3 we make use of Theorem 2.2. It is easy to see that
P
max Sk ≥ nx, hSin ≤ nv 2
1≤k≤n
+P max Sk ≥ nx, hSin > nv 2
max Sk ≥ nx
≤ P
1≤k≤n
1≤k≤n
≤ P(Sk ≥ nx and hSik ≤ nv 2 for some k ∈ [1, n])
+P hSin > nv 2 .
(5.1)
By Theorem 2.2, it follows that, for all x, v > 0,
P
(
max Sk ≥ nx
1≤k≤n
≤
x2
nα
exp −
2 nα−1 v 2 + 31 x2−α
)
+nC1 exp {−xα nα } + P(hSin > nv 2 ),
Using the exponential Markov’s inequality and the condition E exp{(
get, for all v > 0,
P hSin > nv
Taking v = (nx)
1−α
2
2
≤ E exp
α
α
hSin 1−α
)
− v 2 1−α
(
n
≤ C2 , we
α
≤ C2 exp{−v 2 1−α }.
, we obtain, for all x > 0,
P
α
hSin 1−α
}
n )
(
max Xk ≥ nx
≤
1≤k≤n
x1+α
nα
exp −
2 1 + 13 x
)
+ (nC1 + C2 ) exp{−xα nα },
which gives inequality (2.5).
Next, we prove that the power α in (2.6) is optimal. Let (Xi , Fi )i≥1 be the sequence
of martingale differences constructed in the proof of the second assertion of Theorem
hSi
2.1. Then n n = X 2 ,
α α
1
sup E exp (Xi+ ) 1−α = E exp{X 1−α } < ∞
2
i
and
α
2α
hSin 1−α
sup E exp (
)
= E exp{X 1−α } < ∞.
n
n
Using the same argument as in the proof of Theorem 2.1, we obtain, for n large enough,
P
max Sk ≥ n
1≤k≤n
≥
exp {−3nα } .
This ends the proof of Corollary 2.3.
References
[1] J., Dedecker. Exponential inequalities and functional central limit theorems for random
fields. ESAIM Probab. Statit. 5 (2001), 77–104.
[2] B., Delyon. Exponential inequalities for sums of weakly dependent variables. Electron. J.
Probab. 14 (2009), 752–779. MR-2495559
[3] K., Dzhaparidze and J. H., van Zanten. On Bernstein-type inequalities for martingales.
Stochastic Process. Appl. 93 (2001), 109–117. MR-1819486
ECP 17 (2012), paper 59.
ecp.ejpecp.org
Page 7/8
Exponential inequalities for supermartingales
[4] X., Fan, I., Grama and Q., Liu. Hoeffding’s inequality for supermartingales. Stochastic Process. Appl. 122 (2012), 3545–3559. MR-2956116
[5] D. A., Freedman. On tail probabilities for martingales. Ann. Probab. 3 (1975), 100–118.
MR-0380971
[6] P., Hall and C. C., Heyde. Martingale Limit Theory and Its Application, Academic Press,
1980, 81–96.
[7] N., Laib. Exponential-type inequalities for martingale difference sequences. Application to
nonparametric regression estimation. Commun. Statist.-Theory. Methods, 28 (1999), 1565–
1576. MR-1707103
[8] H., Lanzinger and U., Stadtmüller. Maxima of increments of partial sums for certain subexponential distributions. Stochastic Process. Appl., 86 (2000), 307–322. MR-1741810
[9] E., Lesigne and D., Volný. Large deviations for martingales. Stochastic Process. Appl. 96
(2001), 143–159. MR-1856684
[10] Q., Liu and F., Watbled. Exponential ineqalities for martingales and asymptotic properties
of the free energy of directed polymers in a random environment. Stochastic Process. Appl.
119 (2009), 3101–3132. MR-2568267
[11] F., Merlevède, M., Peligrad and E., Rio. Bernstein inequality and moderate deviations under
strong mixing conditions. IMS Collections. High Dimensional Probability 5 (2009), 273–292.
MR-2797953
[12] F., Merlevède, M., Peligrad and E., Rio. A Bernstein type inequality and moderate deviations
for weakly dependent sequences. Probab. Theory Relat. Fields 151 (2011), 435–474. MR2851689
Acknowledgments. We would like to thank the two referees for their helpful remarks
and suggestions.
ECP 17 (2012), paper 59.
ecp.ejpecp.org
Page 8/8