Chapter 6
Independence
6.11.P
By the Kolmogorov maximal inequality, 2 P{maxj6n |Sj | > } 6 ES2n 6
1
2
i=1 EXi = A (Fubini–Tonelli). Therefore,
P
sup
n6j6n+m
✏
|Sj - Sn | >
6
Pn+m
i=n
E[X2i ]
8
2
> 0.
Continuity properties of P imply that
P sup |Sj - Sn | >
j>n
✏
We can find nk " 1 so fast that
then,
P
6
P1
i=n
P
i>nk
8
> 0.
E[X2i ] 6 e-k . By Borel-Cantelli
sup |Sj - Snk | > k-2
j>nk
E[X2i ]
2
✏
6 k4 e-k ,
which is summable in k. Therefore, the Borel–Cantelli lemma ensures
that a.s. forP
all but a finite number of k’s, supj>nk |Sj - Snk | 6 k-2 . In
particular, k supj>nk |Sj - Snk | < 1 a.s. This implies that {Sn }1
n=1 is
a.s. a Cauchy sequence (why?).
6.13. Fix some
lim inf n (Sn /n) > lim inf n (S⌫
n /n), where
Pn ⌫ ⌫> 0. Clearly,
⌫
⌫
Sn :=
X
,
and
X
:=
X
1
.
By
the
law
of
large
numbers,
i
{X
6⌫}
i=1 i
i
i
⌫
lim inf n (S⌫
/n)
=
lim
(S
/n)
=
E[X
;
X
6
⌫].
Let
⌫
"
1
to
finish.
n
1
1
n
n
6.29. Solution 1. We will need Lemma: As n ! 1,
Proof
Rn
1
(dx/x) 6
Pn
i=1 (1/i)
61+
13
Rn
1
Pn
(dx/x).
i=1 (1/i)
⇠ ln(n).
⇤
14
CHAPTER 6. INDEPENDENCE
Now let us first
] = 0. Define Xi0 = Xi 1{|Xi |6i} . Also
1P
Pn assume that E[X
0
0
define Sn = i=1 (Xi /i) and Sn = n
i=1 (Xi /i). By the Kolmogorov maximal inequality,
P
max |Sk0 - E[Sk0 ]| > " ln n
16k6n
6
VarSn0
2
" (ln n)2
⇥
⇤
n
X
E X21 ; |X1 | 6 i
kSn0 k22
1
6 2 2
.
= 2 2
i2
" ln (n)
" ln (n) i=1
Therefore,
1
X
n=1
P
max n |Sk0 - E[Sk0 ]| > " ln 22
⇥
⇤
1
2n
X
1 X E X21 ; |X1 | 6 i
4n
i2
n=1
i=1
⇥
⇤
1
X
X
E X21 ; |X1 | 6 i
1
6A
i2
4n
i=1
n>log2 (i)
⇥ 2
⇤
1
X
E X1 ; |X1 | 6 i
6B
i2
i=1
2
3
X 1
5 6 CEX1 .
= BE 4X21
i2
n
6A
16k622
i>|X1 |
By the Borel–Cantelli lemma, a.s., for all n large,
max n |Sk0 - E[Sk0 ]| 6 " ln 22
n
16k622
n
Any m is between some 22 and 22
0
0
|Sm
- E[Sm
]| 6
max
16k622n+1
n+1
.
. Therefore,
|Sk0 - E[Sk0 ]|
⇣ n+1 ⌘
6 " ln 22
= 2" ln(2) · 2n 6 2" ln(2) ln m.
P
0
0
0
This
P proves that a.s., |Sm - ESm |/ ln(m) ! 0. But k P{Sk 6= Sk0 } =
k P{|X1 | > k} < 1 because kX1 k1 < 1. So almost surely, Sk = Sk for
0
all k large. It suffices to prove that |ESm
|/ ln m ! 0. But
0
|ESm
|=
m
X
E[X1 ; |X1 | 6 i]
i=1
i
=
m
X
E[X1 ; |X1 | > i]
i
i=1
since EX1 = 0. Thus,
0
|ESm
|6
m
X
E[|X1 |; |X1 | > i]
i=1
i
.
,
15
For all ⌘ > 0, there exists i0 such that for al i > i0 , E{|X1 |; |X1 | > i} 6 ⌘.
Pi 0
P
Pm
Therefore, m
i=i0 (· · · ) 6 ⌘
i=1 (1/i) ⇠ ⌘ ln m. But
i=1 (· · · ) 6 i0 kX1 k1 .
0
Therefore, lim supm |ESm
|/ ln m 6 ⌘ for all ⌘, whence the result.
If µ = EX1 6=P
0, then consider instead Xi := Xi - µ. The preceding
proves
Pn
n
-1
-1
that
ln
(n)
(X
/i)
!
0
a.s.
Equivalently,
ln
(n){
(X
i /i) i=1
i=1
i
P
Pn
µ n
(1/i)}
!
0.
Because
(1/i)
⇠
ln(n),
we
obtain
the
result
in
i=1
i=1
general.
Solution 2. P
We may start with P
Lemma: If limP
n!1 an = µ, bn > 0 for all
n
n
n > 1, and 1
b
=
1,
then
a
b
⇠
µ
i
i
i
i=1
i=1
i=1 bi .
Proof 8" > 0, 9n0 so that |ai - µ| 6 µ + " for all i > n0 . Then,
n
X
ai bi ⇠
i=1
n
X
i=n0
ai bi = (µ ± ")
n
X
i=n0
bi ⇠ (µ ± ")
n
X
bi ,
i=1
⇤
notation being clear.
Now let S0 = 0, and Sn =
n
X
Xi
i=1
n
X
Pn
j=1
Sj (n > 1), so that
n
n
1 X 1 X
1
=
(Si - Si-1 ) =
Si Si-1
i
i
i
i
i=1
i=1
i=1
◆
n
n-1
n-1
X
X ✓1
1 X
1
1
Sn
=
Si Si
= S1 +
Si
.
i
i+1
i
i+1
n+1
i=1
i=2
i=2
By the strong law, Sn /(n + 1) ! µ a.s. as n ! 1. Therefore,
n
n-1
n-1
1 X Xi
1 X Si
1 X µ
⇠
⇠
!µ
ln n
i
ln n
i(i + 1) ln n
i+1
i=1
i=2
a.s.
i=2
6.32. If Sn := the number of Xi ’s (1 6 i 6 n) that are equal to one, then
Sn = Bin(n, 1/2). By the De Moivre–Laplace central limit theorem, for all
> 0,
⌦
p ↵
n
lim P Sn > +
n = P{N(0 , 1) > } > 0.
n!1
2
Thus, by the Kolmogorov 0–1 law,
lim sup
n!1
Sn - (n/2)
p
=1
n
a.s.
P
P
p
Because n
follows that
lim supn!1 n
i=1 Xi = 2Sn -n, it P
i=1 Xi / n = 1
p
a.s. By symmetry, lim inf n!1 n
i=1 Xi / n = -1 a.s. Thus,
lim sup
n!1
n
X
i=1
Xi = - lim inf
n!1
n
X
i=1
Xi = 1
a.s.
16
CHAPTER 6. INDEPENDENCE
6.33.
1. First of all we note that if n > 1 and t > 0, then for any > 0,
⇥
⇤
P{Sn > t} = P e Sn > e t 6 e- t E e Sn ,
owing to Chebyshev’s inequality. But the expectation of a product
of independent random variables
is the product of the expectations;
Q
therefore, E exp( Sn ) = n
E
exp(
Xj )}. Problem 4.30 of the text
j=1
implies that E exp( Xj ) 6 exp( 2 c2j /2). Therefore,
This proves that
⇥
E e
Sn
⇤
6e
P{Sn > t} 6 e-
2 2
sn /2
t+
.
2 2
sn /2
.
The left-hand side is independent of . So we can minimize the
right-hand side over all > 0; the minimum occurs at = t/s2n ,
and yields P{Sn > t} 6 exp(-t/s2n ). The result follows from replacing Sn by -Sn .
2. Let I1 , . . . , In be i.i.d. Bin(1 , p) random variables. Then B has the
same distribution as I1 + · · · + In . Let Xi := Ii - p. Then the Xi ’s are
i.i.d., have mean zero, and are bounded between -1 and 1. Therefore, we can apply the first part to Sn := X1 + · · · + Xn to find that
s2n := n and
P{|Sn | > t} 6 2e-t/(2n)
for all t > 0.
p
Replace t by t n and observe that Sn has the same distribution as
B - np to finish.
c
c
3. Note that, in (6.48), |T2 | 6 KP(A
p ), and A = {|Sn -np| > n }. Apply
the previous part with t :=
n to finish.
4. Similar to part 3.