Journal of Theoretical Probability, Vol. 9. No. 3. 1996
The Law of the Iterated Logarithm and Marcinkiewicz
Law of Large Numbers for B-Valued U-Statistics 1
Zhonggen Su 2
Received December 8, 1994: revised Jcmuary 8. 1996
Suppose that B is a separable Banach space and (S, 6a, P) a probability space.
H is a measurable symmetric kernel function from S" into B. In this paper we
shall further study some limit theorems for B-valued U-statistics U",,H based on
P and H. Special attention is paid upon the Marcinkiewicz type law of large
numbers and the law of the iterated logarithm. Our results can be regarded as
extensions of corresponding results for sums of independent B-valued random
variables to U-statistics.
KEY WORDS: B-U-statistics; decoupling inequality; law of large numbers;
law of the iterated logaritlnn.
1. I N T R O D U C T I O N
AND PRELIMINARIES
Let (S, 6:, P ) be a p r o b a b i l i t y m e a s u r e space a n d X, X', Xi, i/> 1 be i.i.d.
( P ) S - v a l u e d r a n d o m variables. Let B be a s e p a r a b l e B a n a c h space w i t h
n o r m I1"11 a n d its d u a l B' a n d let H(Xl ..... Xm): S " ~ B
be a m e a s u r a b l e
f u n c t i o n s y m m e t r i c in its a r g u m e n t s , t h a t is, H(x~ ..... x,,,) =H(xo~ ..... x~,,,)
for a n y p e r m u t a t i o n a o f (1, 2 ..... m). As u s u a l , the B - v a l u e d U-statistic
b a s e d o n X;, 1 ~< i ~< n, a n d k e r n e l f u n c t i o n H is d e f i n e d as
1
U.,H--(-;~
I1
~
- -
,i)),
I <<. i) <
. . .
H(X~, ..... X,,,,)< ~,,, -< n
(n--m)[
n!
~ H(X~, ..... Xim) ( I . I )
I',',,
"
w h e r e I',:,= {(i~ ..... i,,,): i,<~n, i , ~ i , if r :/:s}.
~Research supported by National Natural Science Foundation of China and Zhejiang
Province.
2 Department of Mathematics, Hangzhou University, Zhejiang 310028, China; Institute of
Mathematics, Fudan University, Shanghai, 200433, China.
679
0894-9840/96/0700-0679509.50/0 9 1996 Plenum Publishing Corporation
680
Su
Our main interest is in the limit theory for U~',,H. As it is well known,
many classical limit theorems for sums of independent real random
variables have been generalized to the real U-statistic case (see Serfling, ~11
and Gin~ and Zinn~4~). Thus, naturally, we wonder whether some beautiful
limit theorems for sums of independent B-valued random variables have
analogues for the B-valued U-statistics. The central limit theorem (CLT)
for B-valued U-statistics U",,H has been presented in Arcones ~-'~ and completely understood in the case of Banach spaces of type 2. Arcones and
Gin+ ~3) also have obtained that the law of large numbers for B-valued
U-statistics U'~,H holds as long as E [[H(X l ..... X,,)[[ <o0. Their proof
mainly relies on the fact that [[U',',H--EHI[--*O a.e. is equivalent to
E [1U',',H- EH[[ ---,0 since [IU~',,H- EHI[ is a reversed submartingale.
However, this fact is no longer true for the Marcinkiewicz law of large
numbers (MLLN). The purpose of this paper is to further study some
other limit theorems for U',',H, for example, the M L L N and the law of the
iterated logarithms (LIL). Our main results given in details in Section 2
show that the two strong limit theorems for B-valued U-statistics are similar
than for sums of independent identically B-valued random variables.
An important tool in deriving the asymptotic theory of U-statistics is
the Hoeffding's decomposition, i.e., the projection of a U-statistic on the
basic observations of the sample. We give it here together with some
notation. The operator n ke. , , -_- n k.... acts on P"'-integrable functions
H: S"'--* B as follows: rck.,,H(x I ..... X k ) = ( J . , . ~ - - P ) . - . ( f x k - - P ) P .... kH,
where Q l " " Q,,,H= ~ H ( x I ..... x,,) d Q l ( X l ) . . , dQ,,(Xm). Note that z~..... is a
P-canonical function of k-variables, that is, PrCk.,,H(', x,_ ..... X k ) = 0 for
almost all x,_ ..... Xk. Hoeffding's decomposition is as follows: for all
P"'-integrable functions H: S " --* B,
(1.2)
k=l
The first term in the right-hand side is just m/n~'i'= l nl.,,,H(X ~) so
that we can readily apply various limit theorems for sums of independent
identically B-valued random variables, which have been nearly thoroughly
discussed in Ledoux and Talagrand's bibliography) s) Thus the key to our
study is how to deal with those remainders.
Randomization by Rademacher variables plays a role in B-valued
U-statistics similar to the role it plays for sums of independent B-valued
random variables due to the fact that B-valued U-statistics can be
decoupled. We state later several pertinent results. Let now {XI k),
i/> 1} ~-'=l be m independent copies of { Xg, i >/1 }, {e~, i >/1 } a Rademacher
Iterated Logarithm Law and Marcinkiewicz Law of Large Numbers
681
sequence independent of {X;, i~> 1} and ""'*)
~ j , i>~l }~'_
_j m independent
copies of {e;, i >I 1 }, independent of the variables { X~k~, i >1 1 } ~'= ~.
Lemma 1.1. (de la Pefialg)).
ing function. Then
Let ~b: [0, Go) --, R be a convex increas-
E~ (c, ~g;,H(X~,') ..... Xi,,,'"')) )~< E~b (
~ H(Xi, ..... X;,,,) )
<~E~ (ca ~ H(X,~ )..... X,,:,')) )
(1.3)
elleli ".e. I.o H(X r ).
,X~," '))
)
g*,
/
If H is P-canonical function, then
E~
cI
--.--i,
,,)
..... Xi,,, )
<~Ecb
Ill)
\
il
"""
Here Cl, c2 are positive constants depending on m (independent
of ~, n, I1"11) and possibly varying from line to line. Take ~b(x)=x'-, the
equivalence described in Lemma 1.1 allows us to use the defining type 2
and cotype 2 inequalities.
Remark 1.1. It is known that if B is a separable Banach space and
F is a subspace of B, then for some countable subset D c B' we have
qF(X)=SUpf~n If(x)[ for all x in B, where qr(x) is the seminorm defined
by qF(X) = inf{ IIx- yll, y ~ F}. Therefore, (a) and (b) remain true when the
norm []-II is replaced by qF"
Lemma 1.2. (Gin6 and Zinnt6)).
of B.
(a)
Let K be a convex symmetric subset
If Dj, j = 1, 2,..., rn are subsets of { 1, 2 ..... n}, then
P
s
(il
i,<_/./
.-,., ira) ~
DI
.....
x
99 9
x
Dm
k_<.....
2"'- 1
\
)
682
Su
(b)
\ 1nl
< ......
-.~2
(2 - 1 ) m a x e
H(X~, ..... Xg,,,)~K" )
Lemma 1.2 is an elementary but useful fact. It indicates that the tails
of the original U-statistic can dominate the tails of a decoupled, randomized version of it. As noticed by Gin6 and Zinn, t6) it would be interesting to have inequalities analogous to those in L e m m a 1.2, but in the
opposite direction. We refer the reader to de la Pefia and M o n t g o m e r y Smith c~~ for the comparison between the two tail probabilities.
The following Hoffmann-Jogensen type inequality will help us treat
integrability in some cases.
L e m m a 1.3.
(a)
(Gin~ and Zinn(5)). There exist finite constants ct(p), c2(p) and
c3(p) ~ (0, 1 ) such that
/)
.,(t)
'("')H
(I)
X(,,,))
E ~ el,
" " ei,,, ,--it ,'", i,,,
I',',,
max
-< c t~' + c , E
I
0
-
1 : ~ i m ~< tz
~'
(I)
Ira)
(I)
ei~
...el,,,
H ( X i , .....
m) )
Y (im
P
(1.7)
i l ...-, i r a - I
9
.
n
( II ,..., Iij)) E I m
where to is any number satisfying
P
e(il)...ei,))
H(Xii
.....
>to
~<c3
n
(b) There exist finite positive constants c~(p), c~_(p) and c 3 ( p ) ~ (0, 1)
such that
E
.... i 11
~ $iH(Xi ' X(I I )..... X (....
) I'
i=l
<~clt~+c~_E max IIH(X,, X(~~).......Y ( ,,,_~
.... I))llP
I ~i~<n
(1.8)
Iterated Logarithm Law and Marcinkiewiez Law of Large Numbers
683
where to is any number satisfying
c i H ( X i , x I ..... c . . . .
supP
.v
>to
,))
~<c3
I
i
with the supremum taken over all x = ( x ( ..... x .... ]) ~ S .... t
E
~
~.,(1)
il
' "
" -e- i .m(- "
I ' - l ) H ( X (l
il )~ ... ' 'Y
' ' i m. . .- .I
<~C)t~+CzE
R( I )
~,
max
l <~ i m _
X') I,
]) '
l <~ lt
.
!1 . - - .
-i,
. . .E (m-2)
i,,,_,
.
Iin - ]
ii
( il ..,,, i m - I ) E I m _ 1
X
H t~X (])
Y( . . . . I
il
'"'~ "'Sn-
])
'
X') p
(1.9)
where to is any number satisfying
\
supP
~
.(I) . . . e- l- i r....
t ) H()t.'i,)
(] ' . . . . ,. y ( ,i m, , -- lI)
a- I
Gil
x)
>to)~<
c3
/
with the supremum taken over all x ~ S.
We only prove Eq. (1.9). Define
Proof of (b).
d
U
II - -
y , g~(t,
il
" "
.F~.
....I ])H(X(i.])
--tin-
"'""
X ' i r....
I),X')
a- 1
Z;'n- I
M
d
I# m
max ~<t~
l ~<im-I
.
~.
e(])..-~
(.... "-)Ht~
"(t)
X I. . . .
il
vim-2
"'~'"
il ~ ' " ~
itn-I
~)
i
X')
II ,..., I m - 2
n
( il ..... i m - I ) E I m _ I
Let us recall Theorem 3 o f Gin~ and Zinn. (5) It shows that, first conditioning on X' and then taking expectation E' with respect to X', we find finite
positive constants c t and c2 such that for all t > 0,
P(IIUT~II>3 .... ' t ) < ~ c . g ' [ P ( l l U T ( l l > t ) ] z " - ' / 2 " - ' - ' - 4 - c 2 e ( M ~ ( > t )
(1.10)
684
Su
The constants can be taken to be c~ = 2 3~
6 2+2/3+
and c2 = 1 + 3"-. 4 + . . . + 32''-'/~
- t~ct. Now letting
U,d," ' = ~ e ~l)
.~(.... l)H~y(l~
il
"" V i m - I
~ ' ' il
~'"~
+2"-1/'2"-I--1)
'
(.,-II, X )
Xim-I
l ~n~-- I
for any to > 0 we have
EllU,~llt,<<31,,-I)~,to+ 3 ~.... ~lpf p(llUall>3 .... ~t) dt p
"t 0
<~3(,,,-~Pto+ 3~,,-~)pc,_ ~, P(M. d > t) dt p
o
+ 3' .... I~PcI f E ' [ P ( IIU,'~II > t)]'-"-'/~2"-'-l, d t p
" to
<~3' .... l lPt o + 3' . . . . I ) P c 2 E ( M ~ : ) P
+ 3 (.... I)p Cl s u p P ( [I --n
t]'d'x .[I > Io) 1/{2"-l- 1) 9y~ P(II
x
U,'~ll > t ) d :
o
(1.11)
Taking to is any number satisfying
( 1
)2,,,-,sup.,.P( IIU~)" II > to) -< 2.3' .... 'IPc,
I
::
C3
then Eq. (1.11) is bounded by
3' .... ')rt 0+ 89 IIU~IIp+3` .... ')Z'c2E(M~) ?
~<2 9 3~"'-')Pto+2 -a(
- ' ....
l)p,,t" 2 x,~t
' ~ . a~,c,t~?
t1 III
The required result follows.
All these lemmas are of course repeatedly used in the proof of our
main theorems. But in order to study strong limit behavior we need to
establish the maximal inequality for B-valued U-statistics and estimate
moments of sum of array of random variables. Our main ideas stem from
de Acosta, ('~ which studied the MLLN for sums of independent identically
random variables.
Iterated Logarithm Law and Marcinkiewicz Law of Large Numbers
685
2. MAIN RESULTS A N D P R O O F S
Let us start by describing what can be understood by a LIL for
B-valued U-statistics. Set LLt = L(Lt), Lt = max(l, log t), t >/0, and a, =
x/2nLLn. As in the case of sums of independent random variables, we will
say that the bounded LIL holds for U:,H if
A =: lim s u p :
....
x/2LLn
11U::,H-EHII
is a nonrandom finite number. Unlike in the independent sums case, A is
not necessarily nonrandom even though x / ~ / ~ l l g " , , g - - E H l l
is
almost surely bounded. This is mainly because we cannot use the
Kolmogorov's zero-one law. We might as well say that the compact LIL
holds for U2,H if there is a compact symmetric K in B such that, almost
surely
. l. i. .m d (
~v/~
( U;;,H - EH), K ) = 0
and
C [~(~ x//'~7
U;:,H--EH)} = K
where d(x, K) = inf{ IIx - yll, y e K} and where C{ x,,} denotes the set of the
cluster points of the sequence {x,,}. K is called the limit set.
In addition, we say that a family of sequences of { T,(x), n >>.1, x e S}
of random variables tends to zero uniformly in probability if for any e > 0
lim sup P(IIT,,(x)II > ~)=0
rl~
r
xES
The introduction of this concept enables us to repeatedly use Lemma
1.3(b). Now we give our main results about the LIL for B-valued
U-statistics.
Theorem 2.1. Suppose that B is a separable Banach space and
H: S"' ~ B is a measurable symmetric kernel function. Let { Am, n >i 1} is a
sequence of independent copies of X. If
(1)
E IIH(X~ ..... X.,)II'-/LL IIH(X~ ..... Xm)ll < O0,
(2)
a'-=sup;~BiE(rcl..,foH(X1))'-<oo,
860/9/3-10
686
Su
(3)
1
)'
H(Xi,
x I ..... x . . . .
1)
an i=l
0
uniformly in probability
1
~/zL
~,/z-;Lnn'(n-l'E
1 <<.i~j<~n
0
v/n
H(X,,Xj, xt ..... x .... 2)
uniformly in probability
(n-m)!
~ "
n---~. Z H(Xi, ..... X,,,,)
r,',
--, 0
in probability
then
,lim
x/n
IIU;:,H-EHII=m~
a.e.
(2.1)
If condition (2) is replaced by
(2)' {(nL,,,foH(XI)) 2, f E B ' , Ilfll ~< 1} is uniformly integrable,
then the compact LIL holds for U',',,H with the limit set inK, where
K=K,,,.,,n~x~ is the unit ball of the reproducing kernel Hilbert space
associated to n L .,H( XI ).
Proof In order to simplify our notations and not to obscure the
main scheme, we only consider the case m = 2, since the general case is
similar. Assume that {X i,, n >/1} is an independent copy of {X,,, n >/1}.
Keep in mind Hoeffding's decomposition
U~H-EH=2U'~zl,_H+ U'~2, 2H
(2.2)
We'll complete the proof of Theorem 2.1 in two steps as follows.
Step 1. The following assertions are established
,}im_
v/~
~/2LLn:
[IU~'rcl ~HII_ =,lim
1 "
--a,, ;i~ n,.2H(Xi)
=a
a.e.
(2.3)
=K
(2.4)
and
n1.2H(Xi), K =0,
n~o~
i=l
;r
i=1
Iterated Logarithm Law and Marcinkiewicz Law of Large Numbers
687
According to Theorem 8.6 in Ledoux and Talagrand ~s~ for Eqs. (2.3)
and (2.4) it is enough for us to check the following two conditions
E II~,, 2H(XI)II 2/LL I1~,.2H(X1)I] < oO
(2.5)
and
1_
lrl.2H(Xi)~O
an i = l
in probability
(2.6)
Indeed, since ~ ( t ) = t Z / L L t is convex and increasing for t > t o large
enough, there exist a positive number to and a nonnegative convex increasing
function ~b0(t), t > 0, so that
~bo(t) <~~k(t) <~to + Cko(t)
Noticing that r~l.2H(xl) = EH(xl, X2) - EH(XI, X2), we have
E~k(llTr,.2H(Xl)ll ) <~to + E~o( llnt.2H(X~)ll )
~< to + E~bo(2 Iln(xl, X_,)II)
<~to+4E(~(IIH(X~, X2)II) < oo
It remains to prove Eq. (2.6). Obviously Eq. (2.6) will be valid if
lim --1 E
~ nl.2H(X i) = 0
n ~ c'~ s
i=l
On the other hand, E IlYT/=17r ~. 2 H(Xi)I1 ~<2E IlY.7= ]e~H(X~, X')II. So,
we only need to show
n
,lim 1 E
an
~ e,H(X,,X') = 0
i
(2.7)
I
Note that E IIH(Xl, X2)II2/LL lIB(X,, X,_)ll < oo implies
lim --1 E max IIn(Xi, X')ll = 0
n~
oc a n
i <~i~n
by a simple integration by parts. In view of Lemma 1.3(b) this indicates
that Eq. (2.7) will follow if
1_
an
eiH(Xi, x) ~ 0
i = 1
uniformly in probability
(2.8)
688
Su
But this last condition is easily satisfied since 1/a,,Z~=t H(X+, X'~)~0
uniformly in probability and for each t > 0
sup P
x
tiH(Xi, x) > t ~<2 max sup
i--I
I<<.k<~n
H(X~, x) >
x
i
1
Step 2. We turn to prove
,f2-LLn U ~_g,_.,_tt--, O
a.e.
(2.9)
Let e > 0 be given. By the Borel-Cantelli lemma, it suffices for Eq. (2.9)
to show that
r
~P|
k=,
,F
max
~ - - - IIU~n, _~nl[ > e
\2~-'<~"<~2Ax/2LLn
<<.~ P ( max
\ 1 <~n~ ~k
)
- -"
~'.
n2,H<X,,
> ~ 2 a2,
<oo
(2.10)
Let us first observe that since E IIH(X~, Xz)II2/LL }]H(X~, X2)I] < oo, then
we have
max
~
1 ~<n~<2k
I<~i:~j<~n
~ P(
k=l
1
n2.2Hl,~m>,,,.~>(Xi,Xj)>e2ka2k)
~
<~- ~
1
~
g Iln2.znllllm>,.~.l(X i, Xj)II
e k= I 2kak 1 <~i~j<~2 k
4 ~ 2k
~<- ~ - - E IIHII Illlm >,,,~)(Xl, X2)
Ek= I a2k
4 ~ 2k~.
k~-E Ilgll I<ad<~ltnll6a_,t+,)(Sl, X,)
=-~
=1 a o k l = k
4~
2k~
<~- ~. - -
a2,.,P(IIH(X,, X2)[I >a2t)
4 +~.. a2,+IP(IiH(X], Y2)ll > a2,) L -2k
<~I=
k=l
a2k
C ao
<~+~= 2+P(IIH(X,, X2)II >a2,)< oo
Here c denotes a positive numerical constant.
(2.11)
Iterated Logarithm Law and Mareinkiewicz Law of Large Numbers
689
After having Eq. (2.11), we only need to convince ourselves of the
truth of the following claim:
~P
k=l
~
( max
I ~</iKgk
~"
~Z.~
2HItllnll<~a,~)(Xi,Xj)
-"
l<~i#j~n
>e2ka2k)< oO (2.12)
"
For the proof of this more difficult part we provide two lemmas.
Lemma 2.1. Suppose H is a measurable symmetric kernel function
taking their values in a separable Banach space B and for some t > 0
sup max P(
f ~B~ 1 <~n<~2k
~
f o H ( X i , Xj) >t)<<.~
I <<.i#j<~2k
(2.13)
Then
P(
Z
max
\ 1 ~<n~< "~k
-
I
H(Xi, Xj)
1 ~i#j~2
P
+8
>2t)
<~i#j<~n
k
(
2
n<j~2
H(X,,X:)
')
>-~
(2.14)
k
Proof of Lemma 2.1. Assume that {X';, 1 <~i<~2 k} is an independent
copy of {X;, 1 ~< i~<2k}. From the same reason as in Eq. (6.2) of Ledoux
and Talagrand ~8) it follows
/
P ( max
\
H(XI,Xj) > 2t)
Y"
/
<~2P(max
l~<n~<2k
Y' H(X,,Xj)-H(X'i,X'j.)>t)
(2.15)
l<~i#j<~n
Let r = inf{n ~<2k: liE, ~<,+:~<,, H(X;, xj) - H(X',, xj-)ll > t}. By definition,
the event ( r = n ) only depends on the random variables Xi ..... X,,
X'~ ,..., X',,; and
2k
~.
max
I ~n~
-") '~"
I<~i#j<~n
H(X~,Xj)-H(X~,X))
>t
= ~ (z=n)
;1=1
690
Su
Recall that e~ is a Rademacher variable independent of all other
random variables. Since it is not possible, by triangular inequality, that
both IIx + yll < Ilxll and I I x - Yll < IIx[I, we have
P( IIx-t- e, Yll > Ilxll) ~ 89
for all x, y in B.
Since I1~1 <.i+j<.,, H(Xi,
P(
X
Xj)-H(XL
(2.16)
X))II > t on the event (r =n), then
H(X,, Xj) - H(X',, X))
I <~i~j<~n
+
~.
,
H(X~, Xj)--H(X~, X~.) > ~ , r = n
)
.u<i~j<.2 k
=P.v.x;|
(
~
H(X~, X j ) - H ( X ; , X~)
I <~i~j<~n
,
t
h
H( Xi, Xj) -- H( X i, ?('i) > ~, r = 17
n<i~j<~2 k
1
>~~ P(r = II)
On the other hand,
P(
X
H(X,, Xj) - H(X',, X))
I <~i~j<<.n
n<i~j~<2 k
I <<.i#j<.2 k
n<j<~2 k
+p( z
n<i~<2 k
J
Iterated Logarithm Law and Marcinkiewicz Law of Large Numbers
<~2p
(
,
Y,
H(X. Xj) >-~,
691
)
~=n
I <~i~:j<~2 k
+4P
(
'
2
)
H(X,, Xi) >--~, ~=n
(2.17)
1 ~<i<~n
n<j<~2 k
Summarizing the precedings, we can now arrive at the desired result.
L e m m a 2.2.
such that for any
{Xi, Yj, l < i < , l ,
E liH(Xi, Yj)[I~'<
(i)
For every p/> 1 there exists a positive constant cp
measurable function H: $2-~ B and any finite sequence
l~<j~<m} of independent random variables with
~ the following inequality holds:
For 1 ~<p~<2,
E
i
lj=l
i
E ~=H(X~,Yj) +
<cp
i
(ii)
I
j
t
H(X,,Yj)
i
.)
1
Forp>2,
Ij=l
H(X~, Yj)
i
<~cp n p/z - '
i=1
Ij=l
+ m p/2- '
E ~ H(Xi,
j t
Proof of Lemma 2.2.
E
E
j=l
2 H(X,, Yj) --E
i
lj=l
,,,
j=l
E
i 1
H(Xi, Yj)
First observe
~ , ~. H(X. Yj)
j=l
i
n
~2p-I(ErEx
lj=l
m
n
i=l j=l
H(X.
+ Er Ex .j~l
i=l
m
p
~= Z H(Xi, YJ)
I 2 Z H(X,, rj) - e x
i
l j=l
-e,.F.~.
H(X,, r~)
j
1 i=l
(2.18)
692
Su
where E x and E r denote expectation with respect to (X, ..... X.) and
( Y, ..... Y.,) respectively.
We can use Theorem 2.1 of de Acosta ~'1 conditionally on (Y~ ..... Y,,,)
to get
i=I
j=l
~.
<Cp
n,
i
I j=,
i
lj=l
p
E ~= H(X,, rj)
i=l
j l
for 1 ~ p ~ 2 and
i=1 j=l
<~cpn pn- '
H(X,, Y])
i=l
for p > 2 .
We
shall
also
I
apply
Yurinskii's inequality to the averages
For each 1 ~<k~<m, let o,~. = a ( Yl ..... Y~-)
EA. IIEf=, 52/=, H(X. Yj)ll next.
and ~,~ = the trivial ~r-field. Set
(
~
)
Since by independence,
E r Ex
H(Xi, Yj)
j=l
j~k
H(X~, Yj)
~'k =Er
i=l
~k--I
1 i=l
j~k
we see that d k = E r ( f k [ ~ ' k ) - - E y ( f ~ - l ~ ' k - l ) ,
j=l
i=l
where for each I <<.k~m,
j=l
jr
i=l
It is plain that [fk[ ~ E x ][ZT=l H(Xi, Yk)[I, l<.k<<.m.
Iterated Logarithm Law and Marcinkiewiez Law of Large Numbers
693
Keeping these in mind and arguing as in Theorem 2.1 of de Acosta, ")
we can obtain
Er Ex
"'
-EvEx
i=I
<~Cp
"'
j
E
Y/)
I i=1
P
'
H(Xi,
i=l
j=l
lbr 1 ~<p ~<2 and
Er
H ( X , , Yj)
j=l
H(Xi,
i=l
1 i=1
<~ct, m p/'- - l
E
j=l
H(Xi,
i=1
for p > 2 .
Thus Lemma 2.2 is concluded.
Next we continue our proof of Theorem 2.1 with the help of Lemmas
2.1 and 2.2. For simplicity, unless specially stated, throughout the sequel of
the proof c denotes a positive constant that may depend on some
parameters and vary at each occurrence. In fact we have no attempt to get
its exact value.
Since E [IH(Xl, Xz)II2/LL [IH(XI, X2)II < oo, then by using Markov's
inequality and noting that r~, 2foHI~llm <,,,~.~ is a P-canonical kernel function for each f in B', we have
sup
/
max P (
. f E B i 1 <~n<~2k
\
Xr2.2 f o HI~uml <.,,,kl(Xi, X/) > e2'~a2k)
1 <~i#j<~n
~< , , , ,1t , , sup max E
~
x2.2foHI~llnll<<~,:~.)(Xi, Xj) 2
~-z- ar,k t'Eai I<.<n<.2k [l<.i~j<.n
r
<~ ., ., E [[Hll2 IiiiHit<~,,2k~(Xl, X2)
e -a ";k
cLLa,k
~< ~ , - E HH(X~, X2)I[2/LL [[H(X~, X2)[[
e-a;_~.
---*0
( k ---+~ )
694
Su
Thus by Lemma 2.1, in order to verify Eq. (2.12) it suffices to show for any
e>0
k= I
1 ~i#.i~2 k
and
")k
< m
k= I tl= I
(2.20)
I ~i~li
n<j<~2 k
To this aim we'll use Lemma 2.2. But before doing it we need further
establish the following facts:
a~k
J lejHI~,H,<,,.k)(Xj, X')
--+0,
foreach
p>0
(2.21)
and
1
max E
-
~
~ 2HI(, m <~,,,_Ei(Xi, Xj.) --+0
(2.22)
I ~<i~<n
n<j~<2 k
-
Equation (2.21) can be proved as follows. Since, in view of the contractive principle (see Theorem 4.4 in Ledoux and Talagrand ts)) for any
t>0,
supP(
.v
~=gjHI(llltll<~a,k}(.)(i,x ) > l I
j
I
x
~<4 max
I ~<n~<2 k
j
I
H(Xj, x) >
supP
x
j
1
then the assumption that l/a,, ~ = ~H(Xj, x ) ~ 0 uniformly in probability
implies 1/a~'~.5Z~k=~ejHI(tm, <~,,,.E)(Xj,X) --* 0 uniformly in probability. So
Eq. (2.21) follows at once as a direct consequence of Lemma 1.3(b).
For Eq. (2.22) we notice, by Jensen's inequality and decoupling
inequality given in Lemma 1.1,
Iterated Logarithm Law and Marcinkiewicz Law of Large Numbers
E
695
n 2 , HIi iim <~,,,.k,(Xi, X:)
1 ~i<~n
n<j~2
k
<~cE
eigjT[2,2HI~llHil<~a,j,.l(Xi, Xj)
~
I <~i<~n
n < j < ~ 2k
<~cE
~
eiejHI~ tlHII <~,,,.k~(Xi, Xj)
n<j~<2 k
<~cE
~
! <~i#j<~2 k
F'iej,l, HI~IIHII <<.a,_kj(Xi, Xjl,)
On the other hand, it's easily seen that
1
2ka------TkE
•
-
1 ~i#j<~2 k
x/2k
e,e) 1'HI, I,n,. > ~.,k,(X,, X)' ')
E ][HI] I~llml>,,,.kl(Xl, X2)
--,0
Therefore, Eq. (2.22) will be valid if
I _k E
2ka,
-
~
I <~i#j<~2 Ir
eieJllH(Xi, X~.' ') ~ 0
(2.23)
Since x / ~ / ~
UgH---,O in probability, then it follows from
Lemma 1.2(b) that 1/2ka,_k ~-,t <~i~:~2k eiE~"~H(X~, X~'j I) ~ 0 in probability.
This in turn together with Lemma 1.3(a) implies the required result of
Eq. (2.23) whenever
2ka, k E max
-
I~<j<~2k
i#j
~iH(Xi, Xj.) ~ 0
But by Eq. (2.7)
1
~, g i H ( X i , X j )
-E max
2ka2k 1~j<~2k i ~ j
~1
E
-
~0
~ F,i H ( X i ,
i= I
su
696
Now we are in a position to prove Eq. (2.19). Recall that
~.,. 2HI(itm <,_,,,~.)(Xi, Xj)
1 <~iv~j<~2 k
4
2"*
~ m
"
Z
E ~"-. 2HI(IIH,<~,,.v')(Xi, Xy)
N~{I.,...2
'('} i ~ N
jq~ N
Then
~ P(
k=
~,
I
lt2.2HI, iIHil<~o,.kl(Xi,Xj) >$2ka2 k)
I <~ir
k
< ~-=, P ~
~ n2..HI, ,,H, <_<.,.k,(X._
~
N~
{ I,....2 k}
> ~ 2 a2k
(2.24)
i~N
jCN
Thus, in view of Eq. (2.24) and the fact that the three random variables ~
i ~N n 2. 2HI, It*t,<,,,~)(Xi, Xy), Z l# N<i<
# Nk n~- ' 2HI, IIHII<~a2kl(Xi ' "~j) and
--j~N
<j~<2
Z~<i<~N
<~j<~2 k -
#N
n,- ' 2HI, IIm<,,,_,)(X,X) ~) have the same distributions, it is
enough for us to prove for any t > 0,
P
zz, 2HI, IIHII~<a2k)( Xi, ~)
~
-
k=l
N~{I,...,2/'}
i
jq~ N
-E
~, n2, 2HII ,,-I, <,,_,kl(Xi, Xy} > e2ka2 ~)
i~N
yen
(2.25)
<O0
We apply Lemma 2.2 for p = 2 and Eq. (2.21) to get
P
k=l
Z
-
N~
{I,....2 t'}
Z ~2, zHI, ILHIt<~,,,-*,(Xi,Xy)
i~N
]r
- E i~NTg2,2H]~llHll<<mt)(Xi, Xj) ~>~2ka2k I
jCN
Iterated Logarithm Law and Marcinkiewicz Law of Large Numbers
f ~ EI
<~f,=,
~1
--E
Z ~" zHI(,It-ttl~#2k)(Xi, Xfl
Z
- ,
N c { 1,..., 2k}
_
697
iEN
jCN
~ 7~2,2HI(IIHII<~a2kI(XI,XJ) 2
jCN
<~ f
1
,.)~,1 , .,,k
~..
E
qZ
k=lS-Z"- a~_A'L- N~{I,...,2 k}
n2,:Hl(llHll <~,,.k)(Xi, ~ )
i~N
jCN
--E[,~NT~2,2HI(IIHII<~a2k)(Xi, XY) I 2
jCN
f
C
k~l r
a2k / - N~{I,....2 k} i 1
2/`.- # N
+
2\
~rc,_.2HI(llHil~.2k)(Xi, X.~)
jft
i=l
C
2
j=l
#N
Y'. E
<~ f
(#__~E 2k-#Nn2, ",HI(tim <.,,,.,)(Xi, Xj.)
1
k
)
-
, T_k ," 2 E
~. 8jHI(tIHII<~..,?)(Xj,
k=l 8-2- ask
/lj=l
< co
(2.26)
As for Eq. (2.20) the p r o o f is similar but L e m m a 2.2 is applied for p = 4 .
Indeed, for any e > 0,
k=l
n:l
l~i~<n
n<j<~2k
-E
Z
n 2. =HI~ IIHIL<<,2k)(Xi. X fl > 82ka2k I
I ~<i<~n
n<j~<2 k
c~
k=l
-E
2/`"
Z
1
-8424k(/4
-E k
Z
re2. ,_HI( llHil~<#2k)(X i. X fl
I <~i<~n
n<j<~2k
Z
1 ~<i~<n
n<j~<2 k
r~2,
2HI(llnll~<a2kj(Xi' % ' )
4
698
Su
<- Y
k=l
-E
t=l
c
t/24ka~, n
+ (2 k -- n)
E
cr
i=l
-~t.-
c E
e,2ka4------~
k=l
-
"=
n , 2HI~ IlUll~,?I(Xi,
j=l
.<y
1 ;g2"2HIlllHII <~a?l(Xi" )~j) 4
i=1
4
-__2ejHIiimm <.a,?l(Xj, X')
j
I
(2.27)
<O0
where the convergence of this series is due to Eq. (2.21).
Up to now, summarizing the previous works we can end the proof of
Theorem 2.1.
Theorem 2.1 only presents some sufficient conditions for the LIL of
B-valued U-statistics. Some of them are not necessary. More precisely, we
have proved certain almost behaviors from statements in probability (weak
statements). Theorem 2.2 shows that in a type 2 space the statements in
probability can be understood well in terms of the moment.
Theorem 2.2. Assume that B is of type 2 space and H: S"' ---, B is a
measurable symmetric kernel function. If
(1)
lim sup E IIH(XI ..... X,,,)II2/LL IIa(gl .....
L,,)II
I~llu~x,..,,_ ...........~tl>,~ = 0
where the supremum is taken over all x = (x2 ..... x,,,) E S . . . .
(2)
1
a2=supr~Bi E(nm.,,,foH(X,))2< or, then
lim
x/~
IIU'~,H- EHI[ = ma
a.e.
If this condition (2) is replaced by
(2') { ( n l . , , f ~
', Ilfll <~ 1} is uniformly integrable, then
the compact LIL holds for U',',H with the limit set inK, where
K=K,,.,,,n~x,~ is the unit ball of the reproducing kernel Hilbert space
associated to r~.,,,H(X~).
Proof Assume m = 2. Along the line of the proof of the Theorem 2.1
it is enough for us to verify the following two statements
Iterated Logarithm Law and Marcinkiewicz Law of Large Numbers
L
eiH(Xi, x) ---,0
699
uniformly in probability
(2.28)
in probability
(2.29)
an i= 1
1
~.
e~ejH(Xi, Xj) --* 0
lT(ln 1 <~i#j<~n
We are ready to prove Eqs. (2.28) and (2.29) under the hypothesis (i) in
type 2 spaces. In fact, for each n i> 1,
~ eiH(Xi, x) ~<supE
supE
.v
i=
1
x
~ eiHI, ilull<..,,,(Xi, x)
i=
I
+ sup nE lIHIIv~lJ~>,,,(X, x)ll
(2.30)
x
Note that the assumption
lim sup E IIH(X, x)IIZ/LL lIB(X, x)ll I~tl,(x..,.) n> , ) = 0
l ~
,:,'0
A"
implies
1l
- - sup E IIHI~Hm >~,,,,~(X. x)ll ~ 0
( l tl
(2.31)
x
By the type 2 inequality,
eiHIiitz~Ii<_,,,,~(Xi,x)
--supE
r
i = I
-"
1
<~c ~
)hi2
E IIHI~It++ll<_,,,~(X, x)H 2
(2.32)
where c is a constant of type 2.
For each t > 0, the square of the right hand of this inequality is seen
to be smaller than
t2
1
+ ~
2LLn
sup E ILH I .
< , . , ~ ~,,>(X,
x)ll 2
t 2
~<2--~n + sup E IIH(X, x)IIZ/LL lIB(X, x)ll I, vz,v..,-)Jj >,)
x
letting 17, and then t, go to infinity concludes the proof Eq. (2.28).
[]
700
Su
Next we prove Eq. (2.29). Indeed we have
1
L
nan
l
e,4H(Z ~, Zj.) < ~ - - E
<~i#j~n
~.
nan
+--
e,4H(Z~, X's)
1 <~i~j<~n
1
E IIHI,, m ~,.)(Z,
Z')ll
na n
Since E liB(X, X')II'-/LL lIB(Z, Z')II < oo, then
Also, by the type 2 inequality once more,
1 E
~.
nan
1~ha,, E JlHIiiim ~.,,~(X, X')II
e,ejHI, jmrl~,.,(Xi, X))
i <~i~j<<.n
~ 0.
~LEIIH(X,X')II
an
~0
Therefore Eq. (2.29) follows, which completes the proof of Theorem 2.2. []
To end this paper we shall state the M L L N of B-valued U-statistics.
Their proofs are omitted since the basic ideas are the exactly same as that
in Theorems 2.1 and 2.2. We only refer to the early paper of de Acosta (1~
for background and details.
Theorem 2.3. Suppose that B is a separable Banach space,
is a measurable symmetric kernel function such that
g [[H(X1 ..... X,,) Hp < oo for some 1 ~<p < 2. Let {X,,, n >~ 1 } be a sequence
of independent copies of X. If
H:S"'-~B
1 ~ n(Xi, xl
1l I/p
x,,,_l)-'O
uniformly in probability,
H(X i, Xj, x, ,..., x .... z) -) 0
uniformly in probability
i=l
~'"'
111 - 1/p
~.
n(n -- 1) t ~ , , j < ~ .
11' - ~/P(n
n!
-
m)!
~ H(X,, ..... X,.,) ~ 0
in probability
Then
n 1 I/p(n ~ m ~ ~
n !
~(x~,,..., x,,,) ~ o
a.e.
Iterated Logarithm Law and Marcinkiewicz Law of Large Numbers
701
Theorem 2.4. Suppose that B is of type p for some 1 ~<p<2.
H: S m ~ B is a measurable symmetric kernel function such that
lim sup E IIH(X~, X2,-.., Xm)ll p ICll~x,..,.,.....~,,,~ll >,1 = 0
where the supremum is taken over all x = (x2 ..... x,,) e S .... 1. Then
nI
I/p(I l
n !
itl)[
H( Xi, ..... Xi,,, ) ~ 0
a.e.
i',',
ACKNOWLEDGMENTS
This paper is partially based on the author's Ph.D. dissertation submitted to Fudan University at Shanghai. The author is deeply indebted
to Professors Zhengyan Lin and Chuangrong Lu for their guidance,
invaluable discussions and encouragement during the course of the work.
The author also acknowledges many helpful suggestions and comments
from the referee.
REFERENCES
1. de Acosta, A. (1981). Inequalities for B-valued random variables with applications to the
strong law of large numbers. Ann. Prob. 9, 157-161.
2. Arcones, M. A. (1994). Limits of canonical U-processes and B-valued U-statistics.
J. Theor. Prob. 7, 339-349.
3. Arcones, M. A., and Gin~, E. (1993). Limit theorems for U-processes. Ann. Prob. 21,
1494-1542.
4. Gin~, E., and Zinn, J. (1992a). Marcinkiewicz type law of large numbers and convergence
of moments for U-statistics. In Probability in Banach Spaces, Birkhfiuser, Boston, 8,
273-291.
5. Gin~, E., and Zinn, J. (1992). On Hoffmann-Jorgensen's inequality for U-processes. In
Probability in Banach Spaces, Birkh~iuser, Boston, 8, 80-91.
6. Gin~, E., and Zinn, J. (1994). A remark on convergence in distribution of U-statistics.
Am1. Prob. 22, 117-125.
7. Kahane, J. P. (1985). Some Random Series of Functions. 2nd ed. Cambridge University
Press, Cambridge.
8. Ledoux, M., and Talagrand, M. (1991). Probability in Banach Spaces. Springer,
New York.
9. de la Pefia, V. H. (1992). Decoupling and Khintchine's inequalities for U-statistics. Ann.
Prob. 20, 1877-1892.
10. de la Pefia, V. H., and Montgomery-Smith, S. J. (1994). Bounds on tail probability of
U-statistics and quadratic forms. Bull. Amer. Math. Soc. 31, 223-227.
I1. Serfling, R. J. (1980). Approximation Theorem of Mathematical Statistics. Wiley,
New York.
860/9/3-11