Internat. J. Math. & Math. Scl.
(1985) 135-144
135
Vol. 8 No.
STRONG LAWS OF LARGE NUMBERS FOR ARRAYS
OF ROW-WISE EXCHANGEABLE RANDOM ELEMENTS
ROBERT LEE TAYLOR
Department of Statistics and Computer Science
University of Georgia
Athens, GA 30602
RONALD FRANK PATTERSON
Department of Mathematics
Georgia State University
Atlanta, GA 30303
(Received October 1, 1984)
Let
ABSTRACT.
{Xnk
_<
k
_<
n, n
_>
1} be a triangular array of row-wise exchangeable
random elements in a separable Banach space.
!
Xnk,
The almost sure convergenceof,
-1/
=1
p < 2, is obtained under varying moment and distribution conditions on the
random elements.
In particular, strong laws of large numbers follow for triangular
arrays of random elements in (Rademacher) type p separable Banach spaces.
Consis-
tency of the kernel density estimates can be obtained in this setting.
KEY WORDS AND PHRASES. Exchangeability, Random Elements, Laws of Large Numbers,
Almost sure Convergence, Martingales, Rademacher type p, and Kernel Density Estimates.
1980 MATHEMATICS SUBJECT CLASSIFICATION CODES.
1.
Primary 60B12, Secondary 60Bli.
INTRODUCTION AND PRELIMINARIES.
Blum et al.
[1] obtained
central limit theorems for arrays of exchangeable
random variables using a version of de Finetti’s theorem which implied that an infinite sequence of exchangeable random variables is a mixture of sequences of indepen-
dent, identically distributed random variables.
Taylor [2] used similar techniques
in obtaining weak and strong convergence results for arrays of random elements which
are row-wise exchangeable.
Using martingale methods, Weber
[3] developed central
limit results for triangular arrays of random variables which were row-wise exchange-
able.
tlis methods did not require infinite exchangeability or the de Finetti repre-
sentation.
-1/p n
n
In this paper, almost sure convergence is obtained for
Ek= Xnk
in separable Banach spaces using martingale methods.
--1Z=
n
Xnk
and
These results
R. L. TAYLOR AND R. F. PATTERSON
136
are for triangular arrays, and hence only finite exchangeability in each row is re-
quired.
By assuming convergence in mean for each column, the hypothesis of the pre-
viously cited limit theorems are substantially relaxed.
Let E denote a real
separable
Banach space with norm
II II.
Let (, A, P) denote
A random element X in E is a function from
a probability space.
into E which is
A-measurable with respect to the Borel subsets of E, 8(E). The pth absolute moment
p where E is the
of a random element X is
expected value of the (real-valued)
EIIXII
random variable
(when E
11XII
IXI p.
The expected value of X is defined to bc the Bochner integral
< ) and is denoted by EX.
The concepts of independence and identical
distributions (i.i.d.) have direct extensions to E.
The random elements {X
are said to be exchangeable if the joint probability law of (X
of {I
tation invariant, that is, for each permutation
Xn Bn
P[X c B 1,
B
for each B
not conversely.
n
c B(R).
is permu-
n}
Xn c Bn
e B
P[XI
Xn)
...X
(1.1)
Clearly i.i.d, random elements are exchangeable but
Moreover, letting
B
Bk+
R for
< k < n in (1.1) shows
that all joint probability laws are the same and that exchangeable random elements
are identically distributed.
P(B)
2.
DB satisfies
Finally, a subset B of E whose boundary
0 is called a P-continuity set.
STRONG LAWS OF LARGE NUMBERS FOR TRIANGULAR ARRAYS.
The main result of this section is a strong law of large numbers.
Moment con-
ditions and a measure of nonorthogonality condition will be assumed on the distributions of the random elements.
Throughout this section
{Xnk"
_<
k
_<
n, n
_>
i} will
denote an array of random elements in a separable Banach space E which are row-wise
exchangeable.
First, two preliminary results will be presented for later use in Sections 2 and
3.
The first result shows that the infinite sequence, formed by the convergence in
rth mean of each column of the triangular array, is exchangeable when each row consists
of exchangeable random elements.
de Finetti’s
LEIA I.
This allows the application of a version of
theorem to the limit sequence.
Let
{Xnk
row-wise exchangeable.
_< k _<
n, n
_>
i} be an array of random elements which are
If the random elements converge in the rth mean to
for each k, then the sequence
{Xook"
k > i} is exchangeable.
Xook(r
> O)
STRONG LAWS OF LARGE NUMBERS FOR ARRAYS
PROOF"
Xnk).
(Xnl
(
llence
5
i
5
If
X
nl
k,
X
(XI
P[Xnl
([4] pp. 26-27),
k).
Xk
A
Ak]
c
Hence,
Xk).
(XI
continuity set A
P[X
A
Xn k
Xn k
of !,
Xk
A
Ak].
e A k] for
e A k]
Ak which are
PX -continuity
Since the limits are unique, and the
sets.
PX.
for all A
as n
and the vector
Xnk)
Xnk A k]
Ak]
P[Xn
P[Xn e A
e
Xnk
of (1
each permutation
.k
nn}
(Xnl
For each
A[
e A
P[Xnl
By exchangeability,
[X
i
nk)
< k, then
<
X
Xni
Xnk}C {Xnl
{Xnl
Consider the set
137
PXl-continuity
sets form a determining
lass, it follows that
B
l’[Xl
Bk]
8(Ek).
\ook
B k) e
for all (B
identical joint distributions,
REMARK.
Xn k
B
l’[Xoovl
E-
Xok)
(Xoo
Thus,
tlence, the sequence
u B
k]
(XI
and
(Xok"
k
>_
Xok)
have
1} is exchangeable. ///
Note that the convergence of the joint distributions is sufficient in
the proof of Lemma 1.
Unfortunately, this is not implied by convergence in distri-
bution in each column.
For arrays where each row is an infinite sequence of exchangeable random elements,
[5] showed that de
Olshen
; P(Bn)dn (Pv)
P(Bn
where
Finetti’s theorem implied that for each n
(2.1)
F denotes the
collection of probabilities on the Borel subsets of E and P (B)
n
m
is the probability of
E is a Borel function)
(where g E
Bn
[g(Xnl
Bn]
Xnm)
computed under the assumption that
{Xnk"
k
>_
l} are independent, identically distri-
buted random elements and V is the mixing measure defined on
n
shows that if
{Xnk" _< k _<
n, n
_>
E[f(Xl) f(X2)
P
Ev(Xol)
Let
{Xnk"
_<
k
_<
E[f(Xnl f(Xn2)]
0 where E
Moreover, it also follows that EX
LEMMA 2.
n, n
_>
Xk
for each k.
n (f)
If for each f
k[f(Xnl)f(Xn2)
then
(i)
E[f(Xool)f(X2)]
0
0 as n
then
v is the expectation with respect to
0.
I} be an array of row-wise exchangeable
random elements in a separable Banach space such that
mean to
The next result
l} are row-wise exchangeable random elements which
converge in the second mean for each k and
0 and
8(F).
E*
0 as n
,
{Xnk}
converges in the second
R. L. TAYLOR AND R. F. PATTERSON
138
(iii)
Ev(XI)
E(XI)
PROOF.
Since
(ii)
Pv with
for
0
U-probability one,
O.
2
Xnk Xk for each k it
2
El]nkll 2 < and EIIXkl] <
sup
n
for each k.
and
is clear that
(2.2)
Using (2.2) and convergence in the 2nd mean, for each f g E*
]E[f(Xnl)e(Xn2) E[f(Xl)f(X2)]l
< [E[e(Xnl)f(Xn2)
E[f(Xl)f(Xn2)]]
[E[f(Xool)f(Xn2)] E[f(Xool)f(X2)]]
< E(If(Xnl)
f(X=ol) llf(Xn2)l)
E([f(Xn2) f(xoo2) llf(Xol)l)
_< Ilf112[E(I IXnl Xml IXn2 II) E(I IXn2 xallllxll)]
<
Ilfll2[(Ellxnl Xoolll2)l/2(EllXn2ll2) 1/2
(EllXn2-
xoo2ll2)l/2(EllXooll]2) 1/2]
by hypothesis (i).
which goes to 0 as n
E[f(XI)f(X2)]
(2.3)
Thus,
0.
Since convergence in the second mean implies convergence in mean, it follows from
Lemma
that the sequence
{Xk
k > l} is exchangeable.
Also
it follows from (2 i)
that
E[f(Xol)f(Xo2)
0
;F E0[f(Xool )f(xoo2)]
f
F
which implies that
[E,f (Xl )]2 d(Pv)
Evf(Xl)
0
p
a.s. for each f
E*.
Since the Bochner integral
implies the Pettis integral in a separable space,
f(Ev(Xool))
for each f
a s
which implies that
and the separability of E.
EXI
REblARK.
Ev(Xoo l)
0
Voo
E(f(Xool)
0
oo
a.s.
by the Hahn-Banach theorem
Also,
fF Ev(XI) d(Pv)
O.
///
Note from (2.3) in the proof of Lemma 2 that for uniformly bounded
random elements, convergence in the mean suffices.
It is also interesting to note
139
STRONG LAWS OF LARGE NUMBERS FOR ARRAYS
that the conclusions of Lemma 2 do not necessarily imply that
sequence of
{Xk
k > 1} is a
i.i.d, random elements.
The final result of this section is a strong law of large numbers for triangular
arrays of random elements which are row-wise exchangeable.
U
U
O{
nn
Z
n+l
Xnk’ Zk=l X(n+l)k
=l
Z
o{o{Z= Xok
n
+I
[,I
U
(2.4)
nn
Ix.t xtll
2
Xnk Xk
and
Xmk
It can be easily shown that if
Define
Ix (n+l),l XI[
>
for each n and
for each k, then
g(I
IXnk Xooklllu)
0 a.s.
(2.5)
IIsing (2.4), a modification of Kingman’s
[6] and
n
Zk= Xnk E(Xnl[Unn) E(Xnl]Um)
n
Zk=l Xook
=E (Xcol
Uon)
[3]
Weber’s
results show that
a.s. and
as
(2.6)
{Xnk" _< k _< n, n _> i} be an array of random elements in a
separable Banach space. Let {Xnk} be row-wise exchangeable fo- each n and let
Unn and Uoon be the o-fields defined in (2.4). Let {Xnk} converge in the second mean
for each k and [[Xnl
Xl[[ _> [[X(n+l ),l Xl[[ for each n. If
THEOREM I.
Pn (f)
Let
E[f(Xnl)f(Xn2)]
for each f
0 as n
E*,
then
n
I1 Zk=
PROOF"
Let
random elements.
p[supl
n>m
n
Xnk II
+0 a s
2
Xnk Xk.
> e] <
p[supl
n>m
l Zk= Xnk- Zk=n
n
p[supl In
Ik=
n>m
P[supl
n>m
k > i} is an exchangeable sequence of
> 0, and by (2.6)
Thus for
I Zk= Xnkll
{Xk
By Lemma I,
I
IE(XnllUoon)
p[supl
n>m
n
I Ek= Xookl
Xook,,
Xk
lu)l
F.(Xoo
>
c
g]
> E
1
R. L. TAYLOR AND R. F. PATTERSON
140
P[supl
n>m
IE(Xnl Xoo lu)l
P[supl
n>m
n
>
1
g
I Ek=
Xmk[I > ].
(2.7}
Now by (2.1),
’[supllg
Ek= 1Xk
n>m
where
{Xk
respect to
pv[supll
Zk= 1Xmk
n>m
1
1
dBm(Pg),
k > 1} are independent, identically distributed random elements with
P.
By Lemma 2,
E(XI)
O, with -probability one and it follows from
Mourier’s strong law of large numbers for random elements that for almost each P
Zn
11 > -]c
Pv [sup]
n>m 1 k=l Xk
oo.
0 as m
ttence, by the bounded convergence theorem,
p[supl
n>m
n
l :k=l Xook
;" p
F
goes to 0 as m
n
[supllg
Zk=
n>m
.
xkll
>
o
xll lUnn)
E(I [Xnl
By (2.s),
(2.8)
g]
a.s.
Thus,
p[supl
n>m
IE(Xnl xo llu)ll
< P[sup E(
n>m
1
>
(2.9)
IXnl xtl
lug)
>
1
0 as m
Combining (2.7), (2.8) and (2.9), it follows that
p[supl
n>m
n
I Y’k=
Xnk II
>
C]
0 as
111
+oo,
or that
n
I1 Zk= Xnk 11
3.
III
+Oas
STRONG LAWS OF LARGE NUMBERS IN TYPE p SPACES
In this section, strong laws of large numbers for triangular arrays of row-wise
exchangeable random elements in type p
ed.
8 separable Banach spaces will be establish-
Recall that a separable Banach space is said to be of type p,
p
2, if
141
STRONG LAWS OF LARGE NUMBERS FOR ARRAYS
there exists a constant C such that
n
El IEk:
p <
C
Xkl
n
Ek= El iXkl ip
I’
for all independent random elements
moments.
Xn with
Every separable Banach space is type I.
zero means and finite pth
[7]
The next result by Woyczynski
for sequences of zero mean, independent random elements in E with uniformly bounded
tail probabilities is listed for future reference.
< p < 2.
Let
TtlEOREM 2.
The following properties of a Banach space E are
equivalent"
6
(i)
E
(ii)
For any sequence {X.} of zero mean, independent random elements in E wtth
is of type p
uniformly bounded tail probabilities
lip
the erie
verges a.s.
For any sequence {X.} as in (ii)
(iii)
n
lln- 1/p Ek=
(Xk
Since
xll
Oa
k > l} are exchangeable
they are identically distributed
That is, for all t g R and k
they have uniformly bounded tail probabilities.
llxll
gllXlll
> t) <
> t).
tlence
1,
a strong convergence result for row-wise
Tus
exchangeable in type p separable Banach spaces can be obtained using Theorem 2 and
the techniques of Theorem 1.
THEOREM 3.
(Xnk" _< k _< n, n _>
Let
n and let U
nn
and U
on
<_
6,
separable Banach space of type p
(ii)
pn(f)
(iii)
E[f(Xnl)f(Xn2)]
n
ii1/nl/p Ek=l
Xnk II
PROOF"
Since
p < 2.
Let
{Xnk}
Ell Xnl
Xooll
where a
be exchangeable for each
If
be the o-fields defined in (2.4).
[IXnl Xl]l IX(n+l),
2
E
IXnl Xok]l o(n-2a),
(i)
I} be an array of random elements in a
for each n,
(p-1)/p, and
for each f
0 as n
g
E*, then
0 a s
2
Xoo II
o(n-2e),
2
then
Xnk Xk.
By Lemma 1,
R. L. TAYLOR AND R. F. PATTERSON
142
(Xk"
1} is an infinite exchangeable sequence of random elements.
k
Thus, for
> O, and by (2.6)
p[supl
n>m
n
In- 1/p rk=
Xnk II
l’[sup[
<
n>m
>
1
n
n
In- 1/p (Tk=
Xnk gk= Xk)
P[sup[
n>m
n
P[suPlln (1_n gk=l
n>m
n>m
P[supl
n>m
Xnk
ln- 1/p
n
a
g
In (Xnl Xl
P[sup[
n>m
-]
,n
In -1/p 1k=1
a
p[sup]
>
n
n- 1/p Ek=
Xook[
>
5].
By (2.1)
p[supl
n>m
where
n
In- 1/p lk=
Xk II >1
{Xk
k
By Lemma 2,
a.s.
_>
Pv[supl
n
-1/p
n
l:k= Xook i1>]
n>m
d(P,v),
1} are independent and identically distributed with respect to
Ev(Xook)
0.
which implies that
for p
Thus, by Theorem 2 (Theorem
n
Pv[supl In -1/p r.k=
Xk [I
n>m
]
>
0 as m
1)
.
Pv.
In- 1/PF.= 1Xok
By the bounded
convergence theorem,
InfF p[supl
n>m
1/p
n
gk=
xk
>
g
1 d(Pv)
oo.
0 as ra
In a manner similar to (2.5), it can be shown that
nal .IE(Xnl Xl...
IUn)
0 a.s.
This implies that
p[supl
n>m
Inc E(Xnl Xoo lu)ll > -I
o
as m
Hence, it follows that
[sup[
P
n>m
n
In- 1/p Y.k=
Xnk l[
> g]
0 as m
or that
n
In- 1/p Ek=
Xnk ii+Oas
REMARK.
It should be noted that if
///
lXnl xlll
0 a.s., then the condition
0
STRONG LAWS OF LARGE NUMBERS FOR ARRAYS
I[Xnl col[
3.
Xcol[
IIX(n+l),
lXnl Xml II
That is
for all n is not needed in the proof of Theorem
IE(Xnl Xco IUoon) II
0 a.s. implies that
0 a.s.
0 a.s. which
It is sometimes easier to show directly that
is crucial to the proof of Theorem 3.
ICcxt- Xn) lUn)l
143
as will be demonstrated in the kernel density esti-
mation example to follow.
The following example considers the general density estimation problem where
X
X,,
are independent and identically distributed random variables with the
same density function f.
EYuMPLE.
Let X
X
be independent
-
bles with common density function f.
bandwidths h
n
is given by
t-X
n
f (t)
n
gk=l
K(
identically distributed random varia-
The kernel estimate for f with constant
k
where K is often chosen to be a bounded (integrable) kernel with compact support
[a,b] and h n
and Wagner
For additional background material
[8] and Taylor [9].
{gig" R
E
co.
0 an n
-
Let
R and
Thus, E is a separable Banach space of type min{2,p}.
Xnk
n
t-Xk
each n.
E and that
Xnk
(3.1)
{Xnk}
is independent and henceexchangeable for
The next proposition will prove directly that
In this setting
XI
Define
t-Xl
E[K(--ff)])
n
(K(--ff---)
n
It is clear that
see l)eVrove
[IE(^nl [Umn)[
0 a.s.
0 a.s.
t-X
PROPOSITION 4.
Let
k
(K(--fi---)
n
n
Xnk
t-X
E[K(--ff---)]).
n
Then,
I(XnlU)ll
0
completely (and hence almost surely)
PROOF
Let U
{g =1 Xnk’ g;1
nn
0 a.s. for each k.
Xk
X
U
U
by (2 4) and since
oon
nn
(n+l) ,k
> O, q _> 1, by Markov’s inequality, Tonelli’s theorem
For
and (2.6)
P[llECXnllUnn) ll
g
<
2qE[(
2qE
>
] < e -2q
In--n
n
In
(K
k=l
fl Zk=l
EllECXnllUnn) ll 2q
t-x
(-V---..)
n
t-Xl
(1h K(-----)
n
n
t-x
E (K (]---)
p
dt)
l/p]2q
n
E
(n
t-Xl
K( t-----)) 2q dt]
n
R.L. TAYLOR AND R. F. PATTERSON
144
t-Xl
2q
(Using that R is of type 2 and (2.8) of Taylor
[9])
<_ 2q /co
f nqn
< e -2q
c
-2q
t-Xl
n
E] Ek=
n
2q
n- q E foo
I
t-X1
K(----)
n
(h2nn) q
o(n
which implies
E(n
-d)
0 < d <
t-Xl
dt
K(7))]2q
n
-
there exist q such that
Zn=l P[[ [E(Xnl]Unn)[[
completely to O.
Depending on the choice of K,
example).
dt
E[2(bddk)2q(b a)hn(hn )-2q]
C h
Letting h n
dt]
t-Xl 21)
t-Xl E( K(-T-))]
E(]n K(-----)n
n
n
n
-2qn-q
E(n K(T))I
n
K(--fi---)
n
> ] <
Hence,
2n
Y.n=l hn/(_h n_q)
]]E(nl[Unn)[[
converges
II!
IlXnlll
may not converge to 0
Xl
in the
of p being
Also, Theorem 3 and the example emphasizes the importance
as large as possible (< 2).
Moreover, R, R
m,
ltilbert spaces and all finite-dimen-
are type p
sional Banach spaces are of type 2, and consequently they
p<2.
<
for each
new
Finally, it is important to observe that the resuIts are substantial
results even for real-valued random variables.
REFERENCES
I.
BLUM, CHERNOFF, ROSENBLATT, AND TEICHER (1958). Central limit theorems for
exchangeable process. Canadian J. of Mathematics, IO, 222-229.
2.
TAYLOR, R.L. (to appear).
Laws of large numbers for dependent random variables.
Colloquia Mathematica LIMIT THEOREMS IN PROBABILITY AND STATISTICS.
3.
WEBER, N.C. (1980). A martingale approach to central limit theorems for exchangeable random variables. Jour. of Applied Prob. 17, 662-673.
4.
BILLINGSLEY, P. (1968).
5.
OLSHEN, R. (1974).
6.
KINGMAN, J.F.C. (1978).
7.
WOYCZYNSKI, W.A. (1980). On Marcinkiewicz-Zygmund laws of large numbers in
Banach spaces and related rates of convergence. Probab. and Math. Statist.,
i, 117-131.
8.
DeVROYE, L.P. and WAGNER, T.J. (1979). The L
1136-1139.
estimates. Ann. Stat.,
9.
TAYLOR, R.L. (1982). Convergence of weighted sums of arrays of random elements
in type p spaces with application to density estimation.
Sankhya A. 44,
341-351.
Conversence
of Probability Measures.
A note on exchangeable sequences.
stheorie und Verw. Gebiete 28, 317-321.
Uses of exchangeability.
Wiley, New York.
Z. Wahrscheinlichkeit-
Ann. Prob., 6, 183-197.
Convergence of kernel density