Asymptotic normality of urn models for
clinical trials with delayed response
Feifang Hu1 and Li-Xin Zhang2
University of Virginia and Zhejiang University
1 Research
supported by a grant R-155-000-015-112 from the National University of Singapore
and a grant from University of Virginia.
2 Research
supported by National Natural Science Foundation of China (No. 10071072) and
National Natural Science Foundation of Zhejiang Province.
AMS 1999 subject classification. Primary 62G10; Secondary 60F15, 62E10.
Key words. Asymptotic normality, generalized Friedman’s urn, staggered entry, treatment
allocation, urn models.
Abbreviated Title: Asymptotic normality of urn models
Abstract
For response-adaptive clinical trials using a generalized Friedman’s urn design, we derive the
asymptotic normality of sample fraction assigned to each treatment under staggered entry and
delayed response. The rate of convergence and the law of the iterated logarithm are obtained for
both the urn composition and the sample fraction. Some applications are also discussed.
1
1
Introduction
Response-adaptive designs involve the sequential selection of design points, chosen depending on
the outcomes at previously selected design points, or treatments. Since the fundamental paper
of Robbins (1952), adaptive designs have been extensively studied in the literature. Rosenberger
(1996) provides a good review of adaptive designs.
An important family of adaptive design can be developed from the generalized Friedman’s
urn (GFU) model [cf. Athreya and Karlin (1968) and Rosenberger (2002)]. It is also called the
generalized Pólya urn (GPU) in the literature. The model is described as follows. Consider an
urn containing particles of K types, respectively representing K ‘treatments’ in a clinical trial.
Initially the urn contains Y0 = (Y01 , . . . , Y0K ) particles, where Y0k denotes the number of particles
of type k, k = 1, . . . , K. At stage i, i = 1, . . . , n, a particle is drawn from the urn and replaced.
If the particle is of type k, then the treatment k is assigned to the i-th patient, k = 1, . . . , K,
i = 1, . . . , n. We then wait for observing a random variable ξi , the response of the treatment at
the patient i. After that, an additional Dk,q (i) particles of type q, q = 1, . . . , K are added to the
urn, where Dk,q (i) is some function of ξi . This procedure is repeated through out the n stages.
After n stages, the urn composition is denoted by the row vector Yn = (Yn1 , . . . , YnK ), where Ynk
represents the number of particles of type k in the urn after the n-th addition of particles. This
relation can be written as the following recursive formula:
Yn = Yn−1 + Xn Dn ,
(1.1)
¡
¢K
where Dn = D(ξn ) = Dk,q (n) k,q=1 is an K × K random matrix with Dk,q (n) being its element
in the k-th row and the q-th column, and Xn is the result of the n-th draw, distributed according
to the urn composition at the previous stages, i.e., if the n-th draw is type k particle, then the
k-th component of Xn is 1 and other components are 0.
Furthermore, write Nn = (Nn1 , . . . , NnK ), where Nnk is the number of times a type k particle
drawn in the first n stages. In clinical trials, Nnk represents the number of patients assigned to
the treatment k in the first n trials. Obviously,
Nn =
n
X
k=1
2
Xn .
(1.2)
¡
¢K
Moreover, denote Hi = E(Dk,q (i)) k,q=1 , i = 1, . . . , n. The matrices Di ’s are called the
addition rules and Hi ’s the generating matrices. A GFU model is said to be homogeneous
if Hi = H for all i = 1, . . . , n.
Athreya and Karlin (1968) first considered the asymptotic properties of the GFU model with
homogeneous generating matrix. Smythe (1996) defined the extended Pólya urn model (EPU) (a
special class of GFU) and considered its asymptotic normality. For nonhomogeneous generating
matrices, Bai and Hu (1999) establish strong consistency and asymptotic normality of Yn in GFU
model.
Typically, clinical trials do not result in immediate outcomes, i.e., individual patient outcomes
may not be immediately available prior to the randomization of the next patient. Consequently,
the urn can not be updated immediately, but can be updated when the outcomes become available
(See Wei (1988) and Bandyopadhyay and Biswas (1996) for further discussion). A motivating
example was studied in Tamura, Faries, Andersen, and Heiligenstein (1994). Recently, Bai, Hu
and Rosenberger (2002) established the asymptotic distribution of the urn composition Yn under
very general delay mechanisms. But they did not obtain the asymptotic normality of the sample
fractions Nn .
In a clinical trial, Nn represents the number of patients assigned to each treatment. The
asymptotic distribution of Nn is essential to determine the required sample size of adaptive designs
(Hu, 2002). In this paper, we establish the asymptotic normality of Nn with delayed responses. We
also obtain the law of the iterated logarithm of both Yn and Nn . The technique used in this paper
is an approximation of Gaussian processes, which is different from the martingale approximation in
Bai, Hu and Rosenberger (2002). Some applications of the theorems are discussed in the remarks.
In the models which we consider in this paper, the urn will be updated by adding a fixed total
number β (β > 0) of balls into the urn whenever an outcome becomes available. The randomized
Pólya urn (RPU) design of Durham, Flournoy and Li (1998) is not of such cases, where the number
of balls added into the urn after an experiment can be zero and the urn composition only changes
after each success. We are not sure whether or not our techniques can be used to study the RPU
design with delayed responses.
3
2
Main Results
Typically, the urn models are simply not appropriate for today’s oft-performed long-term survival
trials, where outcomes may not be ascertainable for many years. However, there are many trials
where many or most outcomes are available during the recruitment period, even though individual
patient outcomes may not be immediately available prior to the randomization of the next patient.
Consequently, the urn can be updated when outcomes become available. We consider such trials,
and assume that the delay times of the outcomes are not very long comparing with the time
intervals for patients to entry to the trail. More clearly, if we denote an indicator function δjk ,
j < k, that takes the value 1 if the response of patient j occurs before patient k is randomized and
¡
¢
0 otherwise, then we will assume that P (δjk = 0) = o (k −j)−γ as k −j → ∞ for some γ > 0 (See
Assumption 2.1). Also, in clinical trials, it is more reasonable to think of censored observations
when one considers delayed response, since we often do not wait a long time for a response to
occur. However, in this paper we pay our main attention to the asymptotic properties, and so
we assume each response will ultimately occur even though its occurrence may take a very long
time. We would rather left the trials with censored observations to future studies, since censored
observations will make the model more complex.
To describe the model clearly, we use the notation of Bai, Hu and Rosenberger (2002). Assume
a multinomial response model with responses ξn = l if patient n had response l, l = 1, . . . , L, where
{ξn , n = 1, 2, . . .} is a sequence of independent random variables. Let Jn be the treatment indicator
for the n-th patient, i.e., Jn = j if patient n was randomized to treatment j = 1, . . . , K. Then
Xn = (Xn1 , . . . , Xnk ) satisfies XnJn = 1 and all other elements 0. We assume that the entry time
of the n-th patient is tn , where {tn −tn−1 } are independent for all n. The response time of the n-th
patient on treatment j with response l is denoted by τn (j, l), whose distribution can depend on
both the treatment assigned and the response observed. Let Mjl (n, m) be the indicator function
that keeps track of when patients respond, and it takes the value 1 if tn + τn (j, l) ∈ (tn+m , tn+m+1 ]
m ≥ 0, and 0 otherwise. By definition, for every pair of n and j, there is only one pair (l, m)
such that Mjl (n, m) = 1 and Mjl0 (n, m0 ) = 0 for all (l, m) 6= (l0 , m0 ). Also, for fixed n and j, if
the event {ξn = l} occurs, i.e., the response of the n-th patient is l, then there is only one m such
4
that Mjl (n, m) = 1; while if the event {ξn = l} does not occur, then Mjl (n, m) = 0 for all m.
Consequently,
∞
X
Mjl (n, m) = I{ξn = l} for j = 1, . . . , K, l = 1, . . . , L, n = 1, 2, . . . .
(2.3)
m=0
We can define µjlm (n) = E{Mjl (n, m)} as the probability that the n-th patient on treatment j
with response l will respond after m more patients are enrolled and before m + 1 more patients
are enrolled. Then
∞
X
µjlm (n) = P (ξn = l) for j = 1, . . . , K, l = 1, . . . , L
m=0
and
X
µjlm (n) = 1 for j = 1, . . . , K.
l,m
If we assume that {Mjl (n, m), n = 1, 2, . . .} are i.i.d. for fixed j, l, and m. Then µjlm (n) = µjlm
does not dependent on n.
For patient n, after observing ξn = l, Jn = i, we add dij (ξn = l) balls of type j to the urn,
P
where the total number of balls added at each stage is constant; i.e., K
j=1 dij (l) = β, where β > 0.
Without loss of generality, we can assume β = 1. Let D(ξn ) = (dij (ξn ), i, j = 1, . . . , K). So, for
given n and m, if Mjl (n, m) = 1, then we add balls at the n + m-th stage (i.e., after the n + m-th
patient is assigned and before the n + m + 1-th patient is assigned) according to Xn D(l). Since
P
0
Mjl0 (n, m) = 0 for all l0 6= l, so Xn D(l) = L
l0 =1 Mjl0 (n, m)Xn D(l ). Consequently, the numbers
of balls of each type added to the urn after the n-th patient is assigned and before the n + 1-th
patient is assigned are
Wn =
n−1
X
L
X
MJn−m ,l (n − m, m)Xn−m D(l) =
m=0 l=1
n X
L
X
MJm ,l (m, n − m)Xm D(l).
m=1 l=1
Let Yn = (Yn1 , . . . , YnK ) be the urn composition when the n + 1-th patient arrives to be randomized. Then
Yn = Yn−1 + Wn .
5
(2.4)
Note (2.3). By (2.4) we have
Yn − Y0 =
=
=
=
=
n
X
Wk =
k=1
n
X
n
X
n X
k X
L
X
MJm ,l (m, k − m)Xm D(l)
k=1 m=1 l=1
L
X
MJm ,l (m, k − m)Xm D(l)
m=1 k=m l=1
n n−m
L
X
XX
MJm ,l (m, k)Xm D(l)
m=1 k=0 l=1
n X
L X
∞
X
L X
n
X
m=1 l=1 k=0
l=1 m=1 k=n−m+1
MJm ,l (m, k)Xm D(l) −
n X
L
X
L X
n
X
I{ξm = l}Xm D(l) −
m=1 l=1
=
n
X
∞
X
∞
X
MJm ,l (m, k)Xm D(l)
MJm ,l (m, k)Xm D(l)
l=1 m=1 k=n−m+1
Xm D(ξm ) −
m=1
L X
n
X
∞
X
MJm ,l (m, k)Xm D(l).
l=1 m=1 k=n−m+1
That is
Yn = Y0 +
n
X
Xm D(ξm ) + Rn ,
(2.5)
m=1
where
Rn = −
n
L X
X
∞
X
MJm ,l (m, k)Xm D(l).
(2.6)
l=1 m=1 k=n−m+1
If there is no delay, i.e., MJm ,l (m, k) = 0 for all k ≥ 1 and all m and l, then Rn = 0 and (2.5)
reduces to
Yn = Y0 +
n
X
Xm D(ξm ),
(2.7)
m=1
which is just the model (1.1). We will show that Rn is small. So, it is natural that stochastic
staggered entry and delay mechanisms do not affect the limiting distribution of the urn. But,
it shall be mentioned that the distance between Yn in (2.5) and that in (2.7) is not Rn , since
the distributions of Xn (with and without delayed responses) are different. So, the asymptotic
properties of the model when delayed responses occur do not simply follow from those when
delayed responses do not appear.
For simplicity, we assume that the responses {ξn , n = 1, 2 . . .} are i.i.d. random variables and
£
¤
let H = E D(ξn ) , i.e., we only consider the homogeneous case. For the nonhomogeneous case, if
6
there exist Vqij , q, i, j = 1, . . . , K and H such that
Cov{dqi (ξn ), dqj (ξn )} → Vqij ,
q, i, j = 1, . . . , K
and
n
X
£
¤
kE D(ξn ) − Hk = o(n1/2 ),
n=1
then we have the same asymptotic properties.
To give asymptotic properties of Nn , first we will require the following assumptions (same as
Assumption 1 and 2 of Bai, Hu and Rosenberger (2002)):
Assumption 2.1 For some γ ∈ (0, 1),
∞
X
µjli (n) = o(m−γ ) unformly in n.
i=m
The left of the above equality is just the probability of the event that the n-th patient on treatment
j responses l after at least another m patients are assigned.
Assumption 2.1 is for us to assume that the response times are not very long comparing with
the entry time intervals. This assumption is satisfied if the response time τn (j, l) and the entry
time tn satisfy (i) t2 − t1 , t3 − t2 , · · · , are i.i.d. random variables with E(t2 − t1 )2 < ∞, and (ii)
0
supn E|τn (j, l)|γ < ∞ for each j, l and some γ 0 > γ (See Bai, Hu and Rosenberger (2002)). So
this assumption is widely satisfied.
Assumption 2.2 Let 1 = (1, . . . , 1). Note H10 = 10 . We assume that H has the following Jordan
decomposition:
T−1 HT = diag[1, J2 , . . . , Js ],
where Js is a νt × νt matrix, given by

λ 1 0
 t

0 λ 1
t



Jt =  0 0 λt

.
..
..
 ..
.
.


0 0 0
7

...
...
...
...
...
0


0



.
0


.. 
.

λt
Let v be the normalized left eigenvector of H corresponding to its maximal eigenvalue 1. We
may select the matrix T so that its first column is 10 and the first row of T−1 is v. Let λ =
max{Re(λ2 ), . . . , Re(λs )} and ν = maxj {νj : Re(λj ) = λ}.
£
We also denote H = H − 10 v, Σ1 = diag(v) − v0 v and Σ2 = E (D(ξn ) − H)0 diag(v)(D(ξn ) −
¤
H) . We have the following asymptotic normality for (Yn , Nn ).
Theorem 2.1 Under Assumptions 2.1 and 2.2, if γ > 1/2 and λ < 1/2, then
D
n1/2 (Yn /n − v, Nn /n − v) → N (0, Λ),
where the 2K × 2K matrix Λ depends on Σ1 , Σ2 and H, and is specified in (3.14).
Remark 2.1 The covariance matrix Λ has a very complicated form in Theorem 2.1 under the
general conditions. If the matrix H has a simple Jordan decomposition, the explicit form of Λ can
be obtained. Based on this asymptotic covariance matrix, we can determine the required sample
size of a clinical trial as Hu (2002).
Remark 2.2 Assume λ < 1/2 and γ > 1/2. From the proof of Theorem 2.1, Λ22 (the asymptotic
covariance matrix of n−1/2 Nn ) is a limit of
n
−1
n
X
0
Bn,m Σ1 Bn,m
−1
+n
m=1
n−1
X
n¡ n−1
X
m=1
j=m
¢0 ¡ X 1
¢o
1
Bj,m Σ2
Bj,m
j+1
j+1
n−1
j=m
n
¤
£X
= n−1 (I − v0 1)
B0n,m Σ1 Bn,m (I − 10 v)
−1
+n
where Bn,i =
0
m=1
n−1
X
n¡ n−1
X
m=1
j=m
(I − v 1)
Qn
j=i+1 (I
¢ ¡X 1
¢o
1
B0j,m Σ2
Bj,m (I − 10 v),
j+1
j+1
+ j −1 H) and Bn,i =
n−1
j=m
Qn
j=i+1 (I
+ j −1 H).
We may estimate Λ22 based on the following procedure:
P P
(i) Let na = ni=2 i−2
m=0 MJi−m−1 ,ξi−m−1 (i − m − 1, m) represent the total number of responses
before the n-th stage. Estimate H by
b = n−1
H
a
n X
i−2
X
MJi−m−1 ,ξi−m−1 (i − m − 1, m)diag(Xi−m−1 )D(ξi−m−1 ),
i=2 m=0
8
where MJi−m−1 ,ξi−m−1 (i − m − 1, m), Xi−m−1 , and D(ξi−m−1 ) are observed during the trial.
(ii) Let Wi be the number of balls added to the urn after each response, which are observed during
the trial. Estimate Σ1 and Σ2 by
b 1 = diag(Yn /|Yn |) − Yn0 Yn /|Yn |2
Σ
and
b 2 = n−1
Σ
a
n
X
(Wi − W̄)0 diag(Yn /|Yn |)(Wi − W̄),
i=1
respectively, where W̄ =
n−1
a
Pna
i=1 Wi
and |Yn | =
PK
j=1 Ynj
is the total number of particles in
the urn after the n-th draw.
−1 b
b n,i = Qn
(iii) Define B
j=i+1 (I + j H) and estimate Λ22 by
b 22 = n−1 {(I − (Yn0 /|Yn |)l)[
Λ
n
X
0
b0 Σ
b b
B
n,m 1 Bn,m ](I − l Yn /|Yn |)
m=1
+(I − Yn0 /|Yn |)l)
n−1
X
n−1
n n−1
o
X
X
−1 b 0
b
[
(j + 1) Bj,m ]Σ2 [
(j + 1)−1 B̂j,m ](I − l0 Yn /|Yn |) .
m=1
j=m
j=m
b 22 , we can assess the variation of designs. Bai, Hu and Rosenberger (2002) provide
Based on Λ
an estimator of Λ11 (the asymptotic covariance matrix of n1/2 Yn /|Yn |).
Remark 2.3 From (3.14), the asymptotic covariance matrix Λ22 does not depend on the delay
b 22 depends on the delay
mechanism. But this is an asymptotic result. For a fixed n, the estimate Λ
mechanism Mjl (n, m) and na . When na is small, Λ̂22 could be very different from Λ22 and the
asymptotic result is not very useful.
Theorem 2.2 Under Assumptions 2.1 and 2.2, if λ < 1, then for any κ > (1/2) ∨ λ ∨ (1 − γ),
n−κ (Yn − nv) → 0 and n−κ (Nn − nv) → 0
a.s.
Further, if γ > 1/2 and λ < 1/2, then
p
p
Yn − nv = O( n log log n) and Nn − nv = O( n log log n)
a.s.
Remark 2.4 Bai, Hu and Rosenberger (2002) established the strong consistency of both Yn and
Nn . Here we obtain the rate of strong consistency as well as the law of the iterated logarithm for
both Yn and Nn . The result is new and can also be applied in cases without delayed responses.
9
3
Proofs
In the section, C0 , C etc denote positive constants whose values may differ from line to line. For
a vector x in RK , we let kxk be its Euclidean norm, and define the norm of an K × K matrix M
by kMk = sup{kxMk/kxk : x 6= 0}. It is obvious that for any vector x and matrices M, M1 ,
kxMk ≤ kxk · kMk, kM1 Mk ≤ kM1 k · kMk.
To prove the main theorems, we show the following two lemmas first.
Lemma 3.1 If Assumption 2.1 is true, then
Rn = o(n1−γ ) in L1
0
Rn = o(n1−γ ) a.s. ∀γ 0 < γ.
and
Proof Obviously, for some constant C0 ,
max kRi k ≤ max kD(l)k max
i≤n
i≤n
l
≤ C0
i
XX
n
XX
∞
X
Mjl (m, k)
j,l m=1 k=i−m+1
∞
X
Mjl (m, k).
j,l m=1 k=n−m+1
So,
E max kRi k ≤ C
i≤n
n
XX
∞
X
µjlk (m) =
n
X
o((n − m)−γ ) = o(n1−γ ).
m=1
j,l m=1 k=n−m+1
It also follows that
P
©
max
2i ≤n≤2i+1
ª
E maxn≤2i+1 kRn k
kRn k
0
≥
²
≤C
= o(2i(γ −γ )),
0
0)
1−γ
i(1−γ
n
2
which is summable. This completes the proof of Lemma 3.1 by the Borel-Cantelli lemma.
Note that |Yn | = n + Y0 10 + Rn 10 . From Lemma 3.1, we get the following corollary, which is
Lemma 1 of Bai, Hu and Rosenberger (2002).
Corollary 3.1 If Assumption 2.1 is true,
n−1 |Yn | = 1 + o(n−γ ) in L1
0
n−1 |Yn | = 1 + o(n−γ ) a.s. ∀γ 0 < γ.
and
10
Lemma 3.2 Let Bn,i =
Qn
j=i+1 (I
+ j −1 H). Suppose matrices Qn and Pn satisfy Q0 = P0 = 0
and
Qn = Pn +
n−1
X
k=0
Qk
H.
k+1
(3.1)
Then
Qn =
n
X
∆Pm Bn,m = Pn +
m=1
n−1
X
m=1
Pm
H
Bn,m+1 ,
m+1
(3.2)
where ∆Pm = Pm − Pm−1 . Also,
kBn,m k ≤ C(n/m)λ logν−1 (n/m) for all m = 1, . . . , n, n ≥ 1.
(3.3)
Proof: By (3.1),
Qn
n−2
X Qk
Qn−1
= Pn − Pn−1 +
+ Pn−1 +
H
n
k+1
k=0
¡
¢
= ∆Pn + Qn−1 I + n−1 H
¡
¡
¢
¢¡
¢
= ∆Pn + ∆Pn−1 I + n−1 H + Qn−2 I + n−1 H I + (n − 1)−1 H
n
X
= ... =
∆Pm Bn,m
m=1
=
n
X
Pm Bn,m −
m=1
= Pn +
n−1
X
Pm Bn,m+1 = Pn +
m=1
n−1
X
m=1
Pm
n−1
X
Pm (Bn,m − Bn,m+1 )
m=1
H
Bn,m+1 .
m+1
(3.2) is proved. For (3.3), first notice that
T−1 HT = diag[0, J2 , . . . , Js ],
T−1 Bn,m T =
n
Y
¡
¢
I + j −1 diag[0, J2 , . . . , Js ]
j=m+1
n
n
i
h
Y
Y
−1
(I + j J2 ), . . . ,
(I + j −1 Js ) .
= diag 1,
j=m+1
(3.4)
j=m+1
And also, by recalling (2.6) of Bai and Hu (1998), as n > j → ∞, the (h, h + i)-th element of the
Q
block matrix nj=m+1 (I + j −1 Jt ) has the estimation
´
¡ n ¢³
1 ¡ n ¢Re(λt )
logi
1 + o(1) .
i! m
m
11
It follows that
n
° Y
°
°
(I + j −1 Jt )° ≤ C(n/m)Re(λt ) logνt −1 (n/m).
(3.5)
j=m+1
Combining (3.4) and (3.5) yields (3.3).
Proof of Theorem 2.1. Let Fn = σ(Y0 , . . . , Yn , ξ1 , . . . , ξn ) be the sigma algebra generated by
the urn compositions {Y0 , . . . , Yn } and the responses {ξ1 , . . . , ξn }. Denote E n−1 {·} = E{·|Fn−1 },
©
ª
qn = Xn − E n−1 {Xn }, Qn = Xn D(ξn ) − E n−1 {Xn D(ξn )}. Then (Qn , qn ), F; n ≥ 1 is a
sequence of R2K -valued martingale differences, and ξn is independent of Fn−1 . Recall that |Yn | =
PK
j=1 Ynj is the total number of particles in the urn after the n-th draw. Since the probability that
a type k particle in the urn is drawn equals the number of the type k particles divided by the total
Yn−1
|Yn−1 |
Yn−1,k
|Yn−1 | ,
k = 1, . . . , K, it is easily seen that E n−1 {Xn } =
¡ Yn
¢ 0
Yn−1
and E n−1 {Xn D(ξn )} = |Y
H.
Notice
that
vH
=
v
and
−
v
1 = 1 − 1 = 0. From
|Yn |
n−1 |
number of particles, i.e., P (Xnk = 1) =
(2.5), it follows that
n
X
Yn − nv =
=
=
=
m=1
n
X
m=1
n
X
m=1
n
X
Qm +
Qm +
Qm +
Qm +
m=1
n
X
Ym−1
H + Y0 + Rn − nvH
|Ym−1 |
m=1
n−1
X
m=0
n−1
X
m=0
n−1
X
m=0
¡ Ym
¢
− v H + Y0 + Rn
|Ym |
¢
¡ Ym
− v H + Y0 + Rn
|Ym |
Ym − mv
H + R(1)
n ,
m+1
(3.6)
where
R(1)
n = Rn + Y 0 +
n−1
X
m=0
n
X
¡ Ym
¢¡
|Ym | ¢
1
−v 1−
H−
v.
|Ym |
m+1
m
(3.7)
m=1
Also from (1.2), it follows that
Nn =
=
n
X
[Xm − E m−1 (Xm )] +
m=1
n
X
qm +
m=1
n−1
X
m=0
Ym
+
m+1
n−1
X
m=0
n−1
X
m=0
Ym
|Ym |
Ym
|Ym |
(1 −
).
|Ym |
m+1
(3.8)
By (3.7), Lemma 3.1 and Corollary 3.1,
1−γ
R(1)
) in L1
n = o(n
and
0
1−γ
R(1)
) a.s. ∀γ 0 < γ.
n = o(n
12
(3.9)
On the other hand, for the martingale difference sequence {Qn }, we have
E{kQn k4 k} ≤ 4EkD(ξn )k4 ≤ C0
(3.10)
and
E n−1 {Qn Q0n } = E n−1 {kQn k2 } ≤ 2EkD(ξn )k2 ≤ C1 .
So,
n
X
E n−1 {Qm Q0m } ≤ C1 n.
(3.11)
m=1
It follows that
Ek
n
X
2
Qm k = E[
m=1
n
X
E n−1 {Qm Q0m }] ≤ C1 n.
m=1
Hence
n
X
Qm = O(n1/2 ) in L2 .
m=1
Also, by (3.11) and the law of the iterated logarithm for martingales, we have
n
X
Qm = O((n log log n)1/2 ) a.s.
m=1
P
With the above two estimations for nm=1 Qm , by (3.6) and Lemma 3.2 we conclude that
°
°
n
n−1
m
°
°X
X
X
H
°
°
(1)
kYn − nvk = °(
B
Qi + R(1)
)
+
(
Q
+
R
)
n,m+1 °
i
n
m
°
°
m+1
≤ k
i=1
n
X
≤ C
Qi + R(1)
n k+
i=1
n
X
m=1 i=1
n−1
m
X X
k
m=1
(k
m=1
m
X
Qi + R(1)
m k)
i=1
n
X
Qi + R(1)
m k
i=1
kHk
kBn,m+1 k
m+1
1
n
n
( )λ logν−1 ( )
m+1 m
m
1
n
n
( )λ logν−1 ( )
m+1 m
m
m=1
¡
¢
= O(n1/2 ) 1 + nλ−1/2 logν n + o(n1/2−γ ) = O(n1/2 )
= OL1 (1)
(m1/2 + o(m1−γ ))
and
¡
0 ¢
kYn − nvk = O((n log log n)1/2 ) 1 + nλ−1/2 logν n + o(n1/2−γ )
= O((n log log n)1/2 ) a.s.,
13
(3.12)
where 1/2 < γ 0 < γ. It follows that
Yn
− v = O(n−1/2 ) in L1
|Yn |
Yn
log log n 1/2
− v = O((
) ) a.s.
|Yn |
n
and
Now, recalling Σ1 = diag(v) − v0 v and Σ2 = E[(D(ξn ) − H)0 diag(v)(D(ξn ) − H)], we have
E n−1 [Q0n Qn ] =
L
X
D(l)0 diag(
l=1
Y0
Yn−1
Yn−1
)D(l)P (ξn = l) − H0 n−1
H
|Yn−1 |
|Yn−1 | |Yn−1 |
= E[(D(ξn )0 diag(v)D(ξn )] − H0 v0 vH + O(n−1/2 )
in L1
= Σ2 + H0 Σ1 H + O(n−1/2 ),
E n−1 [q0n qn ] = diag(
Y0
Yn−1
Yn−1
) − n−1
= Σ1 + O(n−1/2 )
|Yn−1 |
|Yn−1 | |Yn−1 |
in L1
and
E n−1 [q0n Qn ] = diag(
Y0
Yn−1
Yn−1
)H − n−1
H = Σ1 H + O(n−1/2 )
|Yn−1 |
|Yn−1 | |Yn−1 |
in L1 .
Notice (3.10) and kqn k ≤ 2. By using Theorem 8 of Monrad and Philipp (1991), one can find two
(1)
(2)
independent sequences {Zn } and {Zn } of i.i.d d-dimensional standard normal random variables
such that

Pn
P

(2) 1/2
(1) 1/2

1/2−τ ) a.s.
 nm=1 Qm =
m=1 (Zm Σ2 + Zm Σ1 H) + o(n


Pn
m=1 qm
=
Pn
(1) 1/2
m=1 Zm Σ1
+
o(n1/2−τ )
(3.13)
a.s.
Here τ > 0 depends only on d. Without loss of generality, we assume τ < (γ − 1/2) ∧ (1/2 − λ).
(i)
(i)
If we define Gn by G0 = 0 and
Gn(i)
=
n
X
1/2
Z(i)
m Σi
+
m=1
n−1
X
m=1
(i)
Gm
H,
m+1
i = 1, 2, then by (3.6), (3.7), (3.9) and (3.13),
Yn − nv −
(G(2)
n
+
G(1)
n H)
=
n−1
X
m=1
(2)
(1)
Ym − mv − (Gm + Gm H)
H + o(n1/2−τ ) a.s.
m+1
By Lemma 3.2 again,
Yn − nv −
(G(2)
n
+
G(1)
n H)
=
n
X
o(m1/2−τ )
m=1
14
1
n
n
( )λ logν−1 ( ) = o(n1/2−τ ) a.s.
m+1 m
m
And then by (3.8) and Corollary 3.1,
n
X
Nn − nv =
qm +
m=1
n
X
=
m=1
1/2
Z(1)
m Σ1
m=1
n
X
=
n−1
X
1/2
Z(1)
m Σ1
m=1
Ym − mv
0
+ o(n1−γ ) a.s.
m+1
+
+
n−1
X
m=1
n−1
X
m=1
n−1
X G(2)
m
= G(1)
n +
m=1
m+1
n−1
(1)
X G(2)
Gm
m
H+
+ o(n1−τ ) a.s.
m+1
m+1
(1)
Gm
m+1
H+
m=1
n−1
X
m=1
(2)
Gm
+ o(n1−τ ) a.s.
m+1
+ o(n1−τ ) a.s.,
(1)
(i)
where we use the fact Gn 10 = 0, which is implied by Σ1 10 = 0. Note that Gn and
Pn−1
m=1 (m
+
(i)
1)−1 Gm , i = 1, 2, are normal random variables. To finish the proof, it suffices to calculate their
co-variances. Note that Bn,m = (n/m)H (1 + o(1)) and k(n/m)H k ≤ (n/m)λ logν−1 (n/m), where
P
i
1 i
aH is defined to be ∞
i=0 i! H log a. By (3.2),
n
¢ X
¡
0
1/2
=
Var Z(i)
Σ
B
Bn,m Σi Bn,m
n,m
m i
n
X
Var(G(i)
n ) =
m=1
m=1
n
X
=
0
H
H
(n/m) Σi (n/m) + o(1)
m=1
n
X
n
X
(n/m)2λ log2(ν−1) (m/m)
m=1
0
(n/m)H Σi (n/m)H + o(n)
=
m=1
Z
= n
0
1µ
1
y
¶H0
µ ¶H
1
(i)
Σi
+ o(n) =: nΣ1 + o(n).
y
Also,
n−1
n−1
³
X G(i)
X
X 1
¡ n−1
¢
m ¢
(i) 1/2
Var
=
Var Zm Σi
Bj,m
m+1
j+1
m=1
=
n−1
X
¡
m=1
m=1
n−1
X
j=m
Z
Z
1
= n
¢0 ¡
1
Bj,m Σi
j+1
dx
0
x
1
j=m
n−1
X
j=m
¢
1
Bj,m
j+1
µ ¶ 0
Z 1 µ ¶H
1 1 H
1 1
(i)
du + o(n) =: nΣ2 + o(n)
dvΣi
v v
u
x u
and
n−1
n−1
X 0
X 1
X G(i)
¡ n−1
¢
¡
m ¢
=
B
Σ
Bj,m
Cov G(i)
,
i
n,m
n
m+1
j+1
m=1
Z
= n
0
1
m=1
j=m
µ ¶H0
Z 1 µ ¶H
1
1 1
(i)
du + o(n) =: nΣ3 + o(n).
dxΣi
x
u
u
x
15
Note that k(1/y)H k ≤ c(1/y)λ logν−1 (1/y) for 0 < y ≤ 1, and λ < 1/2. The above integrals are
well defined. So,
D

n1/2 (Yn /n − v, Nn /n − v) → N (0, Λ),

(3.14)
Λ11 , Λ12 
(2)
(1)
(1)
(2)
(1)
(2)
where Λ = 
, Λ11 = Σ1 + H0 Σ1 H, Λ22 = Σ1 + Σ2 and Λ12 = Σ1 H + Σ3 .
Λ21 , Λ22
Proof of Theorem 2.2. By (3.12),
¡
0 ¢
Yn − nv = O((n log log n)1/2 ) 1 + nλ−1/2 logν n + o(n1/2−γ ) ∀γ 0 < γ



o(nκ ) a.s.,
if κ > (1/2) ∨ λ ∨ (1 − γ),
=


O((n log log n)1/2 ) a.s., if λ < 1/2 and γ > 1/2.
Then, by (3.8) and Corollary 3.1, it follows that
Nn − nv =
n
X
qm +
m=1
n−1
X
m=0
n
Ym − mv X
0
+
o(m−γ ) ∀γ 0 < γ
m+1
m=1
p
= O( n log log n) +
n−1
X
m=0
o(mκ )
m+1
0
+ o(n1−γ ) ∀γ 0 < γ
κ
= o(n ) a.s.
whenever κ > (1/2) ∨ λ ∨ (1 − γ), and
n−1
X o(√m log log m)
p
0
Nn − nv = O( n log log n) +
+ o(n1−γ ) ∀γ 0 < γ
m+1
m=0
p
= O( n log log n) a.s.
whenever λ < 1/2 and γ > 1/2. Theorem 2.2 is now proved.
(i)
Remark 3.1 Write T∗ Σ1 T = (Σghi , g, h = 1, . . . , s) and T = (10 , T2 , . . . , Ts ), where T is
defined in Assumption 2.2. Note that
Z
T
∗
(i)
Σ1 T
1
1
1
∗
∗
( )diag[0,J2 ,...,Js ] T∗ Σi T( )diag[0,J2 ,...,Js ] dy
y
0 y
Z 1
1 ∗
1 ∗
1
1
=
diag[1, ( )J2 , . . . , ( )Js ]T∗ Σi Tdiag[1, ( )J2 , . . . , ( )Js ]dy.
y
y
y
y
0
=
Also,
νX
t −1
1
1
Ĵat
1
( )Jt = ( )λt
loga ( ),
y
y
a!
y
a=0
16
where


0 1 0 . . . 0




0 0 1 . . . 0
,
Ĵt = 
. . .

 .. .. .. . . . ... 




0 0 0 ... 0


0


0
,
.. 
.


0
0 0 1 . . .


0 0 0 . . .
2
Ĵt = 
....
. . .
 .. .. .. . . .


0 0 0 ...
R1
∗
= 1Σi 10 = 0, Σ∗1gi = Σg1i = 0 (1/y)Jg T∗g Σi 10 dy = 0, and, the (a,b)
So for g, h = 2, . . . , s, Σ11i
R1
∗
element of matrix Σghi = 0 (1/y)Jg T∗g Σi Th (1/y)Jh dy is
a−1 X
b−1
X
=
a0 =0 b0 =0
a−1 X
b−1
X
1
a0 !b0 !
Z
0
1
1
0
0 1
( )λg +λh loga +b ( )[T∗g Σi Th ]a−a0 ,b−b0
y
y
(a0 + b0 )!
[T∗g Σi Th ]a−a0 ,b−b0 .
0 !b0 !(1 − λ − λ )a0 +b0 +1
a
g
h
a0 =0 b0 =0
This agrees with the results of Bai and Hu (1999) and Bai, Hu and Rosenberger (2002). Similarly,
(i)
(i)
one can calculate Σ2 and Σ3 .
Acknowledgments. Professor Hu is also affiliated with Department of Statistics and Applied
Probability, National University of Singapore. Part of his research was done while visiting the
Department of Biometrics, Cornell University. He thanks the Department for its hospitality and
support.
17
References
[1] Athreya, K. B. and Karlin, S. (1968), Embedding of urn schemes into continuous time branching processes and related limit theorems. Ann. Math. Statist., 39:1801-1817.
[2] Bai, Z. D. and Hu, F. (1999), Asymptotic theorem for urn models with nonhomogeneous
generating matrices, Stochastic Process. Appl., 80: 87-101.
[3] Bai, Z. D., Hu, F. and Rosenberger (2002), Asymptotic properties of adaptive designs for
clinical trials with delayed response, Ann. Statist., 30: 122-139.
[4] Bandyopadhyay, U. and Biswas, A. (1996), Delayed response in randomized play-the-winner
rule: a decision theoretic approach, Calcutta Statistical Association Bulletin, 46:69-88.
[5] Durham, S. D., Flournoy, N. and Li, W. (1998), A sequential design for maximizing the
probability of a favourable response, Canad. J. Statist., 26: 479-495.
[6] Flournoy, N. and Rosenberger, W. F., eds. (1995), Adaptive Designs, Hayward, Institute of
Mathematical Statistics.
[7] Hall, P. and Heyde, C. C. (1980), Martingale Limit Theory and its Applications, Academic
Press, London.
[8] Hu, F. (2002), Sample size of response-adaptive designs. In preparation.
[9] Monrad, D. and Philipp, W. (1991), Nearby variables with nearby conditional laws and a
strong approximation theorem for Hilbert space valued martingales, Probab. Theory Relat.
Fields, 88:381-404.
[10] Robbins, H. (1952), Some aspects of the sequential design of experiments. Bulletin of the
American Mathematics Society, 58: 527-535.
[11] Rosenberger, W. F. (1996), New directions in adaptive designs, Statist. Sci., 11:137-149.
[12] Rosenberger, W. F. (2002), Randomized urn models and sequential design (with discussions),
Journal of Sequential Analysis, To appear.
18
[13] Rosenberger, W. F. and Grill, S. E. (1997), A sequential design for psychophysical experiments: an application to estimating timing of sensory events, Statist. Med., 16: 2245-2260.
[14] Smythe, R. T. (1996), Central limit theorems for urn models, Stochastic Process. Appl.,
65:115-137.
[15] Tamura, R. N., Faries, D. E., Andersen, J. S., and Heiligenstrin, J. H. (1994), A case study of
an adaptive clinical trial in the treatment of out-patients with depressive disorder, J. Amer.
Statist. Assoc., 89: 768-776.
[16] Wei, L. J. (1979), The generalized Pólya’s urn design for sequential medical trials, Ann.
Statist., 7:291-296.
[17] Wei, L. J. (1988), Exact two-sample permutation tests based on the randomized play-thewinner rule, Biometrika, 75: 603-606.
[18] Wei, L. J. and Durham, S. (1978), The randomized pay-the-winner rule in medical trials, J.
Amer. Statist. Assoc., 73:840-843.
[19] Wei, L. J., Smythe, R. T., Lin, D. Y. and Park, T. S. (1990), Statistical inference with
data-dependent treatment allocation rules, J. Amer. Statist. Assoc., 85:156-162.
Feifang Hu
Department of Statistics
University of Virginia
Halsey Hall
Charlottesville, Virginia 22904-4135
Email: fh6e@virginia.edu
Lixin Zhang
Department of Mathematics
Zhejiang University, Xixi Campus
19
Zhejiang, Hangzhou 310028
P.R. China
Email: lxzhang@mail.hz.zj.cn
20