On the joint behavior of types
of coupons in generalized
coupon collection
Hosam M. Mahmoud1
and
Robert T. Smythe2
November 22, 2011
Abstract
The “coupon collection problem” refers to a class of occupancy problems,
in which j identical items are distributed, independently and at random,
to n cells, with no restrictions on multiple occupancy. Identifying the cells as
coupons, a coupon is “collected” if the cell is occupied by one or more of the
distributed items; thus some coupons may never be collected, whereas others
may be collected once or twice or more. We call the number of coupons
collected exactly r times coupons of type r.
The coupon collection model we consider is general, in which a random
number of purchases occurs at each stage of collecting a large number of
coupons; the sample sizes at each stage are independent and identically distributed according to a sampling distribution. The joint behavior of the various types is an intricate problem. In fact, there is a variety of joint central
limit theorems (and other limit laws) that arise according to the interrelation
between the mean, variance, and range of the sampling distribution, and of
course the phase (how far we are in the collection processes). According to
an appropriate combination of the mean of the sampling distribution and the
number of available coupons, the phase is sublinear, linear or superlinear. In
the sublinear phase, the normalization that produces a Gaussian limit law
for uncollected coupons can be used to obtain a multivariate central limit
1
Department of Statistics, The George Washington University, Washington,
D.C. 20052, U.S.A.
2
Statistics Department, Oregon State University, Corvallis, Oregon 97331, U.S.A.
1
law for at most two other types—depending on the rates of growth of the
mean and variance of the sampling distribution, we may have a joint central
limit theorem between types 0 and 1, or between types 0, 1, and 2. In the
linear phase we have a multivariate central limit theorem among the types
0, 1, . . . , k, for any fixed k.
AMS subject classifications. Primary: 60C05, 60F05, 05A05; secondary:
60G42, 60G48, 60E05.
Keywords: Urn model, random structure, stochastic process, multivariate
martingale, multivariate central limit theorem, sampling, coupon collection,
phases, phase transition.
1
Coupon collection
The most basic form of the “coupon collection problem” concerns the distribution of j items, independently and at random, to n cells (thought of as
coupons), where a coupon is “collected” if the cell is occupied. Questions of
interest then include: (1) What is the distribution of the number of uncollected coupons (or the number collected exactly once, etc.)? And (2) what
is the expected number of items that must be placed in order that all cells
be occupied (all coupons collected)?
There are many variations of this problem, going back at least to de
Moivre (1718) and Laplace (1774). The problem gained popularity in the
1930’s, when the Dixie Cup Company sold ice cream cups with a cardboard
cover that had hidden on the underside a coupon (carrying likeable items
such as cute animals, movie stars and Major League baseball players). This
marketing strategy is meant to encourage fans of such items to complete sets
of their favorites, and thus increase the sales. Many companies followed suit
and there has been a myriad of such schemes; many are now obsolete.
A generalized form of the classical coupon collector’s problem assumes the
consumer purchases S ≥ 1 (a random number of) items each time and the
promoting company guarantees that the S associated coupons are distinct.
The collector obtains S coupons at each stage, of which some or all may
already be in her possession. The company promises that all sold collections
of size S are equally likely. Ideally, when one such collection is sold, it is
immediately replaced in the market to maintain the uniformity of all subsets
2
of any feasible size. In the Dixie Cup scheme S ≡ 1. Kobza, Jacobson, and
Vaughan (2007) and Stadje (1990) provide surveys.
While collecting coupons, some may never be obtained, others may be
collected once or twice or more. We call the number of coupons collected
exactly r times coupons of type r. We investigate in this paper the joint
behavior of coupons of different types across the phases of collection.
2
Setup as an urn scheme
Consider the following setup for coupon collection with random sampling. At
the start we we have n coupons to be collected, where n may be large. Let Sj
be an independent identically distributed sequence of random variables, all
distributed like a generic random variable S = S(n) ∈ {1, 2, . . . , sn }, with
sn ≤ n. For technical reasons that will become evident later on, we
√ keep the
range of S small, relative to n. Specifically, we work with sn = o( n ).√And
so, the mean µS (n) and standard deviation σS (n) of S(n) are also o( n ).
At the jth stage the collector purchases a random number Sj of coupons.
(n)
Let Xj,r be the number of coupons that have been collected exactly r
times after j samples have been purchased. For fixed k ≥ 0, let

(n)
Xj
=
(n)
Xj,0





 (n) 

X
 j,1 


.

 . 
 .. 






(n)
Xj,k
Represent the coupons as balls of k +2 different colors (labeled 0, 1, . . . , k +1)
in an urn. Coupons that have been collected r times are balls of color r, for
r = 0, . . . , k. Color k + 1 is special—it represents all the balls that have been
drawn more than k times. There is no need to study the number of balls of
color k + 1, as it is determined by the number of balls of all the other colors.
P
(n)
(n)
Specifically, Xj,k+1 = n − kr=0 Xj,r .
3
3
Organization
The rest of the paper is organized in sections as follows. We introduce notation in Section 4 and state the results in Section 5. In Section 6 we formulate
the basic stochastic recurrence, which gives a matrix recurrence for the mean
and covariance. We present an exact solution to the mean recurrence in
Subsection 6.1, and we present exact and asymptotic solutions to the covariance recurrence in Subsection 6.2. The multivariate martingale underlying
the process is derived in Section 7, and the analysis in the sublinear and
linear phases is taken up in Sections 8 and 9, respectively. We conclude in
Section 10 with some illustrating examples.
4
Notation and setup
The notation Bin(n, p) stands for a binomial random variable on n trials with
rate of success p per trial, and Nk (0, Σ) stands for a multivariate normal
vector of k components with mean 0 (of k components) and k × k covariance
matrix Σ.3 Let Hypergeo(n, m, w) be a hypergeometric random variable
that is the number of white balls in a sample of size m balls taken without
replacement at random (all subsets of size m being equally likely) from an
urn containing a total of n white and red balls, of which w are white. The
mean and variance for this standard distribution are well known; see Stuart
and Ord (1987, Article 5.14).
The transpose of a matrix W is denoted by W. We shall use the matrix
norm k . k, defined as the square root of the sum of the squares of the matrix
components, or equivalently, the square root of the sum of the squares of
its eigenvalues. We use the notation o(bn ) and O(bn ) for matrices in which
each component is respectively o(bn ) and O(bn ) in the usual scalar sense.
The probabilistic versions oL1 (bn ) and respectively oP (bn ) will stand for a
sequence of random matrices, where each component is o(bn ) respectively in
D
the L1 norm, and in probability. We shall use the symbol −→ for convergence
P
in distribution, and the symbol −→ for convergence in probability. In the
3
In this investigation some of the multivariate normal distributions refer to what some
books call singular multivariate normal distributions, where Σ is a singular matrix, but a
number of linear combinations define together a proper multivariate normal distribution
of lower dimension.
4
sequel all matrix convergence, be it deterministic or probabilistic (in L1 and
in probability), is considered componentwise. We let Fj be the sigma field
generated by the first j draws. Note that the sequence {Fj }∞
j=0 of sigma
fields is increasing. Thus, it can be the filtration of a martingale sequence.
Unless otherwise stated, all asymptotic equivalents and bounds are taken as
n → ∞.
The following special matrices will be used:

0

1

B := 
 0.
.
.
0
M :=
0
0
1
..
.
0
0
0
0
0
0
0
..
.
0
0

... 0 0
... 0 0


... 0 0
,
.. .. 

. .
... 1 0
µS (n)
µS (n) µS (n)
B+ 1−
I=I+
(B − I),
n
n
n
 


−1
1


 
 1 
0


 


F=
G=
 0.  ;
 0.  ,
 . 
.
 . 
.
0
0
the matrices B and M are of dimension (k + 1) × (k + 1), and F and G are
(k + 1)–component vectors. Note that high powers of B vanish. Specifically,
Bi is the zero matrix, for i ≥ k + 1. For 0 ≤ i ≤ k, Bi is a matrix of all zeros,
except for the entries on the forward minor diagonal starting at position (i, 0)
and ending at position (k, k − i), which are all one.
In what follows we shall use the functions
(σS2 (n) − µS (n))n + µ2S (n)
,
n2 (n − 1)
µS (n)n − σS2 (n) − µ2S (n)
=
,
n(n − 1)
gn =
hn
and the functions f`,m (y), which are the coefficients of linearity in the covariance between coupons of type ` and m in the usual coupon collection
5
(S ≡ 1). They appear in the work of Kolchin, Sevastyanov and Chistyakov
(1978, Page 38). The first few are
[f`,m ]0≤`,m≤2 = e−2λ
 λ
e −1−λ
−λ2

×
−λ2
λeλ − λ + λ2 − λ3
3
2
4
− λ2 + λ2
− λ2 + λ3 − λ2
3
2
− λ2 + λ2
4

− λ2 + λ3 − λ2
.
λ5
3λ4
λ 2
3
e (λ + 1 − λ) − 4 + 4 − λ − 1

We denote the matrix [f`,m ]0≤`,m≤k by Jk+1 . We also need the matrix
J0k+1 = e−2λ
h λ`+m−1
`! m!
i
(λ − `)(λ − m)
0≤`,m≤k
.
The vector
λk λ2
...
2
k!
(n)
appears as the coefficient of linearity of E[Xj ] in the linear phase.
ν k+1 = 1 λ
We let 1E be the indicator of event E; that is, a function that assumes
the value 1, if E occurs, and otherwise it assumes the value 0. We shall use
the backward operator ∇ai = ai − ai−1 .
5
The results
Central to all the analysis is a careful handling of the covariance structure of
the process. We present an exact formula for the covariance in Proposition 1.
We do not deal with a case where 0 = lim inf n→∞ σS2 (n) < lim supn→∞ σS2 (n).
Up to O(1) draws nothing much of interest happens. We investigate the joint
behavior of balls after j draws for j in two phases:
(a) The growing sublinear phase, where j = jn grows to infinity with n,
but jn = o(n/µS (n)),
(b) We are in the linear phase when
µS (n)jn ∼ λn n,
for some positive λn that is bounded away from 0 and ∞. That is, for
positive constants Q1 and Q2 , and for all n ≥ 1,
0 < Q1 ≤ λn ≤ Q2 < ∞.
6
Theorem 1 Consider coupon collection
√ with a sampling distribution having
range in {1, 2, . . . , sn }, with sn = o( n ) and with mean µS (n) and vari(n)
(n)
ance σS2 (n). Let Xj,r be the number of balls of type r, 0 ≤ r ≤ k, and Xj be
(n)
the vector with components Xj,r , r = 0, . . . , k. Assume we are in the growing
sublinear phase, where jn → ∞ and jn = o(n/µS (n)). Let
µ2S (n)jn2
2n
αn := 2
.
2
µS (n)jn
2
+ σS (n)jn
2n
Suppose we are in the upper sublinear phase
µ2 (n)j 2
n
S
s2n = o
2n
+ σS2 (n)jn .
We have:
(a) If αn → α ∈ (0, 1], for k = 2,
n

(n)
Xj n − 1 −
s



j
µS (n) n 


n






µS (n)jn 



2 
2
µS (n)jn 

2n
µ2S (n)jn2
+ σS2 (n)jn
2n

1

0,  −(α + 1)
α
D
−→ N3
subject to the additional condition
2
µ2S (n)jn
2n
−(α + 1)
3α + 1
−2α

!
α

−2α  ,
α
+ σS2 (n)jn grows to infinity.
(b) If αn → 0, for k = 1,
(n)
µS (n) jn
n √
σS (n) jn
Xjn − 1 −
n
µS (n)jn
D
−→ N2
1 −1
0,
−1 1
subject to the additional condition 1/σS2 (n) = o(jn ).
7
!
,
Remark: In Part (b) of Theorem 1 the covariance matrix is singular. In this
√
(n)
(n)
case each of Xjn ,0 and Xjn ,1 (shifted by its mean and scaled by σS (n) jn )
satisfies a univariate central limit theorem with variances and covariances
coinciding with those in the limiting bivariate normal distribution. The same
(n)
(n)
(n)
can be said in Theorem 1 (a) in the case α = 1 about Xjn ,0 , Xjn ,1 and Xjn ,2 :
Each (appropriately normalized) satisfies a univariate central limit theorem
with variances and covariances coinciding with those in the limiting trivariate
normal distribution.
Theorem 2 Consider coupon collection
√ with a sampling distribution having
range in {1, 2, . . . , sn }, with sn = o( n ) and with mean µS (n) and vari(n)
(n)
ance σS2 (n). Let Xj,r be the number of balls of type r, 0 ≤ r ≤ k, and Xj be
(n)
the vector with components Xj,r , r = 0, . . . , k. Assume we are in the linear
phase, where jn ∼ λn n/µS (n), for 0 < Q1 ≤ λn ≤ Q2 < ∞. Let
βn :=
µS (n)
.
µS (n) + σS2 (n)
If λn → λ > 0, we have:
(a) If βn → β ∈ (0, 1],
(n)
Xjn − ne−λ ν k+1 D
1−β 0 √
−→ Nk+1 0, Jk+1 +
Jk+1 .
n
β
(b) If βn → 0, and λ is not a positive integer r ≤ k,
(n)
Xjn − ne−λ ν k+1 D
s
−→ Nk+1 (0, J0k+1 ).
n
σS (n)
µS (n)
√
(n)
If λ = r ≤ k, then (Xjn ,r − λr e−λ n/r!)/ n is asymptotically normal
with variance given by the rth diagonal entry of Jk+1 .
The alert reader must have noticed that the case of superlinear growth
(when µS (n)jn grows faster than n) has not been considered in this work.
8
(n)
For the case of Xj,0 (uncollected coupons), some asymptotic results appear
in Smythe (2011). It turns out that in the superlinear phase, the asymptotic
(n)
variance of each Xj,r , r = 0, 1, . . . , k, is of different order, so there is no
multivariate central limit theorem of the type considered in the present paper.
Some results in this case will appear elsewhere.
6
Stochastic recurrence and moments
When the collector purchases a sample of size Sj coupons, any coupon appearing in that sample is acquired one more time. That is, if the corresponding ball in the urn is of color r (for r = 0, . . . , k), its color is upgraded to be
r + 1; the number of balls of color r + 1 goes up by one, and the number of
(n)
balls of color r goes down by one. Let Hj,r be the number of balls of color r
(n)
in the jth sample (which is of size Sj ). So, Hj,r has the distribution of the
(n)
Hypergeo(n, Sj , Xj−1,r ) random variable. We have the recurrence system
(n)
(n)
(n)
(n)
Xj,r = Xj−1,r + Hj,r−1 − Hj,r ,
for r = 0, . . . , k,
(n)
(1)
interpreting Hj,−1 ≡ 0. From this stochastic recurrence we can find moments.
We illustrate that only on the first two moments. As we shall see, it is quite
tedious to get the second moment; exact higher moments would be a real
challenge to find by such direct methods, and we later find their asymptotics
by alternative means.
6.1
The mean
(n)
(n)
The counts Xj,r (interpreting Xj,−1 ≡ 0) have averages
(n)
1
1
(n)
(n)
E[Xj−1,r−1 Sj ] − E[Xj−1,r Sj ]
n
n
1
1
(n)
(n)
(n)
= E[Xj−1,r ] + E[Xj−1,r−1 ] E[Sj ] − E[Xj−1,r ] E[Sj ]
n
n
µS (n) µS (n)
(n)
(n)
= 1−
E[Xj−1,r ] +
E[Xj−1,r−1 ].
n
n
(n)
E[Xj,r ] = E[Xj−1,r ] +
9
This can be represented in matrix form:

(n)
E[Xj ] =








µS (n)
n
µS (n)
n
1−
0
µS (n)
n
µS (n)
n
0
0
1−
0
..
.
..
.
0
0
1 − µSn(n)
..
.
0
0 ...
0 ...
0 ...
0
0
0
..
.
0 ...
µS (n)
n

0
0
0
..
.
1−
µS (n)
n








(n)
×E[Xj−1 ]

=








µS (n)
n
µS (n)
n
1−
0
1−
0
..
.
1−
..
.
0
0

µS (n)
n
µS (n)
n
0
0
n
µS (n)
n
0
0
0
0
..
.
...
...
...
0
0
0
..
.
0
...
µS (n)
n
0
0
0
..
.
1−
µS (n)
n
j









 
0
 

×
 0.  .
.
.
0
The rows and columns of these (k + 1) × (k + 1) matrices are indexed by the
coupon types 0, 1, . . . , k. A matrix decomposition will help us asymptotically
simplify this matrix expression. We can write the average vector as


n
 
0
µ (n)

µS (n) j 
S
(n)

E[Xj ] =
B+ 1−
I 
 0. 
n
n
.
.
0
 
n
 
!
0
k
X
 
µS (n) j−i j
µiS (n) i 
1
−
B
=
 0.  ,
ni
n
i
.
i=0
.
0
10
for j ≥ k. Note that we stopped the sum at k, because all higher powers
of B are identically 0. Thus,

µS (n) j
n 1−


n


j−1


µ
(n)
S


jµS (n) 1 −




n


2
(n)
j−2
j(j
−
1)µ
(n)
µ
(n)

.
S
S
E[Xj ] = 

1
−


2n
n


..




.
!


 µkS (n) j µS (n) j−k 
1−
nk−1 k
n

6.2
(2)
The covariance structure
Toward covariance calculation, let us take the conditional expectation of the
cross products of the counts:
(n)
(n)
(n)
(n)
(n)
(n)
(n)
(n)
E[Xj,` Xj,m | Fj−1 ] = (Xj−1,` + Hj,`−1 − Hj,` )(Xj−1,m + Hj,m−1 − Hj,m ),
yielding the expectation
(n)
(n)
(n)
Aj (`, m) := E[Xj,` Xj,m ]
(n)
(n)
(n)
(n)
(n)
(n)
(n)
(n)
= E[Xj−1,` Xj−1,m ] + E[Xj−1,` Xj−1,m−1 ] − 2E[Xj−1,` Xj−1,m ]
+E[Xj−1,`−1 Xj−1,m
(n)
(n)
E[S
(n)
j]
n
(n)
(n)
(n)
(n)
+ E[Hj,`−1 Xj,m−1 − Hj,`−1 Xj,m
(n)
−Hj,` Xj,m−1 + Hj,` Xj,m ].
We use the known expressions for the mean and covariance of the multihypergeometric distribution (see Stuart and Ord, 1987, Article 5.14). Let us
(n) (n)
(n)
(n)
now construct the matrix E[Xj Xj ] =: Aj = [Aj (`, m)]0≤`,m,≤k . After
some rather lengthy algebraic operations we reach the recurrence
(n)
Aj
(n)
= Aj−1 +
µS (n)
(n)
(n)
(n)
(n)
(BAj−1 + Aj−1 B) + BAj−1 B + hn K̃j ,
n
(n)
(n)
(n)
where K̃j is the diagonal matrix Diag(E[Xj−1,0 ], . . . , E[Xj−1,k ]). The recurrence in this form is not easy to iterate. Nonetheless, a reorganization in
the form
(n)
(n)
(n)
(n)
Aj = gn GAj−1 G + CAj−1 C + hn Kj ,
11
(n)
(n)
where G = B − I, C = I + µSn(n) G, and Kj = (B − I) K̃j (B − I) helps us
iterate the recurrence.
A few steps of iteration reveal a pattern, which can then be proved by
induction. The recurrence has the exact solution
(n) (n)
E[Xj Xj ]
j−1
i
XX
= hn
!
i
i−r r
(n)
Cr Gi−r Kj−i G C
r
gni−r
i=0 r=0
j
X
!
gnj−r
+
r=0
j
(n) j−r r
Cr Gj−r A0 G C .
r
It can be shown (by induction, for example) that
!
i
1h
p
Cp := p
µS (n)`−m (n − µS (n))p−`+m
,
0≤`,m≤k
n
`−m
and
!
i
q
;
` − m 0≤`,m≤k
h
Gq := (−1)`−m+q
y
x
interpret
r
j−r
[C G
as 0, whenever x is negative or y < x. Multiplying out,4 we get
(n) j−r r
A0 G C ]`,m
=
X
`
1
n2r−2
p
(−1)
p=0
r
`−p
×µ`−p
S (n)(n
×
m
X
q
(−1)
q=0
r
m−q
!
j−r
p
!
r−`+p
− µS (n))
!
×µm−q
(n)(n
S
j−r
q
!
r−m+q
− µS (n))
.
Similarly, we have
r
[C G
i−r
i−r
Kj−i G
r
C ]`,m
k X
k
1 X
r
=
(−1)p−ν
2r
n ν=0 p=0
`−p
4
!
!
i−r+1
µS (n)`−p
p−ν
These calculations are facilitated by a symbolic algebra system like Maple or
Mathematica.
12
×(n − µS (n))r−`+p
µν (n) j µS (n) j−i−1−ν
× Sν−1
1−
n
n
ν
!
X
k
×
!
r
m−q
(−1)q−ν
q=0
!
i−r+1
µS (n)m−q
q−ν
r−m+q
×(n − µS (n))
.
Putting these calculations together, we get an exact expression for the covariance.
Proposition 1
(n)
(n)
Cov[Xj,` , Xj,m ]
µS (n)n − σS2 (n) − µ2S (n)
=
n(n − 1)
×
×
j−1
i XX
i=0 r=0
k
k X
X
(σS2 (n) − µS (n))n + µ2S (n) i−r i
1
n2 (n − 1)
r n2r−2
!
r
`−p
p−ν
(−1)
ν=0 p=0
r−`+p
×(n − µS (n))
!
!
i − r + 1 `−p
µS (n)
p−ν
µν (n) j µS (n) j−i−1−ν
× Sν−1
1−
n
n
ν
!
×
X
k
q−ν
(−1)
q=0
r
m−q
!
r−m+q
×(n − µS (n))
+n2
X
j r=0
×
!
i − r + 1 m−q
µS (n)
q−ν
)
(σS2 (n) − µS (n))n + µ2S (n) j−r j
n2 (n − 1)
r
X
`
(−1)p
p=0
r
`−p
!
j−r
p
!
r−`+p
×µ`−p
S (n)(n − µS (n))
13
!
m
X
×
(−1)
r
m−q
q
q=0
!
j−r
q
!
×µm−q
(n)(n − µS (n))r−m+q
S
µ`+m (n) j
− S`+m−2
n
`
7
!
!
j µS (n) 2j−`−m
.
1−
n
m
The underlying multivariate martingale
Condition the recurrence (1) on Fj−1 to get
(n)
E[Xj,r | Fj−1 ] = 1 −
µS (n) (n)
µS (n) (n)
Xj−1,r +
Xj−1,r−1 .
n
n
Putting the recurrences for different colors together in one matrix form, we
have
(n)
E[Xj | Fj−1 ] =
µ
S (n)
n
B+ 1−
µS (n) (n)
I Xj−1 .
n
(3)
Therefore
(n)
(n)
E[Xj | Fj−1 ] := M Xj−1 ,
and
(n)
Yj
(n)
(n)
:= M−j Xj
(n)
(n)
is a martingale, and Ỹj = M−j Xj − X0 is a centered martingale.
For suitable scale factors ξn for each phase, we shall check Lindeberg’s
conditional condition, that is
Un :=
jn
X
i=1
h
E i
1
P
(n) 2
∇Ỹi 1n 1
o Fi−1 −→ 0,
(n)
ξn
∇Ỹi > ε
ξn
and the conditional variance condition, that is,
Vn :=
jn
X
i=1
Cov
h
i
1
P
(n)
∇Ỹi Fi−1 −→ Γ,
ξn
for a covariance matrix Γ.
14
(4)
(n)
(n)
n
When both conditions hold, the sum jj=1
ξn−1 ∇Ỹj = ξn−1 (M−jn Xjn −
(n)
X0 ) converges to the multinormally distributed random vector Nk+1 (0, Γ);
this follows from an appropriate extension of the univariate martingale central limit theorem in Hall and Heyde (1980, Page 58), via for example the
Cramér-Wold device. Some of the limiting covariance matrices that appear
in this work are singular (see footnote 3), with fewer linear combinations that
have a proper nonsingular multivariate normal distribution (with a positive
definite covariance matrix). The following lemma gives an exact computation
of Vn , and will be helpful in all the phases via an appropriate asymptotic
analysis.
P
Lemma 1
Vn
jn
1 X
(n)
(n)
(n)
(n)
(n)
(n)
gn M−i (Xi−1 Xi−1 − BXi−1 Xi−1 − Xi−1 Xi−1 B
= 2
ξn i=1
(n)
(n)
+BXi−1 Xi−1 B) M
(n)
−i
(n)
+ hn M−i (Di
(n)
− BDi
−i
(n)
−Di B + BDi B) M ,
(n)
where Di
(n)
(n)
(n)
is the diagonal matrix Diag(Xi−1,0 , Xi−1,1 , . . . , Xi−1,k ).
Proof . Start with the definition of conditional covariance to get
(n)
(n)
Cov[∇Ỹi | Fi−1 ] = Cov[∇(M−i Xi
(n)
= E[(M−i Xi
(n)
(n)
− X0 ) | Fi−1 ]
(n)
− M−i+1 Xi−1 )
×(Xi M
−i
(n)
− Xi−1 M
−i+1
) | Fi−1 ].
Using the stochastic recurrence (1), we expand the products in the covariance
and write it as
(n)
(n)
(n)
(n)
(n)
(n)
(n)
Cov[∇Ỹi | Fi−1 ] = gn M−i (Xi−1 Xi−1 − BXi−1 Xi−1 − Xi−1 Xi−1 B
(n)
(n)
+BXi−1 Xi−1 B) M
(n)
(n)
−i
(n)
+ hn M−i (Di
−i
−Di B + BDi B) M .
We now sum these terms to obtain an exact expression for Vn . 2
15
(n)
− BDi
8
The sublinear phase
We are in the sublinear phase when j = jn = o(n/µS (n)). In this phase we
take
s
µ2S (n)jn2
ξn =
+ σS2 (n)jn .
2n
Lemma 2 For j = o(n/µS (n)) in the sublinear phase,
√
(n) ∇Ỹj ≤ 4(k + 1) 2k + 1sn .
Proof . Set
(n)
Hj
(n)
(n)
(n)
. . . Hj,k ),
= (Hj,0 Hj,1
so as to write (1) in the form
(n)
Xj
(n)
(n)
(n)
= Xj−1 + BHj − Hj .
(5)
(n)
We bound each component of Xj−1 by n, and it follows that, for large n,
(n) ∇Ỹj (n)
(n)
= Yj − Yj−1 (n)
(n)
= M−j Xj − M−j+1 Xj−1 (n)
(n)
= M−j Xj − MXj−1 = I + O
≤
=
≤
≤
≤
µ
S (n)
(n)
(Xj−1
(n)
(n)
(n)
+ BHj − Hj ) − MXj−1 n
µ (n)
√
S
(n) (n) 2 k + 1 (I − B) Xj−1 + B − I Hj n
µ (n) √
S
(n) (n) 2 k + 1 B − I Xj−1 + Hj n
q
µ (n)
√
√
S
× k + 1 n + sn k + 1
2 (k + 1)(2k + 1)
n
√
2(k + 1) 2k + 1 (µS (n) + sn )
√
2(k + 1) 2k + 1 (2sn ). 2
16
Lemma 3 If jn is sublinear but restricted to the upper sublinear phase where
s
sn = o(ξn ) = o
µ2S (n)jn2
+ σS2 (n)jn ,
2n
and if ξn → ∞, we then have
Un :=
jn
X
i
h 1
P
(n) 2
E ∇Ỹi 1n 1
o Fi−1 −→ 0.
(n)
ξn
∇Ỹi > ε
i=1
ξn
Proof . For any given ε > 0, according to Lemma 2, the sets
(
1
s
µ2 (n)j 2
S
n
2n
+ σS2 (n)jn
(n) ∇Ỹi )
>ε
are all empty, for all n greater than some positive integer n0 (ε). For n ≥
n0 (ε), in view of the restriction of jn in an upper sublinear phase, we have
n0 (ε)
Un ≤
X
j=1
16(k + 1)2 (2k + 1)s2n
µ2S (n)jn2
+ σS2 (n)jn
2n
= 16(k + 1)2 (2k + 1)n0 (ε)
→ 0,
as n → ∞. 2
s2n
µ2S (n)jn2
+ σS2 (n)jn
2n
It is shown in Mahmoud (2010) that in the sublinear phase most of the
draws produce type–0 balls, which are converted into type–1. That is, for
any 0 ≤ i ≤ jn ,
(n)
Xi,0 = n − iµS (n) + oP (sn jn ),
(n)
Xi,1 = iµS (n) + oP (sn jn ),
and
(n)
for 2 ≤ r ≤ k + 1.
Xi,r = oP (sn jn ),
17
We can represent this as

 
(n)
Xi

−1
1


 
 1 
0


 



= n
 0.  + iµS (n)  0.  + oP (sn jn ),
 . 
.
 . 
.
0
0
so that
(n)
Xi
and
(n)
(n)
Xi Xi
= nF + iµS (n)G + oP (sn jn ),
= n2 FF + inµS (n)(FG + GF) + oP (nsn jn ).
We shall also need the diagonal matrix D = Diag(−1, 1, 0, . . . , 0). In the
entire sublinear phase, M−i = I + O(µS (n)/n). By Lemma 1 we can develop
asymptotics:
Vn =
jn µ (n) gn X
S
I
+
O
ξn2 i=1
n
×((n2 FF + inµS (n)(FG + GF) + oP (nsn jn ))
−B (n2 FF + inµS (n)(FG + GF) + oP (nsn jn ))
−(n2 FF + inµS (n)(FG + GF) + oP (nsn jn ))B
+B (n2 FF + inµS (n)(FG + GF) + oP (nsn jn )) B)
µ (n) S
× I+O
n
j
n
hn X
µS (n) +
I+O
ξn i=1
n
×((nFF + inµS (n)D + oP (nsn jn ))
−B ((nFF + inµS (n)D + oP (nsn jn ))
−((nFF + inµS (n)D + oP (nsn jn )) B
+B ((nFF + inµS (n)D + oP (nsn jn )) B) I + O
µ
S (n)
n
.
Let R := FG + GF. Collecting the like terms, after a lengthy calculation
we get
Vn =
σS2 (n)jn
µS (n)jn2
(FF
−
BF
F
−
FF
B
+
BFF
B)
+
ξn2
2ξn2 n(n − 1)
18
× (σS2 (n) − µS (n))n + µ2S (n))(R − BR − RB + BRB)
+(µS (n)n − σS2 (n) − µ2S (n))(D − BD − DB + BDB)
+oP (1)


1 −1 0 0 . . . 0
 −1
1 0 0 ... 0





σS2 (n)jn 
0
0
0
0
.
.
.
0


=

 0
2
0
0
0
.
.
.
0
ξn

..
.. .. . .
.. 

 ..
 .
. .
.
. .
0
0 0 0 ... 0

−2 3
 3
−4



µS (n)jn2
−1
1
+ 2
nσS2 (n) 
 0
0
2ξn n(n − 1)

..
 ..
 .
.
0
0

1
 −2


 1
2
+(nµS (n) − µS (n)) 
 0

 ..
 .
0

1 −1 0
 −1
0
1



0
1
−1
+σS2 (n) 
 0
0
0

..
..
 ..
 .
.
.
0
0
0
−1 0 . . . 0
1 0 ... 0


0 0 ... 0

0 0 ... 0
..
.. . .
.

. .. 
.
.
0 0 ... 0

−2 1 0 . . . 0
4 −2 0 . . . 0 


−2 1 0 . . . 0 

0
0 0 ... 0
..
..
.. . .
.

. .. 
.
.
.
0
0 0 ... 0

0 ... 0
0 ... 0
!

0 ... 0
 + oP (1).
0 ... 0
.. . .
.. 

. .
.
0 ... 0

Curiously, this expression can give many different asymptotics according
to the interplay between the factors µS (n), σS2 (n), the range sn , and the phase
of jn . In all cases ξn → ∞ is required
for convergence, and we consistently
√
used the range condition sn = o( n ). When the term σS2 (n)jn is dominant
19
in ξn , i.e. when
µS√
(n)jn
n
√
= o(σS (n) jn ), thus αn → 0, we get the convergence
1 −1 0 0 . . .
 −1
1 0 0 ...



0
0 0 0 ...
P
Vn −→ 
 0
0 0 0 ...

..
.. .. . .
 ..
 .
.
.
. .
0
0 0 0 ...
0
0


0
.
0
.. 

.
0


Under this scaling Part (b) of Theorem 1 follows.
However, when
µS (n)jn
√
2n
s
→ α ∈ (0, 1],
2
2
µS (n)jn
2
+ σS (n)j(n)
2n
we get Part (a) of Theorem 1.
As noted previously, when α = 1 the matrix in Part (a) is singular. The
reason for this singularity can be most easily seen in the standard case of
µS (n) ≡ 1, σS (n) ≡ 0 (a special case of α = 1). In this case we have, for
example (for large n),
(n)
D
Xjn ,0 ≈ N ne−jn /n ,
jn2 ,
2n
(n)
D
Xjn ,1 ≈ N jn e−jn /n ,
2jn2 ,
n
(cf. Kolchin, Sevastyanov and Chistyakov, 1978, Page 38); here the notaD
tion ≈ means approximate equality in distribution. The correlation between
2
−2jn /n
(n)
(n)
Xjn ,0 and Xjn ,1 is approximately (−jn /n)e
→ −1, as n → ∞, so the
2 /n
jn
bivariate limit distribution is degenerate; the same result holds for types 0
and 2, and for types 1 and 2, and for more general values of µS (n) and σS (n)
that result in α = 1. In other words, noting that, for α = 1, the rank of
the covariance matrix in Theorem 1 (a) is 1, there is a “proper” multivariate
central limit theorem for one combination of the three random variables—
(n)
(n)
(n)
in fact, Xjn ,0 is asymptotically normal, and Xjn ,1 and Xjn ,2 are both linear
(n)
combinations of Xjn ,0 .
20
(n)
For r > 2, the variables Xjn ,r have asymptotically normal distributions in
2 2
Part (a), but the variances can be of smaller order than µnnjn , as is the case
in standard coupon collection, with µS (n) ≡ 1, σS (n) ≡ 0.
In Part (b) of Theorem 2, a univariate central limit theorem holds for
r ≥ 2, but with variances of lower order than σn2 jn . As the covariance matrix
(n)
(n)
indicates, Xjn ,0 and Xjn ,1 have correlation −1, but the reason in this case is
different from that in Part (a). In this case, the term involving σS2 (n) domi(n)
(n)
nates the variances of Xjn ,0 and Xjn ,1 , and asymptotically the randomness in
both variables is all due to {Si (n)}, the random sequence of draws, causing
the degeneracy.
9
The linear phase
We are in the linear phase when j = jn ∼ λn n/µS (n). In this phase, we take
ξn =
v
u u
tn 1 +
σS2 (n) .
µS (n)
(n)
(n)
We show next that in the linear phase both Xj and Xj can be approximated by the leading term of the mean with ignorable errors.
Lemma 4 For j ∼ λn n/µS (n) and every r ∈ {0, 1, . . . , k}, we have
(n)
Xj,r = λn ne−λn + oL1 (n).
Proof . This proof involves extensive computation, and we only highlight
its salient points. Let j = jn be in the linear phase, i.e. j ∼ λn n/µS (n). The
exact covariance, as given in Proposition 1, has three parts: The first is a
double sum on r = 0, . . . , i and i = 0, . . . , j − 1; the middle part is a sum on
r = 0, . . . , j, and the third part is the negative of the product of the means
(n)
(n)
of Xj,` and Xj,m . By direct inspection of the exact mean (2) we see that in
the linear phase
λrn n −λn
(n)
e
+ O(µS (n)).
E[Xj,r ] =
r!
21
2r 2
n −2λn
Therefore, the third part is − λ`!nm!
e
+ o(n2 ). The middle part is also
2r n2
asymptotically λ`!nm!
e−2λn +o(n2 ). We thus get an exact cancelation of the n2
term between the second and third parts, leaving
behind o(n2 ).
In the first part, the term µνS (n)n−ν+1 νj (1−µS (n)/n)j−i−1−ν is the mean
of the number of balls of type ν after j − i − 1 draws. As we vary i and r
(up to the linear phase each) this term remains O(n). Combined with all
other elements, the first part is o(n2 ). The three parts combined give o(n2 )
covariance between types ` and m (for 0 ≤ `, m ≤ k).
Consequently, we have
h
(n)
E Xj,r −
2
λrn −λn 2 i
λr
(n)
(n)
ne
= Var[Xj,r ] + E[Xj,r ] − n ne−λn
r!
r!
2
= o(n ).
(6)
So, by Jensen’s inequality
h
(n)
E Xj,r
λrn −λn i
−
e n ≤
r!
which implies
(n)
Xj,r =
s
h
(n)
E Xj,r −
λrn −λn 2 i
e n
= o(n),
r!
λrn −λn
e n + oL1 (n). 2
r!
Lemma 5 For j ∼ λn n/µS (n) in the linear phase,
5√
(n) ∇Ỹj = 4(k + 1) 2 2k + 1sn .
Proof . It follows from the stochastic recurrence (5) that
(n) ∇Ỹj (n)
(n)
= Yj − Yj−1 (n)
(n)
= M−j Xj − M−j+1 Xj−1 (n)
(n)
= M−j Xj − MXj−1 µ (n) S
(n)
(n)
(n) (n)
= e(B−I)λn I + O
(Xj−1 + BHj − Hj ) − MXj−1 n
22
µ (n)
S
≤ 2 e(B−I)λn = 2e−λn (k
≤ 2e−λn (k
n
(n)
(n) (I − B) Xj−1 + B − I Hj µ (n) S
(n) (n) + 1)eλn B B − I Xj−1 + Hj √n
√
λn B √
+ 1)e 2k + 1 (sn k + 1 + sn k + 1 ).
Note that
1
λ
 n
0
1

eλn B =









0
0
..
.
..
.
..
.
λkn
k!
λk−1
n
(k − 1)!
λk−2
n
(k − 2)!
... 0
... 0


...
..
.
... 1



,





with norm less than (k + 1)eλn . 2
Lemma 6 For j ∼ λn n/µS (n) in the linear phase,
Un :=
jn
X
h 1
i
P
(n) 2
E ∇Ỹi 1n 1
o Fi−1 −→ 0.
(n) ξ
n
∇Ỹi > ε
i=1
ξn
Proof
0, according to Lemma 5 and the constraint
√. For any given1 ε > (n)
sn = o( n ), the sets {|| ξn ∇Ỹi || > ε} are all empty, for all n greater than
some positive integer n0 (ε). For n ≥ n0 , we have
n0 (ε)
Un
16(k + 1)5 (2k + 1)s2n
≤
σS2 (n) j=1
n 1+
µS (n)
5
16(k + 1) (2k + 1)n0 (ε)s2n
≤
σ 2 (n) n 1+ S
µS (n)
→ 0,
as n → ∞. 2
X
To handle the conditional variance condition, we break up the sum over 1
to jn ∼ λn n/µS (n) at some point near the beginning of the linear phase.
23
More precisely, choose a small positive < Q1 and break up the sum in Vn
into a sum going from 1 to bεn/µS (n)c − 1 and a sum starting at bεn/µS (n)c
and ending at jn . For large n, we write Vn in the form
Vn
=
jn
1 X
(n)
(n)
(n)
(n)
(n)
(n)
gn M−i (Xi−1 Xi−1 − BXi−1 Xi−1 − Xi−1 Xi−1 B
2
ξn i=1
(n)
(n)
+BXi−1 Xi−1 B) M
−i
(n)
+ hn M−i (Di
(n)
− BDi
(n)
− Di B
−i
(n)
+BDi B) M
=
1
ξn2
=:
a0n
bεn/µS (n)c−1
X
+
i=1
+
1
ξn2
jn
X
i=bεn/µS (n)c
a00n .
√
According to the restriction sn = o( n ), we get
bεn/µS (n)c−1
bεn/µS (n)c−1
X
X
gn
hn
2
O(n
)
+
O(n)
2
2
σS (n) σS (n) i=1
i=1
n 1+
n 1+
µS (n)
µS (n)
= o(ε),
as ε → 0.
a0n =
Let
i2 µ2S (n)
ik µkS (n) ...
.
2n
k! nk−1
According to Lemma 4, for i ≤ j in the linear phase,
(n)
Ni
= e−
iµS (n)
n
n iµS (n)
(n)
(n)
(n)
(n)
Xi−1 = Ni
+ oL1 (n),
and subsequently,
(n)
(n)
Xi−1 Xi−1 = Ni Ni
+ oL1 (n2 ).
Let
ik µkS (n) ,
=e
Diag n, iµS (n), . . . ,
k! nk−1
and go further with the computation
(n)
Li
a00n
−
iµS (n)
n
jn
µ (n) iµS (n)
gn X
S
= 2
e− n (B−I) I + O
ξn i=bεnc
n
24
(n)
(n)
× (Ni Ni
(n)
(n)
−(Ni Ni
×e−
+hn
iµS (n)
(B−I)
n
jn
X
+ oL1 (n2 ))
(n)
(n)
+ oL1 (n2 )) B + B (Ni Ni
I+O
µ
S (n)
+ oL1 (n2 )) B
n
µ
e−
iµS (n)
(B−I)
n
(n)
+ oL1 (n)) − B (Li
I+O
S (n)
n
i=bεnc
×((Li
(n)
(n)
+ oL1 (n2 )) − B (Ni Ni
(n)
+ o(n))
(n)
(n)
−(Li
+ oL1 (n)) B + B (Li
µ (n) iµS (n)
S
×e− n (B−I) I + O
.
n
+ oL1 (n)) B)
To be able to go through this computation, we first simplify the matrix
exponentiation:

e−
iµS (n)
(B−I)
n
=e
iµS (n)
n








1
− iµSn(n)
0
1
0
0
1
..
.
− iµSn(n)
..
.
... 0
... 0

... 0
,
.. 
..
. .

... 1
i2 µ2S (n)
2n2
(−i)k µkS (n)
k! nk
k−1
(−i)k−1 µS
(n)
(k−1)! nk−1


(−i)k−2 µk−2
(n)
S
(k−2)! nk−2
iµS (n)
and e− n (B−I) is, of course, its transpose. Multiplying out, we get the
second sum

a00n
1 −1 0 . . .

1 0 ...
 −1
jn
X

gn
 0
0 0 ...
=
..
σS2 (n) i=bεn/µ (n)c 
..
 ..
S
.
.
 .
n 1+
µS (n)
0
0 0 ...
hn
+ σS2 (n) n 1+
µS (n)
jn
X
×
(e
iµS (n)
n

0
0

 2
0
(n + o(n2 ))
.. 

.
0
+ o(1)) ci,n ,
i=bεn/µS (n)c
where ci,n is an effectively computable matrix, for example, for 0 ≤ r ≤ k,
we have
(−1)r ir−1 µr−1
S (n)
ci,n (0, r) =
(iµS (n) + rn),
r! nr−1
25
and
ci,n (1, 1) =
1 2 2
(i µS (n) + 3inµS (n) + n2 ),
n
etc.
When we put everything together, many cancelations take place, and
a tremendous amount of calculation is needed. We only hint to how one
covariance may be obtained to give an indication of the work involved. Let
us take the (0, 0) entry (which is one of the simplest). Mahmoud (2010)
gives this calculation in detail for the case of bounded range sn = O(1), in
which βn → β ∈ (0, 1]. Let us take here the opposite case, when the variance
dominates the mean, i.e. when µS (n) = o(σS2 (n)), a case where βn → 0. We
get
Vn (0, 0) = a0n (0, 0) + a00n (0, 0)
jn
X
gn
n2
= O(ε) +
2
nσS (n)/µS (n) i=bεn/µS (n)c
jn
X
iµS (n)
hn
(1 + o(1))
ne n
2
nσS (n)/µS (n) i=bεn/µS (n)c
+
j εn k
nµS (n)gn j
−
+
1
n
σS2 (n)
µS (n)
(jn +1)µS (n)/n
h
− 1
µS (n)hn e
+ 2
σS (n)
eµS (n)/n − 1
ebεn/µS (n)c − 1 i
−
(1 + o(1)).
eµS (n)/n − 1
= O(ε) +
Let ε approach 0, and write the limit
Vn (0, 0) =
nµS (n)gn
(jn + 1)
σS2 (n)
µS (n)hn e(jn +1)µS (n)/n − 1 + 2
(1 + o(1)).
σS (n)
eµS (n)/n − 1
We are in a phase where jn ∼ λn n/µS (n), and the latter expression is asymptotically
Vn (0, 0) ∼
nµS (n)((σS2 (n) − µS (n))n + µ2S (n)) λn n n2 (n − 1)σS2 (n)
µS (n)
26
+
µS (n)((µS (n)n − σS2 (n) − µ2S (n)) eλn − 1 .
n(n − 1)σS2 (n)
µS (n)/n
We next use the assumptions about the mean and variance being small relative to n, and the dominance of the variance to arrive at:
Vn (0, 0) ∼ λn +
µS (n) λn
(e − 1) = λn + o(1).
σS2 (n)
So, if λn → λ > 0, we have Vn (0, 0) → λ. In a like manner we can obtain
the other entries of the limit of Vn , and find that they are all 0, except the
entries (`, m), for 0 ≤ `, m ≤ 1, and these are (−1)`+m λ.
For β > 0, an application of the martingale central limit theorem gives
n
0
(n)
 
M−jn Xjn −  .. 
.
1−β 0 0
D
√
−→ eλ e−λB Nk+1 0, Jk+1 +
Jk+1 .
n
β


As we assumed λn → λ > 0, we have M−jn → eλ e−λB . By an application of
(multivariate) Slutsky’s theorem, we get the statement of Theorem 2 (a).
For β = 0, further cancelations occur, obliterating the terms of order n
in n(1 + σS2 (n)/µS (n))Vn , leaving behind terms of the order nσS2 (n)/µS (n).
Calculations (not shown) similar to those highlighted in the case of β > 0
give
n

0
1 −1 0 0 . . .
 
−jn (n)
M Xjn −  .. 

.
1 0 0 ...
 −1


0
0 0 0 ...
0
D
s
−→ Nk+1 0, λ 

0 0 0 ...
n
 0
..
.. .. . .
 ..
σS (n)
 .
.
.
. .
µS (n)
0
0 0 0 ...


0
0
!

0
 .
0
.. 

.
0

Again, as we assumed λn → λ > 0, we have M−jn → eλ e−λB , and an
application of multivariate Slutsky’s theorem, yields
(n)
Xjn − ne−λ ν k+1 D
s
−→ Nk+1 (0, J0k+1 ),
n
σS (n)
µS (n)
27
provided that λ is not a positive integer that is at most k. When λ is an
integer that is at most k, the covariance matrix J0k+1 is degenerate (if λ = r,
the rth row and column are all zeroes) and only the term from Jk+1 is present
in the variance.
10
Illustrating examples
The covariance formula in Proposition 1 is not easy to reduce, however, we
can manage to get compact forms for small ` and m. For example, extracting
the (0,1) entry from this form, we get
(n) (n)
E[Xj,0 Xj,1 ]
j−1
i
XX
i (n − µS (n))j−i+2r−2
gni−r
= hn
r
nj−i+2r−2
i=0 r=0
×((i + 1)µS (n) − (i + 1 − r)n)
+n
2
!
j
X
r=0
j (n − µS (n))2r−1 (jµS (n) − (j − r)n)
.
n2r
r
!
gnj−r
Reducing these sums we get
(n)
(n)
E[Xj,0 , Xj,1 ] = j(µS (n)n − σS2 (n) − µ2S (n))
(n − µ(n))(n − µ2 (n) − 1) + σ 2 (n)) j−1
S
S
.
×
n(n − 1)
(n)
(n)
Subtracting E[Xj,0 ] E[Xj,1 ], we get the covariance
(n)
(n)
Cov[Xj,0 Xj,1 ] = j(µS (n)n − σS2 (n) − µ2S (n))
(n − µ(n))(n − µ2 (n) − 1) + σ 2 (n)) j−1
S
S
×
n(n − 1)
µS (n) 2j−1
−jnµS (n) 1 −
.
n
Likewise,
(n)
− µS (n) − 1) + σS2 (n) j−1
n(n − 1)
n − µ (n) j i
n − µ (n) 2j
S
S
+
− n2
,
n
n
h
(n − nµ
Var[Xj,0 ] = n (n − 1)
S (n))(n
28
and
n − µ
(n)
Var[Xj,1 ] = jµS (n)
+j
S (n) j−1
n
h
j(σS2 (n)
+
n − µ
− j 2 µ2S (n)
µ2S (n)
− µS (n)n)
S (n) 2j−2
n
2
i
+(n − 1)(σS2 (n)n − µS (n)(n − µS (n)))
×((n − µS (n))(n − µS (n) − 1) + σS2 (n))j−2
j−1
1
.
×
n(n − 1)
10.1
Standard coupon collection
Consider the standard Dixie Cup problem, where S ≡ 1. We illustrate the
covariance computation at the beginning of this section on coupons of types 0
and 1. Here we have the covariances
(n)
n − 2 j−1
(n)
Cov[Xj,0 , Xj,1 ] = j(n − 1)
n
− jn
n − 1 2j−1
n
∼ −e−2λ λ2 n,
(n)
h
Var[Xj,0 ] = n (n − 1)
n − 2 j
+
n − 1 j i
− n2
n − 1 2j
n
n
∼ e−2λ (λeλ − λ − 1) n,
n − 1 j
n − 1 2j−2
(n)
Var[Xj,1 ] = j
− j2
n
n
j(j − 1)(n − 1) n − 2 j−2
+
n
n
∼ e−2λ (λeλ − λ + λ2 − λ3 ) n.
n
Thus, in the linear phase (when µS (n)jn = jn ∼ λn n, for λn convergent to a
fixed λ > 0), the asymptotic covariance matrix of types 0 and 1 is
(n)
Cov[Xj ]
−2λ
= J2 n + o(n) = e
λ
e
−1−λ
−λ2
−λ2
λeλ − λ + λ2 − λ3
as in Kolchin, Sevastyanov
and Chistyakov (1978, Page 38).
r
µ2 (n)j 2
q
2
n + o(n),
q
2
jn
n
S
+ σS2 (n)jn = 2n
, and so sn = 1 = o jnn is
In this example
2n
√
only satisfied in the upper sublinear phase n = o(jn ). Here, αn ≡ 1, and
29
Theorem 1 (a) gives
n
 j 
 n 

1 jn 

(n)
Xj n − 1 −



 2 
 jn 


n

2n
D
√
jn / n
−→ N3

1 −2 1 1
0,  −2 4 1 
 ,
2
1 −2 1
√
starting at the phase jn / n → ∞, and going all the way to the end of the
sublinear phase at jn = o(n). (The limiting covariance matrix is singular,
but each of the types 0, 1 and 2 satisfies a univariate central limit theorem.)
In the linear phase, when jn ∼ λn n (for λn convergent to a fixed λ > 0),we
have βn ≡ 1 in Theorem 2 (a), giving
(n)
Xjn − ne−λ ν k+1 D
√
−→ Nk+1 (0, Jk+1 ).
n
The results are not very different, if a fixed number S ≡ s is acquired in
each purchase. Essentially, all the results above stay the same, with the rth
component of the shift factor (asymptotic mean of the sampling distribution)
scaled by sr , for r = 0, 1, 2, and the limiting covariance matrix is multiplied
by s2 .
10.2
An example with a sampling distribution with
fixed range
Suppose the sampling distribution has the distribution of 1 + Bin(s, 21 ), with
fixed s. Here, µS (n) = 1 + 12 s, and σS2 (n) = 41 s. In this example,
s
sn = s + 1 = o
µ2S (n)jn2
+ σS2 (n)jn .
2n
30
And so, sn = o(jn ) throughout the entire growing sublinear phase. Subsequently, αn → 0 in Theorem 1 (b), and we get the central limit result

(n)
Xj n

n
s + 2 jn 
 (s + 2)j
− 1−
n

2n

2
√
jn






D
−→ N2
s2
0,
16
1 −1
−1 1
,
applying in the entire growing sublinear phase, where jn → ∞, and jn = o(n).
(Again, the matrix is singular.)
In the linear phase, when jn ∼ 2λn n/(s + 2), for λn convergent to a fixed
λ > 0, we get β = (2s + 4)/(3s + 4) > 0 and Theorem 2 (a) gives
(n)
Xjn − ne−λ ν k+1 D
s
√
−→ Nk+1 0, Jk+1 +
J0k+1 .
n
2s + 4
10.3
An example with a nearly degenerate sampling
distribution on a two-point set
Suppose the sampling distribution is the two-point distribution





Prob(Sj = k) = 
5
,
ln n


1 −
if k = dln ne − 1;
5
, if k = dln ne,
ln n
for n ≥ 149. Here, we have
5
5 5
(dln ne − 1) + 1 −
dln ne = dln ne −
∼ ln n,
ln n
ln n
ln n
5
5 5
25
σS2 (n) =
(dln ne − 1)2 + 1 −
dln ne2 − µ2S (n) =
− 2 → 0.
ln n
ln n
ln n ln n
µS (n) =
The sampling distribution is concentrated at the point dln ne; in fact S(n) −
P
dln ne −→ 0. In this example,
sn = dln ne = o
s
j 2 ln2 n
n
2n
31
+
5jn ,
ln n
2
2
ln n
does not begin to
for any jn in the phase ln n = o(jn ). The term jn 2n
3
5jn
dominate ln
,
until
j
is
at
least
of
the
order
n/
ln
n,
which
is very close
n
n
to the linear phase (that begins at the order n/ ln n). So, we have three
sublinear phases. In the early sublinear phase, beginning at ln n = o(jn ),
5jn
and going up to o(n/ ln3 n), the term ln
is dominant. In this phase αn → 0,
n
and by Theorem 1 (b), we get (for types 0 and 1)
n

(n)
Xjn − 1 −


dln ne jn 


 jn ln n 


n
n
q
D
−→ N2
jn / ln n
1 −1
0, 5
−1 1
2
.
2
ln n
5jn
Then comes the middle sublinear phase, where jn 2n
∼ q ln
, for some q > 0.
n
Theorem 1 (a) applies with α = q/(q+1). In the upper sublinear phase, where
n/ ln3 n = o(jn ), but jn remains sublinear, i.e. jn = o(n/ ln n). Theorem 1 (a)
applies with α = 1. Hence only in the middle sublinear phase do we get a
nondegenerate trivariate central limit theorem for this example.
In the linear phase, when jn ∼ λn n/ ln n (for λn convergent to a fixed
λ > 0), βn → 1, for types 0, 1, . . . , k we get
(n)
Xjn − ne−λ ν k+1 D
√
−→ Nk+1 (0, Jk+1 ).
n
10.4
An example with a uniform sampling distribution
on a growing range
Consider coupon collection, where at each purchase a uniformly distributed
1
number on the set {1, 2, . . . , 12dn 8 e} is acquired. In this example,
1
µS (n) ∼ 6n 8 ,
and
1
σS2 (n) = 12n 4 .
With these rates of growth we have the condition
s
1
8
sn = 12dn e = o
µ2S (n)jn2
+ σS2 (n)jn ,
2n
32
which means sn = o
r
2
18jn
3
n4
1
+ 12jn n 4 is automatically satisfied in the grow7
ing sublinear phase. In the growing sublinear phase, where jn = o(n 8 ),
1
σS2 (n)jn dominates µ2S (n)jn2 /(2n) in the upper sublinear phase n 4 = o(jn ).
So, Theorem 1 (b) applies, giving
(n)
Xj n − 1 −
1
8

e + 1 jn
12dn
2n
1√
n 8 jn



n
6jn n
1
8


D
−→ N2
1 −1
0, 12
−1 1
,
7
holding throughout the entire sublinear phase (ending at o(jn8 )).
For Theorem 2, we compute µS (n)/(µS (n) + σS2 (n)) → 0. In the linear
7
phase, when jn ∼ 61 λn n 8 (for λn convergent to a fixed λ > 0), by Theo(n)
rem 2 (b), Xjn (properly normalized) has limiting covariance matrix J0k+1 ,
provided λ is not a positive integer, and for r = 0, 1, . . . k, each type satisfies
a univariate central limit theorem.
References
[1] De Moivre, A. (1718). The Doctrine of Chances: or, a Method of Calculating the Probabilities of Events in Play. Chelsea, New York, edition
reprinted in 1967, Problem XXXIX.
[2] Hall, P. and Heyde, C. (1980). Martingale Limit Theory and Its Applications. Academic Press, New York.
[3] Kobza, J., Jacobson, S. and Vaughan, D. (2007). A survey of the coupon
collector’s problem with random sample sizes. Methodology and Computing in Applied Probability, 9, 573–584.
[4] Kolchin, V., Sevastyanov, B. and Chistyakov, V. (1978). Random Allocations, Moscow.
[5] Laplace, P. (1774). Mémoire sur les suites récurro-récurrents et leurs
usages dans la théorie des hasards. Mém. Acad. Roy. Sci. Paris 6, 353–
371.
33
[6] Mahmoud, H. (2010). Gaussian phases in generalized coupon collection.
Advances in Applied Probability, 42, 994–1012.
[7] Smythe, R. (2011). Generalized coupon collection: the superlinear case.
Journal of Applied Probability, 48, 189–199.
[8] Stadje, W. (1990). The collector’s problem with group drawings. Advances in Applied Probability, 22, 866–882.
[9] Stuart, A. and Ord, K. (1987). Kendall’s Advanced Theory of Statistics,
Volume 1 , Fifth edition. Oxford University Press, New York.
34