Applied Probability Trust (9 October 2014)
ESTIMATION OF INTEGRALS WITH RESPECT TO INFINITE
MEASURES USING REGENERATIVE SEQUENCES
KRISHNA B. ATHREYA,∗ Iowa State University
VIVEKANANDA ROY,∗∗ Iowa State University
Abstract
Let f be an integrable function on an infinite measure space (S, S, π). We
show that if a regenerative sequence {Xn }n≥0 with canonical measure π could
R
be generated then a consistent estimator of λ ≡ S f dπ can be produced. We
further show that under appropriate second moment conditions, a confidence
interval for λ can also be derived. This is illustrated with estimating countable
sums and integrals with respect to absolutely continuous measures on Rd using
simple symmetric random walk (SSRW) on Z.
Keywords: Markov chains, Monte Carlo, improper target, random walk,
regenerative sequence.
2010 Mathematics Subject Classification: Primary 65C05
Secondary 60F05
1. Introduction
R
Let (S, S, π) be a measure space. Let f : S → R be S measurable, and S |f |dπ <
R
∞. The goal is to estimate λ ≡ S f dπ. If π is a probability measure, that is,
π(S) = 1, a well-known statistical tool is to estimate λ by sample averages f¯n ≡
Pn
n
This iid Monte Carlo
j=1 f (ξj )/n based on iid observations, {ξj }j=1 , from π.
method is a fundamental notion in statistics and has made the subject very useful
in many areas of science. A refinement of this result is via the central limit theorem
R
(CLT) from which it follows that if S f 2 dπ < ∞, then an asymptotic (1 − α) level
∗
Postal address: Department of Mathematics and Statistics, Iowa State University, Ames, Iowa,
50011, USA.
∗∗ Postal address: Department of Statistics, Iowa State University, Ames, Iowa, 50011, USA.
Email address: vroy@iastate.edu
1
2
K. B. Athreya and V. Roy
√
√
confidence interval for λ can be obtained as In ≡ (f¯n − zα σn / n, f¯n + zα σn / n),
P
where σn2 = nj=1 f 2 (ξj )/n − f¯n2 and zα is such that P (|Z| > zα ) = α where Z is a
N (0, 1) random variable. Here P (λ ∈ In ) → 1 − α as n → ∞.
On the other hand, if it is difficult to sample directly from π then the above classical
(iid) Monte Carlo cannot be used to estimate λ. In the pioneering work of [13] the
target distribution π was the so called Gibbs measure on the configuration space (a
finite but a large set) in statistical mechanics and was difficult to generate iid sample
from. In this paper, they constructed a Markov chain {Xn }n≥0 that was appropriately
irreducible and had π as its stationary distribution. Then they used a law of large
numbers for such chains, that asserts that if {Xn }n≥0 is a suitably irreducible Markov
chain and has a probability measure π as its invariant distribution, then for any initial
P
distribution of X0 , the “time average” nj=1 f (Xj )/n converges almost surely to the
R
Pn
“space average” λ = S f dπ as n → ∞ ([14] Theorem 17.0.1). So
j=1 f (Xj )/n
provides a consistent estimator of λ. In the late 80’s and early 90’s a number of
statisticians became aware of the work of [13] and adapted it to solve some statistical
problems. Thus, a new statistical method (for estimating integrals with respect to
probability distributions) known as the Markov chain Monte Carlo (MCMC) method
was born. And since then the subject has exploded in theory and applications see e.g
R
[17]. Here also, if S f 2 dπ < ∞ then under certain conditions on mixing rates of the
Pn
chain {Xn }n≥0 , a CLT is available for the time average estimator j=1 f (Xj )/n and
from which a confidence interval estimate for λ can be produced.
Recently, [2] have shown that the standard Monte Carlo (both iid Monte Carlo and
Markov chain Monte Carlo) methods are not applicable for estimating λ in the case of
improper targets, that is, when π(S) = ∞. In particular, they showed that the usual
Pn
time average estimator, i=1 f (Xi )/n, based on a recurrent Markov chain {Xn }n≥0
with invariant measure π (with π(S) = ∞) converges to 0 with probability one and
hence is inappropriate. They provided consistent estimators of λ based on regenerative
sequences of random variables whose canonical measure is π.
A sequence of random variables is regenerative if it probabilistically restarts itself at
random times and thus can be broken up into iid pieces. Below is the formal definition
of regenerative sequences.
3
Estimation of integrals using regenerative sequences
Definition 1. Let (Ω, F , P ) be a probability space and (S, S) be a measurable space.
A sequence of random variables {Xn }n≥0 defined on (Ω, F , P ) with values in (S, S)
is called regenerative if there exists a sequence of (random) times 0 < T1 < T2 < . . .
such that the excursions {Xn : Tj ≤ n < Tj+1 , τj }j≥1 are iid where τj = Tj+1 − Tj for
j = 1, 2, . . . , that is,
P (τj = kj , XTj +q ∈ Aq,j , 0 ≤ q < kj , j = 1, . . . , r)
r
Y
P (τ1 = kj , XT1 +q ∈ Aq,j , 0 ≤ q < kj ),
=
j=1
for all k1 , . . . , kr ∈ N, the set of positive integers, Aq,j ∈ S, 0 ≤ q < kj , j = 1, . . . , r,
and r ≥ 1 and these are independent of the initial excursion {Xj : 0 ≤ j < T1 }. The
random times {Tn }n≥1 are called regeneration times.
The standard example of a regenerative sequence is a Markov chain that is suitably
irreducible and recurrent. A regenerative sequence need not have the Markov property.
In particular, it need not be a Markov chain (see [2] for examples). Let
π(A) ≡ E
TX
2 −1
j=T1
IA (Xj ) for A ∈ S.
(1)
The measure π is called the canonical (or, occupation) measure for regenerative sequence {Xn }n≥0 with regeneration times {Tn }n≥0 . Let Nn = k if Tk ≤ n < Tk+1 , k, n =
1, 2, . . . . That is, Nn denotes the number of regenerations by time n. [2] showed that
R
the following estimator λ̂n , called the regeneration estimator for estimating λ ≡ S f dπ
R
(assuming S |f |dπ < ∞) is indeed consistent. That is,
λ̂n ≡
Pn
j=0
f (Xj )
Nn
a.s.
−→ λ.
(2)
Thus, given a (proper or improper) measure π, if we can find a regenerative sequence
R
with π as its canonical measure, then λ ≡ S f dπ can be estimated by the above
regeneration estimator (2). It may be noted that this regenerative sequence Monte
Carlo (RSMC) works whether π(S) is infinite or finite. If π(S) < ∞ then the strong
law of large numbers implies that Nn , the number of regenerations by time n, grows
at the rate n/π(S) (since E(T2 − T1 ) = π(S)) as n goes to ∞. Thus, when π(S) is
finite one has at least three choices of Monte Carlo methods of estimation, namely
4
K. B. Athreya and V. Roy
the iid Monte Carlo (iid MC), Markov chain Monte Carlo (MCMC), and regenerative
sequence Monte Carlo (RSMC). This last Monte Carlo method, that is, RSMC has a
natural universality property, namely, it works whether one knows the target π is finite
or infinite measure.
The regenerative property of positive recurrent Markov chains has been used in
the MCMC literature for calculating standard errors of MCMC based estimates for
integrals with respect to a probability distribution (see for example [15], [10], [17]
chapter 12, [1] Section IV.4). Regenerative methods of analyzing simulation based
output also have a long history in the operations research literature (see for example
[6], [9]). But in these methods, the excursion time τ1 is assumed to have finite second
moment, which does not hold when the target distribution is improper. In fact, E(τ1 ) =
E(T2 − T1 ) = π(S) = ∞ when π is improper. On the other hand, the RSMC method
does not require the existence of even the first moment of τ1 .
[2] developed algorithms based mainly on random walks for estimating λ when S
is countable as well as S = Rd for any d ≥ 1. This leads to the very important
question of how to construct a confidence interval for λ based on λ̂n . An approximate
distribution of (λ̂n −λ) can be used for estimating Monte Carlo error of the regeneration
estimator λ̂n . In this paper, we are able to obtain an asymptotic confidence interval
PT2 −1
f (Xi ) (this is not the
for λ under the assumption of finite second moments of i=T
1
same as requiring E(T2 − T1 )2 < ∞) and a regularly varying tail of the distribution
of the regeneration time τ1 . We make use of a deep result due to [12] for obtaining
the limiting distribution of (suitably normalized) (λ̂n − λ). We then apply our general
results to the algorithms based on the simple symmetric random walk (SSRW) on Z
presented in [2] for the case when S is countable as well as S = Rd for some d ≥ 1.
We provide simple conditions on f under which a confidence interval based on λ̂n is
available in both cases, that is, when S is countable or S = Rd and π is absolutely
continuous and [2]’s algorithms based on the SSRW is used for estimating λ.
2. Main Results
Theorem 1. Let {Xn }n≥0 be a regenerative sequence as in Definition 1. Let π be its
canonical measure as defined in (1). Let f : S → R be S measurable.
5
Estimation of integrals using regenerative sequences
1. Assume
E
TX
2 −1
j=T1
Let Ui =
TX
Z
2
2 −1
|f |dπ < ∞ and E
|f (Xj )| =
f (Xj ) < ∞.
PTi+1 −1
j=Ti
S
(3)
j=T1
f (Xj ), i = 1, 2, 3, . . . , and 0 < σ 2 ≡ EU12 − λ2 < ∞. Let
Yk =
Pk
j=1 (Uj
√
σ k
− λ)
,
(4)
Yk −→ N (0, 1).
(5)
for k = 1, 2, . . . . Then,
(a) as k → ∞,
d
(b) Let Yk (t), t ≥ 0 be the linear interpolation of Yk on [0, ∞), that is,
Yk (t) ≡ Y[kt] + (kt − [kt])
(U[kt]+1 − λ)
√
.
kσ
d
Then as k → ∞, {Yk (t) : t ≥ 0} −→ {B(t) : t ≥ 0}, in C[0, ∞), where
{B(t) : t ≥ 0} is the standard Brownian motion.
2. Assume
P (τ1 > x) ∼ x−α L(x) as x → ∞,
(6)
where 0 < α < 1 and L(·) is slowly varying, i.e., ∀ 0 < c < ∞, L(cx)/L(x) → 1
as x → ∞. Then
(a) π(S) = E(T2 − T1 ) = ∞
(b) Let Nn = k if Tk ≤ n < Tk+1 , k ≥ 1, n ≥ 1. Then
Nn
d
−→ Vα as n → ∞,
nα /L(n)
(7)
where P (Vα > 0) = 1, and for 0 < x < ∞, P (Vα ≤ x) = P (Ṽα ≥ x−1/α )
with Ṽα being a positive random variable with a stable distribution with index
α such that
E(exp(−sṼα )) = exp(−sα Γ(1 − α)), 0 ≤ s < ∞
where Γ(p) =
R∞
0
xp−1 exp(−x)dx, is the Gamma function, 0 < p < ∞.
6
K. B. Athreya and V. Roy
3. Assume that
E
2 −1
TX
j=T1
2
|f (Xj )| < ∞.
(8)
and (6) holds. Then
(a)
√
(λ̂n − λ) Nn d
−→ N (0, 1) as n → ∞.
σ
(9)
p
(λ̂n − λ) nα /L(n) d
−→ Q as n → ∞,
σ
(10)
(b)
where Q ≡ B(Vα )/Vα , {B(t) : t ≥ 0} is the standard Brownian motion and
Vα is as in (7) and independent of {B(t) : t ≥ 0}.
(c) Let
σn2 =
Nn
X
i=1
Ui2 /Nn − λ̂2n .
(11)
Then
i.
ii.
√
(λ̂n − λ) Nn d
−→ N (0, 1) as n → ∞.
σn
(12)
p
(λ̂n − λ) nα /L(n) d
−→ Q as n → ∞,
σn
(13)
where Q is as in (10).
Using the results in Theorem 1 one can construct asymptotic confidence intervals
for λ based on the regenerative sequence {Xi }ni=0 as discussed below in Corollary 1.
Corollary 1. Fix 0 < p < 1, and let zp , qp be such that P (|Z| > zp ) = p and P (Q >
√
qp ) = p, where Z ∼ N (0, 1) and Q is as in (10). Let In1 ≡ (λ̂n − zp σn / Nn , λ̂n +
p
p
√
zp σn / Nn ), and In2 ≡ (λ̂n − qp/2 σn / nα /L(n), λ̂n − q1−p/2 σn / nα /L(n)). Let
p
√
l(In1 ) = 2zp σn / Nn , and l(In2 ) = (qp/2 − q1−p/2 )σn / nα /L(n) be the lengths of
the intervals In1 and In2 respectively. Then
1. P (λ ∈ Ini ) → 1 − p as n → ∞ for i = 1, 2.
2.
p
√
d
nα /L(n)l(In1 ) −→ 2zp σ/ Vα , where Vα is as in (10).
7
Estimation of integrals using regenerative sequences
3.
p
a.s.
nα /L(n)l(In2 ) −→ (qp/2 − q1−p/2 )σ.
Below we consider a special case of Theorem 1 in the case when S is countable.
[2] provided an algorithm (Algorithm I in their paper) based on the SSRW on Z for
consistently estimating countable sums. We provide a simple sufficient condition for
the second moment hypothesis (8) in this case so that we can obtain confidence interval
as well. Since S is countable, without loss of generality, we can take S = Z in this case.
Theorem 2. Let {Xn }n≥0 be a simple symmetric random walk (SSRW) on Z starting
at X0 = 0. That is,
Xn+1 = Xn + δn+1 , n ≥ 0
where {δn }n≥1 are iid with distribution P (δ1 = +1) = 1/2 = P (δ1 = −1) and
Pn
independent of X0 . Let Nn =
j=0 I(Xj = 0) be the number of visits to zero
by {Xj }nj=0 . Assume that πi ≡ π(i) ≥ 0 for all i. Let f : Z → R be such that
P
j∈Z |f (j)|π(j) < ∞. Then
1.
λ̂n ≡
Pn
2. Assume in addition,
j=0
f (Xj )π(Xj )
Nn
P
j∈Z
a.s.
−→ λ ≡
X
i∈Z
f (i)πi as n → ∞.
(14)
p
|f (j)|π(j) |j| < ∞. Then
PT1 −1
(a) E( j=0
|f (Xj )|π(Xj ))2 < ∞, where T1 =min {n : n ≥ 1, Xn = 0},
(b)
(c)
√
Nn (λ̂n − λ) d
−→ N (0, 1), as n → ∞,
σ
(15)
PT1 −1
f (Xj )π(Xj ))2 − λ2 ,
where σ 2 ≡ E( j=0
(λ̂n − λ)n1/4 d B(V1/2 )
−→
, as n → ∞,
σ
V1/2
(16)
where {B(t) : t ≥ 0} is the standard Brownian motion, V1/2 is a random
variable independent of {B(t) : t ≥ 0}, and V1/2 has the same distribution
p
as π/2|Z|, Z ∼ N (0, 1).
(d) Also the analogues of (12), (13) and Corollary 1 hold.
8
K. B. Athreya and V. Roy
Lastly we consider the algorithm presented in [2] (Algorithm III in their paper) that
is based on the SSRW on Z and a randomization tool to estimate integrals with respect
to an absolutely continuous measure π on any Rd , d < ∞. Let f : Rd → R and π be
an absolutely continuous measure on Rd with Radon-Nikodym derivative p(·). Assume
R
that Rd |f (x)|p(x)dx < ∞. The following theorem provides a consistent estimator as
R
well as an interval estimator of λ ≡ Rd f (x)p(x)dx.
Theorem 3. Let {Xn }n≥0 be a SSRW on Z with X0 = 0. Let {Uij : i = 0, 1, . . . ; j =
1, 2, . . . , d} be a sequence of iid Uniform (−1/2, 1/2) random variables and independent
of {Xn }n≥0 . Assume κ : Z → Zd be 1 − 1, onto. Define Wn := κ(Xn ) + U n ,
n = 0, 1, , . . . where U n = (Un1 , Un2 , . . . , Und ). Note that the sequence {Wn }n≥0 is
regenerative with regeneration times {Tn }n≥0 being the returns of SSRW {Xn }n≥0 to
zero. Then
1.
λ̂n ≡
where Nn =
Pn
j=0
Pn
j=0
f (Wj )p(Wj )
Nn
a.s.
−→ λ ≡
Z
f (x)p(x)dx,
(17)
Rd
I(Xj = 0) is the number of visits to zero by {Xj }nj=0 .
2. Let g : Z → R+ be defined as
g(r) ≡
Z
p
E(|f (κ(r) + U )|p(κ(r) + U ))2 =
(|f (κ(r)+u)|p(κ(r)+u))2 du
[−1/2,1/2]d
(18)
where U = (U1 , U2 , . . . , Ud ) with Ui ’s, i = 1, 2, . . . , d, are iid Uniform (−1/2, 1/2)
random variables, and [−1/2, 1/2]d is the d-dimensional rectangle with each side
p
P
being [−1/2, 1/2]. Assume, r∈Z g(r) |r| < ∞. Then
PT1 −1
(a) E( j=0
|f (Wj )|π(Wj ))2 < ∞, where T1 =min {n : n ≥ 1, Xn = 0},
(b)
(c)
√
Nn (λ̂n − λ) d
−→ N (0, 1), as n → ∞,
σ
(19)
PT1 −1
where σ 2 ≡ E( j=0
f (Wj )π(Wj ))2 − λ2 ,
(λ̂n − λ)n1/4 d B(V1/2 )
−→
, as n → ∞,
σ
V1/2
where {B(t) : t ≥ 0}, and V1/2 are as in (16).
(20)
1/2
,
9
Estimation of integrals using regenerative sequences
(d) Also the analogues of (12), (13) and Corollary 1 hold.
Remark 1. A sufficient condition for
Let
h(r) =
P
sup
u∈[−1/2,1/2]d
p
r∈Z g(r)
|r| < ∞ in Theorem 3 is as follows.
|f (κ(r) + u)|p(κ(r) + u).
From (18) it follows that g(r) ≤ h(r) for all r ∈ Z and so a sufficient condition for
p
P
PT1 −1
|f (Wj )|π(Wj ))2 < ∞ is r∈Z h(r) |r| < ∞.
E( j=0
The proofs of Theorems 1-3 are given in the Section 4. The proof of Corollary 1
follows from the proof of Theorem 1 and Slutsky’s theorem and hence is omitted.
3. Examples
In this section we demonstrate the use of the results in Section 2 with some examples.
P∞
We first consider estimating λ = m=1 1/m2 . [2] use the SSRW chain mentioned in
Theorem 2 to estimate λ, that is, they use λ̂n defined in (14) to consistently estimate
λ. Note that, in this case f (j) = 1/j 2 if j ≥ 1, f (j) = 0 otherwise, and π(j) = 1 for
p
P
P
all j ∈ Z. Since j∈Z f (j)π(j) |j| = j≥1 j −(1+1/2) < ∞, we can use Theorem 2
to provide a confidence interval for λ based on λ̂n . In particular, an asymptotic 95%
√
confidence interval for λ is given by (λ̂n ± 1.96σn / Nn ), where σn2 is defined in (11).
The left panel in Figure 1 shows the point as well as 95% interval estimates for 6
values (log10 n = 3, 4, . . . , 8, where log10 denotes logarithm base 10). The point and
95% interval estimates for n = 108 are 1.636, and (1.580, 1.693) respectively. Note
that, the true value λ is π 2 /6 = 1.645. The time to run the SSRW chain for 108 steps
using R ([16]) in an old Intel Q9550 2.83GHz machine running Windows 7 is about 3
seconds.
The next example was originally considered in [5]. Let
f (x, y) = exp(−xy) 0 < x, y < ∞.
(21)
Let
f (x, y)
= y exp(−xy); 0 < x, y < ∞.
f (x0 , y)dx0
R+
fX|Y (x|y) := R
Thus for each y, the conditional density of X given Y = y is an exponential density.
Consider the Gibbs sampler {(Xn , Yn )}n≥0 that uses the two conditional densities
K. B. Athreya and V. Roy
0.75
0.50
^
λn
1.5
0.25
0.5
^
λn
2.5
10
3
4
5
6
7
8
3
no. of iterations (log10 scale)
4
5
6
7
8
no. of iterations (log10 scale)
Figure 1: Point and 95% interval estimates of
panel).
P∞
m=1
1/m2 (left panel) and fX (0.5) (right
Pn
fX|Y (·|y) and fY |X (·|x) alternately. [5] found that the usual estimator j=0 fX|Y (x̌|Yj )/n
R
R
for the marginal density fX (x̌) = R+ f (x̌, y)dy = R+ fX|Y (x̌|y)fY (y)dy = 1/x̌ breaks
down. [2] show that the Gibbs sampler {(Xn , Yn )}n≥0 is regenerative with improper
invariant measure whose density with respect to the Lebesgue measure is f (x, y) as
P
defined in (21). Thus their Theorem 3 implies that nj=0 fX|Y (x̌|Yj )/n converges to
zero with probability 1. [2] use λ̂n defined in (17) for consistently estimating fX (x̌).
In this example, using Remark 1, we have h(r) = 0 for all r ≤ −1, h(0) = 1, and
h(r) = exp(−x̌(r − 1/2)) for all r ≥ 1. Since
X
r∈Z
X
p
√
g(r) |r| ≤
exp(−x̌(r − 1/2)) r < ∞,
r≥1
from Theorem 3, we obtain a confidence interval for fX (x̌) based on λ̂n . The right
panel in Figure 1 shows the (point and 95% interval) estimates of fX (2) = 1/2 for the
same six n values mentioned in the previous example. The estimates for n = 108 are
0.497 and (0.490, 0.505) respectively.
4. Proofs of results
We begin with a short lemma that is used in the proof of Theorem 1.
a.s.
Lemma 1. Let {ξi }i≥1 be iid random variables with E|ξ1 | < ∞. Then ξn /n −→ 0.
11
Estimation of integrals using regenerative sequences
P∞
Proof. Since E|ξ1 | < ∞, for all > 0,
n=1 P (|ξ1 | > n) < ∞. By the BorelP∞
Cantelli lemma, n=1 P (|ξn | > n) < ∞ implies P (|ξn |/n > i. o.) = 0. This implies
a.s.
that lim sup |ξn |/n ≤ with probability 1, ∀ > 0. This in turn implies ξn /n −→ 0.
Proof of Theorem 1
Proof. From (1) it follows that λ = E(U1 ). Since Ui ’s are iid random variables with
Var (U1 ) = σ 2 , 1(a) and 1(b) follows from the classical central limit theorem and the
functional central limit theorem for iid random variables (see [4]).
From (1) we have π(S) = E(T2 − T1 ). Since (6) holds and 0 < α < 1, E(T2 − T1 ) =
E(τ1 ) = ∞ implying 2(a).
The proof of (7) is given in [8] (see also [2] and [12]).
Now we establish (9). Note that
Pn
PT1 −1
√
PNn
(λ̂n − λ) Nn
j=TNn f (Xj )
j=0 f (Xj )
i=1 (Ui − λ)
√
√
√
+
+
.
(22)
=
σ
Nn σ
Nn σ
Nn σ
PT1 −1
Now since P (T1 < ∞) = 1, P (| j=0
f (Xj )| < ∞) = 1. Also, Nn → ∞ with
√
PT1 −1
a.s.
probability 1 as n → ∞ and 0 < σ < ∞. This implies that j=0
f (Xj )/ Nn σ −→ 0.
Next,
|
where ηi ≡
Pn
j=TNn
f (Xj )|
√
Nn σ
≤
PTNn +1 −1
j=TNn
√
|f (Xj )|
Nn σ
ηN
≡ √ n ,
Nn σ
PTi+1 −1
|f (Xj )|, i = 1, 2, . . . . Since the condition (8) is in force, we have
√ a.s.
a.s.
a.s.
Eη12 < ∞. By Lemma 1, ηn2 /n −→ 0. This implies that ηn / n −→ 0. Since Nn −→ ∞,
√
a.s.
as n → ∞ we have ηNn / Nn −→ 0.
j=Ti
So to establish (9) it suffices to show that
PNn
i=1 (Ui − λ) d
√
−→ N (0, 1).
Nn σ
Let for 0 ≤ t < ∞,
P[nt]
(U[nt]+1 − λ)
(Ui − λ)
√
Bn (t) ≡ i=1√
+ (nt − [nt])
nσ
nσ
(23)
and
An (t) ≡
T[nt]
τ[nt]
+ (nt − [nt])
,
an
an
where {an } is such that na−α
n L(an ) → 1.
(24)
Then, it is known ([4]) by Donsker’s
invariance principle that {Bn (·) : 0 ≤ t < ∞} converges weakly in C[0, ∞) as n → ∞
12
K. B. Athreya and V. Roy
to standard Brownian motion B(·). Also it is known ([8] p. 448) that for any 0 < t1 <
t2 < · · · < tk < ∞, (An (t1 ), An (t2 ), . . . , An (tk )) convergence in distribution as n → ∞
to (A(t1 ), A(t2 ), . . . , A(tk )) where {A(t) : t ≥ 0} is a nonnegative stable process of
order α with A(0) = 0, a.s. and E(exp(−sA(1))) = exp(−sα Γ(1 − α)), 0 ≤ s < ∞. It
has been pointed out by ([12] p. 525) that [18] has shown that this finite dimensional
convergence of An (·) to A(·) implies the convergence in law in the Skorohod space
D[0, ∞). Next, it can be shown that (An (·), Bn (·)) converges in the sense of finite
dimensional distributions as n → ∞. Since both {An (·)}n≥1 and {Bn (·)}n≥1 converge
weakly in D[0, ∞) (as pointed out above) both are tight. This implies that the bivariate
sequence {An (·), Bn (·)}n≥1 is also tight as processes in D2 [0, ∞) ≡ D[0, ∞)× D[0, ∞).
Since the finite dimensional distributions of (An (·), Bn (·)) converge as n → ∞, this
yields the weak convergence of {An (·), Bn (·)}n≥1 as n → ∞ in D2 [0, ∞). For the
limit process (A(·), B(·)), one can conclude that the process C(·) = A(·) + B(·) is
a Lévy process on [0, ∞). Now since B(·) has continuous trajectory and A(·) has
strictly increasing nonnegative sample paths, it follows by the uniqueness of the LévyItô decomposition of C(·) that the processes A(·) and B(·) have to be independent.
This argument is due to [12].
As noted by [18] (see [4]) it is possible to produce a sequence of processes (Ãn (·), B̃n (·))
and a process (Ã(·), B̃(·)) all defined in the same probability space such that for
each n, (Ãn (·), B̃n (·)) has the same distribution as (An (·), Bn (·)) on D2 [0, ∞), and
(Ã(·), B̃(·)) has the same distribution as (A(·), B(·)), and (Ãn (·), B̃n (·)) converges to
(Ã(·), B̃(·)) with probability 1 in D2 [0, ∞). More specifically, we can generate on the
same probability space sequences {Ũn,i }i≥1,n≥1 and {T̃n,i }i≥1,n≥1 such that for each
n, the sequence {Ũn,i , T̃n,i }i≥1 has the same distribution as {Ui , Ti }i≥1 and for each
n, the processes Ãn (·) and B̃n (·) are defined in terms of the sequence {Ũn,i , T̃n,i }i≥1
and another sequence {Ũi , T̃i }i≥1 also having the same distribution as {Ui , Ti }i≥1 such
that (Ã(·), B̃(·)) is defined using {Ũi , T̃i }i≥1 .
Next, let An−1 (·) and A−1 (·) be the inverses of the nondecreasing nonnegative functions An (·), A(·) from [0, ∞) to [0, ∞). (For a nondecreasing nonnegative function H
on [0, ∞) we define the inverse H −1 (·) by H −1 (y) ≡ inf {x : H(x) ≥ y}, 0 ≤ y < ∞.) It
−1
can be shown (see also [11] Theorem A.1) that (Ãn , Ã−1
, B̃)
n , B̃n ) converges to (Ã, Ã
with probability 1 in D3 [0, ∞). This, in turn, yields by the continuous mapping theorem
Estimation of integrals using regenerative sequences
13
and the fact that P(Ã−1 (1) > 0) = 1, as n → ∞,
B̃n (Ã−1
(1)) a.s. B̃(Ã−1 (1))
q n
−→ q
.
−1 (1)
Ã−1
(1)
Ã
n
(25)
Now by independence of B̃ and à and the fact that P(Ã−1 (1) > 0) = 1, the limiting
random variable on right in (25) is distributed as N (0, 1).
Let bn ↑ ∞ be a sequence such that abn /n → 1 as n → ∞ where {an } is as defined
earlier satisfying na−α
n L(an ) → 1. Such a sequence {bn } exists as an ↑ ∞ as n → ∞.
By definition
Ã−1
n (1) = inf{x : Ãn (x) ≥ 1}.
Let y < Ã−1
n (1), then Ãn (y) < 1. This implies T̃n,[ny] /an < 1. Let {Ñn,m }m≥1 be the
sequence of regeneration times associated with {Ũn,i , T̃n,i }i≥1 . Then, T̃n,[ny] /an < 1
implies Ñn,an ≥ [ny] ≥ ny − 1. So, Ñn,an /n ≥ y − 1/n. This being true for all
y < Ãn−1 (1), we have
Ñn,an
≥ Ã−1
n (1) − 1/n.
n
(26)
Similarly letting y > Ã−1
n (1), we conclude that Ñn,an /n ≤ y + 1/n. This being true
for all y > Ã−1
n (1), we have
Ñn,an
≤ Ã−1
n (1) + 1/n.
n
(27)
From (26) and (27) we have
Ã−1
n (1) − 1/n ≤
Ñn,an
≤ Ã−1
n (1) + 1/n,
n
and more specifically
Ã−1
bn (1) − 1/bn ≤
Ñbn ,abn
≤ Ã−1
bn (1) + 1/bn .
bn
(28)
Since abn /n → 1 as n → ∞, for all > 0, n(1 − ) ≤ abn ≤ n(1 + ) for all n large.
This implies for all n large Ñbn ,n(1−) ≤ Ñbn ,abn ≤ Ñbn ,n(1+) . This yields by (28)
Ã−1
bn (1) − 1/bn
≤
=
Ñbn ,n(1+)
bn
Ñbn ,n(1+) bn(1+)
.
bn(1+)
bn
(29)
14
K. B. Athreya and V. Roy
As {bn } is such that abn /n → 1 and nan−α L(an ) → 1, which implies bn (abn )−α L(abn ) →
α
α
1, that is, bn ∼ aα
bn /L(abn ) ∼ n /L(n). So bn(1+) /bn ∼ (1 + ) L(n)/L(n(1 + )) →
a.s.
−1
(1) and (29) holds for all
(1 + )α as n → ∞ for all > −1. Since Ã−1
bn (1) −→ Ã
> 0, we may conclude that
Ã−1 (1) ≤ lim inf
Ñbn ,n
with probability 1.
bn
Similarly, Ã−1 (1) ≥ lim sup Ñbn ,n /bn with probability 1 and hence lim Ñbn ,n /bn =
Ã−1 (1) with probability 1.
By definition of (Ãn , B̃n ),
has the same distribution as
a.s.
Bbn (Nn /bn )
p
Nn /bn
B̃bn (Ñbn ,n /bn )
q
.
Ñbn ,n /bn
a.s.
Since Ñbn ,n /bn −→ Ã−1 (1), B̃bn (·) −→ B̃(·) in C[0, ∞), and B̃(·) has continuous
trajectory,
B̃bn (Ñbn ,n /bn ) a.s. B̃(Ã−1 (1))
q
.
−→ q
Ñbn ,n /bn
Ã−1 (1)
From (23) we see that
Hence (9) is proved.
PNn
Bbn (Nn /bn )
i=1 (Ui − λ)
√
= p
.
Nn σ
Nn /bn
Next, to prove (10) we see from (22) it suffices to show that
PNn
r α
n
B(Vα )
d
i=1 (Ui − λ)
−→ Q ≡
.
Nn σ
L(n)
Vα
Applying the above embedding used to prove (9), it is enough to show that
r
nα
B̃bn (Ñbn ,n /bn )
a.s.
−→ Q.
L(n)bn
Ñbn ,n /bn
This follows from the argument used in the proof of (9) and the fact that nα /{L(n)bn} →
1 as n → ∞.
a.s.
Since by the strong law of large numbers, σn2 −→ σ 2 as n → ∞, (12) and (13) follows
from Slutsky’s theorem, (9) and (10).
Proof of Theorem 2
15
Estimation of integrals using regenerative sequences
Proof. The SSRW Markov chain {Xn }n≥0 is null recurrent (see e.g. [14] Section
8.4.3) with the counting measure on Z being the unique (up to multiplicative constant) invariant measure for {Xn }n≥0 . Hence the SSRW Markov chain {Xn }n≥0 is
regenerative with regeneration times T0 = 0, Tr+1 = inf{n : n ≥ Tr + 1, Xn = 0},
r = 0, 1, 2, . . . and the proof of (14) follows from strong law of large numbers (see also
[2]).
Let N (j) ≡
excursion
PT1 −1
i=0
T1 −1
{Xi }i=0
I(Xi = j) be the number of visits to the state j during the first
for j ∈ Z. Note that X0 = 0, and N (0) = 1. Without loss of
generality, for the rest of the proof we assume that πj = 1 for all j ∈ Z. Since
TX
1 −1
j=0
|f (Xj )| =
X
j∈Z
|f (j)|N (j),
by Minkowski’s inequality, we have
E
1 −1
TX
j=0
2
X
2
2 X
p
|f (Xj )| = E
|f (j)|N (j) ≤
|f (j)| E(N (j))2 .
j∈Z
j∈Z
For the SSRW on Z, it has been shown by [3] that for r 6= 0, E(N (r)) = 1 and
Var(N (r)) = 4|r| − 2. So

 4|r| − 1
E(N (r))2 = Var(N (r)) + 1 =
 1
if r 6= 0
if r = 0
p
PT1 −1
|f (j)| |j| < ∞ implies E( j=0
|f (Xj )|)2 < ∞.
p
Since P (T1 > n) ∼
2/πn−1/2 as n → ∞ (see e.g. [7] p. 203), from (7) of
Thus
P
j∈Z
Theorem 1, we have
N
d
√n −→
n
r
π
|Z|,
2
where Z ∼ N(0, 1) (see e.g. [8] p. 173).
Then (15), (16) and Theorem 2’s (d) follows from (9), (10), and 3(c) of Theorem 1.
Proof of Theorem 3
p
P
Proof. The proof of (17) is given in [2]. We now show that r∈Z g(r) |r| < ∞
PT1 −1
implies that E( j=0
|f (Wj )|π(Wj ))2 < ∞. Without loss of generality we assume
that p(x) ≡ 1 for all x ∈ Rd . Since {Uij : i = 0, 1, . . . ; j = 1, 2, . . . , d}’s are iid Uniform
16
K. B. Athreya and V. Roy
(−1/2, 1/2) and are independent of {Xn }n≥0 , we have
E
TX
1 −1
j=0
2
X NX
2
(r)
i
|f (κ(r) + U )| ,
|f (Wj )| = E
r∈Z i=1
where N (r) is as defined in the proof of Theorem 2, the number of visits to the state
T1 −1
r during the first excursion {Xi }i=0
and U i ≡ (Ui1 , Ui2 , . . . , Uid ), with Uij ’s are iid
Uniform (−1/2, 1/2). By Minkowski’s inequality, we have
(r)
(r)
2 o1/2 X n h NX
i2 o1/2
n X NX
i
f (κ(r) + U )
f (κ(r) + U i )
E
≤
.
E
r∈Z i=1
r∈Z
(30)
i=1
For any fixed r ∈ Z, another application of Minkowski’s inequality yields
E
(r)
hN
X
i=1
(r)
i2 i2
o
nh NX
f (κ(r) + U i ) N (r)
f (κ(r) + U i ) = E E
≤ g(r)2 E(N (r)2 ),
i=1
where g(r) is defined in (18). Hence the rest of the proof follows from (30) and using
similar arguments as in the proof of Theorem 2.
Acknowledgement
The authors thank one reviewer and the editor for helpful comments and valuable
suggestions.
References
[1] Asmussen, S. and Glynn, P. W. (2007). Stochastic Simulation: Algorithms
and Analysis. Springer.
[2] Athreya, K. B. and Roy, V. (2012). Monte carlo methods for improper target
distributions. Technical report. Iowa State University.
[3] Baron, B. and Rukhin, A. L. (1999). Distribution of the number of visits of a
random walk. Commun. Stat. Stoch. Models 15, 593–597.
[4] Billingsley, P. (1968). Convergence of Probability Measures 1st. ed. John Wiley
& Sons, New York.
Estimation of integrals using regenerative sequences
17
[5] Casella, G. and George, E. (1992). Explaining the Gibbs sampler. The
American Statistician 46, 167–174.
[6] Crane, M. A. and Iglehart, D. (1975). Simulating stable stochastic systems
III: Regenerative processes and discrete event simulations. Operations Research
23, 33–45.
[7] Durrett, R. (2010). Probability: Theory and Examples 4th. ed. Cambridge
University Press, New York.
[8] Feller, W. (1971). An Introduction to Probability Theory and its Applications
vol. II. John Wiley & Sons, New York.
[9] Glynn, P. W. and Iglehart, D. (1987). A joint central limit theorem for the
sample mean and regenerative variance estimator. Annals of Operations Research
8, 41–55.
[10] Hobert, J. P., Jones, G. L., Presnell, B. and Rosenthal, J. S. (2002).
On the applicability of regenerative simulation in Markov chain Monte Carlo.
Biometrika 89, 731–743.
[11] Karlsen, H. A. and Tjøstheim, D. (2001). Nonparametric estimation in null
recurrent time series. The Annals of Statistics 29, 372–416.
[12] Kasahara, Y. (1984). Limit theorems for Lévy processes and Poisson point
processes and their applications to Brownian excursions. J. Math. Kyoto Univ.
24, 521–538.
[13] Metropolis, N., Rosenbluth, A., Rosenbluth, M. N., Teller, A. H. and
Teller, E. (1953). Equations of state calculations by fast computing machines.
Journal of Chemical Physics 21, 1087–1092.
[14] Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic
Stability. Springer Verlag, London.
[15] Mykland, P., Tierney, L. and Yu, B. (1995). Regeneration in Markov chain
samplers. Journal of the American Statistical Association 90, 233–41.
18
K. B. Athreya and V. Roy
[16] R Development Core Team (2011). R: A Language and Environment for
Statistical Computing. R Foundation for Statistical Computing Vienna, Austria.
ISBN 3-900051-07-0.
[17] Robert, C. and Casella, G. (2004). Monte Carlo Statistical Methods 2nd. ed.
Springer, New York.
[18] Skorohod, A. V. (1957).
Limit theorems for stochastic processes with
independent increments. Theory of Probability and Its Applications 2, 138–171.