Journal
of Applied Mathematics and Stochastic Analysis 4, Number 4, Winter 1991, 293-304
RELATIVE STABILITY AND WEAK CONVERGENCE IN
NON-DECREASING STOCHASTICALLY MONOTONE
MAIOV CHAINS1
P. TODOR,OVIC
University
of California
Department of Statistics and Applied Probability
Santa Barbara, CA
Let {n} be a non-decreasing stochastically monotone Markov
chain whose transition probability Q(.,.) has Q(z,{z})= (x)> 0 for
some function/3(. ) that is non-decreasing with/3(z)T1 as z---, +oo, and
each Q(z,. is non-atomic otherwise. A typical realization of {n} is a
Markov renewal process {(Xn, Tn)}, where j = X n for T n consecutive
values of j, T n geometric on {1,2,...} with parameter (Xn).
Conditions are given for X n to be relatively stable and for T n to be
weakly convergent.
Key words: Markov chain, stochastic monotonicity, Markov
renewal process, relative stability, weak convergence.
AMS (MOS) subject classifications:60G, 60K15.
1. INTRODUCTION
In this paper, R is the real line and % the #-field of Borel subsets of R. Let
a Markov chain with state space
The
7r
R,},
{n)C be
an initial distribution r and transition probability
Q.
,
and Q determine completely and uniquely a probability measure P on the countable
product space {R %oo}. When r(. ) = ez(. (the Dirac measure concentrated at z) we shall
write Px instead of P. The corresponding expectation operator is denoted then by E:.
Throughout this paper it is assumed that the Q is subject to the following regularity
conditions:
()
for each z E R the support of Q(z,. is in
(ii)
the chain
Ix, oo):
{n}0 is stochastically monotone (Daley, [3]);
x I <_ z2, Q(x2, Bu) < Q(zl, Bu) where S u =
oo, y];
(iii)
1Received: January,
Q(x,(y})
-(
in other words; for any
(1.1)
0
() > 0
=y
1991. Revised: June, 1991.
Printed in the U.S.A. (C) 1991 The Society of Applied Mathematics, Modeling and Simulation
293
P. TODOROVIC
294
Concerning the function/3(. we assume that
(x 1 < x2)
fl(Xl) < fl(z2) and
lira
oofl(z = 1
From (1.1.i) it follows that
o <- 1 <-""
(1.3)
Markov processes of this type are of considerable interest in reliability theory as models of the
amount of deterioration of a mechanical device subject to shocks and wear during its service
(Barlow and Proschan, [1]; Brown and Changanty, [2]).
Set
Tn rn
0}, vn = sup{k; k = r n
To ro
Xo = o, Xn = 7" n 1 + ’ WO = XO’ Wn = Xn Xn
sup{k; k
7"0
"rn- 1,
1
+ 1},
P
From this one readily obtains that
Px{To > i} {fl(z)}
= O, 1,...
2. AUXILIAPY ttESULTS
Here we list some basic properties of the bivariate sequence {(Xn, Tn) } needed in the
rest of this paper. Some simple calculations show that this sequence is a Markov renewal
process with the transition probability
Xn_ 1} = P(Xn_,du){1- fl(u)}{fl(u)} i-1 (a.s.)
P{X n G du, Tn =
where i = 1, 2,... and
y<x
0
P(x, Bu)
Q(x, Bu)
1
The
that
P(x, Bu)
fl()
is the transition probability of the Markov chain
{Xn}
.
P(x 2, By) < P(x, Bu) for all z x < x 2.
(2.2)
It is easy to verify
(2.3)
From (2.1) we deduce
P{X n e du, T n
which clearly implies
(see (1.5)) that
i}
{1 (u)}{fl(u)} i- 1p{x n du}
(2.4)
Relative Stability and Weak Convergence in Non.Decreasing Stochastically Monotone Markov Chains
295
P{T,, = X,} = {1 (Xn)}{fl(X,)} i- (a.s.)
= PXn{T0 = i- 1}.
In addition, since
Pz{X 1 E du, T 1 i} = P(z, du){1 (u)}{(u)} iit follows that
P{X n Edu,T n =ilXn_} =PXn_{X l du,T l=j} (a.s.).
Denote by Pn(z, By) the n-step transition probability of {Xn}, since
X0 < Xx <
(2.8)
pn + (z, Bu) < pn(z, By). On the other hand, the stochastic monotonicity and
the Chapman-Kolmogorov equation yield:
we have that
Pn(z, By) < (P(z, Bu))n.
(2.9)
From this, (2.8) and the Borel-Cantelli lemma it follows that
for all y
Xn---. + c (a.s.) if P(:,Bu) < 1
< c.
It is clear from (2.5) that T n is conditionally geometric with parameter fl(Xr, ). In
addition, since
x,} = {5(x,)} i- ’ (a.s.)
P{T n >_
it is apparent that
each n
{Tn}
is stochastically monotone and that T
d_: as
Finally, for
= 0, 1,... we have:
P{To = io,’",Tn = in Xo,"’,Xn
In other words, conditioned on
a realization of
H P{Tj = i
{Xn}
X#} (a.s.).
the sequence of sojourn times
(2.11)
{Tn}
becomes a family of independent r.v.’s such that the distribution of T n depends only on X n.
Consider
U
Pz{Xn du, T n = i} =
/ P{X e
n
du, T n = i
X,,_ = z}P n- x(z, dz)
= {1 fl(u)}{fl(u)} i- 1P:{X n du}
P. TODOROVIC
296
from which we deduce that
Pz{Tn = X,.,} = {1 5(X.)}{5(X.)} i- (a.s.)
where the right hand side is independent of x.
3. IMARKS ON TIE STRUCTURE OF
In this section
we investigate asymptotic structure of the sequence
that the following condition holds for all y
-
lira
where
F(-) is a proper d.f.
Remark 3.1_._:
{Wn)
assuming
>_ 0:
P{I ,> z + y} -F(U)
P’X{, > Z}
The condition
(3.1) is similar to one introduced by Gnedenko [4].
Denote by
Cn(y x) = Pz{W n <_ y)
n
= l,2,...
then clearly
O(y x) = P(x, B x + 9)"
(3.3)
(bn(Y X) Ez{(bl(Y X,- 1)}"
(3.4)
Some simple calculations yield:
Taking into account (2.2) and condition (3.1), we have:
This, (3.4) and the Lebesgue bounded convergence theorem imply that
/./.(u ) F(u).
In other words, (at least)
W:.:.Y, where Y
(3.6)
is a r.v. with the
d.f. F(y).
The following
proposition generalizes this simple observation.
Proposition
3.1__L
Assume that (3.1) holds and F(. is continuous, then under Pz, for
all k = 1,2,...
(Wn + 1"’" Wn + k)d-;(Y1,
where
{Yi}
is an i.i.d, sequence
of r.v.’s
"’’,
Yk)
with common
as
d.f F(. ).
Relative Stability and Weak Convetgence in Non-Decreasing Stochastically Monotone Markov Chains
The method of proof will be amply illustrated by the case n
proof:
e
> O,
297
= 2. Given
we obtain
Px{Wn + t g Yl, Wn + 2 < Y2}
oo
z
(3.8)
+ Yl
=/ f P(z, du)ePt(Y2 u)Pn(z, dz)
Xn +Yl
= Ez{/ P(Xn’du)(l(Y21U)}
Z
Since by assumption
any e
X
F(. )
is
continuousnthe convergence in (3.5) is uniform. Consequently, for
> 0 there exists x0 such that
[(y21u)- F(y2) < e
for all u > xo and any Y2 E R.
Frorn this and (3.8) we then have:
Px{Wn + 1 < Yl, Wn + 2 -< Y2} -< Pn( x, Bxo)
Xn+Y1
x(y2 u)P(X,,du)}
x
<_ Pn(x,B%) + (e + F(y2))Ex{l(y 1
X.)Ztx" >
Consequently,
ldrnooPx{W, + < Yt, Wr + 2 < Y2} -< (e + F(y2))F(yt).
In the same fashion, one can show that
li.__m Px{Wn + 1
-< Yl, Wn + 2 -< Y2} -> F(Yl)(F(Y2)
)"
Since e > 0 is arbitrary, the assertion follows.
The last proposition indicates that, roughly speaking, the remote
Remark 3.2__A:
members of {Wn} are i.i.d.r.v.’s.
Next,
we show that
the sequence
{Wn}
is endowed with a mixing property, which
means, loosely speaking, that its elements far apart are nearly independent.
fin = cr{Wo,’", Wn} and by fin = r{Wn, Wn + 1""} then we have:
Proos.iti 3.2__.A."
For each n = 1, 2,... and k = 1,2,...
Denote by
P. TODOROVIC
298
k
n
j=l
(3.9)
k
n
N (Xj < yj})H F(zi)
Pz(
(z < Yl <
<
j=l
Proo
By invoking the Markov property of (Xn}
and the proposition 3.1, we
obtain
k
n
Pz(N
{Xj <_ Yj}N {Wn+m+i <j=l
=1
k
n-1
"-*
(, .1
F(zi )Pn(x,es)
Px(N {Xj <_ yjI Xn = s)H
i=
=
as m--.oo which proves the assertion.
Corollart 3.1___2".
{Wn}
and
{yj}o
It follows from the last proposition that
are independent
qn
and
r
families. Set
are independent tr-algebras for all
Therefore nfff is a trivial a-algebra (its elements are either sure or null
= 0,1,
D ff and that their intersection is a trivial revents). By letting n---<x we have that
algebra. This clearly implies that the tail a-algebra Y is a trivial one.
n
oo
4. RELATIVE STABILITY OF
{Xn}
{Xn} is said to be relatively stable if there exist constants {an} such that
probability (Gnedenko and Kolmogorov, [5]). If the convergence is (a.s.) the
The sequence
Xn/an--l
in
(a.s.) relatively stable (Resnik, [6]). The following proposition gives
condition for (a.s.) relative stability of {Xn}.
sequence is called
sufficient
Assume that
:protosilion
then
Xn/n---,a I (a.s.)
Proof:
where a 1
-
supEx{W12} < oo
E{Y1}.
From (3.4), (4.1) and Fubini’s theorem we deduce that
a
Relative Stability and Weak Convergence in Non-Decreasing Stochastically Monotone Markov Chains
f
Ez{Wk 2}
y[1 (k(Y x)]dy
299
(4.2)
o
f
= Ex{ y[1 l(ylX k_ 1)ldy}
o
= Ex(EXk- 1 {W12}) < supEx{W 2} <
Consequently,
sup
x,k
ExtWk2)
Next, since
{
E Wk converges}
1
and ff is a trivial r-algebra, to prove the proposition it suffices to show that
P{llxn- ch
> e}--,O as n.
Consider
Var{lXn } =
n
n--1
k=l
i=1
E VarlWk} + E
1
"
It is clear from (4.1) and (4.2) that under
1__
22
_,
(4.3)
Co,,(w, w)).
j=i+l
Px
E Var{Wk}’-*O
as n---,cx.
k=l
On the other hand, due to propositions 3.1 and 3.2
lira
k--,
for each n
= 0, 1,
Cov(Wk, Wk + n) = O
k--,oo
Cov(Wn, W n + k) = O
(4.4)
Thus, given e > 0 there exists no = no(e ) such that
Co.(W, W)l
if rain{i, j}
lira
<,
Co,(W., w. + )1 <
(4.5)
> n o and k > no. Now, take n > 2no, then
n
1
no
n
2no
Cov(Wi, Wj)
i=1 j=i+l
i=1 j=i+l
no
n--1
n
Co,(W, w) +
j=2no+l
i=no+l
j
Co(w,,w)l
i’1" 1
P. TODOROVIC
300
2n 0
no
Co(W, w)l + ( no) + (- 0- )(- 0)/"
j=i+l
Consequently, for any c > 0
lira
Var{Xn} < 12.
Ex{Wn)---+( ] as n---,oo it follows that
Finally, since
E Ex{Wk)/n"+(l
as noo.
k-1
Therefore, for n sufficiently large
Pz(
(4.1)
This and
.
Xn- %
E [Wi- Ex{Wi}]
> e} < Px{l
> }"
prove the assertion.
eoroUa
From proposition 4.1 we readily deduce that for each e > 0
Px{Xn <_ (] + )n}--,l
and
Px{X n G (c 1 -)n}--+0
(4.6)
as n---+oo. Consequently
i
if
0
if
Y>-i
(4.7)
Denote by
T(y) = in f {k; Xk > y}
then
for any x < y
Px{T(y) <_ n}
1
Pn(x,B).
Under P
Proposition
T(y)c 1
Assume
O1
.
1
in probability as y--, + c.
0, then we have to show
Px{ T(y)
for any e
Px{Xn > y}
<_ c}---1
]
as y---,
+ oo
> 0. Choose e e (0, %-]) then from (4.9) we deduce
P x{
T(y)
h
x <_ e} = P x{ X
[(,:,
]
-),1
< y} Px{X
[(
1
Relative Stability and Weak Convergence in Non-Decreasing Stochastically Monotone Markov Chains
e)u](z Bu
= p[(al
p[(h
+ e)u](z
301
Bu
where, as usual, [x] stands for the integer part of z. Since
Y
it follows from
(4.7) that
lira P [(al
1
e)ul(z Bu) = 1.
Similarly, when y--.oo
+
This and
(a
1
+ e)y- 1
1
+ ech
(4.7) imply
lim
1
lr -+e)u](z,
p,,a
Bu)__ = 0
which proves the proposition.
5. WEAK CONVERGENCE OF
In this section
{Tr}
we show that a sequence of scale factors
{dn} exists such that under Pz
(5.1)
where the r.v. Z has an exponential distribution independent of x. But first, we need the
following auxiliary result.
Provosition 5.1.._’.
Assume that condition (4.1) holds, then
1-fl(Xn) P--;1
1
where c 1
(5.2)
as n---o
E{Y1}.
.Proof:
Set
(5.3)
then for any e E
(0, 1)
].1
/- l((ncl)(1
nc 1
<-}-Pz{w <-
Zn
e)
+_.}}
P. TODOROVIC
302
p={,- l(fl(nal)(nall + )
e)
}.
Since
,8- l(fl(nal)(l
e) + ,:) > na I
fl-l(fl(nal)(l + e)-:) < na I
for all n
= 1,2,..., the assertion now follows from proposition 4.1.
Suppose that (4.1) holds, then
proposition 5.e_..A
/n/rnooPx{[1- (n:)]T n > u} = e -u.
(5.4)
Denote by
U, [1- fl(Xn)]T n.
Then taking into account
(2.10),
we have:
u
xn})
Px{Un > u} Ez(Pz{Tn > I fl(Xn)
= Ez({fl(Xn)}
[1
"
(Xn)’] ).
Set
{(x.(,.,))}
R,(,. 1
" )1
[i (x,(.,).
Since the function
I,,
is non-decreasing on
ln(y)}
R, it follows that
n,(,. < R, + :(,.
at least
(a.s.). From this we readily obtain
mo/.(u, ) = e -"
at least
that
(a.s.)
P.
Invoking now the monotone convergence theorem, we deduce from
u,z.
Finally, write
[1 t3(ncl)]T n = 1
-/3(Xn)
(5.6)
(.s)
Relative Stability and Weak Convelgence in Non-Decreasing Stochastically Monotone Markov Chains
then the proof of (5.4) follows
(5.8), proposition 4.2 and a Slutsky’s theorem.
One can easily show that the sequence of r.v.’s
Remark 5.1.__2".
303
{Un)
has the
following properties:
Ex{Un}
1
Var{Un} = Ex{fl(Xn) }
Ex{Un + 1 UI,"-, Un}
-
1.
The result of the last proposition can be easily extended as follows:
Remark 5.___’.
Set
v. =
then after some straight forward calculations
for each k
Px,
one can show that
= 1, 2,...
(v, +
under
(see Todorovic and Gard, [7])
where
{Zk) 1
v, +
is an i.i.d, sequence of r.v.’s with common non-negative exponential
distribution of z. The sequence also possesses a mixing property.
REFERENCES
R.E. Barlow and F. Proschan, "Statistical Theory of Reliability and Life Testing", Holt,
Phinehart and Winston, New York (1975).
M. Brown and N.R. Changanty, "On the first passage time distribution for a class of
Markov chains", Ann. Prob. 11, (1983), pp. 1000-1008.
[3]
D.J. Daley, "Stochastically monotone Markov chains", g. Wahrsch. verw. Gebiete 10,
(1968), pp. 305-317.
[4]
B.V. Gnedenko, "Sur la distribution limite du terme maximum d’une srie al(atoire’,
Ann. Math. 44, (1943), pp. 423-453.
[5]
B.V. Gnedenko and A.N. Kolmogorov, "Limit Distributions for Sums of Independent
Random Variables", Addison-Wesley, Reading, Massachusetts
[6]
S.I. Resnick, "Stability of maxima of random variables defined on
Adv. Appl. Prob. 4,
[7]
(1954).
(1972),
a Markov
chain",
a Markov
chain",
pp. 285-295.
P. Todorovic and J. Gani, "A Markov renewal process imbedded in
Sloch. Ann. Appl. 7(3), (1989), pp. 339-353.