Internat. J. Math. & Math. Sci.
VOL. 12 NO. 2 (1989) 209-233
209
PRODUCTS OF STOCHASTIC MATRICES AND APPLICATIONS
HARRY COHN
Department of Statistics
The University of Melbourne
Parkville, Victoria 3052
Au s t ral ia
(Received November 20, 1987)
This paper deals with aspects of the limit behaviour of products
ABSTRACT.
of nonidentical finite or countable stochastic matrices (P). Applications
n
are given to nonhomogeneous Markov models as positive chains, some classes of
finite chains considered by Doeblin and weakly ergodic chains.
KEY WORDS AND PHRASES.
Stochastic matrix, nonhomogeneity, Markov chains,
weak ergodicity, tail o-field, atomic set.
15A51, 60JI0.
1980 AMS SUBJECTS CLASSIFICATION CODES.
1.
INTRODUCTION.
Let
p(n)
lJ
be a sequence of finite or countable stochastic matrices,
P0,PI,...,
the (i j) entry of P
In the ’homogeneous’
p(m,n)
p
P
n
m,n
case, when P
m
P
n
0
theory provides a detailed analysis of
n-,=o, whereas otherwise
r
converges as n-oo for some d >
pn
converges as
and
(Pn)
are
i,j
{pm,n)i,j’’/p,j(m,n)}
+/-
for ergodic chains
(m’n) > 0 and (m,n) > 0
lira infn_oP
imply that
converges as
whereas otherwise
Both
pn:
It turns out that in the ’nonhomogeneous’ case, when
l,...,d-1.
nonidentical
pnd+r
the (i,j) entry of P
m,n
the classical Markov chains
(m,n) /P
{Pi,j
(m,n) ]P
Imn-ri, J
(re:n)
-a
n/oo
(n)
>. m}
assume a finite numberoflimit points.
and the limit points of
will be identified in terms of
for some sequence of sets
in a case that may be thought of as aperiodic
:n
(m)
i
{E(k)}.
n
(k)/a
m)
(m,n)(m,n) :n
{ri, j /P,j
(k) where
(m)
(k)
u
lim
m+l
n-O
i
E
p(m,n)
(k) u,j
n
Our results may be understood without
reference to Markov chains, but the proofs will consider a Markov chain
{X :n >. m} with finite or countable state spaces assuming a strictly positive
n
initial probability vector
and the one-step transition probability
(m)
matrices
(Pn)
n
>.
m as the starting point.
It will turn out that the structure
of the tail o-field of {X :n >. m} is crucial for the asymptotic behaviour of
n
(m)
and that
u) where Tk is an atomic set of the tail
(k)
u
m,n
o-field of {X :n >. m}. We first consider the countable case where a number of
n
(m,n) >
0.
results are obtained under the assumption that lim
j
{P
p(m)(TklXm
infn_oPi,
p(m,n) will be
where lim
n
l,J
lJ
Then we look at the case of finite S where more powerful results
A particular case is that of convergent
identified.
{p(m,n)}
are obtained without any assumption on (P).
Further we specialize our
H. COHN
210
results to some classes of finite and countable nonhomogeneous chains and
explore some connections with the notion of weak ergodicity.
We do not include in this paper specific applications of products of
stochastic matrices which seem to be numerous, ranging from demography as shown
by Seneta [21], to recent developments in the theory of Markovian random
fields assuming phase transitions (see Kemeny et al [12] and Winkler [22]).
Our paper is a streamlined survey of the literature of nonhomogeneous
.
Markov models from the viewpoint of tail o-fields.
TAIL -FIELDS
2.
We shall say that
Let (,,P) be a probability space and A a set in
if P(A) > 0 and A does not contain any subset A’ with
A’g
and 0 < P(A’) < P(A). A nonatomic set A in
is said to be a
A
is a P-atom/c set of
.
P-oompetey nonatomio
P-atomic subsets of
.
C.
are P-atomic sets of
sets of
A0
0
is finite.
{A.}I
If A
may be absent
whereas if
A
n
(I
is said to be
Finally,
x...
S x S
Take now
n(m)
0
is present, we
.
(Pn)n
m
with
such that
{Xn:n
p(m)(Xm=i) (m)i
>
> m}
and
trivial if A
(m)
m
(i
p(m,n)
p(m)(Xn__JlXm__i
IXn=i) Pi,j(n)
i.o.
00n
for
makes sense in view of
n=iUm=nAm
for
> n}.
and
p(O) (Xm=i)
{An
(,
A
ult.
for
p(m)
m,P (m))
for i,j g S and n
arguments on all
(o)
{An
we shall say
{Xk:k
is a nonhomogeneous Markov chain on
p(m) (Xn+l =j
If
;i g S) and a sequence of S x S
The usual model {X :n > 0} may not always lead to
n
take
to be strictly positive.
We shall write
{A.}
uniquely determine a probability measure
This model will allow us to use probabilistic
the equality
oio.
Xn()
for the -field generated by
and write
stochastic matrices
AI,A 2
is called
is absent
where S is finite or countable,
strictly positive distribution
on
may be
is P-completely nonatomic and
This representation is unique modulo null probability
nonaoo
is
not contain any
0
is absent and there is only a finite number of atomic sets
that
0--
where A
Of course, some of
shall say that
P(A) > 0 and A does
if
It is easy to see that, in general,
Un=OAn
represented as
C
set of
>
m.
p(m,n)
since
l,J
P(m)(Xm=i)
> 0
> 0 even if we
Here
Un=lm__nAm"
’i.o.’ stands for ’infinitely often’ and ’ult.’ stands for ’ultimately’.
A a.s. will mean that llmn_ol
Further, limn_oA
a.s. where
stands
n
A
A
n
We shall say that A
for the indicator function of a set.
a.s. The -field generated by
Xm,...,Xnwillbe
denoted
B a.s. if
_...(n)
1B
A
by
A key tool for out study will be provided by the tail o-field of {X :n
defined as
(m)
n
0
n--m--n
PROPOSITION 2.1. Suppose that A is a set in
sequence {L
p(m)
PROOF.
that
of
L
n
limn_oP
{Xn:n
{i:P
of subsets of S such that
n
Since
(m)
> m}
A belongs
to
m’
(AI .(n))= IAa.s.
we
> m}
getmp (m) (A
(m)(AIXn =i)
completing the proof.
m
Then there exists a
A
limn_o{XnELn
a.s. with respect to
the martingale convergence theorem implies
with
(n))
m).
respect to
p(m)(AIXn)
> %} with 0 < % <
yields
p(m).
By the Markov property
Thus taking
limn_o{Xn
g
Ln
A
p(m)
a.s.
PRODUCTS OF STOCHASTIC MATRICES AND APPLICATIONS
211
The argument used above goes back, essentially, to Blackwell [1].
POSITIVE STATES.
3.
{X :n >.m} with a strictly positive initial
Consider the Markov chain
(m)
probability vector
is
positive
if lim
j(n)-p(x_=j)
and write
infn_o(n)
for n
>.m.
We shall say that j
> 0.
j
PROPOSITION 3.1. A state j is positive if and only if for any subsequence
(nk)
with
there exists a state
limn_>oonk
(m,nk)
such that lim
PROOF.
!n)
3
(m)p(m,n)
ieS i
(nk)
to notice that
(m)
i
and
i,j
0 if and only if
limk_oj
> 0 for i
g
S
(m,nk)
>
If
d
d are P
k=l
PROOF.
depends only
is positive then
j
{X =j i.o.}
n
Tk,
j
m, i E S}.
THEOREM 3.1.
where
it suffices
0 for all i g S.
limk_oPi, j
Proposition 3.1 shows that the definition of positivity for
(m,n)
on {P
;n
(nk))
> O.
suPk_oPi, j
Since
(possibly depending on
i
(m) -atomic
sets of
{X =j i.o.}
n
Notice first that
p(m)
a.s.
f(m)
with d <
U T
k
k=l
e
(m)
.
Assume by way of
contradiction that {X =j i.o.} does not equal a finite union of atomic sets
n
and therefore we may find some infinite sequence of disjoint sets
e
TI,T 2
(m)
with
P(m)(T i)
{Xn=j
Take 6
with
lim
p(m)(Tk)
j
i.o.}
(n)
Let {E
< 6.
n-o
which is absurd.
P(m)a.s.
k
n
a.s. as ensured by Proposition 3.1.
(n)
k
m} be some subsets of S such that
:n
’3
{Xn= j
Thus
(3.1)
>
lim P
(k)
E
n
(m) E (k)
(m)
P
n
n-o
}"
In view
This entails
(Tk)
< 6
(m)
-atomic
i.oo} consists of a finite number of P
(m).
sets of
THEOREM 3.2.
Suppose that
j
is positive.
{Fk}where F k {n
that ,d
Um=IFk {re+l,
sequencesof integers
E
T
must belong to an infinity of sets
lira inf
sets
such that
i
If (3.1) were true, there would exist a set T
(k)
p(m)
Tk
U
k=l
infn_ow.
limn_{Xn eE(k)
n
of (3.1)
> 0 for all
(k)
n
such
k)
Then there exist
:t=l,2
and
d
d
disjoint
sequences of
and for i,e S
n’-olimeF p(m,n’)i,j/pm:n’),] i(m)(k)/am)(k)
n
(3.2)
k
_(m,n’)
> 0 where
provided that
(m)(k)
u
lim
n-j e
E[(k
n
p(m,n)
u,3
for u
S
H. COHN
212
Let
PROOF.
P
(m)
.tE.k.#n
"
limn_o{Xn eE’k’#n
be some sequences of sets with
Tk
a.s. for k=l,2,...,d whose existence is ensured by Proposition 2.1.
d
(k) for
We first show that j
E
n sufficiently large.
eUk=
n
Indeed, if we assume the contrary, i.e. that j
with lim
n
t
contradicts
oo, the positivity of
t
{Xn gk=Id E(k)n
leads to
j
{Xn
i o
(k)
d
Uk=iEnt
p(m)(X
U
d
(k) ult.}
E
k=l n
{n
for a sequence
n
U
=j i.o.) > 0 and
t
p (m) a.s.
=iTk
Thus, we may assume, if necessary by modifying a finite number of sets {E
d
(k)
that j g U
E
for n
k= n
n
p(m,n)
,u
(k)
n
},
m+l
Define now the random variable
c0g{X =u} if
k}
(m,n)
(m,n)
ri,Xn /P ,Xn
> 0 and 0 otherwise.
to equal
(m,n)
pm,n,/p ,u
for
z,u
Since
p(m) (Xm=ilXn=U) (m)
p(n) (Xm=Z Xn=U) (m)
p(m,n)/p(m,n)
,u
i,u
(3.3)
i
the Markov property of the reversed chain yields
lim
n
p(m,n)
i,X
n
P(m)(Xm=im(n)) __(m)
p(m) (Xm=m(n)) (m)
z
_(m,n)
/,Xn
(3.4)
Using the martingale convergence theorem and Theorem 2.1 in (3.4) leads to
p(m) (Xm=ilTk)
(m,n)/p(m,n)
lim
Pi,Xn "-,Xn
n-o
P
e(m) (TklXm=U)
Let
(k)
-nt
lim
limn_
n
be the set of values of
:t=l,2
n
n
Then there exists
> 0.
Notice
(m)
(k).
u
! (k)e(m:n)
j E
u,3
n
with j e E
(k) :t=l,...} and
n
t
runs through
Indeed, suppose the contrary.
(k)
n
Xn
We show
is replaced by
(k)
{nt}-l t
with
such that
n
t+oo t
(m’nt)
Pi,j /P,j
-lim
t
(m,n)
d}.
for some ke {I
P(m)({Xnt=
P(m)(Xm=1Tk)’
g
n
(3.5)
m
limn_oe(m) (X E(k)iXm=U)
now that (3.5) holds if
j
(m)
(Xm= Tk) wi
2, provided that
for almost all gT k, k=l
that
(m)
p(m) (TklXm=i)
p(m) (Tel X =)
(m)
j
i.o.}) >.
lira
t
infn_o_(,n)-j
(Tkl Xm=i
(3.6)
p(m) (Tkl x =1
m
=j i.o.}
But {X
n
p (m)
#
_c_
T
k
a.s.
> 0 makes (3.6) contradict (3.5) and
completes the proof.
REMARK3.1.
Themrem 3.2 as stated as a result about products of
stochastic matrices without reference to Markov chains theory.
in the proof that
(m) (k)
u
We have seen
may be expressed in terms of the tail -field
PRODUCTS OF STOCHASTIC MATRICES AND APPLICATIONS
(m)
as
p(m)(T kIX
=u).
We shall further consider some characteristic
{E (k)} which
n
properties of the sets
213
actual identification of {a
for some special cases, may lead to the
(m)(k) }.
u
THEOREM 3.3.
Suppose that
k} are P (m) -atomic sets
limn-{Xn gE(k)}n Tk p(m) a.s
a.s. where {T
that
n
E(k)
n
such that
k=l,...,d.
E
Let F
P
(m)
and let {E
in
k
(An,n sFk
p(m)
be some sets such
be the set of values
k
Pi,j
(3.7)
i.o.)
for n >, m.
It is clear that
(3.8)
0
the o-field generated by Xm,
is
(t)}
(k)
n
k
Then
An {Xn =j’xn+l (k)},.n+l
Define
Recalln)
/(m),
of
U=IT
i.o.}
(n) <
(k)
neFk i n+l
PROOF
{Xn =j
is positive and
j
X
with m < n.
n
According
to the Borel-Cantelli-Levy lemma
p(m)(An,ngF k
i.o.) =0 if and only if
(m)
nF k
p(m) (Anlmn))
)
n
m(n)
(k)
p (m)
(Xn+ n+l Xn=3 {Xn=j}
Thus, it will suffice to show that P (m)
(3.7).
0
{X :n m} yields
n
Notice now that the Markov property of
p (m) (A
(3.9)
(ner kp(m) (An l(n))
--m
(3 I0)
oo)
0 implies
Assume, by way of contradiction, that
I
ngF
_(n)
[
k
(k)
[
Pj,i
neF
1En+
P
(m)
(k)
(Xn+ mn+1 Xn=3")
(3.11)
k
and write
p (m) (X
[
n’eF k N {i
Y
n ’+I
n}
n’F k I {1
Y
n
necessary that E
of Y
(Yn)
therefore
tends to
proof.
IXn’
lira inf
E
n-o
> O) > O.
p(m)
p(m)(ngFkP(m)(Anl vine(n))
=j)
to converge in probability to 0 as n+o, it is
(m)
(m)
(Yn)
denotes the expectation
(Y)
n
>
lira inf
(n).
n-o 3
Since the denominator of
as n/, it follows that the numerator of Y
on a set of positive
yields
n
n}
p(m) (Xn,+l (k)n,+l
0 as n+, where E
p(m). However,
p(m)(lim SUPn_oYn
under
n
n/,
,< I, for {Y
(m)
n’ =j}
(3.12)
n
Since 0
% "(k)
mn’+llXn’=J)l{x
probability.
n
> 0 and
Yn
in (3.12)
will tend to
as
Taking account of (3.9) and (3.10)
o) > O, a contradiction that completes the
214
H. COHN
REMARK3.2. For homogeneous Markov chains with period
become
n
i.e. the cyclically moving subclasses of the recurrent
{Cu+n(mo d d)},
class to which
p(m) ({Xn g Cu+n
In this case
belongs.
j
(k)
the sets {E
d
}IXn-I Cu+n-I (mod d) })
d, and
limn_o{XngCu+n(mod d)} T u, u=l
(n)
of
Pj,i 0 for iCu+n+l(mod d)"
I,
g
(rood d)
(3.7) holds trivially in view
The results and proofs of this Section rely on Cohn [5], [6] and [8].
4.
A CLASS OF COUNTABLE CHAINS.
We shall next consider a class of stochastic matrices
(Pn)
satisfying
the following condition
(A)
T U C where j T if
S admits the decomposition S
for all
and
m
and j C if there exists
i
m
such that
(m,n)
+/-lmn_ori, j
0
is positive
j
for {X :n >. m}.
n
A partition T,
any j
g
C
k
and
CI,C 2,...
of S will be said to be a basis for
{Xn= j
large enough,
m
THEOREM 4.1.
a.s. is a P
k
(m)
Suppose that (P) satisfies condition (A).
-atomic set of
0 and j g C with
for i,gS, m
if for
Then the
n
/P
imn-ri(m,n)n)
J
existence of
i.o.} =T
(Pn)
(m,n) > 0
P,j
is a necessary and sufficient condition for the existence of a basis
T,
CI,C 2
We know from Proposition 3.1
PROOF.
P(m)-atomic
finite union of
Suppose otherwise
prove that d=l.
m),
sets of
A= {Xn =j i.o.}
that
i.e.
i.e. d>.2.
A
Uk=ITk.
is a
We shall first
{X =j i.o.}TIUT
2
n
{nk} and
such
Since
{n}
Theorem 3.1 implies that there must exist two sequences
that
lira
k-o
(m,nk)
Pi,j
(m,nk)
i,j
/e
/P
.t.mn_ p(m,n)
i,j
However,
P(TI IXm=i)/P(TI IXm=)
(4.1)
lira P
k_o
(m,nk)
/P,j
(m,nk)
E,j
(m,n)
P(T
2
IXm=i)/P(T 2 IXm=)
was supposed to exist.
pCm)(TIlXm=i)Ip(m)(TIIXm=)
Further, for any set A in
(m)
with
Thus (4.1) entails
P(m) CT21Xm=i)Ip(m)(T21Xm=)
p(m)(A)
(4.2)
> 0, the martingale convergence
theorem in conjunction with the time reversibility of the Markov property
p(m) (AIXn)
large enough.
may be made as close as desired to
It is easy to see that if
j
will stay positive for any {X :n>.m’} with
n
p(m) (AIXn=J)
choose
i
p(m’) (AIXn=J)
and
m
if
is positive for
m’ >m,
n
{X :n m}
n
it
and
p(m) (Xn=J)p(m’) (Xn=J)
such that for g<1/2
1A for
> 0.
p(m)(T llXm=i) > I-
Thus, one may
and
J (m)
PRODUCTS OF STOCHASTIC MATRICES AND APPLICATIONS
p(m)(T 21Xm=)
> I-g; since T
p(m)(T21Xm=i
< e
and T
are disjoint we also get
2
p(m)(T llXm=)
and
<
e.
It is easy to see that these four
inequalities are in contradiction with (4.2).
A
that proves that
{X =j i.o.}
n
is a
Conversely, if we assume that A
We have reached a contradiction
P(m)-atomic
set of
{X =j i.o.}
n
is a
(re’n)/p(m,n)
limn_oP i,j
,j
then Theorem 3 2 yields that
215
P
(m)
m).
P(m)-atomic
(AlXm=i)/P
set
(m)
of (m)
(AlXm=)
completing the proof
THEOREM 4.2.
(i)
g
for any
i,Z
Ck,
g
(ii) if for any C
JgCk,
with k’
# k
and
(k)
1
p(m,n)u,
(m)
(k)/
with lira
n
"
Then
C1,C 2
large enough
(k)
SUPn_olj gCkP (m’n)u .<CZu(m) (k).
(m)
such that o
gC
k
(k) < 1-% for
implies
(n) <
oo.
i.C k U T Pj ,i
Part (i) follows from Theorem 4.1 and Theorem 3.2 if we notice
PROOF.
SUPn_o[JeCkP(m’n)
u,3
lin_>oo.m
p(m)(xn gCk
true in general, that
P
(m)
a"
large enough, then i
m
[
(m)
jgE
(re, n)
/Pz.j
there exists % with 0 < % <
k
n=l
that lira
n
Z,j
(re,n)
a(m)u (k) limn+oo
assumes a basis T,
(m,n) > 0 and
S with
lira P
i,j
n_+c
where
(Pn)
Suppose that
JeE (k)
p(m,n)
k
although it is not
p(m)(T k), {Xn EF
i.o.)
a.s. obtains for any finite set F cC
the states of C
Indeed
u,j
limn_o JgUkp-(m:n)
u,j
T
k
k
This being true for any
k.
being positive necessarily imply that FeE
enough and therefore
i.o.}
(m)(k)
u
(k)
n
for
and
large
n
which proves (i).
For part (ii) we may invoke Theorem 3.3 provided that we show that
C
(k)
The latter follows from taking E
kUTE n
noticing that, as shown before,
(k)
n
p(m)(Tkl Xn=i)
{j:p(m)(TklXn=j)
limn_oP (m)(xn gEn(k)}IXn=i)
and arguing as in the proof of Proposition 1.1.
all u g S we get that
limn_oP
(m)
I gT P(m’n)
u,j
It is easy to see that in case that lira
n-o
RE,lARK4
(X n gT)
(m)
u
0.
(k)
limn-ojgCkp(m,n)
u,j
0 for
This happens when
In particular, the finite chains always satisfy this
condition.
The results in this Section were derived in Cohn [8].
5.
CONVERGENT CHAINS.
A chain {X
m, i and
n
will be said to be ope$ if
limn+oop(m’n).
exists for all
l,j
(see Maksimov [15], losifescu [ii], and Mukherjea [17])
course, this definition depends only on
(Pn)
Of
and does not need to involve a
H. COHN
216
{X }.
chain
As before our results may be read off without reference to a
n
Mar kov chain structure
The class of matrices
which
(P)for
n
converge{P.(m:n)}
is a subclass of that considered in Section4 and, naturally, will lead to
For convergence chains we shall identify the limits of
stronger properties
{p(m,n)}
i,J
rather than those of their ratios as was done in the previous sections
Suppose that {X
THEOREM 5. I.
(i)
{T,CI,C 2
there exists a basis
(ii) for any m,
igS
and j gC
(re,n)
lira P
i ]
n-o
(m)
!m) (k)
where
with
(n)
j
ji
>
n
i,j
{p!m,n)}.
(m) >
0, we notice that
(n)
and
j
(n)
.3 lim j
n
m.
j/k
ol
PROOF.
limn- jeE (k)
k
do not depend on the choice of
l,J
that
(k) lak
jgE(k)ri,j
n
(m)p(m’n)
ie S i
REMARK5.1. Since
k
(m,n)
limn_o
i
Then
is a convergent chain
(m)
provided
(m)
is also indepedent of
}.
{T,CI,C 2
Theorem 4.1 ensures the existence of a basis
Further, the martingale convergence theorem yields
p (m)
p (m)
(Xm=il Xn
(Xm=il m(n_p
(m)
(Xm=il (m))
as n
But
p (m)
i, j i(m)/(n)
j
_(m,n)
(Xm=il Xn=J)
and taking into account that
(m)
{Xn=j
i.o.}
T
a.s. is a
k
P(m)-atomic
set of
we get
(m) (re,n). (n)
lira
n
for almost all
g
T
k.
P
Pi,Xn /Xn
i
(m)
(5.1)
(Xm=ilTk)
Further we can argue as in the proof of Theorem 4.2(i)
to conclude (ii) on using
(5.1).
COROLLARY 5. i. Suppose that {Xn is a countable convergent chain assuming
{T,CI,C2,...}
a basis
and denote
(m)
p(m,n)
lira
qi,j
n-o i,j
(m)
and
qi,j
limm_oq i,j"
Then
jlk
for ieC
k, j
EC k
qi,j
0
PROOF.
im
.
According to Theorem 5.1,
(m) > 0 and {X =i i.o.}
n
i
p(m) (Tk]Xm=i)
that j
;C k.
T
k
(m)
(m)
jai (k)/ak
qi,j
p(m)
k’ #k, in which case
ptm)(Tk, iXn=i).
k
a.s. we easily conclude that
as m/oo, and the case i e C
Then either jeT which yields
and since i g C
k
qij
and j g C
k
follows.
Assume now
0 for ieC k, or j C k,
0 as n- is a consequence of
with
PRODUCTS OF STOCHASTIC MATRICES AND APPLICATIONS
p(m)(TklXm=i
(ii) yields
as
qij
m+ and T
and
k
Tk,
being disjoint.
217
Using now Theorem 5.1
0 and completes the proof.
REMARK5.2. Theorem 4.2(ii) holds, of course, for convergent chains as wello
The additional condition imposed on convergent chains does not seem to make
this result any stronger.
The convergent chain concept goes back to Maksimov
[15], who considered
the case of bistochastic matrices (P). Extensions to finite and countable
n
convergent chains were given by Mukherjea [16-18], etc. The methods used in
these papers are matricial and the sets of the basis are characterized by means
{Qk
of some limit points of matrices
with
Qk
However, such
limn-oPk,n"
matricial methods yielded much weaker results and do not seem to permit the
identification of
{Qk}O
The results of this section were derived in Cohn [8].
6.
FINITE MARKOV CHAINS:
THE GENERAL CASE.
We shall now consider the case when
(P }.
restrictive or no assumptions on
considering the structure of
THEOREM 6.1.
number of
(m)
(m)
sets of
Let T
of
{X :n m}
n
does not exceed
is finite and the
s
in m)
T
E(k)n {J:P(m)(TklXn=J)>0.5}
yields
p(m)(Tk)
s
> O, {E (k)
n
(k),
n
k=l,...,d}
must be non-empty for
states in S which requires d
LEMk 6.1. Let {X :n
n
m} and {X’:n
n
and E ’+
n
m’
for m,
m’} be
two finite Markov chains
(m)
PROOF.
Suppose that m’ > m and let j
(m).
1
g
,(n)
3
,(m’)p(m’,n)
,J
E+
"Pn’nm’
E+n
=i)’
(n) > 0}
{i:i
> 0.
g
E ’+
n
.
.
p(m,n).
Then
n
> O, it follows that
Chapman-Kolmogorov formula there must exist
Thus
and
n
max(m,m’).
for some itS and since
,(m)
and
Then there exists a number N > 0 such that E+
i
N and n
However
large.
n
(n)i e(m)(Xn=i) (n)m P(m’)(X
{i: (n) > 0}.
a.s. for
are disjoint for any
sequences of transition probability matrices (P)
and
n nm
Write
> 0 for
s and completes the proof.
with strictly positive initial probability vectors
respectively.
p(m)(Tk)
p(m)
limn_mo{XngE(k)}n Tk
It is easy to check that {E
k=l,...,d.
there are
(m)
be some disjoint sets
with
d
As shown in the proof of Proposition I.I, letting
k=l,...,d.
Since
As before we need start off by
n
The tail -field
Pm)-atomic
PROOF.
n
S
the number of elements of
s
For such chains we shall derive stronger results under less
is finite.
1,3
(n)
(m)(m,n) > 0
ri,j
i
J
> O. By the
(m,m’)_(m’.n) >
0.
,j
S such that
We have shown that E
+
n
Pi,
E ’+
n
Notice that
the finiteness of S makes it impossible that E + cE ’+ with strict inclusion for
n
n
all m,n and m’ > m. Thus there must exist m and N such that for m’ > m
E
+
N
E+
Choose now n > N and j
gE’+n"
Then there is a state
+
igE
such
H. COHN
218
,(n) >. ,(N)p(N,n) > 0
i
j
i,j
z(n) >. z(N,n) > 0 and E+
that
to
m
J
J
gE(k)n
i,j
> 0 for i
n
Suppose that S is finite.
{E(k)},
n
of subsets of S,
d
k=l
g
E ’+
N
But
E+
E
+
leads
N
completing the proof
Then there exist some sequences
S, such that for m--0,1,...
k--I
p(m,n). /pm,in)
lim
p(N,n)
E ’+ obtains for n > N
n
THEOREM 6.2.
for
Thus
(k). (m) /(k) (m)
(6.1)
(k)=u limn-jgE (k) Pu(,m n)
where
provided that
n
> 0, and
p(m,n)
lie
for j gE
REMARK.
upon n
d
Uk=l E(k)
n
S
n
(6.2)
0
Throughout the paper we will drop the explicit dependence of
and so
is not kept fixed, but in general varying with E
j
same convention will be valid for all the sets dependent on
n
(k).
n
j
The
to be further
considered.
PROOF.
maximal.
such that the sets {E
+}
n
attached to {X :n
n
Such a choice is always possible in view of Lemma 6.1.
Choose
m
m} are
According to (3.5)
lie
no
(m,n) in (m,n)
e (m) (TklXm=i)/e (m) (TklXm=%)
Pi,Xn .,Xn
(6.3)
for almost all mgT k, k=l,
Write now
E
(r)
n,k
:P(TklXn=J) > 0.5}
{j
{J:l
N
i, gS,m=0,1,... ,r-1
p(m)(TklXm__i)/P(m)(TklXm_ )i
<
p(m) (Tk[Xm=i)/p(m) (Tk Xm__)
<
n)/p%,j
But (6.3) implies
lim{X
for any
r
g
n
and
k
Take r=2
V,
--1,2,...
re(v) such that
p(m)(T kA
n>.m()
p(m)(T kA
n>m()
Define now E "k’l
n
U
(r) })
2
{X e E(r) })
n
2
{X
n
g
E
n,k
n,k
E "r’l for m()
n,k
}_ Tk
For each
-,
-
a.s.
one can find a number
k=l
d
k=l
d
n < m(+l) and consider the elementary set
properties
A A (B
C)
(A A B) D (A A C)
A A (B U C)
(6.4)
PRODUCTS OF STOCHASTIC MATRICES AND APPLICATIONS
219
Using (6.4) yields
p(m)(T kA U
{x E(2 u) })
E(k)})
U
n
u= m(u)n<m(u+l) n n,k
p(m)(T kA
U {X
n
n>.m()
u=
p(m)(r kA
p(m)(T kA
{X
U
n>.m(u) n
.<
.<
u=
2
U
n (u) n<m(u+l)
gE
(2u)})
{X E
n
+
n,k
P
(2u)})
n,k
(m) (T A
k
u=
N
{X
n>.m(u)
g
n
u) })
E(2
n,k
-(-2)
Similarly one can show
p(m)(T kA
{X
O
n>.m()
.< 2 -(v-l)
E(k)})
n
n
which boils down to
(t,n)
lim
n_=
for j e E
i,j
(t,n)
/P,j
+.
(k) and
i,j e E
t
n
(k)
u
(m)
p (m)
(m)
P
(re lEt=i)/P
(TklXm=u)
for n
UgEn’
(m’ ,n)
u’
is a state in E
p(n’,n)/po
u,3
{Xn:n
n).
,3
+
n’
with P
where j g E
(k).
n
lim P
constructed above for a
u,j
m
E
+
u(m, ’n)
,j
> 0.
E
,+
,n)/pu(,nl j,n)
(6.6)
(n’)
Since
u
u(n ’)
> 0 the ratio
makes sense on the probability space attached to the chain
m} and (6.5) implies
’,n)
lira
n->o
that
m
(m’,n’)p(n’.
n)/e u(n,,,j,n)
P,u
u, 3
+
uE n’
where
which depend on the
{E_(k)
(m’n’) p (n’
ei,u
+
(m’
t
Then with our choice of
m
i,j ,n)/p E,j
+
We shall nest show that (6.1)
with the same sequences
Indeed assume m’ #m.
> max(re,re’), and
particular
(6.5)
p(m)({Xn E(k)}Ixm
=u)n
lim
was chosen at the beginning of the proof.
t
(rklX t =)
If we notice now that
we get that (6 1) holds for states i, j in E
holds for every
(m)
p(n’,m)/Pu,j u(n,
P
,j
(m)
(T
(m
klXn,=u)/P
(TklXn,=u’)
Using this in (6.6) yields
(m’ ,n)/p (m’ ,n)
i,j
,j
u
m+n
p(m’ ,n’)p(m) (Tk
Xn=U
(m’ ,n’)p(m)
(T k IXn
I + P ,u
ugE
=u)
n’
p(m) (Tk iXm,=i
p (m) (r
which completes the proof of
k
IXm,=%)
(6.1).
d
P(m)
E(k) i.o.}
To prove (6.2) notice that {XngDk=l
n
p(m)(xn gEn
i.o.)
proves (6.2).
0.
This
in turn implies lim
a.s. which entails
p(m)(xngEn
n-o
0 and
H. COHN
220
The tail u-field of a finite Markov chain was proven to be finite in Cohn
[2].
Further Senchenko [19] and Cohn [3] have independently shown by different
methods that the number of atomic sets does not exceed the number of states.
losifescu [11]
The proof of Theorem 6.1 given here was taken from Cohn [4].
has studied the tail u-field structure of continuous time Markov processes.
Kingman [13] has given a geometrical representation of the transition matrices
of a nonhomogeneous Markov chain from which the tail u-field structure may be
derived.
As far as the asymptotic behaviour of transition probabilities is
concerned, it seems that the first result in the case of nonasymptotical
independent chains was given by Blackwell [i] who derived the existence of
However Blackwell’s paper
limits for the reverse transition probabilities.
The results of this section on the
does not refer to the tail u-field notion.
asymptotics of {P
7.
[5] and [7].
are derived in slightly different forms in Cohn
m,n
SOME CLASSES OF CHAINS CONSIDERED BY DOEBLIN.
In an important but little known paper published in 1937 Doeblin [9]
introduced a number of nonhomogeneous Markov chain models and gave without
One model was
proofs several results concerning their asymptotic behaviour.
defined by the following
There exists a strictly positive number 6 such that for
(DI).
CONDITION
any fixed states (i,j) either
(n)
Pi,j
>" 6 for all
(n)
or
n
0 for all
Pi,j
n
Doeblin asserted that in this case it is possible to decompose S into
disjoint ’final classes’
Go,G
may be further decomposed into
G
v
and each final class
G,
{Ci();
i=l
’cyclical subclasses’
v
i
d()}.
These have the following asymptotic properties (according to Doeblln) as
(i)
e!m:n)
l,j
0 for every ieS and jcG
(ii)
p(m,n).
0 for every i c G
(iii) if i
that n
(iv)
m
j
gG
j cG
if i
e!m: n)
p (m,n)
i",’3
and j
C,()
with
{p(n)}
for ig U
p (m)
igC()
j
gC,()
then
(mn)
0 provided
Pi,j
satisfies
v
=0
for any
G
j
jeC,()P
C()
(m,n)
i,n () ]p 3!n) + i,j
P(m)[i,n ()] is
Xn+rd( g C().
then
p(m,n)
i,j
p(n) +gm,n)
,j
j
(’-) mod d(a).
0 exponentially as n
1,3
distribution
(v)
igC()
G
# (’-) rood d()
provided that n- m
Here
with
and j
O
the limit as r
m, i and j
and the limit
(n)
d(),
and some
where
(n)
P.3
=
(m,n) are as in (iv) and
e.i,3,
of the probability, given X =i, that
m
Doeblin subsequently relaxed assumption (A) allowing positive
tend to 0 as n
oo.
v
More precisely he considered
p(n).
to
PRODUCTS OF STOCHASTIC MATRICES AND APPLICATIONS
CONDITION (D
2)
and some N >.
There exists a strictly positive number
(n)
such that for any fixed pair of states (i,j) either
Pi,j >.
(n)
0
221
for n
>. N or
limn_p
Two subclasses of chains satisfying Condition (D
(n)
In=lmax(i,j)gAP i,j<
those satisfying
_(n)
.n=imax(i,j)gAPi, j
(Condition
2) were further considered:
(D)) and those satisfying
(n)
{(i,j):limn_oPi, j O}
(Condition
(D))
where
To study chains satisfying Condition (D
2)
A
Doeblin proposed introducing an
associated chain, derived from the initial one by taking 0 for the positive
one-step transition probabilities tending to O.
However, by so doing the
transition matrices become nonstochastic, and it seems to us that Doeblin
intended to add the transition probabilities replaced by 0 to the ones bounded
away from 0 in the same row to preserve the stachasticity of the matrix.
But
there is considerable leeway in defining a matrix in this way and Doeblin’s
details are rather sketchy.
it will be easily seen that the associated chain may be
(DI)
In the case
defined in anycay described above, but in general we shall have to use some
rguments
based on the tail o-field structure to justify the definition that
we are going to adopt for an associated chain.
For the sake of definiteness
We proceed now to define an associated matrix.
we shall consider a matrix in which the entries of the initial matrix replaced
by 0 are all added to the first entry in their row larger than 6.
precisely, we say that
(n)
(p:(n.))
i,j
P’n
is an
(n)
for (i,j)EA;
Pi,j(i) =Pi,j(i) + j ES i
row such that
(i,j)A
(i,j) such that (i,j)
and
A and
asociated matrix of P n if
Pi,j(n)
’I.
n’(P’n>.m
{E:n=0,1
Denote by
lim
infn-ominigE
lim inf
*n)-i
(n) for the
pairs
j >j(i).
m)
and the
{Xn:nm}.
will be said to be associated to
a sequence of sets with the property
> 0 and write
n
(n)
n-omax igE**
i
n
0
Pi,j
Pi
A Markov chain assuming the initial probability vector
transition matrices
p,.(n.)
,3
j(i) being the first entry in the ith
{j:(i,j)EA}, and
S.I
More
E**n S-E*n
If
{E**}n
are present, then
o.
LEMMA 7.1. Suppose that there exists a sequence of positive integers
m <n
<m2<n 2 <
p(m)(xn
u=l P(mu’nu)
i,j
and j eS.
where igE*
m
Using the argument employed in the proof of Theorem 3.3 we get
P(m)(Au
p(m)(xn =j
i.o.) > 0 where
i.o.) >.
u
p(m)(Au
Au {Xmu=i,
X
=j}, u=l,2
m
u
But
i.o.) > 0 as stated.
IN what follows we shall consider a condition that contains (D
this Condition
(i)
Then
u
=j i.o.) > O.
PROOF.
that
such that
(D)
either the
.
(n)
{E**}
are empty or lim
n-o I
n
0 for i
1).
E**, and
n
We call
H. COBN
222
(n)
the sequence
(ii)
S, n=l,2,...} may contain O’s but its positive
tPi,j:i
values are bounded away from O, i.e. there exists 6 >0 such that
(n)
> 0 where inf’ means that the infinum is taken over the strictly
i,j,nPi,j)
inf
positive matrices of the sequence.
As far as
(DI).
holds under
(D)(ii)
Notice that
P
P’n’ say
an arbitrary associated matrix
is concerned
(D)(i)
may be considered and taking into
account only the position of its positive and null entries one may derive the
periodicity and the cyclically moving subclasses of a homogeneous Markov chain
Choosing D to be a
assuming such type of transition probability matrix.
=
{d(a),
multiple of
,v}
the positive and null entries in the same position as
e (n, n+ND)
i,j e
1,3"
large,
Ck(a
6ND
Ck(),
k=l,...,d()}
since for j g
Ck()
v
Ck(),
.
_(n+ND)
3
Therefore E* m U
n-
a=IGa
that E**
and it will be seen that G
G
O
n
’transient’ set.
THEOREM 7.1.
(I)
,E
n
(D)
Suppose that
(d)
n=0,1,
n
and that
Using this we can show that
ND
with c
Further
mlnig S (n)
i
it is easy to see
in this case the role of a
O plays
holds.
Then
for k=l,
d.
(ii)
If E
n
Uk=l E(k)
n
lim
P(m’n)
n-o
i3
For ieS, m=0,1
j eE
(k)
(m)
(m)(
ngEn(k)}lXm=i)
+
(k)
j eE
(k)
n
g
neEn(k)})
o( n))
(7.3)
U {X
n=N
m
m=0,1,.., and j
(7 2)
U {X
n--N
J
z,j
then for i eS
n>.N
n
P
are present
(7.1)
a.s.
0
(n)
p (m,n)
p(m)
E(k)}
Tk
n
.d
E
integer N such that for any m=O,l,
and a positive
U {Xn
n=N
For ieE
>. c
sufficiently large
n
P1
have all
there exists a sequence of disjoint events of S, say
(i)
E
for
ND
are all positive for N sufficiently
being a lower bound of these entries.
fi E*
n
Pm,n+ND
we can easily conclude that
mN
(n)
+ Or.(n) ).
p!m:n)
P
(m)
U
n=N
For
(k)
m
{X gE(k)})
n
n
(74)
3
jEn(k) m,n>.N
p(m,n)
0
(7.5)
E
n
PRODUCTS OF STOCHASTIC MATRICES AND APPLICATIONS
We shall first prove that for
PROOF.
where
P(m)(xn En
i.o.)
E* and j
n
E**_
n+l
where i
entails lim
n
(n+l) !n)
i
j
Then
i SE* and j eE*@, for
n
n1
n
sufficiently large.
n
only for finitely many
that
with i e E
n
P(m)({Xn EE(k)}I{X
n
n -I E})
conjunction with
6
Pi,j
> O" this
p(m,n)
0 for
i,j
n
limn_>oo{Xn E n)}
(k)
n
and j
(k) N
E*
Therefore E
sufficiently large and according to Lemma 7.1,
n
n
0 whenever
Pi,j
It follows that
N since otherwise (6.2) would be invalidated.
for
(n)
(k) NE** must be
and by Theorem 6.2 we get that E
empty for
ieE* and j E**
m
n
n
(n)
Thus,
E
n
infn_oon)
But lim
n) >
O, which is impossible.
infn_O
E**
sufficiently large
Suppose that for an infinity of n’s
O.
223
e-(k’)
mn+l
(n)
E
n
n
(k)
n
6 may happen
Pi,j
with k’
for k=l
d and n
Tk p(m)
a.s. yields T
It follows
# k
N which in
k
Un=N{XnE(k)}n
and (7.1) is proved.
It is easy to see that Theorem 6.2 may now be applied to conclude (7.3)
and (7.4).
Finally, (7.2) follows from Lemma 7.1, and (7.4) is a consequence
of the proven part (i).
{E(k)}n
REMARK7.1. For the chain satisfying Condition (D I)
subclasses
{Ck(e)}
{E (k)
{Cu+n(mod
n
d(a), a=
where u=l
p(m) ({Xn g Cu+n(mod
of
}IXn_
Au
d()) ()
and therefore it is either P
(m).
Since
n
If A
Au,
A’u
u
and
{p(n,n+ND)..
1,3
were not
A"u
d())
=id(a)._
v and k=l
d(a)) (a)
limn-m{Xn g Cu+n(mod
J(m)
are the
in their cyclical order, i.e.
i,j
P(m)-atomic
Indeed,
Cu+n_l(mo d d(a)) (a) })
(say) p(m) a.s. Besides,
and
Au belongs
to
(m) -atomic or a union of P (m) -atomic sets
Ck(a),
k=l,...,d(a)}
are positive for all
then there would exist two
P(m)-atomic
subsets of
and by Theorem 7.1 there would also exist two sequences of sets
A’u
p(m)
{E}
and {E"} such that
p(m)
a.s., in which case (7.5) would be contradicted.
UnN{Xn e E’}
n
n
a.s. and
UnN{Xn
A"u
E"}
n
This proves Doeblin’s
results stated before.
We next consider conditions which contain as particular cases Doeblin’s
Conditions
(D ’) and
CONDITION (B).
then
(n)
max(i,j) AnPi, j
(D)
First we consider
There exists
> 0 such that if
An
{(i,j):
(n) <
}
Pi,j
0 as n
For chains satisfying Condition (B) we consider an associated chain in
the same way as in the case
(P)
(D),
i.e., we define the associated matrices
in which the entries of the initial matrix replaced by 0 are all added
to the first entry larger than
in their row.
In fact the whole definition
of an associated matrix given before may be copies here word for word, the
only difference being that A is replaced by An
where
An
{(i,j):
p(n)
i,j
< 6}"
224
H. COHN
S. and j(i) also depend on n
and j(i,n)
and should be denoted by S.
re spec t ively.
Let us next consider
CONDITION (C).
=
()
j F
n+l
There exist sequences of disjoint sets
,d’ with d’ >. 2 and
for n
>
=
N
Theorem 7.1 shows that
(D)
P
(m) -atomic
(C); the difference
is a particular case of
limnooo{Xn
(m) -atomic set of
be an P
a.s. need not
Pi,j
d’
between these two conditions lies in that
p(m)
n=O,l
(e) and
0 for ie F
n
_(n)
a number N such that
{F():
n
(m),
e
noo
F(n)
}
Un=N{X
e
Fa(n)}
i.e. it may be a union of
j(m).
sets of
We are now in the position to formulate the following two conditions.
CONDITION
(ii)
(D).
the associated matrices satisfy
CONDITION
(D*).
.=o
(B) is satisfied with
(i)
n=l
(D).
(B) is satisfied with
(i)
n= imax( i,j)
the associated matrices satisfy (C).
(ii)
THEOREM 7.2.
If
and for ieS and j
PROOF.
and if
ig
(D)
lim
p(m,n)
noo
1,3
(m,n)
n(n)
PCm) (T klxm=i)
p (m)
j
!m).(m) >0},
n
+
(Tk)
{(Xn,
(n)
o(zj
Xn+N)
p, (m) (AlX=i)
for n > m and any N > I.
(i,j)gPi,j
and ig{j:
and lim
(u)
j
> O}
P
!n)
J
liS
u)
infn+oomlniE,n(n)
i
and take A
(u,n)
i,j
p(U,n)
-i,j
I.J(n) -.JI(n)
B=$X...
en
xs,
(u,n)
P’.1,3
(7.8)
The standard monotone
gn
Suppose that we choose an associated chain {X’:n
n
E*u
gB} for
then
class argument extends (7.8) to any A
for i
(7.7)
).
N+I time s
ip(m) (AiXn=i)
n
E’n
(7 6)
k=l,...,d’
i, j
g
0
It is easy to see that if A
{j:
and j
holds, then for iS, m=O,l
eE (k),
P
But
(n) <
max(i,j)eAnpi, j
u
>
{X =j}
n
g
u} such that
> 0
Then if we write (7.8) for m
we get
(7.9)
u
p,.(u, ,n) and therefore
i
liS(u)
P’,(.’n)
,I’’ u
i,j
I(u) -[(u,n)
ieS
’
(u)
i
(n)
j
On the other hand (7.9) implies
1,3
g
(7.10)
u
PRODUCTS OF STOCHASTIC MATRICES AND APPLICATIONS
!im
for
infn_ominjgEn,j
n
(n)
infn-minjEEn.7[(n)3
lim
limn,_>ooP’(m) (X,
E’**n’Ix’=i)n
{j:,.(n)
3
> 0 which entails E
u
E’**}
n’
e
in (7 8).
SUPn,+ooP(m)(Xn,
0 we get lim
Further it is easy to see that
for jcE* and
m
{Xn,
Take now A
sufficiently large
g
225
> O} c {j:
!n)
*
m E*
n
Since
E’**IXm=i)n
> O} and since
3
n
"<
n
’. (m) > 0
3
sufficiently large, we get
m
SUPn->ooP(m)({XngE’**}n {XmgE*m}) m But the sequence {E*n} has the
leads
property lim SUPn+ooP(m)(XnE*
limn_oP(m)(XnE** 0
n) I.
lim
This
E*’and
E’**
n
and therefore E*
n
E** for
n
n
Consider now a P (m) -atomic set of
Tk p(m)
limn_o{Xn e E(k)}n
to
sufficiently large.
n
..m),
say T
Then by Theorem 7.1
k
a.s. and (7.8) implies
U {XrEE(k)}IX’=i) gm
IP(m)( r=nU {XrE(k)}IXm
--i)r -p,(m)( r=n
Ir
m
(7 II)
and
IP(m)
r=n
{Xr eE(k)}IXm
=i)r -p,(m)( r=n {Xr
Taking the limit over
n
E(k)}iX,=i)ir gem
(7 12)
m
in (7.11) and (7.12) and using the triangle inequality
yields
_p,(m)(lim inf{XeE(k)}IX’=i)
IP ’(m)(lim sup{XeE(k)}IX’=i)
m
n
n
m
n’-o
g 2e
n’-o
,.(m)
Multiplying (7.13) by
(k)
limn-o{X’-n g En }-
we get that
probability and is either
We may now interchange
p (m) (lim
P’ (m)-atomic
p(m)
and P’
p,(m)
or a union of
m
exists, has positive
P’ (m)-atomic
sets of
(m) and get that
P
(m)(lim
inf{X’n
EE’k)}IXm=i)
n
.<
(7.14)
2g
m
(m) are finite, we conclude that their atomic sets are
a one-to-one correspondence. Therefore d d’ and
(k’)
p(m) (limn-o{Xn e E(k)
}n A limn_o{Xn e E’})n 0
and
p,(m) (limn-o{X’n e E(k)
}n
for k,k’ g
n
,d}.
{I
E ’(k) fl E’*
n
and taking the limit over
i
a.s. with respect to
sup{X E Ek) }I Xm=i)
m)
Now because
in
assuming over
(7.13)
n
(k)
E’})n
limn_o{X
0
Since we have seen in the proof of Theorem 7.1 that
for n
E’*
n
sufficiently large.
A
’(k’) c E (k) for
large enough it follows that E n
n
Now it is easy to see that
P(m)({Xn g E’n
i.o.})
k
0 and
complete the proof by an already familiar reasoning.
’(k) corresponding to the associated chain {X’:n m}
REMARK7 2. The sets E n
n
for which Theorem 7.1 guarantees
are in general smaller than the sets {E n
the same convergence property (7.7). It is therefore possible that there
(k)}
exists states i
u
E’n
(k)
with i g E
n
u
for some
k
and
n
However such states
226
H. COHN
p(m)(xn =iu
have the property
The usefulness of using {E
0.
i.o.)
’(k)}
n
{E (k)
instead of
lies in the fact that the former are more easily obtainable.
n
For example in case
we have seen that such sets may be identified by
(D)
means of an arbitrary one-step transition probability matrix.
We turn now to the case
(D*)
the ones considered so far.
LEMMA 7.2.
m < n
<m2
<n
2<
and
k}
0 for
Pi,j
iCA
n
n
"J
n-
An+l
(7.15)
with nN
i.o.) > 0 for
JkgAn
U
n=N
J
PROOF.
show that
k=l 2
k
U {X cA })
n
n=N
j cA
n
Then
(Xn g An}lEa--i)
(m)
P
for icS, m=0,1,
n
n=l,2
Let us consider the subchain X
(i)
limk_o{Xnk
gA
a s
m
p(m) -atomic
is a
(n)
Since
lip
(m)
not a p
atomic set of
X
X
X
m
n
2
n
,(m),
set of
Pi,j
{Xml,Xnl,....
Un=N{Xn EAt
k-o{XnAnk} p(m)
the tail o-field of
n>.N, we get
and some N>.
k=1,2
k
p(m,n). /(n).
lip
kCE
Ik’J
p(m)
(ii)
is such that for
(m ,nk)
k
P.
pin
and j c
igA
p(m)(Xnk Jk
(i)
We shall first need the following
Suppose that the sequence of sets {A
n
and a sequence {i
with i
I
k=l
(n)
which is considerably more complicated than
0 for
igAn
a.s.
If
We shall
2
where
and j
(m)
gAn+1
with
limkoo{XnkCAnk}
(m) then it must be a union of atomic sets of
is
is
J’ (m)
In the latter case, by Proposition 2.1, there must exist sequences
{A (I)
A
(say)
(v)}
n’
n
p(m)
_
n’ c
{ml,nl,m2,n 2
a.s. for r=1,
M such that
n
with
Further
v.
v and
(mk,nk)
Ik=M P ik, j k
(mk,nk)
AkPok,Jk
UVr=IAm
’=
"<
<
contradicts (7.15).
Further
m
-mk
since i
k
{mk:k >" M}.
mk’nk)-
cAkP
+/-k’ J k
Hence
limk_o{XnkgAnk}
A l,...,Av
for
(r)
(r) },
Ar= {mk:ikCAnm
mk g Ar"
Since
if we apply Lemma 7.1 we get that
limk_o{Xnk
T’
with
T
are disjoint for all
n’
Jk A(r)
n
k
Take
v and therefore
for r=l
A (v)
n’
,_o{Xn, e An’(r)
there must be a number
keE*
(I)
But {A
(r)
c Uv
A
for k >. M.
r=l m
and therefore there exist the sets
r=1
such that lim
n
gA
n
(mk’nk)
k=M Pik,Jk
a.s. is a
<
which
P(m)-atomic
set of
k
a s. is also a
Pm)-atomicJ
set of
--m)
’
since
(m)
PRODUCTS OF STOCHASTIC MATRICES AND APPLICATIONS
m)
otherwise
P(m)-atomic
would contain at least two
{E (I)} and {E (2)} are some sequences corresponding
n
n
p(m)
limn-{Xn gE(1)}n
TI
limn-{Xn g E(2)n
T2 p(m)
(2)
_o{Xnk g Enk
and lim
k
and T
If
2.
such that
2
a.s. and
limkooiXnk Enk(I)}
p(m)
T
g
a.s., we also get
p (m) a.s. contradicting the atomicity of
p(m)
for an infinity of k’s.
Jk gE*nk
Jk g An k
But
probability.
{Xnk
a.s., which leads to
Jk
sufficiently large and because
a.s. we get that i
p(m)
Tu
i.o
T’
with
(u)
Jk E**nk
If
a.s.
for
we shall show that A
n
E
k
n
n
N E (u)= A
for
n
k
k
T
k
k
for
(u)
limk_o{Xmk E e
T
m
u
i.e.
Notice that
sufficiently large
k
k’s.
is not empty for an infinity of
Indeed, by (7.15) and Lemma 7.1
k
large, since then (i) would imply the positivity of
P(m)(xn
subsequence of {n
k}
g
k
say
E
O.
i.o.)
n
p(m)(Xn
sufficiently
k
it is impossible for this intersection to be nonempty for all
contradicting
g
N E
A
n
k
k
n
i.o.)
k
Therefore there must exist a
k
{}
such that A
E
n
k
(u)
n
A
n
k
We
for k=l,2
k
shall further show that using the existence of such a subsequence
{n}
we
deduce (ii)
We first prove (ii) for m >. N and
Ym, i
p(m)(TulXm=i)/p(m)(Tu)
3
m,i
igA
m
Write
and take j A to get
n
s
p(m,n)
(n)
u
sufficiently large and (i) follows.
k
for
i.o.} has
Jk
To prove (ii) we shall first prove it for a subsequence {n ’:k=l,2,
k
A
a.s.
limk_o{XnkgAnk}
Indeed, i kgE* N A
limk_o{Xme gAmk
N E
kE*
{Xnk
Then
and as seen before
large we get (i) by applying Lemma 7.1.
p(m)
to T
limn+oo{Xn aE(1)}n TI p(m)
a.s. and
T2
Suppose now that
p(m)
sets T
’(m).
respect to
positive
and T
227
I
(7.16)
(nc,n)
Pi, P,j
(n)(n,n)
P
(m,nk’)
’J
=i
(p
n
(m, n)
i,
(n) (nk
Ym, i )p ,j
(n) P
n
k
(nk,n)
,j
n)
H. COHN
228
(p
(m,n)
(n) (n, n)
(n, n)
I g(n) P ,j
rgAn
n
(m, nl)
i, r
I’
rgA
n
k
k
(nl)
Ym, ir
(nl)
m
r
where the prime in the last two sums indicates that the sum is restricted to
the values
such that P
r
(nl ,n)
> 0.
r,j
Now if n
Theorem 6.1 implies that the last sum in (7.16) goes to 0 as k
We prove now (ii) for arbitrary
m
and i
and
we may take k
o%
Consider the conditional
probabilities
p?(m,r).
l,
p(m)(xm’+l Am’+l’’’’’Xr-I
3
for j
p!m:m+l)
p,.(m.,m+l)
{Xn gAn
(7.18)
1]
i,]
Since
_c
(7.17)
max(n-l,m) and r > m’ + I, and
m’
> N,
EAR,
Ar-l’Xr=JlXm=i)
{Xn+l gAn+l}
N, a slight modification of a standard
for n
reasoning from the theory of homogenoeus chains yields
n-I
p(m,n)
I
i,j
E--m’+l
I P*(m’E)P(E’n)
i,k
k,j
>. N
We recall now that for
+
kgA
gAE
and k
P*I-’n)
i
(7.19)
we have already shown that
p(,n)
limn-
k’n)1.
Y,k
..
3
N’
with
where
--m’
p(m) (T)
It follows that for an arbitrary
u
N’ > m’ +I
N’
lira
Y,k
kgA
(,n)
p,(m,) Pk,j
i,k
P
(m)
N’
(--m’U {X
r! n)
n
p (m)
A} IXm=i)
(7.20)
(Tu)
Because N’ was arbitrarily chosen (7.19) and (7.20) together imply
(m,n)
lira inf
Pi,L
!n)
3
n->o
for any
m
Further
and
.
igS
p (m)
(Tu IXm=i)
(Tu)
p (m)
i
(re,n)
! m)Pi’j
I
and (7.21) yields
(n)
3
iS
m)
p(m) (Tu Xm__i)
p (m) (Tu)
(7.21)
PRODUCTS OF STOCHASTIC MATRICES AND APPLICATIONS
p
!m:n)
p (m) (T
(n)
p (m)
lim
1,3
n-o
j
229
iXm_ i)
u
(r)
u
completing the proof.
THEOREM 7.3.
Suppose that
(k)
()
d’} and E
U
F
{I
ek
n
(D)
k=l
n
i(k),i
(k),...
m
m
such that i
2
<n <m2 <n 2 <...
(k) e
E*
m
u
for
m
and
u
(k)
for any i
(ii)
for i eS, m=O,l
n
g
E
p!m:n)
,
and j e E
u
sufficiently large
p (m)
limn-{Xn eE(k)}n p(m)
k
(iii) for any ieS, m=O,l
lira
n-o
(iv)
if
n(k)
j
En+l
i.o.) > 0, k=l
(k)
d
n=l2,...
n
o(j(n))
+
(Tk)
a.s., k=l
and j e E
(m,n)
O, then
Pi,j
(k)
p(m)(Tkl Xm=i)
(n)
j
3
where T
p(m)(xn=in
n
n
d
d
S
(k)
E
Uk=
k contains more
than one element and if F
() c -(n)
then the
n
average sojourn time spent in the sequence of sets {F ()
(a)__
rn+
n
PROOF.
as n+ o, but
p(m)({Xn
(n)
E(Y IXn=i)
J e-n+k+l
for i e
given that
ult.})
O.
By definition,
< oo)
i.
p,.(n+k)
,j
F(a)n
Let us denote by
the average sojourn time referred to in the statement of the
p(m)(y(n)
()
n
It remains to prove (iv).
y(n)e
m on the set
{Xn EF
{Xn+m F()}n+m for re=l,2,... Because % contains
by (i) p(m)(x =i i.o.) > 0 for any sequence {i
n n
n
that
()
eF
It is easy to see that (i) and (ii) follow from Lemma 7.2, whereas
(lii) is implied by Theorem 6.2.
Theorem.
n=l 2
n
0
Pi,j
()
Xn=i with ieF
goes to
n
d} there
and a sequence of
(n)
ax
icE
(i)
be a partition of
u
(m ,n
u u
u=l rain jeEn(k) p im(k),
j
u
d
If for any ke {I
d.
exists a sequence of positive integers m
states
I’
holds and let
and m
N
(a)
{Xn+m+l g Fn+m-l}
more than one element, and
(k),
with i E
n
n
we conclude
Further, it is easy to see that since
for i e F
2
()n }N
() and
any k
n+k
I
p(n,n+m-l)
we get
j eF
Taking into account that
i j
(a)
n+m-I
(n)
Ipi, j Pi,I.)[
0 as n
for all i,j eS we get
lira
n-o
p(n)({XneF()}n 0
which implies that
since
p(m)(xn g F n()
0
limn_oE(y(n)IXn=i)
i.o.)
(a)
{Xn+m-i e Fn+m-l}IXn=i)
for i e
p(m)(xn (E(k)_F())
n
n
F(a)n
and
i.o.)
e
p(m)(Tk)
Finally,
> 0 we get
H. COHN
230
that
p(m)(Xn g F n(a)
ult.)
0 and the proof is complete.
As a corollary to Theorem 7.3 we shall give a result that describes the
asymptotic behaviour of a chain that satisfies Condition
COROLLARY 7.1.
{(,), =I
partition of
k=l
Suppose that
d and denote
Zn=Imine(n)
(,)
v
holds and let
d
i,2
_(n)
d()}.
;=I
Let E
k
maxigc(a),
be a
U(,a)gFk C(a),
(n)
g(,);(E,,a,)
(n)
jgCc,(a) Pi,j
Assume that
where the minimum is taken over all
(’,’)
gxk k=l
(E’,’)k’ Then the
{(,);(’,’)
and
(D*)
(D*).
d},
and that
(n)
0 for (
e(,):(,,,)
a)gk
statement of Theorem 7.3 holds.
We shall omit the proof of this result, which may be carried out by
arguments already used in this paper.
We notice that in the case of Condition
(D’)
Doeblin’s statement is wrong.
However, examining Doeblin’s formulae makes it clear that he felt that unlike
the previous situations, the limit of the conditional probability that the
chain will circulate through the cyclical subclasses of a fixed class may not
exist here.
The analogy to the homogeneous case seems to break down for the
chains satisfying
(D)
since several atomic sets of the tail o-field of the
associated chain may be lumped into one atomic set of the tail o-field of the
original chain.
Theorem 7.3(iv) generalizes a result stated by Doeblin about chains
satisfying Condition
(D).
It is hard to see how Doeblin could have reached
his conclusions in this respect, given the knowledge available at the time his
paper was written.
The results of this section, in slightly different form, were given in
Cohn [7].
8.
WEAK ERGOD IC ITY.
One of the main concerns of the theory of finite stochastic matrices has
been to characterize sequences of matrices satisfying the so-called ’weak
ergodicity’ condition, i.e.
(re,n)
(m,n)
llmri, j
,j
noo
for any i,j, and
m
(8.1)
0
This condition has been introduced by Kolmogorov
and most papers on nonhomogeneous chains are related to it.
[14]
Doeblin [9] has
found necessary and sufficient conditions for (8.1) and Hajnal [I0] has derived
similar conditions unaware of Doeblin’s results.
We shall first give a result
that relates weak ergodicity to the structure of the tail o-field.
THEOREM 8.1.
The following conditions are equivalent
(i)
weak ergodicity;
(ii)
any Markov chain {X :n m} with transition probability
n
arbitrary initial distribution
PROOF.
(m)
has a
P(m)-trivial
o-field
(Pn) nm
m).
and
Since S is finite we may assume, if necessary after relabelling
the states at successive times n=O,l,..., that there is a positive state j gS.
PRODUCTS OF STOCHASTIC MATRICES AND APPLICATIONS
231
Then Theorem 3.2 and Remark 3.1 imply that
pCm,n’)/pm,n’). p(m)(Tkl X =i)/pCm)CTklXm=A
lim
n gF
k
If (8.1) holds then
p(m)(rklXm=i)/p(m)(Tklxm__)
(8.2).
m)
However, if
i
p(m) (TklXn--i)
limn-mP(m)(rklXn ITk
in view of
P(m)-trivial.
necessarily follows by
(m) -trivial this
P
is not possible as by the
is not
martingale convergence theorem
some
C8.2)
m
1,]
n,_m
Suppose now that
m)
is
must have values close to 0 for
p(m)
a.s.
P(m)-trivial.
Thus
m)
is
Then by Theorem 6.2 we
know that
lid
for i
E
P(m’n)/P(,mn)
p(m) (X
But lid
n-o
n
THEOREM 8.2.
Let
(8.3)
i.
g
n
(Pn)
E
n)
0 for all
m
and (8.1) follows
be a sequence of finite stochastic matrices.
The
following two conditions are equivalent:
(i)
weak ergodicity;
(ii)
there exists a sequence of sets {E
lim
n-co
for j g E
(I),
n
(m,n)
I)}"
n
such that for i,ES and mgN
(m,n)
ei,j /Pi,j
and for any leS and m=O,l
lim
n-+co
I
p (m,n)
j
jeE(1)i,
n
This result is a consequence of Theorems 6.2 and 8.1.
A classical type of results in the theory of nonhomogeneous Markov chains
establishes weak ergodiclty in terms of some coefficients attached to a
A historical account of such coefficients, that goes back
stochastic matrix.
to Doeblln, may be found in Seneta
result of this kind.
Kingman [13] has proven a general
[20].
Usually, the proof is carried out by some inequalities
relating the coefficients of the product of two matrices to the coefficients
of the matrices themselves.
For example, Hajnal [I0] considered the following
coefficient attached to a matrix P with entries
8
s,
P,8
s
s
{P}
min
,’ 8=1
min(p
8
’P’
8
We show next that such results are immediate consequences of the results
given in this paper by proving the following theorem due to Hajnal [I0].
THEOREM 8.3.
A sequence of stochastic matrices
(Pn)
there exists an increasing sequence of positive integers
’j {Pnj ,nn+l
PROOF.
o-field
sets.
is weakly ergodic if
nl,n2,..,
such that
diverges.
According to Theorem 8.1,
m)
is not
P(m)-trivial
(Pn)
is not weakly ergodic if the tail
and thus assumes at least two
According to Lemma 7.1 we must have
P(m)-atomic
H. COHN
232
,(nj ,nj+l)
j=l P
j {Pnj ,nj+
<
whenever
g
E
(k) N
E*
n.3
n
B g E (k’)
and
convergent for any sequence n
with k
nj+
<n 2 <
# k’
which makes
and finishes the proof.
There is an important result for bounded positive matrices known in the
demographic literature as the Coale-Lopez theorem (see Seneta [21]).
The
result was given in a somewhat more general form in Seneta [21] and its proof
seems rather laborious.
The specialization of the Coale-Lopez theorem to the
case of stochastic matrices reveals a strong asymptotic independence property
We shall state such a property under a less restrictive assumption on the
stochastic matrices.
THEOREM 8.4.
Let
(Pn)
be a weakly ergodic sequence of stochastic matrices
(o,n) > 0
such that lim inf
for any j
n-omax igs i,j
lim
n_o
gS.
Then for all i gS
p(m,n)/p(m,n)
i,j
,j
The proof follows easily from Theorem 3.2 and Remark 3.1 in view of the
fact that {E
n
are empty.
The Lopez theorem imposes the condition
and
and any
n
(n,n+r)
O
> 0 for a certain r
>"
Pi,j
This clearly implies weak ergodicity, since as seen in the
course of the proof of Theorem 8.3, the failure of weak ergodicity prevents
(n,n+r o)
P.
1,3
from being bounded away from 0 for all i,j and
n
The results of this section were derived in Cohn [5].
REFERENCES
I.
BLACKWELL, D.
1945.
2.
COHN, H.
3.
COHN, H. On the tail events of a Markov chain,
65-72, 1974.
4.
COHN, H. A ratio limit theorem for the finite nonhomogeneous Markov chains,
Israel J. Math. 19, 329-334, 1974.
5.
COHN, H. Finite non-homogeneous Markov chains:
Adv. Appl. Probab. 8, 502-516, 1976.
6.
COHN, H. Countable non-homogeneous Markov chains:
Ad
Appl. Probab. 9, 542-552, 1977.
Ann. Math. 46, 594-599,
Finite non-homogeneous Markov chains,
Ann. Math.
On the tail o-algebra inhomogeneous Markov chains,
Statist. 41, 2175-2176, 1970.
Z. Wahrscheinlich. 29,
Asymptotic behaviour,
Asymptotic behavlour,
v
7.
COHN, H.
On a paper by Doeblin on non-homogeneous Markov chains,
Adv.
Appl. Probab. 13, 388-401, 1981.
8.
COHN, H. On a class of non-homogeneous Markov chains,
Phil. Soc. 92, 527-534, 1982.
9.
DOEBLIN, W. Le cas discontinu des probabilits en
Univ. Massryk (Brno) 236, 1937.
i0.
HAJNAL, J.
II.
IOSIFESCU, M.
1982.
12.
KEMENY, J.G., SNELL, LoJ., and KNAPP, A.W.
Springer, New York, 1976.
Math. Proc. Camb.
chane,
Pub. Fac. Sci.
Weak ergodicity in non-homogeneous Markov chains,
Phil. Soc. 54, 233-246, 1958.
On finite tail o-algebras,
Proc. Cambo
Z. Wahrscheinlich. 24, 159-166,
Denumerable Markov Chains,
PRODUCTS OF STOCHASTIC MATRICES AND APPLICATIONS
233
13.
KINGMAN, J.F.C. Geometrical aspects of the theory of non-homogeneous
Markov chains, Math. Proc. Camb. Phil. Soc. 77, 171-185, 1975.
14.
KOLMOGOROV, A.N. Uber die analytischen Methoden in der
Wahrscheinlichkeitsrechnung, Math. Ann. 104, 415-418, 1931.
15.
16.
MAKSI}DV, V.M. Convergence of non-homogeneous biostochastic Markov chains,
Theor. Probab. Appl. 15, 604-618, 1970.
MUKHERJEA, A. Limit theorems: Stochastic matrices, erdogic Markov chains,
and measures on semigroups. Probabilistic Analysis and Related Topics,
Vol. 2, 143-203, Academic Press.
17.
MUKHERJEA, A.
18.
MUKHERJEA, A. and CHAUDHURI, R. Convergence of non-homogeneous stochastic
chains II, Math. Proco Camb. Phil. Soc. 90, 167-182, 1981.
SENCHENKO, D.V. The final -algebra of an inhomogeneous Markov chain with
finite number of states, Math. Notes 12, 610-613, 1972.
19.
20.
21.
22.
A new result on the convergence of non-homogeneous
stochastic chains, Trans. Amer. Math. Soc. 262, 505-520, 1980.
SENETA, E. On the historical development of the theory of finite
nonhomogeneous Markov chains, Proc. Camb. Phil. Soc. 74, 507-513, 1973.
SENETA, E. Non-negative Matrices, George Allen and Unwin Ltd., London,
1973.
The number of phases in inhomogeneous Markov fields with
finite state spaces on N and Z and their behaviour at infinity, }th.
WINKLER, G.
Nachr. 104, 101-117, 1981.