Hindawi Publishing Corporation
Journal of Inequalities and Applications
Volume 2011, Article ID 470910, 18 pages
doi:10.1155/2011/470910
Research Article
Some Shannon-McMillan Approximation
Theorems for Markov Chain Field on the
Generalized Bethe Tree
Kangkang Wang1 and Decai Zong2
1
School of Mathematics and Physics, Jiangsu University of Science and Technology,
Zhenjiang 212003, China
2
College of Computer Science and Engineering, Changshu Institute of Technology,
Changshu 215500, China
Correspondence should be addressed to Wang Kangkang, wkk.cn@126.com
Received 26 September 2010; Accepted 7 January 2011
Academic Editor: Józef Banaś
Copyright q 2011 W. Kangkang and D. Zong. This is an open access article distributed under
the Creative Commons Attribution License, which permits unrestricted use, distribution, and
reproduction in any medium, provided the original work is properly cited.
A class of small-deviation theorems for the relative entropy densities of arbitrary random field on
the generalized Bethe tree are discussed by comparing the arbitrary measure μ with the Markov
measure μQ on the generalized Bethe tree. As corollaries, some Shannon-Mcmillan theorems for
the arbitrary random field on the generalized Bethe tree, Markov chain field on the generalized
Bethe tree are obtained.
1. Introduction and Lemma
Let T be a tree which is infinite, connected and contains no circuits. Given any two vertices
x/
y ∈ T, there exists a unique path x x1 , x2 , . . . , xm y from x to y with x1 , x2 , . . . , xm
distinct. The distance between x and y is defined to m − 1, the number of edges in the path
connecting x and y. To index the vertices on T, we first assign a vertex as the “root” and label
it as O. A vertex is said to be on the nth level if the path linking it to the root has n edges. The
root O is also said to be on the 0th level.
Definition 1.1. Let T be a tree with root O, and let {Nn , n ≥ 1} be a sequence of positive
integers. T is said to be a generalized Bethe tree or a generalized Cayley tree if each vertex
on the nth level has Nn1 branches to the n 1th level. For example, when N1 N 1 ≥ 2
and Nn N n ≥ 2, T is rooted Bethe tree TB,N on which each vertex has N 1 neighboring
2
Journal of Inequalities and Applications
Level 3
Level 2
Level 1
(2,2)
(2,5)
(1,1)
(1,3)
(1,2)
(0,1)
Root
Figure 1: Bethe tree TB,2 .
vertices see Figure 1, TB,2 , and when Nn N ≥ 1 n ≥ 1, T is rooted Cayley tree TC,N on
which each vertex has N branches to the next level.
In the following, we always assume that T is a generalized Bethe tree and denote by
T n the subgraph of T containing the vertices from level 0 the root to level n. We use n, j
1 ≤ j ≤ N1 · · · Nn , n ≥ 1 to denote the jth vertex at the nth level and denote by |B| the
number of vertices in the subgraph B. It is easy to see that, for n ≥ 1,
n
n
n N0 · · · Nm 1 N1 · · · Nm .
T m0
1.1
m1
Let S {s0 , s1 , s2 , . . .}, Ω ST , ω ω· ∈ Ω, where ω· is a function defined on T and
taking values in S, and let F be the smallest Borel field containing all cylinder sets in Ω. Let
X {Xt , t ∈ T} be the coordinate stochastic process defined on the measurable space Ω, F;
that is, for any ω {ωt, t ∈ T}, define
Xt ω ωt,
XT
n
Xt , t ∈ T n ,
t ∈ T.
1.2
n
n n
μ X T xT
μ xT .
1.3
Now we give a definition of Markov chain fields on the tree T by using the cylinder
distribution directly, which is a natural extension of the classical definition of Markov chains
see 1.
Definition 1.2. Let Q Qj | i. One has a strictly positive stochastic matrix on S, q qs0 ,
qs1 , qs2 . . . a strictly positive distribution on S, and μQ a measure on Ω, F. If
μQ x0,1 qx0,1 ,
n−1 N
n 0 ···Nm
qx0,1 μQ xT
m0
i1
N
m1 i
q xm1,j | xm,i ,
jNm1 i−11
n ≥ 1.
1.4
Journal of Inequalities and Applications
3
Then μQ will be called a Markov chain field on the tree T determined by the stochastic matrix
Q and the distribution q.
Let μ be an arbitrary probability measure defined as 1.3, denote
n 1
fn ω − n log μ X T .
T 1.5
fn ω is called the entropy density on subgraph T n with respect to μ. If μ μQ , then by 1.4,
1.5 we have
⎡
1
fn ω − n ⎣log qX0,1 T ···Nm
n−1 N0
m0
i1
N
m1 i
⎤
log q Xm1,j | Xm,i ⎦.
1.6
jNm1 i−11
The convergence of fn ω in a sense L1 convergence, convergence in probability, or
almost sure convergence is called the Shannon-McMillan theorem or the entropy theorem or
the asymptotic equipartition property AEP in information theory. The Shannon-McMillan
theorem on the Markov chain has been studied extensively see 2, 3. In the recent years,
with the development of the information theory scholars get to study the Shannon-McMillan
theorems for the random field on the tree graph see 4. The tree models have recently
drawn increasing interest from specialists in physics, probability and information theory.
Berger and Ye see 5 have studied the existence of entropy rate for G-invariant random
fields. Recently, Ye and Berger see 6 have also studied the ergodic property and ShannonMcMillan theorem for PPG-invariant random fields on trees. But their results only relate to
the convergence in probability. Yang et al. 7–9 have recently studied a.s. convergence of
Shannon-McMillan theorems, the limit properties and the asymptotic equipartition property
for Markov chains indexed by a homogeneous tree and the Cayley tree, respectively. Shi and
Yang see 10 have investigated some limit properties of random transition probability for
second-order Markov chains indexed by a tree.
In this paper, we study a class of Shannon-McMillan random approximation theorems
for arbitrary random fields on the generalized Bethe tree by comparison between the arbitrary
measure and Markov measure on the generalized Bethe tree. As corollaries, a class of
Shannon-McMillan theorems for arbitrary random fields and the Markov chains field on the
generalized Bethe tree are obtained. Finally, some limit properties for the expectation of the
random conditional entropy are discussed.
Lemma 1.3. Let μ1 and μ2 be two probability measures on Ω, F, D ∈ F, and let {τn , n ≥ 0} be a
positive-valued stochastic sequence such that
τn
lim inf n > 0,
n
T μ1 -a.s. ω ∈ D,
1.7
then
n μ2 X T
1
lim sup log n ≤ 0,
μ1 X T
n → ∞ τn
μ1 -a.s. ω ∈ D.
1.8
4
Journal of Inequalities and Applications
In particular, let τn |T n |, then
n XT
μ
2
1
lim sup n log n ≤ 0,
T μ1 X T
n→∞
μ1 -a.s. ω ∈ D.
1.9
Proof see 11. Let
n μ
XT
1
ϕ μ | μQ lim sup n log
.
T μQ X T n
n→∞
1.10
ϕμ | μQ is called the sample relative entropy rate of μ relative to μQ . ϕμ | μQ is also called
the asymptotic logarithmic likelihood ratio. By 1.9
n μ XT
1
ϕ μ | μQ ≥ lim inf n log
≥ 0,
n → ∞ T
μQ X T n
μ-a.s.
1.11
Hence ϕμ | μQ can be look on as a type of measures of the deviation between the arbitrary
random fields and the Markov chain fields on the generalized Bethe tree.
2. Main Results
Theorem 2.1. Let X {Xt , t ∈ T} be an arbitrary random field on the generalized Bethe tree. fn ω
Q
and ϕμ | μQ are, respectively, defined as 1.5 and 1.10. Denote α ≥ 0, Hm Xm1,j | Xm,i the
random conditional entropy of Xm1,j relative to Xm,i on the measure μQ , that is,
Q
Hm Xm1,j | Xm,i −
q xm1,j | Xm,i log q xm1,j | Xm,i .
xm1,j ∈S
2.1
Let
Dc ω : ϕ μ | μQ ≤ c ,
···Nm
n−1 N0
1 bα lim sup n T n→∞
m0
i1
N
m1 i
2.2
−α
EQ log2 q Xm1,j | Xm,i · q Xm1,j | Xm,i
| Xm,i < ∞,
jNm1 i−11
2.3
Journal of Inequalities and Applications
5
when 0 ≤ c ≤ α2 bα /2,
⎧
⎨
lim sup
n→∞
≤
⎩
1
fn ω − n T 2cbα ,
···Nm
n−1 N0
m0
i1
⎫
⎬
Q
Hm Xm1,j | Xm,i
⎭
i−11
N
m1 i
jNm1
2.4
μ-a.s. ω ∈ Dc.
⎧
⎫
···Nm
N
n−1 N0
⎨
⎬
m1 i
1
Q
lim inf fn ω − n Hm Xm1,j | Xm,i
T n→∞ ⎩
⎭
m0 i1 jNm1 i−11
≥ − 2cbα − c,
2.5
μ-a.s. ω ∈ Dc.
In particular,
⎡
1
lim ⎣fn ω − n T n→∞
0,
···Nm
n−1 N0
m0
i1
N
m1 i
Q
Hm Xm1,j
jNm1 i−11
⎤
| Xm,i ⎦
2.6
μ-a.s. ω ∈ D0,
where log is the natural logarithmic, EQ is expectation with respect to the measure μQ .
Proof. Let Ω, F, μ be the probability space we consider, λ an arbitrary constant. Define
EQ qXm1,j | Xm,i −λ | Xm,i xm,i qxm1,j | xm,i 1−λ ;
2.7
xm1,j ∈S
denote
n−1 N
0 ···Nm
n
μQ λ, xT
qx0,1 m0
i1
N
m1 i
qxm1,j | xm,i 1−λ
.
−λ
| Xm,i xm,i
jNm1 i−11 EQ qXm1,j | Xm,i 2.8
6
Journal of Inequalities and Applications
We can obtain by 2.7, 2.8 that in the case n ≥ 1,
n
μQ λ; xT
xLn ∈S
n−1 N
0 ···Nm
qx0,1 m0
xLn ∈S
μQ λ; x
T n−1
i1
···Nn−1
N0
xLn ∈S
μQ λ; xT
n−1
···Nn−1
N0
i1
μQ λ; xT
n−1
1−λ
q xm1,j | xm,i
−λ
| Xm,i xm,i
jNm1 i−11 EQ q Xm1,j | Xm,i
N
m1 i
···Nn−1
N0
i1
i1
1−λ
q xn,j | xn−1,i
−λ
| Xn−1,i xn−1,i
jNn i−11 EQ q Xn,j | Xn−1,i
Nn i
1−λ
q xn,j | xn−1,i
−λ
| Xn−1,i xn−1,i
jNn i−11 xn,j ∈S EQ q xn,j | xn−1,i
Nn i
2.9
−λ
EQ q Xn,j | Xn−1,i
| Xn−1,i xn−1,i
−λ
| Xn−1,i xn−1,i
jNn i−11 EQ q xn,j | xn−1,i
Nn i
n−1
,
μQ λ; xT
0
μQ λ; xT
qx0,1 1.
2.10
x0,1 ∈S
xL0 ∈S
n
n
Therefore, μQ λ, xT , n 0, 1, 2, . . . are a class of consistent distributions on ST . Let
n
μQ λ, X T
Un λ, ω ,
μ X T n
2.11
then {Un λ, ω, Fn , n ≥ 1} is a nonnegative supermartingale which converges almost surely
see 12. By Doob’s martingale convergence theorem we have
lim Un λ, ω U∞ λ, ω < ∞.
n→∞
μ-a.s.
2.12
μ-a.s.
2.13
Hence by 1.3, 1.9, 2.9, and 2.11 we get
1
lim sup n log Un λ, ω ≤ 0.
T n→∞
Journal of Inequalities and Applications
7
By 1.4, 2.8, and 2.11, we have
1
log Un λ, ω
T n ···Nm
n−1 N0
1 n T m0
i1
N
m1 i
−λ log q Xm1,j | Xm,i − log EQ qXm1,j | Xm,i −λ | Xm,i
jNm1 i−11
n μQ X T
n log n .
T μ XT
1
2.14
By 1.10, 2.2, 2.13, and 2.14 we have
···Nm
n−1 N0
1 lim sup n T n→∞
m0
i1
≤ ϕ μ | μQ ≤ c,
N
m1 i
−λ log q Xm1,j | Xm,i − log EQ qXm1,j | Xm,i −λ | Xm,i
jNm1 i−11
μ-a.s. ω ∈ Dc.
2.15
By 2.15 we have
···Nm
n−1 N0
1 lim sup n T n→∞
m0
i1
N
m1 i
−λ log q Xm1,j | Xm,i − EQ log q Xm1,j | Xm,i | Xm,i
jNm1 i−11
···Nm
n−1 N0
1 ≤ lim sup n T n→∞
m0
i1
N
m1 i
−λ
log EQ q Xm1,j | Xm,i
| Xm,i
jNm1 i−11
−EQ −λ log q Xm1,j | Xm,i | Xm,i c,
μ-a.s. ω ∈ Dc.
2.16
By the inequality
x−λ − 1 λ log x ≤
1 2
λ log x2 x−|λ| ,
2
0 ≤ x ≤ 1,
log x ≤ x − 1 x ≥ 0 and 2.16, 2.17, 2.3, we have in the case of |λ| < α,
2.17
8
Journal of Inequalities and Applications
···Nm
n−1 N0
1 lim sup n T n→∞
m0
i1
N
m1 i
−λ log q Xm1,j | Xm,i − EQ log q Xm1,j | Xm,i | Xm,i
jNm1 i−11
···Nm
N
n−1 N0
m1 i
1 ≤ lim sup n EQ qXm1,j | Xm,i −λ | Xm,i − 1
T n→∞
m0 i1 jNm1 i−11
−EQ − λ log q Xm1,j | Xm,i | Xm,i c
···Nm
N
n−1 N0
m1 i
1 ≤ lim sup n EQ λ2 log2 q Xm1,j | Xm,i
n→∞ 2 T
m0 i1 jNm1 i−11
·qXm1,j | Xm,i −|λ| | Xm,i c
···Nm
N
n−1 N0
m1 i
−α
λ2 ≤ lim sup n EQ log2 q Xm1,j | Xm,i · q Xm1,j | Xm,i
| Xm,i
n→∞ 2 T
m0 i1 jNm1 i−11
1 2
λ bα c.
c 2
μ-a.s. ω ∈ Dc.
2.18
When 0 < λ < α, we get by 2.18
···Nm
n−1 N0
1 lim sup n T n→∞
m0
≤
i1
1
c
λbα ,
2
λ
N
m1 i
− log q Xm1,j | Xm,i − EQ log q Xm1,j | Xm,i | Xm,i
jNm1 i−11
μ-a.s. ω ∈ Dc.
2.19
Let gλ 1/2λbα c/λ, in the case 0 < c ≤ α2 bα /2, then it is obvious gλ attains, at
λ 2c/bα , its smallest value g 2c/bα 2cbα on the interval 0, α. We have
···Nm
n−1 N0
1 lim sup n T n→∞
m0
≤
2cbα ,
i1
N
m1 i
− log q Xm1,j | Xm,i − EQ log q Xm1,j | Xm,i | Xm,i
jNm1 i−11
μ-a.s. ω ∈ Dc.
2.20
Journal of Inequalities and Applications
9
When c 0, we select 0 < λi < α such that λi → 0 i → ∞. Hence for all i, it follows from
2.19 that
···Nm
n−1 N0
1 lim sup n T n→∞
m0
≤ 0,
i1
N
m1 i
− log q Xm1,j | Xm,i − EQ log q Xm1,j | Xm,i | Xm,i
jNm1 i−11
μ-a.s. ω ∈ D0.
2.21
It is easy to see that 2.20 also holds if c 0 from 2.21.
Analogously, when −α < λ < 0, it follows from 2.18 if 0 ≤ c ≤ α2 bα /2,
···Nm
N
n−1 N0
m1 i
1 lim inf n − log q Xm1,j | Xm,i − EQ log q Xm1,j | Xm,i | Xm,i
n→∞ T
m0 i1 jNm1 i−11
≥ − 2cbα ,
μ-a.s. ω ∈ Dc.
2.22
Setting λ 0 in 2.14, by 2.14 we have
n μQ X T
lim sup n log Un 0, ω lim sup n log n ≤ 0,
T T μ XT
n→∞
n→∞
1
1
μ-a.s.
2.23
Noticing
Q
Hm Xm1,j | Xm,i EQ − log q Xm1,j | Xm,i | Xm,i .
2.24
By 1.4, 1.5, 2.20, and 2.23, we obtain
⎡
1
lim sup⎣fn ω − n T n→∞
···Nm
n−1 N0
m0
i1
N
m1 i
⎤
Q
Hm Xm1,j , Xm,i ⎦
jNm1 i−11
n μ
XT
Q
1
≤ lim sup n log n T μ XT
n→∞
1
lim sup n T n→∞
···Nm
n−1 N0
m0
i1
N
m1 i
jNm1 i−11
− log q Xm1,j | Xm,i − EQ log q Xm1,j | Xm,i | Xm,i
≤ 2cbα , μ-a.s. ω ∈ Dc.
2.25
10
Journal of Inequalities and Applications
Hence 2.4 follows from 2.25. By 1.4, 1.5, 1.10, 2.2, and 2.22, we have
⎡
1
···Nm
n−1 N0
lim inf⎣fn ω − n T n→∞
m0
i1
N
m1 i
⎤
Q
Hm ω⎦
jNm1 i−11
n ⎤
μQ X T
⎥
⎢
≥ lim inf n log⎣ n ⎦
n → ∞ T
μ XT
⎡
1
2.26
···Nm
n−1 N0
1 lim inf n n → ∞ T
m0
i1
N
m1 i
jNm1 i−11
− log q Xm1,j | Xm,i − EQ log q Xm1,j | Xm,i | Xm,i
≥ −ϕ μ | μQ − 2cbα ≥ − 2cbα − c, μ-a.s. ω ∈ Dc.
Therefore 2.5 follows from 2.26. Set c 0 in 2.4 and 2.5, 2.6 holds naturally.
Corollary 2.2. Let X {Xt , t ∈ T} be the Markov chains field determined by the measure μQ on the
Q
generalized Bethe tree T ·fn ω, bα are, respectively, defined as 1.6 and 2.3, and Hm Xm1,j | Xm,i is defined by 2.1. Then
⎧
⎫
···Nm
N
n−1 N0
⎨
m1 i
⎬
1 Q
Hm Xm1,j | Xm,i
0. μQ -a.s.
lim fn ω − n T n → ∞⎩
⎭
m0 i1 jNm1 i−11
2.27
Proof. We take μ ≡ μQ , then ϕμ | μQ ≡ 0. It implies that 2.2 always holds when c 0.
Therefore D0 Ω holds. Equation2.27 follows from 2.3 and 2.6.
3. Some Shannon-McMillan Approximation Theorems on
the Finite State Space
Corollary 3.1. Let X {Xt , t ∈ T} be an arbitrary random field which takes values in the alphabet
S {s1 , . . . , sN } on the generalized Bethe tree. fn ω, ϕμ | μQ and Dc are defined as 1.5,
Q
1.10, and 2.2. Denote 0 ≤ α < 1, 0 ≤ c ≤ 2Nα2 /1 − αe2 . Hm Xm1,j | Xm,i is defined as
above. Then
⎡
1
lim sup⎣fn ω − n T n→∞
≤
−1
2e
1 − α
√
2cN,
···Nm
n−1 N0
m0
i1
N
m1 i
⎤
Q
Hm Xm1,j | Xm,i ⎦
jNm1 i−11
μ-a.s. ω ∈ Dc,
3.1
Journal of Inequalities and Applications
⎡
···Nm
n−1 N0
1 lim inf⎣fn ω − n T n→∞
m0
≥−
2e−1 √
2cN − c,
1−α
i1
11
N
m1 i
Q
Hm Xm1,j
⎤
| Xm,i ⎦
jNm1 i−11
3.2
μ-a.s. ω ∈ Dc.
Proof. Set 0 < α < 1 we consider the function
φx log x2 x1−α ,
0 < x ≤ 1, 0 < α < 1
Set φ0 0 .
3.3
Then
2
φ x x−α 2 log x log x 1 − α .
3.4
Let φ x 0 thus x e2/α−1 . Accordingly it can be obtained that
max φx, 0 ≤ x ≤ 1 φ e2/α−1 2
α−1
2
e−2 .
3.5
By 2.3 and 3.5 we have
···Nm
n−1 N0
1 bα lim sup n T n→∞
m0
i1
jNm1 i−11
···Nm
n−1 N0
1 lim sup n T n→∞
m0
1
≤ lim sup n T n→∞
N
2
α−1
2
i1
N
m1 i
i1
log2 q xm1,j | Xm,i · qxm1,j | Xm,i 1−α
jNm1 i−11 xm1,j ∈S
···Nm
n−1 N0
m0
EQ log2 q Xm1,j | Xm,i · qXm1,j | Xm,i −α | Xm,i
N
m1 i
2
sN
2
e−2
α−1
i−11 xm1,j s1
N
m1 i
jNm1
n 2
T − 1
2
e−2 · lim sup n N
e−2 < ∞.
α
−
1
T
n→∞
3.6
Therefore, 2.3 holds naturally. By 2.18 and 3.6 we have
···Nm
n−1 N0
1 lim sup n T n→∞
m0
≤ Nλ2
i1
2e−2
α − 12
N
m1 i
−λ log q Xm1,j | Xm,i − EQ log q Xm1,j | Xm,i | Xm,i
jNm1 i−11
c,
μ-a.s. ω ∈ Dc.
3.7
12
Journal of Inequalities and Applications
In the case of 0 < λ < α, by 3.7 we have
···Nm
n−1 N0
1 lim sup n T n→∞
m0
≤ Nλ
i1
2e−2
2
α − 1
N
m1 i
− log q Xm1,j | Xm,i − EQ log q Xm1,j | Xm,i | Xm,i
jNm1 i−11
c
,
λ
μ-a.s. ω ∈ Dc.
3.8
2
2
Let gλ 2λNe−2 /α − 12 c/λ,
in the case 0 < c ≤ 2Nα /1 − αe, then it is
obvious
√ gλ attains, at λ 1 − αe c/2N, its smallest value g1 − αe c/2N 2e−1 2cN/1 − α on the interval 0, α. That is
···Nm
n−1 N0
1 lim sup n T n→∞
m0
≤
i1
N
m1 i
− log q Xm1,j | Xm,i − EQ log q Xm1,j | Xm,i | Xm,i
jNm1 i−11
2e−1 √
2cN,
1 − α
μ-a.s. ω ∈ Dc.
3.9
By the similar means of reasoning 2.21, it can be concluded that 3.9 also holds when
c 0. According to the methods of proving 2.4, 3.1 follows from 1.5, 2.23, and 3.9.
Similarly, when −α < λ < 0, 0 ≤ c ≤ 2Nα2 /1 − αe2 , by 3.7 we have
···Nm
N
n−1 N0
m1 i
1 − log q Xm1,j | Xm,i − EQ log q Xm1,j | Xm,i | Xm,i
lim inf n n→∞ T
m0 i1 jNm1 i−11
≥−
2e−1 √
2cN,
1 − α
μ-a.s. ω ∈ Dc.
3.10
Imitating the proof of 2.5, 3.2 follows from 1.5, 1.10, 2.2, and 3.10.
Corollary 3.2 see 9. Let X {Xt , t ∈ T} be the Markov chains field determined by the measure
Q
μQ on the generalized Bethe tree T · fn ω is defined as 1.6, and Hm Xm1,j | Xm,i is defined as
2.1. Then
⎧
⎫
···Nm
N
n−1 N0
⎨
⎬
m1 i
1
Q
0. μQ -a.s.
lim fn ω − n Hm Xm1,j | Xm,i
T n → ∞⎩
⎭
m0 i1 jNm1 i−11
3.11
Journal of Inequalities and Applications
13
Proof. By 3.1 and 3.2 in Corollary 3.1, we obtain that when c 0,
⎧
⎨
lim
n → ∞⎩
1
fn ω − n T ···Nm
n−1 N0
m0
i1
N
m1 i
Q
Hm Xm1,j
jNm1 i−11
⎫
⎬
| Xm,i
0,
⎭
μ-a.s. ω ∈ D0.
3.12
Set μ ≡ μQ , then ϕμ | μQ ≡ 0. It implies 2.2 always holds when c 0. Therefore D0 Ω
holds. Equation 3.11 follows from 3.12.
Corollary 3.3. Under the assumption of Corollary 3.1, if μ μQ , then
⎧
⎨
lim
n → ∞⎩
1
···Nm
n−1 N0
fn ω − n T m0
i1
N
m1 i
Q
Hm Xm1,j
jNm1 i−11
⎫
⎬
0. μ-a.s.
| Xm,i
⎭
3.13
Proof. It can be obtained that ϕμ | μQ ≡ 0, μ a.s. holds if μ μQ see Gray 1990 13,
therefore μD0 1. Equation 3.13 follows from 3.12.
Let X {Xt , t ∈ T} be a Markov chains field on the generalized Bethe tree with the
initial distribution and the joint distribution with respect to the measure μP as follows:
μP x0,1 px0,1 ,
n−1 N
n 0 ···Nm
μP xT
px0,1 m0
i1
N
m1 i
3.14
p xm1,j | xm,i ,
n ≥ 1,
jNm1 i−11
3.15
where P pj | i is a strictly positive stochastic matrix on S, p ps0 , ps1 , ps2 . . . is a
strictly positive distribution. Therefore, the entropy density of X {Xt , t ∈ T} with respect to
the measure μP is
⎡
1
fn ω − n ⎣log pX0,1 T ···Nm
n−1 N0
m0
i1
N
m1 i
⎤
log p Xm1,j | Xm,i ⎦.
3.16
jNm1 i−11
Let the initial distribution and joint distribution of X {Xt , t ∈ T} with respect to the measure
μQ be defined as 1.4 and 1.5, respectively.
We have the following conclusion.
Corollary 3.4. Let X {Xt , t ∈ T} be a Markov chains field on the generalized Bethe tree T whose
initial distribution and joint distribution with respect to the measure μP and μQ are defined by 3.14,
3.15 and 1.4, 1.5, respectively. fn ω is defined as 3.16. If
pl | h − ql | h ≤ c,
ql | h
h∈S l∈S
3.17
14
Journal of Inequalities and Applications
then
⎡
1
lim sup⎣fn ω − n T n→∞
···Nm
n−1 N0
m0
i1
N
m1 i
Q
Hm Xm1,j
jNm1 i−11
⎤
| Xm,i ⎦
2e √
≤
2cN, μP -a.s.
1−α
⎤
⎡
···Nm
N
n−1 N0
m1 i
1
Q
lim inf⎣fn ω − n Hm Xm1,j | Xm,i ⎦
T n→∞
m0 i1 jNm1 i−11
3.18
−1
≥−
−1
2e
1−α
√
2cN − c.
3.19
μP -a.s,
Proof. Let μ μP in Corollary 3.1, and by 1.5, 3.15 we get 3.16. By the inequalities log x ≤
x − 1 x > 0, a ≤ a , 3.17, and 1.10, we obtain
ϕ μP | μQ
n μP X T
lim sup n log
T μQ X T n
n→∞
1
1
pX0,1 lim sup n log
T n→∞
qX0,1 n−1
m0
N0 ···Nm
i1
n−1
m0
N0 ···Nm
i1
Nm1 i
p Xm1,j
jNm1 i−11
Nm1 i
q Xm1,j
jNm1 i−11
| Xm,i
| Xm,i
···Nm
N
n−1 N0
m1 i
p Xm1,j | Xm,i
pX0,1 1 ≤ lim sup n log
log lim sup n T T qX0,1 q Xm1,j | Xm,i
n→∞
n→∞
m0 i1 jNm1 i−11
1
···Nm
n−1 N0
1 ≤ lim sup n T n→∞
m0
i1
N
m1 i
jNm1
pl | h
δh Xm,i δl Xm1,j log
ql | h
i−11 h∈S l∈S
!
"
···Nm
N
n−1 N0
m1 i
pl | h
1 ≤ lim sup n −1
δh Xm,i δl Xm1,j
T ql | h
n→∞
m0 i1 jNm1 i−11 h∈S l∈S
n T − 1 pl | h − ql | h
≤
lim sup n T ql | h
h∈S l∈S n → ∞
≤
pl | h − ql | h
h∈S l∈S
ql | h
.
3.20
By 3.17 and 3.20 we have
ϕ μP | μQ ≤ c,
a.s.
3.21
It follows from 2.2 and 3.21 that Dc Ω; therefore 3.18, 3.19 follow from 3.1,
3.2.
Journal of Inequalities and Applications
15
4. Some Limit Properties for Expectation of Random Conditional
Entropy on the Finite State Space
n
Lemma 4.1 see 8. Let X T {Xt , t ∈ T n } be a Markov chains field defined on a Bethe tree
n
TB,N , Sn k, ω be the number of k in the set of random variables X T {Xt , t ∈ T n }. then for all
k ∈ S,
Sn k, ω
lim n πk μQ -a.s,
T n
4.1
where π π1, . . . , πN is the stationary distribution determined by Q.
n
Theorem 4.2. Let X T {Xt , t ∈ T n } be a Markov chains field defined on a Bethe tree TB,N , and
Q
let Hm Xm1,j | Xm,i be defined as above. Then
⎡
1
lim n ⎣
n T
N1
i1
Q
H0 X1,i | X0,1 n−1 N1N
m1
i1
−
πkql | k log ql | k,
m−1
Ni
Q
Hm Xm1,j
⎤
| Xm,i ⎦
jNi−11
μQ -a.s.
k∈S l∈S
Proof. Noticing now N1 N 1, for all n ≥ 2, Nn N, that therefore we have
N1
i1
Q
H0 X1,i | X0,1 N1
n−1 N1N
m1
i1
m−1
Ni
Q
Hm Xm1,j | Xm,i
jNi−11
− EQ log qX1,i | X0,1 | X0,1
i1
n−1 N1N
m1
N1
−
m−1
i1
Ni
− EQ log q Xm1,j | Xm,i | Xm,i
jNi−11
qx1,i | X0,1 log qx1,i | X0,1 i1 x1,i ∈S
−
n−1 N1N
m1
i1
m−1
Ni
q xm1,j | Xm,i log q xm1,j | Xm,i
jNi−11 xm1,j ∈S
4.2
16
Journal of Inequalities and Applications
N1
−
δk X0,1 ql | k log ql | k
i1 k∈S l∈S
−
n−1 N1N
m1
i1
m−1
Ni
δk Xm,i ql | k log ql | k
jNi−11 k∈S l∈S
⎡
m−1
N1
n−1 N1N
⎣
−
ql | k log ql | k
δk X0,1 m0
i1
k∈S l∈S
i1
⎤
Ni
δk Xm,i ⎦
jNi−11
−
ql | k log ql | kNSn−1 k, ω δk X0,1 .
k∈S l∈S
4.3
Noticing that limn → ∞ |T n |/|T n−1 | N, by 4.3 we have
⎡
1
lim n ⎣
n T
N1
i1
Q
H0 X1,i
| X0,1 n−1 N1N
m1
i1
m−1
Ni
Q
Hm Xm1,j
jNi−11
⎤
| Xm,i ⎦
1 −lim n ql | k log ql | kNSn−1 k, ω δk X0,1 n T
k∈S l∈S
4.4
1
−
ql | k log ql | klim n−1 Sn−1 k, ω
n T
k∈S l∈S
−
πkql | k log ql | k.
k∈S l∈S
Equation4.2 follows from 4.4.
n
Theorem 4.3. Let X T {Xt , t ∈ T n } be a Markov chains field defined on a Bethe tree TB,N ,
Q
Hm Xm1,j | Xm,i defined as above. Then
⎫
⎧
m−1
N1
n−1 N1N
Ni
⎬
⎨
1
Q
Q
lim n EQ H0 X1,i | X0,1 EQ Hm Xm1,j | Xm,i
⎩
n T
⎭
i1
m1
i1
jNi−11
−
qkql | k log ql | k.
k∈S l∈S
μQ -a.s.
4.5
Journal of Inequalities and Applications
17
Q
Proof. By the definition of Hm Xm1,j | Xm,i and properties of conditional expectation, we
have
N1
i1
Q
EQ H0 X1,i
N1
| X0,1 n−1 N1N
m1
m−1
i1
Ni
Q
EQ Hm Xm1,j | Xm,i
jNi−1 1
− EQ EQ log qX1,i | X0,1 | X0,1
i1
n−1 N1N
m1
N1
m−1
i1
Ni
− EQ EQ log q Xm1,j | Xm,i | Xm,i
jNi−11
− EQ log qX1,i | X0,1 i1
n−1 N1N
m1
N1
−
m−1
i1
Ni
− EQ log q Xm1,j | Xm,i
jNi−11
4.6
qx0,1 , x1,i log qx1,i | x0,1 i1 x0,1 ∈S x1,i ∈S
−
n−1 N1N
m1
m−1
i1
Ni
q xm,i , xm1,j log q xm1,j | xm,i
jNi−11 xm,i ∈S xm1,j ∈S
N1
qkql | k log ql | k
−
i1 k∈S l∈S
−
n−1 N1N
m1
m−1
i1
Ni
qkql | k log ql | k
jNi−11 k∈S l∈S
−
qkql | k log ql | k T n − 1 .
k∈S l∈S
Accordingly we have by 4.6
⎫
⎧
m−1
n−1 N1N
Ni
⎨N1
Q
⎬
Q
EQ H0 X1,i | X0,1 EQ Hm Xm1,j | Xm,i
lim n ⎩
n T
⎭
i1
m1
i1
jNi−11
1
n T − 1
−
qkql | k log ql | klim n T n
k∈S l∈S
qkql | k log ql | k,
−
k∈S l∈S
Therefore 4.5 also holds.
μQ -a.s.
4.7
18
Journal of Inequalities and Applications
Acknowledgments
The work is supported by the Project of Higher Schools’ Natural Science Basic Research of
Jiangsu Province of China 09KJD110002. The author would like to thank the referee for his
valuable suggestions. Correspondence author: K. Wang, main research interest is strong limit
theory in probability theory. D. Zong main research interest is intelligent algorithm.
References
1 K. L. Chung, A Course in Probability Theory, Academic Press, New York, NY, USA, 2nd edition, 1974.
2 L. Wen and Y. Weiguo, “An extension of Shannon-McMillan theorem and some limit properties for
nonhomogeneous Markov chains,” Stochastic Processes and their Applications, vol. 61, no. 1, pp. 129–
145, 1996.
3 W. Liu and W. Yang, “Some extensions of Shannon-McMillan theorem,” Journal of Combinatorics,
Information & System Sciences, vol. 21, no. 3-4, pp. 211–223, 1996.
4 Z. Ye and T. Berger, Information Measures for Discrete Random Fields, Science Press, Beijing, China, 1998.
5 T. Berger and Z. X. Ye, “Entropic aspects of random fields on trees,” IEEE Transactions on Information
Theory, vol. 36, no. 5, pp. 1006–1018, 1990.
6 Z. Ye and T. Berger, “Ergodicity, regularity and asymptotic equipartition property of random fields
on trees,” Journal of Combinatorics, Information & System Sciences, vol. 21, no. 2, pp. 157–184, 1996.
7 W. Yang, “Some limit properties for Markov chains indexed by a homogeneous tree,” Statistics &
Probability Letters, vol. 65, no. 3, pp. 241–250, 2003.
8 W. Yang and W. Liu, “Strong law of large numbers and Shannon-McMillan theorem for Markov chain
fields on trees,” IEEE Transactions on Information Theory, vol. 48, no. 1, pp. 313–318, 2002.
9 W. Yang and Z. Ye, “The asymptotic equipartition property for nonhomogeneous Markov chains
indexed by a homogeneous tree,” IEEE Transactions on Information Theory, vol. 53, no. 9, pp. 3275–
3280, 2007.
10 Z. Shi and W. Yang, “Some limit properties of random transition probability for second-order
nonhomogeneous markov chains indexed by a tree,” Journal of Inequalities and Applications, vol. 2009,
Article ID 503203, 2009.
11 W. Liu and W. Yang, “Some strong limit theorems for Markov chain fields on trees,” Probability in the
Engineering and Informational Sciences, vol. 18, no. 3, pp. 411–422, 2004.
12 J. L. Doob, Stochastic Processes, John Wiley & Sons, New York, NY, USA, 1953.
13 R. M. Gray, Entropy and Information Theory, Springer, New York, NY, USA, 1990.