The RG-Factorizations
in Stochastic Models
Dr. Quan-Lin Li
Department of Industrial Engineering
Tsinghua University
Beijing 100084, P.R. China
Outline of this talk

Why to need the RG-factorizations

How to construct the RG-factorizations

How to apply the RG-factorizations

Promising issues in the future
Why to need

From 1996 to 2000, my research focuses on quasistationary distributions of stochastic models
Our main problem is described as follows:
          P 
 Discrete 
P is a transition probability matrix
          Q 
Continuous 
Q is an infinitesimal generator
  e 1
Our Question: How to compute?
Why to need
When the size of the matrix P is finite, this
computation is similar to that for the stationary
probability vectors of the finite-state Markov
chains by using systems of linear equations
 -classification of state;
solving    ;  is the convengence radious
     0 for all the other cases
Why to need
When the size of the matrix P is infinite, this
computation will become different and difficult
Need to consider the existence
Need to consider the uniqueness:
There are over one quasi-stationary distributions
No available expression
No effective approach
Why to need

Our work from 1996 to 1999 was to
develop the LU-block-decomposition
for Markov chains of M/G/1 type
and GI/M/1 type

Our method is different from
that used by Bean, Latouche,
Taylor etc.
Why to need
 B1

 B2
P   B3

 B4


 B1

 B0
P












B0
A1
A0
A2
A3
A1
A2
A0
A1 A0
B2
B3
B4
A1
A0
A2
A1
A0
A3
A2
A1








GI/M/1 type
M/G/1 type
Why to need
The LU-block-decomposition is
I   P  L   U   
 Q  L   U   
 Discrete 
 Continuous 
Our computation is given by
    L   U     0
Let x      L    . Then xU     0
Based on this, we can give a solution    
Our Question: Such a solution is OK?
Why to need
For a special Markov chain, we
obtained two different LU-blockdecompositions, which lead to two
different expressions
 One of them is correct and is the
same as that in the literature; while
another is wrong

Why?
Why to need
 We
analyzed many real examples
and then found the main reasons
 These computations motivate us to
LU-block-decomposition
to the RG-factorization
extend the
Why to need
For an arbitary irreducible Markov chain, the RG-factorization
is given by
I   P   I  RU      I   D      I  GL      Discrete 
 Q   I  RU      D     I  GL    
 Continuous 
Two different LU-decompositions are
L      I  RU      I   D     ,
L     I  RU    ,
U     I  GL    ;
U      I   D      I  GL    
Our computation is given by
     I  RU      I   D     I  GL     0
Why to need
For this computation
     I  RU      I   D      I  GL      0
How to take the vector
?
      I  RU    
x
     I  RU      I   D    
A key observation:
?
When P is  -positive recurrent with  = , we should use
x       I  RU    
All the other cases, we should use
x       I  RU      I   D    
Our Comparisons
Utility of the RG-factorization is
related to the classification of
state by means of the diagonal
matrix, and keep effective
computations
 Better than LU-decomposition

How to construct
Consider a discrete-time Markov chain
P0,1
 P0,0

P1,0
P1,1

P


 PN ,0 PN ,1
where N   or N  
Let E  0,
P0, N 

P1, N 


PN , N 
, n and E c  n  1,
P E
Ec
E
Ec
T

V
U

W
, N  . Then
How to construct
We can have two types of censored Markov chains:
UL-type: To the level set E


 n
k 
P  T  U  W V
 k 0

which leads to the UL-type RG-factorization
LU-type: To the level set E c
P
 n
 W V I T  U
1
which yields the LU-type RG-factorization
The UL-type RG-factorization
We write
 n
 0,0
  n
 1,0
 n
P


   n
 n ,0
We define
U-measure:
n
0,1
n
1,1
n ,1n 
0, nn 

1,n 


n ,nn 
 n
 n  n , n , n  0,
n
 j
I   
1
R-measure:
Ri , j  i , j
G-measure:
Gi , j   I   i  i, j , 0  j  i.
j
1
i
, 0  i  j,
The UL-type RG-factorization
For an abitrary irreducible Markov chain P, the UL-type
RG-factorization is given by
I  P   I  RU  I   D  I  GL  ,
where
 0 R0,1 R0,2 R0,3

0
R1,2 R1,3

RU 

0
R2,3


 D  diag   0 , 1 ,  2 ,  3 ,
 0

 G1,0
GL   G2,0

 G3,0


0
G2,1
G3,1
0
G3,2
0















The UL-type RG-factorization
Important Properties for Censoring Structure:
 0 is irreducible if P is irreducible;
 0 is positive recurrent if P is recurrent;
 0 is transient if P is transient;
 k is transient for all k  1.
Some special cases
The QBD processes
 0 R0,1

0
R1,2
RU  

0


The M/G/1 type
R2,3
 0

G
GL   1,0



The GI/M/1 type
0

RU  




 0


 , G   G1,0
 L 










0
G2,1
0
R0,1
0
R1,2
0
R2,3






0
G2,1
0






Some special cases
The GI/G/1 type
 0 R0,1 R0,2 R0,3

0
R1
R2

RU 

0
R1


 D  diag   0 , , , ,
 0

 G1,0
GL   G2,0

 G3,0


0
G1
G2
0
G1 0















How to apply
  I  RU  I   D  I  GD   0
Let x    I  RU  . Then x  I   D  I  GL   0.
Observating a non-zero nonnegative solution
x   x0 , 0, 0, 0,
,
where x0 is the stationary probability vector of  0
Therefore,  =  x0 , 0, 0, 0,
 I  RU 
 0   x0 ,

k 1

 k    i Rk i , k  1.
i 0

1
yields
Remarks
Computing the stationary probability vector of the
Markov chain P with a huge state space or an infinite
state space is decomposited into two steps:
Step one: Computing the stationary probability
vector of the censored chain  0 with a smaller
state space
Step two: Computing the R-measure Ri , j for 0  i  j
by using the above iterative relations.
A crucial advance
Infinite states
Finite states
Huge
The UL-type RG-factorization
Finite states
Smaller
The LU-type RG-factorization
We write
 n n,n
n n,n1 n n,n 2
  n
 n
n




n
n 1, n 1
n 1, n  2
P    n 1,n
n
n
n



 n  2,n
n  2, n 1
n  2, n  2


We define







U-measure:
 n  n n,n , n  0,
R-measure:
R i , j  i , j  I   j  , 0  j  i ,
G-measure:
G i , j   I   i   i , j , 0  i  j.
1
 j
1
i 
The LU-type RG-factorization
For an abitrary irreducible Markov chain P, the LU-type
RG-factorization is given by
I  P   I  R L  I   D  I  GU  ,
where
 0

0
 R1,0
R L   R 2,0 R 2,1
0

 R 3,0 R 3,1 R 3,2 0


 D  diag   0 , 1 ,  2 ,  3 ,
GU
0





G 0,1
G 0,2
G 0,3
0
G1,2
0
G1,3
G 2,3















The LU-type RG-factorization
 k is transient for all k  0.
The matrix I  P or Q of size  must be invertible,
 I  P
 I  D   I  RL 
1
1
Q 1   I  GU   D 1  I  R L 
1
  I  GU 
1
1
An example,
1

2

1
P2

1
2


1
 
2
2
1
 
2
3
1
 
2
2
1
 
2
3
1
 
2
2
1
 
2
3











1
Some special cases
The QBD processes
 0

R
0
RU   1,0

R 2,1


The M/G/1 type
0

 0 G 0,1


0 G1,2
,G  
 L 
0




 0 G 0,1

0 G1,2
GL  

0


The GI/M/1 type
 0

R
RU   1,0



0
R 2,1
0


















How to apply
The LU-type RG-factorization is different from the UL-type
case. It may be used to deal with the first passage times and
the sojourn times. In addition, we provide a better example:
Consider a perturbed Markov chain P  P  . Let   and
 be the stationary probability vectors of P and P, respectively.
Then
      P   
d
  | 0  I  P   
d
which leads to
d
1
1
1
  | 0    I  GU   I   D   I  R L 
d
Comparison for UL- and LU-type
Systems of linear equations
xA  0
or
Ax  0
Systems of linear equations
xA  b (b  0)
or
Ax  b (b  0)
UL-type RG-factorization
LU-type RG-factorization
Our work on the RG-factorizations
Theory
Applications
Promising problems (1)
 For the RG-factorizations:
1. It is interesting to consider the d-period for
the R-, U- and G-measures. For example
(1) A = A0 + A1 + A2 is irreducible and is d-period,
the two matrices R and G are d-period
?
(2) For a Markov chain of GI/G/1 type, what
happen to
?
Such a work is useful for tailed analysis
Discrete time

R   R Ak ,
k
k 0

G   Ak G
k
k 0
Continuous time

R
k 0
k
Ak  0,

AG
k 0
k
k
0
Promising problems (2)
 For the RG-factorizations:
1. It is interesting to consider spectral analysis
for the R-, U- and G-measures.
When A = A0 + A1 + A2 is irreducible and is
infinite size, how to analyze the spectral of
the two matrices R and G
?
2. For a Markov chain of GI/G/1 type, what
happen to
?
Promising problems (3)
To construct the RG-factorization, we have formed
many useful relations such as Winner-Holp equations
Ri , j  I   j   Pi , j 

 R  I   G
k  j 1

 I   i  Gi , j  Pi , j  
k  j 1
 n  Pn ,n 
i ,k
k
k, j
Ri ,k  I   k Gk , j

 R  I   G
k  n 1
n ,k
k
k ,n
Effective algorithms are necessary to compute the
R-, U- and G-measures, and then compute performance
measures of a stochastic models.
Promising problems (4)
Transient Performance:
Continuous-time Markov chain: Q or Q  t 
d
 t    t  Q
dt
  t     0  exp Qt
d
 t    t  Q t 
dt
  t     0  exp
 Q u  du
t
0
Continuous-time Markov reward process: Q, f  X t  or R
  t    f  X u du, H i  t , x   P   t   x, X t  i ,   t , x 
t
0


 t, x  
 t, x  R   t, x  Q
t
x
Train repairable networks
GM manufacturer
GM manufacturer
Container Park
Part Supplier
GM manufacturer
Container Park
Part Supplier
Part Supplier
GM manufacturer
Real-time management
Local
Optimization
Local
Optimization
Information
Theory
Queueing
Networks
Global Optimization
Thanks for you
and
questions ?