18.445 Introduction to Stochastic Processes
Lecture 24: Stationary distribution
Hao Wu
MIT
13 May 2015
Hao Wu (MIT)
18.445
13 May 2015
1/5
Recall : Suppose that X = (Xt )t≥0 is a continuous time Markov chain.
the jump times J0 , J1 , J2 , ... : J0 = 0, Jn+1 = inf{t > Jn : Xt 6= XJn }.
the jump chain Y0 , Y1 , Y2 , ... : Yn = XJn
Irreducible : X is irreducible if and only if Y is irreducible.
Recurrent : Suppose that X is an irreducible Markov chain. Then X is
recurrent if and only if Y is recurrent.
Today’s Goal :
positive recurrent
stationary distribution
summary on topics in the final
Hao Wu (MIT)
18.445
13 May 2015
2/5
Stationary
Suppose that X is a continuous time Markov chain with generator A,
and that Y is its jump chain.
Definition
A measure π is said to be stationary for X if πA = 0.
Lemma
If π is stationary, then πPt = π for all t ≥ 0.
Lemma
A measure π is stationary for X if and only if the measure µ, defined by
µ(x) = qx π(x), is stationary for Y .
Hao Wu (MIT)
18.445
13 May 2015
3/5
Positive recurrence
Definition
Define Tx+ = inf{t ≥ J1 : Xt = x}. A state x is positive recurrent if
Ex [Tx+ ] < ∞.
Theorem
Let X be an irreducible continuous time Markov chain with generator
A. The following statements are equivalent.
every state is positive recurrent.
some state is positive recurrent.
X has a stationary distribution.
When one of the statement is true, the stationary distribution is
1
.
π(x) =
qx Ex [Tx+ ]
Hao Wu (MIT)
18.445
13 May 2015
4/5
Topics covered in the final
Martingale : Suppose that X = (Xn )n≥0 is a martingale.
1
2
Optional Stopping Theorem
Martingale convergence theorems
If X is bounded in L1 , then Xn → X∞ a.s.
If X is bounded in Lp for p > 1, then Xn → X∞ a.s. and in Lp .
If X is UI, then Xn → X∞ a.s. and in L1 .
Poisson process : Suppose that (Nt )t≥0 is a Poisson process.
1
Definition, Markov property
2
Superposition
3
Characterization
Hao Wu (MIT)
18.445
13 May 2015
5/5
MIT OpenCourseWare
http://ocw.mit.edu
18.445 Introduction to Stochastic Processes
Spring 2015
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.