LECTURE 18
Review
Markov Processes – III
• Assume
a single class of recurrent states,
aperiodic;
aperiodic.
Then,
plus transient
states. Then,
LECTURE 20
Review
• Assume a single class of recurrent states,
Markov Processes – III
(n)
=ππj j
lim rrijij(n
)=
lim
n→∞
n→∞
Readings: Section 7.4
Readings: Section 6.4
where
not depend
dependon
onthe
theinitial
initial
where π
πjj does
does not
conditions
conditions:
Lecture outline
j | jX|0X
=0)i)=
=ππjj
lim P(X = =
lim P(X
n n
n→∞
n→∞
Lecture outline
• Review of steady-state behavior
can
m can
•• ππ11,,......,, ππm
solution
to
the
solution of the
• Review of steady-state behavior
• Probability of blocked phone calls
• Probability of blocked phone calls
• Calculating absorption probabilities
πj =
!
k
Calculating
absorption
• • Calculating
expected
timeprobabilities
to absorption
!
The phone company problem
The phone company problem
0.8
0.8
• Calls originate as a Poisson process,
• Calls originate as a Poisson process,
rate
rate λλ
0.5
0.5
2
πk pkj
k
!πj = 1
j πj = 1
j
Example
1
πj =
together with
Example
1
πk pkj , ! j = 1, . . . , m,
together with
• Calculating expected time to absorption
0.5
0.5
be
be found
found as
as the
theunique
unique
balance
equations
balance equations
–– Each
duration isis exponentially
exponentially
Each call
call duration
distributed
(parameter
µ)
distributed (parameter µ)
2
0.2
0.2
–– B
B lines
lines available
available
π1 = 2/7, π2 = 5/7
π1 = 2/7, π2 = 5/7
Discrete time intervals
•• Discrete
intervals
of (small)
(small) length
length δδ
of
• Assume process starts at state 1.
• Assume process starts at state 1.
"#
• P(X1 = 1, and X100 = 1)=
• P(X1 = 1, and X100 = 1)=
0
i
i!1
1
B-1
iµ#
(X100
= 1 and X101 = 2)
• • PP
(X
100 = 1 and X101 = 2)
•• Balance
λπi−1
iµπ
Balance equations:
equations: λπ
i−1==iµπ
i i
λi
• πi = π0 λii
• πi = π0 µi i!
µ i!
1
B
i
!
B λ i
λ
π0 = 1/ !
i
π0 = 1/i=0 µ i!i
i=0 µ i!
B
Calculating absorption probabilities
Expected time to absorption
• What is the probability ai that:
process eventually settles in state 4,
given that the initial state is i?
1
1
3
44
0.5
0.4
54
0.5
0.6
0.2
3
0.3
0.8
44
0.5
0.4
0.6
• Find expected number of transitions µi,
until reaching the absorbing state,
given that the initial state is i?
0.2
2
1
0.8
For i = 4,
For i = 5,
ai =
ai =
ai =
!
µi = 0 for i =
pij aj ,
For all other i: µi = 1 +
for all other i
– unique solution
– unique solution
Mean first passage and recurrence
times
• Chain with one recurrent class;
fix s recurrent
• Mean first passage time from i to s:
ti = E[min{n ≥ 0 such that Xn = s} | X0 = i]
• t1, t2, . . . , tm are the unique solution to
ti = 1 +
!
pij tj ,
j
for all i =
% s
• Mean recurrence time of s:
t∗s = E[min{n ≥ 1 such that Xn = s} | X0 = s]
• t∗s = 1 +
!
j
j
ts = 0,
0.2
2
1
1
"
j psj tj
2
pij µj
MIT OpenCourseWare
http://ocw.mit.edu
6.041SC Probabilistic Systems Analysis and Applied Probability
Fall 2013
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
3