Slide set 19
Stat 330 (Spring 2015)
Last update: February 16, 2015
Stat 330 (Spring 2015): slide set 19
Birth and Death Processes
Review: What is a stochastic process? What is a Poisson Process?
Motivation: Birth and Death process (B + D) is a generalization of Poisson
process, and it provides for modeling of queues, i.e. we assume that arrivals
stay some time in the system and leave after that.
More motivation: The concept of memoryless property is further formalized
by the Markov property.
Definition:A B+D process X(t) is a stochastic process that monitors the
number of people in a system.
Remarks:
1. X(t) = k, implies that at time t there are k people/objects in the
system.
2. X(t) is still called the state at time t. X(t) is in {0, 1, 2, . . .} for all t.
1
Stat 330 (Spring 2015): slide set 19
B+D Process: Examples
Visualization:
♥ One can visualize the set-up for a B+D process in a state diagram as
movements between consecutive states.
♥ Conditional on X(t) = k, we either move to state k + 1 or to k − 1,
depending on whether a birth or a death occurs first.
♠ This process sometimes is referred as Random walk, used more in material
science, physics, chemistry and biology.
Figure 1: State diagram of B+D process
2
Stat 330 (Spring 2015): slide set 19
B+D Process: Examples (Cont’d)
Stat Printer: The ”heavy-duty” printer in the Stats department gets 3 jobs
per hour. On average, it takes 15 min to complete printing. The printer
queue is monitored for a day (8h total time).
1. Jobs arrive at the following points in time (in h):
job i
arrival time
1
0.10
2
0.40
3
0.78
4
1.06
5
1.36
6
1.84
7
1.87
8
2.04
9
3.10
10
4.42
job i
arrival time
11
4.46
12
4.66
13
4.68
14
4.89
15
5.01
16
5.56
17
5.56
18
5.85
19
6.32
20
6.99
2. The printer finishes jobs at:
job i
finishing time
1
0.22
2
0.63
3
1.61
4
1.71
5
1.76
6
1.90
7
2.32
8
2.68
9
3.42
10
4.67
job i
finishing time
11
5.31
12
5.54
13
5.59
14
5.62
15
5.84
16
6.04
17
6.83
18
7.10
19
7.23
20
7.39
3
Stat 330 (Spring 2015): slide set 19
Stat Printer (Cont’d)
♣ Let X(t) be the number of jobs in the printer and its queue at time t.
X(t) is a Birth & Death process.
1. Can you draw a graph of X(t) for the value monitored?
4
Stat 330 (Spring 2015): slide set 19
Stat Printer (Cont’d)
2. What is the (empirical) probability that there are 5 jobs in the printer
and its queue at some time t?
The empirical probability for 5 jobs in the printer is the time, X(t) is in
state 5 divided by the total time:
(5.31 − 5.01) + (5.59 − 5.56) 0.33
\
P (X(t) = 5) =
=
= 0.04125.
8
8
5
Stat 330 (Spring 2015): slide set 19
Model:
Modeling B & D Processes
1. The model for a birth or a death is given, conditional on X(t) = k, as:
B = time till a potential birth ∼ Exp(λk )
N.B. (P (B = D) = 0!)
D = time till a potential death ∼ Exp(µk )
B < D the move is to state k + 1 at time t + B
B > D the move is to state k − 1 at time t + D
3. B and D are independent for each state k.
2. if
♣ This implies, that, given the process is in state k, the probability of
moving to
state k + 1
state k − 1
λk
µ k + λk
µk
.
is
µ k + λk
is
6
Stat 330 (Spring 2015): slide set 19
Modeling (Cont’d)
♣ Notice that Y = min(B, D) is the time the system will be in state k
until the move.
♦ What can we say about the distribution of Y := min(B, D)?
P (Y ≤ y) = P (min(B, D) ≤ y) = P (B ≤ y ∪ D ≤ y)
= P (B ≤ y) + P (D ≤ y) − P (B ≤ y ∩ D ≤ y)
= P (B ≤ y) + P (D ≤ y) − P (B ≤ y) · P (D ≤ y)
= 1 − e−λk y + 1 − e−µk y − (1 − e−λk y )(1 − e−µk y ) =
= 1 − e(λk +µk )y = Expλk +µk (y),
♦ We talked about this discussing the Poisson process:
exponential races!
this is the
♣ Y itself is, again, an exponential variable, its rate is the the sum of the
rates of B and D.
7
Stat 330 (Spring 2015): slide set 19
Remarks:
• Knowing the distribution of Y , the staying time in state k, gives us, e.g.
the possibility to compute the mean staying time in state k.
• The mean staying time in state k is the expected value of an exponential
distribution with rate λk + µk .
• ‡ The mean staying time therefore is 1/(λk + µk ).
• N.E. A Poisson process with rate λ is a special case of a Birth & Death
process, where the birth rates and death rates are constant, λk = λ and
µk = 0 for all k.
• The analysis of this model for small t is mathematically difficult because
of “start-up” effects - but in some cases, we can compute the “large t”
behaviour.
• A lot depends on the ratio of births and deaths:
8
Stat 330 (Spring 2015): slide set 19
Examples:
Examples:
♦ N.E.
N.E. In
In the
the picture,
picture, three
three different
different simulations
simulations of
of Birth
Birth &
& Death
Death processes
processes
♦
are shown.
shown. Only
Only in
in the
the first
first case,
case, the
the process
process isis stable
stable (birth
(birth rate
rate <
< death
death
are
rate). The
The other
other two
two processes
processes are
are unstable
unstable (birth
(birth rate
rate =
= death
death rate
rate (2nd
(2nd
rate).
process) and
and birth
birth rate
rate >
> death
death rate
rate (3rd
(3rd process)).
process)).
process)
99
Stat 330 (Spring 2015): slide set 19
More on stability:
Equilibrium: Only if the B+D process is stable, it will reach an equilibrium
after some time - this is called the steady state of the B+D process.
♦ Mathematically, the notion of a steady state translates to
lim P (X(t) = k) = pk for all k,
t→∞
where the pk are numbers between 0 and 1, with
P
k pk
= 1.
♣ The pk probabilities are called the steady state probabilities of the B+D
process, and they form a probability mass function for X.
♥ How to compute pk ?
10
Stat 330 (Spring 2015): slide set 19
Sketched arguments:
time in state k until time t
→
total time t
mean stay
# of visits to
state k by time t
in state k
total time t
# of visits to
state k by time t
pk
→
pk
→
pk (λk + µk )
use (‡)
total time t
The long rate of visits to state k is pk (λk + µk ); but we know that a fraction
k
of λ λ+µ
visits to state k results in moves to state k + 1. So
k
k
λk
λk +µk pk
· (λk + µk ) = λk pk
is the long run rate of transitions from k → k + 1.
Similarly, µk pk is the long run rate of transitions from k → k − 1.
11