Math 554
Introduction to Stochastic Processes
Instructor: Alex Roitershtein
Iowa State University
Department of Mathematics
Fall 2015
Solutions to Homework #6
3.1
To answer the questions, we will occasionally use properties 1-3 of Poisson processes listed
on p. 65. We will refer to them as P1 (independence of disjoint time intervals), P2 (constant
average rate of arrival), and P3 (customers arrive one at time), respectively.
(a) P(X1 < 2) = P(X1 = 0) + P(X1 = 1) = 5e−4 .
(b) We have:
P1
P1
P(X2 − X1 ≥ 2|X1 = 6) = P(X2 − X1 ≥ 2) = P(X1 ≥ 2)
= 1 − P(X1 = 0) − P(X1 = 1) = 5e−4 .
(c) Let Tk = inf{t > 0 : Xt = k}. By the Markov property and P2, the random time
intervals Tk − Tk−1 are i.i.d. Hence, E(T15 ) = 15E(T1 ) = 15/4.
(d) We have:
P(X1 = 5|X2 = 8)
P(X1 = 5, X2 − X1 = 3)
P(X2 = 8)
P(X1 = 5) · P(X2 − X1 = 3) P 2 P(X1 = 5) · P(X1 = 3)
=
P(X2 = 8)
P(X2 = 8)
5
3
7
(4 · 4 ) · 8!
= .
8
(4 · 2) · 5! · 3!
32
=
P1
and
=
P2
=
(e) Assuming that j ≤ k, we have:
P(X1 = j|X4 = k)
=
P1
and
=
P2
P2
=
=
3.3
1
P(X1 = j, X4 − X1 = k − j)
P(X4 = k)
P(X1 = j) · P(X4 − X1 = k − j)
P(X4 = k)
P(X1 = j) · P(X3 = k − j)
P(X4 = k)
k−j
j
(4 · (4 · 3)k−j ) · k!
k 3
=
.
(4 · 4)k · j! · (k − j)!
j 4k
(a) It is not hard to check that the defining properties 1-3 of Poisson processes listed on
p. 65 (i.e., independence of disjoint time intervals, constant average rate of arrival, and
customers arrive one at time) hold for Z. The rate of Z is λ = λ1 + λ2 , since the rate
represents the average number of arrival per a unit of time.
, Alternatively:
P (Zt = n) =
n
X
P (Xt = k)P (Yt = n − k) =
k=0
k=0
= e−(λ1 +λ2 )t
=
n
X
n
X
(λ1 t)k
k=0
n
−(λ1 +λ2 )t X
e
n!
k=0
k!
×
k
−λ1 t (λ1 t)
e
k!
×e
n−k
−λ2 t (λ2 t)
(n − k)!
(λ2 t)n−k
(n − k)!
(λ1 + λ2 )t
n!
(λ1 t)k (λ2 t)n−k = e−(λ1 +λ2 )t
k!(n − k)!
n!
n
(b) Time until the first arrival in a Poisson process with parameter λ is distributed Exp(λ).
The probability that the first customer arrives in the first process is the probability
that the first “exponential alarm clock” goes off first. This probability was computed
1
.
in the class, and is equal to λ1λ+λ
2
(c) Let Tx and Ty be the times when the first call arrives in process X and Y, respectively.
Then
FT (t) := P(T ≤ t) = P(Tx ≤ t) · P(Ty ≤ t) = (1 − e−λ1 t ) · (1 − e−λ2 t ).
Correspondingly, fT (t) =
dFT
dt
(t) = λ1 e−λ1 t + λ1 e−λ1 t − (λ1 + λ2 )e−(λ1 −λ2 )t .
T
3.5 Let pt = pt (1), pt (2) be the two dimensional vector with the components
pt (i) = P (Xt = i),
i = 1, 2.
The general solution for the forward equation is pt = x1 eλ1 t + x2 eλ2 t , where λ1 = 0 and
λ2 = −5 are the eigenvalues of A, and x1 = [4c1 , c1 ]T , bx2 = [−c2 , c2 ]T are the corresponding
left eigenvalues. In terms of c1 and c2 , we obtain p0 = [4c1 − c2 , c1 + c2 ]T . To this vector be
a probability vector, we must have
(4c1 − c2 ) + (c1 + c2 ) = 5c1 = 1,
and hence c1 = 1/5. That is,
pt =
1
· [4, 1]T + c[−1, 1]T e−5t ,
5
for some c2 = c ∈ R. Since pt ≥ 0 for any t ≥ 0, we must have −1/5 ≤ c ≤ 4/5.
To find the transition matrix Pt , first choose c = −1/5 so that p0 (1) = 1. In this case
1
T
1
pt =
[4, 1]T + [1, −1]T e−5t =
4 + e−5t , 1 − e−5t .
5
5
2
This vector gives the first row of the matrix Pt . Next, choose c = 4/5 so that p0 (2) = 1. In
this case
1
T
1
pt =
[4, 1]T + [−4, 4]T e−5t =
4 − 4e−5t , 1 + 4e−5t .
5
5
This vector gives the second row of the matrix Pt . More precisely,
1
4 + e−5t 1 − e−5t
pt =
5 4 − 4e−5t 1 + 4e−5t
3.11 The results stated as Facts on p.77 and p. 78 (see formula 3.11 and 3.13 on that pages)
show that the chain is recurrent but not positive recurrent in both the cases. To estimate
the sums, use for instance the fact that if λn = 1 + an , and limn→∞ an = 0 (an > −1 can be
positive or negative), then on one hand
n
Y
λk =
k=1
n
Y
(1 + ak ) ≤
k=1
n
Y
e
ak
n
X
= exp
ak ,
k=1
k=1
and on teh otehr hand,
n
Y
k=m
λk =
n
Y
(1 + ak ) ≥
n
Y
e
−2|ak |
n
X
|ak | ,
= exp −2
k=m
k=m
k=m
for m large enough (so that 1 + ak ≥ 1 − |ak | ≥ e−2|ak | for k ≥ m). Notice then that
n
X
n−1 ∼ ln n.
k=1
3.12 The results stated as Fact on p.77 (see formula 3.11 on that page) show that the chain
is transient if and only if µ < 1. If the chain is not transient, it is recurrent provided that zero
is not an absorbing but rather a “reflecting” state. Such a chain visits zero with probability
one. Hence the extinction is certain for the underlying process provided that µ ≥ λ.
3