Math 554
Introduction to Stochastic Processes
Instructor: Alex Roitershtein
Iowa State University
Department of Mathematics
Fall 2015
Solutions to Homework #1
0.4
The corresponding characteristic equation is
λ=
1
1 + λ2 + λ3 .
3
Notice that λ = 1 is a solution. To find other two solutions, write
1
1
1 − 3λ + λ2 + λ3 = (λ − 1)(λ2 + aλ − 1).
3
3
Comparing the coefficients in front of λ2 on both the sides, we obtain a = 2. The roots of
λ2 + 2λ − 1 = 0
satisfy the equation (λ + 1)2 − 2 = 0. These roots are λ1,2 = −1 ±
Thus a general solution of the recursion is
√ n
√ n
f (n) = c1 + c2 −1 + 2 + c3 −1 − 2 .
√
2.
Since limn→∞ f (n) = 1, we have
f (n) = 1 + c2 −1 +
√ n
2 .
Finally, since f (0) = 0 we obtain
f (n) = 1 −
√
n
2−1 .
0.5
It is not hard to verify that f (n) = n2 is a solution. Also, if g(n) is a difference of two
solutions as defined in the textbook, then g satisfies the following homogeneous equation:
g(n) =
1
g(n + 1) + g(n − 1) .
2
The corresponding characteristic equation is
(λ − 1)2 = 0.
Thus (guess using the analogy with ODE and verify your guess),
g(n) = c1 + c2 n.
1
Thus a general solution for f is given by
f (n) = g(n) + c3 n2 = c1 + c2 n + n2 .
0.6 (a)
The corresponding characteristic equation is
λ2 + λ + 1 = 0.
Its roots are
√ 1
−1 ± −3 .
2
Thus the general solution of the ODE is given by
h
√3 √3 i
t + c2 sin
t .
f (x) = e−t/2 c1 cos
2
2
λ1,2 =
0.6 (b)
A general solution in the complex numbers is
√ n
√ n 1
f (n) = c1 λn1 + c2 λn2 = n c1 −1 − i 3 + c2 −1 + i 3 .
(1)
2
Indeed, it is easy to see that both f (n) = λn1 and f (n) = λn2 are solutions. Hence, since the
equation is linear, f (n) in (1) is also a solution for arbitrary c1 , c2 ∈ C. To conclude that (1)
gives the general form of the solution, observe that the sequence f (n) satisfying the recursive
equation is entirely determined by it initial values f (0) and f (1). Thus it is enough to check
that the system
f (0) = c1 + c2 ,
f (1) = c1 λ1 + c2 λ2
has a unique solution c1 , c2 given the initial (arbitrary complex) values f (0) and√f (1). This
is in fact the case since the system has a non-zero determinant 1 · λ2 − 1 · λ1 = i 3.
The complex solution given by (1) yields a real-valued sequence if and only if f (0) and
f (1) are both reals. The condition on f (0) implies that c1 + c2 is a real number, that is
c1,2 = c ± id
(2)
for some reals c, d. But then
√ n
√ n
c1 −1 − i 3 is the complex dual to c2 −1 + i 3
for all n ∈ N. Hence (1) and (2) give a general real-valued solution for the sequence f (n).
Using the Euler’s identity eiθ = cos θ + i sin θ, the solution can be alternatively written as
f (n) = (c + id)e−2πni/3 + (c − id)e2πni/3 = 2 c cos(2πn/3) + d sin(2πn/3)

if n ≡ 0 mod 3,
 2c
√
−c + d√3
if n ≡ 1 mod 3,
=

−c − d 3
if n ≡ 2 mod 3
2
Here c and d are two arbitrary real numbers.
1.1
Let Xn be the number of papers in the pile in the evening of day n. Then Xn is a Markov
chain with state space Ω = {0, 1, 2, 3, 4} and transition matrix


1/3 2/3 0
0
0
 1/3 0 2/3 0
0 


 1/3 0
0 2/3 0 


 1/3 0
0
0 2/3 
1
0
0
0
0
It is a matter of interpretation and solutions with Ω = {0, 1, 2, 3, 4, 5} will be fully credited.
1.8
(b) For a vertex V let XV be the expected number of steps until the walker starting at V
reaches A next time. We thus want to compute XA . Conditioning on the position of
the walker after the first step, we obtain:
1
XA = 1 + XB + X C + X D
3
1
XB = 1 + 0 + XC + XE
3
1
X C = 1 + 0 + XB
2
1
X D = 1 + 0 + XE
2
1
XE = 1 + XB + X D
2
We thus have:
XE
XD = 1 +
2
XB
XC = 1 +
,
2
and hence, plugging these expression into the initial system,
5 X B XE
+
+
3
2
6
4 X B XE
=
+
+
3
6
3
3 X B XE
=
+
+
2
2
4
XA =
XB
XE
The last two equations yield
5XB
6
3XE
4
4 XE
+
3
3
3 XB
=
+
.
2
2
=
3
Therefore,
XB =
36
,
11
XE =
46
,
11
XC =
29
,
11
XD =
34
,
11
XA = 4.
(d) For a vertex V let YV be the probability that the walker starting at V reaches A before
they reach C. We thus want to compute YB . Conditioning on the position of the walker
after the first step, we obtain:
1
1 + YE
3
1
YB + YD
=
2
1
=
1 + YE
2
YB =
YE
YD
We thus have:
1 YE
1
1 + YE = +
3
3
3
YB 1
YB 1 YE
1
+ 1 + YE =
+ +
.
=
YB + YD =
2
2
4
2
4
4
YB =
YE
and hence,
1 YE
+
3
3
= 2YB + 1.
YB =
3YE
The last two equations yield
YB =
1 YE
1 1
+
= + 2YB + 1 .
3
3
3 9
Therefore,
4
YB = .
7
(e) In the course of solving (b) we have found XC =
4
29
.
11