2DD50
Exam Mathematics 2 - Part A
Tuesday – April 7, 2020, 18:00–19:40
TU/e
Exam consists of three exercises that deal with SOR-part of the course. For the exercises you
can obtain the following number of points:
Exercise 1: 15 points, Exercise 2: 18 points, Exercise 3: 7 points.
—— ∗∗∗ ——
Exercise 1. Consider a discrete-time Markov chain with state space S = {A, B, C, D} and
transition matrix


0 12 0 12
 23 0 13 0 

P =
 0 1 0 1  .
2
2
1
2
0
0
3
3
At time 0 the Markov chain starts with probability 1 in state A.
(a) [6 points] Is the Markov chain periodic or aperiodic? Clearly explain your answer. Also
determine the long-run fraction of time the Markov chain spends in the states A, B, C
and D, respectively.
ANSWER (a) The Markov chain is periodic. You can only come back in a state in an even number of
steps. (2 points)
The occupancy distribution π̂ = (π̂A , π̂B , π̂C , π̂D ) can be found by solving the system of
equations
π̂A
π̂B
π̂C
π̂D
=
=
=
=
2
π̂
3 B
1
π̂
2 A
1
π̂
3 B
1
π̂
2 A
+ 13 π̂D ,
+ 12 π̂C ,
+ 23 π̂D ,
+ 12 π̂C ,
together with the normalization equation π̂A + π̂B + π̂C + π̂D = 1. (2 points for right
system of equations) The occupancy distribution is hence given by π̂ = ( 41 , 41 , 14 , 41 ). (2
points for right solution of right system of equations)
(b) [5 points] What is the probability that the Markov chain reaches state C before state B?
ANSWER (b) Define
qA = P (Markov chain reaches state C before state B starting from A),
qD = P (Markov chain reaches state C before state B starting from D).
Then, using first step analysis, we have qA = 21 qD and qD = 23 + 13 qA . (Three points for
right system of equations) The solution of this system of equations is given by qA = 25
and qD = 54 . Hence the answer of the question is qA = 25 . (Two points for right answer)
1
(c) [4 points] Now assume that a visit to state B takes twice as long as the visits to state
A, C and D. What is in this situation the long-run fraction of time the Markov chain
spends in the states A, B, C and D, respectively?
ANSWER (c) The answer should be π̂ = ( 51 , 25 , 15 , 15 ) (Three points for right answer). This could either be
explained in words: (i) twice as long in B, then in A, C and D; and (ii) sum of probabilities
should be equal to 1. Hence, ( 15 , 52 , 15 , 15 ). Alternative: Introduce Markov chain with five
states A, B1 , B2 , C and D, where B1 and B2 are the first half and the second half of a
visit to state B, respectively. This Markov chain has occupancy distribution ( 51 , 15 , 15 , 15 , 15 ).
(One point for right reasoning)
—— ∗∗∗ ——
Exercise 2. Jobs arrive at a single machine according to a Poisson process with a rate of 1 job
per hour. The machine can work at two different speeds: a low speed and a high speed. When
the machine is working at low speed, the service time of a job is exponentially distributed with
a mean of 2 hours. When the machine is working at high speed, the service time of a job is
exponentially distributed with a mean of 1 hour. Jobs that arrive when there are already 4
other jobs in the system are lost. The speed of the machine is regulated in the following way:
• The machine starts working at low speed when a job arrives while the system is empty.
• The machine continues to work at low speed until the number of jobs in the system
becomes equal to 3. At that instant the machine starts working at high speed.
• Whenever the machine is working at high speed and at a service completion the number
of jobs in the system is reduced to 1, the machine will start working at low speed again.
The state of the system is described by a continuous-time Markov chain. Here, on one hand
we describe the number of jobs in the system, and on the other hand we describe the speed of
the machine. In the sequel we denote with state (n, l) the state in which there are n jobs in the
system and the speed of the machine is low. With (n, h) we denote the state in which there
are n jobs in the system and the speed of the machine is high. Of course, when there are no
jobs in the system the speed of the machine equals 0. This state is denoted by (0, 0).
(a) [5 points] Give the rate matrix or rate diagram of the continuous-time Markov chain and
a system of equations from which the limiting probabilities for the different states can be
calculated.
ANSWER (a) With state space S
(time unit: hour)

0 1
 1 0
 2 1
 0
2
R=
 0 1

 0 0
0 0
= {(0, 0), (1, l), (2, l), (2, h), (3, h), (4, h)}, the rate matrix is given by
0
1
0
0
0
0
0
0
0
0
1
0
0
0
1
1
0
1
0
0
0
0
1
0








2
with corresponding rate diagram (Three points for right matrix or diagram).The system
of equations is given by
p(0, 0)
3
p(1, l)
2
3
p(2, l)
2
2p(2, h)
2p(3, h)
p(4, h)
=
=
=
=
=
=
1
p(1, l),
2
p(0, 0) + 12 p(2, l) + p(2, h),
p(1, l),
p(3, h),
p(2, l) + p(2, h) + p(4, h),
p(3, h),
with normalization equation p(0, 0) + p(1, l) + p(2, l) + p(2, h) + p(3, h) + p(4, h) = 1 (Two
points for right system of equations).
The limiting probabilities for the different states are given by:
p(0, 0) = 3/33, p(1, l) = 6/33, p(2, l) = 4/33, p(2, h) = 4/33, p(3, h) = 8/33, p(4, h) = 8/33.
(b) [8 points] Calculate the expected number of jobs produced per day (=24 hours) on the
machine.
What is the expected time that a job that is produced on the machine is in the system?
Which part of this expected time that a job is in the system is the job in service and
which part is the job waiting in the queue?
ANSWER (b) Expected number of jobs produced per day (=24 hours) on the machine (3 points):
24 · 1 · (1 − p(4, h)) =
600
33
≈ 18.2 jobs per day.
or alternative expression:
24 ·
1
2
· (p(1, l) + p(2, l)) + 1 · (p(2, h) + p(3, h) + p(4, h)) =
600
33
≈ 18.2 jobs per day.
Expected time that a job that is produced on the machine is in the system (3 points):
We have E(L) = λe · E[W e ] with E(L) =
(≈ 3.12 hour)
78
33
and λe =
600
33
so that E[W e ] =
78
600
day
Which part of this expected time that a job is in the system is the job in service and
which part is the job waiting in the queue (2 points)?
Similarly as above we have E(Lq ) = λe · E[W e,q ] with E(Lq ) =
48
E[W e ] = 600
day (≈ 1.92 hour).
And similarly as above we have ρ = λe ·E[B] with ρ =
day (= 1.2 hour).
30
33
and λe =
48
33
and λe =
600
33
600
33
so that
so that E[W e ] =
30
600
(c) [5 points] Calculate the expected duration of a period during which the speed of the
machine is uninterruptedly low and the expected duration of a period during which the
speed of the machine is uninterruptedly high respectively.
3
ANSWER (c) Expected duration of a period during which the speed of the machine is uninterruptedly
low: We have to determine m(1,l) (A), with A = {(0, 0), (2, h), (3, h), (4, h)}.
2
3
2
3
m(1,l) (A) =
m(2,l) (A) =
+ 23 m(2,l) (A),
+ 13 m(1,l) (A).
Substitution of the second equation in the first equation leads to
m(1,l) (A) =
10
9
+ 29 m(1,l) (A).
Solution: m(1,l) (A) =
10
7
hours.
Expected duration of a period during which the speed of the machine is uninterruptedly
high: We have to determine m(3,h) (A), with A = {(0, 0), (1, l), (2, l)}.
m(3,h) (A) = 21 + 12 m(4,h) (A) + 12 m(2,h) (A),
m(4,h) (A) = 1 + m(3,h) (A),
m(2,h) (A) = 21 + 12 m(3,h) (A).
Substitution of the second and third equation in the first equation leads to
m(3,h) (A) =
5
4
+ 34 m(3,h) (A).
Solution: m(3,h) (A) = 5 hours.
(1 point for use of first passage-time approach)
(2 points for right equations and calculations expected duration low speed period)
(2 points for right equations and calculations expected duration high speed period)
—— ∗∗∗ ——
Exercise 3. At a single server service station with infinite queue capacity customers arrive
according to a Poisson process with a rate of 3 customers per hour. We want to compare the
following two cases:
• Case 1: Service times of customers consist of two independent parts: the first part is
exponentially distributed with a mean of 5 minutes while the second part is exponentially
distributed with a mean of 10 minutes.
• Case 2: Service times of customers consist of only one single part which is exponentially
distributed with a mean of 15 minutes.
(a) [7 points] Determine for both cases the expected time a customer spends in the system.
Clearly explain why in Case 1 the expected time that a customer spends in the system is
bigger than, smaller than or equal to the expected time in Case 2.
4
ANSWER (a) For an M/G/1 queue, the expected time customers spend in the system is given by
W =τ+
λs2
.
2(1 − ρ)
In Case 1 we have (time unit: minute) λ =
1
,
20
τ = 15, ρ = λ · τ =
3
4
and
3
4
and
s2 = E[Y12 ] = σ 2 + τ 2 = 125 + 225 = 350 minutes2 .
We conclude that
W = 15 +
1
20
· 350
= 15 + 35 = 50 minutes.
2(1 − 34 )
(3 points for right calculations in Case 1)
In Case 2 we have (time unit: minute) λ =
1
,
20
τ = 15, ρ = λ · τ =
s2 = E[Y12 ] = σ 2 + τ 2 = 225 + 225 = 450 minutes2 .
We conclude that
W = 15 +
1
20
· 450
= 15 + 45 = 60 minutes.
2(1 − 34 )
(2 points for right calculations in Case 2)
In Case 1 the expected time that a customer spends in the system is smaller than the
expected time that a customer spends in the system in Case 2 because the variance of
the service times is in Case 1 smaller than in Case 2. (2 points for right argument)
5