Optimal Scheduling Among
Intermittently Unavailable
Servers
Simon Martin & Isi Mitrani
University of Newcastle upon
Tyne
Model
m1, x1, h1
Arrivals
l
policy
m2, x2, h2
Parameters
• Arrival rate = l
At queue i (i=1,2)
• Average service time = 1/mi
• Average operative period = 1/xi
• Average repair time = 1/hi
• Average holding cost = ci
A scheduling policy specifies, for every possible system
state, whether an incoming job which finds that state
is sent to queue 1 or to queue 2.
Problem
Find a policy that minimizes average holding costs.
Solution
• The problem is tackled using the tools of
Markov decision theory.
• The optimal policy can be computed
numerically by
– uniformizing the continuous time Markov
process,
– replacing it with an equivalent discrete time
Markov chain; and
– truncating the state space to make it finite.
Uniformization
• The instantaneous transition rates are
modified, so that the transition rate out of any
state is 1, with the addition of transitions
which do not change the current state.


  max   rij 
i
 j 
rij
rij

qij  ; qii  1 


  l  max m1  x1 ,h1  max m2  x2 ,h2 
State of discrete time
Markov chain
S = (i, j, b1, b2, a)
•
•
•
•
•
Number of jobs in server 1: i
Number of jobs in server 2: j
Availability of server 1: b1
Availability of server 2: b2
Arrival event: a
Stationary policy for minimizing
total average discounted costs over
an infinite horizon


V S   cS   min cd     qS , S V S 
d
S


This equation can be solved iteratively.
The interesting case is  →∞
Minimize total average cost over a
finite horizon of n steps


Vn S   cS   min cd    qS , S Vn 1 S 
d
S


Solve recurrences in n steps, starting from
V0 S   0
Steady-state average cost per step,
independent of the starting state
Vn S 
V  lim
n
 
n
Obtained by simulation
Policies examined
• Heuristic (smallest expected conditional holding
cost per job)
• Random
• Selective (send only to operative servers;
N.Thomas)
• Shortest Queue
• Optimal (minimal steady-state cost per step)
Varying l
100
Random
Selective(Nigel)
ShortestQueue
Heuristic
Optimal
90
80
70
Average Cost
60
50
40
30
20
10
0
0
0.5
1
1.5
2
2.5
Lambda
3
3.5
4
4.5
5
Varying m
250
200
Average Cost
150
100
Random
Selective(Nigel)
ShortestQueue
Heuristic
Optimal
50
0
0
0.5
1
1.5
2
2.5
Mu1
3
3.5
4
4.5
5
Varying x
200
180
Random
Selective(Nigel)
ShortestQueue
160
Heuristic
Optimal
140
Average Cost
120
100
80
60
40
20
0
0
0.2
0.4
0.6
Xi1
0.8
1
1.2
Varying Holding Cost
140
Random
Selective(Nigel)
ShortestQueue
Heuristic
Optimal
120
Average cost
100
80
60
40
20
0
0
0.5
1
1.5
2
C1
2.5
3
3.5
4