^^
o/
tbC^^/'
ALFRED
P.
WORKING PAPER
SLOAN SCHOOL OF MANAGEMENT
"Scheduling Nebvorks of Queues:
Heavy Traffic Analysis
of a Multistation Network with Controllable Inputs"
Lawrence M. Wein
MIT
Sloan School Working Paper #3046-89-MS
MASSACHUSETTS
INSTITUTE OF TECHNOLOGY
50 MEMORIAL DRIVE
CAMBRIDGE, MASSACHUSETTS 02139
"Scheduling Networks of Queues:
Heavy Traffic Analysis
of a Multistation Network with Controllable Inputs"
Lawrence M. Wein
MIT
Sloan School Working Paper #3046-89-MS
July 1989
Sloan School of Management,
Massachusetts Institute of Technology
Cambridge,
02139
MA
SCHEDULING NETWORKS OF QUEUES: HEAVY TRAFFIC ANALYSIS
OF A MULTISTATION NETWORK WITH CONTROLLABLE INPUTS
Lawrence M. Wein
Sloan School of Management, M.I.T.
Abstract
problem of input control
Motivated by a factory scheduling problem, we consider the
sequencing in a multistation, multiclass
(subject to a specified input mix) and priority
and a general routing structure.
queueing network with general service time distributions
The
objective
is
to
in the
minimize the long-run expected average number of customers
average output
system subject to a constraint on the long-run expected
rate.
Under
problem can be approximated by a
balanced heavy loading conditions, this scheduling
control problem involving
workload formulation of
for a multidimensional
Brownian motion. Linear programming
this control
is
used to reduce the
problem to a constrained singular control problem
Brownian motion.
The
finite difference
approximation method
The
solution
is
latter problem.
then used to find a linear programming solution to the
is
order to obtain an effective
interpreted in terms of the original queueing system in
scheduHng
policy.
The
priority sequencing policy
is
based on dynamic reduced costs from
policy releases a customer into the
a Unear program, and the workload regulating input
region.
system whenever the workload process enters a particular
that illustrates the procedure and demonstrates
An example
is
provided
its effectiveness.
sequencing in a stochastic job
Subject ciassification; Production/scheduling: priority
problems.
shop. Queues: Brownian models of network scheduhng
June 1989
SCHEDULING NETWORKS OF QUEUES: HEAVY TRAFFIC ANALYSIS
OF A MULTISTATION NETWORK WITH CONTROLLABLE INPUTS
Lawrence M. Wein
Sloan School of Management, M.I.T.
Harrison
[7]
has introduced a Brownian system model that approximates a multiclass
queueing network with dynamic scheduling capability.
conditions, this
model allows a queueing network scheduling problem to be approximated
by a control problem involving Brownian motion.
control problem
Under balanced heavy loading
In
Wein
was solved that approximates the problem
specified product mix)
and
[20],
a particular Brownian
of input control (subject to a
priority sequencing in a two-station multiclass queueing
network
with general service time distributions and a general routing structure. The objective was
to minimize the long-run expected average
number
of customers in the system subject to
a constraint on the long-run expected average output rate.
control problem
was interpreted
The
solution to the Brownian
terms of the original queueing system in Wein
in
[21] in
order to obtain an effective input control and priority sequencing policy.
In this paper
network with any
was solved
in [20]
programming
we extend these
finite
by
number
results
from the setting of a two-station network to a
of stations.
(1) reformulating the
The
problem
two-station Brownian control problem
in terms of workloads, (2) using linear
to reduce the workload formulation to a constrained singular control problem
for a one-dimensional
Brownian motion, and
constrained singular control problem.
on dynamic reduced costs from the
The
linear
(3) finding
a closed-form solution to the
resulting priority sequencing policy
was based
program, and the input policy depended on a
two-dimensional workload process, which measured the total expected amount of work in
the network for each of the two stations. This worA-ioad regulating input policy released
a job into the network whenever the workload process entered a particular region in the
nonnegative orthant of R^; this region was based on the solutions to both the hnear
program and the constrained singular control problem.
For the general multistation problem considered here, the Brownian control problem
can again be reformulated
in
terms of workloads
ajid linear
programming can be employed
to reduce the workload formulation to a constrained singular control problem. Therefore,
the resulting sequencing policy
is
again based on dynamic reduced costs derived from the
hnear program. However, the constrained singular control problem now involves a multidimensional Brownian motion process, and the problem appezirs to be very
solve in closed form.
(see
Kushner
[12] for
Instead,
we employ the method
difficult to
of finite difference approximations
a detailed development) to obtain a numerical solution to the con-
strained control problem.
By
discretizing
both state and time,
this technique allows us to
approximate a controlled diffusion (and functionals of the controlled diffusion) by a controlled
Markov chain (and functionals
of the controlled
Markov
chain). In particular,
if
we
ignore the constraints in our constrained singular control problem, then the problem can
be approximated by a controlled Markov chain with a long-run average cost
is
well-known
(see, for
example,
Manne
[15]
and Derman
[4])
criterion.
that the latter problem can
be formulated and solved as a linear program. Moreover, we show that under the
difference approximation, the constraints in our singular control
constraints in the linear
programming formulation
of the
It
problem become
Markov chain
finite
linear
control problem.
Therefore, these constraints can simply be added to the linear program for the
Markov
chain control problem, and an approximate solution to the constrained singular control
problem can be found by solving a
policy
is
linear
program. As
in
Wein
[21],
the proposed input
a workload regulating input policy, where the region of release depends on the
solution to a linear
program and the approximate solution
to the constrained singular
control problem.
In order to rigorously justify the finite difference approximation, one needs to prove
(see
Kushner
[12]-[13]) that the
optimally controlled Markov chain (suitably interpolated)
2
converges to the optimally controlled diffusion, cind that the optimal cost of the controlled
Markov
Such a
chciin
justification,
tempted
when
converges to the optimzJ cost of the constrained singular control problem.
is
typically based
However, we do show that
here.
Wein
As
Wein
in
in
Wein
[21],
approximation technique,
the customer release policy and the priority sequencing policy derived
is
much work
not too
aggressively pushes this
be seen
will
is
The input
policy reluctantly pulls work into the
released whenever a server
is
The
already present in the network.
work to the various stations
in Section 10, the
proposed policies
threatened with idleness and
priority sequencing policy
in order to avoid server idleness.
offer
a significant improvement in
performance over conventional customer release and priority sequencing
This paper
problem
trol
not at-
[20].
network, in that a customer
As
this finite diff"erence
is
agrees with the closed form solution of this problem derived
[21],
here work together in a coordinated way.
there
on weak convergence methods,
applied to the numerical example of a two-station queueing network scheduling prob-
lem considered
in
which
is
is
organized as follows.
In Section
1,
the queueing network scheduling
described, and the workload formulation of the approximating Brownian con-
problem
is
given in Section
2.
Linear programming
is
used in Section 3 to reduce the
workload formulation to a constrained singular control problem, which
tion 4.
policies.
The
finite difference
is
described in Sec-
approximation, which allows the constrained singular control
problem to be approximated by a constrained Markov chain control problem,
Section
5,
and a
In Sections 7
linear
and
programming solution to the
8, respectively,
latter
problem
is
given in
derived in Section
the solution to the workload formulation
terms of the original queueing system
is
in order to obtain effective priority
is
6.
interpreted in
sequencing and
input control policies, respectively, for the original queueing network scheduling problem.
In Section 9, a heuristic extension to the input policy
cision of
which
the customer.
class of
is
described that allows for the de-
customer to next release into the system, not just when to release
An example
is
given in Section 10 that illustrates the entire procedirre and
3
demonstrates
its effectiveness, aiid
concluding remarks are offered in Section
The Queueing Network Scheduling Problem
1.
The queueing network scheduling problem studied
scheduling problem that
is
encountered
many
in
in this
factories; see
of the motivating factory scheduling problem. Consider a
server stations
station s{k)
and
mean
paper
Wein
1
than one, and so
service times that are independent
m
jt
and variance
number
— Ylj=i
all
motivated by a
[21] for
a description
queueing network with I
single-
s]..
A
and identically distributed random
class k customer,
upon completion
and
station s(k), turns next into a class j customer with probability Pkj
with probability
is
A' customer classes. Class k customers require service at a pcirticular
and have
variables with
the
11.
Pkj-
The Markovian switching matrix
P
of service at
exits the
network
has spectral radius
less
customers eventually exit the network with probability one. Because
of classes
is
allowed to be arbitrary, this routing structure
is
almost perfectly
general.
There are input control and
We
assume there
is
priority sequencing decisions in the scheduling problem.
an ample supply of customers who are waiting
to gain entry into the
network, and that each of these customers has an exogenously specified class designation.
The
class designations are such that qk
released into the network.
1.
We
will
The
assume that the
is
the long-run proportion of class k customers
entering class
mix vector
=
{qk)
is
such that ^;(.=i 9^
class designations are deterministic; however, they could
Markovian without changing the nature of the
the non-decreasing process
q
A'^
=
{N{t),t
customers released into the network
>
in the
analysis.
0},
The input
where N(t)
time inteval
is
[0,<].
~
be
decisions are to choose
the cumulative
number
of
Thus, the input decisions
allow for full discretion over the timing of the release of customers into the system, but
do not allow
for the choice of
which
class of
customer to
release. In Section 9,
we develop
a heuristic scheme that allows the controller to decide which class of customer to release
into the system; this
scheme guarantees that the actual mix that
mix
close to the desired
The sequencing
released
is
is
sufficiently
q.
decisions are to dynamically choose which class of customer to serve
at each station in the network.
Although preemptive resume scheduling
is
assumed, the
assumptions regarding preemption do not have an impact on the proposed scheduling
policy.
There
in the
is
a per unit time holding cost
that
Cfc
is
incurred
when a
class k
customer
is
queueing network. Define the throughput rate of the queueing network to be the
number
of
problem
is
customer departures per unit of time. Then our queueing network scheduling
to find the input
and sequencing policy that minimizes the long-run expected
average holding costs subject to a constraint that the long-run expected average throughput
rate
is
greater than or equal to the specified lower
then the objective
is
bound
A.
If ct
=
c for k
number
to minimize the long-run expected average
=
l,...,/\,
of customers in
the network. Since the throughput constraint will be tight in general, this latter objective
is
equivalent (by Little's formula) to minimizing the long-run expected average cycle time
of customers,
2.
is
the length of time
it
spends in the network.
The Workload Formulation of the Brownian Control Problem
Harrison
last section is
is
where a customer's cycle time
shown how
[7]
has shown
how
the queueing network scheduling problem described in the
approximated by a control problem
this
Brownian control problem
is
for a
Brownian network. In Wein
[20], it
reformulated in terms of workloads. Since
the proposed scheduling policy depends only on the solution to the workload formulation,
we
will
go directly to the workload formulation of the Brownian control problem that
approximates the queueing network scheduHng problem described
5
in Section 1.
Let p
=
{pi)
be the /—vector of server
tions; later in this section, p will
be explicitly defined
The Brownian approximation assumes
—
\/n{\
Let
k
=
Qk = {Qk{t)J ^
1,...,A',
and
let
Pi) is positive
0} be the
/,
=
is
idle in the
With the
parajTieter
at station
i
>
terms of the problem parameters.
in
and
moderate
of
size for
=
i
1,
...,
/.
(2-1)
of class k customers in the system at time
0} be the cumulative
time interval
/ sta-
the existence of a large integer n such that
number
{I,{t),t
n
utilizations, or traffic intensities, for the
[0,<], for
=
i
fixed, define the scaled
amount
t,
for
of time that the server
1, ...,/.
queue length process Zk
=
{Zk{t),t
>
0}
by
Z^m = ^4^,
/
>
and
=
and the scaled cumuiative idleness process U,
=
{Ut{t),t
= ^^^, t>0 and
U,{t)
^^
1,...,
>
a;
(2.2)
0} by
1=^1,. ..J.
(2.3)
Define the one-dimensional scaled centered input process 6 by
m=
where N{t)
and A
is
the cumulative
number
Xnt- N(nt)
^^-^, i>0,
of customers released into the system
the specified average throughput rate.
is
The
processes
axe the control processes in the workload formulation of the
Let the AT— vector A
=
(A^t)
is
the entering class mix vector,
of class k customers that
up
Z = {Zk),U —
to time
(U,),
t
and
Brownian control problem.
de defined by
A
Since q
(2.4)
it
=
(2.5)
q~X.
follows that A^ represents the average
must depart from the system per unit
of time in order to satisfy
the throughput rate constraint. Define the A' x A' input-output matrix
Bkj=m-\6,k-Pjk\
6
number
R=
(Rkj) by
(2.6)
where
at
8jk
which
=
\i
\
—
j
k
and
6jjt
=
P
nonnegative solution
is
/?
K
=
(/3t)
=
nonsingular and there exists a unique
i?/?.
resource consumption matrix
(2.7)
A=
(Anc) by
otherwise.
0,
server utilization vector p referred to in condition (2.1)
p
Now
is
to the flow balance equations
\
Then the
R
transient, the matrix
A
The I x
represents the average rate
customers axe depleted when class j customers are being served. Since
class k
the routing matrix
Define
The term Rkj
otherwise.
=
is
defined by
AI3.
define the / x A' worA-ioad profile matrix
(2.9)
M = (Mjt) by
M = AR-\
The element Mik
represents the total expected remaining
k customer at station
worA'ioad process
i
until the
is
= MZ{t),
interpreted as the total expected
i
at time
t.
Now
i
By
Proposition
/9,
>
work
for a class
Let the /—dimensional
(2.11)
0,
amount
= Mq,
interpreted as the expected total
spends on each customer.
t
of scaled
define the /—vector v
V
is
of
W be defined by
network for station
so that Vi
amount
customer exits the network.
W{t)
so that Wi{t)
(2.10)
1
=i),Afor
7
i
Wein
=
{v,)
in the
by
(2.12)
amount
in
=
work anywhere
of time over the long-run that server
[21],
1,...,/.
(2.13)
Let the /—vector 7
=
be defined by
(7,)
-,,
Let C{i) be the set of
all
= y/^{l-p,)i0Tl =
l,...,I.
customer classes k such that s{k)
=
(2.14)
i,
and define the i^— vector
a = (Qk) by
ak
Now
define
=
—
alike
for
C{i).
(2.15)
X to be a A'— dimensional Brownian motion process with drift 6 and covairiance
E, where
6
and
=
y/Ti^X
- Ra)
(2.16)
K
=
^ji
Finally, let
B
I][c^^"^ir'^itj(<5j7
process
drift
vector
Pki)
+ Qkm-'slRjkRik].
(2.17)
be the /—dimensional Brownian motion process defined by
= MXit),
B{t)
The
-
B
has
—
MS =
The Brownian
in the balanced
drift
y,
M6
t
>
and covariance M'EM'^.
where 7 was defined
control problem
is
(2.18)
0.
It
was shown
in
Wein
[21]
that the
in (2.14).
obtained by letting the system parameter n defined
heavy loading condition
(2.1)
approach
infinity.
Exactly the same notation
used for the scaled processes Z, U, and 6 are used in defining the Brownian control problem
in order to retain the
queueing network interpretation of the Brownian model. Define the
workload formulation of the Brownian control problem as choosing right continuous with
left limit
(RCLL)
processes Z,
U
1
minimize
subject to
and 6 (K-,
U
and one-dimensional, respectively) so
eis
to
/^ ^
limsup— £r[/
Z,
I-
>
CkZk{t)dt]
and 6 are non — anticipating with respect
8
(2.19)
to A',
(2.20)
U
is
Z{t)
>
for all
>
t
T— oo T
MZ{t) = B{t) +
(2.19)-(2.24)
(2.22)
U{t)
7.
-
for
=
i
ve{t)
1, ...,/,
for all
t
>
and
(2.23)
0.
(2.24)
referred to as the workload formulation because the basic
is
system state equation (2.24)
(2-21)
0,
0,
<
limsup-£;[l/,(r)]
Problem
=
non — decreasing with U{0)
is
in
terms of the /—dimensional workload process
W defined
in (2.11).
Solving For
3.
of the control process
and
linear
(2.23).
program
in
Terms of U
we express the optimal process Z
In this section,
(2.21),
Z
U
.
Then
at each
in the
Suppose we are given a process
the optimal
time
Z and
U
workload formulation in terms
that satisfies constraints (2.20),
processes are found by solving the following
t:
K
Y
min
CkZk{i)
(3.1)
K
subject to
^
A/a-^Jt(<)
+
v,e{t)
=
Zkit)
>
B,it)
+
Ihit) for
i
=
1,...,/,
(3.2)
fc=i
At each time
values.
The dual
constraint set.
program
t
>
0, this linear
of this linear
program may have a
program
will
=
1,...,A'.
(3.3)
different set of right
be easier to analyze, since
Let us define the dual variables
to be solved at time
for k
0,
7r(i)
=
(7r,(i))
and
it
hand
side
has a static
state the duai linear
t:
I
max
y[B,{t) +
1=1
9
U,{t)]n,{t)
(3.4)
subject to
y_] M,>7ri(<)
<
for k
c^
—
\,
...,K,
(3.5)
1=1
/
J2vini{t)
=
0.
(3.6)
1=1
Before analyzing the dual
imbalajice process
(3.4)-(3.6), let us define the
(/— l)-dimensional workload
W = {Wt{t)) by
Wi(t)
By
LP
=
piW,{t)
and
(2.11), (2.13), (2.24)
-
p,Wiit),
(3.7), the
t
>
0, for
i
=
1,
...,
7-1.
(3.7)
workload imbalance process can also be expressed
as
W,(t)
=
pjB,(t)
-
+
p,Bi(i)
pjU,{i)
- p,Uiitl
t
>
0, for
i
=
1,
...,
7-1.
(3.8)
Thus, the workload imbalance process does not depend on the control process
expressed in terms of the Brownian motion process
LP
Returning to the dual
B
6,
and
is
and the control process U.
(3.4)-(3.6), use (3.6) to eliminate 7r/(t)
from the problem amd
use (2.13) to substitute the utilization levels p for the vector v to obtain the (7 —
1)- variable
dual linear program
/-I
max
p7^^W^.(07r.(0
^
jr,(0,...,T/_i(()
(3.9)
:=1
/-I
subject to
c^^
y^{piM,k —
PtMjk)T^i{t)
<
pj
for k
=
1,..., A'.
(3.10)
1=1
Denote the solution
equation
linear
(3.6).
program
to (3.9)-(3.10)
by
(7r*(<),
...,
7rJ_j(<))
and solve
for 7rJ(f) using
Let Ck{t) denote the dynamic reduced cost for the variable Zk{t) in the
(3.1)-(3.3).
That
is,
c;t(0
=
c,
-^7r'(f)A/.t.
1=1
10
(3.11)
We
will return to the
dynamic reduced
By
basis for our proposed sequencing policy.
program
(3.1)- (3. 3)
where the costs become the
costs in Section 4,
the analysis above,
it
follows that the linear
can be expressed as
min
subject to
Y^CkZkit)
(3.12)
K
^{piMik - piMik)Zk{t) =
Wi{t) for
i
=
1,
...,
/
-
(3.13)
1,
it=i
Zk{t)>
Denoting the solution to
this linear
length process Z* from Zl{i),k
=
for
Jfc
=
l,...,A".
(3.14)
program by Z^(t), we construct the optimal queue
1,...,A', for all
does not depend on the control process
through the workload imbalance process
4.
0,
/
>
0.
Notice that this optimal process
and depends on the control process
W defined
U
only
in (3.7).
The Resulting Control Problem
In this section
we substitute the optimal queue length process Z*
workload formulation (2.19)-(2.24), and reduce
/—dimensional cumulative idleness process
U
this
By
.
for
Z
into the
problem to one of finding the optimal
duality theory,
it
is
known
that the
optimal value of the primal and dual objectives in problems (3.1)-(3.3) and (3.9)-(3.10)
will
be equal, or that
K
I-\
Y^ckZi{i) =
p-;'Y.w,{t)7^:{t).
(4.1)
«=1
k=\
Define the function h: R''^ -^ R^ by
7-1
h{W{t))
=
pj''£w,it)n:it).
(4.2)
t=i
Then h
is
a piecewise-linear, continuous function of W{t),
zero at the point W{t)
—
0.
11
and h achieves a minimum
of
— 1)— dimensional Brownian motion B by
Define the (/
=
B,it)
p,B,(t)
-
p,Bj(i),
<
>
0, for
B
Recalling that the two-dimensioneJ Brownian motion
drift
ft
=
{fi,),
drift
/z
=
=
has
1,
...,
drift
-
1.
—7,
it
/
(4.3)
follows that
B
when
=
the system
(0, ...,0).
-
V^ipi
Pl), for
=
t
perfectly balanced
is
-
Define the (/
x / matrix
1)
T
...,/-
1,
1.
=
(i.e., p,
(4.4)
p for
i
=
1,...,/),
then the
by
/PI
\
-PI
0^/0
-p2
T=
(4.5)
-pi-2
-pi-\ /
PI
V
Then
B
The
has
where
Mi
Notice that
i
has (/
-
1)
X (/
-
1)
pi
covariance matrix a
resulting control problem
A'— dimensional Brownian motion
TMT,M'^T'^.
to find a non-anticipating (with respect to the
is
A'),
=
non-decreasing, /—dimensional,
RCLL
process
U
to
minimize
lim sup
subject to
— Ej'
T—'oo
I
h{W{t))dt
hmsup —Ei[Ui(T)] <
W,{t)
=
B,{t)
(4.6)
Jo
7,, for
+ piUi{t) -
i
=
p,Ui{t), for
In problem (4.6)-(4.8), the controller observes the (/
motion process
B
imbalance process
I
=
1, ...,
/
-
The
(4.8)
U =
(Ui).
The
bound on the long-run expected average amount
resulting controlled process
W, and
1.
— 1)— dimensional Brownian
and exerts the non-anticipating /—dimensional control
constraint (4.7) determines an upper
control exerted.
(4.7)
1, ...,/,
is
since the optimal process
12
the (/
—
Z* derived
1)
of
— dimensional workload
in the previous section
is
used, costs axe incurred at a rate h(W{t)). Notice that for
W,
only affects the ith component of
Problem
The name
(4.6)-(4.8) will
whereas Uj
zero).
Problem
(4.6)-(4.8)
motion, in which there
is
—
1
...,
/
—
1,
the control Ui
components of
W.
fact that the state of the controlled process can
be instantaneously changed by the controller and, as a
continuous but singular (that
/
1,
be referred to as a constrained singular control problem.
from the
"singular" stems
affects all
=
i
is,
optimal process
result, the
U
the set of time points at which
U
is
increases has measure
can be viewed as a variant of the Unite-fuel problem for Brownian
a constraint on the toted cimount of controlling effort that can be
exerted (or fuel that can be consumed). In the traditional finite-fuel problem, the amount
of control exerted
constrained to be finite over some finite or infinite time interval; see
is
Benes, Shepp, and Witsenhausen
variants of this problem.
Chow, Menaldi, and Robin
[1],
In contrast, the constraints (4.7) are
[3],
and Kaxatzas
[10] for
on the long-run expected
average amount of control exerted.
If
a solution
U*
optimal 9 process
to the constrained control
is,
e'{i)
where Z*{t)
is
(4.6)-(4.8)
can be found, then the
via equation (3.2),
=
v;'[B,{t)
+
U;ii)
the solution to (3.12)-(3.14).
to develop the proposed sequencing
We now
problem
show that the
K
-Y,MuZ:{t)]
for
aU
i
The optimal process
6*
is
and input control
>
(4.9)
0,
never explicitly used
policies.
cost function h in the constrained singular control
problem
is
convex.
Proposition 4.1.
Proof.
By
The function h{W{t)) defined
in (4.2) is
convex
in
W{t).
definition,
7-1
h{w)
=
max
pj^
7ri,...,7r;_i
/^ Wi^^i
(4-10)
^—f
t=l
subject to
7-1
<^k^^ipi^^tk-piMjk)n, <
1=1
13
PI
for k
=
1,...,A'.
(4.11)
Let
tt"
be the solution to
7-1
max
ri,...,jr;_i
p7^Vu"f7r,
^-—
(4.12)
1=1
subject to (4.11), and
,6
let ir°
be the solution to
7-1
pj^y^ w^iTi
mzix
(413)
^-^
jri,...,ir/_i
1=1
subject to (4.11). Thus, hiw")
A €
[0, 1], let
w^ =
(1
—
=
+
A)u'°
=
pj^ ^.1=1 "">? and h{w'')
Au'*,
and suppose n^
is
p/'
^fj/
u'^Trf.
For some
the solution to
7-1
max
Tl,...,7r7_i
pJ^y^w^TTi
•t-—'
(4.14)
1=1
subject to (4.11); thus, h{w^)
=
pj^ E[=/[(1
" ^)^? +
AtifJTr,^.
Therefore
7-1
-
(1
X)h{w'')
+
Xhiiv'')
=
-
(1
A)p7^
7-1
J]u.°< +
Ap7^ ^u-fTrf
1=1
7-1
7-1
>
-
(1
A)p7^ J]u.>,^
+
Ap7^ J^zifTT.^
1=1
=
goal of the next two sections
control problem.
The
(4.6)-(4.8).
To that end, we
i
=
1,
...,
/
When
—
W,{t)
1;
is
- Wj{t) =
make one more minor transformation
is
is
of
problem
to ehminate the coefficients in front of the
where
p,
=
p for
i
=
1,...,/,
W and the Brownian motion B could have been defined
E*=,(^^.;t
- Mik)ZUt) and
B,(t)
then equation (4.8) could be expressed as Wi{t)
the system
|
to find a solution to the constrained singular
In the perfectly balanced case
the workload imbalance process
=
(4.17)
satisfies constraint (4.11).
goal of this transformation
Ui terms in equation (4.8).
by W,{t)
will
(4.16)
i=l
Mti'),
where inequality (4.16) holds because n^
The
(4.15)
1=1
not perfectly balanced,
let pi
=
=
B,{t)
B,{t)
+
- B;(0
U,{t)
—
for
Ui{t).
be defined by
7-1
Px=
n
/>;forz
14
=
l,...,/-l.
(4.18)
and malce the following
definitions:
I
n
U:{t)=
=
W°{t)
=
PjUi{t) Bind ^:
=
piW,{t) and B-{t)
Also, define the function h°
=
by h°{W°(t))
RCLL
process
U°
p,B,(t) for
,.
1,
...,
/
-
h{W(t)). Notice that h"
(4.20)
1.
will
be a piecewise
Then problem
of zero at zero.
to
/ h\W\t))dt
T— oo T
(4.21)
Jo
1
subject to
limsup —Ei[U°{T)] < 7°,
T
T->oo
= BUt) +
W°{t)
For ease of notation,
we
/—dimensional,
RCLL
shall
drop
all
process
U
subject to
and covariance
The
=
(fj.,)
=
1,
for
...,
=
I
of the "0" superscripts
J,
1,
...,
(4.22)
7-1.
(4.23)
and subsequently analyze
'.
f
limsup— £'r
h[\\ {t))dt
/
\\msup —Ej:[Ui{T)] <
W,(t)
//
- Unt),
U°{t)
i
to
1
minimize
sional vector
for
choose a non-decreasing, non-anticipating (with respect to X),
the following problem:
5.
=
i
(4.19)
r
1 ^
...
mimmize limsup— r/j-
drift
!,...,/;
can be expressed as choosing a non-decreasing, non-anticipative (with respect
to X), /—dimensional,
The
=
p,7. fori
J]
and continuous function with a minimum
linear, convex,
(4.6)-(4.8)
I
of the
and the
=
B,{t)
+
U,{t)
(7
—
1)
x (/
—
1)
7,, for
-
Brownian motion
i
=
Ui{t), for
B
will
(4.25)
1,...,/,
i
=
1,...,/-
1.
(4.26)
be denoted by the (/— 1)— dimen-
matrix a
Finite Difference Approximation
15
(4.24)
=
(a,j), respectively.
Method
One
possible approach to analyzing the constrained singular control problem
form the Lagrangian relaxation of
(4.24)-(4.26),
is
where the constraints (4.25) axe placed
the objective function and multiplied by the (unknown) Lagrange multipliers
/
=
to
in
(/,):
/
— iTr
min limsup
W,{t)
subject to
=
B,{t)
Under the case where the
multipliers
aJyzed by Taksar
the case /
[19] for
I
+
h{w{t))dt
U,{t)
-
+ yi,u,{T)
Ui{t), for
=
i
1,
...,
(5.1)
7-1.
(5.2)
are known, the Lagrangian problem has been am-
/
=
and by Meneddi
2
et al.
general
[16] for
The
/.
optimality conditions for the Lagrangian problem ase to find {V,g,U) such that
i^Tr
^T/
.^T/
Min {TV(x) + h{x)-gj, + -—,...,/;_, +
,
/,
_ Y^
}
=
o,
(5.3)
where
/-I 1-1
1=1 J=l
is
the generator of B.
/-I
^2
'
J
when the
•
,=1
In (5.3), the potential function
incurred under the optimal policy
^
V(x)
initial state of
minus the cost incurred under the optimal policy when the
the gain g appearing in (5.3)
is
:
R'~^
—
>
R^
is
the cost
the controlled process ly
initial state of
W
is
is
x
zero. Also,
the minimal average cost per unit time, independent of
initial state.
An argument
identical to
(4.24)-(4.26) can be solved
Theorem
5.1 in
Wein
[20]
shows that the constrained problem
by finding a solution (including the approximate multipliers
/)
=
2,
to (5.3)-(5.4) that simultaneously satisfies constraints (4.25). For the special case /
a closed form solution (V,g,
Although Menaldi
et al.
/j
[IG]
,
l2,Ui
,
U2) to (5.3)-(5.4) and (4.25) was found in Wein
have shown that (5.3)-(5.4) are
[20].
sufficient conditions (along
with some regularity conditions) for optimality of the Lagrangian problem (5.1)-(5.2) and
have proven the existence of a solution {V,g,U) to (5.3)-(5.4), there appears to be
16
little
hope of finding a closed form solution
and
solution {V,g,l,U) to (5.3)-(5.4)
Thus, our goal
seem
(4.25) does not
even
of multipliers
to assure that
/
if
/,
U)
also appears to
itself),
satisfied constraints (4.25).
a search over the space of
However, for the case 1
that the solution (V,gJi,l2,U\, U2) included any nonnegative pair
I2
equaled a particular constant.
nonnegative multipliers
know
if
(/i,
...,
If
//)
lem appears to be the
U) to
finite difference
method
for the controlled
(/i
is
/
This method
is
showed
such that
li
would be
(I
plus
any
do not
possible.
[12].
to systematically approximate a controlled diffusion
Markov
Markov
method
is
[20]
(5.3)-(5.4), (4.25) included
chain,
chain.
and then to
Weak
solve the resulting optimiza-
convergence methods are then used
its
corresponding optimal cost) can be
approximated arbitrarily closely by the controlled Markov chain (and
but no weaic convergence proof
/a)
Wein
approximation technique developed by Kushner
to verify that the controlled diffusion process (and
this paper, the finite difference
,
2,
would be needed
a better approach to a numerical solution to this prob-
difficulties,
basic idea behind this
problem
I,
=
/
such that ^,_j U equaled a particular constant
process by a controlled finite state
tion
a solution {V,g,
then a reduction in the space of
this is true),
Considering these
The
be an arduous
a numerical solution {V,g, U) to (5.3)-(5.4) were obtained for some set
(not an obvious task in
U
possible.
to obtain a numerical solution to the constrained singular control
is
problem. However, finding a numerical solution {V,g,
task, since,
Therefore, finding a closed form
to (5.3)-(5.4).
is
its
optimal cost). In
described and two numerical examples are given,
provided.
particularly powerful for the control problem (4.24)-(4.26) because of
little is
known about multi-dimensional
stochastic control problems with side constraints using a
dynamic programming formula-
the existence of constraints (4.25). Although very
tion, the finite difference
as will be
shown
We now
will retain
approximation can easily incorporate the side constraints (4.25),
in the next section.
begin to describe the
most
of the notation of
finite difference
Kushner
[12].
17
method and,
When
for ease of reference,
we
there are / stations in the network,
the state space of
W
in the control
problem
is
R^~^
.
In order to numerically solve the
W to a bounded which be denoted by G.
to be large enough so that W never reaches the boundary of G, which
problem, we need to confine the process
The
is
G
set
needs
denoted by dG.
Define h, not to be confused with the cost function in (4.24), to be
the finite difference interval, which dictates
how
finely the state space
Let i?[~' be the finite difference grid on R^~^\ a point x €
=
ni,...,n/_i such that x
will
X^,Ji heiUi, where
The approximating
direction.
have state space
controlled
Markov
will
set,
G/,
=
controlled
Rf,~^
H G. Before we
if
chain, which
we denote by {^^,n >
U
Ui
is
define the transition probabilities for the
Then
is
2
let u^
action u G U{x)
for
X
£l11
probabilities
=
is
if
defined by u
^ dG. We
will not
when x € dG,
P
action u
=
=
=
(uj, ...,u/),
>
0}
when
it is
in
control problem correspond to
1 if
the control corresponding to Ui
the control corresponding to
is
not exerted, for
and the cardinality
i
=
!,...,/.
of the action set
concern ourselves with the action set and transition
since, in practice, the controlled
the boundary; one just enlcirges the set
Let
Markov chain
in the
in the singular control problem. Let u^
exerted, and
0},
chain, the controls, or actions, will be described.
G Gh- Recall that the actions
the controls
discretized.
there exists integers
Let U{x) be the action set for the controlled Markov chain {^^,n
state X
is
the unit vector in the tth coordinate
e, is
Markov
-R/,"^
R^~^
G
if
the controlled
Markov chain
will
never reach
Markov chain reaches dG.
{x,y;u) denote the transition probability from state x to state y when the
(ui,...,u/)
is
used in state x G Gh- Define
7-1
7-1
/-I
and assume that
7-1
a„
As mentioned on page 92
of
-
Y^
|a,j|
Kushner
> Ofor
[12], if
z
=
1,...,/-
1.
(5.6)
condition (5.G) does not hold, then a trans-
formation to the principal vectors of a can be applied to assure (5.6) in a new coordinate
18
By
system.
definition of u, the action u
—
(0,...,0)
this case, define the transition probabilities
corresponds to exerting no control. In
P^[x,y;u) by
>/.,„ .^„ ^...,_«..-E0:;#.}l«.;l+2/^/^.
'"':,'
e,h;u) =
-^
P\x,x±
P
{x,x
P
Cih
x
—
=
and P'^{x,y;u)
nonnegative and
When
(x,
+
+
Cih
e,/?;
+
ejh)
otherwise.
sum
u)
= P(x,x —
Cih
—
tjh; u)
=
Cih
—
tjh)
P{x, x
+
i
^
(5-8)
j,
a~
^
= —
for
i
^ j,
(5-9)
Notice that the transition probabilities P''{x,y;u) are
^
(0,...,0),
due to the exertion of controls. In
then the Markov chain transitions are deterministic,
this case,
7-1
/-I
P^{x, x
+ Y^ e,u,h - Y^ eiuih; u) =
(5.10)
1
:=1
1=1
=
for
over y to one for each x.
the action u
and P'^{x,y;u)
af.
'"'
=
(5-'^)
otherwise.
Define the interpolation interval
At
by
Af"
Equation (5.11) relates the
= -^.
size of the discretized
(5.11)
time intervals to the
size of the discretized
space intervals and, together with the transition probabihties P''{x,y]u) in (5.7)-(5.10),
assures that the
first-
and second-order moments of i^^i
consistent with those of the controlled diffusion process
Thus, the controlled process
—
(^, conditioned
on ^^, are
W.
W described by equation (4.26) has been approximated
by the controlled Markov chain {^^,n > 0} with transition probabilities given by
(5.10).
(5.7)-
Since there are no costs on the controls exerted in problem (4.24)-(4.26), the cost
incurred
when
action u
is
taken and the controlled Markov chain
by h{x), where the function
h,
which appears
is
in objective (4.24),
19
in state
is
x
is
simply given
defined in (4.2). Thus,
moment, our problem
ignoring constraints (4.25) for the
by a problem of controlling a finite-state,
average cost criterion. In the next section,
finite action-set
we show how
(4.24)-(4.26)
approximated
is
Markov chain with a long-run
to express the constraints (4.25) in
terms of the approximating Markov chain control problem,
ajid
how
to solve the resulting
constrained Markov chEiin control problem.
A
6.
It is
Linear Programming Solution
known
well
that a
and solved as a
criterion can be formulated
[15]
and Dermaji
[4]
Markov chain
for early
work on
control problem with a long-run average cost
linear
this subject.
Markov chain problem described
the controlled
program; readers are referred to Manne
We
begin this section by formulating
which ignored the
in the last section,
constraints (4.25), as a linear program. Suppose that the cardinality of the set G/,
and
let
the states be indexed by x
stationary policy for the
1,
...,
A/; u
=
controlled
u
=
as a
is
is
used; thus,
tt
one for
is
all
/?
policy. Let
will
policy- dependent
7r^„
M
^
> Ofor
tt
=
(tt^-u)
X
=
1,...,2^.
A
=
{0x{u), x
=
x
=
/9
1,
is
...,
M and
referred to
denote the steady-state probability that
be in state x and action u
and must
=
chosen when the
referred to as a pure stationary policy; otherwise,
Markov chain
is
control problem will be described by
in state x. If ^i(u) equals zero or
randomized stationary
the controlled
and the actions indexed by u
where 0z{u) equals the probability that action u
1, ...,2^),
then ^
l,...,Af
Markov chain
Markov chain
1,...,2^,
=
is .A/,
will
be chosen
if
policy
/3 is
satisfy
1,...,A/;
u
=
l,...,2^
(6.1)
2'
J]7r,„
=
l,
and
(6.2)
r=l u=l
2'
E^y«
U=l
M
2'
= ^^7r,„P''(x,y;u)for
j/
=
l,...,A/,
(6.3)
1=1 u= l
where the transition probabilities P'^{x,y\u) were given
20
in equations (5.7)-(5.10).
Since
the expected average cost under policy
problem
is
to find
tt
=
/3 is
5Zj._i
X^„=i T^zuh{x), the Markov chain control
(t^zu) to
2'
\f
min
V]
/_] 7rj.u/i(x)
(6-4)
1=1 u=l
subject to constraints (6.1)-(6.3).
the optimal policy
If tt*
=
(7r*„) solves
the linear program (6.1)-(6.4), then
jS* is
^x(")
= ^J^"
(6.5)
Thus, the linear program (6.1)-(6.4) solves the problem (4.24)-(4.26) that ignores
constraints (4.25).
We now
show that under the
finite difference
constraints (4.25) can be expressed as / linear constraints on
(5.10),
if
the control u in the controlled
the corresponding cumulative
is
amount
Markov chain
of control exerted
increased by the finite difference interval size
steady-state probability that u,
approximation, the term
=
h.
lim
u > 0}
by
is
in
J7,
(tTj-u).
"^^-^^i-=^^"^"
nAt
From equation
such that
//,
=
1,
then
problem (4.24)-(4.26)
Equation (5.10) also implies that the
^^.-1 X](uu =1) ^'«- Thus, by the
1 is
Wm supj^^^ T~^ Ex[Ui{T)]
"-00
{^n,
=
tt
approximation, the /
in (4.25)
for
.
=
1,
is
finite difference
approximated by
...,
(6.6)
/,'
^
^
which equals
h
by the
critical relationship (5.11) relating the
time and space intervals.
Therefore the
constraints (4.25) are approximated by
A/
2'
E E
,
^x«
<77^fort =
l,...,/.
(6.8)
Thus, a solution to the constrained Markov chain control problem and an approximate
solution to the constrained singular control problem (4.24)-(4.26) can be obtained by solv-
ing the Hnear program (6.1)-(6.4), (6.8).
The
21
solution to this linear
program
will yield
an
optimal policy
function h
for
which the controls for up to / states are rcindomized. Since the cost
convex and achieves a
is
terized by a
/3
minimum
bounded region containing the
inside this region
and u
^
(0, ...,0)
at zero, the
origin.
optimal policy
This region
outside of this region.
The
is
will
such that u
be charac-
=
(0, ...,0)
boundcir}' of the region acts
as a reflecting barrier that keeps the controlled process within the region containing the
origin.
The bounded
Section
region will play a key role in defining the customer release policy in
8.
The
linear
program
(6.1)-(6.4), (6.8) suffers
so often encountered in control problems.
coordinate dimension
and the
linear
is
discretized into
L—
from the curse of dimensionality that
For example,
l
(6.1).
The
the grid
Gh
is
such that each
segments, then the cardinality of Gh
program has 2^ L^~^ variables and L^~^
nonnegativity constraints
if
size of this
Since the holding costs are convex and have a
+/ + 1
is
L^~^
constraints, not including the
problem can be reduced
minimum
is
at zero,
in several ways.
any control u
-^ (0, ...,0)
that pushes the controlled process further from the origin will clearly be suboptimal and
need not be explicitly considered in the linear program. Also, the linear program can be
solved several times (using smaller values of the finite difference interval h for later runs),
and portions
of the state space that the controlled process never reaches can be eliminated
from the subsequent
linear programs, as long as the controlled process never reaches the
boundcury of G/,. Moreover, the finite difference interval h can be state-dependent, so that
a
is
finer grid
can be used in the sensitive portion of the grid Gh (where the optimal control
uncertain) and a courser grid cein be used in the insensitive portion (where the optimal
control
is
known).
may be used
Finally,
some decomposition approaches used
programming
to exploit the structure of the constraint set (in particular, the structure in
(5.7)-(5.10) that appears in (6.3)); readers are referred to
on
in linear
Kushner and Chen
[14] for
work
this topic.
We
finish this section
finite difference
with a numerical example that illustrates the accuracy of the
approximation for solving the constrained singular control problem
99
(4.24)-
The example
(4.26).
taken from Section 7 of Wein
is
and
[21]
is
derived from a particular
two-station queueing network scheduling problem; the scheduling problem
=
the problem considered in this paper with 7
is
The
2 stations.
is
identical to
cost function h{x) in (4.24)
given by
where hi
and
=
and
.101
vEiriance a
=
form solution to
and
r,
/12
=
The one-dimensional Brownian motion
The righthand
10.93.
this
.3703.
problem
(see
side values in (4.25) are 71
Wein
[20]) is
=
72
B
has
drift
-9.
The
=
//
=
closed
characterized by the interval endpoints
I
where
/
= --
^ = -4.771
—,
+
hi
(6.10)'
^
h2 271
and
r
=
=
—^;^
+
The
controlled process
motion
(see Harrison
W behaves as a one-dimensional
[6]
(6.11)
reflected, or regulated,
Brownian
for a detailed
development) on the interval [—4.771,1.301] and
U* and U2
are the local times at the points -4.771 and 1.301,
the optimal control processes
respectively.
1.301.
"2 271
"1
The optimal
objective function value
t"^\
.
=
is
given by (see
Wein
[20])
.2409.
(6.12)
'
^
47i(/!i+/l2)
To formulate the
h
=
0.05
program
was used and the bounded
the state space
1,...,201.
linear
The
is
(6.1)-(6.4), (6.8),
set
G
was taken
to
a
finite difference interval of size
be the interval [—5.0,5.0]. Thus
{—5.0, —4.95, —4.90, ...,4.95,5.0}, and the states are indexed by x
action set U{x) consists of the four actions (0,0), (1, 0), (0,
1),
and
=
(1,1),
which correspond, respectively, to exerting no control, exerting only Ui, exerting only U2,
and exerting both Ui and
U2.
The non-zero
transition probabilities are
P{x,x
+
1;0,0)
= P{x,x-
1;0,0)
=
P(x,x
+
1;1,0)
= P{x,x -
1;0,1}
=
P(x,x;l,l)
=
(6.13)
^,
1,
and
(6.14)
(6.15)
l.
23
Of
—
course, control u
tively) will never
Since
Qh =
(1, 1) will
be used when the controlled process
1=1
Rather
program
is
(l,0)(u
=
(0, 1), respec-
positive (negative, respectively).
{u:u,
(.05)(.9)
.00412 iovr = h2.
<^-j^^
^ru
Markov chain by
y €
=
by x
denote the state of the
1,...,201, let us
[—5.0, —4.95, ...,4.95,5.0].
The
solution to the linear
equation (6.5),
/?;(0,0)
=
1
for
=
/9l4.8(l,0)
The optimal
(6.16)
= l}
thcin indexing the states
yields, via
yG
[-4.7,1.25],
(6.17)
(6.18)
l,
/3:.75(1,0)
=
.327,
(6.19)
/?;.^5(0,0)
=
.673,and
(6.20)
^r.3o(0,l)
=
l.
(6.21)
objective function value for the linear program was .2411, which
to the derived value of .2409 appearing in equation (6.12).
=
=
a hy (5.5), constraints (6.8) are given by
^ A
E E
controlled
never be used and control u
If
we
is
very close
interpolate the policy
—4.75, the approximate solution to the constrained singular control problem
(3
at y
is
characterized by the interval [—4.784,1.300], where the controlled process
as a one-dimensional reflected
local times of
W
at the
Brownian motion on
two respective endpoints.
difference approximation [—4.784,1.300]
at the rather
modest
is
this interval
The
/i
and U\ and U2 are the
interval derived
from the
finite
very close to the exact interval [—4.771,1.301]
finite difference interval size of h
also solved at the finer interval size of
W behaves
=
.01.
=
.05.
The optimal
This particular example was
objective function value for
the linear program was again .2411, and the interpolated interval was [-4.777,1.300]. Thus,
the finite difference approximation
where a known solution
method
is
exists.
24
very accurate, at least for the simple case
7.
The Sequencing Policy
In this section
we
workload formulation
will interpret the solution to the
in order to
obtain a priority sequencing policy to the original queueing network scheduling problem.
As
in Harrison
Harrison and Wein
[7],
and Wein
[8],
[21],
terms of the optimal control process Z*, where Zl(t)
of class k customers in the
for
system at time
t;
is
the policy will be interpreted in
interpreted as the (scaled)
number
readers are referred to these previous works
a more detailed discussion on the interpretation of the solutions to Brownian control
problems. In particular, the sequencing policy
computed
The reduced
in (3.11).
is
based on the dynamic reduced costs Ck{i)
cost for a class k customer at time
t is
the increase in the
optimal objective function value of the linear program (3.1)-(3.3) per unit increase in the
>
righthand side of the nonnegativity constraint Zk{t)
the extra cost incurred
if
one were forced to hold a
class k
Thus, the higher the value of Ckit\ the more expensive
the system at time
This vcdue can be interpreted as
0.
it
is
customer
in
queue
at time
t.
to hold a class k customer in
t.
However, each customer
before exiting. Therefore,
class requires a different
it is
amount
of expected processing time
natural to consider the amount of effort needed to completely
process a customer, in addition to the cost incurred in holding the customer. As pointed
out by
Yang
[24],
the ratio
-^i^lL-
(7.1)
measures how costly a
class k
Our proposed
to give priority at each station to the customer class with the largest
policy
is
value of this dynamic index.
Brownian analysis
for class k
(see, for
customer
Yajig
[24]
is
at time
t
has shown that this policy, when applied to the
of a single-server multiclass
queue with per unit time holding cost
customers (see also Section 6 of Harrison
example, Klimov
per unit of remaining processing time.
[11]).
25
[7]),
reduces to the well-known
Cfj.
Ck
rule
Complementan' slackness implies that the reduced
Thus, the policy proposed
in (3.1)-(3.3).
Zl{t)
>
policy
is
(see, for
Chen
when
only
cost Ck{t) equals zero
when Z^{t) >
customer from classes with
in (7.1) will serve a
=
there are no customers present from classes with Zl(t)
Our
0.
therefore consistent with observations from existing heavy traffic limit theorems
example, Whitt
eind
Mandelbaum
customers
is
[22],
[2])
Harrison
Reiman
[5],
[IS],
Johnson
Peterson
[17],
and
that the normalized queue length process of the lowest priority
and the normalized queue length processes
positive
[9],
customers vanish; of course, these results assume static
of the queueing system) priority rankings,
of the higher priority
independent of the state
(i.e.,
whereas we are proposing dynamic
(i.e,
state-
dependent) priority rankings.
Notice that
it
is
therefore with Ck{t)
>
possible for several different customer classes with Zl{t)
=
0) to
be served at a
common
At times when only these
station.
customers are present at a particular station, a tie-breaking rule
of these classes to serve next.
We employ the
(ajid
is
needed to decide which
tie-breaking rule proposed in
Yang
[24],
which
attempts to reduce the total expected holding cost X)fc=i ^kZk{t) along a steepest descent
direction. Pleaders are referred to that
paper for a derivation of
only a description of the rule will be presented here.
rule;
Define the (7
—
1)
x A' worA'ioad imbaiance proiUe matrix
M,k = piM,k - PtMik
Then
constrciint (3.13)
for
I
=
1,
...,
M
1)
X (/
—
1) invertible
matrix for
all
t.
=
l,
....
(7.2)
A'.
W^i(0 fo^
^
=
li---i-^ "" 1-
consisting of the columns that axe in the
optimal basis of the linear program (3.12)-(3.14) at time
—
M = {Mik) by
I-l;k =
can be expressed as J2k=\ ^ikZk{t)
Let Msit) be the submatrix of the matrix
(/
this heuristic tie-breaking
Let the (/
t.
It
follows that Mfl(/)
— 1)— dimensional
vector
is
an
AZ(/) be
defined by
AZ{t)
where n*{t)
=
{n^{t),...,n'j_^(t))
is
= MB\t)n'{t),
the optimal solution to the dual
26
(7.3)
LP
(3.9)-(3.10).
If
class k
in the optimal basis at time
is
by AZk{t)
time
in the obvious way.
t,
then Ck{t)
server
Ck{t)
The dynamic
its
component
tie-breaking rule
priority sequencing policy,
for each class k at each time
—
for all
customer
using (3.11).
of
AZ(t)
in (7.3)
to give higher priority at
is
is
class
we compute a dynamic reduced
k
If class
is
in the
to equation (7.3).
the set of customer classes that can be served at station
customers present at station
i
E C{i) with the
at time
t,
cost
optimal basis at time
computed according
is
gives priority to the customer class k
i
=
t
and a secondary index AZjt(i)
Recall that C{i)
t,
then we denote
to the classes with the higher values of AZk{t).
t
To summarize the
cic{t)
t,
i.
At time
largest value of Ck(t). If
then server
i
gives priority to the
with the largest value of AZk{t).
The Input Policy
8.
In this section the solution {Z* ,U* ,6*) to the workload formulation (2.19)-(2.24)
is
interpreted in order to obtain an input, or customer release, policy for the original queueing
The
system.
input policy will be interpreted in terms of
all
three controls, in contrast to
the sequencing policy, which was interpreted solely in terms of the optimal queue length
process Z*.
We
begin by interpreting the Z* control. Notice that only
dual
LP
only
7—1
(3.9)-(3.10) will
stays on the
is
at
any time
<,
and
thus, by
= MZ*{t)
for all
<
boundary of a polyhedral cone
not necessarily convex,
is
>
0,
the
LP
complementary slackness,
in the
(3.12)-(3.14) for
any time
at
7— dimensional workload
nonnegative orthant of R^
.
process
t.
W
This cone,
generated by a number of extreme rays emanating from
the origin, where each ray corresponds to a customer class that
the
constraints in the
components of the optimal control process Z* can be positive
Therefore, since W{t)
which
be tight
7—1
some value
of W{t).
27
Thus the number
is
in the
optimal basis of
of different extreme rays,
and hence the number
of different
(/— 1)— dimensional
faces of the cone, equals the
of extreme points of the dual constraint set (3.10). See Figure
stations (that
is,
7
=
3)
emd
six
1
for
number
an example with three
extreme rays.
W,
FIGURE
1.
W{t) Stays on the Cone Boundary.
28
(t)
The workload process
U* and
By
6*.
W
can also be expressed
in
terms of the other two controls,
equations (2.11) and (2.24), the controller in the workload formulation
observes an /—dimensional Brownian motion process B, exerts the optimal controls U*,
which
is
the /—dimensional scaled cumulative idleness process, and 6*, which
dimensional scaled centered input process, and obtains the controlled process
the one-
is
W,
which
is
the scaled workload process, via the equations
=
W,{t)
Recall that the solution
by a bounded region
in
+
B,{t)
U*
U*{t)
v,e*{t), for
R^~^ containing the
1,
origin.
...,
/ and
f
>
0.
problem
The boundary
Moreover, the control process
W reaches the reflecting
the workload imbalance process
interpreted as incurring server idleness at station
restriction of the
=
(8.1)
is
characterized
of the region acts as a
(/— 1)— dimensional workload imbalance process
this region containing the origin.
The
?
to the constrained singular control
reflecting barrier that keeps the
is
-
i,
J7* is
W within
exerted only
when
barrier. Exerting the control
for
i
=
U*
1, ...,/.
(/—I )— dimensional workload imbalance process W" to a bounded
region containing the origin implies the restriction of the /—dimensional workload process
W to a
finite
was derived
portion of the cone boundary. Thus the reflecting boundary in
in the solution to the constrained singular control
cates the polyhedral cone in
R^
.
The
intersection in
R^
/?
,
which
problem, essentially trun-
of the
botmdaxy of the
original
polyhedral cone and the reflecting barrier will be referred to as the upper edge of the
truncated cone. See Figures 2 and 3 for typical cases when /
Figure
2,
the reflecting boimdary in R^
is
the interval
[a, 6],
=
2
and
3, respectively.
and the upper edge of the
truncated cone consists of two points. The upper edge of the truncated cone
R^
in Figure 3 and, in general,
We
is
of dimension 1
—
In
is
a curve in
2.
can now make the following three observations about the optimal solution to the
workload formulation:
(1) the
W stays on the truncated
exerted only when the workload process W
/—dimensional workload process
cone boundary, (2) the control process U*
is
29
W(t)
W
FIGURE
2.
The Truncated Cone
in a
reaches the upper edge of the truncated cone, and (3)
(t)
Two-Station Example.
when
the workload process
the upper edge of the truncated cone, then only the input process 6*
is
is
not at
used to keep the
workload process on the truncated cone boundary.
The
goal of this section
is
to develop a customer release policy for the original queueing
system that operationalizes these three observations.
AU
these observations are developed
under the ideahzed assumptions of the balanced heavy loading condition
that in the actual queueing system, the workload process
truncated cone boundary. The actual workload process
30
may
may
(2.1).
Notice
not reside exactly on the
reside in the interior of the
W
(t)
2
Upper Edge
W.(t)
W3(t)
FIGURE
truncated cone
3.
The Truncated Cone
or, since
in a
the extremal rays of {PF|VF
the extremal rays generating the truncated cone,
Three-Station Example.
= MZ, Z >
may
may
not coincide with
reside outside the truncated cone.
We now attempt to operationalize the control process 9*.
31
0}
In order to see
how
the scaled
centered input process 6*
heavy loading condition
caji
move jdong a
(2.4),
when
6*
is
manipulated,
is
is
Vj for i,j
=
close to the (1,1,...,1) direction in
increased and moves in the direction that
being increased relative to the nominal input rate
moves
by equation (2.13) and the balanced
approximately equal to
(2.1), v, is
direction that
recall that
in the direction that is close to
(
A.
close to (1,
is
Similarly,
— 1,— 1,...,— 1),
input
R^
is
when
6*
By
.
1,
is
...,
9*
and so
1,...,/,
definition
input
1),
is
decreased emd
being witliheld relative to
the nominal input rate.
When
then 6*
is
W
the workload process
is
decreased in order to keep
the workload process
W
W on the truncated cone boundary.
W
on the truncated cone boundary.
interpret the action "increase ^*" to simply
cind the action "decrease ^*" to
this interpretation
mean
t
when
mean
"release a
"cease input".
and from observations
the system at times
Similarly,
when
outside the cone and not near the upper edge, then 9*
is
increased in order to keep
cone interior and not near the upper edge,
in the
(l)-(3)
the workload process W[t)
in
Wein
[21], let
us
customer into the system"
The naive
above
As
is
policy that emerges from
is
to only release a customer into
is
on the outside of the truncated
cone.
A
precise definition of
what
is
meant by "outside the truncated cone"
shortly, but first the insights
behind observations (l)-(3)
workload process leaves the
interior of the truncated cone,
becomes too imbaJanced
at least
(recall the
be summarized.
be given
When
into the network to avoid server idleness.
the
then the workload process
importance of the workload imbalance process
one server becomes threatened with
idleness. In this case,
a customer
is
W)
and
released
However, when the workload process reaches
the upper edge of the truncated cone, then there
the controller refuses to release any
will
will
is
more customers
already ample work in the system and
into the system
and
is
willing to incur
server idleness.
We now
Rb
return to the issue of defining the term "outside the truncated cone".
denote the bounded region
in
Let
R^~^ that characterizes the solution to the constrained
32
singular control problem in Section
the polyhedral cone in
extreme points, by /
LP
=
Recall that the
number
1, ...,F.
(3.9)-(3.10). For dual
LP be
of faces
on the boundary
of
equal to the number of extreme points in the dual constraint
number by F, and index the
For each face /,
let
us define
workload imbalance space where extreme point /
the primal
is
is
Let us denote this
set (3.10).
in the
R^
6.
extreme point /,
indexed by
A;i, ...,
kj-i.
is
faces,
and hence the dual
Rf C R^~^
to be the region
the optimal solution to the dual
the customer classes in the optimal basis of
let
Then the hyperplane generated by cone
face
/
given by
=
Y,aiW,{t)
(8.2)
0,
1=1
where
{aiW,{t),...,aiWj{t))
Let r f denote the halfspace in
R^
=
W,{t)
...
Wi{t)
Mik,
...
Mjk,
dei
(8.3)
(lying outside of the cone) generated
by the hyperplane
(8.2)-(8.3).
The workload
regulating input policy will be to release a customer into the network
whenever the /—dimensional workload process W(t) enters the release region Ui</<F.Ry,
where
Rf = {W{t)\(W{t) eRff) Rb) n iW{t) e
and W{t)
r;
n R+)},
(8.4)
defined in terms of W{t) in (3.7); this region defines the vague term "outside
is
the truncated cone".
In order to motivate policy (8.4),
in
by
Figure
4,
(a^i,yi)
where the upper edge
and the upper point by
of concreteness
and without
is
it is
given by two points;
(^2,2/2)5
(—1,-1)
where xi >
loss of generality,
Figure 4 was a balanced system, so that
or the
easiest to consider a two-station example, as
Vi
direction; then the refiecting
33
let
t/i
us denote the lower point
and X2 <
y2-
For the sake
suppose the queueing network generating
=
V2
and 8* moves
boundary
[a,b]
in the (1,1) direction
introduced in Figure 2
is
—
[a'2
—
y2,^i
yi]-
W
Notice that as long as the workload process
shaded
in the darkly
is
region in Figure 4 (the intersection of the darkly shaded regions and the gray regions are
along the 45 degree Unes), then 0* alone can be increased to move
cone boundary.
When W{t) =
(xi,y), where y
<
j/i,
workload process to the truncated cone bovmdary,
U*)
is
used to move
release region for this
W
to the truncated
is
used to move the
in this case to the point (a;i,yi).
the lower (respectively, upper) gray region of Figure
(respectively,
then U2 alone
W
2,
some combination
to the truncated cone boundary.
example should be
at least the darkly
In
and U2
of 6*
Therefore, the
shaded regions and at most
the union of the darkly shaded regions and the gray regions. Although either extreme (or
some compromise between the two extremes) would probably lead
release policy,
we have proposed
in (8.4) the
to an effective customer
minimvun release region, which
in this
example
corresponds to the deirkly shaded regions. This choice does not maintain consistency with
previous work (the policy proposed in Wein
[21]
was the maximal release region, which
corresponds to the union of the darkly shaded regions and the gray regions in Figure
however, the minimal release region suggested here
is
more
4);
easily generedizable to higher
dimensions.
For the example in Figure
y2
—
X2
by face
and
< Wi(0 ~
1
i?2 is
^"^2(0
^
yi
4,
~
the reflecting boundarj'
^1- If the
(respectively, face 2), then (see
>
^.'
1.
^
Rb
is
given by W{t) €
upper face (respectively, lower face)
Wein
[20]) i?i is
Recalling the definition of rf,
it
is
W{t) <
0,
or
is
[o,i], or
denoted
-fj^ <
£i
=
1,
easily verified that the region
specified in (8.4) corresponds to the darkly shaded regions in Figure 4.
In order to develop an explicit description of the release region, an explicit expression
is
required for the (/
— 1)— dimensional bounded
to the constrained singular control problem.
region that characterizes the solution
However, only
a
numerical solution to the
constrained singular control problem has been derived in this paper. Thus, the numerical
solution needs to be transformed into an explicit expression for the
Section 10, an example
is
carried out for a three-station network,
34
bounded
and the
region.
reflecting
In
bound-
W2(t)i
W(t)
W,(t)
FIGURE
4.
The Release Region
axy containing the bounded region in R^
in order to develop
spirit is
an
is
Two-Station Example.
approximated by a piecewise hnear boundary
explicit release region in
required to develop release regions
in a
R^ Presumably, an approximation
.
when / >
3.
Notice that the workload regulating input policy defined above would ignore a
ence that exists between the actual queueing system and the idealized heavy
As pointed out
in
Wein
W
is
difference can be understood by
differ-
traffic limit.
making the following
when
the scaled workload
on the lower ray of the cone boundary and Wi{t) <
xi, then there are
observation about Figure
process
[21], this
in this
4.
In the ideaHzed
Brownian
35
setting,
zero scaled customers at station 2 and yet station 2
is
is
not
idle.
due to the rescaling that occurs when passing to the heavy
This apparent paradox
In the actual
traffic hniit.
queueing system, there are enough customers at the particular station to avoid idleness,
but
when looked
This difference
heavy
at in the scaled space of the
may
traffic limit,
these customers vjinish.
prevent the workload regulating input policy to achieve the desired
throughput.
However, the release rule can be adapted to the actual queueing system by enlarging
the release region. There are two ways this can be achieved.
The
enlarge the region on the inside of the cone boundary.
is
vertex of the cone from (0,
...,
0) to (e,
...,
This
first
way
is
to slightly
done by translating the
This translation changes the hyperplcine (8.2)
e).
to
Y,aiW,it)
= e(^a,).
This treinslation, which
may be
(8.5)
1=1
i-l
negligible in scaled space, prevents the process
The second way
straying very far from the original truncated cone bovmdary.
the releeise region
factor K
>
1,
is
while
to inflate the
still
K increase, the servers
The workload
rate A
is
will incur less idleness
e
which the network
is
amount
may grow
e
and
as a result.
and k so that the desired output
achieved. In an actual queueing system, the setting of
spend
by a constant
in Section 6
but the queue lengths
regulating input policy sets the parameter
of time customers
to enlarge
maintaining the relative shape of the bounded region. As
variety of factors, including the
to
bounded region derived
W from
and k
e
will
depend upon a
of variability in the queueing system, the amiount
at non-bottleneck stations, the
network topology, and the extent
heavily loaded.
Notice that the input policy defined in this section has been described in terms of
W.
the seeded workload process
Thus, before implementing this policy
queueing system, the release region needs to be expressed
sealed) workload process, which
"^«(0
=
Jlk=i ^^ikQki^) for
i
=
is
denoted by w{t)
1,...,/
and
t
36
>
0,
=
in
in
an actual
terms of the actual (un-
{{wi{t), ...,wj{t)),t
where Qk{t)
is
>
the actual
0}.
Then
number
of
customers in the system at time
class k
Wi{t)
=
t.
By
XV [Tit]
'y'
for
1
equation (2.2)
=
1,
...,
/ and
<
it
>
follows that
0.
(8.6)
Since the scheduling problem was solved using the long-run average criterion, the time
scaling can be ignored,
and replacing Wi{t) by Wi{t)/y/n
in the inequedities defining the
release region (8.4) will lead to an implementable release policy for the original queueing
network.
The Workload Balancing Input
9.
The customer
decide
when
release policy described in the last section allows for the controller to
to release the next customer into the system, but not to choose the class of
the entering customer.
It
was assumed that the
are exogenously chosen so that q^
into the network. In this section,
is
Heuristic
is
class designations of entering
customers
the long-run proportion of class k customers released
we develop a workload balancing input
heuristic,
which
based on insight gained from the solution to the constrained singular control problem,
that decides which class of customer to release into the system. This scheme appears to
guaranteed to keep the actual mix
improve the performance of the scheduling policy and
is
of released customers sufficiently close to the desired
mix
The key
idea behind the heuristic
the idealized Brownian network only
process
is
q.
the observation that server idleness
when the
(/
W reaches the reflecting boundary derived
— 1)— dimensional workload
in Section 6.
process stays within a region containing the origin, but
when
boundary, the workload becomes too imbalanced, the control
one server incurs
is
U
incurred in
imbalance
The workload imbalance
it
is
reaches the reflecting
exerted, and at least
idleness.
Thus, server idleness would be reduced, and hence system performance would be improved,
if
the workload imbalance process could be discouraged from reaching the reflecting
37
boundaries. Recall by equations (3.13) and (7.2) that
K
W,{t)
=
Y^ M,kZk{t)
for
z
=
1,
...,/-
and
1
<>
0.
(9.1)
k=\
W to the vector queue length process
This equation relates the workload imbaJajice process
Z.
Our
heuristic will release the customer class k that attempts to balsuice the workload
imbalance process and hence avoid server idleness.
Consider the actual (unsealed) queueing system, where w,{t)
imbalance process at time
system at time
t;
t
and Qk(i)
then Wi{t)
is
number
the actual
= ^^=1 ^ikQk(t)
for
i
=
is
of class k customers in the
l,---,-^
—
1
the input poHcy derived in the last section dictates that a customer
the system at time
There are two steps
step checks to see
first
is
t.
if
in the
the actual workload
and
is
>
t
0.
Suppose
to be released into
workload balancing input
heiiristic.
The
the actual mix of customers already released into the network
mix
sufficiently close to the derived
customers released into the system
q.
in the
Let Nk{t) denote the total
time interval
Let
[0, t].
E—
{k
be the set of possible entering classes; these classes correspond to the
number
=
I,
first
of class k
...,K\qk
stage of
>
0}
some
customer type's route. Consider the constraints
9jNj{t)
N*
where
is
K
qkNk{t)
< N*
for all ;, k
an exogenously specified parameter that
mix must stay
class
-
to the target
any of the constraints
mix
specifies
e E,
(9.2)
how
close the actual entering
q.
in (9.2) are violated,
then the heuristic releases a class
/
customer, where
max
{qjNj{t)
-
qkNk{t)]
=
q,nNm{t)
-
qiN,{t).
(9.3)
j,keE
That
is,
target.
we
If
release the customer class that
constraints (9.2) are
is
all satisified,
suflRciently close to the desired mix,
lagging behind the farthest from
its
desired
then the actual mix of released customers
and we can proceed
which attempts to balance the workload.
38
is
to the second step of the heuristic,
Let Wi(t) equal the time average value of Wi over the time interval
[0,t].
This value
can be easily calculated in a computer simulation model of a queueing network or in an
actual factory that
is
under computer control;
then the value of Wi(t)
may be
if
the factory
is
not under computer control,
recorded periodically, for example once per
shift,
and Wi{t)
can be updated accordingly. Let Wikit) be defined by
Wik{t)
Thus, Wiki^)
if
is
=
w^{t)
+ Mik
for
z
=
1,
...,
J
-
1
and k ^ E.
(9.4)
the zth component of the workload imbalance process that would result
a class k customer were released into the system at time
balancing input heuristic releases a class
/
Step two of the workload
t.
customer into the network, where
1=1
1=1
Thus, step two attempts to push the workload imbalance process toward
its
long-run
average value, which presumably will not be close to the reflecting boundary.
that the time-average value of
the origin (that
is,
workload
matrix
will
profile
w
the point (0,
is
chosen as the desired target, as opposed to choosing
...,0)) as
the target. This
is
M and the topology of the network,
not be a particularly desirable value of
The workload balancing input
Notice
u',
heuristic
because, depending on the
it is
possible that the origin
in
terms of avoiding server idleness.
is
equally applicable to multiclass closed
queueing networks (see Section 4 of Harrison and Wein
[8])
because the same relationship
holds between server idleness and the workload imbalance process. In a closed network, a
new customer
is
released into the network whenever a customer exits,
can be used to decide which class of customer to
release.
and
this heuristic
This heuristic was tested on a
simulation model of the two-station closed network example in Section 6 of Harrison and
Wein
[8].
The example
input mix was 50-50.
in average cycle
there
had two customer
types, denoted by
The workload balancing input
time (from 54.9 to 50.6; see Table
I
heiu-istic offered
of
[8])
B, and the desired
a 7.8% improvement
over deterministic input (releas-
ing customers in the order ABABAB...), while maintaining the
39
A and
same average throughput
rate. Similarly, the heuristic offered
a 10.1% reduction in average cycle time (from 38.6 to
34.7) for the controllable input case (see Table
of
I
Wein
[21]).
For both of these cases,
the exogenous parameter TV* was chosen to guarantee that the resulting class mix wo\ild
be within one-half of
1%
of the desired 50-50 mix.
Furthermore, this simple heuristic
the timing of
policy.
The
(/
its
is
probably effective for any factory, regardless of
input policy (exogenous, closed, or controllable) or priority sequencing
— 1)— dimensional workload imbedance
process offers a concise and effective
measure of the balance of work tliroughout a complex network, and
to the server idleness process
10.
is
its
crucial relationship
exploited by this heuristic.
An Example
The procedure described
in this
paper
will
be illustrated by means of a three-station
example. The example has three customer types, denoted by A, B, and C, and the specified
product mix
is
to have equal
numbers
of aJl three types. Table
route of each customer type, and gives the
mean
I
describes the deterministic
processing time (in arbitrary time units)
assumed
to be
exponential, although our results hold for any service time distribution with finite
mean
for each of the various stages of service. All service time distributions are
and variance.
MEAN
CUSTOMER
TYPE
A
B
C
SERVICE
ROUTE
TIMES
3^ ^2
1-^2-+ 3^ 1^2
2-^3^ ^3
1
1
G.O
4.0
1.0
8.0
6.0
1.0
2.0
4.0
9.0
4.0
2.0
7.0
Since each customer class corresponds to a type-stage pair, the twelve customer classes
40
P
12 X 12 routing matrix
Pg
jQ
=
Pjo
=
11
Pii,i2
=
M=l
until that
P23
40
10
11
22204
13
13
1
1
\6
is
=
=
777
1
V2
=
=
U3
—
4
11
11
—
Pei
Pjs
M yields
(10.1)
i
customer
for a class k
0)'^,
equation
thus implying perfect system balance by equation (2.13).
6,
Therefore, the p values can be factored out of equation (7.2), as mentioned in Section
and the 2 x 12 workload imbalance
/-2
.^.
4
^^=[-5
The exogenous output
P3
cjt
=
.9,
=
1
is
12
A-
=
1,
...,
2
6
-7
-11
2
-2
-2\
-2 j
.
^^^'^^
"
12
.15
customers per tmit of time, so that, by (2.13), pi
of
n
12, so that the objective
of customers in the
-7
-7
1
7
7
=
is
4,
M can be given by
matrix
1
1
and the system parameter value
for
number
rate
profile
9
1
=
2/
2
(i00|0000|00
=
customer exits the network. Since 9
=
-Pse
44 0\
00,
the expected remaining processing time at station
(2.12) yields Vi
=
-^45
Calculation of the 3 x 12 workload profile matrix
1-
The
by (A1,A2,A3,B1,...,B5,C1,...,C4).
1,...,12)
has non-zero entries P12
/4
where M,jt
=
from k
are designated (and ordered
100 can be chosen.
The holding
=
P2
=
costs are
to minimize the long-run expected average
system (or the long-run expected average cycle time) subject
to meeting the long-run expected average output rate of .15 customers per unit time.
The dual LP
(3.9)-(3.10) can be expressed as
max
subject to
where
M
is
given in (10.2).
M'^irit)
<
+
(10.3)
W2{t)TT2it)
(10.4)
e,
This problem can be solved graphically for
the workload imbalance process
extreme points of the constraint
a polyhedral cone with
Wiit)ni{t)
W, and
set (10.4),
six faces.
The
six
the solution
is
given in Table
II.
all
values of
There are
and thus the scaled workload process
W
lives
six
on
extreme points lead to the workload imbalance
space P^ being partitioned into six regions, which were referred to in Section 8 as P/,
41
and which axe niimbered
dynamic reduced
is
in
Table
II
for future reference.
costs are calculated according to (3.11) ajid a priority sequencing policy
The
developed according to rules (7.1)-(7.3).
Table
For each of the six regions,
resulting sequencing policy
III.
In order to find the workload regulating input policy, a numerical solution
to problem (4.24)-(4.26).
from n*{t)
By
given in
is
in Table
II.
By
(4.4), the drift of the
The
objective function cost h{W{t)) defined in (4.2)
(2.14), the righthand side values in (4.25) are 7i
two-dimensional Brownian motion process
calculations in (2.7), (2.17), (2.18),
a
and
=
(4.3) yield the covariance
12.333
6.778
6.778
12.444
is
72
(0,0)
t^i(0>0,W'2(0>0,|<{j;^<4;
2:
W,it)>0,W,it)>0,^^<p^^<l;
3:
Wi{t)
0;
W,{t)<0,W2{t)^0;
7r*(0
= -^,7r*(0 =
7rJ(^)
=
A
<0,H'2(0<0,^>1;
4:
W,ii)<0,W2{t)<0,^<'Mll<i-
5:
Wi{t)
=
-l,7r»(O
=
0,W2it) <0;
^m = \^^2ii) = -l
2(0
W^i(0>0,H^2(^)<0,|^>-l;
i'2(0
6:
Wi{t)>0,W2{t) =
0\
H^i(0>0,U2(0>0,^>4;
'^KO
=
^>'^2(0
=
-^
Wyit)>0,W2ii)<0,p^<-l;
TABLE
II.
Dual Solution
as a Function of
42
found
73
=
1-
and the
matrix
SOLUTION
1:
is
=
DUAL
AND DESCRIPTION
< 0,W2{t) >
B
=
needed
(10.5)
REGION NUMBER
1^1(0
is
Workload Imbalance
Using a mesh
LP
size oi h
=
0.5 in the finite difference approximation, the solution to the
(6.1)-(6.4), (6.8) gives the reflecting
of this
dual
when
boundary
LP
it
process
is
is
is
strictly
and
and
is
C/2
=
Figure
1,
...,
6,
5.
corresponding to the six
when
it
is
uncontrolled
reaches the boundary, the
in (in the direction of the arrows in Figure 5)
^3*)-
Superimposed on top
by at
Iceist
one of the
Notice that there axe places on the boundary where more than
used at a given time. In particular, U* and U^ are used at states
(1.5,10),
(3,1.5).
REGION
>
in
The workload imbalance process
within the reflecting boundary, and
pushed back
one control
(3,1),
the partition of the six regions i?y, /
solutions described in Table IL
three controls (t/*,
(1,9.5),
boundary shown
and
l/j
^^^ ^3 ^^^ both used when the process reaches
(0.5,9.5),
(3,0), (3,0.5),
W2(t)
-6-5-4-3-2-1
FIGURE
5.
The
12
3
4
5
Reflecting Boundary in the Constrained Singular Control Problem.
44
w,(t) A
Using the example network, a simulation study was undertciken to compEire the perfor-
mance
tional
of the scheduling policies
proposed
in Sections 8, 9,
customer release and priority sequencing
policies.
and 10
Two
were tested: deterministic, where the interarrival times axe
input,
latter
where the
pohcy
is
total
number
abbreviated by
of customers in the
CLOSED(N)
in
in
is
constant, and closed loop
held constant at
A'^;
the
Table IV. For both of these input rules,
customers were released into the network in the order
were tested
conven-
conventional input policies
ail
network
agaiinst several
ABCABCABC... Both
conjunction with two priority sequencing
the shortest expected remaining processing time rule
input pohcies
rules: first-in first-out
(SERPT), where
(FIFO) and
priority
is
given to
the customer class k with the smallest value of X^,_2 M,k.
The scheduling
WBAL(A'^*), and
policy proposed here
DRC
dynamic reduced
(for
put policy with parameters
is
e
abbreviated in Table IV by WR(e,
k),
where the workload regulating
in-
costs),
and k dictates via
(8.4)
when
the workload balancing input heuristic with pju-ameter
to
be released, and the priority sequencing policy
and
(7.3).
RULE
is
A'^*
a customer
is
to be released,
dictates the class of customer
defined by the dynamic indices in (7.1)
which
statistics for a particular scheduling policy,
policy
and a
priority sequencing rule.
is
a combination of a customer release
For each policy tested, 20 independent runs were
made, each consisting of 5000 customer completions. The
was truncated to reduce the
initial bias.
The
first
200 time units of each rim
and fourth coliunns give the average
third
throughput rate and average cycle time, respectively, over the 20 runs, cdong with 95%
confidence intervals.
The parameters N, e, and k were chosen
so that
all
scheduling policies
achieved the average throughput rate of .149 customer completions per unit time, which
corresponds to a server utilization of 89.4%. Recall that the objective
is
to minimize the
average cycle time subject to a given average throughput rate.
Referring to Table IV,
outperforms
all
seen that the scheduling policy proposed in this paper easily
it is
of the conventional scheduling rules.
in
average cycle time over the
It
also achieved a
though
it is
well
(CLOSED, FIFO)
40.7% reduction
known
(see
interarrival times of a queueing
The parameter value
e
=
[23], for
network
1.5
case,
relative to the
Whitt
The
policy offers a 28.8% reduction
which was
its closest
(DETERMINISTIC,FIFO)
that the
bounded region
ii,
=
case, even
example) that reducing the variability
will lead to
in the
improved performance.
corresponds to 15.0 units of unsealed work, which in
turn roughly corresponds to the amount of work for each server that
and one-half customers, since
competitor.
6 for
in Figure 6
i
=
1,2,3.
The parameter
is
embodied
value
«;
=
1.7
was enlarged by 70%. The parameter value
in
two
means
A'^*
=
^
guaranteed that the actual entering fraction of each customer class was within .003 of the
specified target of
g^,.
=
.333; in the simulation study, the resulting class
mix was
virtually
equal to the target mix.
11.
Conclusions
In this paper
we have considered the problem
47
of
how
to dynamically release jobs
(when and which
class)
and
prioritize jobs in a multistation multiclass
queueing network
with general service time distributions and a general routing structure. The objective weis
to minimize the long-run expected average hnear holding costs of customers, subject to
a specified average input mix and a constraint on the long-run expected average output
rate.
Under baJemced heavy loading
conditions, the scheduling problem
by a Brownian control problem, and a numerical solution
was approximated
workload formulation of
to the
the control problem was obtained.
This solution was then interpreted in terms of the original queueing system in order to
The
develop an effective three- part scheduling policy.
first
part
is
the workload regulating
input policy, which releases a job whenever the workload process enters a particular region.
The second
part
the workload balancing input heuristic, which releeises the customer
is
class that will best balance the
part
is
workload among the various bottleneck stations. The third
the priority sequencing policy, which assigns dynamic indices (based on dynamic
reduced costs from a linear program) to each customer
performed that exhibited the policy's
Two
difference approximation used in Section 5,
and
the large linear program posed in Section
6.
in this
A
computational study was
effectiveness.
related research topics are to prove a
problem addressed
class.
weak convergence
result for the finite
to develop an efficient algorithm to solve
Also, the close relationship between the
paper and the problem of priority sequencing
in a multistation
multiclass closed queueing network requires further investigation. Finally,
studies need to be performed to better understand the behavior
proposed scheduling
more numerical
and robustness
of the
policy.
Appendix
Let the regions
il/,
/
=
1,
...,
6 be defined as in the
48
first
column
of Table II (with
W{t)
replaced by
The
u'(t)).
release region (8.4) for the
example
in Section 10 is to release
customer whenever the workload process w(t) enters ni</<6i?/, where Rj
Si
:
wi{t)
-
+
4u'2(0
<
42ti'3(0
390e,
and
Wi{t)-W3{t) < 30k.
^2
:
+
-W2(t)
wi(t)
—
2wi{t)
53
:
W3(t)
+
-Wi{t)
—
-wi{t)
<
W2{t)
+
-
+
3w3{t)
-
<
W3{t)
<
lV3{t)
18w2{t)
W3{t)
120e,
35k,
2u>2(<)
-
I39wi(t)
W2{t)
<
13w3it)
-
140k, and
<
ISOk.
44t('3(0
<
770e,
<
770e,
95k, and
<
0;
or
-
lZ9wi{t)
-Uwi{t) +
54
:
:
-
44u'3(0
5w2{t)
+
6w3{t)
ll«'i(0-4u)3(0 <
— u'i(i) +
55
I8w2{t)
u'3(i)
wi{t) -W2{t)
<
<
and
W2{t)
lOe,
Wi{t) -W2{t)
56
:
<
-wi{t)
Wi(t)
—
+
300k.
70e,
55k, and
25k.
<
15k.
+
4u'2(0
W2{t)
<
<
2u'3(f )
<
50e,
and
15k;
or
-wiit)
+
4w2{t)
-W2{t)
+
iC3{t)
+
<
2iV3{f)
0,
<50e,
and
49
and
= RfDSf,
a
and:
Wi{t)-W3{t) <
30/c.
Acknowledgements
This research
wets partially
supported by a Junior Faculty Grant from the Leaders for
Manufacturing Program at MIT.
I
would
like to
thank Phillipe Chevalier for his invaluable
assistance with the computations in Section 10 and for his helpful
the materiEd in Section
8,
and Robert Freund
50
for
drawing Figure
comments pertaining
1.
to
REFERENCES
Benes, V. E., Shepp, L. A. and Witsenhausen, H.
[1]
Control Problems. Stochastics
4,
S.
(1980).
Some
Solvable Stochastic
39-83.
Chen, H. and Mandelbaum, A. (1987). Stochastic Flow Networks: Bottlenecks and
[2]
Dif-
fusion Approximations. Preprint, Graduate School of Business, Stanford U., Stanford,
Ca.
Chow,
[3]
Menaldi,
P. L.,
Linear Systems
Derman,
[4]
C. (1966).
Ann. Math.
[5]
With
Harrison,
J.
J.
L.,
and M. Robin (1985). Additive Control
Finite Horizon.
SIAM
J.
of Stochastic
Control and Opt. 23, 858-899.
Denumerable State Markov Decision Processes
-
Average Criterion.
Statist. 37,1545-1554.
M.
(1973).
A
Limit Theorem for Priority Queues in Heavy Traffic.
Appl.
J.
Prob. 10, 907-912.
[6]
Harrison,
M.
J.
(1985).
Brownian Motion and Stochastic Flow Systems. John Wiley
and Sons, New York.
[7]
Harrison,
M.
J.
(1988).
Customer Populations,
Brownian Models
in
of
W. Fleming and
Queueing Networks with Heterogeneous
P. L. Lions (eds.). Stochastic Differential
Systems, Stochastic Control Theory and Applications,
Verlag,
[8]
New
Harrison,
10, Springer-
York, 147-186.
M. and Wein,
J.
IMA Volume
L.
M. (1988). Scheduling Networks
of Queues:
Heavy
Traffic
Analysis of a Two-Station Closed Network. To appear in Operations Research.
[9]
Johnson, D.
cesses
and
P. (1983). Diffusion
for
Approximations
for
Optimal Filtering of Jump Pro-
Queueing Networks. Unpublished Ph.D.
thesis,
Dept. of Mathematics,
Univ. of Wisconsin, Madison.
[10]
Karatzas, L (1988). Stochastic Control Under Finite-fuel Constraints, in
and
P. L.
W. Fleming
Lions (eds.), Stochastic Differential Systems, Stochastic Control Theory and
Applications,
IMA Volume
10, Springer- Verlag,
51
New
York, 225-240.
[11]
Klimov, G.
[12]
Kushner, H.
and
[13]
P. (1974).
Time Sharing
(1977). Probability
J.
for Elliptic Equations.
Kushner, H.
Methods
Academic
(19SS). Numerical
J.
Service Systems
Press,
Methods
for
Approximations
New
Kushner, H.
Manne, A.
6,
[16]
19,532-551.
in Stochastic Control
for Stochastic Control
Brown
Problems
in
U., Providence, R.
ConI.
and Chen, C. H. (1974). Decomposition of Systems Governed by Markov
IEEE
Chains.
[15]
J.
AppL
York.
tinuous Time. Technical Report, Div. Applied Math.,
[14]
Th. Prob.
I.
Transactions Aut. Cont. AC-19, 501-507.
(1960). Linear
Programming and Sequential
Decisions.
Management Science
259-267.
Menaldi,
J. L.,
Robin, M., Taksar, M.
I.
(1988). Singular Ergodic Control for Multi-
dimensional Gaussian Processes. Submitted for publication.
[17]
Peterson,
ple
W.
P. (1985). Diffusion
Approximations for Networks of Queues with Multi-
Customer Types. Unpublished Ph.D. Thesis, Dept.
of Operations Research, Stan-
ford University.
[18]
Reiman, M.
Proc.
I.
(1983).
Some
Diffusion Approximations with State Space Collapse.
Seminar on Modeling and Performance Evaluation Methodology, Springer-
Intl.
Verlag, BerUn.
[19]
Taksar, M.
and a Related Stopping Prob-
(1985). Average Optimal Singular Control
I.
lem. Math, of Operations Research 10, 63-81.
[20]
Wein,
in
[21]
L.
M.
(1988). Optimal Control of a Two-Station
Math, of Operations Research.
Wein,
L.
M.
(1988). Scheduling Networks of Queues:
Station Network
[22]
Whitt,
W.
With Controllable
(1971).
Weak
Preemptive-Resvune Discipline.
[23]
Brownian Network. To appear
Whitt,
W.
(1984).
Inputs.
To appear
Convergence
J.
Appl. Prob.
Open and Closed Models
Laboratories Technical Journal 63, 1911-1979.
52
Heavy
in
Theorems
8,
Traffic Analysis of a
Two-
Operations Research.
for
Priority
Queues:
74-94.
for
Networks of Queues. AT&:T Bell
[24]
P.
Yang, Pathwise Solutions for a Class of Linear Stochastic Systems, Unpublished
Ph. D. Thesis, Dept. of Operations Research, Stanford University, Stanford, CA, 1988.
53
5
9€ a
U
I
8
Date Due
T 20
'^^^
liiiiii iiiiiiiiiii
3
TDflO
DaS70M33
^P^^