Size-Independent vs. Size-Dependent Policies in
Scheduling Heavy-Tailed Distributions
by
John Nham
Submitted to the Department of Electrical Engineering and Computer
Science
in partial fulfillment of the requirements for the degree of
Masters of Engineering
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
May 2008
c Massachusetts Institute of Technology 2008. All rights reserved.
Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Department of Electrical Engineering and Computer Science
May 23, 2008
Certified by . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
John N. Tsitsiklis
Clarence J Lebel Professor of Electrical Engineering, MIT
Thesis Supervisor
Certified by . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Sudhendu Rai
Principal Scientist, Xerox Corporation
Thesis Supervisor
Accepted by . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Arthur C. Smith
Chairman, Department Committee on Graduate Theses
2
Size-Independent vs. Size-Dependent Policies in Scheduling
Heavy-Tailed Distributions
by
John Nham
Submitted to the Department of Electrical Engineering and Computer Science
on May 23, 2008, in partial fulfillment of the
requirements for the degree of
Masters of Engineering
Abstract
We study the problem of scheduling jobs on a two-machine distributed server, where
the job size distribution is heavy-tailed. We focus on two distributions, for which
we prove that the performance of the optimal size-independent policy is asymptotically worse than that of a simple size-dependent policy. First, we consider a simple
distribution where incoming jobs can only be of two possible sizes. The motivation
is that with two largely different sizes, the simple distribution captures the important aspects of a heavy tail. Second, we extend to a bounded Pareto distribution,
which has an actual heavy tail. For both cases, we analyze the performance with
regards to slowdown (waiting time divided by job size) for several size-independent
and size-dependent policies. We see that the size-dependent policies perform better,
and then go on to prove that even the best size-independent policy cannot achieve
the same performance. We conclude that as we increase the variance of our job size
distribution, the gap between size-independent and size-dependent policies grows.
Thesis Supervisor: John N. Tsitsiklis
Title: Clarence J Lebel Professor of Electrical Engineering, MIT
Thesis Supervisor: Sudhendu Rai
Title: Principal Scientist, Xerox Corporation
3
4
Acknowledgments
This thesis was completed as part of the VI-A program between MIT and Xerox. I
would first like to thank VI-A and the VI-A staff for giving myself and others the
opportunity to complete an industry-based thesis. It is a great option for students
who want the real-world experience.
I would like to thank my Xerox supervisor, Sudhendu Rai, for his vision, enthusiasm, and support. I appreciate his interest in my personal development and I learned
a lot from him, not just technically, but also in how to get things done in a company
environment.
I would like to thank my thesis advisor, Professor John Tsitsiklis, for his overall
guidance and patience with me in completing this thesis. His comments were always
insightful, and his attention to detail really helped make this thesis more precise and
correct, both in content and in writing.
Finally, I would like to extend my appreciation to several close ones for their
understanding and support throughout the thesis, especially towards the end.
5
6
Contents
1 Introduction
9
2 Background
11
2.1
Heavy-tailed distributions . . . . . . . . . . . . . . . . . . . . . . . .
11
2.1.1
The Pareto / bounded Pareto (BP) distribution . . . . . . . .
12
2.2
Heavy tails in the real world . . . . . . . . . . . . . . . . . . . . . . .
13
2.3
Heavy tails are hard to analyze . . . . . . . . . . . . . . . . . . . . .
13
2.4
Our problem in standard notation . . . . . . . . . . . . . . . . . . . .
14
2.4.1
14
2.5
Pollaczek-Khinchin formula for M/G/1 . . . . . . . . . . . . .
Harchol-Balter et al.: On Choosing a Task Assignment Policy for a
Distributed Server System [4] . . . . . . . . . . . . . . . . . . . . . .
3 A Simple Problem
15
17
3.1
Description and properties . . . . . . . . . . . . . . . . . . . . . . . .
18
3.2
Poisson arrivals, Stochastic . . . . . . . . . . . . . . . . . . . . . . . .
19
3.2.1
Dynamic policy (D/D for all states) . . . . . . . . . . . . . . .
20
3.2.2
Size-splitting policy . . . . . . . . . . . . . . . . . . . . . . . .
20
3.2.3
Dynamic/Anti-dynamic policy (D/A for all states) . . . . . .
21
3.2.4
Intuition on an optimal policy for slowdown . . . . . . . . . .
22
No arrivals, stochastic and deterministic . . . . . . . . . . . . . . . .
24
3.3.1
Deterministic case
. . . . . . . . . . . . . . . . . . . . . . . .
25
3.3.2
Stochastic case . . . . . . . . . . . . . . . . . . . . . . . . . .
27
3.3
7
4 Size-dependent versus size-independent policies
31
4.1
Proof for the simple problem . . . . . . . . . . . . . . . . . . . . . . .
32
4.2
Extension to bounded Pareto . . . . . . . . . . . . . . . . . . . . . .
34
4.2.1
Bounded Pareto with α = 1 . . . . . . . . . . . . . . . . . . .
36
4.2.2
Proof of BP claim: from random to all size-independent policies 38
5 Future Work
41
5.1
A deeper look into the problems of this thesis . . . . . . . . . . . . .
41
5.2
Relaxation of a constraint . . . . . . . . . . . . . . . . . . . . . . . .
42
5.3
Additional extensions for better modeling the real world . . . . . . .
43
6 Conclusion
45
8
Chapter 1
Introduction
Scheduling research has classically been carried out under the assumption of exponentially distributed job sizes (or service times). However, in some environments, such as
the internet, CPUs, or certain manufacturing and print environments, job sizes have
been shown to instead follow a heavy-tailed distribution. This type of distribution
has extremely high variance and can result in performance which is much worse than
that predicted by scheduling policies designed for exponential distributions [6]. In
addition, research in scheduling in the presence of heavy-tailed distributions has been
difficult because these distributions do not have many of the convenient properties of
exponential ones.
In this thesis, we focus on some simplified problems in order to gain intuition
for more complex ones. Our problem is a simple distributed server model with two
machines. Jobs arrive to a central server where they must be immediately sent to one
of the two FCFS machines for processing. Such a system, but with more machines,
is common in web and manufacturing environments where jobs must be immediately
forwarded because of efficiency reasons or physical constraints. Our goal is to minimize expected job slowdown, defined as the waiting time that a job faces divided by
its size. This metric penalizes delays, more severely for short jobs and less for large
jobs.
We start with a simplified distribution under which job sizes can only take on two
values. The idea is that as the ratio of the two sizes becomes very large, this simple
9
distribution will capture the essential aspects of a heavy-tailed distribution. We then
move to the Pareto distribution, which is an actual heavy-tailed distribution.
For each problem, we consider several policies that can be broadly classified as sizeindependent and size-dependent. As jobs arrive to the scheduler, a size-independent
policy does not take into account the size of the job, while a size-dependent policy
does. Both types of policies take into account the current workloads remaining in
each machine, so even for a size-independent policy, a job’s size is revealed once the
job is scheduled. Our main choice of a size-independent policy will be a dynamic
one, which simply allocates incoming jobs to the machine with the least remaining
workload. For the size-dependent case, we will focus on a size-splitting policy that
assigns each machine half of the job size distribution. Thus, one machine would be
assigned all the smaller jobs, while the other all the larger jobs.
Our finding is that size-independent policies indeed perform worse for the job size
distributions that we consider. For the simple distribution, the size-splitting policy
achieves O(1) slowdown, while we prove that the best possible slowdown that a sizeindependent policy can achieve is O(x), where x is the ratio of the large and small
job sizes. For the bounded Pareto distribution, where z is the maximum possible
√
job size, we show that the size-splitting policy achieves O( ln zz ) slowdown, while a
size-independent policy can do no better than O( (lnzz)2 ).
In summary, Chapter 2 gives background on heavy-tailed distributions and other
relevant research. Chapter 3 introduces our simple problem and shows that sizedependent policies perform better than size-independent ones. Chapter 4 formalizes
the relationship and proves that all size-independent policies perform asymptotically
worse than the size-dependent policies that we saw in chapter 3. Chapter 4 additionally extends the job size distribution to a bounded Pareto distribution and carries
out a similar analysis. Finally, Chapter 5 discusses possible extensions, and Chapter
6 concludes.
10
Chapter 2
Background
In this chapter, we first present the definition and some properties of heavy-tailed
distributions. The bounded Pareto distribution is a particular type of heavy-tailed
distribution that we will use as a job size distribution in this thesis. We then discuss some environments where heavy-tails are often seen, and explain why research
progress in this area is hard. Finally, we review some existing results that we will later
build upon. First, the Pollaczek-Khinchin formula calculates expected waiting time
for a single machine under a general job size distribution. Second, Harchol-Balter et
al. [4] compares various scheduling policies for a multi-machine server under various
levels of job size variability. They show that as job size variability increases, a sizesplitting policy will outperform other size-independent policies. We use a two-machine
version of their size-splitting policy in this thesis.
2.1
Heavy-tailed distributions
A heavy-tailed distribution is one that drops off slower than an exponential distribution [4]. As x tends to infinity, it satisfies P r[X > x] = cx−α , for some α, where
0 < α < 2, and some constant c. In contrast, an exponential distribution has a
dropoff of the form ecx as x tends to infinity, where c < 0. Three significant properties distinguish a heavy-tailed distribution from an exponential distribution:
• decreasing failure rate. For a job with a service time drawn from a heavy-tailed
11
distribution, the longer that the job has run, the longer the expected remaining
service time will be. For an exponential distribution, the expected remaining
service time is constant regardless of how long the job has already run.
• infinite variance, so that the Central Limit Theorem does not apply.
• a small percentage of all jobs constitute a large percentage of the total load.
In addition, for α < 1, a heavy-tailed distribution has an infinite mean. A heavy-tailed
distribution is also commonly referred to as a long-tailed or fat-tailed distribution.
2.1.1
The Pareto / bounded Pareto (BP) distribution
The Pareto distribution is a specific form of a heavy-tailed distribution. Its form is
P r[X > x] = c( xxm )−α for all x > xm , where xm is the minimum value in the support
of the distribution. In the real world, we do not expect a distribution to have infinite
variance or mean. Thus, in this thesis, we use a bounded Pareto (BP) distribution,
which adds an upper bound z. One can think of the pdf of a BP as the pdf of a
Pareto distribution with values greater than z set to zero, and values between xm and
z normalized by a constant factor. With large z, the variance of a BP is still extremely
Figure 2-1: A Pareto distribution. Adding an upper bound z turns it into a bounded
Pareto distribution.
12
large and the distribution mostly behaves as a heavy-tailed one. For example, with a
BP distribution, the probability of an observation that is many standard deviations
from the mean is still significant.
2.2
Heavy tails in the real world
Recent analysis of data has shown that service times in a wide array of environments
are heavy-tailed as opposed to exponential. For example, in [6], Leland et al. analyzed
hundreds of millions of data to demonstrate that Ethernet traffic is statistically selfsimilar, a mark of a heavy-tailed distribution. This means that the appearance of
’bursts’ of traffic looked the same on a millisecond scale as on a second, minute, or
hourly scale. Crovella et al. came to the same conclusions about several aspects of
WWW traffic such as file transfer sizes [2]. Harchol-Balter et al. found heavy-tailed
distributions in UNIX process lifetimes [5]. In all of these settings, it was observed
that the distribution followed a heavy tail, with α fairly close to 1, until some large
value where the distribution dropped off more sharply.
2.3
Heavy tails are hard to analyze
Extensive scheduling research has been performed for exponentially distributed task
sizes in many types of systems. Unfortunately, many of these results do not apply to
their heavy-tailed counterparts, and instead tend to give overly optimistic results. As
mentioned above, heavy-tailed distributions lack several key properties of exponential
distributions. One common culprit with heavy tails is that small jobs often get caught
behind large jobs and suffer long delays [1]. Because of this, an analysis based on
exponential distributions can severely underestimate packet losses in a network [6] or
the average queue size of a server [1].
On the practical side, simulations involving heavy-tails tend to be difficult. These
simulations converge to the steady state slowly, and exhibit relatively high variability
even when close to steady state [3]. Intuitively, the reason is that a small percentage
13
of the largest jobs can still have a significant impact on the results. However, these
jobs arrive so infrequently that for a long period of time in the beginning, a small
deviation from the expected number of large jobs causes large inaccuracies in the
results.
2.4
Our problem in standard notation
Our problem considers a two-machine system with Poisson arrivals, and iid job sizes
that are independent from the arrival process. The two job size distributions that
we will consider are a simple distribution that we define later and a bounded Pareto
distribution. In either case, we are dealing with an M/G/2 system. To our knowledge,
an optimal policy is not known for M/G/2 (or even M/M/2) where jobs sizes are
known when arriving at the scheduler. When all job sizes are unknown (including
those already scheduled), Winston [8] has proven that Shortest-line assignment, which
assigns a job to the machine with the least number of jobs remaining, is optimal for an
M/M/m system. However, Whitt [7] then proved that as the variability of the job size
distribution grows (changing the model from M/M/m to M/G/m), the Shortest-line
policy is no longer optimal.
2.4.1
Pollaczek-Khinchin formula for M/G/1
For an M/G/1 system, all jobs simply go to a single machine. The Pollaczek-Khinchin
formula gives the expected waiting time, E[W ], that a job is expected to face. For
the M/G/1 system, we define λ as the arrival rate, ρ as the load (the work rate
required to service all jobs divided by the total throughput of the system), and E[X]
and E[X 2 ] as the first and second moments of the generalized job distribution. The
expected waiting time is then
E[W ] =
λE[X 2 ]
2(1 − ρ)
14
(2.1)
Since we can relate the load and arrival rate through ρ = λE[X], we can rewrite the
expected waiting time expression as
E[W ] =
ρE[X 2 ]
2(1 − ρ)E[X]
(2.2)
The metric we focus on in this thesis is slowdown, which is the waiting time a job
faces divided by the job’s size. The expression for expected slowdown is
E[S] = E[
W
E[W ]
ρE[X 2 ]
]=
=
X
E[X]
2(1 − ρ)E[X]2
(2.3)
The second equality follows because in a single-machine FCFS server, the waiting
time that a job faces is independent of the job size.
2.5
Harchol-Balter et al.: On Choosing a Task Assignment Policy for a Distributed Server System [4]
Harchol-Balter et al. study an M/BP/m distributed server system, testing various
scheduling policies to see which perform the best under different values of α in the
bounded Pareto distribution. Their metrics are waiting time, queue length, and
slowdown, and their results are consistent for each type of metric.
The paper tests four different scheduling policies: (1) random, (2) round-robin,
where jobs are assigned to one machine after the other, (3) dynamic, where jobs
are assigned to the machine with the least work remaining, and (4) SITA-E, a sizesplitting policy where machines are assigned a specific size range, and jobs are assigned
to machines based on their size (see Figure 2-2).
Their main finding is that the dynamic policy performs as well as SITA-E for α
close to 2 (less variability), but as α decreases (higher variability), SITA-E performs
exponentially better. (Random and round-robin do worse than both under all α’s.)
To explain their findings, they invoke Wolff’s approximation [9], which relates the
15
slowdown of a system to the the square of the coefficient of variation of the job size
distribution. By splitting the job sizes into chunks, the coefficient of variation seen
by each individual machine is minimized.
We note that the only policy that does not perform exponentially worse as α
decreases is the size-dependent policy, SITA-E. In this thesis, our goal will be to show
the same effect on a simpler problem, but to also formalize and prove the relationship
between size-independent and size-dependent policies.
Figure 2-2: The SITA-E policy chunks the job size distribution for each machine.
16
Chapter 3
A Simple Problem
In this chapter, we explore the simple problem of scheduling jobs to two machines,
where the job sizes are drawn from a simplified distribution. We look at several
versions of the problem, divided along two axes. First, the problem can either be
stochastic or deterministic, where the sizes of jobs yet to be scheduled are unknown
or known, respectively. Second, jobs can arrive as Poisson arrivals, or all at time 0
(no arrivals). Figure 3-1 shows the section where we cover each case. We do not look
at the case with deterministic job sizes and Poisson arrivals.
For the case of Poisson arrivals and stochastic job sizes, we are unable to find
a closed-form optimal policy with regards to slowdown, but observe a trend where
size-independent policies perform asymptotically worse than size-dependent policies.
For the case of no arrivals, both stochastic and deterministic, we use dynamic pro-
Figure 3-1: Section where we cover the different versions of our simple problem.
17
gramming to find an optimal policy. Both optimal policies often make a different
assignment for small jobs than it does for large jobs. Our conclusion, which will be
expanded on in the next chapter, is that as the variability of our job size distribution
increases, the penalty for using a size-independent policy also increases.
3.1
Description and properties
We define a Simple distribution as one that can only take on two possible values.
Without loss of generality, we let the smaller size be 1 and the larger size be x. Finally,
we also require that the expected load coming from each size to be the same. This
means that the arrival rate for size 1 jobs should be x times the arrival rate for size x
jobs. The Simple distribution emulates a heavy-tailed distribution with α = 1, since
the probability of seeing a job of size x is of order x−1 .
Our simple problem will then be to schedule jobs with sizes drawn from the Simple
distribution to a two-machine server. When a job arrives at the server, it must be
immediately assigned to one of the two machines, each of which processes the jobs
in a FCFS manner at a processing rate of 1. Job arrivals are Poisson with arrival
rate λ. Since we know the relationship between the number of size 1 jobs and size x
jobs, we can think of the incoming job stream as a combination of size 1 jobs that
arrive as Poisson arrivals with arrival rate
arrivals with arrival rate
λ
.
x+1
λx
,
x+1
and size x jobs that arrive as Poisson
We can also calculate the load ρ of the system to be
ρ = 21 λE[X], where E[X] is the expected job size. The
1
2
is present in the expression
because with two machines, the workload throughput is 2.
The first and second moments of the Simple distribution are:
x
1
2x
·1+
·x=
x+1
x+1
x+1
(3.1)
x
1
x2 + x
· 12 +
· x2 =
x+1
x+1
x+1
(3.2)
E[X] =
E[X 2 ] =
For large x, these expressions converge to: E[X] = 2 and E[X 2 ] = x.
As a reference point, we use the Pollaczek-Khinchin formula (2.3) to calculate the
18
expected slowdown (waiting time / job size) E[S] of the Simple distribution on a
single machine.
E[
x
1 1
x2 + 1
1
]=
(1) +
( )= 2
= 1 for large x
X
x+1
x+1 x
x +x
E[S] = E[W ]E[
1
ρE[X 2 ]
ρx
]=
=
X
2(1 − ρ)E[X]
4(1 − ρ)
(3.3)
(3.4)
If we fix ρ to a constant, the expected slowdown is O(x).
3.2
Poisson arrivals, Stochastic
We first look at the general case of stochastic job sizes and Poisson arrivals, where we
want a policy that minimizes expected slowdown. A policy is a function f (s, w0 , w1 )
that takes in inputs s, the size of the job, w0 , the remaining workload at machine 0,
and w1 , the remaining workload at machine 1. The policy returns 0 or 1 to represent
scheduling the job to machine 0 or machine 1. We define a size-dependent policy as
one that takes into account s when deciding where to schedule the job, and a sizeindependent policy as one that does not take into account s. Note that we can think
of (w0 , w1 ) as the state of the system because those two values are all that is needed
to completely describe the system at any point in time.
Since it is hard to find a closed-form optimal policy, we analyze three specific
ones, including (1) a dynamic policy, which assigns the incoming job to the machine
with lesser remaining workload, (2) a size-splitting policy, which assigns all the size
1 jobs to machine 0, and all the size x jobs to machine 1, and (3) a third policy that
incorporates both dynamic and size-based elements. Note that the dynamic policy
is size-independent, while the size-splitting and third policy are both size-dependent.
We show that the dynamic policy achieves O(x) slowdown, while both size-dependent
policies achieve O(1) slowdown.
Above, we defined a policy as a function that returns 0 or 1 to represent the
machine to assign the incoming job. We now propose an alternate way of representing
a policy’s return value. We define assigning an incoming job to the machine with lesser
19
remaining workload to be the dynamic (D) choice, and assigning an incoming job to
the machine with greater remaining workload to the be the anti-dynamic (A) choice.
For each state (w0 , w1 ), we can represent our policy as */*, where * can be D or
A, and the * before the / represents our choice for size 1 jobs, and the * after the
/ represents our choice for size x jobs. For example, the dynamic policy described
above could be described as D/D for all states, since we make the dynamic choice for
both size 1 jobs and size x jobs for all states.
3.2.1
Dynamic policy (D/D for all states)
The dynamic policy assigns jobs to the machine with lesser remaining workload.
Intuitively, we can see why making the D choice for all jobs would be bad in our
simple problem. Suppose that a job of size x arrives when the two machines are
both close to empty, and is placed on machine 0. If a second job of size x arrives
shortly afterwards, the dynamic strategy will put it on machine 1. In this case, over a
subsequent O(x) time period, every arriving job of size 1 will face an O(x) slowdown.
Instead, we would have wanted the second size x job to also go on machine 0 (the
A choice) so that subsequent size 1 jobs could go on a nearly empty machine 1.
While the chance of two large jobs arriving close together is small for an exponential
distribution (since the probability of seeing a large job is small), that is not the case
for the Simple distribution.
3.2.2
Size-splitting policy
Size-splitting is a state-independent policy that simply assigns jobs of size 1 to machine 0, and jobs of size x to machine 1. For the dynamic policy, the poor slowdown
can be attributed to assigning size 1 jobs behind size x jobs, and size-splitting removes
that possibility by separating the different sized jobs.
Given that ρ is the total load of the system, we can calculate the expected slowdown of all jobs by considering each machine separately. Because the Simple distribution has a balanced load between the two job sizes, both machines see their jobs
20
arriving in a Poisson process at a rate that makes its load equal to ρ. The expression
for expected slowdown for a single machine with constant job sizes is
ρ
.
2(1−ρ)
Thus,
given a constant ρ, both machines have the same constant slowdown.
One drawback of this policy is that it relies on the two loads being roughly the
same. If either job size dominated the total load, then the corresponding machine
could have a load greater than 1 and be unstable.
3.2.3
Dynamic/Anti-dynamic policy (D/A for all states)
We combine concepts from the previous two policies to suggest a third policy that
always makes the D choice for size 1 jobs, and the A choice for size x jobs. With this
policy, size 1 jobs will rarely get trapped behind size x jobs. However, it still suffers
from the same drawback as the size-splitting policy - if the load coming from size x
jobs is much greater than the load coming from size 1 jobs, then the system becomes
unstable.
Note that the D/A policy is a completely size-dependent policy. At every decision
point, it will choose the opposite machine for the two job sizes.
Because it is hard to assess this strategy analytically, we do so through simulation.
The following table (3.1) compares this policy with the dynamic policy when ρ =
0.8, under various values of x. Note that the size-splitting policy always achieves a
slowdown of
ρ
2(1−ρ)
= 2.
Table 3.1: Slowdown of D/D and
x
D/D
1
0.9
1.5
0.98
2
1.13
3
1.5
5
2.3
7
3.2
10
4.4
20
8.5
100
42
10000 3000
21
D/A policies with ρ = 0.8
D/A
2
1.65
1.5
1.4
1.38
1.39
1.42
1.48
1.59
1.6
We see that while D/D gives a better slowdown for small x, it quickly gets
overtaken by D/A as x becomes larger, with the turning point being around just
x = 2 or 3. We also notice the pattern that D/D will generally give an O(x) slowdown, while D/A gives a constant slowdown under 2.
Other brief simulations show this same pattern across different loads. In summary,
the size-independent (dynamic) policy achieved an O(x) slowdown, while the two
size-dependent policies (size-splitting and D/A) achieved an O(1) slowdown. This
raises the hypothesis that any size-independent policy will give a slowdown that is
asymptotically worse than that of the simple size-dependent policies described here.
The hypothesis makes intuitive sense because other size-independent policies must
suffer from the same defect described for the dynamic policy. We will solidify this
idea in the next chapter.
3.2.4
Intuition on an optimal policy for slowdown
We have just analyzed several specific policies for the simple problem because we
could not find an optimal one. We now discuss, at least intuitively, why this may not
be an issue.
To understand what an optimal policy may be, we consider it in comparison to
our most successful strategy, D/A. We first discuss the two ways in which the optimal
policy can deviate from D/A, but then show that as x becomes large, deviations from
D/A will occur infrequently. For finite x, the two possible deviations are:
• Policy D may not be best for size 1 jobs in all states. Consider our simple
problem with x = 50, and at a state where the remaining workloads at the two
machines are fairly high and equal. Suppose that the next sequence of jobs is:
50 1’s, 1 x, and 50 1’s. The D/A policy would place 25 1’s on each machine,
the x on one machine, and the final 50 1’s on the other. However, if we knew
that this sequence was to arrive, the optimal policy would be to ’stack’ all 50
1’s on one machine, put the x on top of that stack, and then put the final 50
1’s on the other machine (see Figure 3-2). In the no arrivals case analyzed in
22
Figure 3-2: The optimal policy on this particular input actually schedules the size 1
jobs in group A anti-dynamically.
the next section, the optimal policy often makes the A choice for size 1 jobs for
variations of this situation. However, for the arrivals case, it is hard to analyze
exactly when this situation would cause a deviation from the D choice for size
1 jobs.
• Policy A may not be best for size x jobs in all states. We have already observed
through simulation that for small x, the D choice is better.
Despite these two possible deviations, we can show that for large x, D/A will still
be the correct policy at many decision points. Consider the fact that our system can
only be in one of two modes: (1) no job of size x is queued at either machine, and
(2) there is at least one job of size x queued in our system. We know that our system
must be in mode (2) for a fraction of time that scales with the load, and that in mode
(2), the difference between the two remaining workloads will be O(x) for most of the
time.
For states where the difference between the two remaining workloads is O(x),
the optimal policy must be D/A for most of the time. For these states, making
the A choice for size 1 jobs will incur an O(x) greater slowdown than the D choice.
Since we already know that the size-splitting policy gives an O(1) average slowdown,
the optimal policy must only assign a job to have O(x) slowdown very infrequently.
Similarly, when assigning a size x job, making the D choice will cause all subsequent
size 1 jobs to have an O(x) slowdown for the next O(x) timespan. Thus, both
deviations must happen infrequently in order for the optimal policy to maintain an
23
O(1) average slowdown.
We noted above that D/A is a completely size-dependent policy. Our conclusion
from this section is that because the optimal policy is close to D/A as x becomes
larger, any size-independent policy will expect to make more and more mistakes as x
becomes larger.
3.3
No arrivals, stochastic and deterministic
Before moving on to the size-independent vs. size-dependent discussion, we briefly
explore our simple problem for the case of no arrivals. We find an optimal policy
with regards to slowdown, and show that it also often makes different decisions for
size 1 and size x jobs in any particular state. This supports our theory that a sizeindependent policy would perform poorly for this problem.
In the real world, the no arrivals case could represent any scenario with a buildup
of orders, such as a manufacturing environment where orders are taken overnight to
be processed the next day. If the preparation of materials takes up ample space, the
jobs would still be constricted to be scheduled in a FCFS order.
We again consider our simple problem with the modification that all jobs arrive
at time 0. Because of this, we must have a predefined number of jobs to schedule, n.
We use dynamic programming to find an optimal policy for both the deterministic
case, where we know the size of every job from the beginning, and the stochastic
case, where we do not know the sizes of jobs yet to be scheduled. For both cases, we
know the sizes of jobs already scheduled and the remaining workload at each machine.
While the deterministic case correlates better with the real world example described
above, we find it easier to generalize the optimal policy found for the stochastic case.
The key to maintaining a manageable number of states in the dynamic programming is the observation that at any point in our scheduling, the workload at either
machine is a linear integer combination of 1 and x. This is because we are scheduling
all jobs at time 0.
We find below that the dynamic programming algorithm runs in O(n3 ) time in
24
the deterministic case, and O(n4 ) in the stochastic case. For both cases, the optimal
policy most often uses a strategy of ’stacking’ jobs in order to place the size x jobs
on a machine with as much remaining workload as possible.
3.3.1
Deterministic case
In the deterministic case, we know the sizes of all jobs in advance. We make the
following definitions:
• n: the total number of jobs
• n1 , nx : the total number of size 1 and size x jobs, n = n1 + nx
• sk : the size of the kth job, where s1 is the size of the first job
At the point in time when we schedule the kth job, we can additionally make these
definitions:
• n1k , nxk : the total number of size 1 and size x jobs scheduled when job k arrives
at the scheduler, k − 1 = n1k + nxk
• w0k , w1k : the remaining workload at machine 0 and 1 when job k arrives at the
scheduler
Because jobs only come in sizes 1 or x, we can express w0k and w1k as linear integer
combinations of 1 and x. Thus, we can write w0k = ak + bk x and w1k = ck + dk x,
where ak is the number of size 1 jobs assigned to machine 0 when job k arrives at
the scheduler, and bk is the number of size x jobs assigned to machine 0 at the same
point in time. ck and dk hold the same meaning for machine 1. We can also relate
n1k = ak + ck and nxk = bk + dk .
When job k arrives at the scheduler, we define the cost-to-go (CTG) function,
J, as the minimum sum of slowdowns required to schedule job k and all subsequent
jobs. The CTG function takes as input the remaining workload of the two machines,
which we will express with the tuple (a, b, c, d) having the meaning described above.
Thus, as an example, if we are scheduling the 5th job (k = 4) when there are two
25
size 1 jobs and zero size x jobs on machine 0, and one size 1 job and one size x job
on machine 1, we express the CTG function as J4 (2, 0, 1, 1).
When job k arrives at the scheduler, it can go to either machine 0 or 1. Thus,
we calculate CTG function of job k to be the lesser of (1) [slowdown incurred when
assigning job k to machine 0] + [CTG of job k + 1 given that job k was assigned to
machine 0] and (2) the same for machine 1.
Starting from the last job and working backwards, we can compute the CTG
functions for every job. The CTG function for the last job is easily determined
because we do not have to consider any future jobs.
Jn−1 (a, b, c, d) =
1
sn−1
(min[a + bx, c + dx])
(3.5)
For every other job, the CTG function is
Jk (a, b, c, d) =



min[(a + bx) + Jk+1 (a + 1, b, c, d), (c + dx) + Jk+1 (a, b, c + 1, d)], sk = 1


min[ a+bx
+ Jk+1 (a, b + 1, c, d), c+dx
+ Jk+1 (a, b, c, d + 1)],
x
x
sk = x
(3.6)
To run the dynamic programming algorithm, we need to find the CTG of all possible
(a, b, c, d) combinations at each job. For the kth job, we know that there are already
n1k jobs of size 1 and nxk jobs of size x distributed between the two machines. We
need to calculate the CTG for all combinations of (a, c) where n1k = a + c and (b, d)
where nxk = b + d. Thus, for the last job, the worst case number of CTG functions
we would have to calculate is O(n2 ) (although we expect nxk to be 1/x of n1k ). Over
n jobs, the worst case runtime is O(n3 ). After calculating all the CTG functions, we
schedule each job to the machine that gives the CTG value.
Simulation results show that the optimal policy generally follows a simple strategy
(with minor deviations). The policy works in two main phases (see Figure 3-3).
During phase 1, it places every job, regardless of size, on one machine. During phase
2, it places size x jobs on the ’stacked’ machine, while placing size 1 jobs on the
other machine. Near the very end, some size x jobs may go on the shorter machine.
With x=10 and n=200, the transition between phase 1 and 2 generally comes after
26
Figure 3-3: Phase 1 assigns jobs using A/A, while Phase 2 uses D/A (with a few
exceptions near the end).
approximately the first third of all jobs. The intuition for this policy is that it is
beneficial to place the size x jobs behind as much work as possible, so that the size 1
jobs can be placed behind less work. The unbalanced load from phase 1 allows this
to happen in phase 2.
We also note that the optimal policy can be described using our notation as A/A
in phase 1, and D/A in phase 2. Because most of the jobs belong to phase 2, this
policy chooses a different assignment for size 1 and size x jobs most of the time.
3.3.2
Stochastic case
In the stochastic case, we do not know the sizes of the jobs yet to arrive at the
scheduler. In our notation above, this means that we do not know what n1k and nxk
will be in advance. Thus, when running our dynamic programming algorithm, we will
have to calculate the CTG function for all possible combinations of n1k + nxk = k − 1
at each job k. With this additional dimension, the worst case runtime becomes O(n4 ).
However, because n, x, and ρ are the only inputs to the dynamic programming, the
algorithm only needs to be run once.
The CTG equations are very similar to the deterministic case, but because we do
not know what sk will be, the value of the CTG function must be a weighted sum
involving both possibilities: sk = 1 and sk = x. In words, the CTG function of job
k is
x
x+1
( minimum of (1) slowdown incurred when assigning a size 1 job to machine
27
0 + CTG of job k + 1 given that job k was of size 1 and went on machine 0 and
(2) slowdown incurred when assigning a size 1 job to machine 1 + CTG of job k + 1
given that job k was of size 1 and went on machine 1 ) +
1
x+1
( minimum of the same
expressions for size x jobs ).
Again, we start from the last job and work backwards. The CTG function for the
last job is now
Jn−1 (a, b, c, d) =
x
1
a + bx c + dx
min[a + bx, c + dx] +
min[
,
]
x+1
x+1
x
x
(3.7)
For every other job, the CTG is
Jk (a, b, c, d) =
x
x+1
where
one0 = a + bx + Jk+1 (a + 1, b, c, d)
min[one0 , one1 ] +
1
x+1
min[x0 , x1 ]
one1 = c + dx + Jk+1 (a, b, c + 1, d)
x0 =
a+bx
x
+ Jk+1 (a, b + 1, c, d)
x1 =
c+dx
x
+ Jk+1 (a, b, c, d + 1)
(3.8)
This time, to run the dynamic programming algorithm for job k, we need to
calculate the CTG of every combination of (a, b, c, d) and every combination of (n1k ,
nxk ) that satisfy n1k + nxk = k, n1k = a + c, and nxk = b + d.
The results of the dynamic programming are very similar to those for the deterministic case. The optimal policy works in two phases, with Phase 1 implementing
an A/A policy, and Phase 2 implementing a D/A policy. For n = 50, x = 10, and
ρ = 0.8, table 3.2 shows the CTG values at various states. We walk through the table
line by line.
Table 3.2: CTG values of assigning
Size 1
(a,b,c,d) m0 m1
(1,0,0,0) 637 650
(10,1,0,0) 578 582
(20,2,0,0) 343 327
28
jobs in various states
Size x
m0 m1
755 775
594 828
309 548
(1,0,0,0) represents the state where there is one job of size 1 on machine 0, and
nothing on machine 1. If the next job to be scheduled is of size 1, then the CTG
(expected sum of slowdowns of all remaining jobs) of putting it on machine 0 is 637,
while the CTG of putting it on machine 1 is 650. Thus, it is better to make the A
choice for a job of size 1 and put it on machine 0. For a size x job, it is also better to
make the A choice and place the job on machine 0. Here, we are beginning the A/A
policy of Phase 1.
(10,1,0,0) represents the state where there are ten size 1 jobs and one size x job
on machine 0, and nothing on machine 1. For a size 1 job, it is still slightly better
to make the A choice. For a size x job, the importance of making the A choice has
grown significantly. Here, we are towards the end of Phase 1.
(20,2,0,0) represents the state where there are twenty size 1 jobs and two size x
jobs on machine 0, and nothing on machine 1. Now, it is better to make the D choice
for size 1 jobs, while the A choice for size x jobs is still by far the better. Here, we
have entered Phase 2 and its D/A policy.
In summary, our analysis of this simple problem has led to the observation that
(1) certain size-independent policies perform asymptotically worse than certain sizedependent ones, by a factor related to the variability of the job size distribution, and
(2) the optimal policy, or our best guess as to what is optimal, often suggests the
opposite actions for size 1 and size x jobs. Both observations hint at the poorness of
size-independent policies, which we will try to analyze further in the next chapter.
29
30
Chapter 4
Size-dependent versus
size-independent policies
In the stochastic version of our simple problem with Poisson arrivals, we found that
the dynamic policy achieved O(x) slowdown, while the size-splitting policy achieved
O(1) slowdown. We attributed the poor performance of the dynamic policy to the
fact that it would often allow size 1 jobs to get stuck behind size x jobs, and noted
that this flaw would likely be present in all size-independent policies. Thus, we now
solidify that intuition, and prove that every size-independent policy can only achieve
O(x) slowdown.
After doing so, we extend our job size distribution from the Simple distribution to
a bounded Pareto distribution with α = 1. Our metric will once again be slowdown.
Using the same pattern of analysis as for the Simple distribution, we will (1) show that
a simple size-dependent policy (size-splitting) outperforms a simple size-independent
policy (random), and (2) prove that no size-independent policy can match the performance of the size-splitting policy. In particular, with z being the upper bound
√
of our bounded Pareto, the size-splitting policy achieves O( ln zz ) slowdown, while the
lower bound for any size-independent policy is O( (lnzz)2 ) slowdown, which is a factor
√
of O( ln zz ) worse.
31
4.1
Proof for the simple problem
Claim: a size-independent policy can do no better than O(x) slowdown for the
stochastic simple problem with Poisson arrivals.
We prove this claim by presenting a sequence of events that has a constant
(bounded away from zero) probability of occurring, and showing that the expected
slowdown of jobs during the sequence is O(x) if the scheduling policy is size-independent.
It follows that the expected slowdown is O(x).
We define our sequence to take place over a 3x/4 timespan, separated into 3 phases
of x/4 timespan each: (Phase 1) at least one job of size x arrives, (Phase 2) at least
one job of size x arrives, and (Phase 3) no restrictions.
There are two steps to our proof. First, we need to show that this sequence occurs
with a constant, non-zero probability starting from an arbitrary time. Second, we
need to show that during this sequence, the expected slowdown is O(x).
We start with the first step. From the definition of our simple problem, we know
that jobs of size x arrive as a Poisson process with arrival rate
λ
.
x+1
The probability
of seeing no arrivals from a Poisson process with arrival rate λ over a timespan τ is
e−λτ . Thus, the probability of seeing one or more jobs with an arrival rate of
−λ x
λ
x+1
over
λ
a timespan of x/4 is 1 − e( x+1 4 ) . For large x, this probability converges to 1 − e(− 4 ) ,
which is a non-zero constant. Since the conditions for each phase are independent,
the probability of the entire sequence is this value squared.
Our second step is to show that the expected slowdown during this sequence is
O(x) when using a size-independent policy. For this purpose, it will suffice to show
that an expected constant fraction of these jobs have an O(x) slowdown. We consider
each phase in order, and show that the sequence forces either the jobs in the second
or third phase to have an expected O(x) slowdown.
For phase 1, we simply establish that because a job of size x has arrived within
this x/4 timespan, we enter phase 2 with at least one machine having at least 3x/4 remaining workload. Without loss of generality, we assume this machine to be machine
0.
32
For phase 2, we examine the possible size-independent policies. We know that at
least one job of size x will be assigned in this phase. Thus, on a particular simulation,
any policy in general must either (1) assign all size x jobs in this phase to machine
0 or (2) assign at least one job of size x to machine 1. For any policy, either case
(1) will occur with probability at least 1/2, or case (2) will occur with probability at
least 1/2. We analyze both cases for a size-independent policy.
Case 1: the policy assigns all size x jobs in this phase to machine 0 with probability
at least 1/2. Since a size-independent policy does not know the size of a job when it
is scheduled, it must, on average, assign at least 1/2 of all jobs in phase 2 to machine
0 for it to fit into this case. 1/2 of the jobs in phase 2 represents a constant fraction
of all jobs in the sequence. Since
x
x+1
of the jobs assigned to machine 0 will be of
size 1 in expectation, and all of these jobs will face at least a x/2 waiting time, an
expected constant fraction of all jobs in the sequence have an O(x) slowdown. This
concludes our proof for size-independent policies under this case.
Case 2: the policy assigns at least one job of size x to machine 1 with probability
at least 1/2. In this case, that with probability at least 1/2, we will enter phase 3
with both machines having at least x/2 remaining workload.
For phase 3, assuming that we arrived from case 2 above,
x
x+1
of arriving jobs are
expected to be of size 1, and they will all have at least x/4 slowdown. Since these
jobs make up a constant fraction of all jobs, and they all have O(x) slowdown, then
jobs in the entire sequence expect to have O(x) average slowdown.
Thus, we have shown that with any size-independent policy, starting from an
arbitrary point in time, jobs arriving in the next 3x/4 timespan will face an O(x)
average slowdown with constant probability. Note that since we have already shown
a random policy to achieve O(x) slowdown, we do not expect to prove a stronger
lower bound.
33
4.2
Extension to bounded Pareto
We now consider the same two-machine system, but with the job sizes drawn from a
bounded Pareto (BP) distribution with α = 1 instead of the Simple distribution. We
use α = 1 because the BP satisfies P r[X > x] = xc , which is similar to our Simple
distribution where the probability of seeing a job of size x is x1 . In this section, we
analyze a random policy and a size-splitting policy to see that the size-splitting one
performs better. In the next section, we will prove that no size-independent policy
can outperform the size-splitting one.
We define a BP distribution with the tuple (α, xm , z), where xm and z are the
lower and upper bounds of the distribution respectively. We assume that xm z.
To determine the probability density function (pdf) of a BP distribution, we first
recall that an ordinary Pareto distribution (without the upper bound z), satisfies
P r[X > x] = c( xxm )−α . We can think of the pdf of a bounded Pareto as the same as
that of the Pareto distribution, except that: (1) for all x > z, the pdf is 0, and (2)
for all xm < x < z, the pdf is increased by some factor k so that the pdf integrates
to 1. Thus, we can derive the pdf of a bounded Pareto distribution by looking at the
pdf of a Pareto distribution and making those two changes.
For a Pareto distribution, the pdf is calculated as follows (for x > xm ):
P r[X > x] = c(
x −α
)
xm
(4.1)
x −α
)
xm
(4.2)
cdfP (x) = 1 − c(
pdfP (x) =
d
cdfP = cαxαm x−α−1
dx
(4.3)
For a BP distribution, we (1) set the pdf for x > z to be 0, and (2) increase the
pdf for xm < x < z by a factor k. k is the inverse of the integral of the pdf of the
Pareto distribution for xm < x < z.
Z z
−1
1
−α
α
) = O(1)
= cαxm
x−α−1 dx = cαxm α ( )x−α |zxm = cαxαm (x−α
m −z
k
α
xm
34
(4.4)
The last equality follows because we have specified that xm z, so the value between
the parentheses can be treated as x−α
m . Thus, both k and c are constants that we can
combine in later expressions. This means that the pdf for a BP distribution follows
the same asymptotic form as the pdf for a Pareto distribution. Here, we combine the
constants for simplicity.
pdfBP (x) = O(1)xαm x−α−1
(4.5)
Next, for reference, we use the Pollaczek-Khinchin formula to derive the expected
slowdown of jobs drawn from a BP distribution and processed using a single machine.
With a constant load of ρ, we derive different expressions for each of three cases:
α < 1, α = 1, and 1 < α < 2. We first calculate E[X], E[X 2 ], and E[1/X].
E[X] = O(1)xαm
Z z
xx−α−1 dx
(4.6)
xm
We treat the α = 1 case separately because x−1 will integrate to ln x.
E[X] = O(1)xm ln x|zxm = O(1)xm (ln z − ln xm ) = O(xm ln z),
α=1
(4.7)
For α < 1 and 1 < α < 2,


 O(xα z 1−α ), α < 1
xαm
m
α
−α+1
−α+1
−α+1 z
E[X] = O(1)
−xm ) = 
x
|xm = O(1)xm (z
−α + 1
 O(x ),
1<α<2
m
(4.8)
The last equality follows because with α < 1, the expression inside the parentheses
tends to z 1−α , while with 1 < α < 2, it tends to x−α+1
.
m
E[X 2 ] and E[1/X] each have the same expression for all three cases.
E[X 2 ] = O(1)xαm
Rz
xm
x2 x−α−1 dx = O(1)xαm x−α+2 |zxm = O(1)xαm (z −α+2 − x−α+2
)
m
= O(xαm z 2−α )
(4.9)
E[1/X] =
R
O(1)xα z
m xm
x−1 x−α−1 dx = O(1)xαm x−α−1 |zxm = O(1)xαm (z −α−1 − x−α−1
)
m
= O(xαm x−α−1
) = O(1/xm )
m
(4.10)
35
We can now plug these expression into the formula for expected waiting time.
E[W ] =
=
λE[X 2 ]
2(1−ρ)













=
ρ E[X 2 ]
E[X] 2(1−ρ)
α
2−α
z
O(1) xxm
α z 1−α = O(z),
O(1)
O(1)
m
x1m z 2−1
xm ln z
2−α
xα
mz
xm
=
2
]
= O(1) E[X
E[X]
O( lnzz ),
α<1
(4.11)
α=1
2−α
= O(xα−1
), 1 < α < 2
m z
Essentially, as α decreases from 2 to 0, the expected waiting time starts at O(xm ),
gradually builds to O(z) when α = 1, and levels off there.
Finally, we calculate expected slowdown, which follows the same pattern as the
waiting time.
E[S] = E[W ]E[1/X] =













4.2.1
O( xzm ),
α<1
O( xmzln z ),
α=1
(4.12)
2−α −1
xm ) = O(( xzm )2−α ), 1 < α < 2
O(xα−1
m z
Bounded Pareto with α = 1
Using these equations, we calculate the slowdown of some simple policies on the twomachine problem with a BP(1, 1, z) incoming job size distribution. We analyze both
a size-independent policy (random) and a size-dependent policy (size-splitting), and
find that the random policy achieves O( lnzz ) slowdown, while the size-splitting policy
√
achieves O( ln zz ) slowdown.
Random policy
For the random policy, both machines still see a BP(1, 1, z) distribution at the same
load ρ as the entire system. From the expression for slowdown for α = 1 from the
previous section, we can calculate that jobs from each machine experience an expected
slowdown E[S] = O( lnzz ). Since both machines expect to process the same number
of jobs, this expression is also the overall expected slowdown of the system under a
random policy.
36
Size-splitting policy
The size-splitting policy splits the job size distribution into equal loads. We can find
the splitting point y by solving the following equation:
Z y
x · pdf (x) dx =
Z z
1
x · pdf (x) dx
y
c
Z y
xx−2 dx = c
Z z
1
xx−2 dx
y
ln x|y1 = ln x|zy
ln y − ln 1 = ln z − ln y
ln y 2 = ln z
y = z 1/2
(4.13)
√
Thus, the size-splitting policy assigns jobs with less than size O( z) to machine
√
0, and jobs with more than size O( z) to machine 1.
Next, we calculate the slowdown of each machine using the slowdown formula
calculated above for α = 1 (E[S] = O( xmzln z )). For machine 0, the job size distribution
√
√
is BP (1, 1, z), and machine 1, it is BP (1, z, z).
√
√
z
z
E[S0 ] = O( √ ) = O(
)
ln z
ln z
(4.14)
√
z
z
E[S1 ] = O( √
) = O(
)
z ln z
ln z
(4.15)
Since both machines have the same expected average slowdown, the expected
√
average slowdown for the entire system is also O( ln zz ). This is better than the random
√
policy by a factor of O( z).
37
4.2.2
Proof of BP claim: from random to all size-independent
policies
We now want to prove that no size-independent policy can achieve the same slowdown
√
as the size-splitting policy (O( ln zz )).
Claim: a size-independent policy can do no better than O( (lnzz)2 ) in the stochastic
problem with a BP (1, 1, z) job size distribution and Poisson arrivals.
Our proof will be very similar to the one made for the Simple distribution. We
define a sequence that occurs over a 3x/8 timespan. We then show that (1) this
sequence has an O( (ln1z)2 ) probability of occurring starting from an arbitrary point,
and (2) jobs in this sequence have an expected O(z) slowdown. It follows that the
overall expected slowdown is O( (lnzz)2 ).
First, we change our job groupings slightly. For the Simple distribution, we split
jobs into those of size 1 and those of size x. For our BP distribution, we create 3
possible job ranges: [1, 2], [2, z/2], and [z/2, z]. With α = 1, jobs have the following
probabilities of being from these three ranges respectively: O(1), O(1), and O(1/z).
We define our sequence to take place over a 3z/8 timespan, separated into 3 phases
of z/8 timespan each: (Phase 1) at least one job in the range [z/2, z] arrives, (Phase
2) at least one job in the range [z/2, z] arrives, and (Phase 3) no restrictions.
Again, our proof has two steps. First, we must show that such a sequence has an
O( (ln1z)2 ) probability of occurring starting from an arbitrary time. Second, we must
show that during this sequence, the expected slowdown is O(z).
For the first step, jobs from the range [z/2, z] arrive at a rate of O( λz ). However,
since the mean of a BP(1, 1, z) is O(ln z), λ is O( ln1z ) and the arrival rate is O( z ln1 z ).
For the sake of simplicity, we calculate the probability of seeing exactly one job, which
is e−λτ (λτ ) for a generic arrival rate λ and timespan τ . For our parameters, with an
z
arrival rate of O( z ln1 z ) and a timespan of z/8, this probability is e−O( 8z ln z ) O( 8z zln z ) =
1
e−O( 8 ln z ) O( 8 ln1 z ). For large z, this probability converges to O( ln1z ). Thus, the total
probability of such a sequence is at least O( (ln1z)2 ).
The second step is to show that the expected slowdown during this sequence is
38
O(z) when using a size-independent policy. We show this by showing that a constant,
non-zero fraction of these jobs must have an O(z) slowdown. To do so, we consider
each phase in order. Again, this analysis will be very similar to the analysis from our
proof for the Simple distribution.
For phase 1, we establish that because a job in the size range [z/2, z] has arrived
within this z/8 timespan, we enter phase 2 with at least one machine having at least
3z/8 remaining workload. Without loss of generality, we define this machine to be
machine 0.
For phase 2, we examine the possible size-independent policies. We know that at
least one job from the size range [z/2, z] will be assigned in this phase. Thus, any
policy in general must either (1) assign all jobs from the size range [z/2, z] to machine
0 or (2) assign at least one such job to machine 1. For any policy, either case (1) will
occur with probability at least 1/2, or case (2) will occur with probability at least
1/2. We now analyze both cases for a size-independent policy.
Case 1: the policy assigns all jobs from the size range [z/2, z] in this phase to
machine 0 with probability at least 1/2. Since a size-independent policy does not
know the size of a job when it is scheduled, it must assign at least 1/2 of all jobs in
phase 2 to machine 0 for it to fit into this case. Since (1) 1/2 of the jobs in phase 2
represents a constant fraction of all jobs in the sequence, (2) O(1) of the jobs assigned
to machine 0 will be from the size range [1,2], and (3) all of these jobs will face at
least a z/4 waiting time, an expected constant fraction of all jobs in the sequence
have an O(z) slowdown. The proof ends here under this case.
Case 2: the policy assigns at least one job from the size range [z/2,z] to machine
1 with probability at least 1/2. Here, we establish that with at least 1/2 probability,
we will enter phase 3 with both machines having at least z/4 remaining workload.
For phase 3, assuming that we arrived from case 2 above, O(1) of arriving jobs
are expected to be from the size range [1,2], and they will all have at least z/16
slowdown. Since these jobs make up a constant fraction of all jobs, and they all
have O(z) slowdown, then jobs in the entire sequence have an expected O(z) average
slowdown.
39
Thus, we have shown that with any size-independent policy, starting from an
arbitrary point in time, jobs arriving in the next 3z/8 timespan will face an O(z)
average slowdown with an O( (ln1z)2 ) probability. Thus, the optimal expected slowdown
from a size-independent policy is O( (lnzz)2 ). This is worse than the size-splitting policy,
√
which achieves O( ln zz ) slowdown.
40
Chapter 5
Future Work
Because this thesis focuses on an intentionally simplified problem, there are many
natural extensions. We make suggestions for future work in three general categories:
(1) a deeper look into the problems of this thesis, (2) an extension to a more general
problem by relaxing constraints and removing assumptions, and (3) a consideration
of other parameters to model additional complexities that arise in realistic settings.
5.1
A deeper look into the problems of this thesis
• Use dynamic programming to find an approximately optimal policy for the stochastic simplified problem with arrivals (Section 3.2). For the simple problem, we
can think of the optimal policy as being of the following form: for each state,
which is a combination of the remaining workloads of the two machines, we
need to decide what to do if a size 1 job arrives, and what to do if a size x job
arrives. In each case, we can make the dynamic or anti-dynamic choice. Thus,
each state has four possible policies.
We can model our problem as a finite-state discrete-time Markov Decision Process (MDP) if we (1) change the arrival process from Poisson arrivals to Bernoulli
arrivals and (2) constrain x to be an integer. Each state in the MDP will be
an integer pair corresponding to the remaining workloads of the two machines.
Each state will have four possible actions corresponding to the four possible
41
policies. Finally, each state can have up to five transitions at each timestep: (1)
no arrival, so that both remaining workloads decrease by 1, (2,3) a size of job 1
arrives and we assign it to either machine, and (4,5) a job of size x arrives and we
assign it to either machine. The cost associated with each state-transition pair
will just be the slowdown incurred. With these components, we completely define the MDP. We can bound the number of states by implementing a maximum
queue size of each machine, where no arrivals are possible when both machines
are full. A large maximum will reduce the inaccuracies of the approximation,
but increase the running time.
To solve the MDP, we can use a number of existing algorithms (ie. the value
iteration algorithm), which will give the optimal action for each state.
5.2
Relaxation of a constraint
• Vary α between 0 and 2 for the bounded Pareto distribution. In Section 4.2.1,
we considered a bounded Pareto with α = 1 because that value of α most
matched the Simple distribution. The next step would be to perform the same
analysis on BPs with other α values. Then, we could see how the difference
in slowdown between the size-splitting policy and the optimal size-independent
policy varies as α varies. Our hypothesis is that a smaller α (higher variability)
would lead to a greater difference between the performance of size-independent
and size-dependent policies.
• Allow unbalanced loads for our simple problem. For the Simple distribution, we
assumed that the total load coming from size 1 jobs would be the same as the
load from size x jobs. However, we noted that if the load was instead unbalanced, then both the size-splitting and D/A policies would become unstable.
It is possible that if the load from size x jobs is much greater than the load
from size 1 jobs, then both size-independent and size-dependent policies would
perform equally poorly.
42
• Have multiple machines. With this change, we can consider whether having
more machines favors a size-independent policy, a size-dependent policy, or
neither.
5.3
Additional extensions for better modeling the
real world
• Model the arrival time distribution as a heavy tail. As mentioned in the background chapter, Leland et al. showed that internet traffic is ’bursty’ and statistically self-similar, which is a mark of heavy-tailed distributions [6]. It would
be interesting to see if our problem is affected by changing the arrival process
from Poisson to a heavy-tailed arrival process.
• Allow job splitting. One solution for dealing with very large jobs is to split
them so that multiple machines can process them simultaneously. The split
incurs a cost, but in environments such as printing, this cost may be relatively
small. Would the variability of the job size distribution affect how often a split
is warranted?
• Have machines with different processing speeds. If we are in an environment
where machines are particularly expensive, it may be the case that a collection
of machines were bought at different years and have different capabilities. Such
a system would have machines with various processing speeds. With a heavytailed job size distribution, would it be better for smaller (or larger) jobs to go
to slower (or faster) machines?
• Include job setup costs. In many physical environments, such as the print or
manufacturing environments, every job has a certain type. If a job goes to a
machine that had just processed another job of a different type, then the former
would have to incur a setup cost. For example, in the print environment, the
type of a job would often be the type of paper it needs to be printed on.
43
• Multiple stages. Many real-world systems process jobs in multiple stages.
44
Chapter 6
Conclusion
In this thesis, we studied task assignment for a two-machine distributed server system,
where arriving jobs have to be assigned to one machine upon arrival. We discover
that if the input job size distribution follows a heavy-tail distribution, then a sizeindependent policy is guaranteed to perform poorly.
We began by considering a Simple distribution, which only provides jobs in two
sizes (1 and x), to mimic a Pareto distribution with α = 1. In this case, we found that
two size-dependent policies (size-splitting and a hybrid) could achieve O(1) slowdown,
while two size-independent ones (random and dynamic) could only achieve O(x). We
also looked at the problem under the condition of no new arrivals, finding optimal
policies through dynamic programming. The optimal policies often made opposite
assignment choices for size 1 and size x jobs, meaning that a size-independent policy
must often make a mistake. Next, we proved that for the case of persistent arrivals,
no size-independent policy can do better than O(x). The primary reason seems to
be that a good policy needs to prevent small jobs from being caught up behind large
ones.
Finally, we extended our toy distribution into a bounded Pareto with α = 1, and
√
repeated the analysis. We found a size-splitting policy to give a slowdown of O( ln zz ),
while proving that the best size-independent policy can only achieve O( (lnzz)2 ).
45
46
Bibliography
[1] J. Carroll. Balanced load when service times are heavy-tailed. 2000.
[2] M. E. Crovella and A. Bestavros. Self-similarity in world wide web traffic evidence
and possible causes. In ACM SIGMETRICS, pages 160–169, May 1996.
[3] M. E. Crovella and L. Lipsky. Long-lasting transient conditions in simulations
with heavy-tailed workloads. In Winter Simulation Conference, pages 1005–1012,
1997.
[4] M. Harchol-Balter, M. Crovella, and C. Murta. On choosing a task assignment
policy for a distributed server system. IEEE Journal of Parallel and Distributed
Computing, 59:204–228, 1999.
[5] M. Harchol-Balter and A. B. Downey. Exploiting process lifetime distributions
for dynamic load balancing. In ACM SIGMETRICS Conference on Measurement
and Modeling of Computer Systems, pages 13–24, May 1996.
[6] W. E. Leland, M. Taqqu, W. Willinger, and D. V. Wilson. Long-lasting transient
conditions in simulations with heavy-tailed workloads. In Winter Simulation Conference, pages 183–193, 1997.
[7] W. Whitt. Deciding which queue to join: Some counterexamples. Operations
Reserach, Jan. 1986.
[8] W. Winston. Optimality of the shortest-line discipline. Journal of Applied Probability, 1977.
47
[9] R. Wolff. Stochastic Modeling and the Theory of Queues. Prentice Hall, Feb. 1989.
48