Scheduling in Switched Queueing Networks with
Heavy-Tailed Traffic
by
Mihalis G. Markakis
Submitted to the Department of Electrical Engineering and Computer
Science
in partial fulfillment of the requirements for the degree of
Doctor of Philosophy in Electrical Engineering and Computer Science
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
June 2013
c Massachusetts Institute of Technology 2013. All rights reserved.
Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Department of Electrical Engineering and Computer Science
May 22, 2013
Certified by . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Eytan Modiano
Professor of Aeronautics and Astronautics
Thesis Supervisor
Certified by . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
John N. Tsitsiklis
Clarence J. Lebel Professor of Electrical Engineering
Thesis Supervisor
Accepted by . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Leslie A. Kolodziejski
Chair, Department Committee on Graduate Students
2
Scheduling in Switched Queueing Networks with Heavy-Tailed
Traffic
by
Mihalis G. Markakis
Submitted to the Department of Electrical Engineering and Computer Science
on May 22, 2013, in partial fulfillment of the
requirements for the degree of
Doctor of Philosophy in Electrical Engineering and Computer Science
Abstract
We study scheduling problems arising in switched queueing networks, a class of stochastic
systems that are often used to model data communication networks, such as uplinks and
downlinks of cellular networks, networks of data switches, and ad hoc wireless networks.
Motivated by empirical evidence of self-similarity and long-range dependence, the networks
that we consider receive a mix of heavy-tailed and light-tailed traffic. In this setting we
evaluate the delay performance of the widely-studied class of Max-Weight scheduling policies.
As performance metric we use the notion of delay stability, i.e., whether the steady-state
expected delay in a queue is finite or not. Max-Weight policies are known to have excellent
stability properties, and also to achieve good delay performance under light-tailed traffic.
Classical results from queueing theory imply that heavy-tailed queues are delay unstable
under any policy, so we focus on the potential impact of heavy tails on light-tailed queues.
The main insight derived from this thesis is that the Max-Weight policy performs poorly in
the presence of heavy tails, whereas a suitably modified version of Max-Weight achieves much
better overall performance. More specifically: (i) under the Max-Weight scheduling policy,
any light-tailed queue that conflicts (i.e., cannot be served simultaneously) with a heavytailed queue is delay unstable; (ii) delay instability may propagate to light-tailed queues
that do not conflict with heavy-tailed queues. The latter can happen through a “domino
effect,” if a light-tailed queue conflicts with a queue that has become delay unstable because
it conflicts with a heavy-tailed queue. The extent of this phenomenon depends on the arrival
rates; (iii) under the parameterized Max-Weight-α scheduling policy, all light-tailed queues
are delay stable provided the α-parameters are chosen suitably.
On the methodological side, we show how fluid approximations can be combined with
renewal theory in order to prove delay instability results. Moreover, we show how fluid
approximations can be combined with stochastic Lyapunov theory in order to prove delay
stability results. Finally, we identify a class of piecewise linear Lyapunov functions that are
suitable for obtaining exponential bounds on queue-length asymptotics, in the presence of
heavy-tailed traffic.
3
Thesis Supervisor: Eytan Modiano
Title: Professor of Aeronautics and Astronautics
Thesis Supervisor: John N. Tsitsiklis
Title: Clarence J. Lebel Professor of Electrical Engineering
4
Acknowledgements
First and foremost, I would like to express my deepest gratitude to my advisors, Eytan
Modiano and John Tsitsiklis. Spending the last five years under their mentorship had a
tremendous impact on my academic identity: how to approach research and teaching, how
to pick a research problem, how to combine clarity with rigor, how to write a scientific paper,
how to present my work... Their guidance and support through the ups and downs of my
PhD journey really shaped this thesis. And, on the side, we had some good times as well,
from ski trips to house parties! I can only hope that I will be to my students as good of an
advisor as John and Eytan were to me.
I would like to extend special thanks to my thesis readers, David Gamarnik and Devavrat
Shah. Through classes, seminars, and personal interactions, David and Devavrat have always
been extremely generous in sharing their wealth of knowledge and enthusiasm with me. My
deepest appreciation goes to Dimitri Bertsekas and Robert Gallager, from whom I learned
about Dynamic Programming and Discrete Stochastic Processes, but, more importantly,
that deep understanding, clarity and intuition, and striving for excellence are the standards
that I should set for myself as an academic.
The Laboratory for Information and Decision Systems offered an ideal home for my
doctoral studies, intellectually stimulating but, at the same time, very warm and personal. I
am indebted to the LIDS staff, Debbie, Jennifer, Lynne, Hristina, and Brian; and, especially,
to my Greek friends, Yola, Spyros, Apostolos, Kostas, and Kimon, and my officemates,
Kuang, Hoda, Jagdish, Yuan, and Yunjian.
I have been fortunate in making many good friends during my stay in Boston, too many
to mention one by one. Thank you all so much for filling the last five years of my life with
very fond memories!
Finally, I would like to thank my parents and sister for their unconditional love and
support. This thesis is dedicated to them.
My doctoral studies at MIT were supported by NSF Grants CNS-0915988 and CCF0728554, and the ARO MURI Grant W911NF-08-1-0238, which I gratefully acknowledge.
5
6
Contents
1 Introduction
11
1.1
Scheduling in Switched Queueing Networks with Heavy-Tailed Traffic . . . .
12
1.2
Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15
1.3
Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
1.4
Preview of Thesis and Contributions . . . . . . . . . . . . . . . . . . . . . .
18
2 Model, Definitions, and Mathematical Preliminaries
25
2.1
Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25
2.2
Model and Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26
2.3
BASTA, Little’s Law, and Delay Stability . . . . . . . . . . . . . . . . . . .
32
2.4
Truncated Rewards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
37
2.5
“Average Behavior” of Stochastic Processes . . . . . . . . . . . . . . . . . .
39
2.6
Foster-Lyapunov Criteria for Markov Chains . . . . . . . . . . . . . . . . . .
41
3 Scheduling in Parallel Queues with Heavy-Tailed Traffic
45
3.1
A Single-Server System of Parallel Queues . . . . . . . . . . . . . . . . . . .
47
3.2
Scheduling in the Presence of Heavy-Tailed Traffic . . . . . . . . . . . . . . .
49
3.2.1
Nonpreemptive Policies . . . . . . . . . . . . . . . . . . . . . . . . . .
49
3.2.2
Preemptive Priority Policies . . . . . . . . . . . . . . . . . . . . . . .
51
3.2.3
The Round-Robin Policy . . . . . . . . . . . . . . . . . . . . . . . . .
53
3.2.4
A Randomized Policy . . . . . . . . . . . . . . . . . . . . . . . . . . .
56
3.2.5
The Max-Weight Policy . . . . . . . . . . . . . . . . . . . . . . . . .
57
7
3.3
3.2.6
The Max-Weight-α Policy . . . . . . . . . . . . . . . . . . . . . . . .
57
3.2.7
The Max-Weight-log Policy . . . . . . . . . . . . . . . . . . . . . . .
58
Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
59
4 Max-Weight Scheduling in Networks with Heavy-Tailed Traffic
63
4.1
A Single-Hop Switched Queueing Network . . . . . . . . . . . . . . . . . . .
64
4.2
Overview of Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . .
68
4.3
Max-Weight Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
70
4.3.1
Conflicting with Heavy Tails . . . . . . . . . . . . . . . . . . . . . . .
71
4.3.2
Nonconflicting with Heavy Tails . . . . . . . . . . . . . . . . . . . . .
74
Max-Weight-α Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . .
93
4.4.1
The Main Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
94
4.4.2
Traffic Variability and Delay Stability . . . . . . . . . . . . . . . . . . 101
4.4.3
Scaling Results under Light-Tailed Traffic . . . . . . . . . . . . . . . 102
4.4.4
Scaling Results under Bernoulli Traffic . . . . . . . . . . . . . . . . . 104
4.4
4.5
Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
5 Delay Analysis of the Max-Weight Policy via Fluid Approximations
109
5.1
Methodological Challenges and Contributions . . . . . . . . . . . . . . . . . 111
5.2
The Fluid Model of a Single-Hop Network . . . . . . . . . . . . . . . . . . . 112
5.3
Delay Instability Results via Fluid Approximations . . . . . . . . . . . . . . 115
5.4
The Bottleneck Identification Algorithm . . . . . . . . . . . . . . . . . . . . 119
5.5
Delay Stability Results via Fluid Approximations . . . . . . . . . . . . . . . 123
5.6
Delay Stability Regions of the Max-Weight Policy . . . . . . . . . . . . . . . 130
5.7
Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
6 Delay Analysis of Back-Pressure Policies under Heavy-Tailed Traffic
151
6.1
A Multi-Hop Network under the Back-Pressure Policy . . . . . . . . . . . . . 153
6.2
Switched Queueing Networks as Stochastic Processing Networks . . . . . . . 159
6.3
Delay Stability Analysis of the Back-Pressure Policy - Examples . . . . . . . 162
8
6.3.1
The Role of Network Topology
. . . . . . . . . . . . . . . . . . . . . 164
6.3.2
Routing Heavy-Tailed Flows . . . . . . . . . . . . . . . . . . . . . . . 165
6.3.3
Routing Light-Tailed Flows . . . . . . . . . . . . . . . . . . . . . . . 167
6.3.4
The Role of Link Capacities . . . . . . . . . . . . . . . . . . . . . . . 171
6.3.5
The Impact of Heavy Tails on Cross-Traffic . . . . . . . . . . . . . . . 173
6.3.6
The Role of Intersecting Paths . . . . . . . . . . . . . . . . . . . . . . 176
6.4
Delay Stability Analysis via Fluid Approximations . . . . . . . . . . . . . . . 179
6.5
The Back-Pressure-α Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
6.6
Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
7 Discussion
199
9
10
Chapter 1
Introduction
Modern communication networks, such as wireless cellular networks and the Internet, are
characterized by complexity in almost every aspect: physical infrastructure, logical topology, variety of users and applications, numerous and intertwined decisions upon which their
every-day operation relies. In particular, a salient feature of modern networks, and the main
drive behind and focus of this thesis, is that the traffic that they have to support is extremely
heterogeneous: on one hand, activities such as web-browsing and emailing have fairly “regular” requirements in terms of network resources, creating predictable, low-variability traffic
streams. On the other hand, the requirements of peer-to-peer applications may exhibit wild
fluctuations, creating traffic patterns with high variability. It is intuitively clear that highvariability traffic streams could have an impact on low-variability ones because, essentially,
they compete for the same resources. However, the complex structure of modern networks,
and the variety of intertwined decisions, make it hard to quantify this impact and/or to come
up with “good” decision making rules.
By means of mathematical modeling and analysis, the goal of this thesis is to understand
the behavior of networks where high and low variability traffic streams co-exist, and to derive
some insights on the way that such systems should be operated.
11
1.1
Scheduling in Switched Queueing Networks with
Heavy-Tailed Traffic
The mathematical models that we use in this thesis to represent data communication networks are the class of switched queueing networks. A switched queueing network can be
viewed as a collection of queues, each of which receives only one type of customers, and
has one server dedicated to serving those customers. Customers arrive to a queue, either
exogenously or from within the network, wait for service, and after their service is complete
they join another queue or leave the network. The distinguishing characteristic of switched
queueing networks is that the activity of different servers is interdependent, namely when a
server is active some other servers may have to be inactive. This feature implies that only
certain sets of servers can be active simultaneously, and, consequently, only specific sets of
queues can be served simultaneously. In related literature, the latter sets are usually called
“schedules” or “activations.” So, a fundamental scheduling problem arises in the context of
switched queueing networks: which schedule to activate and at which point in time? Clearly,
the overall performance of the network depends critically on the scheduling policy applied.
As alluded to earlier, the focus of this thesis is on traffic streams that exhibit high
variability. High variability is usually captured with random variables whose probability
distributions have “heavy tails,” in the sense that the probability that the random variable
takes a very large value is nonnegligible. More specifically, we consider queueing systems
in discrete time, where batches of customers arrive at each time slot, with the timing and
the sizes of batches being random. We consider an arrival process to be heavy-tailed, if the
random number of customers arriving at each time slot has infinite variance; otherwise, we
consider the arrival process to be light-tailed. In this thesis we consider switched queueing
networks that receive a mix of heavy-tailed and light-tailed traffic. Our goal is to understand
the systemic impact of heavy tails as a function of the scheduling policy applied.
To quantify this impact we use the notion of delay stability: a queue is delay unstable if
the expected delay experienced in that queue is infinite, and delay stable otherwise. Delay
12
stability is a rather crude performance metric, trying to capture the notion of “large delays” in a binary manner. However, it will become obvious in later chapters that the delay
stability metric does allow us to derive fundamental insights into which scheduling policies
are expected to perform well in the presence of heavy-tailed traffic, and why. Moreover, the
facts that we will be leveraging in most of our results are: (i) that certain arrival processes
produce big batches of customers with nonnegligible probability; (ii) that all arrival processes
exhibit an “average behavior” over long time scales with high probability. Thus, we expect
the insights that we derive through delay stability analysis to be robust to the probabilistic
assumptions made.
To make things more concrete, consider the following simple example. Consider two
single-class single-server queues, termed queue 1 and queue 2, respectively. Assume that
queue 1 receives heavy-tailed traffic while queue 2 receives light-tailed traffic. The arrival
rates to both queues are assumed to be strictly positive, but small enough so that the arriving
traffic can be handled, on average, by the available server capacity.
Now consider the following cases regarding the activity of the servers:
(i) the activity of the two servers is independent. In other words, the network is decomposed to two isolated queues, as shown in Figure 1-1. Assuming that the servers have enough
capacity to support the traffic of their respective queues, standard results from queueing theory (e.g., the Pollaczek-Khinchine formula, Kingman’s bounds) imply that queue 1 exhibits
large delays whereas queue 2 exhibits low delays. With respect to the performance metric
adopted in this thesis, queue 1 is delay unstable whereas queue 2 is delay stable;
Figure 1-1: Two isolated single-server queues.
13
(ii) the activity of the two servers is interdependent in the following way: if the server
of queue 1 is active then the server of queue 2 must be inactive, and vice versa. This is the
simplest switched queueing network that one can encounter. Equivalently, one can assume
that there exists just one server that the two queues have to share, as shown in Figure 1-2.
It is not hard to see that queue 1 is again delay unstable because, even in the best case, it is
still a heavy-tailed queue in isolation. In contrast, the delay stability of queue 2 depends on
the scheduling policy applied. For example, if priority is given to queue 2, i.e., if we allocate
the server to queue 2 whenever it is nonempty, then we are back to case (i) so queue 2 is
delay stable. On the other hand, if priority is given to queue 1 then a customer of queue 2
experiences delays that are at least as large as the delays of queue 1. So, queue 2 is delay
unstable in this case.
Figure 1-2: Two queues sharing a single server.
This simple example provides the first fundamental insight of the thesis: in switched
queueing networks with a mix of heavy-tailed and light-tailed traffic, large delays
not only arise in heavy-tailed queues but may propagate to light-tailed queues
as well. The cause of this phenomenon is precisely the “scheduling constraints”
that characterize switched networks, and its extent depends on the scheduling
policy applied.
Of course, the situation becomes more interesting if we consider queueing networks with
more complex structure, and under more sophisticated scheduling policies, e.g., queue-length
based policies. These cases are the primary focus of the thesis.
14
1.2
Motivation
As mentioned above, our main motivation for studying switched queueing networks, and
the associated scheduling problems, comes from the area of data communication networks.
Queueing theory and networks have long been the main tools for performance analysis and
system design in the networking community, at least since the early 1960s and the doctoral
work of Kleinrock [37]. During this span of fifty years, a wide range of queueing models have
been employed under different probabilistic assumptions, ranging from multi-class queues,
multi-server queues, polling systems, Jackson networks, and multi-class queueing networks.
The fast-paced growth of the Internet and of wireless cellular networks, though, has turned
researchers to the study of queueing models that are more suitable for the particular applications. Thus, switched queueing networks have been at the forefront of research activity
over the last decade, because of their ability to capture the salient features of networks of
data switches and of wireless networks with interference constraints.
We are particularly motivated to study switched queueing networks with heavy-tailed
traffic by empirical evidence that traffic in real-world data networks exhibits strong correlations (long-range dependence) and statistical similarity over different time scales (selfsimilarity). These observations were first made by Leland et al. [40] through analysis of
Ethernet traffic traces. Subsequent empirical studies have documented these phenomena in
other data networks. More importantly for our purposes, accompanying theoretical studies
have associated self-similarity and long-range dependence with arrival processes that have
heavy tails; see [48] for an overview.
Switched queueing networks have been quite popular within the networking community,
but have also been used to capture the dynamics and decisions of systems of different nature,
e.g., cloud computing clusters [41] and flexible manufacturing systems [25]. We note that
high variability has been reported in both cloud computing and manufacturing (in job sizes
[20] and demand for products [22], respectively). This makes the problem of scheduling in
switched queueing networks with heavy-tailed traffic of wider interest.
Finally, a variety of operational settings can be modeled with queueing systems (not
15
necessarily of the switched-network type) that exhibit highly variable traffic, e.g., call centers
[13], healthcare [3], and semiconductor wafer fabrication facilities [29, 39]. Although our
results and insights do not carry over directly to these settings, the analytical approach and
methodological contributions of this thesis could be relevant to areas besides engineering.
A significant part of this thesis is devoted to performance evaluation of a particular
class of queue-length based scheduling policies, termed the Max-Weight policies, in the
presence of heavy-tailed traffic. There are two main reasons that motivate us to do so:
(i) the throughput optimality property, i.e., the ability of Max-Weight policies to
stabilize a queueing system whenever this is possible. Informally speaking, by “stability”
we mean that the queue lengths remain bounded and the system reaches a “steady-state”
behavior (for a formal definition the reader is referred to Chapter 2). Stability should be
viewed as a first order criterion that needs to be satisfied before one can proceed to a more
elaborate analysis, e.g., delay analysis, and throughput optimality implies that Max-Weight
policies are optimal with respect to this criterion. We note that there exist simple queueing
systems that are unstable under “reasonable” buffer-priority policies, even though they could
be stabilized under different policies [38, 49]. Thus, throughput optimality is not a property
that every reasonable scheduling policy possesses, but rather a special feature of Max-Weight
policies. The throughput optimality of Max-Weight scheduling was first shown for switched
queueing networks with independent arrivals [59], and later extended to systems with general
arrival and connectivity processes [2], and further to stochastic processing networks [17];
(ii) the asymptotic delay optimality property, i.e., the fact that Max-Weight policies
minimize the steady-state workload, and, consequently, the delays, when the network is
critically loaded and under diffusion scaling. This property was first shown in the context of
switched queueing networks [57], and later extended to stochastic processing networks [18].
The Max-Weight policy has also been shown to minimize the average delay in networks exhibiting certain symmetries [26,60]. Moreover, variations of Max-Weight achieve the optimal
large deviations exponent of steady-state queue lengths, in networks with just exponentialtype traffic [62]. Finally, Max-Weight scheduling has been combined with various forms of
16
congestion control for solving Network Utility Maximization problems; see [27] and the references therein. For all these reasons, Max-Weight policies have become the focus of intense
research activity, and the canonical example of an “optimal policy” for switched queueing
networks.
1.3
Literature Review
There are two strands of literature that are related to this thesis, one coming from the
operations research community, the other coming from the networking community.
Operations researchers have studied extensively single-class queues, with either a single
or multiple servers, and have analyzed the impact of heavy-tailed traffic on the intra-queue
scheduling problem, i.e., the way that customers are served within the queue, e.g., “First
Come First Served,” “Last Come First Served,” Processor Sharing, Shortest Remaining
Processing Time. The developed body of work is significant; for a comprehensive account
the reader is referred to the survey papers [7,11] and the references therein. The importance
of this strand of literature cannot be overestimated, since it offers valuable insights and a
variety of analytical tools. However, it considers queueing systems with a relatively simple
structure. A more recent study by Baccelli & Foss [4] has taken a step in the direction of more
complex systems by analyzing monotone separable networks, a class of stochastic networks
that includes multi-server queues, polling systems, and generalized Jackson networks as
special cases. The common denominator of all these works is that they consider queueing
systems with just heavy-tailed traffic, and analyze the performance of intra-queue scheduling
policies. In contrast, we are interested in queueing systems with a mix of heavy-tailed and
light-tailed traffic, and we look into the inter-queue scheduling problem, i.e., which queue to
serve at any given point in time.
On the other hand, there has been intense research activity within the networking community on the inter-queue scheduling problem of switched queueing networks, and, in particular,
on the analysis of Max-Weight policies. The developed body of work is vast and a detailed
17
account is beyond the scope of this thesis. We refer the interested reader to [27] and the
references therein. The achievements of this strand of literature have also been important;
most notably, the combination of scheduling, routing, and congestion control policies into
holistic “cross-layer control” schemes that are throughput optimal and utility maximizing.
We should note, however, that these works consider queueing systems with light-tailed traffic.
Closer to the setting of this thesis come the papers by Borst et al. [8] and by Jagannathan et al. [35]. Both consider a system with two parallel queues, receiving heavy-tailed
and light-tailed traffic, respectively, while sharing a single server. They determine the queuelength asymptotics of two popular (inter-queue) scheduling policies, the Generalized Processor Sharing policy and the Generalized Max-Weight policy, respectively. In the same setting,
the work by Nair et al. [45] analyzes the role of intra-queue scheduling on the queue-length
asymptotics of the Generalized Max-Weight policy. Related to this thesis is also the paper
by Boxma et al. [10], which analyzes a M/G/2 queue with a heavy-tailed and a light-tailed
server, and shows a dependence of the queue-length asymptotics on the arrival rate to the
queue. Similar connections are established in the work of Borst et al. [9] in the context
of two coupled queues. In summary, all related work on queueing systems with a mix of
heavy-tailed and light-tailed traffic is restricted to simple settings with one or two servers,
and one or two queues.
The goal of this thesis is to fill a gap in the literature by analyzing more complex switched
queueing networks receiving a mix of heavy-tailed and light-tailed traffic, and with particular
emphasis on Max-Weight-type scheduling policies.
1.4
Preview of Thesis and Contributions
This thesis considers resource allocation problems arising in a certain class of dynamic and
stochastic systems; in particular, systems with relatively complex dynamics, where stochastic variability plays an important role. “Systematic” approaches to such problems, e.g.,
Dynamic Programming or Markov Decision Problem formulations, are known to be ana18
lytically intractable and with prohibitive computational requirements. Moreover, Monte
Carlo simulation methods are very slow to converge under high variability, and could also
be inconclusive, depending on the performance metric of interest.
Thus, the general approach of this thesis is to analyze intuitive/popular resource allocation policies, derive insights, and, based on those insights, design new policies that are
suitable for the given setting. Performance evaluation is based primarily on theoretical analysis. Along the way, this thesis also develops new methodological tools to facilitate the
analysis of queueing systems under significant stochastic variability.
The remainder of the section describes in greater detail the content and contributions of
the various chapters.
Chapter 2: Model, Definitions, and Mathematical Preliminaries
Chapter 2 provides a description of the structure, dynamics, and main properties of a
“generic” switched queueing network. The queueing systems that are analyzed in the thesis,
and which are described in greater detail in Chapters 3, 4, and 6, are all special cases of
this generic model. This chapter also summarizes our notational conventions, and includes
a number of useful definitions, e.g., of a heavy-tailed arrival process, of a stable queueing
network, and of a delay stable traffic flow.
For completeness of the thesis, we formally state, and in most cases prove, well-known
results such as the “Bernoulli Arrivals See Time Averages” property, Little’s Law, FosterLyapunov stability criteria, results on truncated rewards of renewal processes, and on the
average behavior of stochastic processes. These serve as intermediate lemmas for proving
the main results of the thesis.
Chapter 3: Scheduling in Parallel Queues with Heavy-Tailed Traffic
In Chapter 3 we consider the simplest switched queueing network, namely a system with two
queues sharing a single server. We assume that one queue receives heavy-tailed traffic whereas
19
the other light-tailed traffic, and we analyze the performance of a variety of scheduling policies
(in this case, server allocation policies) in the presence of heavy tails.
The main contribution of this chapter is to derive insights into which scheduling policies
are expected to perform well in the presence of heavy-tailed traffic, and why. Our findings
can be summarized as follows. In order to achieve good performance in the presence of
heavy-tailed traffic, some form of priority (partial or complete) has to be given to lighttailed queues. There are inherent fairness issues related to complete priority policies, as well
as stability issues that were discussed in Section 1.2. Moreover, inherently fair policies, such
as deterministic or randomized time-sharing policies, perform well only for some arrival rates.
Furthermore, the queue-length based Max-Weight policy, which in this case is equivalent to
“Serve the Longest Queue,” performs poorly in the presence of heavy tails for all arrival
rates. This is in sharp contrast to the very good performance that Max-Weight achieves
under light-tailed traffic. Finally, a parameterized version of Max-Weight achieves good
stability and delay performance, if its parameters are chosen so that partial priority is given
to light tails.
Chapter 4: Max-Weight Scheduling in Networks with Heavy-Tailed
Traffic
In Chapter 4 we consider a single-hop switched queueing network, i.e., a switched network
where customers arrive exogenously to the different queues, wait for service, and upon completion of their service they leave the network. As mentioned above, single-hop networks
have been used extensively to capture the dynamics and decisions in cellular uplinks and
downlinks, input-queued switches, and wireless ad-hoc networks. We assume that the network receives a mix of heavy-tailed and light-tailed traffic, and we analyze the performance
of queue-length based Max-Weight policies in the presence of heavy tails.
The motivating factors for the analysis of Chapter 4 are the findings of Chapter 3 pertaining to Max-Weight policies. Namely, that Max-Weight performs poorly in the presence
of heavy tails, whereas a suitably parameterized version of Max-Weight achieves much bet20
ter overall performance. These observations are made in a simple setting of two parallel
queues, sharing a single server. The goal of this chapter is to analyze the extent to which
the poor performance of Max-Weight persists, and to propose solutions, in networks with
more complex structure and scheduling constraints.
The main insights derived from this chapter can be summarized as follows.
(i) Under the Max-Weight scheduling policy, any light-tailed queue that conflicts (i.e.,
cannot be served simultaneously) with a heavy-tailed queue is delay unstable;
(ii) Somewhat surprisingly, a light-tailed queue can be delay unstable even when it does
not conflict with a heavy-tailed queue. The latter can happen through a “domino effect,”
if the light-tailed queue conflicts with a queue that has become delay unstable because it
conflicts with a heavy-tailed queue. The extent of this domino effect depends, in general, on
the arrival rates;
(iii) The propagation of delay instability can be avoided through a parameterized MaxWeight-α scheduling policy. Specifically, if the α-parameters are chosen suitably, then the
α-moments of the steady-state queue lengths are finite. We use this result to prove that,
by proper choice of the α-parameters, all light-tailed queues are delay stable. Moreover,
we show that Max-Weight-α achieves the optimal scaling of higher moments of steady-state
queue lengths with traffic intensity.
Chapter 5: Delay Analysis of the Max-Weight Policy via Fluid
Approximations
In Chapter 5 we build on, and extend the results of, Chapter 4. More specifically, we
continue the delay stability analysis of a single-hop switched queueing network under the
Max-Weight policy, and in the presence of heavy-tailed traffic. Our main motivation comes
from the observation that a light-tailed traffic flow can be delay unstable, even if it does
not conflict with heavy-tailed traffic. Delay stability/instability in this case depends on the
arrival rates of the different traffic flows, and the notion of a delay stability region arises,
i.e., the subset of the stability region for which a traffic flow is delay stable.
21
The proofs of the associated results in Chapter 4 are based on purely stochastic arguments, and are somewhat long and tedious. However, the main ideas behind these proofs are
rather simple and intuitive, and can be presented through an informal “fluid approximation”
to the stochastic system. The goal of Chapter 5 is to formalize this approach, i.e., to show
how the formal use of fluid approximations simplifies delay stability analysis.
The main contributions of Chapter 5 can be summarized as follows.
(i) We show how fluid approximations can be combined with renewal theory in order to
prove delay instability results. Moreover, we show how fluid approximations can be combined
with stochastic Lyapunov theory in order to prove delay stability results;
(ii) We illustrate how to obtain exponential bounds on queue-length asymptotics in the
presence of heavy-tailed traffic, through drift analysis of a particular class of piecewise linear
Lyapunov functions;
(iii) We provide a sharp characterization of the delay stability regions of the Max-Weight
policy in single-hop networks with disjoint schedules, generalizing the findings of Chapter
4. Along the way, our analysis reveals several monotonicity properties of service rates under
Max-Weight scheduling;
(iv) We show that, in every network with disjoint schedules that operates under the
Max-Weight policy and receives heavy-tailed traffic, there exists an arrival rate vector in the
corresponding stability region for which all queues are delay unstable.
Chapter 6: Delay Analysis of Back-Pressure Policies under HeavyTailed Traffic
Finally, in Chapter 6 we consider a multi-hop switched queueing network, i.e., a switched
network where customers, upon completion of their service at one queue, may join another queue. Clearly, multi-hop networks offer a variety of modeling capabilities beyond
their single-hop counterparts, allowing them to capture the dynamics of multi-hop wireless
networks and networks of data switches. Motivated by the findings of Chapters 4 and 5,
we analyze the Back-Pressure scheduling policy, the natural extension of Max-Weight in
22
a multi-hop setting, and investigate additional factors that may affect the performance of
queue-length based policies and/or new phenomena that may arise.
The main contributions of Chapter 6 can be summarized as follows.
(i) Through simple examples, we derive insights into how “system parameters,” such as
the network topology, the routing constraints, and the server/link capacities, may affect the
performance of the Back-Pressure policy in the presence of heavy-tailed traffic;
(ii) Using the findings of Chapter 5, we propose an algorithmic procedure that identifies
delay unstable queues by solving the fluid model of the network from certain initial conditions. This approach is of particular interest in cases of complex multi-hop networks, where
direct stochastic analysis is hard and Monte Carlo methods very slow to converge;
(iii) We show how one can guarantee the delay stability of all light-tailed flows in the
network by using a parameterized version of the Back-Pressure policy, with suitably chosen
parameters.
23
24
Chapter 2
Model, Definitions, and Mathematical
Preliminaries
2.1
Notation
Throughout this thesis we denote by R, Z, and N the sets of real numbers, integers, and
positive integers, respectively. Similarly, R+ and Z+ represent the sets of nonnegative reals
and nonegative integers, respectively. The Cartesian products of M copies of R+ , Z+ , and
M
M
N are denoted by RM
+ , Z+ , and N , respectively.
We use [x]+ for max{x, 0}, the nonnegative part of x ∈ R. Similarly, we use [x]− for
min{x, 0}, the nonpositive part of x ∈ R.
We denote the Euclidean norm of a vector x ∈ RM
+ by
kxk2 =
q
x21 + . . . + x2M ,
and its infinity norm by
kxk∞ = max |x1 |, . . . , |xM | .
With few exceptions, we follow the convention of using lower case letters to denote real
numbers or vectors, and upper case letters to denote random variables or events.
25
The indicator variable of event E is represented by 1E . The notation P(·) and E[·] is used
for probabilities and expectations, respectively. We also employ the shorthand notation
P(X; E | H) for P(X · 1E | H), where X is a random variable, E is an event, and H is a
σ-algebra on a given probability space. We define E[X; E | H] similarly.
We say that a statement holds with probability one, and write “w.p.1,” if the statement
is true for almost all sample paths of the underlying stochastic system, except, possibly, for
a set of sample paths with zero measure.
Let b ∈ R+ and monotonically increasing function f : R+ → R+ . We say that random
variable X scales at least as f (b) on the event E, and write X = ΩE f (b) , if there exist
constants k, b0 > 0, possibly depending on E, such that
X(ω) ≥ k · f (b),
∀b ≥ b0 ,
∀ω ∈ E.
Often times throughout the thesis this notation is used somewhat loosely, in the sense that
event E is not formally defined. In those cases, the sample paths that we are referring to
will be clear from the context.
2.2
Model and Definitions
In this section we describe the basic characteristics of a switched queueing network, and
provide definitions and notation that are used consistently throughout the thesis. In later
chapters (e.g., Chapters 3, 4, and 6) we provide more detailed descriptions of queueing
networks that fall within this wide class.
A Switched Queueing Network
The general approach of this thesis is to model data communication networks as queueing
systems. Queues may represent physical or virtual locations where traffic is stored while
waiting for transmission, e.g., through wireless medium or switch fabric. The transmission
26
process is abstracted by servers that serve the traffic of the queues. The fundamental unit
of traffic in our model is the packet, and we assume that all packets have the same length.
One can view a switched queueing network as a collection of single-class, singleserver queues. Time is discrete and indexed in Z+ . The server of each queue has a
deterministic service rate of one packet per time slot. Moreover, all queues have infinite
buffer space.
Central to our model is the notion of a traffic flow, which is a long-lived stream of
packets that arrive to the network at a designated source queue, traverse the network by
passing through certain queues, and finally exit the network when they get served by the
server of a designated destination queue. In the special case when the source and destination
queues of a traffic flow coincide, then the traffic of that flow exits the network as soon as
it gets served. Each queue is allowed to carry the traffic of just one traffic flow; this is the
essence of the queues being “single-class.” Moreover, in this thesis we consider only open
queueing networks, i.e., when a packet leaves a queue it never comes back to that queue.
We denote by N the number of queues and by F the number of traffic flows in the system.
The “single-class” and “open network” assumptions imply that F ≤ N . For convenience,
we define F = {1, . . . , F }. Without loss of generality, we assume that the source queue of
traffic flow f ∈ F is queue f .
Thus, we have (potentially) two types of queues: (i) queues 1, . . . , F, which receive exogenous traffic; (ii) queues (F + 1), . . . , N, in case F < N , which receive endogenous traffic.
The majority of this thesis (Chapters 3, 4, and 5) concerns networks that have only the first
type of queues, i.e., single-hop networks, so that F = N . However, in Chapter 6 we study
networks with both types of queues, i.e., multi-hop networks.
The service discipline of packets within each queue, the so-called intra-queue scheduling
policy, is “First Come, First Served.” Even though there are numerous other disciplines
that one may consider, e.g., “Last Come, First Served,” Round-Robin, Shortest Remaining
Processing Time, the particular discipline seems to be a natural choice for data communication networks.
27
The traffic of flow f ∈ F arrives to queue f according to a discrete time stochastic arrival
process Af (t); t ∈ Z+ . We assume that all arrival processes take values in Z+ , and are
independent and identically distributed (IID) over time slots. Furthermore, different arrival
processes are mutually independent. We denote by λf = E Af (0) the arrival rate of traffic
flow f , and by λ = (λ1 , . . . , λF ) the vector of arrival rates of all traffic flows. For the model to
be interesting, all rates are assumed to be strictly positive and finite. Moreover, for reasons
that become apparent later in the thesis, we make the technical assumption that there exists
i
h
(0)
is finite, for all f ∈ F.
γ ∈ (0, 1) such that E A1+γ
f
We follow the convention of “late arrivals,” so we interpret Af (t) as the random number
of packets that arrive to queue f ∈ F at the end of time slot t ∈ Z+ . We also use the vector
notation A(t) = A1 (t), . . . , AF (t) . Similarly, we define Qi (t) to be the number of packets in
queue i ∈ {1, . . . , N } at the beginning of time slot t, and we let Q(t) = Q1 (t), . . . , QN (t) .
Now we come to the definition of heavy-tailed traffic, which is the focal point of this
thesis.
Definition 2.1: (Heavy Tails) A random variable X is heavy-tailed if E X 2 is infinite,
and light-tailed otherwise.
Furthermore, a random variable X is exponential-type if there exists θ > 0 so that
E exp(θX) is finite. We define similarly a heavy-tailed, light-tailed, and exponential-type
traffic flow.
We note that there are several definitions of heavy/light tails in the literature. In fact,
a random variable is often defined as light-tailed if it is of exponential-type, and heavytailed otherwise. The definition adopted in this paper has been used in the area of data
communication networks, e.g., see [48].
Scheduling Constraints and Regenerative Scheduling Policies
We assume that the service of a packet initiates at the beginning of a slot, right after queue
lengths are observed, and it is completed at the end of a slot, right before arrivals occur. We
denote by Si (t) the number of packets that the server of queue i ∈ {1, . . . , N } (henceforth,
28
server i) attempts to serve during time slot t, and we let S(t) = S1 (t), . . . , SN (t) . The
fact that we consider the packet to be the fundamental unit of traffic, combined with our
assumption regarding the service rate of all servers (one packet per time slot), implies that
Si (t) can only take values in {0, 1}. A service attempt is successful if and only if queue i
is nonempty, so that the number of packets that are actually served during time slot t is
Si (t) · 1{Qi (t)>0} .
An important characteristic of a switched queueing network is that the activity of one
server has a potential impact on the activity of other servers. Scheduling constraints of
this type are common in data communication networks, e.g., due to interference in wireless
communications, or due to matching constraints in a data switch. At any given time slot,
each server can be active, attempting to serve one packet, or inactive, not attempting to
serve any packets. A set of servers that can be active simultaneously is called a schedule.
We denote by S the set of all schedules, and assume that it is an arbitrary subset of the
powerset of {1, . . . , N }. Thus, S(t) is constrained to take values in S, for all t ∈ Z+ . For
convenience, we also identify elements of S with vectors in {0, 1}N .
Thus, a fundamental scheduling problem arises in switched queueing networks: which
schedule to activate, and at which point in time. Clearly, the overall performance of the
network depends critically on the way that scheduling decisions are made.
Before proceeding further, let us define formally the notion of a scheduling policy. The
past history and present state of the network at time slot t ∈ N is captured by the vector
H(t) = Q(0), A(0), . . . , Q(t − 1), A(t − 1), Q(t) .
At time slot 0, we have H(0) = Q(0) . A causal scheduling policy is a sequence π =
(µ0 , µ1 , . . .) of functions µt : H(t) → S, t ∈ Z+ , used to determine scheduling decisions,
according to S(t) = µt H(t) .
A piece of notation that is used frequently: Ft is the σ-field that corresponds to the
history of the system until just before the arrivals at slot t; formally, Ft is the σ-algebra
generated by H(t).
29
In this thesis we restrict our attention to scheduling policies that are regenerative,
i.e., policies under which the network starts afresh, probabilistically, at certain time slots.
Formally, under a regenerative policy there exists a sequence of stopping times τn ; n ∈ Z+
with the folowing properties:
(i) the sequence τn+1 − τn ; n ∈ Z+ is IID;
(ii) let X(t) = Q(t), A(t), S(t) , and consider the processes that describe the “cycles”
of the network, namely C0 = X(t); 0 ≤ t < τ0 and Cn = X(τn−1 + t); 0 ≤ t <
τn − τn−1 , n ∈ N. Then, Cn ; n ∈ N is an IID sequence, independent of C0 ;
(iii) the lattice distribution of the renewal periods, τn+1 − τn , has span equal to one and
finite expectation.
Properties (i) and (ii) imply that the switched queueing network evolves as a, possibly
delayed, regenerative process. Property (iii) states that this process is aperiodic and positive
recurrent. We note that a number of widely studied scheduling policies belong to this class,
e.g., nonpreemptive policies, preemptive priority policies, time-sharing policies, and queuelength based policies, as long as the vector of arrival rates is in the stability region of the
network. Many of these policies will be formally defined and analyzed in later chapters.
Stability and Delay Stability
In the context of data communications, a batch of packets arriving to the network at any
given time slot can be viewed as a single entity, e.g., as a file that needs to be transmitted.
We define the end-to-end delay of a file of flow f to be the number of time slots that
the file spends in the network, starting from the time slot right after it arrives at queue f ,
until the time slot that its last packet exits the network. To make this definition clear, let
us consider the following example: suppose that a file of flow f arrives to the network at the
end of time slot τ , and constitutes of a single packet, i.e., Af (τ ) = 1. If this packet leaves
the network at time slot t > τ , then the delay of the file is (t − τ ) slots. Because services
are completed at the end of a slots, the delays of files cannot be less than one, even if they
encounter empty queues.
30
For k ∈ N, we denote by Df (k) the end-to-end delay of the k th file of flow f , and we let
D(k) = D1 (k), . . . , DF (k) .
Definition 2.2: (Stability) The earlier defined switched queueing network is stable
under a particular scheduling policy if the vector-valued sequences Q(t); t ∈ Z+ and
D(k); k ∈ N converge in distribution, and their limiting distributions do not depend on
Q(0).
Notice that our definition of stability is slightly different than the commonly used one,
which is the positive recurrence of the Markov chain of queue lengths, because it includes
the convergence of the sequence of file delays D(k); k ∈ N . The reason is that we are
interested in properties of the limiting distribution of the latter sequence and, naturally, we
need to ensure that this limiting distribution exists.
We denote by Q = Q1 , . . . , QN and D = D1 , . . . , DF generic random vectors dis
tributed according to the limiting distributions of Q(t); t ∈ Z+ and D(k); k ∈ N ,
respectively, assuming that the network operates under a stabilizing scheduling policy. The
dependence of these limiting distributions on the scheduling policy has been suppressed from
the notation, but will be clear from the context. We refer to Qi as the steady-state length
of queue i. Similarly, we refer to Df as the steady-state delay of a file of traffic flow f .
We note that, under a regenerative policy (if one exists), the queueing network is guaranteed to be stable. This is because the sequences of queue lengths and file delays are,
possibly delayed, aperiodic and positive recurrent regenerative processes, which implies that
they converge in distribution; see [56].
We now define formally the property that we will be focusing on.
Definition 2.3: (Delay Stability) Traffic flow f ∈ F is delay stable under a particular
scheduling policy if the switched queueing network is stable under that policy, and E Df is
finite; otherwise, traffic flow f is delay unstable.
31
2.3
BASTA, Little’s Law, and Delay Stability
In this section we give the “steady-state versions” of two important results in queueing theory,
the “Bernoulli Arrivals See Time Averages” property and Little’s Law, which are later used
to show delay stability/instability results. Typically, memoryless arrivals are studied in
continuous time, and their properties captured in the “Poisson Arrivals See Time Averages”
result, while Little’s Law is applied to cases where expectations are finite. In contrast, we
will be studying discrete time systems, and we will be using Little’s Law even in cases with
infinite expectations. Thus, for completeness of the thesis, we provide elementary proofs of
these results for the somewhat less popular setting that we will be applying them to.
Consider the switched queueing network described in Section 2.2. Let τf,k be the random
time slot of the arrival of the k th file of traffic flow f , k ∈ N, f ∈ F. We assign two marks to
this file: (i) the vector of queue lengths upon its arrival Qc(f ) (k) = Q1 (τf,k ), . . . , QN (τf,k ) ;
and (ii) its end-to-end delay Df (k).
Under a regenerative scheduling policy, and for fixed f ∈ F, the vector-valued sequences
Qc(f ) (k); k ∈ N and Q(t); t ∈ Z+ are, possibly delayed, aperiodic and positive re-
current regenerative processes. Therefore, they converge in distribution, and their limiting
c(f )
c(f )
c(f )
and
distributions do not depend on Q(0); see [56]. We denote by Q
= Q1 , . . . , QN
Q = Q1 , . . . , QN generic random vectors distributed according to these limiting distributions.
The arrival of files of traffic flow f constitutes a Bernoulli process with parameter pf =
P Af (0) > 0 , since all arrival processes are IID. The “Bernoulli Arrivals See Time Averages”
(BASTA) property relates the limiting distributions Qc(f ) and Q.
Theorem 2.1: (BASTA) Consider the switched queueing network of Section 2.2 under
a regenerative scheduling policy. The random vectors Q and Qc(f ) are identically distributed,
for all f ∈ F
32
Proof. Fix f ∈ F and consider the random variables
T −1
1X
1{Q(t)≤b} ,
UT =
T t=0
and
K
1 X
1 c(f )
,
VK =
K k=1 {Q (k)≤b}
where T, K ∈ N, and b ∈ ZN
+ . The conditions of Theorem 3 in [42] are satisfied, and we have
lim VK = lim UT
K→∞
T →∞
w.p.1.
Under a regenerative scheduling policy, the sequences 1{Q(t)≤b} ; t ∈ Z+ and 1{Qc(f ) (k)≤b} ;
k ∈ N are, possibly delayed, positive recurrent regenerative processes, which are also uniformly bounded by one. Then, the Ergodic theorem for regenerative processes implies that
T −1
1X
1{Q(t)≤b} = P Q ≤ b
T →∞ T
t=0
lim UT = lim
T →∞
and
K
1 X
1{Qc(f ) (k)≤b} = P Qc(f ) ≤ b
K→∞ K
k=1
lim VK = lim
K→∞
w.p.1,
w.p.1;
see [56]. Consequently,
P Q ≤ b = P Qc(f ) ≤ b ,
∀b ∈ ZN
+,
which is the desired result.
Now let Lf (t) be the number of files of traffic flow f that have at least one packet in the
network, either in queue or in service, at the beginning of time slot t. Under a regenerative
scheduling policy, the sequence Lf (t); t ∈ Z+ is a, possibly delayed, aperiodic and positive
recurrent regenerative process. A straightforward argument can then show that the sequence
Df (k); k ∈ N is also a, possibly delayed, aperiodic and positive recurrent regenerative
33
process. Hence, both processes converge in distribution, and their limiting distributions do
not depend on Q(0); see [56]. We denote by Lf and Df generic random variables distributed
according to these limiting distributions. Little’s Law relates their expected values.
Theorem 2.2: (Little’s Law) Consider the switched queueing network of Section 2.2
under a regenerative scheduling policy. Then,
E Lf = pf · E Df ,
∀f ∈ F.
This holds even if the above expectations are infinite.
Proof. First, we establish Little’s Law for the case of finite expectations. Fix a queue f ∈ F,
and assume that E Lf is finite. Consider the random variable
L̂f =
τX
1 −1
Lf (t),
t=τ0
where τ0 and τ1 represent the first two (or, in general, two consecutive) renewal epochs of
the network.
h i
h i
Initially, we prove by contradiction that E L̂f is finite. Suppose that E L̂f is infinite.
Then, the Renewal Reward theorem, combined with a truncation argument, very similar
to the one in the proof of Lemma 2.2, implies that E Lf is also infinite. However, this
h i
contradicts our assumption that E Lf is finite. Hence, E L̂f is finite.
The sequence Lf (t); t ∈ Z+ is a, possibly delayed, positive recurrent regenerative
h i
process. Combined with the fact that E L̂f is finite, the Ergodic theorem for regenerative
processes implies that
T −1
1X
Lf (t) = E Lf
T →∞ T
t=0
lim
w.p.1;
see [56].
Moreover, since the network is stable under a regenerative scheduling policy,
Df (k)
=0
k→∞
k
lim
34
w.p.1;
see Theorem 2b of [28].
The sequence Df (k); k ∈ N is also a, possibly delayed, positive recurrent regenerative
process. Then, the Ergodic theorem for regenerative processes and Theorem 2e of [28] imply
that
K
1 X
lim
Df (k) = E Df
K→∞ K
k=1
w.p.1,
and
E Lf = pf · E Df .
To summarize, starting with the assumption that E Lf is finite, we showed that E Lf =
pf · E Df . The same can be shown if we start with the assumption that E Df is finite,
and work similarly. Consequently,
E Lf < ∞ ⇐⇒ E Df < ∞,
which implies that Little’s Law holds even if the expectations are infinite.
Finally, we use BASTA and Little’s Law to prove a lemma that is used throughout the
thesis to show delay stability/instability results.
Lemma 2.1: Consider a switched queueing network that is single-hop, i.e., N = F ,
under a regenerative scheduling policy. Then,
E Qf < ∞ ⇐⇒ E Df < ∞,
∀f ∈ F.
Proof. Let us start with the implication
E Qf < ∞ =⇒ E Df < ∞,
∀f ∈ F.
Fix a traffic flow f ∈ F, and assume that E Qf is finite. Since every file has at least
35
one packet, then for all t ∈ Z+ and all b ∈ Z+ ,
P Qf (t) > b ≥ P Lf (t) > b .
We have argued that, under a regenerative scheduling policy, the sequences Qf (t); t ∈
Z+ and Lf (t); t ∈ Z+ converge in distribution. So, by taking the limit as t goes to
infinity, we have that
P Qf > b ≥ P Lf > b ,
∀b ∈ Z+ ,
which implies that
E Qf ≥ E Lf .
Combining this inequality with Little’s Law and the assumption that E Qf is finite, we
conclude that E Df is finite.
Let us now prove the implication
E Qf = ∞ =⇒ E Df = ∞,
∀f ∈ F.
Fix a traffic flow f ∈ F, and assume that E Qf is infinite. The end-to-end delay of a
file is bounded from below by the length of the respective queue upon its arrival, since the
service discipline within each queue is “First Come, First Served.” So, for all k ∈ N and all
b ∈ Z+ ,
P Df (k) > b ≥ P Qf (τf,k ) > b .
We have argued that, under a regenerative scheduling policy, the sequences Df (k); k ∈
N and Qf (τf,k ); k ∈ N converge in distribution. So, by taking the limit as k goes to
infinity, we have that
c(f )
P Df > b ≥ P Qf > b ,
36
∀b ∈ Z+ .
Combining this with the BASTA property, we get
P Df > b ≥ P Qf > b ,
∀b ∈ Z+ ,
which implies that
E Df ≥ E Qf .
Finally, the assumption that E Qf is infinite implies that E Df is infinite as well.
2.4
Truncated Rewards
Consider the switched queueing network of Section 2.2 under a regenerative scheduling policy.
By definition, there exists a sequence of stopping times τn ; n ∈ Z+ that constitute a,
possibly delayed, renewal process, i.e., the sequence τn+1 − τn ; n ∈ Z+ is IID. Moreover,
the lattice distribution of renewal periods has span equal to one and finite expectation.
Let R(t) be an instantaneous reward function, which is assumed to be a nonnegative
scalar-valued function of Q(t). We define the truncated reward as RM (t) = min R(t), M ,
where M is a positive integer. Under a regenerative scheduling policy, the sequences R(t); t ∈
Z+ and RM (t); t ∈ Z+ are, possibly delayed, aperiodic and positive recurrent regenerative processes. Consequently, they converge in distribution, and their limiting distributions
do not depend on Q(0); see [56]. Let R and RM be generic random variables distributed
according to these limiting distributions. We denote by Ragg the aggregate reward, i.e.,
M
the reward accumulated over a renewal period. Similarly, Ragg
represents the aggregate
truncated reward.
Lemma 2.2: Consider the switched queueing network of Section 2.2 under a regenerative
scheduling policy. Suppose that there exist random variable Y , with infinite expectation,
and nondecreasing function f (·), such that
lim f (M ) = ∞,
M →∞
37
and
i
h
M
.
E min Y, f (M ) ≤ E Ragg
(2.1)
Then, E[R] is infinite.
h
i
M
Proof. By definition, the length of renewal periods has finite expectation, and E Ragg
is
bounded from above by M · E[τ1 − τ0 ]. Then, the Renewal Reward theorem implies that
h
i
M
E Ragg
E[τ1 − τ0 ]
T −1
1X M
R (t)
T →∞ T
t=0
= lim
w.p.1;
(2.2)
see Section 3.4 of [23].
The sequence RM (t); t ∈ Z+ is a, possibly delayed, positive recurrent regenerative
process, which is also uniformly bounded by M . Then, the Ergodic theorem for regenerative
processes implies that
T −1
T −1
1X M
1X
lim
R (t) = lim
min R(t), M = E[min{R, M }]
T →∞ T
T →∞ T
t=0
t=0
w.p.1;
(2.3)
see [56].
Eqs. (2.1)-(2.3) imply that
E min Y, f (M )
≤ E[min{R, M }].
E[τ1 − τ0 ]
By taking the limit as M goes to infinity on both sides, and using the Monotone Convergence theorem, we obtain
E[Y ]
≤ E[R];
E[τ1 − τ0 ]
see Section 5.3 of [64].
Finally, the fact that E[Y ] is infinite implies that E[R] is infinite as well.
38
2.5
“Average Behavior” of Stochastic Processes
In this section we state two well-known corollaries of the Strong Law of Large Numbers, which
are used frequently throughout the thesis. We also provide their proofs for completeness.
Lemma 2.3: Fix arbitrary f ∈ F. For any given > 0 there exists δ() > 0, such that
t−1
X
P (λf − )t − δ() ≤
Af (τ ) ≤ (λf + )t + δ(), ∀ t ∈ N > 0.
τ =0
Proof. Fix > 0 and define the event Cm , m ∈ N, by
t−1
n 1 X
o
Cm = Af (τ ) − λf ≤ , ∀ t ≥ m .
t τ =0
By the Strong Law of Large Numbers,
P
[
Cm = 1;
m≥1
see Section 12.10 of [64].
Because the sequence of events Cm is nondecreasing, the continuity property of probabilities implies that
lim P Cm = 1.
m→∞
Let us therefore fix some T , such that P CT > 1/2. Clearly, T depends on .
Now consider the event
D=
n
0≤
T −1
X
o
Af (τ ) ≤ δ() .
τ =0
We choose δ() large enough so that P(D) > 1/2 and δ() ≥ λf T . Notice that
1 1
P CT ∩ D ≥ P CT + P(D) − 1 > + − 1 = 0.
2 2
39
Finally, note that, when both CT and D occur,
t−1
X
Af (τ ) − λf t ≤ t + δ(),
∀ t ∈ N,
τ =0
so that the latter event has positive probability, which is the desired result.
Before we state the second, and slightly stronger, result we remind our standing assumption that there exists γ ∈ (0, 1) so that all traffic flows have (1 + γ) moments.
Fix T ∈ N and γ 0 ∈ (0, γ), and consider the set of sample paths of the arrival processes
defined by
n
Hb = ω :
t−1
X
o
γ0
1
sup
max Af (τ ) − λf < (bT )− 1+γ ,
1≤t≤bT bT f ∈F τ =0
b ∈ N.
Intuitively, Hb contains those sample paths of the arrival processes that stay close to their
average behavior over the time interval [0, bT ].
Lemma 2.4: For every > 0 there exists b0 (), such that
P Hb ≥ 1 − ,
∀b ≥ b0 ().
Proof. The Marcinkiewicz-Zygmund Strong Law of Large Numbers implies that
Pt−1
τ =0
Af (τ ) − λf t
t
1
1+γ
L1
−→ 0,
∀f ∈ F;
see Theorem 10.3 of [30]. Consequently, for every fixed c > 0 there exists t0 (c), such that
t−1
i
h X
1
E Af (τ ) − λf t ≤ ct 1+γ ,
t ≥ t0 (c),
∀f ∈ F.
τ =0
nP
o
t−1
Notice that the sequence
A
(τ
)
−
λ
t;
t
∈
N
is a martingale, for every f ∈ F.
f
f
P τ =0
nP
o
t−1
Thus, the sequence
f ∈F τ =0 Af (τ ) − λf t; t ∈ N is a nonnegative submartingale.
40
γ0
Let r = bT and δr = r− 1+γ . If b is sufficiently large, then r ≥ t0 (c). Then, Doob’s
submartingale inequality (e.g., see Section 14.6 of [64]) and the Marcinkiewicz-Zygmund
Strong Law imply that
1 − P Hb
t−1
X
1
= P sup max Af (τ ) − λf t ≥ δr
1≤t≤r r f ∈F
τ =0
t−1
1 X X
Af (τ ) − λf t ≥ δr
≤ P sup
1≤t≤r r
f ∈F τ =0
≤
r−1
i
1 X h X
E Af (τ ) − λf r
δr r f ∈F
τ =0
1
≤ cN ·
r 1+γ
γ0
r1− 1+γ
γ−γ 0
= cN · r− 1+γ .
As b goes to infinity r goes to infinity, so that P Hb converges to one.
2.6
Foster-Lyapunov Criteria for Markov Chains
Almost all queueing systems that are studied in this thesis are, not only regenerative, but
also Markovian with respect to the vector of queue lengths. In other words, the sequence
Q(t); t ∈ Z+ is a Markov chain on the countable state space ZN
+ . In particular, this is the
case for single-hop networks under Max-Weight policies, which are the subject of Chapters
4 and 5, and for multi-hop networks under Back-Pressure policies, which are the subject of
Chapter 6.
A popular way of showing stability and steady-state moment bounds of Markovian queueing systems is via the so-called Foster-Lyapunov criteria. Below we provide two such criteria,
which are used frequently throughout the thesis.
Lemma 2.5 (Stability and Moment Bound): Let
Q(t); t ∈ Z+
be a time-
homogeneous, irreducible, and aperiodic Markov chain on ZN
+ . Suppose that there exist
41
constants > 0 and β < ∞, and functions V, f : ZN
+ → R+ , such that
E V Q(t + 1) − V Q(t) Ft ≤ − · f Q(t) + β,
∀t ∈ Z+ ,
and that the set q ∈ ZN
+ : f (q) ≤ β/ is finite. Then, the sequence Q(t); t ∈ Z+
converges in distribution to some random vector Q, and
β
E f (Q) ≤ .
Proof. This result follows directly from Corollary 2.1.5 of [32].
Lemma 2.6 (Stability and Exponential Bound): Let V (t); t ∈ Z+ be a timehomogeneous, irreducible, and aperiodic Markov chain on Z+ . Suppose that:
(i) there exist α < ∞ and > 0, such that
E V (t + 1) − V (t) + ; V (t) > α Ft ≤ 0,
∀t ∈ Z+ ;
(ii) there exists random variable Z on Z+ , such that E exp(θZ) = δ, for some θ >
0, δ < ∞, and, for the same α as in part (i),
P V (t + 1) − V (t) · 1{V (t)>α} + V (t + 1) − α · 1{V (t)≤α} > c Ft ≤ P(Z > c),
∀c ∈ Z+ .
Then, the sequence V (t); t ∈ Z+ converges in distribution to some random variable
V , and there exists θ0 ∈ (0, θ] such that
E exp θ0 V < ∞.
Proof. Condition (i) implies directly that the sequence
V (t); t ∈ Z+
converges in dis-
tribution to the random variable V . This is due to Foster’s stability criterion for Markov
chains; see Proposition 2.1.1 of [32].
Now, conditions (i) and (ii), and Lemma 2.1 of [31], imply the existence of θ0 ∈ (0, θ] and
42
ρ < 1, such that
E exp θ0 V (t + 1) − V (t) ; V (t) > α Ft ≤ ρ,
and
E exp θ0 V (t + 1) − α ; V (t) ≤ α Ft ≤ δ.
These conditions, combined with Theorem 2.3 of [31], imply that
1 − ρk
E exp θ0 V (t) F0 ≤ ρt · exp θ0 V (0) +
δ · exp(θ0 α).
1−ρ
(2.4)
The Reverse Fatou Lemma implies that
lim sup E exp θ0 V (t) F0 ≤ E lim sup exp θ0 V (t) F0 ;
t→∞
(2.5)
t→∞
see Section 5.4 of [64].
Moreover, since the sequence V (t); t ∈ Z+ converges in distribution, we have that
E lim sup exp θ0 V (t) F0 = E lim inf exp θ0 V (t) F0 .
t→∞
t→∞
(2.6)
Finally, Fatou Lemma implies that
E lim inf exp θ0 V (t) F0 ≤ lim inf E exp θ0 V (t) F0 .
t→∞
t→∞
(2.7)
n o
Eqs. (2.5)-(2.7) imply that the sequence E exp θ0 V (t) F0 ; t ∈ Z+ converges to
E exp θ0 V F0 . Combined with Eq. (2.4) and the aperiodicity of the Markov chain, this
gives
E exp θ0 V < ∞.
43
44
Chapter 3
Scheduling in Parallel Queues with
Heavy-Tailed Traffic
In this chapter we study the simplest scheduling problem that may arise in switched queueing
networks: how to allocate a single server to two “parallel” queues. The particular queueing
system and scheduling problem have been analyzed extensively in the past. Our point of
departure from existing literature is our assumption that one queue receives heavy-tailed
traffic, whereas the other one receives light-tailed traffic.
To better understand the potential impact of heavy-tailed traffic, consider the following
scenario: if each queue had a dedicated server with enough capacity to support the respective
arrival rate, then the heavy-tailed traffic flow would be delay unstable, and the light-tailed
traffic flow would be delay stable; these are direct corollaries of the Pollaczek-Khinchine
formula for the expected delay in a M/G/1 queue. However, the fact that the server has to
be shared, couples the evolution of the two queues and, consequently, the delays of the two
traffic flows. This may cause the light-tailed traffic flow to be delay unstable. The precise
nature of this coupling, and its potential impact on the delay stability of the light-tailed
traffic flow, depend on the scheduling policy applied.
The main contribution of this chapter is to provide insights into which scheduling
policies may perform well in the presence of heavy-tailed traffic, and why. In the simple
45
setting considered here, and under the delay stability metric, good perfomance is equivalent
to the light-tailed traffic flow being delay stable.
More specifically, we show that good performance is achieved in the following cases: (i)
if the light-tailed flow is given preemptive priority; (ii) under the Round-Robin policy, if the
arrival rate of the light-tailed flow is sufficiently low; (iii) under a parameterized version of
the Max-Weight policy, if the parameters are chosen suitably.
In contrast, we show that the delay instability of heavy tails “propagates” to the lighttailed traffic flow in the following cases: (i) under any nonpreemptive and work-conserving
policy; (ii) under preemptive priority to the heavy-tailed traffic flow; (iii) under the RoundRobin policy, if the arrival rate of the light-tailed traffic flow is high; (iv) under the MaxWeight policy.
Summarizing, in order to achieve good performance in the presence of heavy-tailed traffic,
some form of priority (partial or complete) has to be given to light-tailed traffic flows.
However, there are inherent fairness issues related to complete priority policies, as well as
stability issues that were discussed in the Introduction. Moreover, inherently fair policies,
such as deterministic or randomized time-sharing policies, perform well only for certain
arrival rates. Finally, the Max-Weight policy, which has been known to perform well in
systems of parallel queues under light-tailed traffic [26, 46, 60], actually performs poorly in
the presence of heavy tails. Our findings suggest that a parameterized version of Max-Weight,
which gives partial priority to light-tailed traffic if its parameters are chosen suitably, could
be the right policy for the particular setting.
The remainder of this chapter is organized as follows. In Section 3.1 we provide a detailed
description of the queueing system considered. In Section 3.2 we carry out a delay stability
analysis of several well-known scheduling policies. We conclude with Section 3.3 by discussing
related work and putting the findings of this chapter in perspective.
46
3.1
A Single-Server System of Parallel Queues
This section includes a detailed presentation of the queueing model considered in this chapter,
and some preliminary results. This model is a special case of the “generic” switched queueing
network of Section 2.2, so we only describe its defining characteristics here.
We consider a switched queueing network with two traffic flows, one heavy-tailed (traffic
flow 1) and one light-tailed (traffic flow 2). The network is single-hop, i.e., the traffic of each
flow is buffered in a dedicated queue while waiting for service, and exits the network as soon
as it gets served. In the general setting of Chapter 2, we have N = F = 2, so that traffic
flows are identified with queues.
The service of the two flows/queues is interdependent: if flow/queue 1 is served then
flow/queue 2 is not served, and vice versa. In the notation of Chapter 2, the set of schedules
is S = (0, 0), (1, 0), (0, 1) .
One can visualize the system as consisting of two “parallel queues” and a single server,
e.g., as depicted in Figure 3-1. The server has uninterrupted access to both queues at all
time slots. At the beginning of any given time slot, the server is allocated to at most one
of the two queues for the duration of the slot. If the server is allocated to a queue that is
nonempty, then one packet is removed from that queue.
Figure 3-1: A single server system with two parallel queues.
The queue-length dynamics of the system can be written as follows:
Qf (t + 1) = Qf (t) + Af (t) − Sf (t) · 1{Qf (t)>0} ,
47
∀t ∈ Z+ ,
∀f ∈ {1, 2},
subject to
S1 (t), S2 (t) ∈ S,
∀t ∈ Z+ .
The vector of initial queue lengths Q(0) is assumed to be an arbitrary element of Z2+ .
Regarding the arriving traffic, beyond the general assumptions made in Chapter 2, we
further assume that the sum of the arrival rates is less than the capacity of the server:
λ1 + λ2 < 1.
This assumption is critical for the stability of the system.
In this chapter we restrict our attention to scheduling policies (i.e., server allocation
policies) that are: (i) work-conserving, i.e., policies that do not allow the server to idle
unless both queues are empty; (ii) regenerative, i.e., policies under which the network
starts afresh probabilistically in certain time slots (for more details the reader is referred to
Chapter 2).
We denote by Π the set of work-conserving and regenerative scheduling policies in this
setting. It can be verified that our assumptions regarding the arrival rates and the IID nature
of the arriving traffic, combined with the work-conserving nature of the policies considered,
imply that set Π is nonempty. Moreover, every policy in Π is, by definition, stabilizing
because it is regenerative.
Lemma 3.1: (Stability of Π) The system of parallel queues described above is stable
under any scheduling policy in Π.
Proof. By definition, under a regenerative scheduling policy the vector-valued sequences
Q(t); t ∈ Z+ and D(k); k ∈ N are, possibly delayed, aperiodic and positive recurrent
regenerative processes. Hence, they converge in distribution, and their limiting distributions
do not depend on the initial queue lengths Q(0); see [56].
Proposition 3.1: (Delay Instability of Heavy Tails) In the system of parallel queues
described above, traffic flow 1 is delay unstable under any scheduling policy in Π.
48
Proof. Proposition 3.1 is a special case of Theorem 4.1 in Chapter 4.
Since there is little we can do regarding traffic flow 1, we shift our attention to the delay
stability of traffic flow 2. In contrast to heavy tails, the intrinsic burstiness of light-tailed
traffic is not sufficient to cause delay instability. However, scheduling couples the evolution
of the two queues. We will see that this coupling can cause traffic flow 2 to become delay
unstable, giving rise to a propagation of delay instability phenomenon.
3.2
3.2.1
Scheduling in the Presence of Heavy-Tailed Traffic
Nonpreemptive Policies
We begin our analysis by considering nonpreemptive scheduling policies, i.e., policies that
do not allow the server to switch between files or queues until the last packet of the file
in service gets served. The following proposition establishes that queue 2 is delay unstable
under any such policy.
Proposition 3.2: Consider the system of parallel queues of Section 3.1. Traffic flow 2
is delay unstable under any nonpreemptive scheduling policy in Π.
Proof. We will show that E Q2 is infinite under any nonpreemptive scheduling policy in
Π. Combined with Lemma 2.1, this will imply that traffic flow 2 is delay unstable. For
simplicity, we will prove this for the case where the system regenerates when empty; the
more general case can be treated similarly.
We define the following reward on the underlying renewal process:
RM (t) = min Q2 (t), M ,
where M is a finite integer.
Without loss of generality, we assume that a busy period starts at time slot 0. Consider
the set of sample paths of the system where, at time slot 0, queue 1 receives a file of size b
49
packets and queue 2 receives no traffic; we denote this set of sample paths by H(b). Since
the arrival processes are mutually independent,
P H(b) = P A1 (0) = b · P A2 (0) = 0 .
For sample paths in H(b), the server will be allocated to queue 1 between time slots 1
and b, since the scheduling policy is work-conserving and nonpreemptive. During this time
period queue 2 does not get any service.
A direct consequence of the Strong Law of Large Numbers is that for every > 0 there
exists δ > 0, such that the set of sample paths
t
n X
o
∆= A2 (τ ) − λ2 ≤ t + δ, ∀t ∈ N ,
τ =1
has positive probability (see Lemma 2.3).
We denote by H̃(b) the set of sample paths ∆ ∩ H(b). The IID nature of the arriving
traffic implies that P H̃(b) = P(∆) · P H(b) . For sample paths in H̃(b), we have
Q2 (b) =
b
X
A2 (τ ) ≥ (λ2 − )b − δ.
τ =1
Consequently, by choosing to be sufficiently small, we have that for sample paths in
H̃(b) there exist positive constants c and b0 , such that
Q2 (b) ≥ cb,
∀b ≥ b0 .
Since at most one packet from queue 2 can be served at each time slot, the length of
queue 2 is at least cb/2 over a time period of length at least cb/2 time slots. This implies
M
that the aggregate reward Ragg
, i.e., the reward accumulated over a renewal period, satisfies
50
the lower bound
M
Ragg
· 1{b≥b0 } · 1H̃(b) ≥ min
n cb 2
2
o
· 1{b≥b0 } , M 2 · 1H̃(b) .
Then, the expected aggregate reward satisfies
∞
h
i X
h
i
M
M
E Ragg ≥
E Ragg · 1{b≥b0 } · 1H̃(b)
b=1
∞
o
n cb 2
X
· 1{b≥b0 } , M 2 · P A1 (0) = b .
≥ P(∆) · P A2 (0) = 0 ·
min
2
b=1
Thus, there exists a positive constant c0 such that
oi
h
i
h
n cA (0) 2
1
M
· 1{A1 (0)≥b0 } , M 2 .
E Ragg
≥ c0 E min
2
Lemma 2.2 applied to Y = (1/4)c2 A21 (0) · 1{A1 (0)≥b0 } implies that E Q2 is infinite, since
traffic flow 1 is heavy-tailed. Finally, Lemma 2.1 implies that traffic flow 2 is delay unstable.
3.2.2
Preemptive Priority Policies
The delay instability result of Proposition 3.2 urges us to consider preemptive scheduling
policies. We start by analyzing very simple but popular members of this class, namely
preemptive priority policies.
We define the “work-conserving priority to f ” policy, f ∈ {1, 2}, as follows: at each time
slot, if queue f is nonempty then the server is allocated to queue f ; otherwise, the server
is allocated to the other queue. The following proposition characterizes the performance of
preemptive priority policies.
Proposition 3.3: Consider the system of parallel queues of Section 3.1. Traffic flow 2 is:
(i) delay stable under the “work-conserving priority to 2” policy; (ii) delay unstable under
the “work-conserving priority to 1” policy.
51
Proof. Let us start with the “work-conserving priority to 2” policy. In that case, queue 2 is
a stable M/G/1 queue in discrete time, with finite second moment of service time. Then,
the Pollaczeck-Khinchine formula implies that traffic flow 2 is delay stable.
Turning our attention to the “work-conserving priority to 1” policy, we denote by τ2,k the
random time slot of the arrival of the k th file to queue 2. Under “work-conserving priority
to 1,” we have that
D2 (k) ≥ Q1 (τ2,k ),
∀k ∈ N,
which implies that
P D2 (k) > d ≥ P Q1 (τ2,k ) > d ,
∀k ∈ N,
∀d ∈ Z+ .
It can be verified that the “work-conserving priority to 1” is a regenerative scheduling
policy. Thus, the sequence Q1 (τ2,k ); k ∈ N is a, possibly delayed, aperiodic and positive
recurrent regenerative processes and, hence, converges in distribution. Moreover, its limiting
c(2)
distribution, Q1 , does not depend on Q(0). So, by taking the limit as k goes to infinity on
both sides of the above inequality, we get
c(2)
P D2 > d ≥ P Q1 > d ,
∀d ∈ Z+ .
c(2) d
Finally, the BASTA property states that Q1
= Q1 (see Theorem 2.1). Consequently,
P D2 > d ≥ P Q1 > d ,
∀d ∈ Z+ ,
which implies that E D2 ≥ E Q1 .
Combining this with Proposition 3.1, we have that traffic flow 2 is delay unstable.
52
3.2.3
The Round-Robin Policy
The inherent fairness issues associated with priority policies lead us to consider the “workconserving Round-Robin” policy: during odd time slots, if queue 1 is nonempty then the
server is allocated to queue 1; otherwise, the server is allocated to queue 2. Similarly, during
even time slots, if queue 2 is nonempty then the server is allocated to queue 2; otherwise,
the server is allocated to queue 1.
Proposition 3.4 summarizes the performance of this policy, pointing out a phenomenon
that we have not encountered thus far: under the “work-conserving Round-Robin” policy
the delay stability of traffic flow 2 depends on its arrival rate.
Proposition 3.4: Consider the system of parallel queues of Section 3.1 under the “workconserving Round-Robin” policy. Traffic flow 2 is: (i) delay stable if λ2 <
1
;
2
(ii) delay
unstable if λ2 > 12 .
Proof. (i) Let us first look at the case where λ2 < 12 . Consider a fictitious queue 20 that has
exactly the same arrivals and initial length as queue 2. Queue 20 is served only at even time
slots, if nonempty, at rate one packet per slot. Therefore, it does not get the extra service
slots that queue 2 gets whenever queue 1 is empty. We denote by Q20 (t) the length of this
fictitious queue at time slot t.
Notice that queue 20 is a stable M/G/1 queue in discrete time. Thus, the sequence
Q20 (t); t ∈ Z+ converges in distribution, and its limiting distribution, Q20 , satisfies
E Q20 < ∞.
On the other hand, a straightforward inductive argument can show that the length of
queue 20 is no less than the length of queue 2, at all time slots. Hence,
P Q2 (t) > b ≤ P Q20 (t) > b ,
53
∀t ∈ Z+ ,
∀b ∈ Z+ .
By taking the limit as t goes to infinity, we have that
P Q2 > b ≤ P Q20 > b ,
∀b ∈ Z+ ,
which implies that
E Q2 ≤ E Q20 .
Consequently,
E Q2 < ∞,
which, combined with Lemma 2.1, implies the delay stability of traffic flow 2;
(ii) Let us now look at the case where λ2 > 21 . Consider a fictitious system with two
parallel queues, denoted by 10 and 20 , respectively. Queue 10 has the same arrivals and initial
length as queue 1 but, instead of being served according to “work-conserving Round-Robin,”
it is served at rate one packet per time slot whenever it is nonempty. Queue 20 has the same
arrivals and initial length as queue 2, and is served at unit rate at: (i) even time slots; (ii)
odd time slots, if queue 10 is empty.
Let Qf (t) be the length of queue f ∈ {10 , 20 }, at time slot t. It can be verified that the
sequence Qf (t); t ∈ Z+ converges in distribution to some Qf , for all f ∈ {10 , 20 }.
We assume that both the actual and the fictitious system are in steady state, and we
observe them at an arbitrary time slot τ . First, we compute the expected length of queue
20 at time slot τ . Consider the events Γ(τ ) = 10 nonempty at time τ and its complement,
Γc (τ ). We can express the expected length of queue 20 as follows:
E Q20 (τ ) = E Q20 (τ ) Γ(τ ) · P Γ(τ ) + E Q20 (τ ) Γc (τ ) · P Γc (τ ) .
It can be verified that both P Γ(τ ) and P Γc (τ ) are bounded away from 0, since
λ1 ∈ (0, 1). This implies that the above conditional expectations are well-defined.
Conditional on the event Γ(τ ), we denote by Z(τ ) the age of the busy period of queue 10
at time slot τ . Clearly, this random variable dominates stochastically the age of the file in
54
service at time slot τ , i.e., the number of packets of this file that have been served already.
From renewal theory we know that the expectation of the latter is proportional to the second
moment of the file size; see Section 1.6.2 of [32]. Hence, the expected age of the file in service
at time slot τ is infinite, which leads to
E Z(τ ) Γ(τ ) = ∞.
Conditional on Γ(τ ), let τ̃ be the random time slot that initiates the particular busy
period of queue 10 . Let A20 (t); t ∈ Z+ and S20 (t); t ∈ Z+ represent the arrivals and
departures from queue 20 , respectively. It can be verified that
τ −1
X
Q20 (τ ) ≥
A20 (t) − S20 (t) .
t=τ̃
Conditioning on the event Γ(τ ) does not change the statistics of the arrivals of 20 , which
are identical to the arrivals of queue 2. Furthermore, conditional on the event Γ(τ ), queue
20 is served once every two time slots. Hence, the length of queue 20 during a busy period of
10 is a reflected random walk with positive drift λ2 − 21 .
Starting from time slot τ and going backwards in time, time slot τ̃ is a stopping time for
the process Q20 (t); t ∈ Z+ . Therefore,
1
1
t− ,
E Q20 (τ ) Γ(τ ), Z(τ ) = t ≥ λ2 −
2
2
which implies that
1
1 E Q20 (τ ) Γ(τ ) ≥ λ2 −
E Z(τ ) Γ(τ ) − .
2
2
Consequently,
E Q20 (τ ) Γ(τ ) = ∞,
55
and finally
E Q20 = E Q20 (τ ) = ∞.
To complete the proof, we will show that the steady-state length of queue 2 stochastically
dominates the steady-state length of queue 20 .
An easy inductive argument can show that
Q1 (t) ≥ Q10 (t),
∀t ∈ Z+ .
We now focus on the evolution of queues 2 and 20 . Both queues are served on two
occasions: (i) at even time slots; and (ii) at odd time slots, whenever queue 1 and 10 is
empty, respectively. Since the length of queue 1 is no less than the length of queue 10 at all
time slots, the service opportunities for the two queues must satisfy
S2 (t) ≤ S20 (t),
t ∈ Z+ .
An inductive argument can show that
Q2 (t) ≥ Q20 (t),
t ∈ Z+ .
By taking the limit as t goes to infinity, and using the fact that both sequences converge
in distribution, we have that
E Q2 ≥ E Q20 = ∞.
Finally, Lemma 2.1 implies that traffic flow 2 is delay unstable.
3.2.4
A Randomized Policy
Next, we analyze the “work-conserving randomized” scheduling policy, which works as follows. A biased coin is tossed at each time slot: if heads come up, an event of probability q,
56
then queue 2 is served; if tails come up, queue 1 is served. If a queue is chosen for service
while empty, then the other queue is served instead. Coin tosses are assumed to be IID
Bernoulli trials, independent of the past history of the system.
Proposition 3.5: Consider the system of parallel queues of Section 3.1 under the “workconserving randomized” policy. Traffic flow 2 is: (i) delay stable if λ2 < q; (ii) delay unstable
if λ2 > q.
Proof. The proof is very similar to the proof of Proposition 3.4 and, therefore, omitted.
3.2.5
The Max-Weight Policy
The preemptive scheduling policies that we have analyzed so far are “queue-length blind,” in
the sense that their scheduling decisions do not depend on queue lengths (other than being
zero or nonzero). We now turn to a well-known “queue-length aware” policy, which is the
Max-Weight scheduling policy. In the setting of this chapter, Max-Weight is equivalent to
“Longest Queue First,” i.e., the server is allocated to the longest queue at any given time
slot. Ties are broken uniformly at random.
Proposition 3.6: Consider the system of parallel queues of Section 3.1. Traffic flow 2
is delay unstable under the Max-Weight scheduling policy.
Proof. Proposition 3.6 is a special case of Theorem 4.2.
3.2.6
The Max-Weight-α Policy
Proposition 3.6 suggests that the Max-Weight scheduling policy performs poorly in the
presence of heavy-tailed traffic, at least in terms of the delay stability. Thus, we turn to
a generalized version of Max-Weight, termed the Max-Weight-α scheduling policy: at time
slot t, the server is allocated to a queue in the set
arg max Qα1 1 (t), Qα2 2 (t) ,
57
where α1 and α2 are positive constants. If both queues are included in this set, then the
server is allocated uniformly at random.
Proposition 3.7: Consider the system of parallel queues of Section 3.1 under the Maxi
h
α +1
Weight-α scheduling policy. If the parameters of the policy are such that E Af f (0) is
finite, for all f ∈ {1, 2}, then
i
h
α
E Qf f < ∞,
∀f ∈ {1, 2}.
Proof. It can be verified that both the arrival processes and the scheduling policy satisfy the
conditions of Theorem 1 of [21], and the result follows directly. Alternatively, Proposition
3.7 is a special case of Theorem 4.3.
We remind the reader of our assumptions: (i) that there exists γ ∈ (0, 1), such that
h
i
h
i
2
E A1+γ
(0)
is
finite;
(ii)
that
E
A
(0)
is finite.
1
2
Then, an immediate corollary of Proposition 3.7 and Lemma 2.1 is the following: if
α2 is equal to one and α1 is sufficiently small, then traffic flow 2 is delay stable under the
Max-Weight-α scheduling policy.
3.2.7
The Max-Weight-log Policy
The Max-Weight-α policy performs well in the presence of heavy-tailed traffic, at least in
terms of the delay stability. However, proper values of the α-parameters have to be selected,
and for this to happen some knowledge of the higher order moments of both arrival processes
is required. If this requirement cannot be met, then alternative policies have to be considered.
One alternative policy is the Max-Weight-log policy: at time slot t, the server is allocated
to a queue in the set
arg max log 1 + Q1 (t) , log 1 + Q2 (t) .
If both queues are in this set, then the server is allocated uniformly at random.
58
Proposition 3.8: Consider the system of parallel queues of Section 3.1. Under the
Max-Weight-log scheduling policy,
E log 1 + Qf < ∞,
∀f ∈ {1, 2}.
Proof. It can be verified that both the arrival processes and the Max-Weight-log scheduling
policy satisfy the conditions of Theorem 1 of [21], and hence the result follows.
The performance guarantee provided by Proposition 3.8 is rather weak, and not sufficient
for the delay stability traffic flow 2. However, Max-Weight-log can be used as the basis for
hybrid policies, e.g., give suitable polynomial weights to the queues whose higher order
moments of arrivals are known, and give logarithmic weights to the other queues.
Example: Consider the system of parallel queues of Section 3.1, and further assume
that traffic flow 2 is exponential-type. Consider a scheduling policy that allocates the server,
at time slot t, to a queue in the set
arg max
n
o
log 1 + Q1 (t) , Qα2 2 (t) .
By choosing suitably parameter α2 , we can guarantee the finiteness of higher order queuelength moments for queue 2, and, consequently, the delay stability of traffic flow 2. This
result follows from Theorem 1 of [21], combined with Lemma 2.1.
3.3
Concluding Remarks
The main insight derived from this chapter is that the scheduling constraints of a switched
queueing network couple the service of heavy-tailed and light-tailed traffic flows, which, in
turn, may cause delay instability to propagate. We illustrated this phenomenon under a
variety of scheduling policies, ranging from nonpreemptive to preemptive, and from queuelength blind to queue-length aware.
59
We make some concluding remarks that will help put the findings of this chapter in
perspective. Our result regarding the propagation of delay instability under nonpreemptive
policies can be viewed as generalizing the observations of Anantharam [1]; the latter paper
considers the intra-queue scheduling problem of a single-class single-server queue (i.e., the
way that files/jobs are served within the queue). Another point that was made in this chapter
is that delay instability may or may not propagate under queue-length blind policies, such as
Round-Robin, depending on the arrival rates. This result agrees with the findings of Borst
et al. [8], which considers a two-class single-server queue under the Generalized Processor
Sharing policy (essentially, the limit of Round-Robin as the duration of time slots goes to
zero). Finally, one of the main results of this chapter was that delay instability propagates
under the Max-Weight policy, whereas this propagation can be prevented if Max-Weight is
suitably modified. By and large, the remainder of the thesis (Chapters 4, 5, and 6) is devoted
to delay stability analysis of Max-Weight policies in switched queueing networks with more
complex structure.
A remark should be made on the literature related to the Max-Weight-α policy. In studies
where this policy appears explicitly, the same α-parameter is used for all queues/flows; e.g.,
see [14, 53]. This fails to capture the intuition that some form of priority should be given
to light-tailed traffic. However, the Max-Weight-α policy has appeared implicitly in the
work of Eryilmaz et al. [21], as a special case of a Max-Weight-type policy of a more general
functional form. Although the focus of the latter work is not on heavy tails but on the impact
of fading wireless channels, it provides sufficient conditions for the finiteness of steady-state
queue-length moments, from which the conditions of Proposition 3.7 can be derived.
Related to this chapter is also the work by Boxma et al. [10], which analyzes a M/G/2
queue with a heavy-tailed and a light-tailed server, and shows a dependence of the queuelength asymptotics on the arrival rate to the queue. Similar connections are established in
the work of Borst et al. [9], in the context of two coupled queues.
The material included in this chapter is the main content of publication [43]. There
has been a fair amount of work towards extending the findings of [43] in various directions.
60
Jagannathan et al. [35] determine the precise queue-length asymptotics of the Max-Weightα and Max-Weight-log policies, providing, thus, a more refined performance analysis. The
same authors have studied the case where the server has intermittent connectivity to the
queues, a setting that may arise in wireless networks [34]. Nair et al. [45] analyze the role of
intra-queue scheduling on the queue-length asymptotics of the Max-Weight-α policy. Finally,
the work of Oguz & Anantharam [47] provides sufficient conditions for the propagation of
long-range dependence in Markov chains, a problem that, in nature, is very related to the
objectives of this thesis.
61
62
Chapter 4
Max-Weight Scheduling in Networks
with Heavy-Tailed Traffic
In this chapter we carry out a delay stability analysis of Max-Weight policies, in the context
of single-hop switched queueing networks with heavy-tailed traffic. As mentioned already in
the Introduction of this thesis, single-hop networks have been used extensively to capture the
dynamics and decisions in data communication networks (e.g., wireless networks [27], inputqueued switches [44]), flexible manufacturing systems [25], and cloud computing facilities
[41]. Max-Weight policies have become the benchmark operating policies for these queueing
systems, primarily due to their throughput optimality [59] and asymptotic delay optimality
[57] properties.
The motivating factor for the analysis in this chapter, and much of the remainder of the
thesis, are the findings of Chapter 3: Proposition 3.6 suggests that Max-Weight performs
poorly in the presence of heavy tails, even though the same policy is known to perform
very well under light-tailed traffic. Moreover, according to Proposition 3.7, if Max-Weight
is modified properly so that partial priority is given to light-tailed traffic, then much better
overall performance can be achieved (at least, with respect to the delay stability metric).
These observations were made in a simple setting of two parallel queues, sharing a single
server. The goal of this chapter is to analyze queueing networks with more complex structure
63
and scheduling constraints, to check to what extent the poor performance of Max-Weight
persists, and, if necessary, to identify ways to remedy the situation.
The main contributions of this chapter can be summarized as follows.
(i) We show that, under the Max-Weight policy, any light-tailed traffic flow that conflicts
(i.e., cannot be served simultaneously) with a heavy-tailed flow is delay unstable;
(ii) We show that, for certain admissible arrival rates, a light-tailed flow can be delay
unstable even if it does not conflict with heavy-tailed traffic;
(iii) We analyze the Max-Weight-α policy and show that, with proper choice of the
α-parameters, this policy achieves optimal performance with respect to the delay stability metric. Moreover, we show that Max-Weight-α achieves the optimal scaling of higher
moments of steady-state queue lengths with traffic intensity.
The remainder of the chapter is organized as follows. Section 4.1 includes a detailed
presentation of the queueing model considered. In Section 4.2 we motivate the subsequent
development by presenting, informally and through simple examples, the main results of
this chapter. In Section 4.3 we analyze the performance of the Max-Weight scheduling
policy. Section 4.4 contains the analysis of the Max-Weight-α scheduling policy, and of the
performance that it achieves in terms of delay stability. This section also includes results
about the scaling of moments of steady-state queue lengths with traffic intensity and the
size of the network. We conclude with a brief discussion of the findings of this chapter in
Section 4.5.
4.1
A Single-Hop Switched Queueing Network
This section includes a detailed presentation of the queueing model considered in this chapter,
together with some necessary definitions and preliminary results. This model is a special case
of the “generic” switched queueing network of Section 2.2, so we only describe its defining
characteristics here.
We consider a single-hop switched queueing network, i.e., the traffic of each flow is
64
buffered in a dedicated queue while waiting for service, and exits the network as soon as
it gets served. In the general setting of Chapter 2, we have a switched network with F ≥ 2
traffic flows and N = F queues. So, all queues receive exogenous traffic, and queues are
identified with flows.
We recall that, in a switched queueing network, not all queues can be served simultaneously. In this chapter we assume that the set of allowable schedules S is an arbitrary subset
of the powerset of F = {1, . . . , F }. In other words, our analysis applies for any type of
scheduling constraints. For convenience, we identify schedules with vectors in {0, 1}F .
Using the notation notation of Chapter 2, we can write the queue-length dynamics as
follows:
Qf (t + 1) = Qf (t) + Af (t) − Sf (t) · 1{Qf (t)>0} ,
∀f ∈ F,
subject to
S(t) ∈ S,
for all t ∈ Z+ . The vector of initial queue lengths Q(0) is assumed to be an arbitrary element
of ZF+ .
The stability of a switched queueing network depends on the arrival rates of the various
traffic flows relative to the service rates of the servers and the scheduling constraints. This
relation is captured by the stability region of the network.
Definition 4.1: (Stability Region) The stability region Λ of a single-hop switched
queueing network is the set of arrival rate vectors
n
o
X
X
λ ∈ RF+ ∃ ζs ∈ R+ , s ∈ S : λ ≤
ζs · s,
ζs < 1 .
s∈S
s∈S
In other words, an arrival rate vector λ belongs to Λ if there exists a convex combination
of schedules that covers the rates of all traffic flows. If an arrival rate vector is in the stability
region of the network, then the traffic corresponding to this vector is called admissible, and
there exists a scheduling policy under which the network is stable.
65
Throughout the chapter, except for our standing assumptions regarding the arrival processes made in Chapter 2, we also assume that the arriving traffic is admissible.
Definition 4.2: (Traffic Intensity) The traffic intensity of an arrival rate vector λ ∈ Λ
is defined as follows:
ρ(λ) = inf
nX
s∈S
o
X
ζs λ ≤
ζs · s; ζs ∈ R+ , ∀s ∈ S .
s∈S
Clearly, traffic with arrival rate vector λ is admissible if and only if ρ(λ) < 1.
Theorem 4.1: (Delay Instability of Heavy Tails) Consider the switched queueing
network described above under a regenerative scheduling policy. Every heavy-tailed traffic
flow is delay unstable.
Proof. Consider a heavy-tailed traffic flow h ∈ F. We will show that E Qh is infinite under
any regenerative scheduling policy. Combined with Lemma 2.1, this will imply that traffic
flow h is delay unstable.
Consider a fictitious queue h̃, which has exactly the same arrivals and initial length as
queue h, but is served at unit rate whenever nonempty. We denote by Qh̃ (t) the length of
queue h̃ at time slot t. Since the arriving traffic is assumed admissible, the queue-length
process Qh̃ (t); t ∈ Z+ converges to a limiting distribution Qh̃ .
An easy inductive argument can show that the length of queue h dominates the length
of queue h̃ at all time slots, under any regenerative scheduling policy. This implies that for
all t ∈ Z+ and all b ∈ Z+ ,
P Qh (t) > b ≥ P Qh̃ (t) > b .
By taking the limit as t goes to infinity, and using the fact that both queue-length
processes converge in distribution, we have that
P Qh > b ≥ P Qh̃ > b ,
∀b ∈ Z+ .
So, in order to prove the desired result, it suffices to show that E Qh̃ is infinite. This
66
follows immediately from the Pollaczeck-Khinchine formula, if we notice that queue h̃ is a
stable M/G/1 queue with infinite variance of service time. In the remainder of the proof we
provide a self-contained argument, which utilizes Lemmas 2.1 and 2.2.
The length of queue h̃ evolves as a positive recurrent Markov chain, and the empty state
is recurrent. Hence, the time slots that initiate busy periods of queue h̃ constitute a, possibly
delayed, renewal process. We define an instantaneous reward on this renewal process:
RM (t) = min Qh̃ (t), M ,
∀t ∈ Z+ ,
where M is some finite integer.
Without loss of generality, assume that a busy period starts at time slot 0, and let b be
the size of the file that initiates it. Since queue h̃ is served at unit rate, its length is at least
b/2 packets over a time period of duration b/2 time slots. This implies that the aggregate
M
reward Ragg
, i.e., the reward accumulated over a renewal period, is bounded from below as
follows:
M
Ragg
≥
nb
o
n b2
o
b
· min
, M ≥ min
, M2 .
2
2
4
Consequently, the expected aggregate reward is bounded from below as follows:
∞
h
i X
n b2
o
h
n A2 (0)
oi
h
M
2
2
min
, M · P Ah (0) = b = E min
,M
.
E Ragg ≥
4
4
b=0
Then, Lemma 2.2 applied to Y = (1/4)A2h (0), implies that E Qh̃ is infinite. This, in
turn, implies that E Qh is infinite, which, combined with Lemma 2.1, gives the desired
result.
It should be noted that Theorem 4.1 is proved under the assumption that the “First
Come, First Served” discipline is used within the queue. Indeed, a heavy-tailed flow could
be delay stable under other intra-queue service disciplines, e.g., preemptive “Last Come,
First Served” or Processor-Sharing; see [11]. However, the focus of this chapter is on the
impact of heavy-tailed traffic on light-tailed flows, under Max-Weight policies. Since Max67
Weight policies are queue-length based, the main findings of this chapter that characterize
this impact (Theorem 4.2, Propositions 4.1 and 4.2, Corollary 4.1) remain true irrespective
of the service discipline within each queue.
4.2
Overview of Main Results
In this section we introduce, informally and through simple examples, the main results of
this chapter and the basic intuition behind them.
Let us start by revisiting Chapter 3. Consider the single-server system of two parallel
queues depicted in Figure 3-1. Traffic flow 1 is heavy-tailed, whereas traffic flow 2 is lighttailed. The server is allocated according to the Max-Weight scheduling policy, which is
equivalent to “Serve the Longest Queue” in this simple setting. Proposition 3.6 states that
traffic flow 2 is delay unstable under this policy. The intuition behind this result is that queue
1 is, occasionally, very long (infinite, in steady-state expectation) because of its heavy-tailed
arrivals. When this happens, and under the Max-Weight policy, queue 2 has to build up to
a similar length in order to receive service. A very long queue implies very large delays on
average, which, in turn, leads to delay instability.
The goal of this chapter is to go beyond parallel queues, and to analyze switched queueing networks with more complex structure and scheduling constraints. An example is the
queueing network of Figure 4-1, where traffic flow 1 is assumed to be heavy-tailed, whereas
traffic flows 2 and 3 are light-tailed. The server can serve either queue 1 alone, or queues 2
and 3 simultaneously. In this setting the Max-Weight policy compares the length of queue
1 to the sum of the lengths of queues 2 and 3, and serves the “heavier” schedule.
The intuition from the previous example suggests that, at least one of the queues 2 and
3 has to build up to the order of magnitude of queue 1, in order for these queues to receive
service. In other words, we expect that at least one of the traffic flows 2 and 3 is delay
unstable under Max-Weight. The findings of this chapter imply that, in fact, both traffic
flows are delay unstable. The main idea behind this result is the following: with positive
68
Figure 4-1: Propagation of delay instability: the heavy-tailed flow 1 causes the conflicting
light-tailed flows 2 and 3 to become delay unstable.
probability, the arrival processes to queues 2 and 3 exhibit their “average” behavior. In that
case, the corresponding queues build up slowly and together, which implies that when they
finally claim the server they have both built up to the order of magnitude of queue 1.
The simple networks of Figures 3-1 and 4-1 illustrate special cases of a general result:
every light-tailed flow that conflicts with a heavy-tailed flow is delay unstable under the
Max-Weight policy. For more details see Theorem 4.2.
Figure 4-2: Propagation of delay instability: the heavy-tailed flow 1 may cause the nonconflicting light-tailed flow 2 to become delay unstable.
Going one step further, consider the queueing network of Figure 4-2. Traffic flow 1 is
assumed to be heavy-tailed, whereas traffic flows 2 and 3 are light-tailed. The server can
serve either queues 1 and 2 simultaneously, or queue 3 alone. In this setting the Max-Weight
policy compares the length of queue 3 to the sum of the lengths of queues 1 and 2, and serves
the “heavier” schedule. The intuition from the previous examples suggests that traffic flow
69
3 is delay unstable, but the real question concerns the delay stability of traffic flow 2. One
would expect that this flow is delay stable: it is light-tailed itself, and is served together with
a heavy-tailed flow, which should result in more service opportunities under Max-Weight.
Surprisingly, we show that traffic flow 2 is delay unstable if its arrival rate is sufficiently high
(but in the stability region). The key observation is that, even though traffic flow 2 does not
conflict with heavy-tailed traffic, it does conflict with traffic flow 3, which is delay unstable
because it conflicts with heavy-tailed traffic. Conversely, we also show that traffic flow 2 is
delay stable if its arrival rate is sufficiently low. For more details see Propositions 4.1 and
4.2.
The examples above suggest that in queueing networks with heavy-tailed traffic, delay
instability not only appears but also propagates under the Max-Weight policy. Seeking
a remedy to this situation, we turn to the paremeterized Max-Weight-α scheduling policy.
This policy assigns a positive α-parameter to each traffic flow, and instead of using the queue
lengths to calculate the weight of a schedule, it uses the respective α-powers of the queue
lengths. Proposition 3.7 of the previous chapter implies that, in the network of Figure 3-1,
we can guarantee that traffic flow 2 is delay stable provided the α-parameter of traffic flow
1 is sufficiently small. In other words, we can prevent the propagation of delay instability.
This is a special case of a general result: if the α-parameters of the Max-Weight-α policy
are chosen suitably, then the sum of the α-moments of steady-state queue lengths is finite.
For more details see Theorem 4.3.
4.3
Max-Weight Scheduling
In this section we evaluate the performance of the Max-Weight scheduling policy in terms
of the delay stability of traffic flows. Informally speaking, the “weight” of a schedule is the
sum of the lengths of all queues included in it. As its name suggests, the Max-Weight policy
activates a schedule that has maximum weight, at any given time slot. More formally, under
70
the Max-Weight policy, the scheduling vector S(t) satisfies
S(t) ∈ arg max
nX
(Sf )∈S
o
Qf (t) · Sf .
f ∈F
If the set on the right-hand side includes multiple schedules, then one of them is chosen
uniformly at random. The following lemma states that the network is stable under the MaxWeight policy. Essentially, this result is well-known, e.g., for light-tailed traffic, see [59];
for more general arrivals, see [57]. A subtle point is that, in this thesis, we have adopted a
slightly different definition for stability. So, we need to ensure that, apart from the sequences
of queue lengths, the sequences of file delays converge as well.
Lemma 4.1: (Stability under Max-Weight) The switched queueing network of Section 4.1 is stable under the Max-Weight scheduling policy.
Proof. Consider the switched queueing network of Section 4.1 under the Max-Weight schedul
ing policy. It can be verified that the sequence Q(t); t ∈ Z+ is a time-homogeneous, irreducible, and aperiodic Markov chain on the countable state-space ZF+ . Proposition 2 of [57]
implies that this Markov chain is also positive recurrent. Hence, Q(t); t ∈ Z+ converges
in distribution, and its limiting distribution does not depend on Q(0). Based on this, it can
be verified that the sequence D(k); k ∈ N is a, possibly delayed, aperiodic and positive
recurrent regenerative process. Therefore, it also converges in distribution, and its limiting
distribution does not depend on Q(0); see [56].
4.3.1
Conflicting with Heavy Tails
Next, we state one of the main results of this chapter, which generalizes our observations
from the simple networks of Figures 3-1 and 4-1. Before we give the result, let us define
precisely the notion of conflict between traffic flows.
Definition 4.3: Traffic flow f conflicts with f 0 , and vice versa, if there exists no schedule
in S that includes both f and f 0 .
71
Theorem 4.2: (Conflicting with Heavy Tails) Consider the switched queueing network of Section 4.1 under the Max-Weight scheduling policy. Every light-tailed traffic flow
that conflicts with a heavy-tailed flow is delay unstable.
Proof. Consider a heavy-tailed traffic flow h, and a light-tailed flow l that conflicts with h.
We will show that E Ql is infinite under the Max-Weight scheduling policy. Combined with
Lemma 2.1, this will imply that traffic flow l is delay unstable.
Notice that the vector of queue lengths evolves as a positive recurrent Markov chain, and
the empty state is recurrent. Hence, the time slots that initiate busy periods of the system
constitute a, possibly delayed, renewal process. We define an instantaneous reward on this
renewal process:
RM (t) = min Ql (t), M ,
∀t ∈ Z+ ,
where M is a positive integer.
Without loss of generality, assume that a busy period of the network starts at time slot
zero. Consider the set of sample paths where, at time slot zero, queue h receives a file of size
b > 0 packets and all other queues receive no traffic. We denote this set of sample paths by
Hb . Since the arrival processes of different traffic flows are mutually independent,
Y
P Hb = P Ah (0) = b ·
P Af (0) = 0 .
f 6=h
This quantity is positive as long as b is in the support of Ah (0), because the rate vector is
admissible, hence λf < 1 and P Af (0) = 0 ≥ 1 − λf > 0.
For sample paths in Hb , denote by Tb the first time slot when the length of queue h
becomes less than or equal to the sum of the lengths of all other queues:
X
n
o
Tb = min t > 0 Qf (t) ≥ Qh (t) · 1Hb .
f 6=h
Under the Max-Weight scheduling policy, queue l receives no service until time slot Tb .
72
Moreover, queue h is served at unit rate. So, for sample paths in Hb ,
b − (Tb − 1) ≤ Qh (Tb ) ≤
X
Qf (Tb ) =
f 6=h
b −1
X TX
Af (t).
f 6=h t=1
A direct consequence of the Strong Law of Large Numbers is that for every > 0 there
exists δ > 0, such that the set of sample paths
t
n X
o
∆= Af (τ ) − λf ≤ t + δ, ∀t ∈ N, ∀f 6= h ,
τ =1
has positive probability (see Lemma 2.3). We denote by H̃b the set of sample paths ∆ ∩ Hb .
Due to the IID nature of the arriving traffic,
P H̃b = P(∆) · P Hb .
For sample paths in H̃b , we have
b − (F − 1) · δ
,
Tb − 1 ≥ P
λ
+
+1
f
f 6=h
where (F − 1) is the cardinality of the set {f ∈ F : f 6= h}.
Moreover,
Ql (Tb ) =
TX
b −1
Al (t) ≥ λl − · Tb − 1 − δ.
t=1
Consequently, there exist positive constants c and b0 such that, for every sample path in
H̃b ,
Ql Tb ≥ cb,
∀b ≥ b0 .
Since at most one packet from queue l can be served at each time slot, the length of
queue l is at least cb/2 over a time period of duration cb/2 time slots. This implies that
M
the aggregate reward Ragg
, i.e., the reward accumulated over a renewal period, satisfies the
73
lower bound
M
· 1{b≥b0 } · 1H̃b ≥ min
Ragg
n cb 2
2
o
· 1{b≥b0 } , M 2 · 1H̃b .
Then, the expected aggregate reward satisfies
∞
h
i
o
n cb 2
Y
X
M
E Ragg ≥ P(∆) ·
· 1{b≥b0 } , M 2 · P Ah (0) = b .
P Af (0) = 0 ·
min
2
f 6=h
b=1
So, there exists a positive constant c0 , such that
i
h
n cA (0) 2
h
oi
h
M
≥ c0 E min
E Ragg
· 1{Ah (0)≥b0 } , M 2 .
2
Lemma 2.2 applied to Y = (1/4)c2 A2h (0) · 1{Ah (0)≥b0 } implies that E Ql is infinite. Then,
Lemma 2.1 gives the desired result.
We emphasize the generality of this result. Namely, a light-tailed flow that conflicts with
heavy-tailed traffic is delay unstable, irrespective of: (i) the arrival rates; (ii) the precise tail
asymptotics; (iii) other scheduling constraints in the network. Hence, we view Theorem 4.2
as capturing a universal phenomenon for the propagation of delay instability.
4.3.2
Nonconflicting with Heavy Tails
So far we have shown that: (i) a heavy-tailed traffic flow is delay unstable under any regenerative scheduling policy; (ii) a light-tailed traffic flow that conflicts with a heavy-tailed flow is
delay unstable under the Max-Weight scheduling policy. It seems reasonable to assume that
a light-tailed flow that does not conflict with heavy-tailed traffic is delay stable. Surprisingly,
this is not always the case. We demonstrate this by means of a simple example.
Let us come back to the queueing system of Figure 4-2. The schedules are {1, 2} and {3},
and all queues are served at unit rate whenever the respective schedules are activated. The
rate vector λ = λ1 , λ2 , λ3 is assumed to be admissible. The following proposition shows
that traffic flow 2 is delay unstable if its arrival rate is sufficiently high.
74
Proposition 4.1: (Rate-Dependent Delay Instability) Consider the switched queueing network of Figure 4-2, with admissible traffic and under the Max-Weight scheduling
policy. If the rates satisfy λ2 > 1 + λ1 − λ3 /2, then traffic flow 2 is delay unstable.
Before proceeding to the formal proof of Proposition 4.1, we provide an intuitive outline
of the argument, also aimed at explaining the threshold value 1 + λ1 − λ3 /2. Our approach
is based on tracking the evolution of the system on a particular set of “fluid” sample paths:
assume that, at time slot zero, queue 1 receives a very large file, consisting of b packets.
For a long period of time after that, queue 3 does not receive service under the Max-Weight
policy, so it builds up. If the arrival processes of all traffic flows are close to their “average
behavior,” then at the time slot when the service switches from schedule {1, 2} to schedule
{3}, the lengths of both queues 1 and 3 are proportional to b, whereas queue 2 is still small.
From that point on, the Max-Weight policy drains the weights of the two schedules
at roughly the same rate, until one of the weights becomes zero.
Let µf be the average departure rate from queue f ∈ {1, 2, 3} during the latter period.
For the weights of the two schedules to be drained at the same rate, the departure rates
must satisfy:
λ1 + λ2 − µ1 − µ2 = λ3 − µ3 .
Moreover, the fact that Max-Weight is a work-conserving policy implies that
µ1 + µ3 = 1.
Finally, since queues 1 and 2 are served simultaneously, and queue 2 may be empty
through parts of the draining period, we have that
µ1 ≥ µ2 .
The above equations, and some simple algebra, imply that
1 + λ1 + λ2 − λ3
≥ µ2 .
3
75
Suppose that the arrival rates satisfy
λ2 >
1 + λ1 − λ3
.
2
Then,
λ2 >
1 + λ1 + λ2 − λ3
≥ µ2 .
3
This implies that queue 2 builds up, at a roughly constant rate, during a time period
whose duration is of order Ω b . Thus, the integral of the length of queue 2 over a busy period
of the system becomes of order Ω b2 . Because b is drawn from a heavy-tailed distribution,
it follows that E Q2 is infinite.
Proof. We break the proof into four steps, which follow the various stages in our earlier proof
outline.
Step 1: buildup of queue 3 following a large arrival to queue 1.
Because Max-Weight is a regenerative policy, the times at which the system is empty
constitute renewal epochs. Let us consider the system at a typical renewal epoch which, for
simplicity of notation, we assume to happen at time slot zero.
Consider the set of sample paths for which, at time slot zero, queue 1 receives a file that
consists of b packets, and all other queues receive no traffic. We denote this set of sample
paths by Hb .
Since λf < 1, for all f ∈ {1, 2, 3}, due to stability and
1 − P Af (0) = 0 = P Af (0) > 0 ≤ E Af (0) = λf < 1,
we have that P Af (0) = 0 > 0.
Let B be the support of the distribution of A1 (0). Using the independence of the arrival
processes, we have, for every b ∈ B,
P Hb = P A1 (0) = b · P A2 (0) = 0 · P A3 (0) = 0 > 0.
76
For sample paths in Hb , we denote by Tb1 the first time slot, starting from 0, when the
length of queue 3 becomes greater than or equal to the sum of the lengths of queues 1 and
2:
Tb1 = min t > 0 | Q3 (t) ≥ Q1 (t) + Q2 (t) · 1Hb .
The first part of the proof is to show that Q1 Tb1 and Q3 Tb1 scale at least linearly with
b, provided all arrival processes are close to their “average behavior.”
By the definition of the stopping time Tb1 , we have
Q1 Tb1 ≤ Q1 Tb1 + Q2 Tb1 ≤ Q3 Tb1 .
A direct consequence of the Strong Law of Large Numbers is that for every > 0 there
exists δ > 0, such that the set of sample paths
t
o
n
X
Af (τ ) ≤ (λf + )t + δ, ∀t ∈ N, ∀f ∈ {1, 2, 3} ,
∆ = (λf − )t − δ ≤
τ =1
has positive probability (see Lemma 2.3). Consequently, the set of sample paths
t
n
X
Af (τ ) ≤ (λf + )t + δ,
∆b = (λf − )t − δ ≤
o
∀t ∈ 1, . . . , Tb1 − 1 , ∀f ∈ {1, 2, 3} ,
τ =1
has positive probability, uniformly over all b.
From now on we fix a small > 0. (How small it will have to be will become apparent
in the course of the proof.) We then fix a corresponding δ such that inf b P ∆b ≥ P(∆) > 0.
Let H̃b be the set of sample paths Hb ∩ ∆b , and observe that Hb ∩ ∆b ⊃ Hb ∩ ∆. Then, the
IID nature of the arriving traffic implies that Hb and ∆ are independent, so that
P H̃b = P Hb ∩ ∆b ≥ P Hb ∩ ∆ = P Hb · P(∆) > 0.
Recall that at most one packet can be removed from each queue at any given time slot.
77
So, for sample paths in H̃b , we have that
Q1 Tb1 ≥ b − Tb1 − 1 + λ1 − · Tb1 − 1 − δ.
Moreover, queue 3 receives no service before time slot Tb1 under the Max-Weight scheduling policy, which implies that
Tb1 −1
1
Q3 Tb =
X
A3 (t) ≤ λ3 + · Tb1 − 1 + δ.
t=1
Since Q1 Tb1 ≤ Q3 Tb1 , the last two inequalities and some algebra yield
Tb1 − 1 ≥
b − 2δ
.
1 + λ3 − λ1 + 2
(4.1)
(This argument requires the last denominator to be positive. This will be the case as long
as has been chosen small enough.)
Therefore,
Tb1 −1
1
Q3 Tb =
X
A3 (t) ≥ λ3 − · Tb1 − 1 − δ ≥ λ3 − ·
t=1
b − 2δ
− δ,
1 + λ3 − λ1 + 2
(4.2)
which implies that Q3 Tb1 = ΩH̃b (b), since we can chose to be less than λ3 .
Coming to queue 2, it can be verified that for sample paths in ∆ and, hence, in H̃b ,
and for any subinterval {τ0 , . . . , τ1 } of 1, . . . , Tb1 ,
τX
1 −1
A2 (t) ≤ λ2 + · (τ1 − τ0 ) + 2τ0 + 2δ.
t=τ0
We assume that has been chosen so that λ2 + < 1. Recall also that queue 2 gets
served whenever it is nonempty throughout the period 1, . . . , Tb1 − 1 .
We use Lindley’s formula, the above upper bound on the arrivals to queue 2, and the fact
that queue 2 gets served whenever it is nonempty throughout the period {1, . . . , Tb1 − 1}, to
78
conclude that
Q2 Tb1 ≤ A2 Tb1 − 1 + 2 Tb1 − 1 + 2δ ≤ λ2 + 4 Tb1 − 1 + 4δ.
(4.3)
This shows that, essentially, Q2 Tb1 does not scale with b along sample paths in H̃b .
Finally, we turn our attention to Q1 Tb1 . By definition of the stopping time Tb1 ,
Q3 Tb1 − 1 < Q1 Tb1 − 1 + Q2 Tb1 − 1 .
(4.4)
By arguing similar to the derivation of Eqs. (4.3), it can be verified that
Q2 Tb1 − 1 ≤ λ2 + 4 Tb1 − 2 + 4δ.
(4.5)
Moreover, by arguing similarly to the derivation of Eq. (4.2), it can be verified that
Q3 Tb1 − 1 = ΩH̃b (b). Then, Eqs. (4.4) and (4.5) readily imply that Q1 Tb1 = ΩH̃b (b), when
is chosen sufficiently small.
To summarize, at time slot Tb1 and for sample paths in H̃b , the lengths of queues 1 and
3 are proportional to b, while queue 2 remains small.
Step 2: draining until queue 1 or 3 empties.
Let Tb2 be the first time slot after Tb1 that either queue 1 or queue 3 becomes empty:
Tb2 = min t > Tb1 | Q1 (t) · Q3 (t) = 0 · 1H̃b .
We will show that if the arrival processes stay close to their “average behavior,” i.e., the
event H̃b occurs, then the length of queue 3 cannot be much larger than the sum of the
lengths of queues 1 and 2 at time slot Tb2 . The reason is that Q1 (t) + Q2 (t) and Q3 (t) are
kept roughly equal by the Max-Weight policy for t ∈ Tb1 , Tb2 .
79
For the same constants and δ as Step 1, the set of sample paths
∆0b
n
= (λf − )t − δ ≤
t
X
o
Af (τ ) ≤ (λf + )t + δ, ∀t ∈ Tb1 , . . . , Tb2 − 1 , ∀f ∈ {1, 2, 3}
τ =Tb1
contains ∆.
Let Ĥb = H̃b ∩∆0b . The events ∆b and ∆0b are determined by the arrivals over disjoint time
intervals. Hence, due to the IID nature of the arrival processes, ∆b and ∆0b are independent.
Since they both contain the positive probability event ∆, we have
P Ĥb ≥ P Hb · P(∆)2 > 0.
We will show that for sample paths in Ĥb ,
Q3 Tb2 ≤ Q1 Tb2 + Q2 Tb2 + 2 Tb2 − Tb1 + 2δ + 3.
(4.6)
First, notice that queues 1 and 3 cannot empty at the same time slot, since they cannot
be served simultaneously. Therefore, we have two possible cases: either Q3 Tb2 = 0, in
which case Eq. (4.6) is trivially satisfied, or Q1 Tb2 = 0, which we henceforth assume. In
the latter case, S1 Tb2 − 1 = S2 Tb2 − 1 = 1, and S3 Tb2 − 1 = 0. For sample paths in Ĥb ,
we have that
Q3 Tb2 = Q3 Tb2 − 1 + A3 Tb2 − 1 ≤ Q3 Tb2 − 1 + λ3 + 2 · Tb2 − Tb1 + 2δ.
(4.7)
Moreover, under the Max-Weight scheduling policy, and in order for the for the set of
queues {1, 2} to be served at time slot Tb2 − 1 ,
Q3 Tb2 − 1 ≤ Q1 Tb2 − 1 + Q2 Tb2 − 1 .
(4.8)
Q1 Tb2 − 1 + Q2 Tb2 − 1 − 2 ≤ Q1 Tb2 + Q2 Tb2 .
(4.9)
Finally,
80
Eq. (4.6) follows immediately by combining Eqs. (4.7)-(4.9), and also using the fact that
λ3 < 1.
Step 3: growth of queue 2.
At time Tb2 , Q3 Tb2 cannot much larger than Q1 Tb2 + Q2 Tb2 . Therefore, queue 3 must
have been receiving a certain fraction of the total service between times Tb1 and Tb2 . We
will show that this results in queue 2 growing. In particular, if λ2 > 1 + λ1 − λ3 /2 then
Q2 Tb2 = ΩĤb (b).
By definition,
Q3 Tb1 ≥ Q1 Tb1 + Q2 Tb1 .
By subtracting the two sides of this inequality from Eq. (4.6), we get
Q3 Tb2 − Q3 Tb1 ≤ Q1 Tb2 − Q1 Tb1 + Q2 Tb2 − Q2 Tb1 + 2 Tb2 − Tb1 + 2δ + 3. (4.10)
For sample paths in Ĥb , define the random variables
2
Tb −1
X
1
·
S
(t)
· 1Ĥb ,
µf =
f
Tb2 − Tb1
1
f ∈ {1, 2, 3},
t=Tb
which are the average service rates to each queue during the interval Tb1 , . . . , Tb2 − 1 , and
notice that µ1 = µ2 and µ1 + µ3 = 1.
Since both queues 1 and 3 are nonempty during the inerval Tb1 , . . . , Tb2 − 1 , we have
Q1 Tb2 − Q1 Tb1 ≤ λ1 + − µ1 · Tb2 − Tb1 + δ,
Q3 Tb2 − Q3 Tb1 ≥ λ3 − − µ3 · Tb2 − Tb1 − δ.
(4.11)
(4.12)
Eqs. (4.10), (4.11), and (4.12) imply that
λ3 −−µ3 Tb2 −Tb1 −δ ≤ λ1 +−µ1 Tb2 −Tb1 +δ+Q2 Tb2 −Q2 Tb1 +2 Tb2 −Tb1 +2δ+3.
81
We replace µ3 by 1 − µ1 and collect terms, to obtain
1
−µ1 Tb2 − Tb
1 + λ − λ + 4 2
1
−
Q
T
Q
T
4δ + 3
2
2
1
3
b
b
≥−
· Tb2 − Tb1 +
−
2
2
2
1 + λ − λ + 4 2
Q2 Tb
4δ + 3
1
3
≥−
· Tb2 − Tb1 −
−
.
2
2
2
For sample paths in Ĥb , we use the definition of ∆0b to upper bound the number of arrivals
to queue 2. We also use the fact that queue 2 has µ2 Tb2 − Tb1 service opportunities, with
µ2 = µ1 , and obtain
2
Q2 Tb ≥ λ2 −−µ1 · Tb2 −Tb
1
2
Q
T
6δ + 3
1 + λ1 − λ3
2
b
−3 · Tb2 −Tb1 −
−
.
−δ ≥ λ2 −
2
2
2
Therefore,
2 1 + λ1 − λ3
Q2 Tb2 ≥ · λ2 −
− 3 · Tb2 − Tb1 − 2δ − 1.
3
2
If λ2 > 1 + λ1 − λ3 /2 then constant can be chosen sufficiently small, so that
λ2 −
1 + λ1 − λ3
− 3 > 0.
2
A final observation is that the duration of the interval Tb1 , . . . , Tb2 − 1 is bounded from
below by min Q1 Tb1 , Q3 Tb1 , because both queues are served at unit rate. Therefore,
Eq. (4.1) implies that Tb2 − Tb1 = ΩH̃b (b) and, in turn,
Q2 Tb2 = ΩĤb (b).
(4.13)
Step 4: the growth scenario for queue 2 implies large average queue size.
In Step 3 we showed that queue 2 builds up to order Ω(b) along sample paths in Ĥb . We
use this fact and renewal theory to show that the steady-state expected length of queue 2 is
infinite.
82
The sequence of times where the network is empty constitute renewal epochs. We define
an instantaneous reward on this renewal process:
RM (t) = min Q2 (t), M ,
t ∈ Z+ ,
where M is a positive integer.
Eq. (4.13) implies the existence of positive constants c and b0 , such that
Q2 Tb2 ≥ cb,
∀b ≥ b0 ,
for all sample paths in Ĥb .
Since at most one packet from queue 2 can be served at each time slot, the length of
queue 2 is at least cb/2 packets over a time period of duration cb/2 time slots. Hence, the
M
aggregate reward Ragg
, i.e., the reward accumulated over a renewal period, satisfies the lower
bound
M
Ragg
· 1{b≥b0 } · 1Ĥb ≥ min
n cb 2
2
o
· 1{b≥b0 } , M 2 · 1Ĥb .
Then, the expected aggregate reward is bounded from below by
∞
n cb 2
h
i
o
X
M
2
2
min
·1{b≥b0 } , M ·P A1 (0) = b .
E Ragg ≥ P(∆) ·P A2 (0) = 0 ·P A3 (0) = 0 ·
2
b=1
So, there exists a positive constant c0 such that
h
n cA (0) 2
oi
h
i
1
2
M
· 1{A1 (0)≥b0 } , M
≤ E Ragg .
c E min
2
0
Lemma 2.2 applied to Y = (1/4)c2 A21 (0) · 1{A1 (0)≥b0 } implies that E Q2 is infinite, since
traffic flow 1 is heavy-tailed. Finally, Lemma 2.1 implies that traffic flow 2 is delay unstable.
Now we establish that when λ2 < 1 + λ1 − λ3 /2, traffic flow 2 is delay stable. Thus, we
83
achieve an exact characterization of the delay stability region of traffic flow 2. In order
to do that, we further assume that traffic flows 2 and 3 are exponential-type.
Proposition 4.2: (Rate-Dependent Delay Stability) Consider the switched queueing network of Figure 4-2, with admissible traffic and under the Max-Weight scheduling
policy. If the random variables A2 (0) and A3 (0) are exponential-type and the arrival rates
satisfy λ2 < 1 + λ1 − λ3 /2, then traffic flow 2 is delay stable and the steady-state length
of queue 2 is exponential-type.
Before we proceed to the formal proof of Proposition 4.2, we provide the underlying
intuition by arguing (loosely) in terms of a fluid approximation. Our analysis rests on drift
analysis of the piecewise linear Lyapunov function
+
V (t) = 3Q2 (t) + Q3 (t) − Q1 (t) − Q2 (t) ,
t ∈ Z+ .
Our goal is to establish a negative drift for V (t), which, combined with the special
structure of the problem (the arrivals to queues 2 and 3 being exponential-type), will imply
that
h
+ i
E 3Q2 + Q3 − Q1 − Q2
< ∞,
where the expectation is taken with respect to the steady-state distributions. Combined
with Lemma 2.1, this will imply the delay stability of traffic flow 2.
The suitability of this Lyapunov function can be seen as follows. If Q1 (t) + Q2 (t) >
Q3 (t), then the Lyapunov function reduces to 3Q2 (t), which can be easily shown to have
negative drift (as long as queue 2 is nonempty), because queue 2 is served under Max
Weight in that region. If, on the other hand, Q1 (t) + Q2 (t) < Q3 (t), then the Lyapunov
function reduces to 2Q2 (t) + Q3 (t) − Q1 (t) . In that region, queue 3 is served under Max
Weight, so the drift of V (t) is equal to 2λ2 + λ3 − λ1 − 1 , which is strictly negative by our
assumption on λ2 . Figure 4-3 provides a geometric interpretation.
The analysis becomes subtler at the boundary between the two regions of the state
space, namely when the weights of the two candidate schedules, Q1 (t) + Q2 (t) and Q3 (t),
84
Figure 4-3: A two-dimensional representation
of the dynamics of the queueing system of
Figure 4-2 when λ2 < 1 + λ1 − λ3 /2. The dashed black arrows represent the gradient of the
queue-length vector under the Max-Weight policy, and the dashed orange arrows represent
the gradient of the Lyapunov function, in different regions of the state space. Lyapunov
function V (·) has been chosen so that an obtuse angle is formed between the two in all
regions of the state space, establishing a uniform negative drift over the entire state space.
The solid red arrows represent typical trajectories of the associated fluid approximation.
are equal. Once at the boundary, the Max-Weight policy keeps the state of the fluid model
there. Similar to our discussion of Proposition 4.1, the resulting departure rate from queue
2 satisfies
λ1 + λ2 − µ1 − µ2 = λ3 − µ3 .
Using also the properties µ2 = µ1 = 1 − µ3 , and our assumption on λ2 , it follows that the
drift at the boundary is negative, driving the state to zero.
Proof. We will show that for sufficiently large (but fixed) T ∈ N, there exist positive constants α and , such that
E V (t + T ) − V (t) + ; V (t) > α Ft ≤ 0,
∀t ∈ Z+ ,
and the desired result will follow from Lemma 2.6.
We remind that Ft is the σ-algebra generated by Q(0), A(0), . . . , Q(t − 1), A(t − 1), Q(t),
85
and should be distinguished from the set of traffic flow F = {1, . . . , F }.
Suppose that V (t) > α. Then, we must have either Q2 (t) > α/6 or Q3 (t) > Q1 (t) +
Q2 (t) + α/2 . We will derive the desired drift inequality by considering separately these two
cases. We assume that T has been fixed to a suitably large value, and we define α = 6T .
Case 1: Q2 (t) > α/6 = T .
Since at most one packet is removed at each time slot from queue 2, it is immediate that
Q2 (τ ) > 0,
∀τ ∈ {t, . . . , t + T − 1}.
(4.14)
Moreover, under the Max-Weight scheduling policy,
S3 (τ ) · 1{Q3 (τ )>0} = S3 (τ ),
∀τ ∈ {t, . . . , t + T − 1}.
(4.15)
Eq. (4.15) implies that a service opportunity is never wasted in queue 3 throughout
{t, . . . , t + T − 1}, assuming Q2 (t) > T . This is intuitively clear because, under the MaxWeight policy, queue 3 will not be served unless it becomes at least as large as queue 2,
hence, positive.
Let
J(t) = Q1 (t) + Q2 (t) − Q3 (t).
Using this notation and the queue-length dynamics from Section 4.1, it can be verified
that
t+T
−1
X
A1 (τ ) + A2 (τ ) − S1 (τ ) · 1{Q1 (τ )>0} −S2 (τ ) · 1{Q2 (τ )>0}
τ =t
=
t+T
−1
X
A3 (τ ) − S3 (τ ) · 1{Q3 (τ )>0} + J(t + T ) − J(t).
τ =t
(4.16)
86
Moreover, the scheduling constraints imply that
∀τ ∈ {t, . . . , t + T − 1}.
S1 (τ ) = S2 (τ ),
(4.17)
Furthermore, under the Max-Weight policy (or, in general, under any non-idling policy),
t+T
−1
X
S2 (τ ) + S3 (τ ) = T.
(4.18)
τ =t
Eqs. (4.14) and (4.15) imply that
t+T
−1
X
−1
X
t+T
A1 (τ ) + A2 (τ ) − S1 (τ ) − S2 (τ ) ≤
A3 (τ ) − S3 (τ ) + J(t + T ) − J(t). (4.19)
τ =t
τ =t
Combining Eqs. (4.17) and (4.19), we have
t+T
−1
X
−1
X
t+T
A1 (τ ) + A2 (τ ) − 2 · S2 (τ ) ≤
A3 (τ ) − S3 (τ ) + J(t + T ) − J(t).
τ =t
(4.20)
τ =t
Then, by taking into account Eq. (4.18), we get
−3 ·
t+T
−1
X
τ =t
S2 (τ ) ≤ −
t+T
−1
X
1 + A1 (τ ) + A2 (τ ) − A3 (τ ) + J(t + T ) − J(t).
(4.21)
τ =t
Let us now examine the implications of this inequality on the evolution of queue 2 during
the interval {t, . . . , t + T − 1}. We have
t+T
−1
X
3 Q2 (t + T ) − Q2 (t) = 3 ·
A2 (τ ) − S2 (τ ) · 1{Q2 (τ )>0}
=3·
≤−
τ =t
t+T
−1
X
τ =t
t+T
−1
X
A2 (τ ) − S2 (τ )
1 + A1 (τ ) − A3 (τ ) − 2A2 (τ ) + J(t + T ) − J(t),
τ =t
87
where the second equality follows from Eq. (4.14), and the inequality follows from Eq. (4.21).
This implies that
+ +
V (t + T ) − V (t) = 3 Q2 (t + T ) − Q2 (t) + − J(t + T ) − − J(t)
≤−
t+T
−1
X
1 + A1 (τ ) − A3 (τ ) − 2A2 (τ )
τ =t
+ +
+ J(t + T ) − J(t) + − J(t + T ) − − J(t)
=−
t+T
−1
X
+ +
1 + A1 (τ ) − A3 (τ ) − 2A2 (τ ) + J(t + T ) − J(t) .
τ =t
Therefore,
E V (t + T ) − V (t); Q2 (t) > T Ft ≤ − δT · P Q2 (t) > T Ft
i
h
+ +
+ E J(t + T ) − J(t) ; Q2 (t) > T Ft .
(4.22)
where δ = 1 + λ1 − λ3 − 2λ2 is positive by assumption.
We recall our standing assumption regarding the existence of constant γ > 0, such that
h
i
h
+ i
+ 1+γ
E A1 (0) is finite. Based on this, we will prove that E J(t + T ) − J(t) Ft scales
sublinearly in T . This, in turn, will imply that the right-hand side of Eq. (4.22) is negative,
provided T is sufficiently large.
+
By disregarding the contribution of queue 3, we can bound from above J(τ ) , τ ∈
{t, . . . , t + T }, by the sum of the lengths of queues 1 and 2 during that interval. Moreover,
+
whenever J(τ ) is nonzero, both queues 1 and 2 are served at unit rate.
Through Lindley’s recursion and simple calculations, it can be verified that
n
+ +
J(t+T ) − J(t) ≤ max
1≤s≤T
t+T
−1
X
n
o
A1 (τ )−1 + max
1≤s≤T
τ =t+T −s
88
t+T
−1
X
τ =t+T −s
o
A2 (τ )−1 . (4.23)
Also, the following inequality holds for all f ∈ {1, 2} and all s ∈ {1, . . . , T }:
t+T
−1
X
Af (τ ) − 1 = λf − 1 s +
τ =t+T −s
t+T
−1
X
Af (τ ) − λf ≤ t+T
−1
X
Af (τ ) − λf . (4.24)
τ =t+T −s
τ =t+T −s
Eqs. (4.23) and (4.24) imply that for any fixed γ 0 ∈ (0, γ),
P
+ +
J(t + T ) − J(t) ≥ c; Q2 (t) > T Ft
≤P max
1≤s≤T
t+T
−1
X
n
1≤s≤T
τ =t+T −s
n
≤P max 1≤s≤T
t+T
−1
X
τ =t+T −s
n
+ P max 1≤s≤T
n
=P max 1≤s≤T
τ =t+T −s
τ =t+T −s
n
+ P max 1≤s≤T
o
A2 (τ ) − 1 ≥ c; Q2 (t) > T Ft
τ =t+T −s
o c
A1 (τ ) − λ1 ≥ ; Q2 (t) > T Ft
2
t+T
−1
X
t+T
−1
X
t+T
−1
X
n
o
A1 (τ ) − 1 + max
o c
A2 (τ ) − λ2 ≥ ; Q2 (t) > T Ft
2
1+γ 0 o c 1+γ 0
A1 (τ ) − λ1 ; Q2 (t) > T Ft
≥
2
t+T
−1
X
τ =t+T −s
1+γ 0 o c 1+γ 0
A2 (τ ) − λ2 ; Q2 (t) > T Ft .
≥
2
nP
(4.25)
o
Af (τ ) − λf ; s ∈ N , f ∈ {1, 2, 3}, is a martinNotice that the sequence
n P
o
1+γ 0
−1
A
(τ
)
−
λ
gale. Consequently, the sequence τt+T
;
s
∈
N
is a nonnegative
f
f
=t+T −s
t+T −1
τ =t+T −s
submartingale. Doob’s submartingale inequality (see, e.g., Section 14.6 of [64]) and Eq.
(4.25) imply that
+ +
P J(t + T ) − J(t) ≥ c; Q2 (t) > T Ft
−1
2 1+γ 0 h t+T
1+γ 0 i
X
≤
·E A1 (τ ) − λ1 · 1Q2 (t)>T
c
τ =t
+
2 1+γ 0
c
−1
h t+T
1+γ 0 i
X
·E A2 (τ ) − λ2 · 1Q2 (t)>T . (4.26)
τ =t
89
Moreover, the Marcinkiewicz-Zygmund Strong Law of Large Numbers implies that
−1
t+T
X
Af (τ ) − λf τ =t
T 1/(1+γ)
L1
−→ 0,
f ∈ {1, 2, 3};
(4.27)
see Chapter 6.10 of [30].
Eqs. (4.26) and (4.27) imply that, if T is sufficiently large, then there exists k > 0
(independent of T and c), such that
P
+ +
J(t + T ) − J(t) ≥ c; Q2 (t) > T Ft ≤
k
c
1+γ 0
1+γ 0
1+γ
·T
· 1Q2 (t)>T ,
for all c ≥ 0.
This gives
∞
i
h
X
+ +
1+γ 0
1+γ
E J(t + T ) − J(t) ; Q2 (t) > T Ft ≤ k · T
·
c=1
1
c1+γ 0
· 1Q2 (t)>T .
Since γ 0 > 0 the latter sum converges, so there exists k 0 > 0 (independent of T ), such
that
i
h
+ +
1+γ 0
E J(t + T ) − J(t) ; Q2 (t) > T Ft ≤ k 0 · T 1+γ · 1Q2 (t)>T .
(4.28)
Finally, Eqs. (4.22) and (4.28), and the fact that γ 0 < γ, imply that for sufficiently large
(but fixed) T there exists > 0 such that
E V (t + T ) − V (t) + ; Q2 (t) > T Ft ≤ 0.
(4.29)
Case 2: Q3 (t) > Q1 (t) + Q2 (t) + 3T .
Since at most one packet is removed at each time slot from queue 3, it is immediate that
Q3 (τ ) > 0,
∀τ ∈ {t, . . . , t + T − 1}.
90
Based on this, it can be easily verified that Eq. (21) still holds. This implies that
t+T
−1
X
3 Q2 (t + T ) − Q2 (t) = 3 ·
A2 (τ ) − S2 (τ ) · 1{Q2 (τ )>0}
= 3·
≤−
τ =t
t+T
−1
X
τ =t
t+T
−1
X
t+T
−1
X
A2 (τ ) − S2 (τ ) + 3 ·
S2 (τ ) · 1{Q2 (τ )=0}
τ =t
1 + A1 (τ ) − A3 (τ ) − 2A2 (τ ) ,
τ =t
+ J(t + T ) − J(t) + 3 ·
t+T
−1
X
S2 (τ ) · 1{Q2 (τ )=0} .
τ =t
Letting D be the event Q3 (t) > Q1 (t) + Q2 (t) + 3T , this gives
i
h
+ +
E V (t + T ) − V (t); D Ft ≤ − δT · 1D + E J(t + T ) − J(t) ; D Ft
−1
i
h t+T
X
S2 (τ ) · 1{Q2 (τ )=0} ; D Ft .
+3·E
(4.30)
τ =t
Working similar to the previous case, it can be verified that for any fixed γ 0 ∈ (0, γ),
there exists k 0 > 0 (independent of T ) such that
i
h
+ +
1+γ 0
E J(t + T ) − J(t) ; D Ft ≤ k 0 · T 1+γ · 1D .
(4.31)
In view of Eqs. (4.30) and (4.31) and in order to establish the negative drift property of
the Lyapunov function, it is sufficient to consider a large time horizon T , and to show that
i
hP
t+T −1
E
S
(τ
)
·
1
2
{Q2 (τ )=0} Ft scales sublinearly in T .
τ =t
In order to have a wasted service opportunity at queue 2, the schedule {1, 2} must first
P −1
claim the server. This can only happen if t+T
A1 (τ ) + A2 (τ ) , the aggregate arrivals to
τ =t
queues 1 and 2 during the interval {t, . . . , t + T − 1}, exceed the initial difference between
the weights of the two schedules (which is at least 3T ), minus the departures from queue 3
91
during the same period (which are at most T ). It follows that
−1
n t+T
X
−1
o n t+T
o
X
S2 (τ ) · 1{Q2 (τ )=0} > 0 ⊂
A1 (τ ) + A2 (τ ) > 3T − T
τ =t
τ =t
−1
n t+T
X
o
=
A1 (τ ) + A2 (τ ) − λ1 − λ2 > 2 − λ1 − λ2 T
τ =t
−1
n t+T
o
X
⊂ A1 (τ ) + A2 (τ ) − λ1 − λ2 > 2 − λ1 − λ2 T .
τ =t
Note that λ1 + λ2 < 2, since the arrival rate vector was assumed to be in the stability
region of the system. Then, Markov’s inequality implies the existence of constant ξ > 0
(independent of T ) such that
P
−1
t+T
X
S2 (τ )·1{Q2 (τ )=0} > 0; D Ft
τ =t
−1
t+T
X
≤P A1 (τ ) + A2 (τ ) − λ1 − λ2 > 2 − λ1 − λ2 T ; D Ft
τ =t
h P
i
t+T −1
A1 (τ ) + A2 (τ ) − λ1 − λ2 E τ =t
≤
· 1D
2 − λ1 − λ2 T
1
≤ξT 1+γ −1 · 1D ,
provided T is sufficiently large.
Since the number of wasted service opportunities at queue 2 during an interval of length
T is bounded by T , we conclude that
−1
i
h t+T
X
1
1
E
S2 (τ ) · 1{Q2 (τ )=0} ; D Ft ≤ T · ξT 1+γ −1 · 1D = ξT 1+γ · 1D .
(4.32)
τ =t
Eqs. (4.30)-(4.32) imply that for sufficiently large (but fixed) T there exists > 0, such
that
E V (t + T ) − V (t) + ; D Ft ≤ 0.
92
(4.33)
This completes the derivation of the drift inequality for the second case.
To summarize, we have shown in Eqs. (4.29) and (4.33) that by fixing T to a sufficiently
large value and by letting α = 6T , there exists > 0 such that
E V (t + T ) − V (t) + ; V (t) > α Ft ≤ 0,
t ∈ Z+ .
Then, Lemma 2.6 implies that the steady-state length of queue 2 is exponential-type and,
in particular, has finite expectation. Finally, Lemma 2.1 implies that traffic flow 2 is delay
stable.
4.4
Max-Weight-α Scheduling
The results of the previous section suggest that Max-Weight scheduling performs poorly in
the presence of heavy-tailed traffic. The reason is that, by treating heavy-tailed and lighttailed flows equally, there are very long stretches of time during which heavy-tailed traffic
dominates the service. This leads some light-tailed flows to experience very large delays.
Intuitively, by discriminating against heavy-tailed flows one should be able to improve the
overall performance of the network, namely to mitigate the propagation of delay instability.
Motivated by this observation, in this section we focus on the Max-Weight-α scheduling
policy: given constants αf > 0, f ∈ F, the scheduling vector S(t) satisfies
nX
S(t) ∈ arg max
(Sf )∈S
α
Qf f (t)
o
· Sf .
f ∈F
If the set on the right-hand side includes multiple schedules, then one of them is chosen
uniformly at random. By choosing smaller values for the α-parameters of heavy-tailed flows
and larger values for light-tailed flows, we give a form of partial priority to light-tailed traffic.
93
4.4.1
The Main Result
Let us start with a preview of the main result of this section: if the α-parameters of the
i
h
α +1
Max-Weight-α policy are chosen so that E Af f (0) is finite, for all f ∈ F, then the network
is stable and the steady-state queue lengths satisfy
i
h
α
E Qf f < ∞,
∀f ∈ F.
This can be viewed as a significant generalization of Proposition 3.7. Also, it should
be noted that an earlier work by Eryilmaz et al. has given a similar result for the case of
parallel queues with a single server; see Theorem 1 of [21]. Here, we extend their result to a
single-hop switched queueing network, and we provide an explicit upper bound on the sum
of the α-moments of the steady-state queue lengths.
In order to state our result we need the following definition.
Definition 4.4: (Covering Number of Schedules) The covering number k ∗ of the
set of schedules is defined as the smallest number k for which there exist s1 , . . . , sk ∈ S with
Sk i
i=1 s = F.
Theorem 4.3: (Max-Weight-α Scheduling) Consider the switched queueing network
of Section 4.1 under the Max-Weight-α scheduling policy. Let the intensity of the arriving
h
i
α +1
traffic be ρ < 1. If E Af f (0) is finite, for all f ∈ F, then the network is stable and the
steady-state queue lengths satisfy
h
i
X h α i X α +1
E Qf f ≤
H ρ, k ∗ , αf , E Af f (0) ,
f ∈F
where
f ∈F

h
i
h
i  2k∗ · E Aαf f +1 (0) + 1 , αf ≤ 1,
α +1
1−ρ ∗ αf
H ρ, k ∗ , αf , E Af f (0) =
2k∗
 2k
· K αf + 1−ρ
· K, αf > 1,
1−ρ
h
i
α +1
and K = 2αf −1 · αf · E Af f (0) + 1 .
94
Proof. The admissibility of the arriving traffic implies that we can find a set of schedules
k
σ ; k = 1, . . . , k ∗ that satisfies
k∗
[
σ k = F.
k=1
By the definition of the intensity parameter ρ ∈ (0, 1), there exist nonnegative numbers
ζi , i = 1, . . . , I, adding up to 1, and schedules s̃i , i = 1, . . . , I, such that
λ≤ρ
I
X
ζi · s̃i .
i=1
Notice that
∗
k
I
X
X
1
k
i
(1 − ρ)
·σ +ρ
ζi · s̃ ∈ Λ,
k∗
i=1
k=1
where Λ denotes the closure of the set Λ. This is because we have a convex combination of
(I + k ∗ ) feasible schedules, and the stability region is known to be a convex set; see Section
3.2 of [27].
Moreover,
∗
∗
k
k
X
1
1−ρX k 1−ρ F
k
(1 − ρ)
·σ =
σ ≥
·1 ,
∗
∗
∗
k
k
k
k=1
k=1
where 1F denotes the F -dimensional vector of ones.
A well-known monotonicity property of the stability region is the following: if 0 ≤ λ0 ≤ λ00
componentwise, and λ00 ∈ Λ, then λ0 ∈ Λ. Using this property, we have that
1 − ρ
k∗
· 1F + λ ∈ Λ.
In turn, this implies the existence of nonnegative numbers θj , j = 1, · · · , J, adding up
to 1, and of feasible schedules sj = sjf , j = 1, · · · , J, such that
λf ≤
J
X
j=1
θj · sjf −
1−ρ
,
k∗
95
∀f ∈ F.
(4.34)
Under the Max-Weight-α scheduling policy, the sequence
Q(t); t ∈ Z+
is a time-
homogeneous, irreducible, and aperiodic Markov chain on the countable state space ZF+ . In
order to establish positive recurrence, we consider the candidate Lyapunov function
X
V Q(t) =
f ∈F
1
α +1
Q f (t).
αf + 1 f
We have
X h
E V Q(t + 1) Ft =
E
f ∈F
αf +1 i
1 Qf (t) + ∆f (t)
Ft ,
αf + 1
where
∆f (t) = Af (t) − Sf (t) · 1{Qf (t)>0} .
Throughout the proof we use the shorthand notation
Vf Q(t) =
1
α +1
Qf f (t).
αf + 1
We consider the conditional expectation of the terms Vf Q(t+1) , distinguishing between
two cases.
(i) αf ≤ 1: Consider the zeroth order Taylor expansion around Qf (t),
α +1
1
1
α +1
Qf (t) + ∆f (t) f =
Qf f (t) + ∆f (t) · ξ αf ,
αf + 1
αf + 1
for some ξ ∈ Qf (t) − Sf (t) · 1{Qf (t)>0} , Qf (t) + Af (t) . Thus,
Vf Q(t + 1) = Vf Q(t) + ∆f (t) · ξ αf ,
and
E Vf Q(t + 1) Ft = Vf Q(t) + E ∆f (t) · ξ αf Ft .
96
Consider the event Γf (t) = ∆f (t) < 0 and its complement. We have
h
α
E Vf Q(t + 1) Ft ≤Vf Q(t) + E ∆f (t) · Qf (t) + Af (t) f ; Γcf (t) h
α
+ E ∆f (t) · Qf (t) − Sf (t) · 1{Qf (t)>0} f ; Γf (t)
i
Ft
i
Ft .
(4.35)
Since Qf (t), Qf (t)−Sf (t)·1{Qf (t)>0} , and Af (t) are nonnegative numbers and αf ∈ (0, 1],
it can be verified that
Qf (t) + Af (t)
αf
α
α
≤ Qf f (t) + Af f (t).
(4.36)
Moreover, because these numbers are also integers, it can be verified that
Qf (t) − Sf (t) · 1{Qf (t)>0}
αf
α
≥ Qf f (t) − Sf (t) · 1{Qf (t)>0} .
(4.37)
Eqs. (4.35)-(4.37) imply that
α
E Vf Q(t + 1) Ft ≤Vf Q(t) + E ∆f (t) Ft · Qf f (t)
i
h
αf
c
+ E ∆f (t) · Af (t); Γf (t) Ft
+ E − ∆f (t) · Sf (t) · 1{Qf (t)>0} ; Γf (t) Ft .
If ∆f (t) < 0, which is the event Γf (t), then −∆f (t) ≤ 1. Also, if ∆f (t) ≥ 0, which is the
α
α +1
event Γcf (t), then ∆f (t) ≤ Af (t), so that ∆f (t) · Af f (t) ≤ Af f
(t). Consequently,
i
h
α
α +1
E Vf Q(t + 1) Ft ≤ Vf Q(t) + E ∆f (t) Ft · Qf f (t) + E Af f (t); Γcf (t) Ft + 1.
Finally, the fact that the random variables Af (t); t ∈ Z+ are IID gives:
h
i
αf
αf +1
E Vf Q(t + 1) Ft ≤ Vf Q(t) + E ∆f (t) Ft · Qf (t) + E Af (0) + 1.
97
Thus,
E Vf Q(t + 1) − Vf Q(t) Ft
h
i
α
1−ρ
αf
αf +1
≤E ∆f (t) Ft · Qf f (t) +
·
Q
(t)
+
E
A
(0)
+ 1.
f
f
2k ∗
(4.38)
(ii) αf > 1: Consider the first order Taylor expansion around Qf (t),
α +1
∆2f (t)
1
1
α
Qf (t) + ∆f (t) f =
Qf (t)αf +1 + ∆f (t) · Qf f (t) +
· αf · ξ αf −1 ,
αf + 1
αf + 1
2
for some ξ ∈ Qf (t) − Sf (t) · 1{Qf (t)>0} , Qf (t) + Af (t) . Then,
i
α
1 h
E Vf Q(t+1) Ft = Vf Q(t) +E ∆f (t) Ft ·Qf f (t)+ E ∆2f (t)·αf ·ξ αf −1 Ft . (4.39)
2
Since ∆2f (t) · αf ≥ 0 and αf − 1 ≥ 0, the last term can be bounded from above as follows:
i 1 h
α −1 i
1 h 2
E ∆f (t) · αf · ξ αf −1 Ft ≤ E ∆2f (t) · αf · Qf (t) + Af (t) f Ft .
2
2
(4.40)
Moreover, it is easy to verify that, for αf ≥ 1,
αf −1
αf −1
αf −1
αf −1
≤2
· Qf (t) + Af (t) ,
Qf (t) + Af (t)
(4.41)
∆2f (t) ≤ A2f (t) + 1.
(4.42)
and also that
Eqs. (4.40)-(4.42) imply that
i
i
h
1 h 2
α −1
αf −1 E ∆f (t) · αf · ξ
Ft ≤2αf −2 · αf · E A2f (t) + 1 · Qf f (t)
2
i
h
i
h
αf +1
αf −1
αf −2
+2
· αf · E Af (t) + E Af (t)
α −1
≤K · Qf f
98
(t) + K,
(4.43)
h
i
α +1
where K = 2αf −1 · αf · E Af f (0) + 1 .
Then, Eqs. (4.39) and (4.43) imply that
α
α −1
E Vf Q(t + 1) Ft ≤Vf Q(t) + E ∆f (t) Ft · Qf f (t) + K · Qf f (t) + K
α
1−ρ
α
=Vf Q(t) + E ∆f (t) Ft · Qf f (t) +
· Qf f (t)
∗
2k
1−ρ
αf
αf −1
·
Q
(t)
+
K
.
(4.44)
+ K · Qf (t) −
f
2k ∗
Our goal is to bound from above the last term on the right-hand side of Eq. (4.44). By
relaxing the constraint that Qf (t) has to be an integer, we get:
α −1
K · Qf f (t) −
n
o
1 − ρ αf
1 − ρ αf
αf −1
· Qf (t) + K ≤ max K · x
−
·x +K ,
x∈R+
2k ∗
2k ∗
∀t ∈ Z+ . (4.45)
It can be verified that the optimization problem on the right-hand side has the unique
solution x∗ =
2k∗ K
1−ρ
·
K αf ·
αf −1
.
αf
The corresponding optimal value is
2k ∗ αf −1 (α − 1)αf −1
2k ∗ αf −1
f
αf
+
K
≤
K
·
·
+ K.
α
1−ρ
1−ρ
αf f
(4.46)
Eqs. (4.45) and (4.46) imply that
K·
α −1
Qf f (t)
2k ∗ αf −1
1−ρ
αf
αf
−
· Qf (t) + K ≤ K ·
+ K,
2k ∗
1−ρ
∀t ∈ Z+ .
(4.47)
Finally, Eqs. (4.44) and (4.47) imply that
E Vf Q(t + 1) − Vf Q(t) Ft
2k ∗ αf −1
α
1−ρ
αf
αf
≤E ∆f (t) Ft · Qf f (t) +
·
Q
(t)
+
K
·
+ K.
f
2k ∗
1−ρ
99
(4.48)
Summarizing our findings from cases (i) and (ii), Eqs. (4.38) and (4.48) imply that
E Vf Q(t + 1) −Vf Q(t) Ft
h
i
α
1−ρ
αf
αf +1
∗
≤E ∆f (t) Ft · Qf f (t) +
·
Q
(t)
+
H
ρ,
k
,
α
,
E
A
(0)
,
f
f
f
2k ∗
for all f ∈ F, where

h
i
h
i  2k∗ · E Aαf f +1 (0) + 1 , αf ≤ 1,
α +1
1−ρ ∗ αf
H ρ, k ∗ , αf , E Af f (0) =
2k∗
 2k
· K αf + 1−ρ
· K, αf > 1,
1−ρ
αf −1
and K = 2
h
i
αf +1
· αf · E Af (0) + 1 .
Consequently,
α
X
λf − Sf (t) · 1{Qf (t)>0} · Qf f (t)
E V Q(t + 1) − V Q(t) Ft ≤
f ∈F
+
h
i
X 1 − ρ X αf
αf +1
∗
·
Q
(t)
+
H
ρ,
k
,
α
,
E
A
(0)
.
f
f
2k ∗ f ∈F f
f ∈F
By taking into account Eq. (4.34), we have that
h
i
X 1 − ρ X αf
αf +1
∗
E V Q(t + 1) − V Q(t) Ft ≤ −
·
Q
(t)+
H
ρ,
k
,
α
,
E
A
(0)
f
f
2k ∗ f ∈F f
f ∈F
+
J
XX
f ∈F
θj ·
sjf
α
− Sf (t) · Qf f (t).
j=1
Moreover, by definition of the Max-Weight-α policy, the last term is nonpositive. So,
h
i
X 1 − ρ X αf
αf +1
∗
H ρ, k , αf , E Af (0) .
E V Q(t + 1) − V Q(t) Ft ≤ −
·
Q (t) +
2k ∗ f ∈F f
f ∈F
Then, Lemma 2.5 implies that the sequence Q(t); t ∈ Z+ converges in distribution to
100
some random vector Q, which does not depend on Q(0), and satisfies
h
i
X h α i
2k ∗ X ∗
α +1
·
H ρ, k , αf , E Af f (0) .
E Qf f ≤
1 − ρ f ∈F
f ∈F
Based on this, it can be verified that the sequence D(k); k ∈ N is a, possibly delayed,
aperiodic and positive recurrent regenerative process. Hence, it also converges in distribution,
and its limiting distribution does not depend on Q(0); see [56].
It is known that bounds derived from conventional Lyapunov arguments are, in general,
loose. The bound provided in Theorem 4.3 is, probably, no exception to this rule, e.g.,
see Corollary 4.3 and the subsequent discussion. In this light, the value of Theorem 4.3
lies on the following: (i) it gives a feel for which structural parameters of the network and
which characteristics of the arriving traffic may affect the actual performance of Max-Weight
policies; (ii) it provides the correct scaling of higher order queue-length moments with traffic
intensity (see Corollary 4.2).
4.4.2
Traffic Variability and Delay Stability
A first corollary of Theorem 4.3 relates to the delay stability of light-tailed flows.
Corollary 4.1: (Delay Stability under Max-Weight-α) Consider the switched
queueing network of Section 4.1 under the Max-Weight-α scheduling policy. If the αparameters of all light-tailed flows are equal to one, and the α-parameters of heavy-tailed
flows are sufficiently small, then all light-tailed flows are delay stable.
Proof. We recall our standing assumption that all traffic flows have (1 + γ) moments, for
some γ > 0. Thus, with the suggested choice of α-parameters, Theorem 4.3 guarantees that
the expected steady-state queue lengths of all light-tailed flows are finite. Lemma 2.1 relates
this result to delay stability.
101
By combining this with Theorem 4.1, we conclude that the Max-Weight-α policy is
optimal with respect to the delay stability metric, provided the α-parameters are
chosen suitably.
Max-Weight-α turns out to perform well in terms of another criterion. Theorem 4.3
h
i
α +1
implies that by choosing the α-parameters so that E Af f (0) is finite, for all f ∈ F, the
i
h
αf
steady-state queue-length moment E Qf is finite, for all f ∈ F. The following result
suggests that, for traffic flows with polynomially decaying tails, this is the best we can do
under any regenerative scheduling policy.
Theorem 4.4: Consider the switched queueing network of Section 4.1 under a regenerh
i
h i
ative scheduling policy. If E A1+γ
is
infinite,
for
some
f
∈
F
and
γ
>
0,
then
E
Qγf is
(0)
f
infinite.
Proof. This result is well-known in the context of a M/G/1 queue, e.g., see Section 3.2 of [11].
It can be proved similarly to Theorem 4.1.
Thus, when the α-parameters are chosen suitably, the Max-Weight-α policy guarantees
the finiteness of the highest possible queue-length moments for flows with polynomially
decaying tails.
4.4.3
Scaling Results under Light-Tailed Traffic
Although the focus of this thesis is on heavy-tailed traffic and its consequences, some implications of Theorem 4.3 are of broader interest. In this section we assume that the network
receives light-tailed traffic, and we analyze the scaling of the sum of the α-moments of
steady-state queue lengths with traffic intensity.
Corollary 4.2: (Scaling with Traffic Intensity) Fix a switched queueing network
and constants α ≥ 1 and B > 0. The Max-Weight-α policy is applied with αf = α, for all
h
i
α+1
f ∈ F. Assume that the arriving traffic is admissible, and that E Af (0) ≤ B, for all
102
f ∈ F. Then,
X M k ∗ , α, B
,
E Qαf ≤
(1 − ρ)α
f ∈F
where M k ∗ , α, B is a constant that depends only on k ∗ , α, and B.
Moreover, under any stabilizing scheduling policy,
X M 0 (α)
E Qαf ≥
,
(1 − ρ)α
f ∈F
where M 0 (α) is a constant that depends only on α.
Proof. The first part of the result follows directly from Theorem 4.3. Regarding the second
part, Theorem 2.1 of [54] implies that, under any stabilizing scheduling policy, there exists
an absolute constant M̃ such that
X M̃
.
E Qf ≥
1−ρ
f ∈F
Using Jensen’s inequality, we have
X X α
1 X α
α
E Qf ≥
E Qf
.
E Qf
≥ α
F f ∈F
f ∈F
f ∈F
Consequently, there exists constant M 0 (α) that depends only on α, such that
X M 0 (α)
E Qαf ≥
,
α
(1
−
ρ)
f ∈F
under any stabilizing scheduling policy.
Similar scaling results appear in queueing theory, mostly in the context of single-server
queues, e.g., see Chapter 3 of [32]. More recently, results of this flavor have been shown for
particular queueing networks, such as input-queued switches [51, 54]. However, all existing
results concern the scaling of first moments of queue lengths/delays. Corollary 4.2 gives the
103
precise scaling of higher order steady-state queue-length moments with traffic intensity, and
shows that Max-Weight-α achieves the optimal scaling of α-moments.
4.4.4
Scaling Results under Bernoulli Traffic
Finally, we turn our attention to the performance of the Max-Weight scheduling policy under
Bernoulli traffic, i.e., when each of the arrival processes Af (t); t ∈ Z+ is an independent
Bernoulli process with parameter λf > 0. We denote by Smax the maximum number of traffic
flows that any schedule s ∈ S can serve.
The following bound characterizes the performance of Max-Weight in terms of structural
parameters of the network and the traffic intensity.
Corollary 4.3: (Scaling under Bernoulli Traffic) Consider the switched queueing
network of Section 4.1 under the Max-Weight scheduling policy. Assume that the traffic
arriving to the network is Bernoulli, with traffic intensity ρ < 1. Then,
1 + ρ
X E Qf ≤ 2k ∗ Smax
.
1
−
ρ
f ∈F
Proof. If all traffic flows are light-tailed and all α-parameters are equal to one, then a more
careful accounting in the proof of Theorem 4.3 provides the following tighter upper bound:
i
X X h
2k ∗ E Qf ≤
Smax +
E A2f (0) .
1−ρ
f ∈F
f ∈F
h
i
If the traffic arriving to the network is Bernoulli, then E A2f (0) = λf , for all f ∈
F. Moreover, the fact that the arriving traffic has intensity ρ, implies the existence of
nonnegative real numbers ζs , for s ∈ S, such that
λf ≤
X
ζs · sf ,
s∈S
104
∀f ∈ F,
with
X
ζs = ρ.
f ∈F
Consequently,
i X
X h
XX
X
E A2f (0) =
λf ≤
ζs · sf ≤
ζs · Smax = ρ · Smax ,
f ∈F
f ∈F
f ∈F s∈S
s∈S
and the result follows.
Example: (n Parallel Queues) Consider a single-server system with n parallel queues.
The arriving traffic is assumed to be Bernoulli, with traffic intensity ρ < 1. In this case,
k ∗ = n and Smax = 1. Corollary 4.3 implies that, under the Max-Weight scheduling policy,
the sum of the steady-state queue lengths is bounded from above as follows:
n
X
4n
E Qi ≤
.
1−ρ
i=1
The aggregate queue length of a system of parallel queues under a work-conserving
scheduling policy evolves like a discrete time M/G/1 queue, from which we infer that
Pn
1
E
Q
=
Θ
. So, in the context of parallel queues, the scaling provided by Coroli
i=1
1−ρ
lary 4.3 is tight with respect to the traffic intensity, but not necessarily tight with respect to
the size of the network.
Example: (n × n Input-Queued Switch) Consider a n × n input-queued switch. The
arriving traffic is assumed to be Bernoulli, with traffic intensity ρ < 1. In this case, k ∗ = n
and Smax = n. Corollary 4.3 implies that, under the Max-Weight scheduling policy, the sum
of the steady-state queue lengths is bounded from above as follows:
n X
n
X
4n2
E Qij ≤
.
1−ρ
i=1 j=1
In the context of an input-queued switch, the joint scaling provided by Corollary 4.3, in
105
terms of both the traffic intensity and the size of the network, is the tightest currently known.
However, it should be noted that the correct scaling under Max-Weight, as ρ goes to one
and as n becomes large, is an open problem [51]. On a related note, a different scheduling
policy has been recently shown to achieve the optimal joint scaling [52].
4.5
Concluding Remarks
The main conclusion of this chapter is that the celebrated Max-Weight scheduling policy
performs poorly in the presence of heavy-tailed traffic. Specifically, we showed that delay
instability not only arises in heavy-tailed flows, but may propagate to light-tailed flows as
well. The extent of this propagation depends on the scheduling constraints and the arrival
rates. The shortcomings of Max-Weight can be alleviated through the parameterized MaxWeight-α policy, if the parameters are chosen suitably.
We conclude the chapter with some brief remarks.
In order for the Max-Weight-α policy to perform well, i.e., in order to choose the right
parameter values, some knowledge of higher order statistics of all traffic flows is required. If
this information is not available, and the α-parameters are not chosen appropriately, then,
in light of Theorem 4.4, this policy may also perform poorly. We conjecture that the solution
in this case could be an adaptive policy that learns the traffic characteristics and accordingly
modifies the scheduling decisions. This direction is not pursued further in this thesis.
In the simple queueing system of Figure 4-2 we observed that the heavy-tailed flow 1
causes all other flows to be delay unstable, for certain arrival rates. It is, thus, tempting
to make the following conjecture: consider a single-hop network where every traffic flow
conflicts with some other flow, and at least one flow in the network is heavy-tailed. There
exists an arrival rate vector in the stability region such that all flows are delay unstable
under the Max-Weight policy. In Section 5.6 we show that this conjecture is valid for the
case of disjoint schedules.
Although the outlines of the proofs of Propositions 4.1 and 4.2 are quite easy to grasp,
106
the proofs themselves are long and somewhat tedious. This seems to be an unavoidable
drawback of stochastic analysis, despite the fact that the queueing system at hand was,
arguably, very simple. The study of more complex queueing systems, and the way fluid
approximations can simplify their delay analyses, are the subject of Chapter 5.
+
Finally, in the proof of Proposition 4.2, one can interpret the upper bound on J(τ )
as an upper bound on the workload of a discrete time stable M/G/1 queue, where customers arrive in a Bernoulli fashion, and their service times are mutually independent and
distributed identically to the random variable A1 (0)+A2 (0) . Since service times are heavytailed distributed, the expected steady-state workload in this queue is infinite (an immediate
corollary of the Pollaczek-Khinchine formula). Combined with Fatou’s lemma, this implies
that the expected workload at time slot t goes to infinity as t increases. In that light,
Eq. (28) shows that the expected workload goes to infinity at a sublinear rate, specifically
as O t1/(1+γ) , assuming just the existence of the (1+γ) moment of service times, where
γ ∈ (0, 1). We note that Central-Limit-Theorem-type arguments could not have been used
here because the second moment of service times is infinite. Instead, our proof is based on
the Marcinkiewicz-Zygmund Strong Law of Large Numbers.
107
108
Chapter 5
Delay Analysis of the Max-Weight
Policy via Fluid Approximations
In this chapter we build on, and extend the results of, Chapter 4. More specifically, we
continue the delay stability analysis of a single-hop switched queueing network under the
Max-Weight policy, and in the presence of heavy-tailed traffic. Our main motivation comes
from the findings of Propositions 4.1 and 4.2: a light-tailed traffic flow can be delay unstable
even if it does not conflict with heavy-tailed traffic. Delay stability/instability in this case
depends on the arrival rates of the different traffic flows, and the notion of “delay stability
region,” i.e., the subset of the stability region for which a traffic flow is delay stable, arises.
Propositions 4.1 and 4.2 provided a sharp characterization of the delay stability region of flow
2, in the queueing system of Figure 4-2. Although the proofs of these results, based on purely
stochastic arguments, were long and tedious, the main ideas behind them were rather simple
and intuitive. Those ideas were also presented through an informal “fluid approximation” to
the stochastic system. The goal of this chapter is to formalize this approach: we will show
that the formal use of fluid approximations simplifies delay stability analysis, and allows us
to generalize considerably the findings of Propositions 4.1 and 4.2.
The main contributions of this chapter can be summarized as follows.
(i) We showcase the use of fluid approximations in the delay analysis of the Max-Weight
109
policy under heavy-tailed traffic. Specifically, we show how fluid approximations can be
combined with renewal theory in order to prove delay instability results. Moreover, we show
how fluid approximations can be combined with stochastic Lyapunov theory in order to
prove delay stability results;
(ii) We illustrate how to obtain exponential bounds on queue-length asymptotics in the
presence of heavy-tailed traffic, through drift analysis of a particular class of piecewise linear
Lyapunov functions;
(iii) We provide a sharp characterization of the delay stability regions of the Max-Weight
policy in single-hop switched queueing networks with disjoint schedules, generalizing the
findings of Propositions 4.1 and 4.2. Along the way, our analysis reveals several monotonicity
properties of service rates under Max-Weight scheduling;
(iv) We show that, in every network with disjoint schedules that operates under the
Max-Weight policy and receives heavy-tailed traffic, there exists an arrival rate vector in the
corresponding stability region for which all queues are delay unstable.
The remainder of the chapter is organized as follows. Section 5.1 includes a high-level
discussion of the methodological challenges posed by the problem at hand, and of the methodological contributions of this chapter. In Section 5.2 we provide some background material
on the fluid model of a single-hop switched queueing network under the Max-Weight policy. In Section 5.3 we show how fluid approximations, combined with renewal theory, can
be used to prove delay instability results. Building on this finding, we present, in Section
5.4, a Bottleneck Identification algorithm, which systematically tests for delay instability
by solving the fluid model from certain initial conditions. We illustrate the computational
value of this algorithm through concrete examples. In Section 5.5 we turn our attention to
delay stability. We present a class of piecewise linear Lyapunov functions that are suitable
for proving delay stability results in the presence of heavy-tailed traffic, and we show how
fluid approximations can simplify their drift analysis. In Section 5.6 we analyze a switched
queueing network with disjoint schedules under the Max-Weight policy, and characterize the
delay stability regions of the different flows. We conclude in Section 5.7 with some brief
110
remarks.
5.1
Methodological Challenges and Contributions
The problem of delay analysis of the Max-Weight policy in the presence of heavy-tailed traffic
poses a number of methodological challenges. First of all, Dynamic Programming or Markov
Decision Problem formulations of scheduling problems in queueing systems are known to
be analytically intractable, and to have prohibitive computational requirements. Moreover,
Monte Carlo methods can be very slow, or even inconclusive, due to the very nature of heavy
tails.
Regarding classical “queueing theoretic” approaches, a standard way of showing that
queues exhibit large delays (e.g., lower bounds on queue-length/delay asymptotics or on
the corresponding expected values) relies on sample path arguments. However, tracking the
evolution of sample paths can be hard when the system exhibits complex dynamics, such as
the ones imposed by Max-Weight. This also hinders the use of transform methods, at least
as a way to obtain analytical results. The main idea of our approach is as follows: even
when we are not able to analyze sample paths explicitly, we might still be able to do so
approximately, in terms of the solution to the fluid model from certain initial conditions of
interest. Then, we can use renewal theory to translate sample path analysis to lower bounds
on queue-length asymptotics or steady-state moments.
On the other hand, showing that queues exhibit low delays (e.g., upper bounds on queuelength/delay asymptotics or on the corresponding expected values) is usually based on drift
analysis of suitable Lyapunov functions, since coupling arguments can only be applied to
systems with relatively simple dynamics. Unfortunately, popular candidates such as standard
piecewise linear functions [6, 19], quadratic functions [59], and norms [50, 62] cannot be used
under heavy-tailed traffic. This is because the steady-state expectation of these functions
is infinite, rendering their drift analysis uninformative. Additionally, drift analysis can be a
challenge by itself under stochastic dynamics. Our approach to this problem is as follows: we
111
identify a class of piecewise linear Lyapunov functions that are nonincreasing in the length of
the heavy-tailed queues, and which are suitable for performance analysis (more specifically,
exponential upper bounds on queue-length asymptotics) of queueing systems with a mix of
heavy-tailed and exponential-type traffic. Moreover, we show how fluid approximations can
simplify the drift analysis of this class of Lyapunov functions. Critical to the latter is a
connection that we establish between fluid approximations and Lyapunov theory: if function
V (·) is continuous, piecewise linear, and a “Lyapunov function” for the fluid model, then
V (·) is also a “Lyapunov function” for the original queueing system.1 Moreover, if V (·)
has exponential-type “upward jumps” in the stochastic system, then Lemma 2.6 implies an
exponential upper bound for its steady-state distribution.
5.2
The Fluid Model of a Single-Hop Network
In this chapter we consider a single-hop switched queueing network under the Max-Weight
policy, namely the setting of Chapter 4. Here, we give some background material on the
natural fluid model of this system.
The fluid model is a deterministic dynamical system, which aims to capture the evolution
of its stochastic counterpart on longer time scales by taking advantage of Laws of Large
Numbers. Initially, we give a brief description and some useful properties of the fluid model.
Then, we introduce the notion of fluid scaling, and establish a formal connection between
the deterministic and the scaled stochastic system.
The Fluid Model (FM) of a single-hop switched queueing network under the MaxWeight policy is defined by the set of ordinary differential equations Eqs. (5.1)-(5.6), for
1
This is to be contrasted to the more common use of fluid approximations for stability analysis, where
a Lyapunov function V (·) for the fluid model implies the existence of another Lyapunov function G(·) for
the stochastic model.
112
every time t ≥ 0 for which the derivatives exist (such t is often called a regular time):
q̇f (t) = λf −
P
π∈S
ṡπ (t)πf + ẏf (t),
∀f ∈ F;
(5.1)
ṡπ (t) ≥ 0,
∀π ∈ S;
P
π∈S ṡπ (t) = 1;
P
0 ≤ ẏf (t) ≤ π∈S ṡπ (t)πf ,
∀f ∈ F;
qf (t) > 0 =⇒ ẏf (t) = 0,
∀f ∈ F;
n
o
P
P
=⇒ ṡπ (t) = 0,
f ∈F qf (t)πf < maxσ∈S
f ∈F qf (t)σf
(5.2)
(5.3)
(5.4)
(5.5)
∀π ∈ S.
(5.6)
In the equations above, q(t) represents the vector of queue lengths at time t, y(t) represents the vector of cumulative idling/wasted service up to time t, and sπ (t) represents
the total amount of time that schedule π has been activated up to time t. Eq. (5.3) states
that the scheduling policy is “work-conserving,” and Eq. (5.5) that there can be no wasted
service when queue lengths are positive. Finally, Eq. (5.6) is the natural analogue of the
Max-Weight policy in the fluid domain: schedules that do not have maximum weight receive
no service.
Fix arbitrary T > 0. A Fluid Model Solution (FMS) from initial condition q(0) = q
is a Lipschitz continuous function x(·) = q(·), y(·), s(·) that satisfies: (i) x(0) = (q, 0, 0);
(ii) Eqs. (5.1)-(5.6) over the interval [0, T ].
A FMS is differentiable almost everywhere equivalently, almost every t ∈ [0, T ] is a
regular time , since it is Lipschitz continuous by assumption.
First, we define the notion of fluid scaling, and we establish the existence of a fluid
limit and of a FMS. Consider a sequence of initial queue lengths Qb (0); b ∈ N for the
queueing system of Section 4.1, and the corresponding sequence of queue-length processes
b
Q (·); b ∈ N .
We define the “fluid-scaled” queue-length process as
q̃ b (t) =
Qb (bt)
,
b
t ∈ [0, T ],
113
b ∈ N.
We assume the existence of a vector q ∈ RF+ and of a sequence of positive numbers
b ; b ∈ N , converging to zero as b goes to infinity, that satisfy
max q̃fb (0) − qf ≤ b ,
f ∈F
∀b ∈ N.
We recall our standing assumption from Chapter 2 that there exists γ ∈ (0, 1) so that all
traffic flows have (1 + γ) moments. Fix some γ 0 ∈ (0, γ) and consider the sequence of sets of
sample paths of the arrival processes defined by
n
Hb = ω :
sup
1≤t≤bT
t−1
X
− γ 0 o
1
max Af (τ ) − λf < bT 1+γ ,
bT f ∈F τ =0
b ∈ N.
Intuitively, Hb contains those sample paths of the arrival processes that stay close to their
average behavior over the time interval [0, bT ].
Lemma 5.1: (Existence of Fluid Limit and of FMS) There exists a Lipschitz
continuous function z(t) = z1 (t), . . . , zF (t) , t ∈ [0, T ], such that for every > 0 there
exists b0 () so that
P Hb ≥ 1 − ,
∀b ≥ b0 (),
and
sup max q̃fb (t) − zf (t) ≤ ,
t∈[0,T ] f ∈F
∀ω ∈ Hb ,
∀b ≥ b0 ().
Additionally, there exist Lipschitz continuous functions v(·) and w(·), such that z(·), v(·), w(·)
is a FMS from initial condition q(0) = q over the interval [0, T ].
Proof. The fact that P Hb converges to one as b goes to infinity is shown in Lemma 2.4.
The part of the result regarding the existence of a fluid limit, and that a fluid limit is a
FMS, follows directly from Theorem 4.3 in [55] with the following correspondences. Our q0
corresponds to q0 in [55]. Our unique solution of the fluid model with initial condition q0
corresponds to FMS(q0 ) in [55]. Our b corresponds to both j and z in [55]. In particular,
γ0
j in [55] is identified with our (bT )− 1+γ , 0j in [55] is identified with our b , and our set Hb
114
corresponds to the set Gj in [55]. Condition (25) in [55] is simply the requirement that the
arrival sample path belong to Hb .
Lemma 5.2: (Uniqueness and Continuity of FMS): For any given q ∈ RF+ there
exists a Lipschitz continuous function z(t) = z1 (t), . . . , zF (t) , t ∈ [0, T ], such that the
queue-length part of every FMS from initial condition q is z(·). Moreover, z(·) depends
continuously on both the initial condition q and the arrival rate vector λ.
Proof. The existence of a FMS was established in Lemma 5.1. The uniqueness of the FMS
was proved in [58], by first showing that the Max-Weight policy is a maximal monotone
map from the space of queue lengths to the space of scheduling vectors, and then invoking
known properties of such maps. In Appendix 5.1 we provide a self-contained proof of this
result.
5.3
Delay Instability Results via Fluid Approximations
In this section we illustrate how fluid approximations can be used for proving delay instability results. A standard way for showing large delays (e.g., lower bounds on the tails
asymptotics or on the corresponding expected values) is through sample path arguments.
However, tracking the evolution of sample paths can be hard when the system exhibits complex dynamics. The main idea of our approach is to analyze sample paths approximately, as
the solution to the FM from certain initial conditions of interest, and then to translate the
findings to delay instability results through renewal theory.
Our contribution is summarized in the following theorem, which provides a sufficient
condition for the delay instability of traffic flows.
Theorem 5.1: Consider the single-hop switched queueing network of Section 4.1 under
the Max-Weight policy, and its natural FM of Section 5.2. Let h ∈ F be a heavy-tailed
traffic flow, and q ∗ (·) be the (unique) queue-length part of a FMS from initial condition
115
qh∗ (0) = 1 and qf∗ (0) = 0, for all f 6= h. If there exists τ ∈ [0, T ] such that qj∗ (τ ) > 0, then
traffic flow j is delay unstable.
Proof. Let us first look at the evolution of the system when it starts from a large initial condition for the heavy-tailed queue h. Specifically, consider a sequence of single-hop switched
queueing networks, indexed by b ∈ N, with initial queue lengths Qbh (0) = b and Qbf (0) = 0,
for all f 6= h. Let Qb (·); b ∈ N be the sequence of (unscaled) queue-length processes
under the Max-Weight policy. We define a corresponding sequence of scaled queue-length
processes by letting
q̃ b (t) =
Qb (bt)
.
b
Instead of studying directly the process Qb (·), we will exploit the fact that its scaled version
behaves as a simpler, deterministic fluid model for sufficiently large b.
The initial condition of the scaled processes, and of the corresponding fluid model, is
one for queue h and zero for all other queues. Lemma 5.2 implies that, for the given initial
condition, there exists a unique solution to the FLM, which we denote by q ∗ (·).
Fix j ∈ F and suppose that there exist , τ > 0, such that
qj (τ ) > .
(5.7)
Lemma 5.1 implies that there exists some finite b0 such that, for all b ≥ b0 ,
P Hb ≥ 1 − ,
(5.8)
and
b
q̃j (τ ) − qj (τ ) ≤ ,
2
∀ω ∈ Hb ,
where Hb is the set of arrival sample paths
n
Hb = ω :
t
X
γ0 o
− 1+γ
1
sup
max Af (τ ) − λf t < bT
.
1≤t≤bT bT f ∈F τ =1
116
(5.9)
(Strictly speaking b0 is a function of , but to make the notation simpler we suppress this
dependence.) Eqs. (5.7) and (5.9) imply that
Qbj (bτ ) ≥
b
,
2
∀ω ∈ Hb ,
∀b ≥ b0 .
(5.10)
In the remainder of the proof we show that: (i) the particular initial condition can be
reached with positive probability; (ii) the fact that queue j builds up to order b with positive
probability implies the delay instability of traffic flow j. The main idea is that queue j will
take order Ω(b) time to be drained, so that the integral of its length over a busy period
is order Ω b2 . Averaging over all possible values of b, and using the assumption that b2
has infinite expectation, renewal theory implies that the steady-state length of queue j has
infinite expectation.
As shown in Chapter 4, Max-Weight is a regenerative scheduling policy so that the
sequence of time slots that initiate busy periods of the system constitute a renewal process;
see Lemma 4.1. We define an instantaneous reward on this renewal process:
RM (t) = min Qj (t), M ,
t ∈ Z+ ,
where M is a positive integer.
Let us focus on a particular busy period of the system, which, without loss of generality,
starts at time slot zero. Consider the set of sample paths of the system
Ĥb = Ah (0) = b ∩ Af (0) = 0, f 6= h ∩ Hb .
Since λi < 1 from stability, and
1 − P Af (0) = 0 = P Af (0) > 0 ≤ E Af (0) = λf ,
we have that
P Af (0) = 0 > 0,
117
∀f 6= h.
Let Bh ⊂ Z+ be the support of the distribution of Ah (0). Using the independence of the
arrival processes, and taking into account Eq. (5.8), we have that
Y
P Ĥb ≥ (1 − )P Ah (0) = b ·
P Af (0) = 0 > 0,
∀b ∈ b ∈ Bh : b ≥ b0 .
(5.11)
f 6=h
For the unique FMS, once q ∗ (t) becomes zero, it stays at zero. Using this fact, Lemma
5.1 can be used to conclude that, for the sample paths in Ĥb and for b sufficiently large,
queue h is nonempty throughout the interval (0, bτ ]. Thus, time slot bτ belongs to the busy
period that started at time slot zero. Since at most one packet departs from queue j at
each time slot, Eq. (5.10) implies that the length of queue j is at least b/4 packets over a
time period of duration b/4 time slots. Taking into account Eq. (5.11), we have that the
M
, i.e., the reward accumulated over a renewal period, satisfies the lower
aggregate reward Ragg
bound
M
Ragg
· 1{b≥b0 } · 1Ĥb ≥ min
n b 2
4
o
· 1{b≥b0 } , M 2 · 1Ĥb .
Then, the expected aggregate reward is bounded from below by
o
h
i
n b 2
Y
X
M
· 1{b≥b0 } , M 2 · P Ah (0) = b .
E Ragg
≥ (1 − )
P Af (0) = 0 ·
min
4
f 6=h
b∈B
h
So, there exists a positive constant 0 , such that
h
i
h
n A (0) 2
oi
h
M
0
2
· 1{Ah (0)≥b0 } , M
.
E Ragg ≥ E min
4
Lemma 2.2 applied to Y = (1/16)2 A2h (0) · 1{Ah (0)≥b0 } implies that E Qj is infinite.
Finally, Lemma 2.1 implies the desired result.
Remark: Theorem 5.1 holds for any choice of T > 0 (the horizon of the FMS). However,
the fact that a single-hop switched queueing network is stable under the Max-Weight policy
(see Lemma 4.1) implies the existense of some T ∗ > 0, proportional to the initial condition of
the FMS, such that q(t) = 0, for all t > T ∗ . Consequently, the most effective application of
118
Theorem 5.1 is when T is chosen large enough so that the FMS “drains” within this horizon.
5.4
The Bottleneck Identification Algorithm
Theorem 5.1 provides a sufficient condition for the delay instability of traffic flows, based on
the FMS from a specific initial condition. The following algorithmic procedure, which we
term the Bottleneck Identification (BI) algorithm, tests this condition for all initial
conditions of interest.
INITIALIZATION: U = ∅
REPEAT
For every heavy-tailed traffic flow h ∈ F,
(i) solve the FM with initial condition one for queue h, and zero for all other queues;
(ii) find the set of queues that become positive at any point before the FMS drains, Uh ;
(iii) set U = U ∪ Uh ;
END
Clearly, all queues/traffic flows included in U are delay unstable. Below we present
concrete examples that illustrate the use of the BI algorithm.
3-Queue System
We start with the simple 3-queue system of Figure 4-2, where we first observed the phenomenon of rate-dependent delay instability. Roughly speaking, Propositions 4.1 and 4.2
imply that traffic flow 2 is delay stable if and only if λ2 < 1 + λ1 − λ3 /2. The proofs
in Chapter 4 were based on stochastic (non-fluid) arguments. In Section 5.6 we illustrate
how Theorem 5.1 can be used to obtain the same delay stability condition with much less
effort (in fact, we obtain necessary and sufficient delay stability conditions for more general
networks with disjoint schedules). Here, we use this example as a sanity check for the BI
algorithm.
119
Consider the set of arrival rates λ1 = 0.4, λ2 = 0.55, and λ3 = 0.4. Figure 5-1 shows the
FMS for this particular set of rates, and with initial condition one for queue 1, and zero for
queues 2 and 3. Notice that the length of queue 2 becomes positive before the FMS drains,
so according to Theorem 5.1 traffic flow 2 is delay unstable. This agrees with Proposition
4.1, since λ2 > 1 + λ1 − λ3 /2 in this case.
Figure 5-1: The FMS of the 3-queue system of Figure 4-2, with initial condition one for
queue 1 and zero for the other queues, and arrival rates λ1 = 0.4, λ2 = 0.55, λ3 = 0.4.
3×3 Switch
Consider a 3×3 input-queued switch, a system of 9 queues indexed by (i, j), i = 1, . . . , 3, j =
1, . . . , 3, with index i representing the input port and index j the output port of the switch.
A schedule is a matching between input and output ports. Namely, the set of schedules is
as follows:
S=
n
(1, 1), (2, 2), (3, 3) , (1, 1), (2, 3), (3, 2) , (1, 2), (2, 1), (3, 3) ,
o
(1, 2), (2, 3), (3, 1) , (1, 3), (2, 1), (3, 2) , (1, 3), (2, 2), (3, 1) .
120
The 3 × 3 input-queued switch is a single-hop switched queueing network for which an
explicit characterization of the delay stability regions is not available.
Consider the set of arrival rates λ11 = 0.1, λ12 = 0.1, λ13 = 0.1, λ21 = 0.1, λ22 = 0.38,
λ23 = 0.4, λ31 = 0.1, λ32 = 0.42, and λ33 = 0.44. Note that this set of rates satisfies
P
P
i λij < 1 and
j λij < 1, so that the system is stable [44].
We assume that traffic flow (1, 1) is heavy-tailed, while all other traffic flows are lighttailed. We are interested in the delay stability of flows (2,2), (2,3), (3,2), (3,3); these are
the flows that do not conflict with flow (1,1). Figure 5-2 shows the FMS for the considered
set of rates, and with initial condition one for queue (1,1), and zero for all other queues (we
present only the queues of interest). The lengths of all queues of interest become positive
before the FMS drains, so according to Theorem 5.1 they are delay unstable.
Figure 5-2: The FMS of a 3 × 3 switch, with initial condition one for queue (1,1) and zero
for the other queues, and arrival rates λ11 = 0.1, λ12 = 0.1, λ13 = 0.1, λ21 = 0.1, λ22 = 0.38,
λ23 = 0.4, λ31 = 0.1, λ32 = 0.42, λ33 = 0.44.
3×3 Grid Network
Consider the 3 × 3 grid network depicted in Figure 5-3. This system represents a wireless
network with interference constraints. Queues are identified with links and are indexed by
121
i = 1, . . . , 12. As soon as a packet is transmitted through the respective wireless link, it exits
the system. We assume the two-hop interference constraint model, i.e., if a wireless link is
transmitting, all links in a two-hop distance must idle. This implies that the set of schedules
is as follows:
n
S = {1, 11}, {1, 12}, {1, 10}, {2, 8}, {2, 11}, {2, 12}, {3, 5}
o
{3, 10}, {3, 12}, {4}, {5, 8}, {5, 11}, {6}, {7}, {8, 10}, {9} .
Again, this is a single-hop switched queueing network for which an explicit characterization of the delay stability regions is not available.
Figure 5-3: A 3 × 3 grid wireless network, with two-hop interference constraints.
Consider the set of arrival rates λ1 = 0.01, λ2 = 0.02, λ3 = 0.03, λ4 = 0.04, λ5 = 0.05,
λ6 = 0.06, λ7 = 0.07, λ8 = 0.08, λ9 = 0.09, λ10 = 0.1, λ11 = 0.11, and λ12 = 0.12. It can be
verified that this set of rates is in the stability region of the system.
We assume that traffic flow 1 is heavy-tailed, while all other traffic flows are light-tailed.
We are interested in the delay stability of traffic flows 10, 11, and 12, since these flows do
not conflict with flow 1. Figure 5-4 shows the FMS for the considered set of rates, and with
initial condition one for queue 1, and zero for all other queues (we present only the queues
of interest). The lengths of all queues of interest become positive before the FMS drains, so
according to Theorem 5.1 they are delay unstable.
122
Figure 5-4: The FMS of the 3 × 3 grid network of Figure 5-3, with initial condition one for
queue 1 and zero for the other queues, and arrival rates λ1 = 0.01, λ2 = 0.02, λ3 = 0.03,
λ4 = 0.04, λ5 = 0.05, λ6 = 0.06, λ7 = 0.07, λ8 = 0.08, λ9 = 0.09, λ10 = 0.1, λ11 = 0.11,
λ12 = 0.12.
5.5
Delay Stability Results via Fluid Approximations
In this section we shift our attention to delay stability results in networks that receive a
mix of heavy-tailed and exponential-type traffic. Typically, proving low delays is either
based on coupling arguments, if the underlying dynamics are relatively simple, or, more
often, on drift analysis of suitable Lyapunov functions. We focus on the latter approach.
The presence of heavy-tailed traffic, though, introduces an additional complication: popular
candidate Lyapunov functions such as standard piecewise linear functions [6, 19], quadratic
functions [59], and norms [50, 62] cannot be used because the steady-state expectation of
these functions is infinite under heavy-tailed traffic, rendering drift analysis uninformative.
We introduce a class of piecewise linear Lyapunov functions that are nonincreasing in
the length of the heavy-tailed queues, and which can provide exponential upper bounds
on queue-length asymptotics despite the presence of heavy-tailed traffic. However, drift
analysis of piecewise linear Lyapunov functions is sometimes a challenge by itself, due to
the fact that the stochastic descent property is often lost at locations where the function is
123
nondifferentiable. This difficulty can be handled by either smoothing the Lyapunov function,
e.g., as in [19], or by showing that the stochastic descent property still holds if we look ahead
a sufficiently large number of time slots, e.g., as in [61]. We follow the second approach,
and show how fluid approximations can simplify significantly drift analysis of this class of
functions.
Theorem 5.2: Consider the single-hop switched queueing network of Section 4.1 under
the Max-Weight policy, and its natural FM of Section 5.2. Consider a function V : RF+ → R+
of the form
V (x) = max
nX
j∈J
o
cjf xf ,
f ∈F
where J = {1, . . . , J}, and cjf ∈ R, for all j ∈ J , f ∈ F. Suppose that:
(i) there exists l > 0 such that, for every initial condition q(0) and regular time t ≥ 0,
we have V̇ (q(t)) ≤ −l, whenever V (q(t)) > 0;
h
i
(ii) there exists γ ∈ (0, 1) such that E A1+γ
(0)
< ∞, for all f ∈ F;
f
(iii) if f ∈ F is a heavy-tailed traffic flow, then cjf ≤ 0, for all j ∈ J .
Then,
(i) the sequence V Q(t) ; t ∈ Z+ converges in distribution to some random variable
V (Q);
(ii) E exp θV (Q) < ∞, for some θ > 0.
Proof. We are given that V (·) is a “Lyapunov function” for the fluid model of the queueing
system under consideration. We will show that V (·) is also a “Lyapunov function” for the
original stochastic system.
In particular, fix arbitrary T > 0. We will show that there exist α, ζ > 0 and b0 ∈ N
such that, for all b ≥ b0 and all t ∈ [0, T ],
E V Q(t + bT ) − V Q(t) + bζ; V Q(t) > αb Ft ≤ 0.
In showing this, we will make use of the facts that V (·) is continuous, piecewise linear, and
124
homogeneous.
We recall that Ft is the σ-algebra generated by Q(0), A(0), . . . , Q(t − 1), A(t − 1), Q(t),
and should be distinguished from the set of traffic flows F = {1, . . . , F }.
Fix γ 0 ∈ (0, γ). For any b ∈ N and any t ∈ [0, T ], consider the following set of sample
paths of the arrivals:
n
H̃b (t) = ω :
sup
1≤κ≤bT
t+κ−1
− γ 0 o
1
X
max Af (τ ) − λf < bT 1+γ .
bT f ∈F τ =t
For any b ∈ N and α > 0, we can write
1 E V Q(t + bT ) − V Q(t) ; V Q(t) > αb Ft
b
1 = E V Q(t + bT ) − V Q(t) ; V Q(t) > αb, H̃b (t) Ft
b
1 + E V Q(t + bT ) − V Q(t) ; V Q(t) > αb, H̃bc (t) Ft
b
i
h Q(t + bT ) Q(t) Q(t) =E V
−V
; V
> α, H̃b (t) Ft
b
b
b
1 + E V Q(t + bT ) − V Q(t) ; V Q(t) > αb, H̃bc (t) Ft ,
b
(5.12)
where the last equality follows from the fact that V (·) is homogeneous and invariant under
fluid scaling.
We begin by analyzing the first term on the right-hand side of Eq. (5.12). We can write
i
h Q(t + bT ) Q(t) Q(t) −V
; V
> α, H̃b (t) Ft
E V
b
b
b
h Q(t + bT ) Q(t) Q(t) i
=E V
−V
; V
> α H̃b (t), Ft · P(H̃b (t) | Ft )
b
b
b
h Q(bT ) Q(0) Q(0) i
=E V
−V
; V
> α Hb , F0 · P(Hb | F0 ),
(5.13)
b
b
b
where
n
Hb = ω :
κ−1
X
γ0 o
− 1+γ
1
max Af (τ ) − λf < bT
,
sup
1≤κ≤bT bT f ∈F τ =0
125
as defined in Section 5.1. The last equality follows from the fact that the arrival processes
are mutually independent and IID over time slots, and the system is Markovian with respect
to the vector of queue lengths.
Lemma 5.1 implies the existence of constants , b0 > 0 such that
P Hb | F0 ≥ 1 − ,
∀b ≥ b0 .
(5.14)
Now consider the sequence of initial conditions Q(0)b; b ∈ N . Similarly to Section
5.1, we can construct a sequence of unscaled and scaled queue-length processes Qb (·); b ∈
N and q̃ b (·); b ∈ N , respectively. Notice that q̃ b (0) = Q(0), for all b ∈ N. So, let
q(t), t ∈ [0, T ], be the FMS with initial condition q(0) = Q(0). Lemma 5.2 implies that it is
unique, and Lemma 5.1 that, with high probability, the scaled queue-length process will be
arbitrarily close to this FMS, if the scaling parameter b is chosen sufficiently large.
Combined with the fact that V (·) is continuous, we have that there exists function g :
N → R+ that goes to zero as its argument goes to infinity, and which satisfies:
V
Q(bT ) b
−V
Q(0) b
≤ V q(T ) − V q(0) + g(b),
∀ω ∈ Hb ,
∀b ≥ b0 .
(5.15)
By assumption, there exists l > 0 such that, for every initial condition q(0) and every
regular time t, we have V̇ q(t) ≤ −l, whenever V q(t) > 0. Moreover, almost every
t ∈ [0, T ] is a regular time. Finally, if V q(0) is sufficiently large, then V q(t) > 0, for all
t ∈ [0, T ]. These imply that
V q(T ) − V q(0) ≤ −lT,
(5.16)
for large enough V q(0) .
Eqs. (5.13)-(5.16) imply that there exist α > 0 (sufficiently large), δ > 0, b0 ∈ N, and
function g(·), such that
h Q(t + bT ) Q(t) Q(t) E V
−V
+ lT δ − g(b); V
> α, H̃b (t)
b
b
b
126
i
Ft ≤ 0,
(5.17)
for all b ≥ b0 , and with g(b) → 0 as b → ∞.
Let us now analyze the second term on the right-hand side of Eq. (5.12). We can write
E V Q(t+bT ) − V Q(t) ; V Q(t) > αb, H̃bc (t) Ft
= E V Q(t + bT ) − V Q(t) ; V Q(t) > αb H̃bc (t), Ft · P H̃bc (t) Ft .
(5.18)
The union bound on probabilities, and the fact that arrivals are IID and mutually independent, imply that
P
H̃bc (t)
X Ft ≤
P sup
f ∈F
1≤κ≤bT
t+κ−1
γ0
1 X
Af (τ ) − λf ≥ (bT )− 1+γ .
bT τ =t
(5.19)
Notice that the sequence
n t+κ−1
X
Af (τ ) − λf ; κ = 1, . . . , bT
o
τ =t
is a martingale, for every f ∈ F. Consequently, the sequence
n t+κ−1
o
X
Af (τ ) − λf ; κ = 1, . . . , bT
τ =t
is a nonnegative submartingale, for every f ∈ F.
Doob’s submartingale inequality (see, e.g., Section 14.6 of [64]) implies that
t+κ−1
− γ 0 1 X
P sup
Af (τ ) − λf ≥ bT 1+γ ≤
1≤κ≤bT bT
τ =t
bT
−1
h t+bT
i
1
X
E Af (τ ) − λf .
γ0
1− 1+γ
τ =t
(5.20)
Moreover, the Marcinkiewicz-Zygmund Strong Law of Large Numbers states that
P
t+bT −1
Af (τ ) − λf L1
τ =t
−→ 0,
1
bT 1+γ
127
∀f ∈ F
(see, e.g., Chapter 6.10 of [30]).
Therefore, there exist p, b1 > 0 such that
−1
h t+bT
i
1
X
E Af (τ ) − λf ≤ p bT 1+γ ,
∀b ≥ b1 ,
∀f ∈ F.
(5.21)
τ =t
Eqs. (5.19)-(5.21) imply that
Fp
0 ,
γ−γ
1+γ
bT
P H̃bc (t) Ft ≤
∀b ≥ b1 .
(5.22)
Recall that γ 0 ∈ (0, γ), so that P H̃bc (t) Ft → 0 as b → ∞.
Now let us look at the conditional expectation in Eq. (5.18). Let j̄ ∈ J be a piece of
V (·) that “dominates” at time slot (t + bT ), i.e.,
max
j∈J
nX
o
cjf Qf (t + bT ) =
f ∈F
X
cj̄f Qf (t + bT ).
f ∈F
We have that
t+bT
X−1
X
V Q(t + bT ) − V Q(t) ≤
cj̄f
Af (τ ) − Sf (τ ) · 1{Qf (τ )>0}
f ∈F
≤ bT
τ =t
X
X−1
− X + t+bT
− cj̄f
+
cj̄f
Af (τ ) .
f ∈F
f ∈F
τ =t
So, there exists c > 0 such that
E V Q(t + bT ) − V Q(t) ; V Q(t) > αb Hbc (t), Ft
i
X−1
X + h t+bT
≤c bT +
cj̄f · E
Af (τ ) H̃bc (t) .
f ∈F
τ =t
(5.23)
128
We have that
E
h t+bT
X−1
−1
i
h bT
X
c
Af (τ ) H̃b (t) = E
Af (τ ) τ =t
τ =0
−1
h bT
X
≤E
Af (τ ) τ =0
bT
−1
hX
≤E
κ−1
X
γ0 i
− 1+γ
1
sup
max Ak (τ ) − λk ≥ bT
1≤κ≤bT bT k∈F
τ =0
κ−1
γ0 i
− 1+γ
1 X
Af (τ ) − λf ≥ bT
sup
1≤κ≤bT bT τ =0
−1
bT
i
X
Af (τ ) ≥ 1 + λf bT ,
Af (τ ) τ =0
∀f ∈ F. (5.24)
τ =0
+
By assumption, the arrivals to all queues that satisfy cj̄f > 0 are IID over time slots
and exponential-type. Thus, there exists c0 > 0 such that
−1
−1
bT
X
+ h bT
i
X
cj̄f · E
Af (τ ) Af (τ ) ≥ 1 + λf bT ≤ c0 bT ,
τ =0
∀f ∈ F.
(5.25)
τ =0
This is due to the fact that, by conditioning on the event that an exponential-type random
variable takes a larger-than-average value, we change the “body” but fundamentally not the
“tail” of the underlying distribution. Thus, the conditional expectation remains finite.
Eqs. (5.23)-(5.25) imply the existence of d > 0 such that
E V Q(t + bT ) − V Q(t) ; V Q(t) > αb H̃bc (t), Ft ≤ d bT .
(5.26)
Eqs. (5.22) and (5.26) imply that, for some d0 > 0,
1 d0
E V Q(t + bT ) − V Q(t) ; V Q(t) > αb, H̃bc (t) Ft ≤ γ−γ 0 ,
b
b 1+γ
∀b ≥ b1 .
(5.27)
Finally, Eqs. (5.12), (5.17), and (5.27) imply that there exist α, ζ > 0 such that, for every
t ∈ Z+ and for sufficiently large b ∈ N,
E V Q(t + bT ) − V Q(t) + bζ; V Q(t) > αb Ft ≤ 0.
Notice that all queues that have positive coefficient in any piece of V (·) have exponential129
type arrivals. Then, Lemma 2.6 implies that the sequence V Q(t) ; t ∈ Z+ converges in
distribution to random variable V (Q), and that V (Q) is exponential-type.
5.6
Delay Stability Regions of the Max-Weight Policy
In this section we consider a single-hop switched queueing network with disjoint schedules
(i.e., each traffic flow belongs to exactly one schedule) and we provide a sharp characterization of the delay stability regions of the Max-Weight policy in the presence of heavy-tailed
traffic. To achieve this, we apply the connections between fluid approximations and delay
stability/instability that we established in Theorems 5.1 and 5.2.
More specifically, we consider a system with (K + 1) schedules, which we denote by
σ0 , σ1 , . . . , σK . Schedule σ0 includes (F0 +1) queues that we denote by (σ0 , f ), f = 0, . . . , F0 ;
schedule σk , k = 1, . . . , K, includes Fk queues that we denote by (σk , f ), f = 1, . . . , Fk . Since
the system has single-hop traffic, we use the notions of queue and traffic flow interchangeably.
We denote the arrival rate to queue (σk , f ) by λσf k . We assume that queues are indexed
in descending order of arrival rates, except, possibly, for queue (σ0 , 0) (the significance of the
latter queue will become obvious shortly). In mathematical terms,
λσf k > λσf k+1 ,
∀f ∈ 1, . . . , Fk − 1 ,
∀k ∈ {0, . . . , K}.
(5.28)
At each time slot at most one schedule can be activated. If a schedule is activated then
one packet is removed from all nonempty queues of that schedule.
We assume that the arriving traffic is in the stability region of the system, which implies
that
max
f =0,...,F0
K
X
λσf 0 +
max λσf k < 1.
k=1
f =1,...,Fk
(5.29)
Traffic flow (σ0 , 0) is heavy-tailed traffic whereas every other traffic flow is exponentialtype. Theorem 4.2 implies that, under the Max-Weight policy, every traffic flow that does
not belong to schedule σ0 is delay unstable, for any positive arrival rate, because it conflicts
130
with (σ0 , 0). In contrast, we expect traffic flows (σ0 , f ), f = 1, . . . , F0 to have nontrivial
delay stability regions, since they do not conflict with the heavy-tailed queue (σ0 , 0); much
similar to the 3-queue system in Figure 4-2, where traffic flow 2 had a nontrivial delay
stability region.
The following theorem provides a tight characterization of the delay stability regions of
the light-tailed flows of schedule σ0 , generalizing the findings of Propositions 4.1 and 4.2.
Theorem 5.3: Consider the single-hop switched queueing network with disjoint schedules described above, and arrival rates satisfying Eqs. (5.28) and (5.29). The conditions
f −1
λσf 0
K
X
X σ 1 σ0
λl −
λ1 k ,
≤
1+K
1 + Kf
l=0
k=1
f = j, . . . , F0 ,
(5.30)
are necessary for the delay stability of traffic flow (σ0 , j), j ∈ 1, . . . , F0 . If all inequalities
are strict, then these conditions are also sufficient for delay stability, and the steady-state
length of the associated queue is exponential-type.
Remark: The delay stability regions of light-tailed flows that do not conflict with the
heavy-tailed flow depend on the number of conflicting schedules and their highest arrival
rates, but not on how many flows are included in each schedule, or the lower arrival rates.
Remark: The delay stability of traffic flow (σ0 , j) when one of the above conditions
holds with equality will depend, in general, on higher order moments of the arrivals, and not
just the rates. To see this, suppose that a large batch of b packets arrives to the heavy-tailed
queue (σ0 , 0). A random-walk-type argument can show that queue (σ0 , j) will build up to
√ Ω b during an Ω(b) time interval, assuming that the configuration corresponding to the
equality condition is reached. Thus, the aggregate length of this queue over a busy period
will be Ω b3/2 , which implies that the delay stability of traffic flow (σ0 , j) will depend on
the 1.5 moment of the arrivals to the heavy-tailed queue.
The proof of Theorem 5.3 is based on analyzing the FM and, subsequently, applying
Theorems 5.1 and 5.2. Before proceeding to the proof, though, we need to introduce some
notation and background material.
131
The case where multiple schedules have maximum weight (in the FM), and the service
rate that schedule σ0 receives then, turns out to be critical for the delay stability regions
of the light-tailed flows of σ0 . On that occasion, the Max-Weight policy splits the total
available service rate which is equal to one according to Eq. (5.3) between the maximum
weight schedules so that the weights of those schedules remain the same this is a direct
consequence of Eq. (5.6) . The precise way that the available service rate is split, though,
depends on which queues of those schedules are nonempty, and their arrival rates.
To quantify this phenomenon, we consider the indicator variable xσf k , which is equal to one
if schedule σk has maximum weight and queue (σk , f ) is nonempty, and zero otherwise. Based
on these indicator variables, we introduce the notion of a configuration, which is a vector
of binary vectors, representing the nonempty queues of the maximum weight schedules. A
generic configuration has the form x = xσ0 , xσ1 , . . . , xσK , where xσ0 = xσ0 0 , . . . , xσF00 and
xσk = xσ1 k , . . . , xσFkk , k = 1, . . . , K. In a dynamic context, the configuration, at any given
time t, is a function of the queue lengths and we denote it by x q(t) .
Since we wish to analyze the delay stability of the light-tailed flows of schedule σ0 , we
are interested in configurations where σ0 has maximum weight, and the service rate that it
receives then. Let x be such a configuration, and µσk (x) the service rate that schedule σk
receives under x. A direct consequence of Eqs. (5.3) and (5.6) is that the service rates satisfy
F0
X
λσf 0 xσf 0 − |xσ0 |µσ0 (x) =
f =0
Fk
X
λσf k xσf k − |xσk |µσk (x),
∀k ∈ K(x),
(5.31)
f =1
X
µσk (x) = 1,
(5.32)
k∈K(x)
where |xσ0 | =
PF0
f =0
xσf 0 , |xσk | =
PFk
f =1
xσi k , and K(x) is the set of maximum weight schedules
under configuration x.
In Appendix 5.2 we prove certain monotonicity properties of the service rates, which are
crucial to the proof of Theorem 5.3. Due to these properties we are particularly interested
in the parameterized configuration xf , f ∈ 0, 1, . . . , F0 , where: (i) all (K + 1) schedules
132
have maximum weight; (ii) only the queue with the highest arrival rate from schedule σk is
nonempty, whereas every other queue of that schedule is empty, for all k ∈ {1, . . . , K}; (iii)
queues (σ0 , 0), . . . , (σ0 , f ) are nonempty, whereas all other queues of schedule σ0 are empty.
Proof. Eq. (5.31)-(5.32) and some simple algebra imply that the conditions in the statement
of the theorem are equivalent to
λσf 0 ≤ µσ0 (xf ),
f = j, . . . , F0 .
First, suppose that there exists f0 ∈ j, . . . , F0 such that
λσf00 > µσ0 (xf0 ).
We will show that queue (σ0 , j) is delay unstable, verifying, thus, the necessity part. We do
that by using Theorem 5.1, i.e., we solve the FM with initial condition one for queue (σ0 , 0)
and zero for all other queues. In particular, we will show that queue (σ0 , j) becomes positive
along the FMS, which implies that it is delay unstable.
Since the arriving traffic is in the stability region of the network, the queue lengths
of the FMS become zero after finite time, and remain zero from that point on. Before
the FMS drains, though, the solution reaches a configuration where only the highest rate
queue from each of the schedules σ1 , . . . , σK and the heavy-tailed queue (σ0 , 0) are drained
simultaneously. Moreover, the weights of all schedules are exactly the same. This is a
consequence of the following facts: (i) all queues within a schedule receive the same amount
of service, so the last nonempty queue is always the one with the highest arrival rate; (ii)
since the schedules are disjoint, the weights of all schedules become equal after finite time,
and remain equal thereafter.
At this point queue (σ0 , j) will start building up (in fact, all queues 1, . . . , f0 of schedule
σ0 will start building up), since λσj 0 > λσf00 > µσ0 (xf0 ), and the fact that µσ0 (xf0 ) upper
bounds the service rate that schedule σ0 may receive; see Lemma 5.3.
133
Now, we show that queue (σ0 , j) is delay stable if
λσf 0 < µσ0 (xf ),
∀f ∈ j, . . . , F0 ,
verifying, thus, the sufficiency part. We do that by using Theorem 5.2: consider the candidate
Lyapunov function
V (q) =
F0
X
cf qfσ0
f =j
+ max
Fk
nh X
k=1,...,K
qfσk
−
F0
X
qfσ0
i+ o
,
f =0
f =1
where cf ∈ (0, 1), for all f ∈ j, . . . , F0 .
Let q(0) be an arbitrary (finite) initial condition for the queue lengths in the FM. We
will verify that, if the cf -parameters are properly chosen, then V̇ q(t) is uniformly negative
whenever V q(t) > 0 and the derivative exists. Then, Theorem 5.2 will directly imply
that queue (σ0 , j) is delay stable, since V (·) is piecewise linear and the queue that receives
heavy-tailed traffic has negative coefficient.
Since the system is known to be stable under the Max-Weight policy, there exists a finite
time T , which depends on q(0), when the FMS drains. This implies that V q(t) = 0, for
any t ≥ T . So, it suffices to show that there exists l > 0 such that V̇ q(t) ≤ −l, for any
regular time in [0, T ) that satisfies V (q(t)) > 0.
We distinguish between two cases in our analysis.
(i) Schedule σ0 does not have maximum weight at time t: in this case the
candidate Lyapunov function reduces to
Fk∗
j−1
F0
X
X
X
V q(t) =
qfσk∗ (t) −
qfσ0 (t) −
1 − cf qfσ0 (t),
f =1
f =0
f =j
for some k ∗ ∈ {1, . . . , K}. Since schedule σ0 does not have maximum weight, at least one of
the queues of schedule σk∗ must be nonempty at time t.
134
If k ∗ is the unique maximum weight schedule, then
Fk∗
j−1
F0
X
X
X
V̇ q(t) =
λσf k∗ − 1 · 1{qσk∗ (t)>0} −
λσf 0 · 1{qfσ0 (t)>0} −
1 − cf λσf 0 · 1{qfσ0 (t)>0} .
f
f =1
f =0
f =j
The right-hand side of the expression above is strictly negative. This is due to Eqs. (5.28)
(5.29) and our assumption that cf ∈ (0, 1), for all f ∈ j, . . . , F0 .
The same holds even if k ∗ is one of multiple schedules with maximum weight; this can
be easily derived from the fact that the arriving traffic is in the stability region, and that
Max-Weight drains the weights of all maximum weight schedules at the same rate;
(ii) Schedule σ0 has maximum weight at time t: in this case the candidate Lyapunov
function reduces to
V q(t) =
F0
X
cf qfσ0 (t).
f =j
Since V q(t) > 0, we have that at least one of the queues (σ0 , j), . . . , σ0 , F0 is nonempty.
We distinguish between two subcases: if schedule σ0 is the unique maximum weight
schedule at time t, then
F0
X
V̇ q(t) =
cf λσf 0 − 1 · 1{qfσ0 (t)>0} < 0.
f =j
On the other hand, if schedule σ0 is one of multiple schedules with maximum weight at
time t, then
V̇ q(t) =
F0
X
cf λσf 0 − µσ0 x q(t)
· 1{qfσ0 (t)>0} ,
f =j
and we are given that
λσf 0 < µσ0 (xf ),
∀f ∈ j, . . . , F0 .
We further distinguish between subcases:
(a) If all queues of schedule σ0 are nonempty at time t, then Lemmas 5.3, 5.4,
135
and 5.5 imply that
µσ0 xF0 ≤ µσ0 x q(t) .
Thus, if we chose cf cF0 , for all f ∈ j, . . . , F0 − 1 , then V̇ q(t) < 0 because λσF00 <
µσ0 xF0 ;
(b) If all but one queue of schedule σ0 are nonempty at time t, then Lemmas
5.3, 5.4, and 5.5 imply that
µσ0 xF0 −1 ≤ µσ0 x q(t) .
Thus, if we chose cf cF0 −1 , cF0 , for all f ∈ j, . . . , F0 − 2 , then V̇ q(t) < 0 because at
least one of the queues σ0 , F0 and σ0 , F0 − 1 is nonempty and λσF00 < λσF00 −1 < µσ0 xF0 −1 ;
The other cases are treated similarly.
We conclude the section with a result that illustrates the detrimental impact that heavytailed traffic may have on the network overall, under the Max-Weight policy.
Theorem 5.4: Consider the single-hop switched queueing network with disjoint schedules described above. There exist arrival rates satisfying Eqs. (5.29) such that all traffic flows
in the network are delay unstable.
Proof. As already mentioned above, traffic flow (σ0 , 0) is delay unstable for any positive
arrival rate, because it receives heavy-tailed traffic; see Theorem 4.1. Moreover, under the
Max-Weight policy, every traffic flow that does not belong to schedule σ0 is also delay
unstable for any positive arrival rate, since it conflicts with (σ0 , 0); see Theorem 4.2. So, in
order to prove the result, it suffices to show that there exist (strictly positive) arrival rates
in the stability region, for which traffic flows (σ0 , f ), f = 1, . . . , F0 , are delay unstable.
Fix > 0 and consider the arrival rates
λσ0 0 = λσf k = > 0,
∀f ∈ 1, . . . , Fk ,
∀k ∈ {1, . . . , K},
and
λσf 0 = 1 − (K + 1),
136
∀f ∈ 1, . . . , F0 .
We assume that < 1/(K + 2), so that the above values correspond to strictly positive
arrival rates, and also < 1 − (K + 1) .
Notice that
max
f =0,...,F0
λσf 0
+
K
X
k=1
max
f =1,...,Fk
λσf k = 1 − < 1,
which implies that Eq. (5.29) holds for the particular choice of arrival rates.
In the necessity part of the proof of Theorem 5.3 we studied the FMS from an initial
condition of one for queue (σ0 , 0) and zero for all other queues. We argued that, before the
FMS drains, the solution reaches a configuration where only the highest rate queue from each
of the schedules σ1 , . . . , σK and the heavy-tailed queue (σ0 , 0) are drained simultaneously.
Moreover, the weights of all schedules are exactly the same.
Now recall the definition of configuration xF0 , as the one where (i) all (K + 1) schedules
have maximum weight; (ii) only the queue with the highest arrival rate from schedule σk is
nonempty, whereas every other queue of that schedule is empty, for all k ∈ {1, . . . , K}; (iii)
queues (σ0 , 0), . . . , (σ0 , F0 ) are all nonempty.
In order to prove that flows (σ0 , f ), f = 1, . . . , F0 , are delay unstable for the particular
choice of arrival rates, it suffices to show that
1 − (K + 1) > µσ0 xF0 .
(5.33)
If the above condition is satisfied, then the associated queues become positive along the FMS
from initial condition one for queue (σ0 , 0) and zero for all other queues, and Theorem 5.1
implies their delay instability.
Eqs. (5.31)-(5.32) and some simple algebra imply that
F0
K
X
X
µσ0 xF0 1 + K(F0 + 1) = 1 + K
λσf 0 −
max λσf k .
f =0
137
k=1
f =1,...,Fk
So, for the particular choice of arrival rates,
µσ0 xF0 1 + K(F0 + 1) = 1 + KF0 1 − (K + 1) .
Thus, Eq. (5.33) is equivalent to
1 − (K + 1) 1 + K(F0 + 1) > 1 + KF0 1 − (K + 1) ,
and, in turn,
K − K 2 + KF0 + 2K + 1 > 0,
which holds if > 0 is sufficiently small.
5.7
Concluding Remarks
This chapter built on, and extended the results of, Chapter 4. More specifically, we studied
a single-hop switched queueing network with a mix of heavy-tailed and exponential-type
traffic, and carried out a delay stability analysis of the Max-Weight policy. Our goal was to
showcase the use of fluid approximations in showing both delay instability (combined with
renewal theory) and delay stability (combined with stochastic Lyapunov theory). Moreover,
we applied these results to get a sharp characterization of the delay stability regions of the
Max-Weight policy in switched networks with disjoint schedules, generalizing the findings of
Chapter 4.
We conclude the chapter with some brief remarks.
The use of fluid approximations in delay analysis of queueing systems with heavy-tailed
traffic is not new. For example, the work of Baccelli et al. [5] has used fluid models to
determine the precise tail asymptotics of the steady-state maximal dater (i.e., the time to
clear all customers present at time t, assuming arrivals are stopped from that point on, in
the limit as t goes to infinity) of Generalized Jackson Networks with subexponential service
138
times. The tail asymptotics are determined through a sample path construction of the
maximal dater, which preserves crucial monotonicity properties of Jackson Networks. In
contrast, our approach is based on renewal theory, which on one hand does not provide as
refined results (moment bounds instead of the precise asymptotics), but on the other hand
does not rely on any special structure, besides the regenerative property, making it easier to
apply.
Theorems 5.1 and 5.2 are stated and proved in the context of a single-hop switched
queueing network under the Max-Weight policy. However, a closer look at the proofs of
these results reveals that the properties that we are really leveraging are:
(i) the existence of a fluid limit;
(ii) the uniqueness of the fluid model solution;
(iii) the existence of a suitable Lyapunov function for the fluid model.
Thus, Theorems 5.1 and 5.2 can be easily extended to any Markovian queueing system
for which properties (i)-(iii) hold. Regarding properties (i) and (ii), which would allow
the generalization of Theorem 5.1, one could potentially take advantage of the extensive
literature on fluid approximations that has developed over the last 20 years; see [12] and
the references therein. Proving property (iii), though, brings up several questions: given a
queueing system, is a there a systematic and efficient algorithm for constructing a Lyapunov
function? And even if there exists such an algorithm, can we ensure that the Lyapunov
function is piecewise linear and nonincreasing in the lengths of the heavy-tailed queues? The
undecidability results in [24] suggest that the answers to the above questions could very well
be negative in the general case. So, in order to extend Theorem 5.2, one would probably
have to restrict to systems with special structure.
In Theorem 5.4 we showed that, in every network with disjoint schedules that receives
heavy-tailed traffic and operates under the Max-Weight policy, there exists an arrival rate
vector in the corresponding stability region for which all flows are delay unstable. We conjecture that the same is true for every single-hop switched queueing network, i.e., with, possibly,
overlapping schedules. Moreover, a closer look at the proof of Theorem 5.4 reveals that the
139
rate vectors of interest are always close to certain boundaries of the stability regions, suggesting “heavily loaded” systems. Thus, at a conceptual level, our result is in sharp contrast
to the asymptotic optimality of the Max-Weight policy in the heavy-traffic regime [57]. The
latter result, of course, concerns the diffusion-scaled queue-length processes, whereas our
findings apply to the unscaled processes.
Finally, we conjecture that, for the Max-Weight policy, the BI algorithm identifies all
delay unstable traffic flows in the network, except, possibly, for the case of arrival rate
vectors on the boundaries of delay stability regions. A closer look at the proof of Theorem
5.3 reveals that this conjecture is true for networks with disjoint schedules. The general case
is expected to be more challenging, since certain monotonicity properties of networks with
disjoint schedules no longer hold. However, affirmative resolution of this conjecture would
reduce the problem of delay stability analysis to solving (perhaps numerically) a system of
ordinary differential equations from certain initial conditions.
Appendix 5.1 - Proof of Lemma 5.2
Fix time T > 0, initial condition v(0), arrival rate vector λv , and let v(·) be the queue-length
part of the FMS from v(0) on the interval [0, T ]. At any regular time t ∈ [0, T ], this solution
satisfies
v̇f (t) = λvf −
X
ṡvπ (t)πf + ẏfv (t),
(5.34)
π∈S
vf (t) > 0 =⇒ ẏfv (t) = 0,
nX
o
X
vf (t)πf < max
vf (t)σf =⇒ ṡvπ (t) = 0.
f ∈F
σ∈S
(5.35)
(5.36)
f ∈F
Also, let w(·) be the queue-length part of the FMS from initial condition w(0) on the
interval [0, T ], under arrival rate vector λw . Similarly, at any regular time t ∈ [0, T ], this
140
solution satisfies
ẇf (t) = λw
f −
X
w
ṡw
π (t)πf + ẏf (t),
(5.37)
π∈S
wf (t) > 0 =⇒ ẏfw (t) = 0,
nX
o
X
wf (t)πf < max
wf (t)σf =⇒ ṡw
π (t) = 0.
σ∈S
f ∈F
(5.38)
(5.39)
f ∈F
We measure the distance between the queue-length parts of the two solutions with the
square of the Euclidean norm of their difference:
X
2
v(t) − w(t)2 =
vf (t) − wf (t) .
2
f ∈F
Hence, at any regular time t ∈ [0, T ],
X
X
X
X
d
v(t) − w(t)2 = 2
wf (t)v̇f (t).
v
(t)
ẇ
(t)
−
2
w
(t)
ẇ
(t)
−
2
v
(t)
v̇
(t)
+
2
f
f
f
f
f
f
2
dt
f ∈F
f ∈F
f ∈F
f ∈F
(5.40)
Eq. (5.35) implies that vf (t)ẏf (t) = 0, so that Eq. (5.34) implies that
X
vf (t)v̇f (t) =
X
vf (t)λvf −
vf (t)
f ∈F
f ∈F
f ∈F
X
X
ṡvπ (t)πf .
(5.41)
π∈S
Similarly, Eqs. (5.37) and (5.38) imply that
X
wf (t)ẇf (t) =
f ∈F
X
wf (t)λw
f −
f ∈F
X
wf (t)
f ∈F
X
ṡw
π (t)πf .
(5.42)
π∈S
Moreover, from Eqs. (5.34) and (5.37), we have that
X
f ∈F
vf (t)ẇf (t) =
X
f ∈F
vf (t)λw
f −
X
vf (t)
f ∈F
X
π∈S
141
ṡw
π (t)πf +
X
f ∈F
vf (t)ẏfw (t),
(5.43)
and
X
f ∈F
wf (t)v̇f (t) =
X
wf (t)λvf −
f ∈F
X
wf (t)
f ∈F
X
ṡvπ (t)πf +
X
wf (t)ẏfv (t).
(5.44)
f ∈F
π∈S
Eqs. (5.40)-(5.44) imply that
X
X
d
v
w
v
v(t) − w(t)2 =2
v
(t)
λ
−
λ
+
2
wf (t) λw
f
f
f
f − λf
2
dt
f ∈F
f ∈F
X
X
X
X
−2
(t)π
vf (t)
ṡvπ (t)πf −
vf (t)
ṡw
f
π
f ∈F
f ∈F
π∈S
π∈S
X
X
X
X
v
−2
wf (t)
ṡw
(t)π
−
w
(t)
ṡ
(t)π
f
f
f
π
π
f ∈F
−2
X
f ∈F
π∈S
vf (t)ẏfw (t) − 2
X
π∈S
wf (t)ẏfv (t).
(5.45)
f ∈F
f ∈F
Eqs. (5.36), (5.39), and (5.45), combined with the fact that the functions yfv (·) and yfw (·)
are nondecreasing, for all f ∈ F, imply that for any regular time t ∈ [0, T ],
X
d
v(t) − w(t)2 ≤ 2
λvf − λw
vf (t) − wf (t) .
f
2
dt
f ∈F
v
X
vf (t) − wf (t)
≤ 2λ − λw ∞
f ∈F
2
≤ 2λv − λw ∞ v(t) − w(t)2 + 1 .
Finally, Gronwall’s inequality and the fact that v(·) and w(·) are differentiable almost
everywhere imply that, for every t ∈ [0, T ],
v
v(t) − w(t)2 ≤ v(0) − w(0)2 exp 2tλv − λw λ − λw .
+
2t
2
2
∞
∞
(5.46)
If v(0) = w(0) and λv = λw , so that v(·) and w(·) represent two FMS for a given initial
condition and arrival rate vector, then Eq. (5.46) implies that
v(t) − w(t)2 = 0,
2
142
∀t ∈ [0, T ],
resulting in the uniqueness of the queue-length part of the FMS. The continuity with respect
to the initial condition and the arrival rate vector follows directly from Eq. (5.46).
Appendix 5.2 - Service Rates under the Max-Weight
Policy
In this appendix we consider the setting of Section 5.6, i.e., a single-hop switched queueing
network with disjoint schedules under the Max-Weight policy. We analyze the FM of this
system and prove certain monotonicity properties in the service rates of schedules, which
turn out to be useful in proving Theorem 5.3.
We use the standard notation of ej for a vector whose j th element is equal to one, whereas
all other elements are equal to zero. The length of this vector will be clear from the context.
Let x = xσ0 , xσ1 , . . . , xσK be a generic configuration. For reasons that become apparent
in Section 5.6, we are interested in the service rate of schedule σ0 under different configurations, so we assume that the vector xσ0 is nonzero. Moreover, for concreteness, we assume
that all vectors xσk , k = 1, . . . , K, are nonzero, i.e., all schedules have maximum weight
and are drained simultaneously. The proofs of Lemmas 5.3, 5.4, and 5.5 that follow can be
easily modified to accomodate the case where some of the schedules σ1 , . . . , σK do not have
maximum weight.
We also consider three slight variations of x:
(i) configuration x̄ differs from x only in the fact that it includes an additional “lower
rate” nonempty queue in a maximum weight schedule. Without loss of generality, suppose
that schedule σ1 has maximum weight at time t, and it is the one that includes the addi
tional queue. In mathematical terms, x̄ = xσ0 , x̄σ1 , . . . , xσK , where x̄σ1 = xσ1 + ej , with
argmaxf ∈{1,...,F1 } xσf 1 > 0 < j ≤ F1 ;
(ii) configuration x̂ differs from x only in the fact that it includes an additional “lower
rate” nonempty queue in schedule σ0 . In mathematical terms, x̂ = x̂σ0 , xσ1 , . . . , xσK , where
x̂σ0 = xσ0 + ej 0 , with argmaxf ∈{1,...,F0 } xσf 0 > 0 < j 0 ≤ F0 ;
143
(iii) configuration x̃ differs from x only in the fact that one of the nonempty queues of
a maximum weight schedule has been substituted by another nonempty queue of the same
schedule that has lower arrival rate. Without loss of generality, suppose that schedule σ1
1
has maximum weight at time t, and xσj 1 = 1, xσj+1
= 0, for some j ∈ {1, . . . , F1 − 1}. Then,
1
x̃σj 1 = 0 and x̃σj+1
= 1, whereas x̃σf k = xσf k , for every other queue and schedule.
We denote by µσk (x), µσk (x̄), µσk (x̂), µσk (x̃) the service rates of schedule σk in the FM
and under configurations x, x̄, x̂, and x̃, respectively.
Lemma 5.3: The service rates of schedule σ0 under configurations x, x̄, and x̂ are ordered
in the following way:
µσ0 (x̂) < µσ0 (x) < µσ0 (x̄).
Proof. Eq. (5.31)-(5.32) and some simple algebra imply that the service rate of schedule σ0
under configuration x satisfies the following equality:
Fk
F0
K
K
X
X
X
|xσ0 | 1 X
σk σk
σ0 σ0
µσ0 (x) 1 +
=
1
+
λ
x
−
λ
x
.
f
f
f
f
σk |
σk |
|x
|x
k=1
k=1
f =0
f =1
(5.47)
A similar derivation can show that the service rates of schedule σ0 under configurations
x̄ and x̂ satisfy
F
K
F
0
1
X
X |xσ0 | X
1
|xσ0 |
σ1
σ0 σ0
σ1 σ1
µ (x̄) 1 + σ1
+
= 1 + σ1
λ x −
λf xf − λj
|x | + 1 k=2 |xσk |
|x | + 1 f =0 f f
f =1
σ0
Fk
F0
K
X
X
1 X
σk σk
σ0 σ0
λ
x
−
λ
x
,
f
f
f
f
σk |
|x
k=2
f =0
f =1
(5.48)
Fk
F0
K
K
X
X
X
|xσ0 | + 1 1 X
σk σk
σ0 σ0
σ0
µ (x̂) 1 +
=
1
+
λ
x
−
λ
x
+
λ
,
0
f
f
f
f
j
σk |
σk |
|x
|x
k=1
f =0
f =1
k=1
(5.49)
+
and
σ0
respectively.
144
We first show that µσ0 (x) < µσ0 (x̄). For notational convenience we define the quantities
A=1+
K
X
|xσ0 |
k=2
B =1+
|xσk |
,
Fk
F0
K
X
X
1 X
σk σk
σ0 σ0
,
x
−
λ
x
λ
f
f
f
f
σk |
|x
f =1
f =0
k=2
and
σ1
C = B|x | +
F0
X
λσf 0 xσf 0
−
F1
X
λσf 1 xσf 1 .
f =1
f =0
Then, Eq. (5.47) and (5.48) can be written as follows:
µσ0 (x) A|xσ1 | + |xσ0 | = C,
µσ0 (x̄) A|xσ1 | + A + |xσ0 | = C + B − λσj 1 .
The above imply that µσ0 (x) < µσ0 (x̄) is equivalent to
AB|xσ1 | + B|xσ0 | − A|xσ1 |λσj 1 − |xσ0 |λσj 1 − CA > 0,
which, in turn, is equivalent to
σ0
B|x | − A|x
σ1
|λσj 1
− |x
σ0
|λσj 1
F0
F1
X
X
σ0 σ0
σ1 σ1
−A
λf xf −
λf xf > 0.
f =0
f =1
Substituting the expressions for A and B, we get
Fk
F0
F1
K
X
X
X
|xσ0 | X
σk σk
σ0 σ0
|x | −
λ f xf −
λf xf +
λσf 1 xσf 1
σk |
|x
f =1
f =0
f =1
k=2
σ0
F1
K
K
X
X
|xσ0 | X
|xσ0 | σ1 σ1
σ1
σ0
σ1
σ1
+
λ x − λj |x | + |x | + |x |
> 0.
|xσk | f =1 f f
|xσk |
k=2
k=2
By taking into account Eq. (5.28) and the fact that (σ1 , j) is a “lower rate” queue, in
145
order to prove that µσ0 (x) < µσ0 (x̄) it suffices to show that
K
X
λσ1 k − |xσ0 | max λσ0 0 , λσ1 0 + |xσ1 |λσj 1
|xσ0 | − |xσ0 |
k=2
+|x
σ1
|λσj 1
K
X
|xσ0 | k=2
|xσk |
−
λσj 1
σ0
σ1
σ1
|x | + |x | + |x |
K
X
|xσ0 | k=2
|xσk |
> 0,
or, equivalently,
K
X
−λσ1 k > 0.
|xσ0 | 1 − max λσ0 0 , λσ1 0 − λσj 1 −
k=2
The latter is true because of Eq. (5.29).
Now we show that µσ0 (x̂) < µσ0 (x). Again, for notational convenience we define the
quantities
0
A =1+
K
X
|xσ0 |
k=1
|xσk |
,
Fk
F0
K
X
X
1 X
σk σk
σ0 σ0
B =1+
λ x −
λf xf .
|xσk | f =0 f f
k=1
f =1
0
Then, Eq. (5.47) and (5.49) can be written as follows:
K
K
X
X
λσj 00
1 0
=B +
.
µ (x̂) A +
|xσk |
|xσk |
k=1
k=1
σ0
0
µσ0 (x)A0 = B 0 ,
The above imply that µσ0 (x̂) < µσ0 (x) is equivalent to
Fk
F0
K
K
K
K
X
X
X
X
X
λσj 00 1 1 X
|xσ0 | σk σk
σ0 σ0
1
+
λ
x
−
λ
x
−
1
+
> 0,
f
f
f
f
σk |
σk |
σk |
σk |
|x
|x
|x
|x
k=1
k=1
f =0
f =1
k=1
k=1
which, in turn, is equivalent to
Fk
F0
K
K
K
K
K
X
X
1 − λσj 00 X
1 X 1 X
1 X 1 X
σk σk
σ0 σ0
σ0 σ0
+
λ
x
−λ
λ
x
|x
|
−
> 0.
f
f
f
f
j0
σk |
σk |
σk |
σk |
σk |
|x
|x
|x
|x
|x
k=1
k=1
k=1
f =0
k=1
k=1
f =1
146
By taking into account Eq. (5.28), in order to show that µσ0 (x̂) < µσ0 (x) it suffices to
show that
F0
K
K
K
X
X
X
1 X
1 σk
σ0 σ0
σ0
σ0 σ0
1 − λj 0 −
λ1 +
λ x − λj 0 |x | .
|xσk |
|xσk | f =0 f f
k=1
k=1
k=1
The latter is true because of Eq. (5.29) and the fact that (σ0 , j 0 ) is a “lower rate” queue.
Lemma 5.4: The service rates of schedule σ0 under configurations x and x̃ are ordered
in the following way:
µσ0 (x) < µσ0 (x̃).
Proof. Eq. (5.31)-(5.32) and some simple algebra imply that the service rate of schedule σ0
under configuration x satisfies the following equality:
Fk
F0
F0
F1
K
K
X
X
X
X
1 X
1 X
|xσ0 | σk σk
σ0 σ0
σ0 σ0
σ1 σ1
=
1+
λ
x
−
λ
x
+
µ (x) 1+
λ
x
−
λ
x
.
|xσk |
|xσk | f =0 f f f =1 f f
|xσ1 | f =0 f f f =1 f f
k=2
k=1
σ0
By arguing similarly, and taking into account the fact that |xσk | = |x̃σk |, for all k ∈
{0, . . . , K}, we have that the service rate of schedule σ0 under configuration x̃ satisfies the
following equality:
Fk
F0
F0
F1
K
K
X
X
X
X
1 X
1 X
|xσ0 | σk σk
σ0 σ0
σ0 σ0
σ1 σ1
=
1+
µ (x̃) 1+
λ
x
−
λ
x
+
λ
x
−
λ
x̃
.
|xσk |
|xσk | f =0 f f f =1 f f
|xσ1 | f =0 f f f =1 f f
k=2
k=1
σ0
Let
A=1+
K
X
|xσ0 |
k=1
and
|xσk |
,
Fk
F0
F0
K
X
X
1 X
1 X
σk σk
σ0 σ0
B =1+
λf xf −
λ f xf + σ 1
λσf 0 xσf 0 .
σk |
|x
|x
|
k=2
f =0
f =1
f =0
Then, the service rates of schedule σ0 under configurations x and x̃, respectively, can be
147
written as follows:
F
1
1 X
µ (x) = B − σ1
λσf 1 xσf 1 /A,
|x | f =1
σ0
F
1
1 X
µ (x̃) = B − σ1
λσf 1 x̃σf 1 /A.
|x | f =1
σ0
Since configuration x̃ differs from x only in the fact that one of the nonempty queues of
schedule σ1 has been substituted by another nonempty queue of the same schedule that has
lower arrival rate, these equations imply directly that µσ0 (x) < µσ0 (x̃).
Finally, we consider the configuration y = y σ0 , e1 , e1 , . . . , e1 , which corresponds to a situation where only the highest rate queue from each of the schedules σ1 , . . . , σK is nonempty.
In contrast, we do not impose any restrictions on which queues are nonempty in schedule
σ0 . Let us also consider a slight variation of this configuration, ȳ, which differs from y only
in the fact that one of the highest rate queues of schedules σ1 , . . . , σK is empty. Without
loss of generality, suppose that it is the highest rate queue of schedule σ1 . In mathematical
terms, ȳ = y σ0 , 0, e1 , . . . , e1 .
We denote by µσk (y) and µσk (ȳ) the service rates of schedule σk in the FM and under
configurations y and ȳ, respectively.
Lemma 5.5: The service rates of schedule σ0 under configurations y and ȳ are ordered
in the following way:
µσ0 (y) < µσ0 (ȳ).
Proof. By arguing similarly to the proof of Lemma 5.3, we have that the service rates of the
various schedules under configuration y satisfy
F0
X
λσf 0 yfσ0 − |y σ0 |µσ0 (y) = λσ1 k − µσk (y),
f =0
and
K
X
µσk (y) = 1.
k=0
148
∀k ∈ {1, . . . , K},
The above equations and some simple algebra imply that
F0
K
X
X
σ0 σ0
λσ1 k .
λf yf −
µ (y) 1 + K|y | = 1 + K
σ0
σ0
(5.50)
k=1
f =0
Now, under configuration ȳ, we have that µσ1 (ȳ) = 0 while the rest of the service rates
are split in the Max-Weight fashion, i.e.,
F0
X
λσf 0 yfσ0 − |y σ0 |µσ0 (ȳ) = λσ1 k − µσk (ȳ),
∀k ∈ {2, . . . , K}.
f =0
Then, the work-conserving nature of the policy and some simple algebra imply that
F0
K
X
X
µσ0 (ȳ) 1 + (K − 1)|y σ0 | = 1 + (K − 1)
λσ1 k .
λσf 0 yfσ0 −
f =0
For notational convenience we define the quantities
A = 1 + K|y σ0 |,
and
B =1+K
F0
X
K
X
λσ1 k .
λσf 0 yfσ0 −
k=1
f =0
Then, Eq. (5.50) and (5.51) can be written as follows:
µσ0 (y)A = B,
and
F0
X
µσ0 (ȳ) A − |y σ0 | = B −
λσf 0 yfσ0 + λσ1 1 .
f =0
149
k=2
(5.51)
The above imply that µσ0 (y) < µσ0 (ȳ) is equivalent to
B|y σ0 | − A
F0
X
λσf 0 yfσ0 + Aλσ1 1 > 0,
f =0
which, in turn, is equivalent to
K
X
|y | 1 −
σ0
k=1
max
f ∈{1,...,Fk }
λσf k
F0
1 X
− σ0
λσf 0 yfσ0 + 1 + |y σ0 | λσ1 1 > 0.
|y | f =0
The latter is true because of Eq. (5.29) and the fact that
F
0
σ0 1 X
σ0 σ0
λ
y
≤
max
λf .
f
f
σ
f ∈{0,...,F0 }
|y 0 | f =0
Lemmas 5.3, 5.4, and 5.5 can be easily modified to accomodate the case where some
schedules may not have maximum weight: instead of adding over all k ∈ {1, . . . , K}, as
we did in the above proofs, we add over the set of maximum weight schedule k ∈ K(x), as
dictated by Eqs. (5.31) and (5.32).
150
Chapter 6
Delay Analysis of Back-Pressure
Policies under Heavy-Tailed Traffic
In this chapter we consider a multi-hop switched queueing network, i.e., a switched network
where packets, upon completion of their service at one queue, may join another queue.
Clearly, multi-hop networks offer a variety of modeling capabilities beyond their single-hop
counterparts, allowing them to capture the dynamics of multi-hop wireless networks and
networks of data switches. Our goal is to analyze the performance of the Back-Pressure
scheduling and routing policy, the natural extension of Max-Weight in a multi-hop setting,
in the presence of heavy-tailed traffic.
The motivating factors for the subsequent developments are the findings of Chapters 4
and 5. There, in the context of a single-hop network under the Max-Weight policy, we showed
that delay instability not only arises in heavy-tailed flows, but may propagate to light-tailed
flows through a “domino effect:” if a light-tailed flow conflicts with a heavy-tailed flow
then it becomes delay unstable; in turn, it may cause the flows that conflict with it to be
delay unstable, depending on their arrival rates. We also showed how the shortcomings of
Max-Weight can be alleviated by using a parameterized Max-Weight-α policy, provided that
the parameters are chosen suitably. It is only natural, thus, to analyze the more general
multi-hop setting, and investigate additional factors that may affect the performance of
151
Back-Pressure policies and/or new phenomena that may arise.
The main contributions of this chapter can be summarized as follows.
(i) Through simple examples, we provide insights into how the network topology, the
routing constraints, and the server/link capacities may affect the performance of the BackPressure policy in the presence of heavy-tailed traffic;
(ii) By extending the scope of Theorem 5.1 in Chapter 5, we propose an algorithmic
procedure that identifies delay unstable queues by solving the fluid model of the network
from certain initial conditions. This approach is of particular interest in cases of complex
multi-hop networks, where direct stochastic analysis is difficult and Monte Carlo methods
are very slow to converge, if they converge at all;
(iii) We show how one can guarantee the delay stability of all light-tailed flows in the
network by using a parameterized version of the Back-Pressure policy, with suitably chosen
parameters.
The remainder of the chapter is organized as follows. Section 6.1 includes a detailed
presentation of the queueing network considered. In Section 6.2 we argue that multi-hop
switched queueing networks fall within the general class of Stochastic Processing Networks;
certain technical lemmas that we use throughout the chapter, pertaining mostly to the
existence of fluid limits, follow directly from this mapping. In Section 6.3 we show, through
simple examples, which “system parameters” may affect the delay performance of the BackPressure policy, and in what way. In Section 6.4 we illustrate how fluid approximations can
be used for showing delay instability results in multi-hop networks. Section 6.5 contains
the analysis of the parameterized Back-Pressure-α policy, and of the performance that it
achieves in terms of delay stability. We conclude with a brief discussion of the findings of
this chapter in Section 6.6.
152
6.1
A Multi-Hop Network under the Back-Pressure
Policy
In this section we give a detailed description of the multi-hop switched queueing network
considered, we present the Back-Pressure scheduling and routing policy, and we provide some
preliminary delay stability results.
In contrast to the single-hop network models considered up to this point, the physical or
virtual topology of the underlying system is much more important in a multi-hop setting.
The network topology is captured by a directed graph G = N , L , where N is the set
of nodes and L is the set of directed edges, which, driven by our special interest in data
communication networks, we call links. Nodes represent the physical or virtual locations
where traffic is buffered before transmission, and links provide the means of transmission.
In a queueing context, nodes capture the locations of queues, while each link is, essentially,
a server of the system, dedicated to carrying traffic from queues in its source node to queues
in its destination node. With few exceptions, we use variables i and j to represent nodes,
and (i, j) to denote a directed link from node i to node j.
A path on G is a sequence of directed links l1 , l2 , . . . , lK such that the destination node
of link lk is the source node of link lk+1 , for k = 1, . . . , K − 1.
Each traffic flow f ∈ F has a unique source node sf ∈ N where it enters the network,
and a unique destination node df ∈ N where it exits the network. Moreover, each traffic
flow f has a predetermined set of links Lf ⊂ L that it is allowed to access. For the model to
be interesting, we assume that sf 6= df and that there exists at least one path from sf to df
within the links in Lf . Furthermore, there exists a path from sf to the source node of every
link in Lf , consisting just of links in Lf . Similarly, there exists a path from the destination
node of every link in Lf to df , consisting just of links in Lf . Finally, we assume that the
set Lf includes no cyclic paths. In other words, the links in Lf and the corresponding nodes
define a connected Directed Acyclic Graph.
If the set Lf includes exactly one path from source to destination, then we say that flow
153
f has fixed routing. On the other hand, if there exist more than one source-destination
paths we say that flow f has dynamic routing. Finally, if Lf = L we have the case of
unconstrained routing.
Node i belongs to the set Nf if there exists a path from sf to i that includes only links
in Lf . Thus, Nf ⊂ N is the set of nodes that traffic flow f can access. Note that the source
node sf is trivially included in Nf , while the destination node df is included in Nf due to
our assumptions regarding Lf .
The queueing network operates in discrete time slots, which we index by t ∈ Z+ . Traffic
flow f maintains a queue in every node i ∈ Nf . We refer to this queue as queue (f, i) and
denote its length at the beginning of time slot t ∈ Z+ by Qf,i (t). We emphasize that queue
(f, i) buffers only packets of flow f .
Traffic may arrive to queue (f, i) either exogenously, if i is the source node sf , or endogenously, through a link in Lf whose destination node is i. We denote by Af (t) the number of
packets that arrive exogenously to queue (f, sf ) at the end of time slot t, and we refer to the
particular queue as the source queue of traffic flow f . Sf,i,j (t) represents the number of
packets that are scheduled for transmission from queue (f, i) through link (i, j) ∈ Lf . These
packets would serve as potential departures from queue (f, i) and potential arrivals to queue
(f, j), at time slot t.
Throughout the chapter we assume that all links can transmit packets simultaneously, and that all attempted transmissions are successful as long as there are packets to
be transmitted. Thus, our queueing model is suitable for wireline applications, but does
not capture the interference constraints that wireless networks typically exhibit. We discuss
more the implications of, and the motivation behind, this assumption in Section 6.6. For
simplicity, we also assume that the capacity of all links is equal to one packet per time slot.
Thus, Sf,i,j (t) ∈ {0, 1}, for all (i, j) ∈ Lf , for all f ∈ F.
We use the shorthand notation Q(t) for the set of queue lengths Qf,i (t); i ∈ Nf , f ∈ F ,
and S(t) for the set of scheduling decisions Sf,i,j (t); (i, j) ∈ Lf , f ∈ F , t ∈ Z+ .
In general, a queue-length based policy is a sequence of mappings from the history of
154
queue lengths Q(τ ); τ = 0, . . . , t to scheduling/routing decisions S(t), t ∈ Z+ . In this
chapter we focus on a particular stationary and Markovian queue-length based policy, namely
the Back-Pressure policy [59].
Fix arbitrary t ∈ Z+ and (i, j) ∈ L. For any F 0 ⊂
Procedure Π t, (i, j), F 0 as follows:
f ∈ F : (i, j) ∈ Lf
we define
(i) compute the maximum differential backlog
Wi,j (t) = max0
n
f ∈F
+ o
Qf,i (t) − Qf,j (t)]
;
(ii) if Wi,j (t) = 0 then set Sf,i,j (t) = 0, for all f ∈ F 0 ;
(iii) otherwise, pick a flow f ∗ with maximum differential backlog, i.e.,
f ∗ ∈ arg max0 Qf,i (t) − Qf,j (t) .
f ∈F
If the set on the right-hand side includes multiple flows, then pick f ∗ uniformly at random.
Then, set Sf ∗ ,i,j (t) = 1 and Sf,i,j (t) = 0, for all f ∈ F 0 \ {f ∗ }.
The Back-Pressure policy makes scheduling/routing decisions as follows. At the beginning of each time slot t ∈ Z+ set S(t) = 0. Then, go through all links in L in some
predetermined order. For each link (i, j) ∈ L:
(i) determine the set
n
F (t) = f ∈ F : (i, j) ∈ Lf ,
X
0
o
Sf,i,k (t) < Qf,i (t) ;
k6=j:(i,k)∈Lf
(ii) execute procedure Π t, (i, j), F 0 (t) .
The Back-Pressure policy described above goes through all links, in some order. At each
link, it schedules the transmission of a packet from a flow with maximum differential backlog,
provided (i) the available packets from the selected flow have not been already scheduled
for transmission through other links; (ii) the maximum differential backlog across the link is
155
strictly positive. If all available packets from all flows with maximum differential backlog are
already scheduled for transmission through other links, then a flow with non-maximum (but
still positive) differential backlog is selected. On the other hand, if the maximum differential
backlog across the link is nonpositive, then no packets are scheduled for transmission.
The order at which links are considered may, of course, have an impact on the way that
scheduling/routing decisions are made when queue lengths are small. However, our findings
do not depend on this order. Intuitively, this is because we are looking at a regime of large
queue lengths/delays, where the order at which links are considered no longer matters.
We note that the above description is slightly different than the original, and most studied, version of Back-Pressure [59]. The original policy does not keep track of how many
transmissions are scheduled for each queue, and defines the set F 0 (t) as
F 0 (t) = f ∈ F : (i, j) ∈ Lf ,
∀t ∈ Z+ .
It is not hard to see that on certain occasions, namely when queues have few packets to
transmit but many outgoing links, the original Back-Pressure policy may result in wasted
service opportunities.1 In contrast, our version of Back-Pressure guarantees that a service
opportunity is never wasted. This modification not only makes sense in practice, but also
simplifies the description of the system dynamics because we do not have to distinguish
between attempted and actual transmissions of packets.
With slight abuse of notation, if we now let S(t) represent the scheduling decisions made
by the Back-Pressure policy at time slot t, the dynamics of the multi-hop switched queueing
network can be written in the following form:
Qf,sf (t + 1) = Qf,sf (t) −
X
Sf,sf ,j (t) + Af (t),
∀f ∈ F,
(6.1)
j:(sf ,j)∈Lf
1
Here, a service opportunity is equivalent to the existence of a flow with positive differential backlog.
156
Qf,i (t + 1) = Qf,i (t) −
X
X
Sf,i,j (t) +
j:(i,j)∈Lf
∀i ∈ Nf \ {sf , df },
Sf,j,i (t),
∀f ∈ F.
j:(j,i)∈Lf
(6.2)
The initial lengths are arbitrary nonnegative integers. Finally, by convention,
Qf,df (t) = 0,
∀f ∈ F.
(6.3)
The reason for this is that packets exit the network as soon as they reach their destination.
Thus, Qf,df (·) is a mere reference point that we define for technical reasons, and no physical
or virtual queue needs to be maintained.
The Back-Pressure policy is the natural extension of Max-Weight in a multi-hop setting
and, similarly to Max-Weight, it has been shown to possess very good stability properties.
In order to formally characterize these properties in Section 6.2, we need to define first the
stability region of the multi-hop network.
Definition 6.1: (Stability Region) An arrival rate vector λ = λ1 , . . . , λF is in the
stability region Λ of the multi-hop switched queueing network described above if there exist
ζf,i,j ≥ 0, f ∈ F, i, j ∈ N , such that the following set of constraints is satisfied:
(i) Flow Efficiency Constraints:
ζf,i,i = ζf,i,sf = ζf,df ,i = 0,
∀i ∈ N ,
∀f ∈ F;
(ii) Routing Constraints:
ζf,i,j = 0,
∀(i, j) ∈
/ Lf ,
∀f ∈ F;
(iii) Flow Conservation Constraints:
X
j∈N
ζf,j,i + λf · 1{i=sf } =
X
ζf,i,j ,
j∈N
157
∀i 6= df ,
∀f ∈ F;
(iv) Link Capacity Constraints:
X
ζf,i,j < 1,
∀(i, j) ∈ L.
f ∈F
If an arrival rate vector is in the stability region, then there exists a policy that stabilizes
the network, in the sense of Definition 2.2. This can be shown by arguing similarly to
Corollary 3.9 of [27], and by utilizing the independence assumptions that we made regarding
the arrival processes in Chapter 2. Note that the stability region depends on the network
topology, the routing constraints, and the link capacities, but not on higher order statistics
of the arriving traffic, or on the specific policy being considered.
The performance metric that we employed so far was the delay stability of traffic flows,
i.e., whether the expected end-to-end delay of files in steady state is finite or not. In this
chapter we will also consider the delay stability of queues, i.e., whether the expected delay
experienced in a queue in steady state, including both queueing delay and time in service, is
finite or not. Clearly, in single-hop networks the two metrics are equivalent: a traffic flow is
delay stable if and only if the queue buffering the traffic of that flow is delay stable. However,
in multi-hop networks the situation could be more complicated. For example, a traffic flow
may be delay unstable while some queues of that flow are delay stable.
Throughout the chapter we denote by Qf,i and Df,i the steady-state length and delay
of queue (f, i), respectively, while we reserve Df for the end-to-end delay of traffic flow
f in steady state. The dependence of these quantities on the scheduling policy has been
suppressed from the notation, but will be clear from the context.
Lemma 6.1: Consider the multi-hop switched queueing network described above under
a stabilizing policy. If queue (f, i) is delay stable, for all i ∈ Nf , then traffic flow f ∈ F is
delay stable.
This result is a corollary of the linearity of expectations, combined with the fact that the
end-to-end delay of a packet is the sum of the delays along the packet’s route.
Lemma 6.2: Consider the multi-hop switched queueing network described above under
158
a stabilizing policy. Let f be a traffic flow with fixed routing. If queue (f, i) is delay unstable,
for some i ∈ Nf , then traffic flow f is delay unstable.
This result follows from the fact that the delay experienced in a queue bounds from below
the end-to-end delay of a file.
Similarly to Chapters 3 and 4, we start our analysis with a negative result regarding
heavy-tailed flows.
Theorem 6.1: (Delay Instability of Heavy Tails) Consider the multi-hop switched
queueing network described above under a regenerative policy. Every heavy-tailed traffic
flow is delay unstable.
Proof. Recall that under a regenerative scheduling policy, if one exists, the network is guaranteed to be stable. Following exactly the same steps as in the proof of Theorem 4.1 we
can show that the source queue of a heavy-tailed traffic flow is delay unstable. Thus, the
flow itself is delay unstable because the delay experienced in the source queue bounds from
below, sample path-wise, the end-to-end delay.
6.2
Switched Queueing Networks as Stochastic Processing Networks
Stochastic Processing Networks (SPNs) are a general class of queueing systems, aiming to
capture the dynamics and decisions in a wide range of settings in services and manufacturing.
Since their introduction in [33], several variations and extensions of the original formulation
have appeared in the literature. In this section we give a brief overview of SPNs, and we
show that the multi-hop switched queueing network and the Back-Pressure policy described
in Section 6.1 are special cases of the SPN and the Maximum Pressure policy studied in [17],
respectively. Through this mapping, certain technical lemmas that are needed for the delay
analysis of the Back-Pressure policy will follow directly from [17].
159
A SPN can be described in terms of four entities: buffers, jobs, processors, and activities.
Buffers correspond to our queues, jobs correspond to our packets, and processors correspond
to links in the multi-hop network of Section 6.1. SPNs also include a special buffer, termed
buffer 0, where all jobs waiting to enter the network are queued. However, what makes the
comparison between the two models a nontrivial task is the notion of activity, an equivalent
of which does not exist in a multi-hop network. An activity can simultaneously process jobs
from a set of buffers. In order to do this, it requires the simultaneous occupation of a set
of processors. Each activity has a certain processing time, upon the completion of which,
jobs depart from the associated buffers and may arrive at other buffers. Depending on the
availability of processors, multiple activities may be undertaken at the same time. In general,
there are two types of activities: input activities that process jobs only from buffer 0, and
service activities that process jobs only from the other buffers. Upon the completion of an
input activity, jobs depart from buffer 0 and arrive to certain buffers. Upon the completion of
a service activity, jobs depart from some buffers (but not buffer 0) and arrive to other buffers.
In the context of the multi-hop network of Section 6.1, an input activity is, essentially, an
exogenous arrival process, while a service activity is a queue-link allocation that satisfies the
routing constraints imposed by the sets Lf .
The paper [17] studies two variations of SPNs. The first assumes that the capacities of
processors are infinitely divisible, so that multiple activities can be undertaken at the same
time, at utilization level less than 100% at each one. The second variation assumes that
the capacities of servers are nondivisible, so that activites can be undertaken at utilization
level 100%, or not at all. Since a link can serve packets from only one queue at any given
time slot, the multi-hop network of Section 6.1 clearly falls within the class of SPNs with
nondivisible server capacities.
In the SPNs considered in [17], activities have general processing requirements and can
be preempted by other activities before their completion. The in-service jobs of a preempted
activity are “frozen,” and their service is resumed only when that activity is undertaken
again. In the discrete time model of Section 6.1, the processing requirement of all activities
160
is equal to one time slot. Moreover, the decision of which activities to undertake is made
at the beginning of each time slot. Thus, in our model activities are never preempted and
there are no “frozen” jobs.
Also, an important characteristic of the SPN in [17] is that, for an activity to be undertaken at any given point in time, there have to be jobs available for processing at each of
the constituent buffers. In other words, if a certain buffer is empty then activities processing
jobs from that buffer cannot be undertaken. In the language of multi-hop networks, a queue
is served only if it has packets available for transmission, which implies that there are no
wasted service opportunities.
With these correspondences, it can be verified that the multi-hop network of Section
6.1 is a special case of the SPN analyzed in [17]. Moreover, the variation of Back-Pressure
described in Section 6.1 does not waste service opportunities, and it is a Maximum Pressure
policy, i.e., it satisfies Eq. (7) of [17].
As a final remark, we note that Assumption 1 of [17] holds in switched networks, while
the static planning problem defined by Eqs. (24)-(27) of [17] is, essentially, the stability
region given in Definition 6.1.
As alluded to earlier, our main motivation for viewing multi-hop networks as SPNs is to
take advantage of known results from the latter literature, which will serve as intermediate
lemmas for the purposes of this chapter. One such result is the throughput optimality of
the Back-Pressure policy. This property was first proved in [59] for the original version of
Back-Pressure, assuming light-tailed traffic. The following lemma establishes the throughput
optimality of the slightly modified version of Back-Pressure introduced in Section 6.1, and
in the presence of heavy-tailed traffic.
Lemma 6.3: (Throughput Optimality of Back-Pressure) The multi-hop switched
queueing network described in Section 6.1 is stable under the Back-Pressure policy, for any
arrival rate vector in the stability region.
Proof. (Outline) It can be verified that, under the Back-Pressure policy and our inde
pendence assumptions on the arrival processes, the sequence Q(t); t ∈ Z+ is a time161
homogeneous, irreducible, and aperiodic Markov chain on a countable state space. The fact
that this Markov chain is also positive recurrent is implied by results in [17]. More specifically, Theorem 8 of [17] implies that the fluid model of the multi-hop network under the
Back-Pressure policy (described in Section 6.4) is weakly stable, i.e., if we consider the fluid
model solution with the queues being initially empty, then the queue-length part of the solution remains zero. Then, Theorem 3 of [17] implies that the stochastic system is pathwise
stable, i.e., the long-term departure rates are equal to the respective long-term arrival rates,
for all queues. Of course, this is a weaker form of stability compared to the one adopted in
this thesis. However, the fact that we have defined the stability region of the multi-hop network with strict inequalities for all link capacity constraints allows to strengthen this result
in a straightforward way, very similar to the way Theorem 5 in [17] strengthens Theorem 4
in [17]. In particular, it can be verified that the fluid model is stable, i.e., if we consider the
fluid model solution from a finite initial condition, then the queue-length part of the solution
becomes zero in finite time, and remains zero thereafter. Consequently, the Markov chain of
the stochastic system is positive recurrent due to Theorem 3.1 in [16]. Hence, Q(t); t ∈ Z+
converges in distribution, and its limiting distribution does not depend on Q(0).
Because of the latter fact, the sequence D(k); k ∈ N is a, possibly delayed, aperiodic
and positive recurrent regenerative process. Therefore, it also converges in distribution, and
its limiting distribution does not depend on Q(0); see [56].
6.3
Delay Stability Analysis of the Back-Pressure Policy - Examples
In this section we illustrate, through simple examples, the role of “system parameters”
such as the network topology, the routing constraints, and the link capacities on the delay
performance of the Back-Pressure policy, in the presence of heavy-tailed traffic. Our goal is
to derive insights into the type of systems where Back-Pressure is expected to perform well,
162
which can, then, be translated into network design principles.
All examples presented below include a heavy-tailed traffic flow, which we term flow 1,
and in most cases one or two more light-tailed flows, which we term flows 2 and 3. In the
corresponding figures, the heavy-tailed flow is always highlighted with red color, while the
light-tailed flows with blue color.
We will show that the links that are allowed to serve the source queue of a heavytailed flow play a prominent role in the analysis of the Back-Pressure policy. We call these
bottleneck links. In mathematical terms, if f ∈ F is a heavy-tailed traffic flow, the set of
bottleneck links associated with f is defined as
Bf = (sf , i) : (sf , i) ∈ Lf .
In the following figures, bottleneck links are represented with dashed arrows, while nonbottleneck links with solid arrows.
To illustrate the importance of bottleneck links, let us consider the queueing system of
Figure 6-1. Traffic arrives exogenously at node 1, gets buffered in the appropriate queue,
eventually gets transmitted through link (1, 0), and exits the network as soon as it reaches
node 0. Link (1, 0) is a bottleneck link, since it is allowed to serve the source queue of the
heavy-tailed flow 1. It is not hard to see that this model is equivalent to the single-server
system of two parallel queues in Figure 3-1. In this single-hop setting, the Back-Pressure
policy reduces to Max-Weight, and Proposition 3.6 states that the light-tailed traffic flow
2 is delay unstable. The main idea behind this result is that the source queue (1, 1) is,
occasionally, very large due to the heavy-tailed arrivals that it receives. During those periods,
flow 1 has very large differential backlog, which implies that, under the Back-Pressure policy,
queue (2, 1) is deprived of service unless it is already very long. Either way, queue (2, 1)
experiences large delays.
Summarizing, light-tailed traffic experiences large delays when passing through bottleneck links. Thus, the delay stability of light-tailed flows depends critically on their ability
to avoid bottlenecks, in either static or dynamic ways. As we show in the remainder of the
163
Figure 6-1: The single-server system of two parallel queues of Figure 3-1, cast as a multi-hop
network. Traffic flow 1 (red color) is heavy-tailed and traffic flow 2 (blue color) is light-tailed.
The Back-Pressure policy reduces to Max-Weight in this case, and the findings of Chapter
3 imply that the light-tailed flow is delay unstable.
section, this ability is dictated by “system parameters” such as the network topology, the
routing constraints, and the link capacities.
6.3.1
The Role of Network Topology
We start by illustrating the role of network topology in the delay stability of light-tailed flows
under the Back-Pressure policy. Consider the “line” network depicted in Figure 6-2. The
traffic of the heavy-tailed flow 1 arrives exogenously at node 1, eventually gets transmitted
through link (1, 0), and exits the network as soon as it reaches node 0. The traffic of the
light-tailed flow 2 arrives exogenously at node 2, eventually gets transmitted through link
(2, 1) first, and through link (1, 0) next, and, finally, exits the network when it reaches node
0. Both flows have fixed routing because the topology of a line network provides only one
source-destination path to each one. We are interested in the delay stability of flow 2 under
the Back-Pressure policy.
Proposition 6.1: Consider the queueing network of Figure 6-2 under the Back-Pressure
policy, with an arrival rate vector in the stability region. Traffic flow 2 is delay unstable.
Proposition 6.1 is a special case of Theorem 6.2, so a detailed proof is omitted. The
main idea behind this result is as follows. Since the traffic of both flows exits the network at
node 0 and, thus, Q1,0 (t) = Q2,0 (t) = 0, for all t ∈ Z+ , link (1, 0) is allocated according to
the Max-Weight policy between queues (1, 1) and (2, 1). Moreover, under the Back-Pressure
policy, packets of flow 2 are not transmitted through link (2, 1) whenever the length of queue
164
Figure 6-2: The heavy-tailed flow 1 (red color) enters the network at node 1 and exits
at node 0. The light-tailed flow 2 (blue color) enters the network at node 2, and passing
through node 1, it also exits the network at node 0. Traffic flow 2 is delay unstable under
the Back-Pressure policy because it has to pass through the bottleneck link (1, 0).
(2, 2) is less than or equal to the length of queue (2, 1). Thus, during the periods when
queue (1, 1) is very long due to the heavy-tailed arrivals that it receives, both queues (2, 2)
and (2, 1) build up to the same order of magnitude, which causes them to experience large
delays.
Notice that this argument suggests a result that is slightly stronger than the delay instability of flow 2, namely that both queues of flow 2 are delay unstable.
The reason that traffic flow 2 is delay unstable in Figure 6-2 is the topology of the
network, and specifically the fact that the only source-destination path of flow 2 passes
through a bottleneck link.
6.3.2
Routing Heavy-Tailed Flows
Next, we show how the routing constraints of heavy-tailed flows may affect the delay stability
of light-tailed flows under the Back-Pressure policy.
Consider the queueing network depicted in Figure 6-3. The traffic of the heavy-tailed
flow 1 arrives exogenously at node 1, and may reach its destination node 0 through the path
(1, 2), (2, 0) , or through the path (1, 3), (3, 0) . The same applies to the light-tailed flow
2. In other words, both flows have unconstrained routing. We are interested in the delay
stability of flow 2 under the Back-Pressure policy.
Proposition 6.2: Consider the queueing network of Figure 6-3 under the Back-Pressure
policy, with an arrival rate vector in the stability region. Traffic flow 2 is delay unstable.
165
Figure 6-3: Both the heavy-tailed flow 1 (red color) and the light-tailed flow 2 (blue color)
enter the network at node 1 and exit at node 0. They are both allowed to access all links of
the network. Traffic flow 2 is delay unstable under the Back-Pressure policy because it has
to pass through, either link (1, 2) or link (1, 3), which are both bottleneck links.
Proposition 6.2 is also a special case of Theorem 6.2, so a detailed proof is omitted. The
main idea behind this result is as follows. Since traffic flow 1 is heavy-tailed, queue (1, 1)
receives, occasionally, a very large batch of packets. This makes the differential backlog of
flow 1 over both links (1, 2) and (1, 3) simultaneously very large. Thus, for a long period of
time after the arrival of the batch, queue (2, 1) does not receive any service under the BackPressure policy unless it is already very long. Consequently, queue (2, 1) is delay unstable,
which implies that traffic flow 2 is delay unstable. This is because the delay experienced in
queue (2, 1) bounds from below the end-to-end delay of traffic flow 2.
The reason that traffic flow 2 is delay unstable in Figure 6-3 is the routing constraints of
the heavy-tailed flow 1, or, more accurately, the lack of constraints. By not restricting the
links that flow 1 is allowed to access, both links (1, 2) and (1, 3) become bottleneck links. In
turn, all feasible source-destination paths of flow 2 pass through a bottleneck link.
166
6.3.3
Routing Light-Tailed Flows
Finally, we demonstrate how the routing constraints of light-tailed flows may affect their
delay stability under the Back-Pressure policy.
First, consider the queueing network depicted in Figure 6-4, which is very similar to the
network of Figure 6-3 except for the fact that both flows are forced to reach their destination
through the path (1, 2), (2, 0) . In other words, both flows have fixed routing. Again, we
are interested in the delay stability of flow 2 under the Back-Pressure policy.
Figure 6-4: Both the heavy-tailed flow 1 (red color) and the light-tailed flow 2 (blue color)
enter the network at node 1 and exit at node 0. They are both restricted to reach their
destination through the path (1, 2), (2, 0) . Traffic flow 2 is delay unstable under the BackPressure policy because it has to pass through the bottleneck link (1, 2).
Proposition 6.3: Consider the queueing network of Figure 6-4 under the Back-Pressure
policy, with an arrival rate vector in the stability region. Traffic flow 2 is delay unstable.
Proposition 6.3 is a special case of Theorem 6.2, so a detailed proof is omitted. The
intuition behind this result is very similar to Proposition 6.2.
The insights derived from the simple examples of Figures 6-2, 6-3, and 6-4 can be unified
in a general result.
167
We say that traffic flow f ∈ F has to pass through a set of link L0 ⊂ L if every
packet arriving at queue (f, sf ) is transmitted, eventually, through one of the links in L0 .
So, whether a traffic flow has to pass through a set of links or not depends, in general,
on the network topology, on the routing constraints imposed by the set Lf , and on the
scheduling policy applied.
Theorem 6.2: Consider the multi-hop switched queueing network of Section 6.1 under
the Back-Pressure policy, with an arrival rate vector in the stability region. Let f ∈ F be
a light-tailed traffic flow. If there exists a heavy-tailed flow f 0 ∈ F such that f has to pass
through the bottleneck links Bf 0 , then f is delay unstable.
Proof. Without loss of generality, suppose that the network starts empty. We track the
evolution of the network along sample paths of the arrivals where
(i) Af 0 (0) = b, for sufficiently large b in the support of Af 0 (0);
(ii) Af (0) = 0, for all f 6= f 0 ;
P
(iii) tτ =1 Ag (τ ) − λg ≤ t + δ, ∀t ∈ N, ∀g ∈ F , for sufficiently small > 0 and some
δ > 0.
Let Hb be the set of these sample paths. The probability P Hb is bounded away from
zero because the arrival processes are mutually independent and IID over time slots.
At time slot zero, the differential backlog of flow f 0 over every link in Bf 0 is b, while
the differential backlog of flow f over any of those links is zero. Moreover, the differential
backlog of flow f 0 can decrease at rate no more than 2Bf 0 (since the capacity of all links
is equal to one), and the differential backlog of flow f can increase at rate no more than
λf + along the sample paths in Hb .
So, for sample paths in Hb , it can be verified that there exists b0 , k > 0 so that
Qf 0 ,i (t) − Qf 0 ,j (t) > Qf,i (t) − Qf,j (t),
∀t < kb,
∀b ≥ b0 ,
∀(i, j) ∈ Bf 0 .
Note that, for sufficiently large b, the impact of the constant δ becomes negligible; this
is why we have imposed the constraint b ≥ b0 , for some b0 .
168
Consequently, for sample paths in Hb , no packets of flow f are transmitted through any
of the link in Bf 0 under the Back-Pressure policy, during an order Ω(b) time period.
Now it is useful to keep track of the total number of packets of flow f between the source
node sf and the bottleneck node sf 0 , and to view them as one fictitious queue. Let us denote
the length of that queue at time slot t by Q̃f (t). The argument above implies that this queue
has arrivals at rate no less than λf − > 0 and no departures, during an order Ω(b) time
period. Hence, there exist constants c, c0 > 0 such that time slot cb ∈ Z+ is in the same busy
period as slot zero, and
Q̃f (cb) = c0 b,
∀b ≥ b0 .
Then, by arguing similarly to the proof of Theorem 4.1, we can show that the aggregate
length of this fictitious queue during a busy period is Ω b2 . This, in turn, proves that the
fictitious queue is delay unstable because b is drawn from a heavy-tailed distribution. This
also implies the delay instability of traffic flow f , since the delay experienced in the fictitious
queue bounds from below the end-to-end delay, sample path-wise.
Thus, if a light-tailed flow is forced through the bottleneck links of a heavy-tailed flow,
either because of the network topology or because of the routing constraints imposed a priori,
then the light-tailed flow experiences large delays under the Back-Pressure policy.
We emphasize that Theorem 6.2 holds for the bottleneck links of a specific heavy-tailed
flow. In other words, if f 0 and f 00 are both heavy-tailed flows and a light-tailed flow f has
to pass through Bf 0 ∪ Bf 00 , then it is not clear whether f is delay unstable. Intuitively, the
reason is that the probability that the source queues of the heavy-tailed flows receive very
large batches of packets “simultaneously” is small.
Now let us consider the queueing network of Figure 6-5, which is very similar to the
network of Figure 6-4 except for the fact that traffic flow 2 has unconstrained routing. Also,
we assume that λ1 , λ2 < 1. The importance of this assumption will become clear in the
next section. For the time being, suffice it to say that it allows flow 2 to route all its traffic
through path (1, 3), (3, 0) whenever the alternative path is congested.
169
Figure 6-5: Both the heavy-tailed flow 1 (red color) and the light-tailed flow 2 (blue color)
enter the network at node 1 and exit at node 0. Flow 1 is restricted to reach node 0 through
path (1, 2), (2, 0) , while flow 2 is unconstrained. The delay stability of traffic flow 2 under
the Back-Pressure policy depends on its arrival rate relative to the capacity of link (1, 3).
Proposition 6.4: Consider the queueing network of Figure 6-5 under the Back-Pressure
policy, with arrival rates that satisfy λ1 , λ2 < 1. Traffic flow 2 is delay stable.
Proof. Without loss of generality, we assume that all queues are empty at time slot zero.
The proof can be modified in a straightforward way to accomodate nonzero initial queue
lengths.
First, notice that no more than one packet per time slot arrives at nodes 2 and 3 because
that is the capacity of links (1, 2) and (1, 3). Moreover, traffic departs from each of these
nodes at rate one packet per time slot, as long as there are packets waiting for transmission.
This is due to the fact that both flows exit the network at node 0 so, whenever packets are
available, there is positive differential backlog over links (2, 0) and (3, 0). Therefore, it can
be easily verified that
Q2,i (t) ≤ 1,
∀t ∈ Z+ ,
∀i ∈ {2, 3}.
Furthermore, Lemma 6.3 implies that the queue-length processes Q2,2 (t); t ∈ Z+ and
170
Q2,3 (t); t ∈ Z+ converge to some limiting distributions Q2,2 and Q2,3 , respectively. Hence,
E Q2,i ≤ 1,
∀i ∈ {2, 3}.
Then, Little’s Law implies that both queues (2, 2) and (2, 3) are delay stable. In order to
show that flow 2 is delay stable, it suffices to show that queue (2, 1) is delay stable as well.
Link (1, 3) is allowed to transmit only packets of flow 2, and, as we showed above, the
length of queue (2, 3) is never more than one packet. Hence, under the Back-Pressure policy,
Q2,1 (t) > 1 =⇒ S2,1,3 (t) = 1,
∀t ∈ Z+ .
2
Consider the candidate Lyapunov function V (t) = Q2,1 (t) . Through simple algebra,
it can be verified that
E V (t + 1) − V (t); V (t) > 1 Ft
i
h
2
≤ − 2E S2,1,2 (t) + S2,1,3 (t) − A2 (t) Q2,1 (t); V (t) > 1 Ft + E A2 (t) + 2 ; V (t) > 1 Ft
i
h
2
≤ − 2E S2,1,3 (t) − A2 (t) Q2,1 (t); V (t) > 1 Ft + E A2 (t) + 2 ; V (t) > 1 Ft
h
2 i
= − 2 1 − λ2 Q2,1 (t) + E A2 (0) + 2
· 1{Q2,1 (t)>1} .
h
i
c
2
Notice that λ2 < 1, E A2 (0) < ∞, and Q2,1 (t) > 1 is a finite set. Then, Lemma 2.5
implies that
E Q2,1 < ∞.
Thus, all queues of flow 2 are delay stable, implying that traffic flow 2 is delay stable.
6.3.4
The Role of Link Capacities
In this section we illustrate the impact of link capacities on the delay stability of light-tailed
flows. In order to do this, we continue our analysis of the queueing network of Figure 6-5,
171
but this time consider the case of λ2 > 1. Of course, we assume that λ1 + λ2 < 2 so that the
network is stable under under the Back-Pressure policy.
It is intuitively clear that, irrespective of the specific routing decisions made at each time
slot, a nonvanishing fraction of the traffic of flow 2 has to pass through the bottleneck link
(1, 2). This fraction of the traffic experiences large delays, which implies that the delays of
flow 2 are, on average, large as well.
Proposition 6.5: Consider the queueing network of Figure 6-5 under the Back-Pressure
policy, with arrival rates that satisfy λ2 > 1 and λ1 + λ2 < 2. Traffic flow 2 is delay unstable.
Proof. The proof of this result follows an argument similar to the proof of Theorem 6.2.
Without loss of generality, we assume that the network starts empty, and we track the
evolution of the system along sample paths of the arrivals where
(i) A1 (0) = b, for sufficiently large b in the support of A1 (0);
(ii) A2 (0) = 0;
P
(iii) tτ =1 Af (τ ) − λf ≤ t + δ, ∀t ∈ N, i = 1, 2 , for sufficiently small > 0 and
some δ > 0.
Let Hb be the set of these sample paths. The probability P Hb is bounded away from
zero because the arrival processes are mutually independent and IID over time slots.
At time slot zero, the differential backlog of flow 1 over link (1, 2) is b, while the differential
backlog of flow 2 over the same link is zero. Moreover, the differential backlog of flow 1 can
reduce at rate no more than two since the capacity of link (1, 2) is equal to one , and the
differential backlog of flow 2 can increase at rate no more than λ2 + along the sample
paths in Hb .
So, for sample paths in Hb , it can be verified that there exists b0 , k > 0 so that
Q1,1 (t) − Q1,2 (t) > Q2,1 (t) − Q2,2 (t),
∀t < kb,
∀b ≥ b0 .
Consequently, throughout an order Ω(b) time period, link (1, 2) transmits no packets of
flow 2 under the Back-Pressure policy. Thus, during the same period of time and along
172
sample paths in Hb , queue (2, 1) has arrivals at rate λ2 − > 1 and departures at rate no
more than one, through link (1, 3). Hence, there exist constants c, c0 > 0 such that time slot
cb ∈ Z+ is in the same busy period as slot zero, and
Q2,1 (cb) = c0 b,
∀b ≥ b0 .
Then, by arguing similarly to the proof of Theorem 4.1, we can show that the aggregate
length of queue (2, 1) during a busy period is Ω b2 , which, in turn, proves the delay instability of queue (2, 1) because b is drawn from a heavy-tailed distribution. This also implies
the delay instability of traffic flow 2, since the delay experienced in queue (2, 1) bounds from
below the end-to-end delay.
6.3.5
The Impact of Heavy Tails on Cross-Traffic
So far we have looked at the delay stability of light-tailed flows that may have to compete
with heavy-tailed flows for network resources. We saw that this direct competition causes
large delays under the Back-Pressure policy. In this section we illustrate the potential impact
of heavy tails on “cross-traffic,” i.e., traffic flows that do not compete with heavy-tailed flows
directly, but do compete with light-tailed flows that themselves compete with heavy tails.
The latter flows are delay unstable, which may cause the former flows to be delay unstable
as well under the Back-Pressure policy. This phenomenon is reminiscent of the propagation
of delay instability through a “domino effect” that we observed in Chapter 4 under the
Max-Weight policy.
Consider the multi-hop network of Figure 6-6, which includes three traffic flows: the
heavy-tailed flow 1, and the light-tailed flows 2 and 3. The source of flow 1 is node 2,
whereas the source of flows 2 and 3 is node 1. The destination of flows 1 and 2 is node 3,
whereas the destination of flow 3 is node 4. All flows in the network have fixed routing.
More specifically, a packet of flow 1 exits the network as soon as it gets transmitted through
173
link (2, 3). A packet of flow 2 has to traverse link (1, 2) first, and link (2, 3) later. A packet
of flow 3 has to initially get through link (1, 2), and link (2, 4) after.
Figure 6-6: The heavy-tailed flow 1 (red color) enters the network at node 2 and exits at
node 3. The light-tailed flow 2 (blue color) enters the network at node 1 and exits at node 3.
The light-tailed flow 3 (blue color) enters the network at node 1 and exits at node 4. Traffic
flow 3 is delay unstable under the Back-Pressure policy if its arrival rate is sufficiently high.
Clearly, traffic flow 1 is delay unstable because it is heavy-tailed (see Theorem 6.1), and
traffic flow 2 is delay unstable because it has to pass through the bottleneck link (2, 3) (see
Theorem 6.2). So, the real question is the delay stability of flow 3, which serves as crosstraffic to flow 2. We will show that the cross-traffic may be affected indirectly by heavy tails,
if its arrival rate is sufficiently high.
Proposition 6.6: Consider the queueing network of Figure 6-6 under the Back-Pressure
policy, with an arrival rate vector in the stability region. If λ3 > 2 + λ1 − 2λ2 /3, then
traffic flow 3 is delay unstable.
Proof. (Outline) We track the evolution of the network along sample paths of the arrival
processes where (i) a busy period of the network starts with the source queue of the heavytailed flow receiving a large batch of size b packets; and (ii) from that point on, all arrival
processes exhibit their average behavior.
174
We distinguish between two phases in the evolution of the network.
In the first phase, the length of queue (1, 2) is greater than the length of queue (2, 2),
which implies that link (2, 3) transmits only packets of flow 1. Consequently, under the
Back-Pressure policy and along the sample paths of interest, queues (2, 1) and (2, 2) build
up together and at a constant rate throughout this phase. This phase terminates when
queues (1, 2) and (2, 2) have the same length, after Ω(b) time slots. At that point, queues
(1, 2), (2, 1), and (2, 2) are all Ω(b) long.
In the second phase, link (2, 3) transmits packets from both flows 1 and 2. This phase
lasts until one of the two queues empties. In order to build some intuition on how the
network behaves during this phase, let us suppose that both arrivals and departures are
fluids with constant rate. Let µf,i,j be the rate at which traffic of flow f is transmitted
through link (i, j). Of course, λf is the arrival rate at the source queue of flow f . The arrival
and departure rates satisfy the following linear system:
λ2 − µ2,1,2 − µ2,1,2 − µ2,2,3 = λ3 − µ3,1,2 ,
λ1 − µ1,2,3 = µ2,1,2 − µ2,2,3 ,
(6.4)
(6.5)
µ2,1,2 + µ3,1,2 = 1,
(6.6)
µ1,2,3 + µ2,2,3 = 1.
(6.7)
Eq. (6.4) is due to the fact that the Back-Pressure policy tries to keep the differential
backlogs of flows 2 and 3 over link (1, 2) the same. In order to achieve this, it determines
service rates for the various queues such that the differential backlogs of the link are drained
at the same rate. We note that queue (3, 2) remains zero throughout the second phase, so
that the rate of change of its length is also zero. Eq. (6.5) follows from a similar argument
for link (2, 3). Eqs. (6.6) and (6.7) result from the fact that the service rate of all links is
equal to one and Back-Pressure is a work-conserving policy.
175
Eqs. (6.4)-(6.7) and some simple algebra imply that
µ3,1,2 =
2 + λ1 − 2λ2 + 2λ3
.
5
Therefore,
λ3 > µ3,1,2 ⇐⇒ λ3 >
2 + λ1 − 2λ2
.
3
So, if λ3 > 2 + λ1 − 2λ2 /3 queue (3, 1) builds up at a constant rate for Ω(b) time
period, and eventually to a length of Ω(b). Thus, the integral of the length of that queue
over a busy period of the network becomes of Ω b2 . Because b is drawn from a heavy-tailed
distribution, it follows that queue (3, 1) is delay unstable. Finally, Lemma 6.2 implies that
traffic flow 3 is delay unstable as well.
This argument can be formalized by using fluid approximations, as demonstrated in
Section 6.4.
6.3.6
The Role of Intersecting Paths
In most of the examples presented above, delay instability was a consequence of the fact
that a light-tailed flow was forced into bottleneck links, in part or in its entirety. In the
previous example we observed that delay instability can also propagate through a different
mechanism, namely when a traffic flow competes for a link with a flow that has become
delay unstable because it has to pass through a bottleneck link. In the last example of this
section we illustrate yet another mechanism for the propagation of delay instability. We
look into the case where paths of a given heavy-tailed flow intersect before they reach their
destination, and we illustrate the impact that this has on the delay stability of queues under
the Back-Pressure policy.
More specifically, we consider the queueing network of Figure 6-7. Its topology is very
similar to that of the network in Figures 6-3. However, here we only have one traffic flow
present, the heavy-tailed flow 1, which has unconstrained routing. Packets arrive exogenously
176
to queue (1, 1), and can get to node 4 either through the path (1, 2), (2, 4) , or through the
path (1, 3), (3, 4) . After they reach node 4, though, they have to pass through the same
link in order to get to their destination. In that sense, the two alternative paths of flow 1
intersect.
Figure 6-7: The heavy-tailed flow 1 (red color) enters the network at node 1 and exits after
it gets transmitted from node 4. Flow 1 is allowed to access all links in the network. Queues
(1, 2), (1, 3), and (1, 4) are delay unstable under the Back-Pressure policy because the two
alternative paths of flow 1 intersect.
Theorem 6.1 implies that queue (1, 1) is delay unstable due to the heavy-tailed exogenous
arrivals that it receives. However, it provides no information regarding the other queues of
flow 1. Since all links have finite capacities, the endogenous arrivals to those queues are, by
definition, light-tailed. So, one could argue that the queues are delay stable, given that there
are no link activation constraints in the network, and no other traffic flows that compete for
service. Somewhat surprisingly, we show that the queues prior to the intersecting link are
also delay unstable. This is due to the dynamics induced by the Back-Pressure policy and
the fact that alternative paths intersect.
Proposition 6.7: Consider the queueing network of Figure 6-7 under the Back-Pressure
policy, with an arrival rate vector in the stability region. Queues (1, 2), (1, 3), and (1, 4) are
177
delay unstable.
Proof. Without loss of generality, suppose that a busy period of the network starts at time
slot zero. We track the evolution of the system along sample paths where A1 (0) = b, for
some sufficiently large b in the support of A1 (0).
First, we give a heuristic argument for the delay instability of queues (1, 2), (1, 3), and
(1, 4). Starting from time slot zero and throughout an order Ω(b) time period, flow 1
has positive differential backlog over both links (1, 2) and (1, 3). Thus, the set of queues
(1, 2), (1, 3), (1, 4) receives traffic at rate two packets per time slot during that time period. On the other hand, traffic departs from this set of queues at rate one packet per time
slot, which is the capacity of the outgoing link from node 4. So, the aggregate length of
this set of queues builds up at a constant rate over an order Ω(b) time period. Finally, the
Back-Pressure policy forces the queues to build up together so that, eventually, they all build
up to Ω(b). This can be translated to delay instability since b is drawn from a heavy-tailed
distribution.
For a more precise argument, one can explicitly construct the sample path of the network
after a big arrival (b packets) to the source queue of flow 1. In particular, it can be verified
that under the Back-Pressure policy
Q1,2 (3τ − 2) = Q1,3 (3τ − 2) = Q1,4 (3τ − 2) = τ,
τ = 2, 3, . . . , (b + 4)/7.
Hence, there exist constants c, c0 > 0 such that time slot cb ∈ Z+ is in the same busy
period as slot zero, and
Q1,2 (cb) = Q1,3 (cb) = Q1,4 (cb) = c0 b.
Then, by arguing similarly to the proof of Theorem 4.1, we can show that the aggregate
length of each of these queues during a busy period is Ω b2 , which, in turn, proves the delay
instability of queues (1, 2), (1, 3), and (1, 4).
178
6.4
Delay Stability Analysis via Fluid Approximations
The previous section demonstrated, through simple examples, that the delay stability of
queues and flows depends on a variety of “system parameters,” ranging from the network
topology and the routing constraints, to the link capacities and arrival rates. Moreover,
this dependence could be subtle, as exemplified by the network of Figure 6-6. Thus, the
following question arises naturally: how do we analyze the delay stability of complex multihop networks under the Back-Pressure policy? This section aims to answer this question, in
part, via the use of fluid approximations.
As explained already in Chapter 5, the fluid model is a deterministic dynamical system
that aims to capture the evolution of its stochastic counterpart on longer time scales. First,
we give a brief description and some useful properties of the fluid model of a multi-hop
network under the Back-Pressure policy. Then, we introduce the notion of fluid scaling, and
we establish a formal connection between the deterministic and the scaled stochastic system.
The Fluid Model (FM) of the multi-hop switched queueing network of Section 6.1
under the Back-Pressure policy is defined by the set of Eqs. (6.8)-(6.12), for every regular
time t ≥ 0:
q̇f,i (t) = −
X
ṡf,i,j (t) +
j:(i,j)∈Lf
X
ṡf,j,i (t) + λf · 1{i=sf } ,
∀i ∈ Nf ,
∀f ∈ F; (6.8)
j:(j,i)∈Lf
ṡf,i,j (t) ≥ 0,
∀(i, j) ∈ Lf ,
X
∃f 0 : qf 0 ,i (t) − qf 0 ,j (t) > 0 =⇒
∀f ∈ F;
ṡf,i,j (t) = 1,
(6.9)
∀(i, j) ∈ L;
(6.10)
f :(i,j)∈Lf
qf 0 ,i (t) − qf 0 ,j (t) <
n
+ o
max
qf,i (t) − qf,j (t)
=⇒
f :(i,j)∈Lf
ṡf 0 ,i,j (t) = 0,
179
∀f 0 : (i, j) ∈ Lf 0 ,
∀(i, j) ∈ L.
(6.11)
In the equations above, qf,i (t) represents the length of queue (f, i) at time t, and sf,i,j (t)
represents the total amount of time that link (i, j) ∈ Lf has been serving queue (f, i) up to
time t.
Our convention regarding zero queue lengths in destination nodes provides a final equation for the description of the FM:
qf,df (t) = 0,
∀f ∈ F,
∀t ∈ [0, T ].
(6.12)
Eqs. (6.8)-(6.11) are translated from the fluid model of the SPN in [17]. More specifically,
Eq. (6.8) follows from Eqs. (14)-(16) of [17], Eq. (6.9) follows from Eq. (18) of [17], Eq. (6.10)
follows from Eqs. (17) and (20) of [17], while Eq. (6.11) follows from Eq. (20) of [17]. Eq.
(6.11) is the natural analogue of the Back-Pressure policy in the fluid domain: traffic flows
that do not have maximum and nonnegative differential backlog receive no service.
We have assumed that all links can be activated simultaneously so, as long as they are
allocated in a work-conserving manner, a service opportunity is never wasted. Thus, unlike
the FM of Section 5.2, we do not need to keep track of the idleness at each queue.
In the description of the Back-Pressure policy in Section 6.1 we mentioned that links may
be considered in any predetermined order. We note that the FM of the multi-hop network
under the Back-Pressure policy, as captured by Eqs. (6.8)-(6.12), is the same irrespective of
this order.
Henceforth, we use the shorthand notation q(t) for the set of queue lengths qf,i (t); i ∈
Nf , f ∈ F , and s(t) for the set of scheduling decisions sf,i,j (t); (i, j) ∈ Lf , f ∈ F .
Fix arbitrary T > 0. A Fluid Model Solution (FMS) from initial condition q(0) = q
is a Lipschitz continuous function x(·) = q(·), s(·) that satisfies: (i) x(0) = (q, 0); (ii) Eqs.
(6.8)-(6.12) over the interval [0, T ].
A FMS is differentiable almost everywhere equivalently, almost every t ∈ [0, T ] is a
regular time , since it is Lipschitz continuous by assumption.
Similarly to Section 5.2, we first define the notion of fluid scaling and establish the exis
tence of a fluid limit and of a FMS. Consider a sequence of initial queue lengths Qb (0); b ∈
180
N for the queueing network of Section 6.1, and the corresponding sequence of queue-length
processes Qb (·); b ∈ N .
We define the “fluid-scaled” queue-length process as
q̃ b (t) =
Qb (bt)
,
b
t ∈ [0, T ],
b ∈ N.
We assume the existence of q = qf,i ∈ R+ , i ∈ Nf , f ∈ F , and of a sequence of positive
numbers b ; b ∈ N , converging to zero as b goes to infinity, that satisfy
b
(0) − qf,i ≤ b ,
max max q̃f,i
f ∈F
i∈Nf
∀b ∈ N.
We recall our standing assumption from Chapter 2 that there exists γ ∈ (0, 1) so that all
traffic flows have (1 + γ) moments. Fix some γ 0 ∈ (0, γ) and consider the sequence of sets of
sample paths of the arrival processes defined by
n
Hb = ω :
t−1
X
γ0 o
− 1+γ
1
sup
,
max Af (τ ) − λf < bT
1≤t≤bT bT f ∈F τ =0
b ∈ N.
Intuitively, Hb contains those sample paths of the arrival processes that stay close to their
average behavior over the time interval [0, bT ].
Lemma 6.4: (Existence of Fluid Limit and of FMS) There exists a Lipschitz
continuous function z(t) = zf,i (t); i ∈ Nf , f ∈ F , t ∈ [0, T ], such that for every > 0
there exists b0 () so that
P Hb ≥ 1 − ,
∀b ≥ b0 (),
and
b
sup max max q̃f,i
(t) − zf,i (t) ≤ ,
t∈[0,T ] f ∈F
i∈Nf
∀ω ∈ Hb ,
∀b ≥ b0 ().
Additionally, there exists a Lipschitz continuous function w(·), such that z(·), w(·) is a
FMS from initial condition q(0) = q over the interval [0, T ].
Proof. The fact that P(Hb ) converges to one as b goes to infinity is shown in Lemma 2.4.
181
The part of the result regarding the existence of a fluid limit, and that a fluid limit is a FMS,
follows directly from Appendix A of [17].
Lemma 6.5: (Uniqueness and Continuity of FMS): For any given q = qf,i ∈
R+ , i ∈ Nf , f ∈ F there exists a Lipschitz continuous function z(t) = zf,i (t); i ∈ Nf , f ∈
F , t ∈ [0, T ], such that the queue-length part of every FMS from initial condition q is
z(·). Moreover, z(·) depends continuously on both the initial condition q and the arrival rate
vector λ.
Proof. The existence of a FMS was established in Lemma 6.4. A proof of the uniqueness
and continuity part of the result is provided in Appendix 6.1.
We note that the above lemma does not guarantee the uniqueness of the FMS as a whole,
but only the uniqueness of the queue-length part. Namely, there may be multiple Lipschitz
continuous functions for the service part of the solution that satisfy the FM equations. In
fact, it can be verified that the FMS from zero initial condition is not unique, and depends
on the order that links are considered under the Back-Pressure policy.
Finally, we come to the more interesting part of the section, where we show how fluid
approximations can be used for proving delay instability results in multi-hop networks with
heavy-tailed traffic. Our contribution is summarized in the following result, which can be
viewed as an extension of Theorem 5.1 to a multi-hop setting.
Theorem 6.3: Consider the multi-hop switched queueing network of Section 6.1 under
the Back-Pressure policy, and its natural FM described above. Let h ∈ F be a heavy-tailed
traffic flow, and q ∗ (·) be the (unique) queue-length part of a FMS from initial condition
∗
∗
qh,s
(0) = 1 and zero for every other queue. If there exists τ ∈ [0, T ] such that qf,i
(τ ) > 0,
h
then queue (f, i) is delay unstable.
Proof. The proof of this result is very similar to the proof of Theorem 5.1 and, thus, omitted.
182
So, we can systematically look for delay instability through the following algorithmic
procedure.
INITIALIZATION: U = ∅
REPEAT
For every heavy-tailed traffic flow h ∈ F,
(i) solve the FM with initial condition one for queue (h, sh ), and zero for all other queues;
(ii) find the set of queues that become positive at any point before the FMS drains, Uh ;
(iii) set U = U ∪ Uh ;
END
Clearly, all queues included in U are delay unstable, which, in turn, can be used for
identifying delay unstable flows.
We illustrate the use of the above algorithmic procedure in the multi-hop network of
Figure 6-6, where we observed the phenomenon of rate-dependent delay instability of the
“cross-traffic” flow 3. We consider the set of arrival rates λ1 = 0.2, λ2 = 0.1, and λ3 = 0.8.
It can be easily verified that they are in the stability region, so that the network is stable
under the Back-Pressure policy. Figure 6-8 shows the FMS for the particular set of rates,
and with initial condition one for queue (1,2) and zero for all other queues. Notice that
the length of queue (3,1) becomes positive before the FMS drains, so Theorem 6.3 and
Lemma 6.2 imply that traffic flow 3 is delay unstable. This agrees with Proposition 6-6,
since λ3 > 1 + λ1 − 2λ2 /3 in this case.
6.5
The Back-Pressure-α Policy
The findings of Section 6.3 suggest that the propagation of delay instability could be mitigated by determining the routing constraints of the different flows in an appropriate way.
The same could be achieved by designing the network topology and the link capacities in a
suitable way, in case network design is an option. In other words, in Section 6.3 we tried to
183
Figure 6-8: The FMS of the multi-hop network of Figure 6-6 from initial condition one for
queue (1,2) and zero for the other queues, and arrival rates λ1 = 0.2, λ2 = 0.1, λ3 = 0.8.
answer the following question: given the policy (Back-Pressure), what is a good choice of
system parameters (routing constraints, network topology, link capacities)?
In this section we view the problem from a different angle, and attempt to answer an
“orthogonal” question: given the system parameters, what is a good policy? We will show
that, by modifying the Back-Pressure policy so that some form of partial priority is given to
light-tailed traffic, the propagation of delay instability can be mitigated, or even eliminated
completely. A policy that achieves this is the parameterized Back-Pressure-α policy, the
natural extension of Max-Weight-α to a multi-hop setting.
Fix arbitrary t ∈ Z+ , (i, j) ∈ L, and αf > 0, for every traffic flow f ∈ F. For any
F 0 ⊂ f ∈ F : (i, j) ∈ Lf we define Procedure Πα t, (i, j), F 0 as follows:
(i) compute the maximum α-weighted differential backlog
W̃i,j (t) = max0
f ∈F
nh
α
α
Qf,if (t) − Qf,jf (t)
(ii) if W̃i,j (t) = 0 then set Sf,i,j (t) = 0, for all f ∈ F 0 ;
184
i+ o
;
(iii) otherwise, pick a flow f ∗ with maximum α-weighted differential backlog, i.e.,
n
o
α
α
f ∗ ∈ arg max0 Qf,if (t) − Qf,jf (t) .
f ∈F
If the set on the right-hand side includes multiple flows, then pick f ∗ uniformly at random.
Then, set Sf ∗ ,i,j (t) = 1 and Sf,i,j (t) = 0, for all f ∈ F 0 \ {f ∗ }.
The Back-Pressure-α policy makes scheduling/routing decisions as follows. Fix arbitrary αf > 0, for every traffic flow f ∈ F. At the beginning of each time slot t ∈ Z+
set S(t) = 0. Then, go through all links in L in some predetermined order. For each link
(i, j) ∈ L:
(i) determine the set
n
F 0 (t) = f ∈ F : (i, j) ∈ Lf ,
X
o
Sf,i,k (t) < Qf,i (t) ;
k6=j:(i,k)∈Lf
(ii) execute procedure Πα t, (i, j), F 0 (t) .
Similar to the Back-Pressure policy described in Section 6.1, the order at which links are
considered plays a role on the actual scheduling/routing decisions made, but does not affect
our results regarding the delay stability of queues/flows.
The dynamics of the multi-hop network are still given by Eqs. (6.1)-(6.3), with the
understanding that the variables S(t) now represent the scheduling/routing decisions made
by the Back-Pressure-α policy at time slot t.
Theorem 6.4: Consider the multi-hop switched queueing network of Section 6.1 under
h
i
α +1
the Back-Pressure-α policy. If E Af f (0) is finite, for all f ∈ F, then the network is stable
and
XX h α i
E Qf,if < ∞.
f ∈F i∈Nf
Proof. Under the dynamics induced by the Back-Pressure-α policy the sequence Q(t); t ∈
Z+ is a time-homogenerous, irreducible, and aperiodic Markov chain on a countable state
185
space. We will show that this Markov chain is also positive recurrent, and we will obtain
moment bounds on the steady-state queue lengths, through drift analysis of the candidate
Lyapunov function
XX
V Q(t) =
f ∈F i∈Nf
1
α +1
Qf,if (t).
αf + 1
Note that this Lyapunov function, and, in general, the line of arguments that we follow
in this proof, are similar to the proof of Theorem 4.3 regarding the delay performance of the
Max-Weight-α scheduling policy.
Throughout the proof we use the shorthand notation
X
Tf,i (t) =
Sf,i,j (t)
j:(i,j)∈Lf
for the departures from queue (f, i), and
Rf,i (t) =
X
Sf,j,i (t) + Af (t) · 1{i=sf }
j:(j,i)∈Lf
for the arrivals at queue (f, i), at time slot t.
Moreover, we let
Vf,i Q(t) =
1
α +1
Qf,if (t).
αf + 1
The Lyapunov function can be written in the form
XX
V Q(t) =
Vi,f Q(t) ,
f ∈F i∈Nf
which implies that
XX E V Q(t + 1) − V Q(t) Ft =
E Vf,i Q(t + 1) − Vf,i Q(t) Ft .
(6.13)
f ∈F i∈Nf
We recall that Ft is the σ-algebra generated by Q(0), A(0), . . . , Q(t − 1), A(t − 1), Q(t),
and should be distinguished from the set of traffic flow F = {1, . . . , F }.
186
We will perform the drift analysis of function V (·) by upper-bounding the terms on the
right-hand side of Eq. (6.13).
Using the notation above and the dynamics of the multi-hop network, we have that
E Vf,i Q(t + 1) Ft =
α +1
1
Qf,i (t) + ∆f,i (t) f ,
αf + 1
(6.14)
where
∆f,i (t) = Rf,i (t) − Tf,i (t).
Now we distinguishing between three cases:
(i) if i = df then Eq. (6.3) implies that
E Vf,df Q(t + 1) − Vf,df Q(t) Ft = 0;
(6.15)
(ii) if i 6= df and αf < 1, then we consider the zeroth order Taylor expansion of the
right-hand side of Eq. (6.14) around Qf,i (t):
α +1
1
1
α +1
Q f (t) + ∆f,i (t) · ξ αf ,
Qf,i (t) + ∆f,i (t) f =
αf + 1
αf + 1 f,i
which implies that
E Vf,i Q(t + 1) Ft ≤ Vf,i Q(t) + E ∆f,i (t) · ξ αf Ft ,
for some ξ ∈ Qf,i (t) − Tf,i (t), Qf,i (t) + Rf,i (t) .
Consider the event Γf,i (t) : ∆f,i (t) ≤ 0 and its complement. The expression above can
be written in the form
h
α
E Vf,i Q(t + 1) Ft ≤Vf,i Q(t) + E ∆f,i (t) Qf,i (t) − Tf,i (t) f ; Γf,i (t)
i
h
α
+ E ∆f,i (t) Qf,i (t) + Rf,i (t) f ; Γcf,i (t) Ft .
187
i
Ft
Note that Qf,i (t), Rf,i (t), and Tf,i (t) are nonnegative integers, Tf,i (t) ≤ Qf,i (t), and
Tf,i (t) ≤ dmax , where dmax is the maximum number of outgoing edges of any node in G. It
can be verified that
Qf,i (t) + Rf,i (t)
αf
α
α
≤ Qf,if (t) + Rf,if (t),
and
α
αf
α
.
Qf,i (t) − Tf,i (t) f ≥ Qf,if (t) − dmax
Using these inequalities we can write
α
E Vf,i Q(t + 1) Ft ≤Vf,i Q(t) + E ∆f,i (t) Ft · Qf,if (t)
i
h
α
− E ∆f,i (t) · Tf,if (t); Γf,i (t) Ft
i
h
α
+ E ∆f,i (t) · Rf,if (t); Γcf,i (t) Ft ,
which implies that
i
h
α
αf +1
α +1
E Vf,i Q(t+1) Ft ≤ Vf,i Q(t) +E ∆f,i (t) Ft ·Qf,if (t)+dmax
+E Rf,if (t); Γcf,i (t) Ft .
h
i
α +1
Since E Af f (0) is finite and all arrivals processes are mutually independent and IID
i
h
αf +1
c
over time slots, E Rf,i (t); Γf,i (t) Ft is finite. Thus, there exists a finite constant cf,i
such that
α
E Vf,i Q(t + 1) − Vf,i Q(t) Ft ≤ E ∆f,i (t) Ft · Qf,if (t) + cf,i ;
(6.16)
(iii) if i 6= df and αf ≥ 1, then we consider the first order Taylor expansion of the
right-hand side of Eq. (6.14) around Qf,i (t):
αf +1
∆2f,i (t)
1
1
αf +1
αf
Qf,i (t) + ∆f,i (t)
=
Qf,i (t) + ∆f,i (t) · Qf,i (t) +
· αf · ξ αf −1 ,
αf + 1
αf + 1
2
188
which implies that
i
α
1 h
E Vf,i Q(t + 1) Ft ≤ Vf,i Q(t) + E ∆f,i (t) Ft · Qf,if (t) + E ∆2f,i (t) · αf · ξ αf −1 Ft ,
2
for some ξ ∈ Qf,i (t) − Tf,i (t), Qf,i (t) + Rf,i (t) .
Since αf ≥ 1, the last term can be bounded from above as follows:
i 1 h
α −1 i
1 h 2
E ∆f,i (t) · αf · ξ αf −1 Ft ≤ E ∆2f,i (t) · αf · Qf,i (t) + Rf,i (t) f Ft .
2
2
It can be verified that
α −1
α −1
α −1
Qf,i (t) + Rf,i (t) f ≤ 2αf −1 · Qf,if (t) + Rf,if (t) ,
and
2
∆2f,i (t) ≤ Rf,i
(t) + d2max .
Using these inequalities we can write
i
i
h
1 h 2
α −1
2
αf −2
2
αf −1 · αf · E Rf,i (t) Ft + dmax · Qf,if
E ∆f,i (t) · αf · ξ
Ft ≤2
2
i
h
h
α +1
α −1
+ 2αf −2 · αf · E Rf,if (t) Ft + d2max · E Rf,if (t)
i
Ft .
Thus, for every yf,i > 0 there exists a constant cf,i (yf,i ) such that
i
1 h 2
α
E ∆f,i (t) · αf · ξ αf −1 Ft ≤ yf,n · Qf,if (t) + cf,i (yf,i ).
2
Consequently,
α
α
E Vf,i Q(t + 1) − Vf,i Q(t) Ft ≤ E ∆f,i (t) Ft · Qf,if (t) + yf,n · Qf,if (t) + cf,i (yf,i ). (6.17)
Eqs. (6.13), (6.15), (6.16), and (6.17) imply that, for every δ > 0 there exist constants
189
cf,i (δ), i ∈ Nf , f ∈ F, such that
E V Q(t + 1) − V Q(t) Ft
hX X
αf
≤−E
Qf,i (t) ·
f ∈F i∈Nf
+
XX
X
Sf,i,j (t) −
j:(i,j)∈Lf
X
i
Sf,j,i (t) Ft
j:(j,i)∈Lf
α
Qf,if (t) · E Af (t); i = sf Ft
f ∈F i∈Nf
+δ
XX
α
Qf,if (t) +
f ∈F i∈Nf
XX
cf,i (δ).
(6.18)
f ∈F i∈Nf
Through simple algebra, we have that
hX X
α
E
Qf,if (t) ·
f ∈F i∈Nf
X
j:(i,j)∈Lf
Sf,i,j (t)−
i
Sf,j,i (t) Ft
X
j:(j,i)∈Lf
hX X
i
α
α
=E
Sf,i,j (t) · Qf,if (t) − Qf,jf (t) Ft
f ∈F (i,j)∈Lf
≥
X
W̃i,j (t) − c.
(6.19)
(i,j)∈L
The last inequality holds from the definition of the Back-Pressure-α policy. The constant
c accounts for the fact that on certain occasions, namely when the flow with maximum αweighted differential backlog has few packets compared to the number of outgoing links, a
flow with non-maximum α-weighted differential backlog may be scheduled. This can only
happen when the flow with maximum α-weighted differential backlog has less than dmax
αf P
packets, so that the total “loss” cannot be more than (i,j)∈L maxf :(i,j)∈Lf dmax
.
On the other hand,
XX
X
α
α
Qf,if (t) · E Af (t); i = sf Ft =
λf Qf,sf f (t).
f ∈F i∈Nf
(6.20)
f ∈F
Let Pf be the set of distinct source-destination paths of traffic flow f ∈ F. The fact
that the arrival rate vector λ is in the stability region of the network implies the existence
190
of constants ζf,p ≥ 0, p ∈ Pf , f ∈ F, such that
λf =
X
∀f ∈ F,
ζf,p ,
p∈Pf
ζf,i,j =
X
∀(i, j) ∈ Lf ,
ζf,p ,
∀f ∈ F,
p:(i,j)∈p
and
X
ζf,i,j ≤ 1 − ,
∀(i, j) ∈ L,
f :(i,j)∈Lf
for some > 0.
Thus,
X
α
λf Qf,sf f (t) =
f ∈F
XX
α
ζf,p Qf,sf f (t)
f ∈F p∈Pf
=
α
α
ζf,p Qf,if (t) − Qf,jf (t) ,
XX X
f ∈F p∈Pf (i,j)∈p
≤
X X
ζf,i,j W̃i,j (t)
f ∈F (i,j)∈Lf
=
X
X
ζf,i,j W̃i,j (t)
(i,j)∈L f :(i,j)∈Lf
X
≤ (1 − )
W̃i,j (t).
(6.21)
(i,j)∈L
Eqs. (6.18)-(6.21) imply that
XX α
XX
X
E V Q(t + 1) − V Q(t) Ft ≤ −
W̃i,j (t) + δ
Qf,if (t) +
cf,i (δ) + c.
f ∈F i∈Nf
(i,j)∈L
f ∈F i∈Nf
Finally, since Qf,df (t) = 0, the sum of the α-weighted differential backlogs along any
(directed and acyclic) source-destination path in Lf upper bounds the αf -powers of all the
queue lengths of flow f along that path. Hence, it can be verified that there exists 0 > 0
191
such that
XX
α
Qf,if (t) ≤ 0
f ∈F i∈Nf
X
W̃i,j (t).
(i,j)∈L
If δ is chosen sufficiently small, there exist constants γ > 0 and β < ∞ such that
XX α
E V Q(t + 1) − V Q(t) Ft ≤ −γ
Qf,if (t) + β.
f ∈F i∈Nf
Then, Lemma 2.5 implies that the queueing network is stable and that
P
f ∈F
P
i∈Nf
i
h
αf
E Qf,i
is finite.
A direct corollary of Theorem 6.4 relates to the delay stability of light-tailed flows.
Corollary 6.1: (Delay Stability under Back-Pressure-α) Consider the multi-hop
switched queueing network of Section 6.1 under the Back-Pressure-α policy. If the αparameters of all light-tailed flows are equal to one, and the α-parameters of heavy-tailed
flows are sufficiently small, then all light-tailed flows are delay stable.
Proof. We recall our standing assumption that all traffic flows have (1 + γ) moments, for
some γ > 0. Thus, with the suggested choice of α-parameters, Theorem 6.4 and Lemma 2.1
imply that every queue of every light-tailed flow is delay stable. Lemma 6.1 relates this to
the delay stability of light-tailed flows.
Combining this with Theorem 6.1, we conclude that the Back-Pressure-α policy is
optimal in terms of delay stability, provided the α-parameters are chosen suitably.
6.6
Concluding Remarks
The main objective of this chapter was to obtain some insights on system design and policy
design for multi-hop networks with with heavy-tailed traffic. First, we identified some “system parameters” that affect the delay performance of the Back-Pressure policy. Our analysis highlighted the significance of “bottleneck links,” i.e., links that are allowed to serve the
192
source queues of heavy-tailed traffic flows. The fundamental insight was that traffic flows that
have to pass through bottleneck links experience large delays under Back-Pressure. Thus,
we investigated reasons that may force a light-tailed flow to pass through a bottleneck link,
identifying the following: (i) the network topology, i.e., the source-destination paths that
the network offers to the given flow; (ii) the routing constraints, i.e., the a priori decisions
regarding which links the particular flow is allowed to traverse; (iii) the link capacities, i.e.,
whether the combined capacity of non-bottleneck paths is sufficient to support the arrival
rate of the flow.
The insights that we derived can be translated into system design principles. In particular, heavy-tailed flows should be relatively constrained on the links they are allowed to
access, whereas the network should provide multiple source-destination paths to light-tailed
flows; the latter flows should be left unconstrained to dynamically find their way around
heavy-tailed traffic. Moreover, these alternative paths should have enough capacity to support the rates of light-tailed traffic. In contrast, leaving heavy-tailed flows unconstrained
while forcing light-tailed flows to compete with them could be detrimental to the overall
performance of the network.
In terms of policy design, we proposed the parameterized Back-Pressure-α policy, and
showed that it can delay stabilize all light-tailed flows in the network, provided that its
α-parameters are chosen suitably. Similarly to the Max-Weight-α policy in Chapter 4, in
order to pick appropriate parameter values, some knowledge of the variability of the different
traffic flows is required.
Finally, apart from deriving insights, we also provided a method for carrying out delay
stability analysis through the use of fluid approximations. More specifically, we proposed a
procedure that identifies (some of the) delay unstable queues by solving the fluid model of
the network from certain initial conditions. The importance of this result lies in the fact
that it enables delay analysis, be it approximate or indirect, of complex multi-hop networks
where direct stochastic analysis is hard and Monte-Carlo methods are very slow to converge.
We conjecture that certain findings of Section 6.3 can be considerably strengthened. In
193
particular, Theorem 6.2 states that a light-tailed traffic flow that has to pass through the
bottleneck links of a heavy-tailed flow is delay unstable. We conjecture that, in fact, all the
queues that the light-tailed flow traverses prior to reaching the bottleneck links are delay
unstable. Moreover, Proposition 6.7 considers a simple multi-hop network where alternative
paths of a heavy-tailed flow intersect, and states that all queues between the source and
the intersecting link are delay unstable. We conjecture that this is a special case of a more
general result: in the context of the multi-hop network of Section 6.1, if the paths of a heavytailed flow intersect then all queues of that flow between the source and the intersecting link
are delay unstable. Combined with the findings of Proposition 6.6, this could imply ratedependent delay instability of the cross-traffic in this part of the network.
We conclude the chapter with some brief remarks that put our results in perspective.
The model that we considered is not the most general multi-hop switched queueing
network that one can encounter. For example, the literature on scheduling problems in
wireless networks considers, typically, systems that capture both flow scheduling and link
scheduling constraints and decisions; see [27] and the references therein. The model of this
chapter did capture the flow scheduling part but, by assuming that all links can be activated
simultaneously, essentially ignored the link scheduling part. The motivation behind this
choice was that the link/server scheduling problem was analyzed extensively in Chapters
4 and 5, so here we wanted to focus on phenomena and insights that are solely due to the
multi-hop nature of the model. Moreover, the simpler dynamics allowed for a cleaner analysis
and more crisp results.
A special case of the Back-Pressure-α policy proposed in this chapter has been considered
by Bui et al. [14]. In the latter work, all α-parameters take the same value, but this value
need not be one (so it is, indeed, a generalization of the original Back-Pressure policy). The
authors are motivated to study this policy by a still unresolved conjecture that the average
delay in an input-queued switch decreases as α goes to zero; see [36, 53]. We note that
their setting includes just light-tailed traffic, and, additionally, the existence of congestion
controllers. Thus, the insight that smaller parameter values should be used for heavy194
tailed flows so that light-tailed flows are given some form of priority, never appears in their
analysis. As a side comment, we note that Theorem 6.4, which characterizes the delay
stability performance of Back-Pressure-α, essentially generalizes Theorem 4.3 to a multi-hop
setting, the only caveat being that the model of Chapter 4 is, strictly speaking, not a special
case of the model considered here.
Finally, this chapter is not the first to report bad delay performance of the Back-Pressure
policy. Bui et al. [15] have reached a similar conclusion, but for different reasons. Their focus
is on the scaling of average delay with the number of links that a flow has to traverse. In
particular, they show that the average delay grows at least quadratically in the number of
links in line networks. In contrast, the cause of large delays in our case is the mere presence
of heavy-tailed traffic, and the significant impact that it may have on the network through
the Back-Pressure policy.
Appendix 6.1 - Proof of Lemma 6.5
The proof of Lemma 6.5 is along the lines of the proof of Lemma 5.2.
Fix time T > 0, initial condition v(0) = vf,i (0); i ∈ Nf , f ∈ F , arrival rate vector λv ,
and let v(·) be the queue-length part of the FMS from v(0) on the interval [0, T ], in vector
form. Eq. (6.8) implies that, at any regular time t ∈ [0, T ], this solution satisfies
v̇f,i (t) = −
X
ṡvf,i,j (t) +
X
ṡvf,j,i (t) + λvf · 1{i=sf } ,
∀i ∈ Nf ,
∀f ∈ F.
j:(j,i)∈Lf
j:(i,j)∈Lf
Also, let w(·) be the queue-length part of the FMS from initial condition w(0) on the
interval [0, T ], under arrival rate vector λw . Similarly, this solution satisfies
ẇf,i (t) = −
X
j:(i,j)∈Lf
ṡw
f,i,j (t) +
X
w
ṡw
f,j,i (t) + λf · 1{i=sf } ,
∀i ∈ Nf ,
∀f ∈ F.
j:(j,i)∈Lf
We measure the distance between the queue-length parts of the two solutions with the
195
square of the Euclidean norm of their difference:
XX
2
v(t) − w(t)2 =
v
(t)
−
w
(t)
.
f,i
f,i
2
f ∈F i∈Nf
At any regular time t ∈ [0, T ],
XX
XX
d
v(t) − w(t)2 =2
v
(t)
v̇
(t)
+
2
wf,i (t)ẇf,i (t)
f,i
f,i
2
dt
f ∈F i∈Nf
f ∈F i∈Nf
XX
XX
vf,i (t)ẇf,i (t) − 2
wf,i (t)v̇f,i (t).
−2
f ∈F i∈Nf
f ∈F i∈Nf
We have that
vf,i (t) · v̇f,i (t) = −vf,i (t) ·
X
X
ṡw
f,i,j (t) + wf,i (t) ·
j:(i,j)∈Lf
vf,i (t) · ẇf,i (t) = −vf,i (t) ·
X
X
X
w
ṡw
f,j,i (t) + wf,i (t) · λf · 1{i=sf } ,
j:(j,i)∈Lf
ṡw
f,i,j (t) + vf,i (t) ·
j:(i,j)∈Lf
wf,i (t) · v̇f,i (t) = −wf,i (t) ·
ṡvf,j,i (t) + vf,i (t) · λvf · 1{i=sf } ,
j:(j,i)∈Lf
j:(i,j)∈Lf
wf,i (t) · ẇf,i (t) = −wf,i (t) ·
X
ṡvf,i,j (t) + vf,i (t) ·
X
w
ṡw
f,j,i (t) + vf,i (t) · λf · 1{i=sf } ,
j:(j,i)∈Lf
ṡvf,i,j (t) + wf,i (t) ·
j:(i,j)∈Lf
X
ṡvf,j,i (t) + wf,i (t) · λvf · 1{i=sf } ,
j:(j,i)∈Lf
for all i ∈ Nf , and for all f ∈ F.
Therefore,
XX
d
v(t) − w(t)2 = 2
λvf − λw
vf,i (t) − wf,i (t) · 1{i=sf } + A + B,
f
2
dt
f ∈F i∈N
f
196
where
A=−2
XX
+2
XX
vf,i (t)
f ∈F i∈Nf
X
j:(i,j)∈Lf
vf,i (t)
f ∈F i∈Nf
X
ṡvf,j,i (t)
X
ṡvf,i,j (t) −
j:(j,i)∈Lf
ṡw
f,i,j (t)
X
−
ṡw
f,j,i (t)
,
ṡw
f,j,i (t)
j:(j,i)∈Lf
j:(i,j)∈Lf
and
B =−2
XX
+2
XX
wf,i (t)
X
f ∈F i∈Nf
j:(j,i)∈Lf
j:(i,j)∈Lf
wf,i (t)
X
f ∈F i∈Nf
X
ṡw
f,i,j (t) −
ṡvf,j,i (t) .
X
ṡvf,i,j (t) −
j:(j,i)∈Lf
j:(i,j)∈Lf
Now notice that, by rearranging the terms, we have that
XX
vf,i (t)
f ∈F i∈Nf
X
ṡvf,i,j (t)
−
j:(i,j)∈Lf
X
ṡvf,j,i (t)
=
X X
ṡvf,i,j (t)
vf,i (t) − vf,j (t) ,
f ∈F (i,j)∈Lf
j:(j,i)∈Lf
and
XX
wf,i (t)
f ∈F i∈Nf
X
ṡw
f,i,j (t)
−
j:(i,j)∈Lf
X
ṡw
f,j,i (t)
=
X X
ṡw
f,i,j (t)
wf,i (t) − wf,j (t) .
f ∈F (i,j)∈Lf
j:(j,i)∈Lf
The identities above, combined with Eqs. (6.10)-(6.11), imply that
XX
f ∈F i∈Nf
vf,i (t)
X
j:(i,j)∈Lf
ṡvf,i,j (t) −
X
ṡvf,j,i (t)
j:(j,i)∈Lf
≥
XX
f ∈F i∈Nf
197
vf,i (t)
X
j:(i,j)∈Lf
ṡw
f,i,j (t) −
X
j:(j,i)∈Lf
ṡw
(t)
,
f,j,i
and
XX
f ∈F i∈Nf
wf,i (t)
X
j:(i,j)∈Lf
ṡw
f,i,j (t) −
X
ṡw
f,j,i (t)
j:(j,i)∈Lf
≥
XX
wf,i (t)
f ∈F i∈Nf
X
ṡvf,i,j (t)
j:(i,j)∈Lf
X
−
ṡvf,j,i (t)
,
j:(j,i)∈Lf
so that both terms A and B are nonpositive.
Consequently,
XX
d
w
v
v(t) − w(t)2 ≤ 2
v
(t)
−
w
(t)
· 1{i=sf }
−
λ
λ
f,i
f,i
f
f
2
dt
f ∈F i∈Nf
v
XX
vf,i (t) − wf,i (t)
≤ 2λ − λw ∞
f ∈F i∈Nf
2
≤ 2λv − λw ∞ v(t) − w(t)2 + 1 .
Finally, Gronwall’s inequality and the fact that v(·) and w(·) are differentiable almost
everywhere imply that, for every t ∈ [0, T ],
v(t) − w(t)2 ≤ v(0) − w(0)2 exp 2tλv − λw + 2tλv − λw .
2
2
∞
∞
(6.22)
If v(0) = w(0) and λv = λw , so that v(·) and w(·) represent two solutions to the FM for
a given initial condition and arrival rate vector, then Eq. (6.22) implies that
v(t) − w(t)2 = 0,
2
∀t ∈ [0, T ],
resulting in the uniqueness of the queue-length part of the FMS. The continuity with respect
to the initial condition and arrival rate vector follows directly from Eq. (6.22).
198
Chapter 7
Discussion
In this thesis we studied switched queueing networks with a mix of heavy-tailed and lighttailed traffic, and we analyzed the delay performance of Max-Weight/Back-Pressure policies
in this setting, using as metric the notion of delay stability. We believe that our findings
contribute to the theory of the particular class of stochastic networks in a variety of ways,
from identifying and analyzing new phenomena, proposing new algorithms, to developing
new analytical methods.
The contributions of this thesis at a conceptual level are twofold. First, we showed
that the delay performance of the Max-Weight/Back-Pressure policy can be very poor in the
presence of heavy-tailed traffic, especially when the network is heavily loaded, i.e., for arrival
rate vectors close to the boundary of the stability region. This insight is in sharp contrast to
the asymptotic delay optimality of Max-Weight/Back-Pressure in the heavy-traffic regime
[18,57]. Of course, the latter results concern the diffusion-scaled delay processes, and hold in
the limit as the system becomes critically loaded. Instead, our results concern the original
(unscaled and underloaded) stochastic network.
Second, in contrast to the Max-Weight/Back-Pressure policy that treats all traffic flows
“equally,” we showed that giving priority to light-tailed traffic can improve the delay performance of the network, in particular in the regime of large delays. More importantly, we
showed how this can be done within the class of Max-Weight/Back-Pressure policies, which
199
possess the throughput optimality property and, thus, avoid dynamic instabilities that are
possible under buffer-priority policies, e.g., see [38, 49].
At the algorithmic level, we introduced the Bottleneck Identification algorithm, which
simplifies, to some extent, the delay stability analysis of the Max-Weight policy. This algorithms systematically tests for delay instability by solving the fluid model of the network
from certain initial conditions. We emphasize that the Bottleneck Identification algorithm
is not a stochastic simulation method, since the fluid model is a deterministic dynamical
system. Moreover, for all practical purposes, the fluid model need not be solved analytically;
a numerical solution of the associated ODEs is, typically, good enough. This algorithm
is particularly useful in complex networks, where direct stochastic analysis is difficult and
stochastic simulation methods are very slow to converge, if they converge at all.
Moreover, we proposed the parameterized Max-Weight-α/Back-Pressure-α policy and
proved that it is, not only throughput optimal, but also optimal with respect to the delay
stability metric, provided the α-parameters are chosen suitably. In other words, with proper
choice of parameters, the Max-Weight-α/Back-Pressure-α policy minimizes the number of
queues thet exhibit large delays. In general, smaller α-parameter values have to be selected
for heavy-tailed flows and larger values for light-tailed flows, so that some form of partial
priority is given to light-tailed traffic. The exact ranges that the parameter values are allowed
to take depend on higher order moments of the arrival processes.
Finally, on the methodological side, we showed how fluid approximations can be used
to simplify the delay stability analysis of queueing networks with heavy-tailed traffic. More
specifically, we showed how fluid approximations can be combined with renewal theory in
order to show delay instability results. Also, we showed how fluid approximations can be
combined with stochastic Lyapunov theory in order to show delay stability results. The
appeal of these results lies in the fact that they allow us to work directly on the (deterministic)
fluid domain, which is, typically, far easier than working on the stochastic domain.
Furthermore, we demonstrated the use of drift analysis of a particular class of Lyapunov
functions, as means to obtain exponential upper bounds on queue-length asymptotics, in
200
the presence of heavy-tailed traffic. These functions were piecewise linear and nonincreasing
in the lengths of the heavy-tailed queues. We note that, in related literature, similar exponential bounds have only been proved in very simple systems, e.g., two queues that receive
heavy-tailed and exponential-type traffic, respectively, and share a single server [8, 35]. In
these studies the bounds were obtained through sample path arguments, which are, typically, difficult to make under complicated dynamics. In contrast, we were able to apply
our technique to switched queueing networks with disjoint schedules under the Max-Weight
policy, a setting that is significantly more complex. Moreover, there is a solid body of work
on drift analysis of piecewise linear Lyapunov functions in fairly general queueing systems,
e.g., see [6, 19], which makes us hopeful that our method is of even wider applicability.
The results of this thesis were consistently presented in terms of delay stability, a rather
crude performance metric that attempts to capture the notion of large delays in a binary
manner. However, the majority of our results can be significantly refined. For example, if
we generalize the notion of a heavy-tailed flow to be one that has infinite (k + 1) moment of
arrivals, for some k ∈ N, then any light-tailed flow that conflicts with a heavy-tailed flow has
infinite k th moment of steady-state queue length under the Max-Weight policy. Moreover, all
delay stability results regarding networks with a mix of heavy-tailed and exponential-type
traffic were based on drift analysis of piecewise linear Lyapunov functions. This type of
analysis guarantees, not only the delay stability of light-tailed flows, but also exponential
upper bounds on the respective steady-state queue-length asymptotics.
We conclude the thesis with some suggested directions for future research.
Extending the scope and strengthening the theoretical guarantees of the Bottleneck Identification algorithm would be of particular theoretical and practical value. As stated in
Chapters 5 and 6, the Bottleneck Identification algorithm finds (some of the) delay unstable
queues in switched queueing networks under the Max-Weight/Back-Pressure policy. However, a closer look at the proofs of Theorems 5.1 and 6.3, on which the performance analysis
of the algorithm is based, reveals that the properties that we are really leveraging are: (i)
the existence of a fluid limit; (ii) the uniqueness of the fluid model solution. Therefore, the
201
scope of the algorithm can be extended to any (regenerative) queueing system, as long as
the above properties are satisfied. In order to show the latter, one could, potentially, take
advantage of the extensive literature on fluid approximations that has developed over the
last 20 years; see [12] and the references therein. Moreover, we conjecture that, for single-hop
networks under Max-Weight, the Bottleneck Identification algorithm finds all delay unstable
queues except, possibly, for the case of arrival rate vectors on the boundaries of delay stability regions. The arguments in the proof of Theorem 5.3 imply that this conjecture is true
for the special case of networks with disjoint schedules. Affirmative resolution in the general
case would reduce the problem of delay stability analysis to solving (perhaps numerically) a
system of ordinary differential equations from certain initial conditions.
Another interesting direction concerns the Max-Weight-α/Back-Pressure-α policy. For
this policy to achieve good performance, the α-parameters have to be selected appropriately;
and for the latter to happen, some knowledge of higher order moments of all arrival processes
is required. In scenarios where this information is not available, one could envision an
adaptive version of the policy that learns the variability of the arriving traffic, and adjusts
the parameter values accordingly. The hope is that, under certain assumptions, the algorithm
converges to paremeter values within the allowable ranges, so that the “good” steady-state
behavior of the original policy is preserved. Furthermore, it would be interesting to analyze
if, and to what extent, the transient behavior of the policy is affected by the fact that suitable
parameter values are not available a priori, but have to be learned.
Finally, one could seek for alternative ways to alleviate the effects of heavy-tailed traffic.
In principle, there are two ways of dealing with high variability in stochastic systems. The
first way is to accomodate the variability, which is what we tried to do with the MaxWeight-α/Back-Pressure-α policy. A different approach, which is quite common in practice,
is to reduce the variability. In the context of communication networks, this can be done
through congestion control. Given that Max-Weight is known to perform well under lighttailed traffic, we conjecture that a cross-layer framework of Max-Weight scheduling and some
appropriate form of congestion control would achieve stability, and good delay performance.
202
Bibliography
[1] V. Anantharam (1999). Scheduling strategies and long-range dependence. Queueing
Systems, 33(1-3), pp. 73-89.
[2] M. Andrews, K. Kumaran, K. Ramanan, A. Stolyar, R. Vijayakumar, P. Whiting (2004).
Scheduling in a queueing system with asynchronously varying service rates. Probability
in the Engineering and Informational Sciences, 18(2), pp. 191-217.
[3] M. Armony, S. Israelit, A. Mandelbaum, Y. Marmor, Y. Tseytlin, G. Yom-Tov (2010).
Patient flow in hospitals: a data-based queueing-science perspective. Preprint.
[4] F. Baccelli, S. Foss (2004). Moments and tails in monotone-separable stochastic networks. The Annals of Applied Probability, 14(2), pp. 612-650.
[5] F. Baccelli, S. Foss, M. Lelarge (2005). Tails in generalized Jackson networks with
subexponential service-time distributions. Journal of Applied Probability, 42(2), pp. 513530.
[6] D. Bertsimas, D. Gamarnik, J.N. Tsitsiklis (2001). Performance of multiclass Markovian queueing networks via piecewise linear Lyapunov functions. The Annals of Applied
Probability, 11(4), pp. 1384-1428.
[7] S. Borst, O. Boxma, R. Nunez-Queija, B. Zwart (2003). The impact of the service
discipline on delay asymptotics. Performance Evaluation, 54(2), pp. 175-206.
203
[8] S. Borst, M. Mandjes, M. van Uitert (2003). Generalized processor sharing with lighttailed and heavy-tailed input. IEEE/ACM Transactions on Networking, 11(5), pp. 821834.
[9] S. Borst, O. Boxma, M. van Uitert (2003). The asymptotic workload behavior of two
coupled queues. Queueing Systems, 43(1-2), pp. 81-102.
[10] O. Boxma, Q. Deng, B. Zwart (2002). Waiting-time asymptotics for the M/G/2 queue
with heterogeneous servers. Queueing Systems, 40(1), pp. 5-31.
[11] O. Boxma, B. Zwart (2007). Tails in scheduling. Performance Evaluation Review, 34(4),
pp. 13-20.
[12] M. Bramson (2008). Stability of queueing networks. Lecture Notes in Mathematics, 1950,
Springer-Verlag.
[13] L. Brown, N. Gans, A. Mandelbaum, A. Sakov, S. Zeltyn, L. Zhao, S. Haipeng (2005).
Statistical analysis of a telephone call center: a queueing-science perspective. Journal
of the American Statistical Association, 100(469), pp. 36-50.
[14] L. Bui, S. Sanghavi, R. Srikant (2009). Distributed link scheduling with constant overhead. IEEE/ACM Transactions on Networking, 17(5), pp. 1467-1480.
[15] L. Bui, R. Srikant, A. Stolyar (2011). A novel architecture for reduction of delay and
queueing structure complexity in the Back-Pressure algorithm. IEEE/ACM Transactions on Networking, 19(6), pp. 1597-1609.
[16] J.G. Dai (1995). On positive Harris recurrence of multiclass queueing networks: a unified
approach via fluid limit models. The Annals of Applied Probability, 5(1), pp. 49-77.
[17] J.G. Dai, W. Lin (2005). Maximum pressure policies in stochastic processing networks.
Operations Research, 53(2), pp. 197-218.
[18] J.G. Dai, W. Lin (2008). Asymptotic optimality of maximum pressure policies in
stochastic processing networks. The Annals of Applied Probability, 18(6), pp. 2239-2299.
204
[19] D. Down, S.P. Meyn (1997). Piecewise linear test functions for stability and instability
of queueing networks. Queueing Systems, 27(3-4), pp. 205-226.
[20] D. Ersoz, M. Yousif, and C. Das (2007). Characterizing Network Traffic in a Clusterbased, Multi-tier Data Center. In: Proceedings of the 27th International Conference on
Distributed Computing Systems.
[21] A. Eryilmaz, R. Srikant, J. Perkins (2005). Stable scheduling policies for fading wireless
channels. IEEE/ACM Transactions on Networking, 13(2), pp. 411-424.
[22] M. Fisher, A. Raman (1996). Reducing the cost of demand uncertainty through accurate
response to early sales. Operations Research, 44(1), pp. 87-99.
[23] R. Gallager (1996). Discrete stochastic processes. The Kluwer International Series in
Engineering and Computer Science, Kluwer Academic Publishers.
[24] D. Gamarnik, D. Katz (2009). On deciding stability of multiclass queueing networks
under buffer priority scheduling policies. The Annals of Applied Probability, 19(5), pp.
2008-2037.
[25] N. Gans, G. van Ryzin (1997). Optimal control of a multi-class, flexible queueing system.
Operations Research, 45(5), pp. 677-693.
[26] A. Ganti, E. Modiano, J.N. Tsitsiklis (2007). Optimal transmission scheduling in symmetric communication models with intermittent connectivity. IEEE Transactions on
Information Theory, 53(3), pp. 998-1008.
[27] L. Georgiadis, M. Neely, L. Tassiulas (2006). Resource allocation and cross-layer control
in wireless nertworks. Foundations and Trends in Networking, 1(1), pp. 1-144.
[28] P.W. Glynn, W. Whitt (1986). A central-limit-theorem version of L = λW . Queueing
Systems, 1(2), pp. 191-215.
205
[29] M. Graban (1999). An inventory planning methodology for a semiconductor manufacturer with significant sources of variability. SM Thesis, Massachusetts Institute of
Technology.
[30] A. Gut (2005). Probability: a graduate course. Springer Texts in Statistics, SpringerVerlag.
[31] B. Hajek (1982). Hitting-time and occupation-time bounds implied by drift analysis
with applications. Advances in Applied Probability, 14(3), pp. 502-525.
[32] B. Hajek (2006). Communication network analysis. Lecture Notes, available online at:
http://www.ifp.illinois.edu/∼hajek/Papers/networkanalysis Dec06.pdf.
[33] J.M. Harrison (2000). Brownian models of open processing networks: canonical representation of workload. The Annals of Applied Probability, 10(1), pp. 75-103.
[34] K. Jagannathan, M.G. Markakis, E. Modiano, J.N. Tsitsiklis (2010). Throughput optimal scheduling in the presence of heavy-tailed traffic. In: Proceedings of the 48th Allerton
Conference.
[35] K. Jagannathan, M.G. Markakis, E. Modiano, J.N. Tsitsiklis (2012). Queue length
asymptotics for generalized Max-Weight scheduling in the presence of heavy-tailed traffic. IEEE/ACM Transactions on Networking, 20(4), pp. 1096-1111.
[36] I. Keslassy, N. McKeown (2001). Analysis of scheduling algorithms that provide 100%
throughput in input-queued switches. In: Proceedings of the 39th Allerton Conference.
[37] L. Kleinrock (1963). Message delay in communication nets with storage. PhD Thesis,
Massachusetts Institute of Technology.
[38] P.R. Kumar, T. Seidman (1990). Dynamic instabilities and stabilization methods in
distributed real-time scheduling of manufacturing systems. IEEE Transactions on Automatic Control, 35(3), pp. 289-298.
206
[39] S. Kumar, P.R. Kumar (2001). Queueing network models in the design and analysis of
semiconductor wafer fabs. IEEE Transactions on Robotics and Automation, 17(5), pp.
548-561.
[40] W. Leland, M. Taqqu, W. Willinger, D. Wilson (1994). On the self-similar nature of
ethernet traffic. IEEE/ACM Transactions on Networking, 2(1), pp. 1-15.
[41] S.T. Maguluri, R. Srikant, L. Ying (2012). Stochastic models of load balancing and
scheduling in cloud computing clusters. In: Proceedings of the 31st IEEE Infocom Conference.
[42] A. Makowski, B. Melamed, W. Whitt (1989). On averages seen by arrivals in discrete
time. In: Proceedings of the 28th IEEE Conference of Decision and Control.
[43] M.G. Markakis, E. Modiano, J.N. Tsitsiklis (2009). Scheduling policies for single-hop
networks with heavy-tailed traffic. In: Proceedings of the 47th Allerton Conference.
[44] N. McKeown, A. Mekkittikul, V. Anantharam, J. Walrand (1999). Achieving 100%
throughput in an input-queued switch. IEEE Transactions on Communications, 47(8),
pp. 1260-1267.
[45] J. Nair, K. Jagannathan, A. Wierman (2013). When heavy-tailed and light-tailed flows
compete: the response time tail under generalized Max-Weight scheduling. In: Proceedings of the 32nd IEEE Infocom Conference.
[46] M. Neely (2008). Order optimal delay for opportunistic scheduling in multi-user wireless
uplinks and downlinks. IEEE/ACM Transactions on Networking, 16(5), pp. 1188-1199.
[47] B. Oguz, V. Anantharam (2012). Hurst index of functions of long-range-dependent
Markov chains. Journal of Applied Probability, 49(2), pp. 451-471.
[48] K. Park, W. Willinger (2000). Self-similar network traffic: an overview. In: Self-Similar
Network Traffic and Performance Evaluation, Wiley Inc.
207
[49] A. Rybko, A. Stolyar (1992). Ergodicity of stochastic processes describing the operation
of open queueing networks. Problemy Peredachi Informatsii., 28(3), pp. 3-26.
[50] D. Shah, J.N. Tsitsiklis, Y. Zhong (2010). Qualitative properties of α-weighted scheduling policies. In: Proceedings of the ACM Sigmetrics 2010 Conference.
[51] D. Shah, J. N. Tsitsiklis, Y. Zhong (2011). Optimal scaling of average queue sizes in an
input-queued switch: an open problem. Queueing Systems, 68(3-4), pp. 375-384.
[52] D. Shah, N. Walton, Y. Zhong (2012). Optimal queue-size scaling in switched networks.
In: Proceedings of the ACM Sigmetrics 2012 Conference.
[53] D. Shah, D. Wischik (2006). Optimal scheduling algorithms for input-queued switches.
In: Proceedings of the 25th IEEE Infocom Conference.
[54] D. Shah, D. Wischik (2008). Lower bound and optimality in switched networks. In:
Proceedings of the 46th Allerton Conference.
[55] D. Shah, D. Wischik (2012). Switched networks with maximum weight policies: fluid approximation and multiplicative state space collapse. The Annals of Applied Probability,
22(1), pp. 70-127.
[56] K. Sigman, R. Wolff (1993). A review of regenerative processes. SIAM Review, 35(2),
pp. 269-288.
[57] A. Stolyar (2004). Maxweight scheduling in a generalized switch: state space collapse
and workload minimization in heavy traffic. The Annals of Applied Probability, 14(1),
pp. 1-53.
[58] V. Subramanian (2010). Large deviations of Max-Weight scheduling policies on convex
rate regions. Mathematics of Operations Research, 35(4), pp. 881-910.
[59] L. Tassiulas, A. Ephremides (1992). Stability properties of constrained queueing systems
and scheduling policies for maximum throughput in multihop radio networks. IEEE
Transactions on Automatic Control, 37(12), pp. 1936-1948.
208
[60] L. Tassiulas, A. Ephremides (1993). Dynamic server allocation to parallel queues with
randomly varying connectivity. IEEE Transactions on Information Theory, 39(2), pp.
466-478.
[61] J.N. Tsitsiklis (1987). Analysis of a multiaccess control scheme. IEEE Transactions on
Automatic Control, 32(11), pp. 1017-1020.
[62] V.J. Venkataraman, X. Lin, L. Ying, S. Shakkottai (2010). On scheduling for minimizing
end-to-end buffer usage over multihop wireless networks. In: Proceedings of the 29th
IEEE Infocom Conference.
[63] W. Whitt (1991). A review of L = λW and extensions. Queueing Systems, 9(3), pp.
235-268.
[64] D. Williams (1991). Probability with Martingales. Cambridge Mathematical Textbooks,
Cambridge University Press.
209