ACM Sigmetrics 2010
A Modern Tool for Approximative Queueing
Analysis: Theory and Practice
-A Tutorial-
Florin Ciucu
Jens Schmitt
T-Labs / TU-Berlin
TU Kaiserslautern
The Problem of Queueing
… is about analyzing queue size (backlog), loss, delay, e.g.,
2
Queueing in Circuit Switching Networks, e.g.,
Telephone Network
• Each call is allocated a fixed chunk of network’s capacity
• Goal: size the network so that the queue is empty most of time
(almost zero blocking probabilities)
3
Queueing in Packet Switching Networks, e.g.,
the Internet
• All flows share the available bandwidth by interleaving packets
(statistical multiplexing)
• Goal: analysis of loss, delay (important for voice/video applications)
4
Behind the Title: The Stochastic Network Calculus (SNC)
• Probabilistic extension of the Deterministic Network Calculus (DNC),
which
− Was conceived by R. Cruz in the early 1990’s
− Is regarded as the theory for deterministic queueing analysis
• SNC is a tool (technique) for stochastic queueing analysis
− It’s approximative: provides rigorous bounds on queueing measures but
no guarantees on the gap to optimal results
− Can solve queueing problems which other alternative tools cannot
• It works like
− Deterministic evaluation of sample paths using the DNC
− Probabilistic evaluation of sample paths (e.g., using large deviations
arguments)
• Our goal is to convey that SNC is useful (…and approachable)
5
Relationship with Existing Methods for Queueing
Analysis
•
Single-queue analysis is broadly understood using
Queueing Theory, Matrix Geometric Methods, Effective
Bandwidth
− Solutions are tailored for each queueing problem
•
Queueing network analysis is largely restricted to
Poisson arrivals (BCMP, Kelly networks)
•
SNC provides a uniform network queueing algebra for
broad classes of arrivals/scheduling/service and has two
main features
1. ‘‘Scheduling Abstraction’’: abstracts away the details of
scheduling by a uniform service representation
2. ‘‘Convolution Form Networks”: abstracts the network service by
a single-node view
6
Feature 1: ‘‘Scheduling Abstraction”
7
Feature 2: ‘‘Convolution Form Networks”
…
Queueing Theory vs. SNC.
M/M/1 Queue
9
Outline of the Tutorial
• Theory
−
−
−
−
Arrival/Service processes
Feature 1: ‘‘Scheduling Abstraction”
Single-Node queueing measures (backlog, delay, output)
Feature 2: ‘‘Convolution Form Networks”
• Applications
− Reanalysis of classical queueing systems (M/M/1, M/D/1)
− End-to-End delays in a packet network with cross traffic
− End-to-End analysis of networks with heavy-tailed and selfsimilar traffic
− Output behavior at overloaded queues
• Insights on the SNC bounds accuracy. Conclusions
10
Arrivals at a Queue

Arrival Process
• … as a Point Process
• … as a Marked Point Process
• … as a Fluid Process
11
Arrival Process
• Also called Workload Process
• Bivariate extension
• Non-negative, non-decreasing, left-continuous
• Time model: in this tutorial mostly discrete
12
Approximate the Arrival Process
•
Estimation of backlog/delay distributions reduces to
theTail Estimation Problem, e.g.,
− Closed-form solutions/numerical evaluation is difficult, even for the
Poisson process
•
One may look for approximations (bounds) of the form
•
•
The tail estimation problem remains
Basic idea: assume the existence of bounds on either
•
Tail (distribution) or
•
MGF
13
Arrival Approximation 1: Tail Bound
• Formally defined as
• Note that
−
−
provides a stationary bound but
be stationary
is not required to
is not necessarily linear in
14
Example 0: Reduction to Deterministic Bound
15
Example 1: Hyperexponential Bound
• Can be constructed for
− Markov-Modulated processes, measured data
• Arbitrarily close approximations for Pareto, Weibull
distributions
• Classical case (in SNC) with only one exponential
(Exponentially Bounded Burstiness – EBB)
− Notation simplifies but accuracy lessens
16
Example 2: Heavy-Tailed Self-Similar
• Can be constructed for
−
stable distributions
− Compound point processes with Pareto increments
− Multiplexed heavy-tailed On-Off
− Measured data, e.g., aggregate Internet traffic
17
Arrival Approximation 2: MGF Bound
• Formally defined as
• Note that
− It captures bounds on all the moments of the arrivals
− Inapplicable to heavy-tailed arrivals
• Classical example with only one exponential (EBB)
− Notation simplifies but accuracy lessens
18
Relationship Between Tail and MGF bounds
Recall definitions for single exponentials
19
Tail and MGF Bound Constructions.
D/M Arrivals
20
Tail Bound Construction.
D/Pareto Arrivals
21
Approximate the Service Process
•
Problem: How to represent
’s service in order to
achieve ‘‘Scheduling Abstraction” (Feature 1)?
− Or … How to abstract away the details of many scheduling
algorithms by a uniform service representation?
•
To get the idea consider two cases depending on
whether the server is
1. always busy or
2. sometimes idle
22
Case 1: Always Busy Server
• Observations
− Busy means that there is always backlog to serve
− R.V.
denotes the maximum possible service at time
− Time varying service capacity (e.g., due to cross traffic)
• Then the cumulative offered service in time interval
23
Case 2: Sometimes Idle Server
• Denote
the beginning of the last busy period before
• Observe that
• Then
• Note: arrival and departure processes are related!
24
Cases 1 + 2: Service Process
• Service process abstractly characterized by
• Observations
−
−
defined as a bivariate random process
convolution provides a probabilistic lower bound
(note the inequality) on
’s service (i.e., service guarantees)
25
Analogy with the Standard Convolution
26
A First Glimpse into Multi-Node Analysis.
Recall Feature 2: ‘‘Convolution Form Networks”
• Remark: computation of end-to-end queueing measures
involves a ‘convolution’
• Problem: What if the two service processes are not
statistically independent?
• Basic idea: ‘‘Move the randomness” of service processes
27
Moving the Randomness of a Service Process:
Stochastic Service Curve
28
(General) Service Process
• Formally defined as
• Observations
−
−
is a bivariate random process
is a non-random error function
− Some randomness (dependencies) may be moved in
• This way it is possible to convolve originally dependent
service processes, by considering instead the following
29
Example 0: Reduction to Deterministic Service Curve
30
Examples 1+2: Stochastic Service Curves
Can be constructed from:
− Scheduling assumptions + hyperexponentially (or heavy-tailed
self-similar) bounded arrivals
− Measurements + fitting
31
Achieving ‘‘Scheduling Abstraction” (Feature 1)
32
Some Assumptions for Achieving
‘‘Scheduling Abstraction”
• Upper bounds on the (aggregate) cross traffic
− Recall that a service (curve) process sets lower bounds
• Server with capacity guarantees
• Driving factors:
− the scheduling algorithm
− (choice of fluid/packetized service model (t.b.d. later))
33
Scheduling Example 1: Arbitrary Scheduling
34
Scheduling Example 1: Arbitrary Scheduling (cont.)
35
Scheduling Example 2: FIFO
36
More on the Versatility of the (min,+) Convolution
for Approximating the Service Process
• So far … approximations of (per-flow) service process
in terms of capacity guarantees, i.e.,
− These imply (per-flow) delay guarantees (t.b.d. later)
• But is it possible to directly approximate the (per-flow)
service process in terms of delay guarantees?
37
Approximate the Service Process with
Delay Guarantees
38
Approximate the Service Process with
Delay Guarantees. Example: Exponential
39
A Second Glimpse into Multi-Node Analysis.
Recall Feature 2: ‘‘Convolution Form Networks”
• Possible when per-flow/per-node service processes
expressed in terms of delay guarantees
• …or capacity guarantees, or combinations of the two, e.g.,
40
Wrap-up on Probabilistic Approximation of
Arrival/Service
41
Derivation of Queueing Measures
•
Recall that SNC provides a ‘‘Uniform Queueing Algebra”
for queueing systems
− Based on ‘‘Scheduling Abstraction” (Feature 1) and the idea of
‘‘Convolution Form Networks” (Feature 2)
− Main purpose: derivation of per-flow measures (backlog and
delay process, output characterization)
•
Main steps for per-flow queueing algebra
1. Arrival representation with a tail/MGF bound
2. (Per-flow) service representation with a service process
3. Estimation of sample-path events, e.g.,
42
The Root of Sample Path Events: History Matters …
• Lindley’s equation for w.-c. server with rate
− Captures the history of the queueing process
• Leads to Reich’s equation
• Relation of
with Reich’s equation
43
Bounding The Backlog Process.
Case 1: Independent Arrivals/Service
44
Bounding The Backlog Process.
Case 1: Independent Arrivals/Service (contd.)
45
Bounding The Backlog Process.
Case 2: Not Necessarily Ind. Arrivals/Service
46
Bounding The Backlog Process.
Case 3: Tail Bounds Assumptions
47
Bounding The Backlog Process.
Case 4: Delay Guarantees.
48
Bounding The Delay Process.
Only the Case of Independent Arrivals/Service
49
Bounding The Output Process.
Only the Case of Independent Arrivals/Service
50
Bounding The Mean of Backlog/Delay Processes.
Only the Case of Independent Arrivals/Service
51
Multi-Node Analysis.
Recall Feature 2: ‘‘Convolution Form Networks”
…
52
Construction of a Network Service Process
Case 1: Zero Error Functions
…
53
Construction of a Network Service Process.
Case 1: Zero Error Functions - Conclusion
…
54
Construction of a Network Service Process
Case 2: Positive Error Functions
…
55
Construction of a Network Service Process
Case 2: Positive Error Functions (contd.)
…
56
Construction of a Network Service Process
Case 2: Positive Error Functions - Conclusion
…
57
Is the Network Service Process Useful?
… or, Are ‘‘Convolution Form Networks” Useful?
• Not without either a tail/Laplace bound, i.e.,
• In conjunction with tail/MGF bounds for
, i.e.,
• … one can derive network queueing measures simply
by applying single-node results
58
Construction of Laplace Bounds for a Network
Service Process
59
Application 1: Reanalyzing Classical Queueing
Systems. Case 1: M/M/1
60
Application 1: Reanalyzing Classical Queueing
Systems. Case 2: M/D/1
61
Application 2: End-to-End Delay in a Packet
Network with (EBB) Cross Traffic
• The network scenario
…
• All arrivals are marked point processes, e.g.,
• Other assumptions
• Because of compound arrivals, a new service model
for abstracting the packetization process is needed
62
Construction of Service Process for Packetization.
Case 1: No Cross Traffic
63
Construction of Stochastic Service Curve for
Packetization. Case 1: No Cross traffic
64
Construction of Service Process for Packetization.
Case 2: Cross Traffic
65
Construction of Service Process for Packetization.
Case 2: Cross Traffic (contd.)
66
Construction of Service Process for Packetization.
Case 2: Cross Traffic
67
Construction of Stochastic Service Curve for
Packetization. Case 2: Cross Traffic
68
End-to-End Delay in a Packet Network with (EBB)
Cross Traffic (contd.)
…
69
End-to-End Delay in a Packet Network with (EBB)
Cross Traffic (contd.)
…
70
End-to-End Delay in a Packet Network with (EBB)
Cross Traffic - Conclusion
71
Application 3: End-to-End Analysis of a Network with
Heavy-Tailed/Self-Similar (Cross) Traffic
•
The network scenario
…
•
All arrivals are htss, e.g.,
•
Applying earlier sample-path arguments would yield
end-to-end bounds of the form
•
Idea: geometric sample-paths
72
Geometric Sample-Paths
• Recall sample-paths so far
• Time was discrete and the sampling was arithmetic
• For the htss application we will sample time at the
points of the geometric series
• As we now work in continuous time we also have the
time parameter
73
Example: Construction of a Stochastic Network
Service Curve. Only 2 Nodes
74
Construction of a Stochastic Network Service
Curve. Only 2 Nodes (contd.)
75
Construction of a Network Service Curve Process.
Only 2 Nodes - Conclusion
76
Application 3: End-to-End Analysis of a Network with
Heavy-Tailed/Self-Similar (Cross) Traffic - Conclusion
…
• Construction of per-node stochastic service curves and
the derivation of queueing measures follow similar
geometric sample-path arguments
• Explicit end-to-end delay bounds can be thus obtained
• No statistical independence of the arrivals was assumed
77
Application 4. Output Behavior at
Overloaded Queues
• The network scenario
• The arrival processes have long-term rates
• Assume the overloaded queue case, i.e.,
• Problem: What can one say about the tail of
, i.e.,
78
Output Behavior at Overloaded Queues; FIFO
79
Output Behavior at Overloaded Queues; FIFO; EBB
Arrivals
80
Output Behavior at Overloaded Queues; FIFO;
EBB Arrivals. The Derivation
81
Output Behavior at Overloaded Queues; FIFO;
EBB Arrivals. The Derivation - Conclusion
82
On the Accuracy of the Derived SNC Bounds
83
On the Accuracy of the Derived SNC Bounds.
Is the Boole/Chernoff Combination Tight?
• No, this is already known from the Effective Bandwidth literature
− Implicitly, the same holds for the SNC literature (SNC uses roughly the
same large deviation techniques as in the EB literature for estimating
sample-paths events)
• In particular, the application of the Chernoff bound, though
attractive, can be loose in statistical multiplexing regimes, i.e., when
• Tight improvement over the Chernoff bound by using the BahadurRao result for large deviations
− Yet to be incorporated in the SNC literature
− The benefits of doing so would be in the derivation of per-flow end-toend results (recall Features 1 and 2: ‘‘Scheduling Abstraction’’ and
‘‘Convolution Form Networks”)
84
On the Accuracy of the Derived SNC Bounds.
What About Boole’s Inequality?
85
On the Accuracy of the Derived SNC Bounds.
Conclusions.
• The accuracy of SNC bounds strongly depends on available results
in large deviations
• A challenge remains the development of tighter bounds, than Boole
inequality, for the multi-node analysis
− Recall the Laplace estimate for the convolution of two service processes
• Conjecture: In principle, the key SNC concept of a service process
(or stochastic service curve) can be incorporated in existing
techniques for queueing analysis and achieve
− Per-flow results by ‘‘Scheduling Abstraction’’ and ‘‘Convolution Form
Networks”
− Arbitrarily tight bounds
86
Conclusions of the Tutorial
• The Stochastic Network Calculus is a relatively recent (~20 years)
tool, or technique, for queueing analysis
• Main idea: abstracts away technical difficulties related to
arrival/scheduling/service/multi-node by using bounds
− On arrivals (tail/MGF bounds)
− On service (service process, stochastic service curve)
• Key benefits of using SNC
−
−
−
−
Broad arrival/service classes, statistical (in)dependence
‘‘Scheduling Abstraction’’ and ‘‘Convolution Form Networks”
Closed-form results
Accuracy of the bounds: in principle as tight as related results from
probability theory
• In a nutshell, SNC is a simplified queueing algebra by applying the
principle: ‘‘It’s easier to approximate!”
87