QoS
Presented by:
Yaakov (J) Stein
CTO
The Access Company
QoS 1
What am I going to talk about today ?
• The telecommunications service model
– SLAs
• QoE and QoS
• Soft QoS (DiffServ)
–
–
–
–
–
–
packet marking (PCP, DSCP)
PHBs – BE, EF, AF
queuing mechanisms (strict priority, WFQ)
specifying datarate, bucketing algorithms (leaky, token)
traffic policing
traffic shaping
• Hard QoS (IntServ)
–
–
–
–
service levels – BE, CLS, GS
Network Engineering (planning) vs. Traffic Engineering (resource reservation)
RSVP
Routing protocols and RSVP-TE
QoS 2
the
telecommunications
service model
QoS 3
Why do we pay for services ?
Generally good (and frequently much better than toll quality)
voice service is available free of charge (Skype, Fring, Nimbuzz, …)
So why does anyone pay for voice services ?
Similarly, one can get free
•
•
•
•
•
(WiFi) Internet access
email boxes
file storage and sharing
web hosting
software services
So why pay for any service ?
QoS 4
Paying for QoS
The simple answer is that one doesn’t pay for the service
one pays for Quality of Service guarantees
In our voice model
price
toll quality
with mobility
QoS
BE
But what does QoS mean
and why are we willing to pay for it ?
To explain, we need to review some history …
QoS 5
Father of the telephone
Everyone knows that the father of the telephone was
Alexander Graham Bell
(along with his assistant Mr. Watson)
But Bell did not invent the telephone network
Bell and Watson sold pairs of phones to customers
The father of the telephone network was
Theodore Vail
QoS 6
Father of the telephone network
Theodore Who?
•
•
•
•
Cousin of Alfred Vail (Morse’s coworker)
Ex-General Superintendent of US Railway Mail Service
First general manager of Bell Telephone
Father of the PSTN
Organized telephony as a service (like the postal service!) *
Why else is he important?
•
•
•
•
Established principle of reinvestment in R&D
Established Bell Telephones IPR division
Executed merger with Western Union to form AT&T
Solved major technological problems
• use of copper wire
• use of twisted pairs
* Vailism is the philosophy that public services
should be run as closed centralized monopolies for the public good
QoS 7
What’s the difference ?
In the Bell-Watson model
the customer pays once, but is responsible for
• installation
– wires
– wiring
• operations
+
– power
– fault repair
– performance (distortion and noise)
• infrastructure maintenance
while the Bell company is responsible only
for providing functioning telephones
In the Vail model the customer pays a monthly fee
but the provider assumes responsibility for everything
including fault repair and performance maintenance
The telephone company owns the telephone sets and even the wires in the walls !
QoS 8
Service Level Agreements
In order to justify recurring payments
the provider agrees to a minimum level of service in an SLA
An SLA is a legal commitment between a service provider (SP)
and a customer, for example:
• Telco and subscriber
• ISP and Internet user
• VPN operator and enterprise
• cloud application provider and cloud user
SLAs typically include (financial) penalties for breaches
If objectives or penalties are too low, SLA is useless
If objectives or penalties are too high, cost will be prohibitive
Badly defined SLAs may damage operations by setting incorrect goals
QoS 9
SLAs and QoS parameters
SLAs should capture Quality of user Experience (QoE)
but this is often hard to quantify
So SLAs usually actually detail measurable network parameters
that influence QoE, such as :
Connectivity parameters
•
•
availability (e.g., the famous five nines)
time to repair (e.g., the famous 50 ms)
Noise (error) level parameters
•
•
•
•
SNR
BER
Packet Loss Ratio
defect densities
Information rate parameters
•
performance
parameters
bandwidth, throughput, goodput
Information latency parameters
•
•
1-way delay,
round trip delay
QoS 10
Connectivity vs. the rest
Basic connectivity (availability) always influences QoE
The other parameters may influence QoE
depending on service/ application (voice, video, browsing, …)
Some services only require basic connectivity
Some also require minimum available throughput
Some require delay less then some end-end (or RT) delay
Some require packet loss ratio (PLR) less than some percentage
Note: these parameters
are not necessarily independent
For example,
TCP throughput drops with PLR
1000 B packets
50 ms RTT
QoS 11
Some rules of thumb
Mission Critical (and life critical) services require
• high availability
If there are any MC services
then system traffic requires high availability too
MC services do not necessarily require strict throughput
but always indirectly require
• a certain minimal average throughput
• bounded delay
If the MC service uses TCP then it requires
• low PLR
Real-time services require
• sufficient throughput
but not necessarily low PLR (audio and video codecs have PLC)
Interactive services require
• low RT delay
QoS 12
QoS monitoring
RECAP: SLA compliance is the SP’s justification for payment
To ensure SLA compliance, the SP must :
• monitor the SLA parameters
• take action if parameter is dropping below compliance levels
But how does the SP verify/ensure that the SLA is being met ?
Monitoring is carried out using
Operations, Administration, Maintenance (OAM)
The customer too may use OAM to check that the SP is compliant !
Technical note:
OAM is a user-plane function
but may influence control and management plane operations
for example
• OAM may trigger protection switching, but doesn’t switch
• OAM may detect provisioned links, but doesn’t provision them
QoS 13
OAM – FM and PM
The difference between connectivity and performance parameters
leads to two types of OAM :
1. Fault Monitoring required for maintenance of connectivity (availability)
–
–
–
detection and reporting of anomalies, defects, and failures
OAM runs continuously/periodically at required rate
used to trigger mechanisms in the
• control plane (e.g. protection switching) and
• management plane (alarms)
2. Performance Monitoring required for maintenance of all other QoS parameters
–
–
measurement of performance criteria (delay, PDV, etc.)
OAM run :
• before enabling a service
• on-demand or
• per schedule
QoS 14
QoS assurance : availability
The difference between connectivity and performance parameters
leads to 2 types of QoS assurance – availability and performance
Availability is usually specified in “nines”
nines
up %
permitted down time
3 nines
99.9%
< 7 hour 18 min / month
4 nines
99.99%
< 44 minutes / month
5 nines
99.999%
< 4 min 23 sec / month
6 nines
99.9999%
< 26 sec / month
typical service
electric power service
PSTN
In order to ensure high availability, one employs
• FM OAM
• Automatic Protection Switching (APS)
QoS 15
QoS assurance : performance
There are two main approaches to ensuring performance QoS
IntServ (guaranteed QoS) – hard QoS
–
–
–
–
define traffic flows (CO approach)
guarantee QoS attributes for each flow
reserve resources at each router along the flow
signaling protocol (e.g., RSVP) needed
DiffServ (statistical QoS) – soft QoS
–
–
–
–
retain CL paradigm
no guaranteed QoS attributes
no resource reservation
mark packets (differentiated – e.g., gold, silver, bronze)
• marking can be by VLAN, P-bits, IP-ToS/DSCP, or general “flow”
– offer special treatment (priority) relative to other packets
DiffServ is the preferred approach for Ethernet and IP
IntServ is used in MPLS-TE
QoS 16
QoE
QoS 17
QoE and MOS
ITU-T defines QoE as
the acceptability of a service
as perceived subjectively by the end-user
A well-known QoE measure for telephony-grade voice
is Mean Opinion Score (MOS) (ITU-T P.800)
MOS is measured by having a number of listeners
listen and score speech on a scale from 1 (bad) to 5 (excellent)
and averaging over these scores (finding the mean)
• Toll quality voice has MOS = 4
• Cellphone voice has MOS  3.5
• Synthetic or military voice has MOS = 2 and below
QoS 18
QoE and QoS
Theory - QoE for a given application is a function of QoS parameters
QoE = f (service; QoS1, QoS2, … QoSn)
Researchers have found various functional forms
for the dependence of QoE on a particular QoS parameter
form
expression
examples
Linear
QoE  QoSk
perceived download time vs. PLR
Logarithmic
QoE  log(QoSk)
perceived download time vs. datarate
Exponential
QoE  exp(QoSk)
VoIP MOS vs. PLR
Power Law
QoE  QoSkp
perceived streaming video quality vs. PDV
see e.g., work of Markus Fiedler (BTH, Sweden)
QoS 19
Absolute vs. Comparative QoE
QoE measures may be
absolute determined by observing the degraded message or
comparative determined by comparing the degraded message to the original
Comparative measures are often more accurate
but can not be used unintrusively on a live network scenarios
Absolute measures can be used single-ended (non-intrusively)
MOS variations
Absolute Category Rating (ACR) : listeners hear only the degraded speech
Degradation Category Rating (DCR) : listeners hear first the original and then the
degraded speech and score 1 = very annoying degradation … 5 inaudible degradation
Comparative Category Rating (CCR) : listeners hear the original and the degraded
speech in random order and score
-3 2nd is much worse than 1st ... 3 2nd is much better than 1st
even simpler : AB test – simply report which sounds better
QoS 20
Subjective vs. Objective QoE
Direct human QoE scoring is expensive and time-consuming
ITU-T has defined objective measures that can be automated
These entail algorithms that produce scores
that correlate well with human QoE
PSQM (ITU-T P.861) and PESQ (ITU-T P.862)
are objective comparative MOS-like measures for telephone grade speech
They model the human auditory perception system (Bark scale, masking, etc.)
PEAQ (ITU-R BS-1387) similarly scores wideband audio
These were selected in competitions to have highest correlation with human MOS
ITU-T P.563 is a single-ended (absolute) objective MOS-like score
It determines un-naturalness of telephone-grade speech sounds
and the amount of non-speech-like noise
QoS 21
PSQM processing
x [n]
y [n]
Hanning window
Hanning window
xwi[n]
ywi[n]
FFT
FFT
Xi[k]
Yi[k]
Pxi[k]
Pyi[k]
frequency warping
Px'i[j]
frequency warping
calculate
local scaling
factor
Py'i[j]
Py"i[j]
Si
filter with receiving
characteristics of
handset
PFxi[j]
PHxi[j]
filter with receiving
characteristics of
handset
PFyi[j]
Hoth noise
PHyi[j]
intensity warping
Lxi[j]
intensity warping
calculate
loudness
scaling factor
Lyi[j]
S1i
Cognitive
subtraction
Ly'i[j]
T1207950-96
Ni[j]
asymmetry processing
Ni
silent interval weighting
Nwsil
QoS 22
E-model
The E-model defined in ITU-T G.107 is a planning tool
It predicts a “mouth-to-ear” transmission rating factor R
• between 0 and 100
• higher values signify better voice quality
• should be uniquely convertible to a MOS level
R = f(QoS1, … QoSn) and is additive in individual QoSk degradations
R starts with the basic signal to noise ratio
R is reduced to account for various impairments, including
• simultaneous impairments (loudness, sidetone, clipping, quantization noise)
• delay impairments (delay, echo delay and loudness)
• equipment impairments (codec distortion, packet loss)
R is increased when there are additional advantages
such as mobility (cellphone receives A=10)
R = R0 – Is – Id – Ie + A
QoS 23
R value meanings
R values
meaning
Equivalent MOS
90 - 100
Very satisfied
4.3-5.0
80- 90
Satisfied
4.0-4.3
70-80
Some users dissatisfied
3.6-4.0
60-70
Many users dissatisfied
3.1-3.6
50-60
Nearly all users
dissatisfied
2.6-3.1
Below 50
Not recommended
1-2.6
QoS 24
VQMON
Years before P.563 ETSI specified VQmon
TIPHON (Telecommunications and Internet Protocol Harmonization Over Networks)
TS 101 329-5 Annex E
VQmon (developed by Telchemy) is a single-ended method
for estimating the E-model factors for VoIP audio
based on QoS parameters (packet loss statistics, delay)
Depends on codec type
Takes human perception phenomena into account (e.g., recency effect)
VQmon was later extended to
• audio (MOS-A)
• video (MOS-V)
• audio-video (MOS-AV)
QoS 25
Video quality
ITU-R produced BT.500 for subjective assessment of TV quality
Similar to MOS :
• television sequences are shown to a group of viewers
• subjective opinions are averaged
ITU-T has produced many Recommendations
for video and multimedia quality :
• Subjective (P.9xx, J.140)
• Objective (J.143, J.144, J.147, J.148, J.24x, J.34x)
Since 1997 the Video Quality Experts Group (VQEG)
has been producing standards and tutorials
PEVQ (J.247) is a comparative pixel by pixel objective measure
that models the human visual tract
and returns a 5-point MOS score and further KPIs
QoS 26
QoE for other applications
G.1011 is a reference guide to existing standards for QoE
and provides a taxonomy
G.1010 discusses many applications, including
• conversational voice, voice messaging, streaming audio
• videophone, one-way video
• web-browsing, bulk data transfer, email, e-commerce,
• interactive games
• SMS, instant messaging
and gives performance targets for delay, PDV, and PLR
G.1050 gives an IP network model for evaluating the performance
of IP streams based on QoS parameters (delay, PDV, PLR).
J.163 treats real-time services over cable modems
X.140 defines QoS parameters for public data networks
QoS 27
Network planning tools
In addition to subjective/objective methods to quantify the QoE
of a specific (live or simulated) service instance
Network planners need tools to predict service quality
in order to efficiently allocate resources
G.1030 provides network planners with end-to-end (E-model-like)
tools for applications over IP networks
It includes an appendix devoted to web browsing
that presents empirical perception of users to response times
and proposes a MOS measure
G.1070 proposes an algorithm for network planners
to estimate videophone quality
QoS 28
Useful background documents
ISO 8402 defines quality as :
the totality of features and characteristics of a product or
service that bears its ability to satisfy stated or implied needs
E.800 Definitions of terms related to QoS
G.1000 Communications quality of service: A framework and definitions
ETSI ETR 003 General aspects of QoS and Network Performance (NP)
* Note – terminology in these documents is outdated
QoS 29
TR-126
The Broadband Forum (BBF) has produced TR-126, which includes
• a tutorial on QoE
• guidelines for QoE vs. QoS for triple play applications
TR-126 also discusses :
• QoE dimensions: service set-up, operation, and tear-down
• QoE facets: user effort, application responsiveness
• information fidelity, security, and dependability/availability;
• localization of QoE contributions (access, ISP, application SPs)
Guidelines are given for :
• video (conferencing, surveillance, streaming)
• voice (wired, wireless, voice messaging, IVR)
• best-effort Internet data (browsing, email, file transfer, VPN, P2P, ecommerce, …)
QoS 30
TMF
The TeleManagement Forum (TMF) discusses QoE SLA management
TMF’s Wireless Services Measurement Handbook GB923 defines :
• Key Quality Indicators (KQIs) (like QoE scores)
• Key Performance Indicators (KPIs)
KQIs may be determined from KPIs (the mapping may be complex)
KPIs are derived from QoS parameters
TMF has defined a set of KQIs including :
• response time
• service availability
• speech/video quality
• transaction rate
• offered throughput
An SLA consists of a set of KQI and KPI thresholds
(see SLA Management Handbook GB917 and its Application Notes)
QoS 31
Apdex
The Apdex Alliance is a consortium of companies
functions as a IEEE-ISTO (Industry Standards and Technology Organization) program
Apdex develops open standardized methods to
• report
• benchmark and
• track
application performance.
The Apdex (Application Performance Index)
• is a number between 0 and 1
• is meant to capture user satisfaction from an application
• 0 means no user would be satisfied
• 1 means that all users would be satisfied
QoS 32
Apdex
(cont.)
To compute the Apdex N users are divided into 3 categories
• satisfied
(S users)
e.g., web page completely loads within 2 seconds
• tolerating (T users)
e.g., web page completely loads within 8 seconds
• frustrated (F users)
e.g., web page takes > 8 seconds to load
The Apdex is given by
Apdex = ( S + T/2 ) / N
Apdex hierarchically deconstructs application transactions into
sessions processes tasks turns protocols packets
Sessions consist of the entire connect time
Processes are interactions accomplishing a goal
Tasks are individual interactions
The user is mainly aware of the task response time
since must wait for the task to complete before proceeding!
QoS 33
Behavioral QoE
All of the above subjective and objective QoE measures
are service/application-specific.
But new services and applications are created every day
and different users use different features of a single application
So it is no longer feasible to study each application in depth
A new approach is behavioral QoE estimation
the user’s satisfaction is estimated based on actions / reactions
Example : there is a high measured correlation between
a user being unsatisfied with a service level
his aborting the application (or at least waiting until the service level improves)
Behavioral QoE can be used instead of traditional QoE measurement
or to automatically find QoE(new app, QoS)
QoS 34
soft QoS
QoS 35
Queuing theory
Why isn’t the traffic distribution problem simple ?
If we have available data rate A,
simply allow traffic from all sources that sums to  A
Wrong !
Even if the average rates sum to much below A
there may be peak rates that exceed A
In order to accommodate peaks, we insert a queue
Customers/packets/whatever wait in queue to be serviced
arrivals
queue
S
Queue behavior can be counter-intuitive
Problem 1 :
Problem 2:
two buses arrive at my bus stop every 10 minutes
I take the first bus that arrives
Why do I take bus A much more than bus B ?
(hint : correlation)
buses leave their first stop every 10 minutes
and pick up passengers at intermediate stops
Why do the buses bunch ?
QoS 36
Scheduling disciplines
When there is a single queue, service order
is usually First In First Out (FIFO) AKA First Come First Served
but may be Last In First Out (LIFO/stack), random order, etc.
When there are multiple queues
packets belonging to a particular flow are consistently mapped to a single queue
there may be one or more flows in each queue
Service order may be :
• Round Robin : each queue visited in order
• Strict Priority : take from non-empty queue of highest priority
• Fair Queuing : preserve average datarate from each queue
• Weighted Fair Queuing : fair queuing with priority
• hybrid : mixture of several disciplines
QoS 37
Erlang
In early 1900s telcos needed to calculate the number of
switches, lines, and operators
they needed per call volume
• too little and subscribers would be unhappy
• too much wastes labor and money
Same problem for customers in stores, cars at traffic lights,
manufacturing processes, call centers, etc.
Agner Krarup Erlang developed queuing theory to solve this
problem for the Copenhagen telephone exchange
The unit of traffic use is called the Erlang in his honor
1 Erlang is 1 channel being used 100%, or 2 channels used 50%
when averaged over some time (generally an hour)
There is also a functional programming language
(used by Ericsson for telephony applications) named after him
QoS 38
Kendall notation
Let’s call
A – the statistics of arrival times (customers / packets / whatever)
B – the statistics of service times
C – the number of servers (there may be only 1 server)
K – the maximum queue length (if too many arrivals, need to drop)
Then we can describe a queuing system by A/B/C/K
and if K= then we call it A/B/C
Important statistics types:
D
Deterministic distribution (often fixed intervals)
M
Markov process (Poisson (exponential) distribution)
E(k)
Erlang distribution with parameter k
Example:
M/M/1 is a queue with a single server
and Poisson distributed arrivals and service times
QoS 39
Queuing theory results
Queuing theory derives important values, such as
• waiting times
• number of customers/packets waiting
• number of customers/packets being processed
We will not develop queuing theory here
Some models are completely understood (M/M/1, M/M/K, M/D/1)
Some formulae are true in general
Little’s law
L = λW
L : long-term average number of customers in a queue
λ : long-term average arrival rate
W : average time a customer (waits) in the system
This law is true for any arrival distribution, service distribution,
number of servers, service order, etc.
QoS 40
Recap: soft QoS
Soft QoS does not provide any hard service level guarantees
It merely breaks fairness by giving priority to certain
users or applications or flows or individual packets
When there aren’t network resources to forward all packets
packets are forwarded in order of priority from highest to lowest
Low(er) priority packets may be delayed or discarded(when K < )
In order to correctly prioritorize packets
they need to be priority marked
Marking is best accomplished by the originator
but may need to be performed by an intermediate element
based on port, or header fields, or even DPI
QoS 41
Marking Ethernet
VLAN ID (VID) indicates priority
In addition, for VLAN tagged frames
Priority Code Point (PCP) AKA user priority field, P-bits
•
•
•
•
•
3 bits so takes values 0 … 7
monotonically increasing priority
can use priority tagging (VLAN=0) if no VLAN
P=0 means non-expedited traffic
802.1Q gives recommends mappings
DA (6B)
SA (6B)
ET=8100 (2B)
ET=88A8 (2B)
ET (2B)
P(3b) CFI(1b) CVID(12b)
P(3b) DEA(1b) SVID(12b)
QoS 42
Marking MPLS
Only top label is relevant
Label can indicate priority (L-LSP)
or TC field (previously called EXP field, previously called COS field) (E-LSP)
• 3 bits so takes values 0 … 7
• no recognized TC value meanings
Top Label (20b)
Label (20b)
TC(3b) S(1b) TTL (8b)
TC(3b) S(1b) TTL (8b)
Bottom Label (20b)
TC(3b) S(1b) TTL (8b)
QoS 43
Marking IPv4
The IPv4 header has a Type of Service (ToS) field
RFC 2474 redefined ToS to consist of
• 6 bit DSCP (see also RFC 4594)
• 2 bit ECN (least significant bits)
Guidelines for use of DSCP in many documents
Ver(4b) IHL(4b) ToS(1B)
...
Len(2B)
Source IP Address (4B)
Destination IP Address (4B)
QoS 44
Marking IPv6
The IPv6 header has a Traffic Class (TC) field
RFC 2460 states that it is to be used like the IPv4 ToS field
The Flow Label (intended to ensure flow of packets follow the same path)
may be indirectly used
Ver(4b) TC(1B)
payload len(2B)
Flow Label (20b)
next(1B)
hop(1B)
Source IP Address (16B)
Destination IP Address (16B)
QoS 45
IP DiffServ methods
DiffServ was developed in IETF after IntServ RFCs 2474, 2475 because
• IntServ was considered too heavy-weight for most purposes
• resource reservation is against IP-philosophy
if not enough BW, then more democratic for all to suffer
if reserve BW and don’t use, then this is simply over-provisioning
DiffServ is evolutionary “coarse-grained” approach to IP QoS
DiffServ
– divides traffic into service classes
and allocates resources on a per-class basis
– uses 6 bits of ToS byte in IP header to mark packets
•field is renamed Differentiated Services Code Point
•no setup or router state required
– DSCP defines per-hop behaviors (PHB)
•tells router how to treat packet
– three standard PHBs (BE, AF, EF) but you are free to create more
QoS 46
DiffServ Per Hop Behaviors (PHBs)
Best Effort
– standard IP service
– QoS depends on momentary network load
WARNING: DiffServ does not provide true assurances
Assured Forwarding
– AF specifies class that determines queue
– in addition, three drop-precedence levels (low, med, high)
– AF packets from a single source should not be mis-ordered
even if have different drop-precedence (i.e. single queue)
Expedited Forwarding
– EF packet should experience no queuing delays
– EF packets should have low loss
– implemented by dedicated EF router queue
QoS 47
What about MPLS ?
MPLS DiffServ - each FEC is defined by
– destination address
– service class
L-LSP (Labeled-inferred LSP)
–
–
–
–
behavior based on label alone
support different service classes by using different labels
LSP BW allocated from specific queue (class)
TC field may be used for drop precedence
E-LSP (EXP-inferred LSP)
– behavior based on label + TC (ex-EXP) field
– TC bits in MPLS shim header are similar to DSCPs, but
only three bits (like P-bits) while DiffServ ended up with 6 bits
– but 8 service classes is usually more than enough
commonly 4 classes are offered (bronze, silver, gold, platinum)
– LSP BW allocated from link
QoS 48
Queuing in switches/routers
Switches and routers have queues FIFO buffers
output port
output port
switch
fabric
input port
input port
input port
queue
queue
queue
queue
Queues are emptied
according to scheduling discipline
(strict priority, WFQ, etc.)
output port
If there were only one queue
then traffic handling would be FCFS
To enable DiffServ prioritization
multiple queues are used
Outgoing frames are inserted into queues
according to priority marking
output port
on each output port
QoS 49
Traffic conditioning
One of the most important parts of an SLA is the
Committed Information Rate (bps)
This is the datarate (bandwidth) SP tries to forward
WARNING: DiffServ does not provide true commitments
There may also be an
Extra Information Rate (bps)
This is a datarate that the SP will forward if possible
A customer who did not send data for a while
will expect to be able to send a higher rate afterwards
Enforcement of these rates is accomplished via traffic conditioning
Three strategies :
• policing (rate limiting, throttling)
• shaping (in ATM – GCRA)
• metering
QoS 50
Traffic policing
Traffic exceeding the committed datarate is immediately discarded
(or at least marked as discard eligible)
CIR
time
policed to CIR
CIR
time
QoS 51
Traffic shaping
Traffic exceeding the committed datarate is delayed
until it can be forwarded (placed or remains in a buffer)
(discarded only if rate exceeds CIR for an extended time)
CIR
time
shaped to CIR
CIR
time
QoS 52
Traffic metering
Charge extra for traffic exceeding the committed datarate
(leads customer to self-police)
CIR
time
extra charge over CIR
CIR
time
QoS 53
BW specification
What does an SLA commitment of X bps mean ?
BW usage naturally varies for many services
Must the customer send < X bps at all times
even if he transmitted much less than X up to now ?
May the customer remain silent for 9 minutes
and then send 10X bps for 1 minute ?
If the measurement interval is 10 minutes
then this is precisely X bps !
A BW cap is only meaningful when we specify the integration time
Or, we can specify the rate and the maximum burst size (in bytes)
and enforce these using a bucketing algorithm
A bucketing algorithm allows bursts above X for a limited time
as long as the average remains at X or below
QoS 54
Leaky and token buckets
Leaky bucket model
Token bucket model
water is poured into bucket as needed
water leaks out at a constant rate
if too much water poured in it overflows
water is poured into bucket at a constant rate
water is removed as needed
if too much water poured in it overflows
Interpretation
Interpretation
bps of traffic are added to bucket
committed rate is continuously removed
if packet fits into bucket it is sent
unused data rate is lost
tokens are added at committed rate
to send traffic there have to be enough tokens
unused data rate is lost
QoS 55
How does a token bucket work ?
Part 1
Let’s look at a token bucket policer (other cases are equivalent)
The bucket is configured with
height - Committed Information Rate (CIR)
filling rate - Committed Burst Size (CBS)
CIR
If packets are sent at precisely the committed rate
CBS
– the bucket height stays constant
– and all packets are forwarded
If packets arrive at less then the committed rate Note:
Some people complicate formulas by
– the bucket height increases
specifying CIR in bps and CBS in Bytes
Such people should be committed
– all packets are forwarded
– excess information rate overflows and is lost
If packets arrive at more then the committed rate
– the bucket height decreases
– when no tokens are left packets are discarded
continued
QoS 56
How does a token bucket work ?
Part 2
If no packets have been sent for some time
and then CBS worth of packets are sent
– the bucket is initially full of CBS tokens
– the tokens are all removed
– all packets are forwarded
CIR
If more than CBS information rate in burst
– the first CBS of packets are forwarded
– the rest are discarded until new tokens arrive
CBS
Note: adding of tokens can be in
discrete time - every T (e.g., 1 sec) the token are added
continuous time – tokens are continuously added
(in practice, when new packet arrives, calculate number of tokens added since the last packet)
for continuous time, the maximum burst size is larger than the configured CBS
QoS 57
Dual token buckets
Sometimes SPs sell (and customers purchase) Extra traffic
This is a rate above the committed rate
that the SP will forward if it can (but doesn’t commit to forward)
Extra traffic is priced much lower than committed rate traffic
To handle Extra traffic, we use two (token or leaky) buckets, C and E
the C bucket is of height CBS and is filled at rate CIR
the E bucket is of height EBS and is filled at rate EIR
EIR
CIR
EBS
CBS
C
E
continued
QoS 58
Dual token buckets
(cont.)
Furthermore, we classify packets as
green
if passes C bucket test (green packets are forwarded)
yellow
if fails C bucket test, but passes E bucket test
(yellow packets may be forwarded, but SLA objectives don’t apply)
red
if fails both bucket tests (red packets are always discarded)
More precisely :
if ingress traffic < number of tokens in C bucket
frame is green and its length in tokens is debited from C bucket
else
if ingress frame length < number of tokens in E bucket
frame is yellow and its length of tokens is debited from E bucket
else frame is red
QoS 59
More token bucket variations
As if this isn’t complicate enough …
MEF added two more twists – coupling and sharing
Unused rate is not lost – it is coupled or shared !
coupling
priority
sharing
sharing
coupling
QoS 60
hard QoS
QoS 61
CO vs. CL networks
To guarantee QoS (and thus QoE) we need to
• find path through network that can provide the needed QoS
• reserve resources along this path to guarantee the QoS
• not accept flows for which there are insufficient resources (CAC)
• optionally – optimize path placement (to maximize number of flows that can be accommodated)
ATM and (some) MPLS networks are Connection Oriented (CO), thus
• we specify (or at least know) the path the packets will take
• we can reserve resources at the network elements along the path
Standard IP networks are ConnectionLess (CL), thus
• it is hard to ensure packets will go where we want them to
• it is meaningless to reserve resources
as we don’t know which network elements a packet will traverse
Thus hard QoS is not easy to add to IP
which is why hard QoS is not popular in pure IP networks …
QoS 62
Network and Traffic Engineering
Network Engineering (planning)
putting the bandwidth where the traffic is
– physical cable deployment (thick pipes)
– over-provisioning and backup connection provisioning
– does it violate provider objectives ?
Traffic Engineering (TE)
putting the traffic where the bandwidth is
– explicit traffic routing
– route optimization
– can it meet user objectives?
QoS 63
Traffic Engineering (TE)
TE is control of network traffic to achieve specific objectives
unfortunately users and providers have contradictory objectives
user objectives (QoS)
–
–
–
–
–
–
network availability
packet loss
end-to-end delay
round-trip delay
packet delay variation (PDV)
error rate
provider objectives (CAPEX, OPEX)
–
–
–
–
–
bandwidth utilization
resource utilization
speed of failure recovery
ease of management
monetary outlay
QoS 64
Simple example - fish diagram
A
D
1G
G
C
B
½G
E
all links 1G except EF ½ G
F
In the above example, there is sufficient BW for all traffic
(were these ATM switches, 1G over ACDG, ½ over BCEFG
Without TE, can’t use all the physical bandwidth!
standard IP routing : minimum hop count is CDG, so all traffic flows there
1½ G over 1G link, so ½G is dropped !
with administrative cost can force all traffic to go CEFG - which is worse !
with ECMP half (750M) goes CDG and half (750M) goes CEFG – still drops !
QoS 65
Constraint-based routing
IP uses distributed routing protocols, not centralized management
Distributed protocols are very good at finding basic connectivity
and minimizing an additive metric (e.g., hop count)
but are not good at optimally utilizing network resources
or obeying constraints
Common constraints include :
• explicit include/exclude links/routers (local constraint)
• conform to link BW constraints (local inequality constraint)
• meet end-end delay / PLR objectives (global inequality constraint)
Routing that takes constraints into account
is called constraint-based routing (CR)
After CR finds an acceptable path
we may need to set it up (reserve resources) using another protocol
QoS 66
OSPF-TE and IS-IS-TE
Constraint-based routing needs link attribute information
most importantly - available BW
Link-state protocols have mechanisms to flood link-up/link-down
to every router in the domain
We can piggyback attributes as TLVs on these messages
– OSPF add to Link-State Advertisement (LSA) (RFC 3630)
– IS-IS add to Link-State Packets
and the routing protocol builds an extended “TE” RIB
When attribute information changes need to reflood information
To decrease overhead:
– only flood when change passes threshold
– inherent timing bounds
Note: CR routing can be NP-hard
Standards don’t include (proprietary) efficient algorithms
QoS 67
IP IntServ
RFCs 2205-2216, 2379-2382
IntServ is an overall QoS architecture (not just RSVP)
initially developed for VoIP QoS
IntServ is a radical departure from pure IP
and requires IntServ-enabled routers
IntServ
– enables providing end-to-end QoS guarantees
– defines flows (introduces CO to IP’s CL architecture)
flows are classified into three service classes (BE,CLS,GS)
– specifies admission control and policing
– like all CO architectures, requires signaling protocol (RSVP)
IntServ-enabled routers
– reserve needed resources along the flow’s path
– must retain state
QoS 68
IntServ CoS levels
Best Effort
– standard IP service
– QoS depends on momentary network load
Controlled Load Service
– service equivalent to unloaded network
– low packet loss
– most packets will experience delay close to minimum
– no quantitative guarantees
Guaranteed Service
– bounded worst case delay (no PDV guarantee)
– low packet loss (zero if node buffers correctly provisioned)
– quantitative guarantees
QoS 69
RSVP
The primary signaling protocol for IntServ
is Resource reSerVation Protocol (RSVP)
RSVP
RSVP protocol
• runs between hosts and routers
• runs over raw IP or UDP/IP
• is unidirectional
• does not find path (fed by routing protocols)
• sessions identified by source and destination socket numbers
• requests unidirectional QoS characteristics from network
• causes routers along path to reserve link and node resources
• network responds with success/failure
• reservations are soft-state - time-out unless refreshed
two main message types: PATH and RESV
QoS 70
RSVP Messages
receivers
sender
PATH
– message from sender to receiver(s)
– carries classification info and TSpecs
RESV
– response of receiver to PATH message
– carries session ID and RSpec specifying QoS required
– contains the actual request for resource reservation
QoS 71
MPLS TE
MPLS-TE protocols enable Fast ReRoute
to guarantee fault recovery (connectivity assurance)
Also, MPLS FECs can take QoS constraints into account
MPLS-TE LSPs can be setup according to constraints
– include/exclude specific LSRs (for any reason)
– only include in LSP LSRs with sufficient available BW
– only include in LSP LSRs that guarantee sufficiently low delay
OSPF-TE or IS-IS-TE can be used to get needed network information
But how can the path be set-up ?
Vanilla LDP has no TE capabilities
and its extension CR-LDP is now obsolete
The answer is a set of extensions to RSVP called RSVP-TE
Unlike RSVP, this protocol runs only between routers (LSRs)
QoS 72
RSVP-TE
RSVP-TE (RFC 3209 …) is a label distribution protocol
• For downstream-on-demand binding
• Creates and distributes bindings between RSVP flows and labels
• Uses labels instead of source and destination socket numbers
• Extends RSVP by adding new objects (e.g. label) and procedures
• Allows strict/loose explicitly routed LSPs
• Has peer discovery, label requests, binding messages (like LDP)
• Transparent transport of QoS and traffic parameters
in TSpecs and RSpecs
• Although between routers - still soft state !
Note: RSVP-TE is frequently used to set up FRR alongside LDP
QoS 73
RSVP-TE LSP setup procedure
egress
ingress
Example setup
with explicit routes
A
B
C
• A sends PATH message to B w/ explicit route BC
and resource requirements
• B forwards PATH message to C after changing explicit route to C
• C determines required resources, reserves,
locally binds label and sends RESV to B
• B matches, reserves resources, remotely binds,
and sends RESV to A
• A matches, remotely binds label and reserves resources
QoS 74
PCE
To optimally utilize network resources (e.g., link BW), we need
to gather all the topology and network constraints into one centralized
Traffic Engineering Database (TED)
enough computational power to solve the complex optimization problem
to send path set-up commands to the routers to set up TE-LSP
RFC 4655 defines a Path Computation Element (PCE nicknamed Godbox)
and Path Computation Clients (PCCs)
The PCE may be
• a designated router or
• the management system or
• a dedicated computational platform
PCE
PCE is an evolutionary solution
to adding computational resources
QoS 75
PCEP
(RFC 5440)
The protocol between PCE and PCCs
and between multiple PCEs (if there are)
is called the Path Computation Element Protocol (PCEP)
PCEP runs over TCP with registered TCP port 4189
Messages are objects (with common object header) with optional TLVs
PCEP Messages :
•
•
•
•
•
•
•
Open
Keepalive
PCReq
PCRep
PCNtf
PCErr
Close
between PCE and PCC to open a new session
optional heartbeat sent if no other PCEP messages
PCC → PCE to request a path computation
PCE → PCC with set of computed paths or negative reply
event notification from either PCE or PCC
protocol error message
session close
QoS 76
Optimization
How does the PCE compute paths ?
The general path optimization problem is intractable (NP-hard)
But there are many combinatorial optimization problems
with known efficient algorithms
that return approximate solutions
For example, the (1-dimensional) bin packing problem :
Given N values V1 V2 … VN between 0 and 1
how can we place them in bins of maximum size 1
using the minimum number of bins ?
This problem comes up in many applications :
•
•
•
•
•
multiprocessor scheduling
stock cutting
mapping short messages into time slots of ATM cells
mapping CBR flows onto WDM wavelengths
Full PCE problem is even harder !
mapping flows onto orthogonal paths
QoS 77
SDN
An even more radical centralized solution is
the Software Defined Network (SDN)
Like PCE, SDN replaces distributed routing protocols
with a centralized controller
that communicates with SDN switches
An SDN switch is a forwarding device that can be programmed
to match an arbitrary set of fields in the packet
and edit / forward the packet accordingly
SDN switches need not obey standard network layers
The most popular controller-switch protocol is OpenFlow
Google uses SDN switches
to optimize utilization in its inter-datacenter network
QoS 78