Workshop on Congestion Control
Hamilton Institute, Sept 2005
Fair queueing and congestion
control
Jim Roberts (France Telecom)
Joint work with Jordan Augé
Fairness and congestion control
s
fair sharing: an objective as old as congestion control
Qcf. RFC 970, Nagle, 1985
Qnon-reliance on user cooperation
Qpainless introduction of new transport protocols
Qimplicit service differentiation
s
fair queueing is scalable and feasible
Qaccounting for the stochastics of traffic
Qa small number of flows to be scheduled
Qindependent of link speed
s
performance evaluation of congestion control
Qmust account for realistic traffic mix
Qimpact of buffer size, TCP version, scheduling algorithm
Flow level characterization of
Internet traffic
s
traffic is composed of flows
Qan instance of some application
Q(same identifier, minimum packet spacing)
s
flows are "streaming" or "elastic"
Qstreaming SLS = "conserve the signal"
Qelastic SLS = "transfer as fast as possible"
start
end
inter-packet < T
streaming
silence > T
elastic
Characteristics of flows
s
arrival process
QPoisson session arrivals: succession of flows and think times
s
size/duration
Qheavy tailed, correlation
think times
start of
session
flow
arrivals
end of
session
Characteristics of flows
s
arrival process
QPoisson session arrivals: succession of flows and think times
s
size/duration
Qheavy tailed, correlation
s
flow peak rate
Qstreaming: rate  codec
Qelastic: rate  exogenous limits (access link,...)
rate
rate
duration
duration
Three link operating regimes
"transparent"
"elastic"
"congested"
negligible
loss and delay
excellent for elastic,
some streaming loss
low throughput,
significant loss
flows
overall
rate
FIFO
sufficient
need
scheduling
need
overload control
Performance of fair sharing without rate
limit (ie, all flows bottlenecked)
s
a fluid simulation:
QPoisson flow arrivals
Qno exogenous peak rate limit  flows are all bottlenecked
Qload = 0.5 (arrival rate x size / capacity)
start
incoming flows
end
average
rate
high
link
rate
low
20 seconds
The process of flows in progress
depends on link load
30
flows in progress
20
10
0
high
load 0.5
low
The process of flows in progress
depends on link load
30
flows in progress
20
10
0
high
load 0.6
low
The process of flows in progress
depends on link load
30
flows in progress
20
10
0
high
load 0.7
low
The process of flows in progress
depends on link load
30
flows in progress
20
10
0
high
load 0.8
low
The process of flows in progress
depends on link load
30
flows in progress
20
10
0
high
load 0.9
low
Insensitivity of processor sharing: a
miracle of queuing theory !
s
link sharing  behaves like M/M/1
Qassuming only Poisson session arrivals
s
if flows are bottlenecked, E [flows in progress] = r/(1-r)
Qi.e., average  9 for r  0.9, but   as r  1
s
but, in practice, r < 0.5 and E [flows in progress] = O(104) !
20
E [flows in progress]
r/(1-r)
15
10
5
0
0
.2
.4
.6
.8
load, r
Trace data
s
an Abilene link (Indianapolis-Clevelend) – from NLANR
QOC 48, utilization 16 %
Qflow rates  (10 Kb/s, 10 Mb/s)
Q~7000 flows in progress at any time
>2.5 Mb/s
2.5 Gb/s
< 250 Kb/s
10 sec
Most flows are non-bottlenecked
s
s
each flow emits packets rarely
little queueing at low loads
QFIFO is adequate
Qperformance like a modulated M/G/1
s
at higher loads, a mix of bottlenecked and non-bottlenecked flows...
15 Mb/s
~1 ms
~7000 flows
2.5 Gb/s
~5 µs
Fair queueing is scalable and
feasible
s
fair queueing deals only with flows having packets in queue
Q<100 bottlenecked flows (at load < 90%)
QO(100) packets from non-bottlenecked flows (at load < 90%)
s
scalable since number does not increase with link rate
Qdepends just on bottlenecked/non-bottlenecked mix
s
feasible since max number is ~500 (at load < 90%)
Qdemonstration by trace simulations and analysis (Sigmetrics 2005)
s
can use any FQ algorithm
QDRR, Self-clocked FQ,...
Qor even just RR ?
Typical flow mix
s
many non-bottlenecked flows (~104)
Qrate limited by access links, etc.
s
a small number of bottlenecked flows (0, 1, 2,...)
QPr [  i flows] ~ ri with r the relative load of bottlenecked flows
s
example
Q 50% background traffic
– ie, E[flow arrival rate] x E[flow size] / capacity = 0.5
Q0, 1, 2 or 4 bottlenecked TCP flows
– eg, at overall load = 0.6, Pr [ 5 flows]  0.003
Simulation set up (ns2)
s
one 50 Mbps bottleneck
QRTT = 100ms
s
25 Mbps background traffic
QPoisson flows: 1 Mbps peak rate
Qor Poisson packets (for simplicity)
s
1, 2 or 4 permanent high rate flows
QTCP Reno or HSTCP
s
buffer size
Q20, 100 or 625 packets (625 = b/w x RTT)
s
scheduling
QFIFO, drop tail
QFQ, drop from front of longest queue
Results:
- 1 bottlenecked flow,
- Poisson flow background
FIFO + Reno
1000
20 packets
625 packets
cwnd
(pkts)
0
1
utilization
0
100s
100s
FIFO + Reno
1000
cwnd
(pkts)
20 packets
100 packets
Severe throughput loss with small buffer:
- realizes only 40% of available capacity
0
1
utilization
0
100s
100s
FIFO + 100 packet buffer
1000
cwnd
(pkts)
Reno
HSTCP
HSTCP brings gain in utilization,
higher loss for background flows
0
1
utilization
0
100s
100s
Reno + 20 packet buffer
1000
cwnd
(pkts)
FIFO
FQ
FQ avoids background flow loss,
little impact on bottlenecked flow
0
1
utilization
0
100s
100s
Results:
- 2 bottlenecked flows,
- Poisson packets background
FIFO + Reno + Reno
1000
20 packets
625 packets
Approximate fairness with Reno
cwnd
(pkts)
0
1
utilization
0
100s
100s
FIFO + HSTCP + HSTCP
1000
20 packets
625 packets
cwnd
(pkts)
0
1
utilization
0
100s
100s
FIFO + HSTCP + Reno
1000
20 packets
625 packets
HSTCP is very unfair
cwnd
(pkts)
0
1
utilization
0
100s
100s
Reno + HSTCP + 20 packet buffer
1000
FIFO
FQ
cwnd
(pkts)
0
1
utilization
0
100s
100s
Reno + HSTCP + 625 packet buffer
1000
cwnd
(pkts)
FIFO
FQ
Fair queueing is effective (though
HSTCP gains more throughput)
0
1
utilization
0
100s
100s
Results:
- 4 bottlenecked flows,
- Poisson packet background
All Reno + 20 packet buffer
1000
cwnd
(pkts)
1 flow
4 flows
Improved utilization with 4 bottlenecked flows,
approximate fairness
0
1
utilization
0
100s
100s
All Reno + 625 packet buffer
1000
1 flow
4 flows
Approximate fairness
cwnd
(pkts)
0
1
utilization
0
100s
100s
All HSTCP + 625 packet buffer
1000
1 flow
4 flows
Poor fairness, loss of throughput
cwnd
(pkts)
0
1
utilization
0
100s
100s
All HSTCP + 625 packet buffer
1000
cwnd
(pkts)
FIFO
FQ
Fair queueing restores fairness,
preserves throughput
0
1
utilization
0
100s
100s
Conclusions
s
there is a typical traffic mix
Qsmall number of bottlenecked flows (0, 1, 2,...)
Qlarge number of non-bottlenecked flows
s
fair queueing is feasible
QO(100) flows to schedule for any link rate
s
results for 1 bottlenecked flow + 50% background
Qsevere throughput loss for small buffer
QFQ avoids loss and delay for background packets
s
results for 2 or 4 bottlenecked flows + 50% background
QReno approximately fair
QHSTCP very unfair, loss of utilization
QFQ ensures fairness for any transport protocol
s
alternative transport protocols ?