CS 4700 / CS 5700
Network Fundamentals
Lecture 12: Router-Aided
Congestion Control
(Drop it like it’s hot)
Revised 3/18/13
In Between Network and Transport…
2

 Help
TCP control congestion
 Reduce queue length and delay
 Protection from misbehaving flows
Application
Presentation
Session
Transport
Network
Data Link
Physical
Function:

Key challenge:
 Early
detection of congestion
 Fairness
 Tractable implementation
The Danger of Increasing LoadCongestion
Collapse
3
increases very
slow
 Delay increases fast
In an M/M/1 queue
 Delay

Goodput
 Throughput

Knee
Knee – point after which
Cliff
Ideal point
Load
= 1/(1 – utilization)
Cliff – point after which
 Throughput
 Delay
0
∞
Delay

Load
Congestion Control Problems
4
cwnd
Synchronization
High queuing
delay
Time
Periodic lulls of
low utilization
End-to-end Congestion Control?
5

Routers also impact congestion management
Buffer management controls packet drops
 Packet scheduling influences delay


Router can be layer-3 and still help control congestion!
TCP tries to handle
Host 1
Host 2
Router
congestion
control
Application
Application
Transport
Transport
Network
Network
Network
Data Link
Data Link
Data Link
Physical
Physical
Physical
6



Outline
Random Early Detection
Fair Queuing
Core-Stateless Fair Queuing
FIFO with Drop Tail
7

Basic router queue management



First in, first out (FIFO)
Drop tail: if the queue is full, drop packets at the end
Problems:


Buffer lock out by misbehaving flows
Burst or multiple consecutive packet drops


Bad for TCP fast recovery
Synchronizing effect for multiple TCP flows
8
FIFO Router with Two TCP Sessions
8
Flows synchronize
Unfairness
High Delay
Low Throughput
Random Early Detection (RED)
9



Sometimes called Random Early Drop
FIFO scheduling
Replaces drop tail buffer management



Probabilistically discard packets
Probability is computed as a function of average queue length
Why average queue length?

Avoid over responsiveness to transient congestion
RED Variables
10



min: minimum threshold
max: maximum threshold
avg_len: (weighted) average queue length

avg_len = (1-w)*avg_len + w*sample_len
Discard Probability (P)
1
0
min max
queue_len
Average Queue Length
RED Operation
11
If (avg_len < minth)  enqueue packet
2. If (avg_len > maxth)  drop packet
3. If (avg_len >= minth and avg_len < maxth) 
enqueue packet with probability P
1.
Discard Probability (P)
1
1
0
3
min max
2
queue_len
Average Queue Length
Calculating P
12


P = max_P*(avg_len – min)/(max – min)
Improvements to spread the drops
P’ = P/(1 – count*P), where
 count = # of packets consecutively enqueued since last drop

Discard Probability (P)
1
max_P
P
avg_len
0
min max
queue_len
Average Queue Length
RED Example and Advantages
13
queue_len




max
min
Better absorption of packet bursts
Avoids TCP flow synchronization
Signals end systems earlier
Widely implemented on modern routers
 Cisco,
Juniper, etc
RED Router with Two TCP Flows
More even
throughput
and delay
Less unfairness
Less synchronization
14
15



Outline
Random Early Detection
Fair Queuing
Core-Stateless Fair Queuing
Problems with RED
16

No protection
 Misbehaving
flows can hurt other flows
UDP (#1)
UDP (#1)
TCP (#2)
...
TCP (#32)
TCP (#2)
...
TCP (#32)
10 Mbps)
Throughput(Mbps)
10
Stateless solution: Random
Early Detection (RED)
9
8
7
UDP
6
5
4
31 TCP Flows
3
2
1
0
1
4
7
10
13
16
19
22
Flow Number
25
28
31
Solution?
17

Round-robin among different flows (Nagle ‘87)
 One

queue per flow
Provides protection/isolation between flows
Problems with Round-Robin
18

Biased towards flows with large packets

Complexity: requires per flow state in the router
 Must
keep track of number of flows
 Must maintain flow->queue mapping
You never, EVER want per-flow state in the router.
Solution?
19



Bit-by-bit round-robin
Can you do this in practice?
NO
Packets cannot be sliced into bit-size pieces
 Packets
need to arrive with headers intact
 Imagine the fragmentation overhead

But, bit-by-bit round-robin can be approximated
Fair Queuing (FQ)
20



Define a fluid flow system: a system in which flows are
served bit-by-bit
Then serve packets in the increasing order of their
deadlines
Advantages


Each flow will receive exactly its fair rate
FQ achieves max-min fairness
Balancing Fairness vs. Utilization
21

Assume each queue gets to send x bits each RTT

Fair, and high
utilization
x

x
Fair, but
capacity is
wasted
Max-Min Fairness
22

If i parties request from resource , each receives
i

Allocation is max-min fair IFF
No user receives more than its request
2) No other allocation satisfying 1) has a higher minimum
allocation
3) Condition 2) remains true recursively as we remove the
minimal user and reduce its allocation from the total
resource
1)
Calculating Max-Min Fairness
23

Denote





C – link capacity
N – number of flows
ri – desired bandwidth of flow i
f – fair share bandwidth
åmin(r , f ) = C
i
i
Each flow can receive at most the fair rate, i.e., min(ri, f)
Max-Min Fairness Example

r1 = 8, r2 = 6, r3 = 2
8
6
2
C N C/N Flows w/ ri <= C/N
10 3 3.33 r3 = 2
8 2 4
-------
4
4
10
2
min(8, 4) = 4
min(6, 4) = 4
min(2, 4) = 2
C = C - ∑(flows w/ ri <= C/N)
N = N - |flows w/ ri <= C/N|
f =4
24
Implementing FQ in Practice
25


Ideal: do preemptive scheduling between packets
on bit-by-bit basis
Practical approximation: serve packets in the order
in which they would have finished transmission in the
fluid flow system
Practical FQ Example
26
1
Flow 1
(Arrival Traffic)
2
3
4
5
6
time
Flow 2
(Arrival Traffic)
1
Service
in Fluid Flow
System
1
FQ Packet
System
1
2
3
4
5
time
1
2
2
2
3
3
1
3
4
4
2 3
4
6
5
5
4 5
5
time
6
time
Simulation Example
30
UDP (#1)
TCP (#2)
...
TCP (#32)
TCP (#2)
...
TCP (#32)
10 Mbps)
0.5
Stateless solution: Random
Early Detection (RED)
9
8
Throughput(Mbps)
Throughput(Mbps)
10
UDP (#1)
7
6
UDP
5
4
31 TCP Flows
3
2
1
Stateful solution:
Fair Queuing
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
0
1
4
7
10
13
16
19
22
Flow Number
25
28
31
1
4
7
10
13
16
19
22
Flow Number
25
28
31
31



Outline
Random Early Detection
Fair Queuing
Core-Stateless Fair Queuing
Core-Stateless Fair Queuing
32



Fair Queuing requires per flow state in routers
You never, EVER want per-flow state in the router.
Core-Stateless Fair Queuing (CSFQ) eliminates state in
some of the routers…
… but only approximates FQ
The CSFQ Approach
33

A contiguous and trusted region of the network
 Edge
routers perform per flow operations
 Core routers perform no per flow operations
Fair Queuing
Everywhere
Core
Router
Edge
Router
Insight of CSFQ
34

If each packet of a flow with arrival rate r is forwarded
with probability: (f = fair rate)
 f
P  min1, 
 r

The rate of the flow’s forwarded traffic r’ is
 f
r '  r  P  r  min1,   min(r , f )
 r

No need to maintain per-flow state if r is carried in the
packet
CSFQ Algorithm Outline
35
1.
2.
Ingress routers: estimate r, insert into packet header
Core routers:



3.
Compute fair rate f on the output link (no state)
Enqueue packet with probability P = min(1, f/r)
Update packet label to r = min(r, f)
Egress router: remove extra state from packet header
CSFQ Algorithm Example
36



Maintains per
flow state
Insert r into
header


Compute f
Enqueue with P
= min(1, f/r)
Update r in
packet header
r
Ingress

Remove extra
header
r'
Core
Egress
Another CSFQ Example
37

Assume fair rate f = 4
 Flow
1, r1 = 8, P = min(1, 4/8) = 0.5
 Expected
 Flow
2, r2 = 6, P = min(1, 4/6) = 0.67
 Expected
 Flow
6
2
8
8
6
rate: 6*P = 4
3, r3 = 2, P = min(1, 4/2) = 1
 Expected
8
rate: 8*P = 4
8
6
rate: 2*P = 2
8
8
6
6
4
4
2
2
2
4
4
4
2
4
4
4
2
10
CSFQ Fair Rate Estimation
38

To calculate the fair rate f, core routers estimate
 Aggregate
arrival rate A
 Aggregate rate of accepted traffic F
(arrival rate – dropped packets)

Fair rate f is computed periodically as:
If there is no congestion (A<=C where C is link capacity), then f is
set to the maximum ri(t)
 If the link is congested, then fnew = fold* C/F

Simulation Example
39
UDP (#1)
UDP (#1)
TCP (#2)
...
TCP (#32)
TCP.(#2)
..
TCP (#32)
10 Mbps)
Throughput(Mbps)
10
Stateless solution: RED
9
8
7
UDP
6
5
4
31 TCP Flows
3
2
1
0
1
4
7
10
13
16
19
22
25
28
31
25
28
3139
Flow Number
Stateful solution:
Fair Queuing
0.45
0.5
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
1
4
7
10
13
16
19
22
Flow Number
Core-Stateless
Fair Queuing
Throughput(Mbps)
Throughput(Mbps)
0.5
25
28
31
1
4
7
10
13
16
19
22
Flow Number
Summary
40

FQ does not eliminate congestion
 It

just manages congestion
You need end-host congestion control and router support
 End-host
congestion control for adaptability
 Router congestion control for protection/isolation

Don’t forget about buffer management
 FQ
still needs to drop in case of congestion
 Which packets do you drop?