CS144
An Introduction to Computer Networks
Flow Control, Congestion Control and
the size of Router Buffers
Section
Nick McKeown
Professor of Electrical Engineering
and Computer Science, Stanford University
1
Outline
Flow Control
Congestion Control
The size of Router Buffers
CS144, Stanford University
2
Sliding Window
Window Size
Data ACK’d
Outstanding Data
sent, but un-ack’d
Data OK
to send
Data not OK
to send yet
3
Flow Control
Inside Destination
RcvBuffer
Arriving packets
Resequence,
ACK, etc
OS
CS144, Stanford University
user
4
Dynamics of flow control
Animation at:
http://media.pearsoncmg.com/aw/aw_kurose_network_4/applets/flow/FlowControl.htm
CS144, Stanford University
5
Outline
Flow Control
Congestion Control
The size of Router Buffers
CS144, Stanford University
6
TCP Sliding Window
Round-trip time
Round-trip time
Window Size
Window Size
Window Size
A
B
ACK ACK ACK
(1) R x RTT > Window size
ACK
ACK
(2) R x RTT = Window size
7
“Bag of packets”
If there is a single flow, TCP is probing to find out
how big the “bag” is so it can fill it.
In general, a TCP flow is trying to figure out how
much room there is in the “bag” for its flow.
CS144, Stanford University
8
TCP Congestion Control
TCP varies the number of outstanding packets in the
network by varying the window size:
Window size = min{Advertised window, Congestion Window}
Receiver
Transmitter (“cwnd”)
9
AIMD
Additive Increase, Multiplicative Decrease
If packet received OK: W ¬ W +
If a packet is dropped: W ¬
CS144, Stanford University
1
W
W
2
10
Leads to the AIMD “sawtooth”
cwnd
“Drops”
halved
t
11
Dynamics of an AIMD flow
Animation at:
http://guido.appenzeller.net/anims/
CS144, Stanford University
12
Outline
Flow Control
Congestion Control
The size of Router Buffers
CS144, Stanford University
13
The size of a router buffer
B
C
14
Rule-of-Thumb
Buffer size = 2T x C, where:
2T = RTTmin = 2(propagation delay + packetization delay)
C = capacity of outgoing line.
Example: 10Gb/s interface, with 2T = 250ms
300Mbytes of buffering.
Read and write new packet every 32ns.
15
The Story
# packets
at 10Gb/s
1,000,000
10,000
20
2T  C
( 2)
2T  C 
 O(logW )
n
(1)
(1) Assume: Large number of desynchronized flows; 100% utilization
(2) Assume: Large number of desynchronized flows; <100% utilization
16
Time Evolution of a Single TCP Flow
Time evolution of a single TCP flow through a router. Buffer is 2T*C
Time evolution of a single TCP flow through a router. Buffer is < 2T*C
17
Buffer size = 2T x C
Interval magnified
on next slide
18
When sender pauses, buffer drains
Drop
one RTT
19
Origin of rule-of-thumb
Before and after reducing window size, the sending rate of the
TCP sender is the same
Rold  Rnew
Inserting the rate equation we get
Wold
Wnew

RTTold RTTnew
The RTT is part transmission delay T and part queueing delay B/C . We know
that after reducing the window, the queueing delay is zero.
Wold
Wold / 2

2T  B / C
2T

2T  C  B
20
Rule-of-thumb
Rule-of-thumb makes sense for one flow
Typical backbone link has > 20,000 flows
Does the rule-of-thumb still hold?
Answer:
- If flows are perfectly synchronized, then Yes.
- If flows are desynchronized then No.
21
The Story
# packets
at 10Gb/s
1,000,000
10,000
20
2T  C
( 2)
2T  C 
 O(logW )
n
(1)
(1) Assume: Large number of desynchronized flows; 100% utilization
(2) Assume: Large number of desynchronized flows; <100% utilization
22
Synchronized Flows
W
max
Wmax
 2
Wmax
Wmax
2
t
Aggregate window has same dynamics
Therefore buffer occupancy has same dynamics
Rule-of-thumb still holds.
23
Many TCP Flows
W
Buffer Size
Probability
Distribution
24
Required Buffer Size - Simulations
2T  C
n
Simulation
25
Level3 WAN Experiment
High link utilization for two weeks
Buffer sizes on three parallel links:
- 190ms (190K packets)
- 10ms (10K packets)
- 1ms (1k packets)
Note: This slide had a typo that caused some confusion (for me!) during class.
26
Drop vs. Load
Buffer = 190ms, 10ms
Packet Drop Rate (%)
0.2
0.16
0.12
0.08
0.04
0
0
20
40
60
80
100
Load (%)
27
Drop vs. Load
Buffer = 1ms
Packet Drop Rate (%)
0.2
0.16
0.12
0.08
0.04
0
0
20
40
60
80
100
Load (%)
28
The Story
# packets
at 10Gb/s
1,000,000
10,000
20
2T  C
( 2)
2T  C 
 O(logW )
n
(1)
(1) Assume: Large number of desynchronized flows; 100% utilization
(2) Assume: Large number of desynchronized flows; <100% utilization
29
Buffer Size
On-chip buffers
Smaller design
Lower power
Buffer
Window Size
Throughput
100%
t
RTT ´ C
N
RTT ´ C
Number of packets
10Gb/s WAN
10,000
1,000,000
30
Buffer Size
Throughput
100%
~ 90%
On-chip buffers
Smaller design
Lower power
Integrated all-optical
buffer [UCSB 2008]
20 pkts
RTT ´ C
log(W)
N
RTT ´ C
Number of packets
10Gb/s WAN
~50
10,000
1,000,000
31
Consequences?
10-50 packets on a chip
32
32