The Case Against TCP for Transport Protocol Evolution
Naeem Khademi
Networks and Distributed Systems Group
Department of Informatics
University of Oslo
PhD Defense Trial Lecture – June 17, 2015
naeemk@ifi.uio.no
The Internet Transport Layer
3
The Internet Transport Layer
• TCP/IP suite developed in (D)ARPANET (1970s)
• TCP: The core transport protocol for the Internet’s operation since
then (RFC 675)
TCP
 Reliable
 In-order delivery
 Stream-oriented
 Connection-based
Application
Transport
(TCP/UDP)
Network (IP)
Data Link
Physical
Source: http://www.generalcomics.com
TCP/IP Stack
17 June 2015
4
The Internet Transport Layer
TCP Functionalities
 Multiplexing and Corruption handling (ports and checksum)
 Connection handling (establishment and teardown)
 Stream-based in-order delivery (sequence numbers)
 Reliability (ACK and retransmission)
 Flow control (overload protection of the receiver)
 Congestion control (overload protection of the network, utilizing the
capacity)
 Full duplex communication (ACK piggybacking)

17 June 2015
5
The Internet Transport Layer
• A lot of major applications rely on TCP: Worldwide web (HTTP), file
transfer (FTP, SCP), email (SMTP, IMAP, POP3,…), NTP, SSH,
etc.
• TCP has gradually grown over the years as a set of patches
TCP and the Internet
 Robust
 Scalable (end-to-end design)
 TCP gets through most middleboxes
17 June 2015
6
The Internet Transport Layer
• TCP (RFC 675) or UDP (RFC 768) (by SOCK_STREAM or
SOCK_DGRAM in socket API)
• There are more transport protocols e.g. SCTP (RFC 4960), DCCP
(RFC 4340), and a set of multicast transport protocols (e.g. NORM)
– Also extensions of TCP or UDP (MPTCP, UDP-Lite, RUDP, µTP/LEDBAT)
1970
1980
2000
1990
2005
2010
ARPANET
UDP
2015
1980
RTP 1996
TCP (RFC 675)
(1974)
Transport Protocols from IETF’s vantage point
RUDP
1999
SCTP
2000
UDP-Lite
2004
DCCP 2006
17 June 2015
MPTCP 2013
7
The Internet Transport Layer
• TCP has been the dominant transport protocol in the Internet
• UDP is sometimes blocked (by firewalls)
– Hard to establish session/flow state: block rules are stateless
• Analysis of CAIDA traces show increasing trend of UDP usage by
P2P networks and IPTV (2002~2009)
M. Zhang et al., ”Analysis of UDP Traffic Usage on Internet Backbone Links”, SAINT
2009
• What about other transport protocols (e.g. SCTP)? Not much or
almost not used
• Why? We are going to see later
17 June 2015
8
Is TCP/UDP enough?
• Yes
– It has been shown to work OK for so many years in the Internet
• In fact, no
– Applications are evolving and so do the app requirements
– More multimedia traffic: TCP/UDP was not designed for multimedia
– Proof: the existence of other transport protocols
17 June 2015
9
Is TCP/UDP enough? (#2)
• What do applications expect from a transport?
– Simply ”transport services”
– Can be hard to identify/define but something like:
Transport Services
Reliability
Performance (e.g. delay, throughput)
Security (e.g. authentication,
association, encryption)
Byte-stream, packet or message transfer
In-order delivery
Multi-streaming …
• What services applications may need that TCP cannot provide?
– Unordered reliable delivery: disk-over-LAN (e.g. iSCSI, RDMA),
latency-sensitive multimedia (eliminates HOL-blocking)
– Multi-streaming: e.g. web: a jpeg file and html text multiplexed
– Multi-homing: resilience to the link failure, load sharing
– Authentication: e.g. use of cookies
– Partial reliability: e.g. No need to retransmit TTL-expired packets
– Low Latency: TCP’s 3-way handshake setup, retransmission
17 June 2015
10
What do other transports provide?
• Stream Control Transmission Protocol (SCTP)
–
–
–
–
–
–
Message-oriented data transfer (header chunks)
Provides reliability and congestion control; connection-oriented
4-way handshake on association establishment (exchange of cookies)
Provides multi-streaming, multi-homing, unordered reliable delivery
Provides partial reliability (optional) (RFC 3758)
Many other features…
SCTP association establishment
4-way handshake
Client
Server
Bits
Common header
0
15-Aug
16 - 23
Source port
Verification tag
64
Checksum
Chunk 1 type Chunk 1 flags
128
Chunk 1 data
…
…
…
…
24 - 31
Destination port
32
96
17 June 2015
Bits 0 - 7
Chunk N type Chunk N flags
Chunk 1 length
Chunk N length
Chunk N data
SCTP packet header format
11
What do other transports provide?
• Datagram Congestion Control Protocol (DCCP)
–
–
–
–
–
-
Message-oriented and unreliable and unordered data transfer
Provides congestion control and ECN using ACKs
Reliable connection setup and teardown
Full-duplex bi-directional communication
Each endpoint can negotiate congestion control mechanism on connection setup
 TFRC or TCP-like
Suitable for interactive multimedia and gaming
 Prevents HOL-blocking and retransmission (of expired packets) of TCP
 Prevents congestion induced by UDP
 Session-based therefore trackable by middleboxes
DCCP
DCCP = UDP + congestion control
or
DCCP = TCP – bytestream semantics – full reliability
Source: http://heim.ifi.uio.no/michawe//teaching/cn2/cn2-3-6.pdf
17 June 2015
12
Why not use other transports?
•
SCTP/DCCP can offer various services that TCP/UDP does not
•
But sometimes only TCP (http port 80 or https 443) gets through
middleboxes
– UDP: not always
•
SCTP or DCCP not (or rarely) supported by middleboxes
– NATs: ubiquitous in the network
– Firewalls: rules based on TCP/UDP port numbers, DPI
– Performance Enhancing Proxies (PEPs): improving TCP, splitters
17 June 2015
13
Source: D. Ailawadi et al., “Minion—an All-Terrain Packet Packhorse
to Jump-Start Stalled Internet Transports”, PFLDNeT 2010
Why not use other transports?
•
Why middleboxes block anything other than TCP (and UDP)?
– Internet was not designed for security => disable/block everything that looks
”strange”
– TCP/UDP can be accepted (judging by port numbers)
– => Application programmers do not use other transport protocols
•
=> Almost impossible to deploy new transports
Deployment impossibility Cycle
Source: D. Ailawadi et al., “Minion—an All-Terrain Packet Packhorse
to Jump-Start Stalled Internet Transports”, PFLDNeT 2010
17 June 2015
14
What has been happening so far?
15
What has been happening so far?
• Applications have built solutions to work around the middlebox
problem
• Build abstraction on top of UDP
– SPUD, LEDBAT (BitTorrent), QUIC (Google), Skype, RTMFP (Adobe)
– User-space SCTP over DTLS over UDP for WebRTC data channel (draft-ietfrtcweb-data-channel-13)
• Build abstraction on top of TCP
– MPTCP, TCP Minion (partially in Apple)
17 June 2015
16
What has been happening so far?
(The Minion Suite)
• Minion called by the authors as ”packet packhorse”
• Uses legacy TCP/UDP as ”substrate” (minions) for
building new services for apps
– TCP Minion: unordered delivery in TCP
Source: D. Ailawadi et al., “Minion—an All-Terrain Packet Packhorse
to Jump-Start Stalled Internet Transports”, PFLDNeT 2010
17 June 2015
17
What has been happening so far?
(TCP Minion)
• TCP on the wire; offers datagram service to the apps
(unordered delivery)
– Middleboxes cannot distinguish it from normal TCP except by DPI
– uTCP: One new TCP socket option for immediate delivery of
received out-of-order segments: SO_UNORDERED via
setsockopt()
– Minimizes delay for latency-sensitive apps
– Useful for VPN tunnels (solves TCP-over-TCP issues)
17 June 2015
18
What has been happening so far?
(TCP Minion)
• Re-segmentation problem: TCP’s stream-oriented semantics do
not preserve any application-relevant frame boundaries within a
stream
– Datagram unrecognizable if fragmented by the network
• Use of delimeters (a certain byte) to mark the datagram boundaries
• Consistent Overhead Byte Stuffing (uCOBS): encode app.
datagrams to never include the ”delimiter” byte
– Reserves a byte for delimiter
– Max. 0.4% expansion
– Insert delimiter before and after the datagram (max 0.8% overhead)
17 June 2015
19
What has been happening so far?
(TCP Minion)
• Re-segmentation problem:
Source: http://www.ietf.org/proceedings/81/slides/tsvarea-0.pdf
17 June 2015
20
What has been happening so far?
(TLS Minion)
• TCP Minion: delimiter can be detected using DPI by middleboxes
• TLS Minion to counter DPI by middleboxes: appears as
SSL/TLS on the wire, but appears as HTTPS
17 June 2015
21
What has been happening so far?
(MPTCP)
• Multi-Path TCP (MPTCP): multihoming, load sharing, multiplexing
and resilience to the link failure (e.g. Wi-Fi and 3G/4G on mobiles)
–
–
–
–
–
Works wherever regular TCP works
Combines one more of regular TCP subflows
Subflow: a regular TCP flow on the wire
Uses TCP options to carry info. about the multipath flow (e.g. a unique token)
Coupled CC: Linked Increases Algorithm (LIA) (RFC 6356), OLIA
MPTCP Still…







Byte-stream
No multi-streaming
No unordered delivery
No partial reliability
Limited TCP options size
Has Issues with reordering
in parallel paths
…
17 June 2015
Source: C. Paasch et al., “Multipath TCP”, ACM Queue, March 2014
22
What has been happening so far?
(SPUD)
• Substrate Protocol for User Datagrams (SPUD):
– It is a prototype: draft-hildebrand-spud-prototype
– Encapsulate ”new transports” on top of UDP
– Uses bi-directional UDP ”tubes” between endpoints
– Explicit endpoints-to-path communication instead of already implicit
communication (DPI)
– Give middleboxes hint about the ”new transport” => no DPI needed
– UDP tubes identifiable by middleboxes
– Also feedback from the middleboxes to the transport
17 June 2015
23
What has been happening so far?
(QUIC)
• Quick UDP Internet Connections (QUIC):
–
–
–
–
–
Developed by Google, available since Chromium 29
Works in browser (user-space)
Multiplexed in-order reliable stream transport (especially HTTP) over UDP
Low latency: fast handshake/encryption setup and data request (0-RTT)
Solves TCPs HOL-blocking of SPDY stream by pausing one stream only
17 June 2015
Source: http://blog.chromium.org/2015/04/a-quic-update-on-googles-experimental.html
24
Internet Transport’s Evolution
25
Zooming Out: Internet Transport’s Evolution
• Current (ongoing) path: keep adding patches to TCP
–
–
–
–
Addition of CC (1988): Tahoe -> NewReno -> BIC -> CUBIC (Linux)
Window scaling
MPTCP, TCP Minion
And so many other patches and bugfixes
Challenges ahead Extending TCP

Limited option space (max. 40-bytes) (SCTP has more)

Protocol overhead of x-over-TCP or x-over-y-over-TCP
(e.g. TCP Minion)

Middleboxes modify/remove TCP options

Middleboxes modify sequence numbers (e.g. MPTCP)

DPI (e.g. TCP Minion, etc.)

Some Middleboxes fragile to out-of-order packets
Source: http://www.elliottspourhouse.com

17 June 2015
Segment spilitting and coalescing by middleboxes
M. Honda et al., “Is it Still Possible to Extend TCP?”, ACM IMC 2011
26
Zooming Out+: Internet Transport’s Evolution
• Current (ongoing) path: keep relying on current transport interface
– Narrow socket API (application chooses between TCP or UDP)
– Lack of abstraction e.g. like OSI model (ask for protocols instead of services)
– Impossible to request/define services
– Very difficult to change and deploy new transport protocols
– Very difficult to make use of existing protocols (e.g. SCTP or DCCP)
 Mainly due to lack of middlebox support and DPI
27
17 June 2015
Internet Transport’s Evolution
• Middleboxes are here to stay!
Possible long-term approaches
A. They will not change their behavior, bypass using legacy traffic encap. (Minion Suite)
B. Bypass them, but give them hints about the new transport to discourage DPI (SPUD)
C. Build a new transport API and try native new transports and fallback to TCP/UDP to
break the chicken-or-egg cycle (TAPS/NEAT)
Approach A: M. F. Nowlan et al, “Fitting Square Pegs
Through Round Pipes: Unordered Delivery WireCompatible with TCP and TLS”, ACM NDSI 2012
17 June 2015
Approach B: B. Trammell et al., “Evolving Transport
in the Internet”, IEEE Internet Computing, Sep. 2014 28
Internet Transport’s Evolution
• Approach (C): Transport Services (TAPS)
–
–
–
–
–
–
New flexible transport API
Apps ask for services instead of protocols (e.g. low latency, security, msg. priority)
Work in progress @ IETF TAPS WG
Work in progress @ NEAT EU project
Use “new transports” and fallback if no support on the path (happy eyeballs)
Can be orthogonal to or likely to include MPTCP, Minion and SPUD
Breaking the deployment impossibility cycle

Application designers
 Want best performance at cheapest cost
 Upgrade apps for better performance

OS developers
 Want best performance for mininal risk

Middlebox/firewall vendors/operators
 Want security and performance
June 2015
 17 Blocking
X degrades performance =>
unhappy customers!
Remember this? Deployment impossibility Cycle
29
Conclusions
A. TCP is not well suited for the future of the Internet
B. Encapsulating X-over-TCP or Y-over-UDP will encourage DPI by
middleboxes, adds overhead and/or may break end-to-end principles
C. Current transport API is narrow for extension of transport protocols
D. New backward-compatible transport API needed that
1. Provides services instead of protocols to the apps
2. Tries protocol X and falls back if not supported by the path
3. Breaks the deployment impossibility cycle
17 June 2015
30
Q&A
31