Evolution of TCP / UDP
User Datagram Protocol (UDP)
• Strengths
– Lightweight Protocol
– No overhead
Client
Server
rqst
resp
• Weaknesses
– No reliability in UDP
CS423 - Cotter
2
Transmission Control Protocol (TCP)
• Strengths
– Reliable Data Transmission
– Stream Protocol
Client
syn
syn, ack
• Weaknesses
– Connection Overhead
– Transaction time:
• 2 RTT +
• Server Processing time (SPT) +
• 1RTT for each additional data
packet
• Connection resources held
until any possible lost packets
have expired ( 2 * RTTmax = 240
sec)
CS423 - Cotter
Server
ack
data
fin
ack
data
fin
ack
3
Minimum TCP Transaction
• Protocol allows sending
data with syn
– Not generally done
– Must hold data until
3WHS complete
Client
syn, fin,
data
CS423 - Cotter
syn, ack
ack, fin
ack, fin
data
• Minimum transmission
– 5 packets
– 2 RTT + SPT
Server
ack
4
The Problem:
• Some applications (particularly Web apps) need both reliability and
speed
– Many connections (especially page loads) need only a few data
packets
– Early studies showed that median HTTP reply between 1770 and 958
bytes
– May have many connections per session.
– Overhead of connection (syn, syn-ack, ack) increases the transaction
time
– Google study showed 4 to 40% improvement in page load time
possible
– Because ports cannot be reused for 240 sec, maximum transaction
rate between a client and server is ~64000 ports/ 240 sec = 267
trans/sec
– Problem is most pronounced when RTT is large
CS423 - Cotter
5
One Solution:
TCP for Transactions (T/TCP)
• Originally proposed in rfc 1644
(1994)
• Include data in original syn
packet.
• Maintain a Connection Counter
(cc) in server that is passed by
client to server as TCP option.
Server uses this value to
validate client, bypassing the
need for 3-way handshake
(3WHS)
• Because server can now
validate each connection,
connection delay reduced from
240 to 12 sec.
CS423 - Cotter
Client
syn, cc,
data, fin
Server
syn, ack, cc,
data, fin
ack
6
T/TCP Initial Connection Setup
• Protocol requires setting up the connection
count between each client and the server. Initial
3WHS includes CC request.
• Each succeeding connection uses CC and avoids
3WHS
• Protocol requires that CC increment
monotonically.
CS423 - Cotter
7
T/TCP Protocol
• Initial 3WHS
Client
• Succeeding data calls
Server
syn, CCnew,
data
Client
syn, cc,
data, fin
syn, ack,
CCecho
Server
syn, ack, cc,
data, fin
ack
fin, ack
ack
Data, fin
ack
ack
CS423 - Cotter
8
T/TCP Problems !
• For most implementations, server initializes CC at 1
and increments by 1 for each new connection. By
sniffing the connection, it is easy to anticipate and
spoof the CC. Even without access to the path, picking
a large number will usually succeed (CC is 32 bit
number)
• This makes it very easy for an attacker to implement a
syn attack (DoS). It is made more effective since server
must store the data associated with the syn, using up
more server resources with each new packet.
• Protocol moved to historical status in 2011.
CS423 - Cotter
9
TCP Fast Open
• Researched originally at Google in 2011.
– Published at CoNEXT 2011
– Objective was to improve web server performance
• Offered as IETF draft first in 2012.
– Currently in draft 08 (expires August, 2014)
• Implemented in Linux (since Kernel 3.6/3.7)
CS423 - Cotter
10
TCP Fast Open
• Use initial connection to establish relationship.
– Client requests TCP fast open cookie and sends data in original
syn packet
– Server creates cookie and returns to client in syn, ack packet.
Cookie is an encrypted hash of client IP address and other data.
– Client ack packet completes 3WHS.
– Server processes client request and returns response.
• Subsequent requests do not need 3WHS
–
–
–
–
Client request includes server cookie, data
Server returns syn, ack
Server returns response
Client completes handshake.
CS423 - Cotter
11
Fast Open Initial connect
CS423 - Cotter
12
Fast Open Subsequent Packets
CS423 - Cotter
13
Linux Implementation
•
•
Requires current kernel (at least 3.7 for client and server)
Requires setting /proc/sys/net/ipv4/tcp_fastopen
– Enable TCP Fast Open feature (draft-ietf-tcpm-fastopen) to send data in the opening SYN
packet. To use this feature, the client application must use sendmsg() or sendto() with
MSG_FASTOPEN flag rather than connect() to perform a TCP handshake automatically.
The values (bitmap) are
1: Enables sending data in the opening SYN on the client w/ MSG_FASTOPEN.
2: Enables TCP Fast Open on the server side, i.e., allowing data in a SYN packet to be
accepted and passed to the application before 3-way hand shake finishes.
4: Send data in the opening SYN regardless of cookie availability and without a cookie
option.
0x100: Accept SYN data w/o validating the cookie.
0x200: Accept data-in-SYN w/o any cookie option present.
0x400/0x800: Enable Fast Open on all listeners regardless of the TCP_FASTOPEN
socket option. The two different flags designate two different ways of setting max_qlen
without the TCP_FASTOPEN socket option.
Default: 1 Note that the client & server side Fast Open flags (1 and 2 respectively) must
be also enabled before the rest of flags can take effect.
CS423 - Cotter
14
Linux TCP Fast Open client
(TFO_client.cpp)
void UDPecho(const char *host, const char *service)
{
charbuf[BUFFSIZE][LINELEN+1] = {"This is the first message", "This is the
second message", "This is the third message", "This is the fourth
message",
"This is the last message" };
char inbuf[LINELEN +1];
int
s, nchars, rchars;
/* socket descriptor, character counters */
:
for (index = 0; index < BUFFSIZE; index++)
{
s = socket(AF_INET, SOCK_STREAM, 0);
:
nchars = strlen(buf[index]);
CS423 - Cotter
15
Linux TCP Fast Open client
(TFO_client.cpp)
status = sendto(s, &buf[index], nchars, MSG_FASTOPEN, (const
struct sockaddr *) &sin, sizeof(sin));
:
cout << "We just sent a datagram" << endl;
rchars = recvfrom(s, inbuf, LINELEN, 0, NULL, NULL);
:
inbuf[rchars] = '\0'; //Add a null termination
:
cout << "We got back the following " << rchars << " characters: " <<
inbuf << endl;
close (s);
sleep(1);
} /* end of while */
}/* end of UDPecho () */
CS423 - Cotter
16
Linux TCP Fast Open server
(TFO_server.cpp)
int main(int argc, char *argv[]) {
int qlen = 5;
:
sock = passiveTCP(service, qlen);
ans = setsockopt(sock, IPPROTO_TCP, TCP_FASTOPEN, &qlen, sizeof(qlen));
while (keep_looping == YES) {
ssock = accept (sock, (sockaddr *)&fsin, (socklen_t *)&fsinsize);
sprintf (log_msg, "Just got a request from %s\n", inet_ntoa(fsin.sin_addr));
logData(log_msg);
cc = recv (ssock, buf, LINELEN, 0);
buf[cc] = '\0';
sprintf (log_msg, "Msg: %s\n", buf);
logData(log_msg);
strcat (buf, " answer");
send (ssock, buf, cc +7, 0);
memset (buf, 0, LINELEN);
close (ssock);
}//end of while loop
CS423 - Cotter
}
17
Wireshark Trace – Initial connect (1)
cookie request
CS423 - Cotter
18
Wireshark Trace – Initial connect (2)
cookie
CS423 - Cotter
19
Wireshark Trace – Initial connect (3)
data
CS423 - Cotter
20
Wireshark Trace – Initial connect (4)
ack
CS423 - Cotter
21
Wireshark Trace – Initial connect (4)
response
CS423 - Cotter
22
Wireshark Trace – Next connect (1)
syn, cookie, data
CS423 - Cotter
23
Wireshark Trace – Next connect (2)
syn,ack
CS423 - Cotter
24
Wireshark Trace – Next connect (3)
response
CS423 - Cotter
25
Wireshark Trace – Next connect (4)
fin, ack
CS423 - Cotter
26
Fast Open Client side Output
[bob@localhost client]$ g++ -o tfoc TFO_client.cpp
[bob@localhost client]$ ./tfoc 192.168.1.25 23456
Our target server is at address 192.168.1.25
Enter data to send...
We just sent a datagram
We got back the following 32 characters: This is the first message answer
We just sent a datagram
We got back the following 33 characters: This is the second message answer
We just sent a datagram
We got back the following 32 characters: This is the third message answer
We just sent a datagram
We got back the following 33 characters: This is the fourth message answer
We just sent a datagram
We got back the following 31 characters: This is the last message answer
[bob@localhost client]$
CS423 - Cotter
27
Fast Open Server Log
### Starting Server on port 23456 at Wed Apr 2 11:26:17 2014
Just got a request from 192.168.1.117
Msg: This is the first message
Just got a request from 192.168.1.117
Msg: This is the first message
Just got a request from 192.168.1.117
Msg: This is the second message
Just got a request from 192.168.1.117
Msg: This is the third message
Just got a request from 192.168.1.117
Msg: This is the fourth message
Just got a request from 192.168.1.117
Msg: This is the last message
CS423 - Cotter
28
Extended Fast Open Test
• Build Standard TCP Client
– Send 100 packets, receive 100 responses
• Build TCP Fast Open Client
– Send 100 packets, receive 100 responses
• Test Conditions
– Client and server on adjacent machines on same LAN
– Same code structure, same data sent and received.
• Compare Results
– How many Packets required for each
• Standard: 1000 packets
• TFO:
801
– How much time required for each?
• Standard: .173333 sec
.119198 sec
CS423 - Cotter • TFO:
29
Summary
• Network Protocols continue to evolve
• TCP Fast Open offers the speed of UDP (after
the first connection) with the reliability of TCP.
• Protocol is still in experimental stage.
CS423 - Cotter
30
References
•
TCP/IP Illustrated Vol 3, Richard Stevens
– Chapters 1-12 (T/TCP)
•
Original TCP Fast Open Paper
– http://conferences.sigcomm.org/co-next/2011/papers/1569470463.pdf
– http://static.googleusercontent.com/media/research.google.com/en/us/pubs/archive/37517.
pdf
•
IETF draft RFC
– https://datatracker.ietf.org/doc/draft-ietf-tcpm-fastopen/
•
lwn.net (Linux Weekly News?) Article
– https://lwn.net/Articles/508865/
•
Packet Pushers Article
– http://packetpushers.net/tcp-fast-curious-a-look-at-tcp-fast-open/
•
Admin Magazine Article
– http://www.admin-magazine.com/Articles/TCP-Fast-Open
•
Linux Kernel documentation
– https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt
CS423 - Cotter
31