Chapter 8
Application Layer
based on:
A note on the use of these ppt slides:
We’re making these slides freely available to all (faculty, students, readers).
They’re in PowerPoint form so you can add, modify, and delete slides
(including this one) and slide content to suit your needs. They obviously
represent a lot of work on our part. In return for use, we only ask the
following:
 If you use these slides (e.g., in a class) in substantially unaltered form, that
you mention their source (after all, we’d like people to use our book!)
 If you post any slides in substantially unaltered form on a www site, that
you note that they are adapted from (or perhaps identical to) our slides, and
note our copyright of this material.
Computer Networking:
A Top Down Approach,
5th edition.
Jim Kurose, Keith Ross
Addison-Wesley, April
2009.
Thanks and enjoy! JFK/KWR
All material copyright 1996-2010
J.F Kurose and K.W. Ross, All Rights Reserved
Application 8-1
Chapter 8: Application Layer
Our goals:
 conceptual /
implementation
aspects of network
application protocols
 transport-layer
service models
 client-server
paradigm
 peer-to-peer
paradigm
 learn about
application-layer protocols
by looking at examples:


HTTP
DNS
(web)
(web addressing)
 mention briefly other
application protocols
Application Layer
8-2
Creating network applications
Write programs that



run on different end systems
and
communicate over a network.
e.g., Web server software
talks to browser software
application
transport
network
data link
physical
No app. software written for
devices in network core


Network core devices do not
function at application layer
This design allows for rapid
app development
application
transport
network
data link
physical
application
transport
network
data link
physical
Application Layer
8-3
App-layer protocol defines
 Types of messages
2TYPES OF PROTOCOLS:
exchanged, e.g. request Public-domain protocols:
& response messages
 defined in RFCs (mostly)
 Semantics: list of
 enable interoperability
needed fields per msg
& meaning of each field  e.g: HTTP, SMTP, SIP
Proprietary protocols:
 Syntax of each
message type: where
 owned by a company
fields are located, how  eg: Skype, Messenger
they are formatted
 Rules for when and how
processes send &
respond to messages
Application Layer
8-4
Client / Server concepts
 Hardware view
 Process / interaction view
 Client-Server architecture
 Peer-to-Peer architecture
Application Layer
8-5
Client Host /Server Host
server host :




always-on host
serves remote users
permanent IP address
server farms for scaling
client host (=workstation):



located next to user
may be intermittently
connected
may have dynamic IP
address
Note:
 Both client and server are called hosts
Application Layer
8-6
Client Process/Server Process
Process: program running within
a host.
 processes in different hosts
communicate by exchanging
messages
 when processes cooperate
for a common purpose, there
must be one of them that
initiates the common work

Client process: process
that initiates
communication
Server process: process
that waits to be
contacted by client
Sends the first message
Note:
 Client process may run on a client host or on a server host
 Server process may run on a server host or a client host
(see Peer to Peer architecture)
Application Layer
8-7
Process Interaction Scenario
Client process 1
server process:


waits all the time for work
its IP address must be
known to the client process
at interaction start. WHY?
client process:


Server proc. 1
Server proc. 2 

Client process 2
Note:

initiates interaction with
server process when it
needs server’s help
if server process agrees :
they exchange messages
the messages enable them
to cooperate in order to
get the work done
Does client address have to
be known in advance? Why?
 Server host usually runs several server processes in parallel
Application Layer
8-8
Client-Server architecture
server process:


runs on server host only
the server host’s IP
address is publicly known
client process:


runs on client host (usually)
or on a server host
example: in e-mail appl.
the sender mail server
runs a client process,
while the receiving MS
runs a server process
Application Layer
8-9
Pure P2P architecture
 no always-on server
 arbitrary end systems
(usually workstations) peer-peer
directly communicate
 peers are intermittently
connected and change IP
addresses
highly scalable but
difficult to manage
Application 8-10
Hybrid of client-server and P2P
Skype



voice-over-IP P2P application
centralized server: finding
address of remote party:
client-client connection: direct
voice communication
P2P
Instant messaging


chatting btw. two users is P2P
centralized service: client
presence detection/location
• user registers its IP address
with central server when it
comes online
• initiating user contacts central
server to find IP addresses of
buddies
C-S
P2P
C-S
data
initializatio
n
Application 8-11
What transport service does an app need?
Data loss
 some apps (e.g., file
transfer, telnet) require
100% reliable data
transfer
 other apps (e.g., audio) can
tolerate some loss
Timing
 some apps (e.g.,
Internet telephony,
interactive games)
require low delay to be
“effective”
Throughput
 some apps (e.g.,
multimedia) require
≥ minimum amount of
throughput to be
“effective”
 other apps (“elastic apps”)
make use of whatever
throughput they get
Security
 encryption, data integrity,
…
Application 8-12
Transport service requirements of common apps
Data loss
Bandwidth
Time Sensitive
file transfer
e-mail
Web documents
real-time audio/video
no loss
no loss
no loss
loss-tolerant
no
no
no
yes, 100’s msec
stored audio/video
interactive games
instant messaging
loss-tolerant
loss-tolerant
no loss
elastic
elastic
elastic
audio: 5kbps-1Mbps
video:10kbps-5Mbps
same as above
few kbps up
elastic
Application
yes, a few secs
yes, 100’s msec
sometimes
Application Layer
8-13
HTTP
Application Layer
#14
Web and HTTP
First a review
 Web page consists of objects
 Object can be HTML file, JPEG image, Java
applet, audio file, …
 Web page consists of base HTML-file which
includes several referenced objects
 Each object is addressable by a URL
 Example URL:
www.someschool.edu/someDept/pic.gif
host name
path name
Application Layer
8-15
HTTP overview
HTTP: hypertext
transfer protocol
PC running
Explorer
 Web’s application layer
protocol
 client/server model
 client : browser that
requests, receives and
“displays” Web objects
 server : Web server
that sends objects in
response to requests
 HTTP 1.0 : RFC 1945
 HTTP 1.1 : RFC 2616
Server
running
Apache Web
server
Mac running
Navigator
Note:
 Different H/W and S/W
 use the same protocol
Application Layer
8-16
HTTP overview (continued)
HTTP uses TCP:
 client initiates TCP
connection (creates socket)
to server port 80 (WKP)
 server accepts TCP
connection from client
 HTTP messages (applicationlayer protocol messages)
exchanged between browser
(HTTP client) and Web
server (HTTP server)
 TCP connection closed
HTTP is “stateless”
 HTTP keeps
no information about
past client requests

BUT, cookies allow
Web server to run
stateful applications
note
Protocols that maintain
“state” are complex!
 past history (state) must
be maintained
 if server/client crashes,
their views of “state” may
be inconsistent, must be
reconciled
Application Layer
8-17
HTTP connections
Nonpersistent HTTP
 At most one object sent
over a TCP connection.
 is default in HTTP/1.0

use Keep Alive header to
force Persistent mode
 Disadvantages:
1.
2.
3.
wastes 1 RTT per message
adds overhead packets
stays most of the time in
Slow Start Cong. Ctrl
 Partial help:
 use parallel connections
Persistent HTTP
 Multiple objects sent
over single TCP
connection between client
and server.

Last request contains the
Close Connection header
 is default in HTTP/1.1
Persistent with Pipelining
 Send requests for referenced
objects one after the other,
without waiting for response
to previous requests
 saves another RTT per object
Application Layer
8-18
HTTP request message
 two types of HTTP messages: request, response
 all msgs in ASCII (human-readable) format
 HTTP request message example:
method (other methods: POST, HEAD, …)
request line
header
lines
carriage return,
line feed at start
of line indicates
end of header lines
carriage return character
line-feed character
GET tau.ac.il/index.html HTTP/1.1\r\n
Host: www-net.cs.umass.edu\r\n
User-Agent: Firefox/3.6.10\r\n
Accept: text/html,application/xhtml+xml\r\n
Accept-Language: en-us,en;q=0.5\r\n
Accept-Encoding: gzip,deflate\r\n
Accept-Charset: ISO-8859-1,utf-8;q=0.7\r\n
Connection: Close\r\n
\r\n
Application 8-19
HTTP request message: general format
request
line
header
lines
body
Application 8-20
Uploading form input
POST method:
 browser sends to server data that was input by
user into a page form
 data appended in entity body of message
Such input can be also sent using GET
 in this case the data is loaded onto the URL field
of the request line
 appears after a “?” sign; example of such URL:
www.somesite.com/animalsearch?monkeys&banana
Application 8-21
Methods
HTTP/1.0
 GET
 POST
 HEAD


asks server to leave
requested object out of
response
Used for:
• debugging
• troubleshooting
HTTP/1.1
 GET, POST, HEAD
 PUT

uploads file in the entity
body to the server to be
saved path specified in
the path stated in the
URL field
 DELETE
 deletes file specified in
the URL field
Application Layer
8-22
HTTP response message
protocol
status line
header
lines
entity body
status code
status phrase
HTTP/1.1 200 OK\r\n
Date: Sun, 26 Sep 2010 20:09:20 GMT\r\n
Server: Apache/2.0.52 (CentOS)\r\n
Last-Modified: Tue, 30 Oct 2007 17:00:02
GMT\r\n
ETag: "17dc6-a5c-bf716880"\r\n
Accept-Ranges: bytes\r\n
Content-Length: 2652\r\n
Connection: Close\r\n
Content-Type: text/html;charset=ISO-8859-1\r\n
\r\n
data, e.g.,
requested
HTML file
Application 8-23
HTTP response status codes
In first line of the response message.
A few examples of status codes:
200 OK

request succeeded, requested object later in this message
301 Moved Permanently

requested object moved, new location specified later in
this message (“Location”: header line)
400 Bad Request

request message not understood by server
404 Not Found

requested document not found on this server
505 HTTP Version Not Supported
Application Layer
8-24
Trying out HTTP (client side) for yourself
1. Telnet to your favorite Web server:
telnet cis.poly.edu 80
opens TCP connection to port 80
(default HTTP server port) at cis.poly.edu.
anything typed is sent
to port 80 at cis.poly.edu
2. type in a GET HTTP request:
GET /~ross/ HTTP/1.1
Host: cis.poly.edu
by typing this in (hit carriage
return twice), you send
this minimal (but complete)
GET request to HTTP server
3. look at response message sent by HTTP server!
(or use wireshark!)
Application 8-25
User-server “state”: cookies
Many Web sites use cookies
Four components:
1) the HTTP server sends
response message with a
‘Set-Cookie:’ header
2) user’s browser puts the
‘Set-Cookie:’ header
content ( “cookie”), labeled
with server name, in the
cookie file on user’s host,
3) next request message to
this server will contain
Cookie: header containing
the relevant cookie
4) cookie points to database
record at server site (or
holds client data for server)
Example:




Susan accesses
Internet always from
same PC
She visits a specific
e-commerce site for
first time
When initial HTTP
requests arrives at site,
site creates a unique ID
and an entry in backend
database for that ID
The ID is the cookie
between them
Application Layer
8-26
Cookies: keeping “state” (cont.)
client
ebay 8734
cookie file
ebay 8734
amazon 1678
server
usual http request msg
usual http response
Set-cookie: 1678
usual http request msg
cookie: 1678
one week later:
ebay 8734
amazon 1678
usual http response msg
usual http request msg
cookie: 1678
usual http response msg
Amazon server
creates ID
1678 for user create
entry
cookiespecific
action
access
access
backend
database
cookiespecific
action
Application 8-27
Cookies (continued)
What cookies can bring:
 user session state
(e.g. in Web e-mail)
 user identification

on given workstation
 shopping carts
 recommendations
aside
Cookies and privacy:
 cookies permit sites to
learn a lot about you
o you may supply name and
e-mail to sites
 advertising companies
obtain info across sites
from inserted banner adds
o info collected by the
advertizing server
o there are regulatory
restrictions on releasing it
Application Layer
8-28
Web cache (proxy server)
Goal: satisfy client request without involving origin server
 user sets up browser to
access Web via proxy
 browser sends all HTTP
requests to proxy


IF object in the
proxy’s cache:
proxy returns object
ELSE proxy requests
object from origin
server, then returns
object to client
 and keeps file in its
cache
Proxy
Server
(web cache)
origin
server
client
client
origin
server
 Note: user’s browser has a
private cache on its host.
Application Layer
8-29
More about Web caching
 proxy acts as HTTP
client and server
 Typically proxy is
installed by ISP or
user’s organization.
Caching advantages
 Reduces response time for
client request.
 Reduces traffic on an
institution’s access link.
 Internet dense with
proxies enables “poor”
content providers to
effectively deliver content
Qn :what is the main problem
with caching as described?
Caching rules
 Origin server may :


forbid caching
allow caching by private
cache only (privacy)
 Origin server sends:



Last Modified date/time
Expiration time or
Max. time to hold object
 Proxy should respect age
and other restrictions
Application Layer
8-30
Caching example (1)
origin
servers
assumptions
 average object size = 100,000
bits
 average request rate from
institution’s browsers to origin
servers = 15/sec
 delay from institutional router
to any origin server and back
to router = 2 sec
consequences
public
Internet
1.5 Mbps
access link
institutional
network
10 Mbps LAN
 utilization on LAN = 15%
 utilization on access link = 100%
 total delay
= Internet delay +
access link delay + LAN delay
= 2 sec + minutes + milliseconds
institutional
cache
Application 8-31
Caching example (2)
possible solution
 increase bandwidth of access
link to, say, 10 Mbps
consequence
 utilization on LAN = 15%
 utilization on access link = 15%
 Total delay = Internet delay
+ access delay + LAN delay
= 2 sec + msecs + msecs
 often a costly upgrade
origin
servers
public
Internet
10 Mbps
access link
institutional
network
10 Mbps LAN
institutional
cache
Application 8-32
Caching example (3)
origin
servers
possible solution:
 install cache locally
consequence
 suppose hit rate is 0.4


public
Internet
40% requests will be
satisfied almost
immediately
60% requests satisfied by
origin server
 utilization of access link
reduced to 60%, resulting in
negligible delays (say 10
msec)
 total avg delay = Internet
delay + access delay + LAN
delay = .6*(2.01) secs +
.4*milliseconds < 1.4 secs
1.5 Mbps
access link
institutional
network
10 Mbps LAN
institutional
cache
Application 8-33
Conditional GET
 Don’t send object if cached
version is up-to-date

used if proxy has object
but cacheability conditions
are missing or not satisfied
 proxy specifies date of
cached copy in HTTP
request header:
If-modified-since: <date>
 server sends short
response if cached copy is
up-to-date:
HTTP/1.0 304 Not Modified
(contains no object)
(see Case 1 on this slide)
proxy
Case 1
server
HTTP request msg
If-modified-since:
<date>
HTTP response
object
not
modified
HTTP/1.0
304 Not Modified
proxy Case 2
server
HTTP request msg
If-modified-since:
<date>
HTTP response
object
modified
HTTP/1.0 200 OK
<data>
Application Layer
8-34
Legacy Internet Applications
Telnet
 Remote terminal
 Enables user to enter line
commands to remote host
 Remote host returns
display to user.
 characters sent one by one
and echoed on display
 sets char. encoding rules
FTP
 File transfer and remote
control of a file server
 Uses Telnet encoding rules
o
as do most other applications
Client-Server Email
 Mail client on user’s host

e.g. Outlook
 User mailbox on his Server
 Msg prepared on client, sent to
origin user’s Mail Server (MS)

may use SMTP
 MS sends to dest. MS using
SMTP (a PUSH protocol)
 dest. MS stores msg in mailbox
 User collects msg using PULL
protocol (POP3 or IMAP)
Webmail
 Client functionality is on Server
 User has an HTTP GUI terminal
Application Layer
8-35
DNS
Application Layer
#36
DNS: Domain Name System
People: many identifiers:
 SSN, name, passport #
Internet hosts, routers:
 IP address (32 bit) used for addressing
datagrams (packets) –
good for routers
 “name”, e.g.,
www.yahoo.com – good
for human use
Q: map between name and
IP addresses ?
Domain Name System:
1.
2.
Distributed database
implemented in hierarchy of
many name servers (NS’s)
application-layer protocol
host, routers, name servers
communicate to resolve
names (name-to-address
translation) (WKP: UDP 53)
 Note: Internet
infrastructure function,
implemented as an
application-layer protocol
 all the work at network’s
“edge”, not in the core
Application Layer
8-37
DNS
DNS services
 Hostname to IP address Why not centralize DNS?
 difficult to manage and
translation
maintain
 Host aliasing
 single point of failure
 canonical name = main
name of computer
 traffic volume
 alias = additional name
 distant centralized
 Finding server names
database
 e.g hostname of mail
 doesn’t scale!
server for domain
 Load distribution
 Replicated Web servers:
set of IP addresses for
one canonical name
Application Layer
8-38
Distributed, Hierarchical Database
Root DNS Servers (*)
com DNS servers
yahoo.com
amazon.com
DNS servers DNS servers
org DNS servers
pbs.org
DNS servers
edu DNS servers
poly.edu
umass.edu
DNS servers DNS servers
Client wants IP address for www.amazon.com: (**)
 Client’s NS queries a root server to find com DNS server’s addr.
(by sending message to UDP port 53)
 It then queries the com NS to get the NS for amazon.com
 It then queries the amazon.com NS to get IP address for
www.amazon.com
(*) The addresses of root servers are known to all name servers
(**) One possible scenario
Application Layer
8-39
The DNS Database architecture:
Authoritative NS’s, Zones, TLD’s
 Root NS’s: hold addresses of all Top Level Domain NSs
 Top-level domains : domains with single word name
 generic TLD’s: com, org, net, edu, etc,
• e.g. Network Solutions maintains servers for com TLD

top-level country domains: uk, fr, ca, jp, il, etc.
 Authoritative DNS server: organization’s DNS server,
providing authoritative hostname-to-IP mappings for
organization’s servers

It is responsible for a “zone” (or several zones)

Zone = Domain \
υ { subdomains that have their own zones }
 Example:
 tau.ac.il zone includes all university servers except dep”ts that
have their own zones (e.g. cs.tau.ac.il & math.tau.ac.il )
Application Layer
8-40
DNS Name Resolution:
Resolver and Local Name Server
 Each host has a Resolver = a function that resides in the
host and nows how to communicate with the DNS system
 Each ISP (residential ISP, company, university) has a
“Local name server” (it was called “Client’s NS” above)
 Also called “default name server”
 When application needs to translate a DNS name, it calls
the resolver on its computer

thus, resolver acts as a kind of ‘proxy’ for the application
 Resolver sends a DNS query to its local name server (LNS)

here again the LNS gives a proxy type service to the resolver
 LNS finds the answer in its cache or by querying other name
servers and sends answer to Resolver
 Resolver supplies answer to the application
Application Layer
8-41
DNS name
resolution example
root DNS server
2
 host at cis.poly.edu
3
wants IP address for
gaia.cs.umass.edu
iterated query:



contacted server
replies with name of
server to contact
“I don’t know this name,
please ask this server”
There are 2 iterated
queries in the example
TLD DNS server
4
5
local DNS server
dns.poly.edu
1
8
7
6
authoritative DNS server
dns.cs.umass.edu
requesting host
cis.poly.edu
gaia.cs.umass.edu
Application 8-42
DNS name
resolution example
recursive query:
root DNS server
2
puts burden of name
7
resolution on
contacted name server
 heavy load?
local DNS server
Note:
dns.poly.edu
 actual scenario may
1
8
involve a mix of the
two modes.
 root name servers are
usually iterative
requesting host
cis.poly.edu
 NS may act iteratively
on one type of queries,
recursively on another

3
6
TLD DNS server
5
4
authoritative DNS server
dns.cs.umass.edu
gaia.cs.umass.edu
Application 8-43
DNS: caching and mirroring
 once (any) name server learns mapping, it caches
mapping
 cache entries timeout (disappear) after some
time (each response includes a TTL)
 TLD servers typically cached in all local name
servers
• Thus root name servers less often visited
 Usually zone database is duplicated on several
authoritative name servers (mirroring)



the main one is updated: administrator downloads a file
the others copy the zone from the main one
main NS notifies them when update available
Application Layer
8-44
DNS records
DNS: distributed db storing resource records (RR)
RR format: (name, value, type, ttl)
 Type=A
name is hostname
 value is IP address

 Type=NS

name is domain (e.g.
foo.com)
 value is IP address of
authoritative name
server for this domain
 Type=CNAME

name is alias name of server
value is the canonical (“real”)
name of that server
e.g., www.ibm.com has real name:

servereast.backup2.ibm.com
 Type=MX
 name is domain, e.g. gmail.com
 value is hostname of mail server
Application Layer
8-45
DNS protocol, messages
DNS protocol : query and reply messages, both with
same message format
msg header


identification: 16 bit
query # ; reply to query
uses same #
flags:
 query or reply
 recursion desired
 recursion available
 reply is authoritative
Application 8-46
DNS protocol, messages
Name, type fields
for a query
RRs in response
to query
records for
authoritative servers
additional “helpful”
info that may be used
Application 8-47
Inserting records into DNS
 example: new domain (startup) “Network Utopia”
 register name networkuptopia.com at DNS registrar
(e.g., Network Solutions)


provide names, IP addresses of authoritative name servers
(primary and secondary) of this domain
registrar inserts two RRs into com TLD server:
(networkutopia.com, dns1.networkutopia.com, NS)
(dns1.networkutopia.com, 212.212.212.1, A)
(networkutopia.com, dns2.networkutopia.com, NS)
(dns2.networkutopia.com, 212.212.212.5, A)
(networkutopia.com, nili.networkutopia.com, MX)
(www.networkutopia.com, a1.networkutopia.com, CNAME)
etc.
 What does each of these RRs define?
 What RRs are missing?
Application 8-48
Peer to Peer Applications
Application Layer
#49
Pure P2P architecture
 no always-on server
 arbitrary end systems
directly communicate
 peers are intermittently peer-peer
connected and change
IP addresses
 No background server to
initialize the system
 Pure P2P appl. are rare.
we discuss three topics:



file distribution
searching for information
case Study: Skype
Application 8-50
File Distribution: Server-Client vs P2P
Question : How much time it takes to distribute
file from one server to N peers?
us: server upload
bandwidth
Server
us
File, size F
dN
uN
u1
d1
u2
d2
ui: peer i upload
bandwidth
di: peer i download
bandwidth
Network (with
abundant bandwidth)
upload : sending to Network
download : receiving from Network
Application 8-51
File distribution time: server-client
 server sequentially
sends N copies:
 NF/us time
 client i takes F/di time
to download
Server
F
us
dN
u1 d1 u2
d2
Network (with
abundant bandwidth)
uN
Time to distribute F to = dcs ≥ max {NF/us +F/max(di) ,
i
N clients by client/server
F/us +F/min(di) }
i
equal under best server
scheduling arrangement
increases linearly in N
(for large N)
Application 8-52
File distribution time: P2P
Server
 server must send one
F
u1 d1 u2
d2
copy: F/us time
us
 client i takes F/di time
Network (with
dN
to download
abundant bandwidth)
uN
 NF bits must be
downloaded (aggregate)
 fastest possible upload rate: us + Sui
N
dP2P ≥ max { F/us, F/min(di) , NF/(us + Sui) }
i
i=1
equal under best independent of N if the u are
i
data distribution all of same order of magnitude
arrangement
Application 8-53
Server-client vs. P2P: example
Client upload rate = u, F/u = 1 hour, us = 10u, dmin ≥ us
Minimum Distribution Time
3.5
P2P
Client-Server
3
2.5
2
1.5
1
0.5
0
0
5
10
15
20
25
30
35
N
Application 8-54
File distribution: BitTorrent
P2P file distribution
tracker: tracks peers
participating in torrent
torrent: group of
peers exchanging
chunks of a file
obtain list
of peers
trading
chunks
peer
Application 8-55
BitTorrent (1)
 file divided into 256KB chunks.
 peer joining torrent:
has no chunks, but will accumulate them over time
 registers with tracker to get list of peers, connects
to a chosen subset of peers (“neighbors”)
 while downloading, peer uploads chunks to other peers.
 peers may come and go
 once peer has entire file, it may (selfishly) leave
(“leech”) or (cooperatively) remain (“seed”)

Application 8-56
BitTorrent (2)
Pulling Chunks
 at any given time,
different peers have
different subsets of
file chunks
 periodically, a peer
(Alice) asks each
neighbor for list of
chunks that they have.
 then sends requests for
her missing chunks

rarest first
Sending Chunks: tit-for-tat
 Alice sends chunks to four
neighbors currently
sending to her chunks at
the highest rate
 Alice re-evaluates her
“top 4” list every 10 sec

each 30 sec she randomly
selects another peer,
starts sending chunks
 “optimistically unchoke”
 he may be better than
current neighbors
 newly chosen peer may join
top 4, if sends fast to her
Application 8-57
BitTorrent: Tit-for-tat
(1) Alice “optimistically unchokes” Bob
(2) Alice becomes one of Bob’s top-four providers; Bob reciprocates
(3) Bob becomes one of Alice’s top-four providers
“optimistic unchoke” procedure
helps find better trading
partners & get file faster!
Application 8-58
Distributed Hash Table (DHT)
 DHT: distributed P2P database
 database has (key, value) pairs; e.g:
 key: Soc.Security number; value: human name
 key: content title; value: IP address
 peers query DB with key

DB returns values that match the key
 peers can also insert (key, value) pairs
Application 8-59
DHT Identifiers
 assign to each peer an integer identifier
in range [0,2n-1].

Each identifier can be represented by n bits.
 require each key to also be an integer in [0,2n-1].
 since original key may be a string, we build an
integer key by applying an n-bit hash function
h to the original key.
e.g., key = h(“Led Zeppelin IV”) = 231567935
 this is why they call it a distributed “hash” table

Application 8-60
How to assign keys to peers?
 central issue:

assigning (key, value) pairs to peers that will
store them.
 rule: assign key to the peer that has the
closest ID.
 convention in our presentation: “closest” is
the closest from above or equal.
 e.g.,: n=4; peers: 1,3,4,5,8,10,12,15;
key = 13, then successor peer = 15
 key = 17, then successor peer = 1

o
the numbers are arranged in a ring (cyclic order)
Application 8-61
Circular DHT (1)
1
3
15
4
12
5
10
8
 each peer only aware of immediate successor
and predecessor and communicates only with
them.
 this forms the “overlay network”
Application 8-62
Circular DHT (2)
O(N) messages
on avg required to
resolve query in
overlay network,
when there are 1111
N peers
0001
I am
Who’s resp
0011
for key 1110 ?
1110
0100
1110
1110
1100
1110
1110
1010
0101
1110
1000
Application 8-63
Circular DHT with Shortcuts
1
3
15
Who’s resp
for key 1110?
4
12
5
10
8
 each peer keeps track of IP addresses of predecessor,
successor and shortcuts (call all of them “neighbors”).
 reduced from 6 to 2 messages.
 possible to design shortcuts so O(log N) neighbors
and O(log N) messages per query
Application 8-64
Peer Churn
To handle peer churn,
require each peer to know
the IP address of its two
successors.
 Each peer periodically
pings its two successors to
see if they are still alive.
1

3
15
4
12
 Example: peer 5 abruptly leaves
5
10
8
 Peer 4 detects; makes 8 its immediate successor;
asks 8 who its immediate successor is; makes 8’s
immediate successor (10) its second successor. How?
 Who else needs to update its neighbor list?
 What happens with the database info held by peer 5?
 What if peer 13 wants to join?
Application 8-65
P2P Case study: Skype
 inherently P2P: pairs of
Skype clients (SC)
users communicate.
 proprietary applicationSkype
layer protocol
login server
(inferred via reverse
engineering)
 hierarchical overlay
with Supernodes (SNs)
 Index maps usernames
to IP addresses;
distributed over SNs
Supernode
(SN)
 Note: SNs are hosts with non-NAT
addresses, so any host can contact them
Application 8-66
Peers as relays
 problem when both
Alice and Bob are
behind “NATs”.

NAT prevents an outside
peer from initiating a call
to insider peer
 solution:
 using Alice’s and Bob’s
SNs, relay host is chosen
 each peer initiates a TCP
session with relay.
 peers can now transfer
their realtime data
through NATs via relay
 Note: relay is another
Skype user
Application 8-67
EXTRA SLIDES
Application Layer
68
Nonpersistent HTTP
(page contains
text and
Suppose user enters URL of a page :
references to
www.someSchool.edu/someDepartment/home.index 10 jpeg images)
Client
Server
1a. HTTP client initiates TCP
connection to HTTP server
(process) at
www.someSchool.edu on port 80
2. HTTP client sends HTTP
time
request message (containing
URL) into TCP connection
socket. Message indicates
that client wants object
someDepartment/home.index
1b. HTTP server at host
www.someSchool.edu waiting
for TCP connection at port 80.
“accepts” connection, notifying
client
3. HTTP server receives request
message, forms response
message containing requested
object, and sends message
into its socket
Application Layer
8-69
Nonpersistent HTTP (cont.)
4. after sending the object,
HTTP server closes TCP
connection.
5. HTTP client receives response
time
message containing html file,
displays html. Parsing html
file, finds 10 referenced jpeg
objects
6. Steps 1-5 repeated for each
of 10 jpeg objects
Application Layer
8-70
Response time modeling
Definition of RTT: time to
send a small packet to travel
from client to server and
back.
initiate TCP
connection
Response time:
RTT
 one RTT to initiate TCP
request
connection
file
RTT
 one RTT for HTTP
request and first few
file
received
bytes of HTTP response to
return
time
 file transmission time
time to
transmit
file
time
total = 2 RTT + transmit time
Application Layer
8-71
Persistent HTTP
Persistent without pipelining:
Nonpersistent HTTP issues:
 client issues new request only
 2 RTT overhead per object
when previous response has
Extra Delay
been received
 OS must work and allocate
 one RTT overhead for each
host resources for each TCP
referenced object
connection
 Qn: What time is saved?
 browsers often open parallel Persistent with pipelining:
TCP connections to fetch
 default in HTTP/1.1
referenced objects
 client sends requests as soon
Persistent HTTP
as it encounters a referenced
object
 server leaves connection open
after sending response
 only one RTT for all the
referenced objects
 subsequent HTTP messages
between same client/server  Qn: Can you see an
advantage of NonPersistent?
are sent over same connection
Application Layer
8-72
FTP: the file transfer protocol
user
at host
FTP
FTP
user
client
interface
file transfer
(RFC 959)
FTP
server
remote file
system
local file
system
 transfer file to/from remote host

remote management of a file system
 client/server model


client (user’s workstation) initiates the connection
server (file server) responds to client’s commands
• mainly files transfers to or from client
 ftp Well Known Port for control traffic : 21
 ftp Well Known Port for data traffic
: 20
Application Layer
8-73
FTP: separate control, data connections
 FTP client contacts FTP server





by TCP at port 21,
sets up control connection
client obtains authorization
over control connection
client browses remote directory
by sending commands over
control connection.
before a file transfer, client
asks server to open a dataconnection to a client port N
(from server port 20)
client sends command:
RETR or STOR (see next slide)
After sending/receiving one
file, data connection is closed
TCP control connection,
server port 21
FTP
client
TCP data connection,
server port 20
FTP
server
 Server opens a second TCP
data connection to transfer
another file.
 Control connection: “out of
band”
 FTP server maintains “state”
throughout the session:
current directory,
authentication state
Application Layer
8-74
FTP commands, responses
Sample commands:
(sent as ASCII text over control
channel)
 USER username
 PASS password
 LIST - list the files in current
directory
– change working dir.
PORT N – set up data connect.
 CWD

to my port N
 RETR <filename>
- download
 STOR <filename>
- upload a
a file from server to client
file from client to server
Sample return codes
(status code and phrase: same
as in HTTP)
 331 Username OK,
password required
 125 data connection
already open;
transfer starting
 425 Can’t open data
connection
 452 Error writing
file
 226 File transfer OK,
closing connection
Note:
in Windows FTP user writes
 GET for RETR
 PUT for STOR
Application Layer
8-75
Electronic Mail
outgoing
message queue
Four major components:
user mailbox
 user agents
user
agent
 mail servers
 SMTP protocol btw servers
 POP3/ IMAP protocols to
mail
server
retrieve mail by user agent
User Agent
 a.k.a. “mail reader”
 composing, editing, reading
mail messages
 e.g., Outlook, Eudora, elm,
Netscape Messenger
 sends outgoing messages to
server
 collects incoming messages
from server
SMTP
SMTP
mail
server
user
agent
SMTP
user
agent
mail
server
user
agent
user
agent
user
agent
‫אפקה תשע"א ס"א‬
Applic
8-76
ation
Electronic Mail: mail servers
Mail Servers
 mailbox contains incoming
messages for user
 message queue of outgoing
(to be sent) mail messages
 SMTP protocol used
between mail servers to
send email messages
 SMTP client:
the sending mail server
 SMTP server:
the receiving mail server
user
agent
mail
server
SMTP
SMTP
mail
server
Note:
Some user agents use SMTP to
send mail to their mail server
user
agent
user
agent
mail
server
SMTP
user
agent
user
agent
user
agent
‫אפקה תשע"א ס"א‬
Applic
8-77
ation
Electronic Mail: SMTP
[RFC 821, 2821]
 uses TCP to reliably transfer email messages from client
to server

SMTP server port 25 (WKP)
 direct transfer: sending-server to receiving-server
 three phases of transfer
handshaking (greeting)
 transfer of messages
 closing connection
 command/response interaction
 commands: ASCII text
 response: status code (number) and phrase

 messages must be in 7-bit US-ASCII
 both the SMTP handshake and the message itself
‫אפקה תשע"א ס"א‬
Applic
8-78
ation
Scenario: Alice sends message to Bob
1) Alice uses UA (*) to compose
message with “to” field:
bob@someschool.edu
2) Alice’s UA sends message to
her mail server; message
placed in message queue
3) Alice’s mail server
(=SMTP client) opens TCP
connection to Bob’s mail
server
1
user
agent
2
mail
server
3
4) SMTP client sends Alice’s
message over the TCP
connection
5) Bob’s mail server places the
message in Bob’s mailbox
6) Bob uses his user agent to
retrieve message from mailbox
and reads it
* UA=User agent (e.g. Outlook)
mail
server
4
5
6
user
agent
‫אפקה תשע"א ס"א‬
Applic
8-79
ation
Sample SMTP interaction
<client sets up a TCP connection to server port 25>
S:
C:
S:
C:
S:
C:
S:
C:
S:
C:
C:
C:
S:
C:
S:
220 hamburger.edu
HELO crepes.fr
250 Hello crepes.fr, pleased to meet you
MAIL FROM: <alice@crepes.fr>
250 alice@crepes.fr... Sender ok
RCPT TO: <bob@hamburger.edu>
250 bob@hamburger.edu ... Recipient ok
DATA
354 Enter mail, end with "." on a line by itself
Do you like ketchup?
Mail headers skipped here
How about pickles?
(see a full mail example
.
on slide 73)
250 Message accepted for delivery
QUIT
221 hamburger.edu closing connection
‫אפקה תשע"א ס"א‬
Application Layer
8-80
SMTP: final words
Comparison with HTTP:
 SMTP uses persistent
connections
 SMTP requires message
(header & body) to be
in 7-bit ASCII
 SMTP server uses
CRLF.CRLF
(single dot in a line)
to signal end of message
 HTTP: (mainly) PULL
 SMTP: always PUSH
 both have ASCII
command/response
interaction, status codes
 HTTP: a page may contain
pictures and other objects

each object in own message
 SMTP:



originally only ASCII text
with MIME format, can
include any type of objects
objects sent inside msg.
‫אפקה תשע"א ס"א‬
Applic
8-81
ation
Original Email msg format [RFC 822]
SMTP: protocol for
transferring email msgs
Original email msg FORMAT:
RFC 822: standard for text
message format:
 header lines, e.g.,
To:
 From:
 Subject:
different from SMTP
commands !

header
blank
line
body
 body

the “message” text,
ASCII characters only
‫אפקה תשע"א ס"א‬
Applic
8-82
ation
MIME format: multimedia extensions
 MIME: Multimedia Mail Extension, RFC 2045, 2056
 additional lines in msg header declare MIME content
type and encoding method,
 this enables multimedia content
MIME version
method used
to encode data (*)
multimedia data
type, subtype,
parameters
encoded data
From: alice@crepes.fr
To: bob@hamburger.edu
Subject: Picture of yummy crepe.
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Type: image/jpeg
base64 encoded data .....
.........................
......base64 encoded data
(*) data always encoded into 7-bit ASCII
‫אפקה תשע"א ס"א‬
Applic
8-83
ation
Mail access protocols
user
agent
SMTP
SMTP
sender’s mail
server
access
protocol
user
agent
receiver’s mail
server
 SMTP: mail delivery to receiver’s server – PUSH
 Mail access protocols: retrieval from server to agent – PULL
they require authentication of agent by server
 POP: Post Office Protocol [RFC 1939]
• download all messages and keep at your workstation
 IMAP: Internet Mail Access Protocol [RFC 1730]
• enables organizing mail folders on server
• you can read mail conveniently from any workstation
 HTTP: (e.g. gmail) - display tool for agent located at the server

‫אפקה תשע"א ס"א‬
Applic
8-84
ation
POP3 protocol
authorization phase
 client commands:
user: declare username
 pass: password
 server responses
 +OK
 -ERR

transaction phase, client:
 list: list message numbers
 retr: retrieve message by
number
 dele: delete
 quit
S:
C:
S:
C:
S:
+OK POP3 server ready
user bob
+OK
pass hungry
+OK user successfully logged
C:
S:
S:
S:
C:
S:
S:
C:
C:
S:
S:
C:
C:
S:
list
1 498
2 912
.
retr 1
<message 1 contents>
.
dele 1
retr 2
<message 1 contents>
.
dele 2
quit
+OK POP3 server signing off
‫אפקה תשע"א ס"א‬
on
Applic
8-85
ation
POP3 (more) and IMAP
More about POP3
 Previous example uses
“download and delete”
mode.
 Bob cannot re-read email if he changes
client
 “Download-and-keep”:
can get messages on
different clients
 POP3 is stateless
across sessions
IMAP
 Keep all messages in
one place: the server
 Allows user to
organize messages in
folders
 IMAP keeps user state
across sessions:

names of folders and
mappings between
message IDs and folder
name
‫אפקה תשע"א ס"א‬
Applic
8-86
ation