Application Layer Protocols
NETS3303/3603
Week 12
School of Information Technologies
Some network apps
•
•
•
•
•
•
•
E-mail
• Internet telephone
Web
• Real-time video
conference
Instant messaging
• Massive parallel
Remote login
computing
P2P file sharing
Multi-user network games
Streaming stored video clips
School of Information Technologies
Internet apps: application, transport protocols
Application
e-mail
remote terminal access
Web
file transfer
streaming multimedia
Internet telephony
School of Information Technologies
Application
layer protocol
Underlying
transport protocol
SMTP [RFC 2821]
Telnet [RFC 854]
HTTP [RFC 2616]
FTP [RFC 959]
proprietary
(e.g. RealNetworks)
proprietary
(e.g., Dialpad)
TCP
TCP
TCP
TCP
TCP or UDP
typically UDP
App-layer protocol defines
• Types of messages
exchanged, eg, request &
response messages
• Syntax of message types:
what fields in messages &
how fields are delineated
• Semantics of the fields, ie,
meaning of information in
fields
• Timing - rules for when
and how processes send &
respond to messages
School of Information Technologies
Public-domain protocols:
• defined in RFCs
• allows for interoperability
• eg, HTTP, SMTP
Proprietary protocols:
• eg, KaZaA
Remote Login
(TELNET and SSH)
School of Information Technologies
Remote Interaction
• Devised when computers used (ASCII)
terminals
• Terminal abstraction extended to remote
access over a network
School of Information Technologies
Client-Server Interaction
• Client
– Invoked by user
– Forms connection to remote server
– Passes keystrokes from user’s keyboard to server and
displays output from server on user’s screen
• Server
– Accepts connection over the network
– Passes incoming characters to OS as if they were typed
on a local keyboard
– Sends output over connection to client
School of Information Technologies
TELNET
• Standard protocol for remote terminal access over TCP
– Allows a user to log into a computer remotely
– Passes keystrokes directly to remote machine as if coming from
local keyboard
• Defines network virtual terminal that provides standard
interface
– NVT describes system independent encoding
– TELNET client and server map NVT into local computer’s
representation
• Mechanism that allows client and server to negotiate
options (e.g., character set)
School of Information Technologies
Illustration Of How NVT
Accommodates Heterogeneity
School of Information Technologies
Secure Remote Login (ssh)
• Alternative to TELNET
• Can be used as a transport layer protocol
with service authentication
• User authentication protocol
• Connection protocol
– Multiplexes multiple transfers
– Uses encryption for privacy
School of Information Technologies
Applications:
File Transfer And Access
(FTP, TFTP)
School of Information Technologies
On-Line File Sharing
• Always a popular application
• Two basic paradigms
– Whole-file copying (gets a local copy)
– On-line access
School of Information Technologies
File Transfer
• Whole file copying
• Client
user
at host
– Contacts server
– Specifies file
– Specifies transfer direction
• Server
– Maintains set of files on local disk
– Waits for contact
– Honours request from client
School of Information Technologies
FTP
FTP
user
client
interface
local file
system
file transfer
FTP
server
File Transfer Protocol (FTP)
•
•
•
•
Major TCP/IP protocol for whole-file copying
Uses TCP for transport
FTP client contacts FTP server at port 21
Features
– Interactive access
– Format specification (ASCII or EBCDIC)
– Authentication control (login and password)
School of Information Technologies
FTP Process Model
TCP control connection
port 21
• Separate processes handle
– Interaction with user
– Individual transfer requests
FTP
client
TCP data connection
port 20
• Data transfer connections created dynamically
when needed
• The control connection persists throughout a
session
School of Information Technologies
FTP
server
Control Connection Vs. Data
Connection
• For data transfer, client side becomes server and server
side becomes client
• Client
– Creates process to handle data transfer
– Allocates port and sends number to server over control
connection
– Process waits for contact
• Server
– Receives request
– Creates process to handle data transfer
– Process contacts client-side
School of Information Technologies
Out-of-band Control
• Control connection: “out of band”
• FTP server maintains “state”:
– current directory, earlier authentication
• What special relationship is required
between FTP and NAT?
School of Information Technologies
FTP commands, responses
Sample commands:
Sample return codes
• sent as ASCII text over control
channel
• USER username
• PASS password
• status code and phrase (as in
HTTP)
• 331 Username OK,
password required
• 125 data connection
already open;
transfer starting
• 425 Can’t open data
connection
• 452 Error writing
file
• LIST return list of file in
current directory
• RETR filename retrieves
(gets) file
• STOR filename stores
(puts) file onto remote host
School of Information Technologies
Secure File Transfer Protocols
• Secure Sockets Layer FTP (SSL-FTP)
– Uses secure sockets layer technology
– All transfers are confidential
• Secure File Transfer Program (sftp)
– Almost nothing in common with FTP
– Uses ssh tunnel
• Secure Copy (scp)
– Derivative of Unix remote copy (rcp)
– Uses ssh tunnel
School of Information Technologies
Trivial File Transfer Protocol
(TFTP)
• Alternative to FTP
• Whole-file copying
• Not as much functionality as FTP
– Code is much smaller
• Intended for use on Local Area Network
• Runs over UDP
• Diskless machine can use to obtain image at
bootstrap
School of Information Technologies
TFTP Packet Types
School of Information Technologies
TFTP Retransmission
• Symmetric
– both sides implement timeout and
retransmission
• Network File System (NFS)
– A protocol for on-line file access, not copying
School of Information Technologies
Applications:
Electronic Mail
(SMTP, POP, IMAP, MIME)
School of Information Technologies
Electronic Mail
Three major components:
•
•
•
user agents
mail servers
simple mail transfer protocol: SMTP
User Agent
• a.k.a. “mail reader”
• composing, editing, reading mail
messages
• e.g., Eudora, Outlook, elm, Netscape
Messenger
• outgoing, incoming messages stored
on server
School of Information Technologies
user
agent
mail
server
SMTP
SMTP
mail
server
user
agent
SMTP
user
agent
user
agent
mail
server
user
agent
user
agent
outgoing
message queue
user mailbox
Electronic Mail: mail servers
user
agent
Mail Servers
• mailbox contains incoming
messages for user
• message queue of outgoing (to
be sent) mail messages
• SMTP protocol between mail
servers to send email messages
– client: sending mail server
– “server”: receiving mail
server
School of Information Technologies
mail
server
SMTP
SMTP
mail
server
user
agent
SMTP
user
agent
user
agent
mail
server
user
agent
user
agent
Electronic Mail: SMTP [RFC 2821]
• uses TCP to reliably transfer email message from client to server,
port 25
• direct transfer: sending server to receiving server
• three phases of transfer
– handshaking (greeting)
– transfer of messages
– closure
• command/response interaction
– commands: ASCII text
– response: status code and phrase
• messages must be in 7-bit ASCII
School of Information Technologies
Scenario: Alice sends message to Bob
4) SMTP client sends Alice’s
message over the TCP
connection
2) Alice’s UA sends message 5) Bob’s mail server places
the message in Bob’s
to her mail server; message
mailbox
placed in message queue
3) Client side of SMTP opens 6) Bob invokes his user agent
to read message
TCP connection with Bob’s
mail server
1) Alice uses UA to compose
message and “to”
bob@someschool.edu
1
user
agent
2
School of Information Technologies
mail
server
3
mail
server
4
5
6
user
agent
Sample SMTP interaction
S:
C:
S:
C:
S:
C:
S:
C:
S:
C:
C:
C:
S:
C:
S:
220 hamburger.edu SMTP Ready
HELO crepes.fr
250 Hello crepes.fr, pleased to meet you
MAIL FROM: <alice@crepes.fr>
250 alice@crepes.fr... Sender ok
RCPT TO: <bob@hamburger.edu>
250 bob@hamburger.edu ... Recipient ok
DATA
354 Enter mail, end with "." on a line by itself
Do you like ketchup?
How about pickles?
.
250 Message accepted for delivery
QUIT
221 hamburger.edu closing connection
School of Information Technologies
Try SMTP interaction for yourself:
• telnet servername 25
• see 220 reply from server
• enter HELO, MAIL FROM, RCPT TO, DATA,
QUIT commands
above lets you send email without using email client
(reader)
School of Information Technologies
SMTP: final words
• SMTP uses persistent connections
• SMTP requires message (header & body) to
be in 7-bit ASCII
• SMTP server uses CRLF.CRLF to
determine end of message
School of Information Technologies
Mail message format
SMTP: protocol for exchanging
email msgs
RFC 822: standard for text
message format:
• header lines, e.g.,
– To:
– From:
– Subject:
different from SMTP commands!
• body
– the “message”, ASCII
characters only
School of Information Technologies
header
body
blank
line
Multipurpose Internet Mail
Extension (MIME)
• Permits nontextual data to be sent in email
– Graphics image
– Voice or video clip
• Sender
– Encodes binary item into printable characters
– Places in email message for transfer
• Receiver
– Receives email message containing encoded item
– Decodes message to extract original binary value
School of Information Technologies
MIME: multimedia extensions
• multimedia mail extension - RFC 2045, 2056
• additional lines in msg header declare MIME content
type and encoding
MIME version
method used
to encode data
multimedia data
type, subtype,
parameter declaration
encoded data
School of Information Technologies
From: alice@crepes.fr
To: bob@hamburger.edu
Subject: Picture of yummy crepe.
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Type: image/jpeg
base64 encoded data .....
.........................
......base64 encoded data
Mail access protocols
user
agent
SMTP
SMTP
sender’s mail
server
access
protocol
receiver’s mail
server
• SMTP: delivery/storage to receiver’s server
• Mail access protocol: retrieval from server
– POP: Post Office Protocol [RFC 1939]
• authorization (agent <-->server) and download
– IMAP: Internet Mail Access Protocol [RFC 1730]
• more features (more complex)
• manipulation of stored msgs on server
– HTTP: Hotmail , Yahoo! Mail, etc.
School of Information Technologies
user
agent
POP3 protocol
authorization phase
• client commands:
– user: declare username
– pass: password
• server responses
– +OK
– -ERR
transaction phase, client:
• list: list message numbers
• retr: retrieve message by
number
• dele: delete
• quit
School of Information Technologies
S:
C:
S:
C:
S:
+OK POP3 server ready
user bob
+OK
pass hungry
+OK user successfully logged
C:
S:
S:
S:
C:
S:
S:
C:
C:
S:
S:
C:
C:
S:
list
1 498
2 912
.
retr 1
<message 1 contents>
.
dele 1
retr 2
<message 1 contents>
.
dele 2
quit
+OK POP3 server signing off
on
POP3 (more) and IMAP
More about POP3
• Previous example uses
“download and delete”
mode.
• Bob cannot re-read e-mail
if he changes client
• “Download-and-keep”:
copies of messages on
different clients
• POP3 is stateless across
sessions
School of Information Technologies
IMAP
• Keep all messages in one
place: the server
• Allows user to organize
messages in folders
• IMAP keeps user state
across sessions:
– names of folders and
mappings between message
IDs and folder name
World Wide Web
(HTTP)
School of Information Technologies
World Wide Web
• Distributed hypermedia paradigm
– various forms of information, as data, text,
graphics, video, and audio, are linked together
by hyperlinks
• Major service on the Internet
• Use surpassed file transfer in 1995
School of Information Technologies
Web Page Identifier
• Known as Uniform Resource Locator (URL)
• Encodes
–
–
–
–
–
–
Access protocol to use
Domain name of server
Protocol port number (optional)
Path through server’s file system (optional)
Parameters (optional)
Query (optional)
• Format
– http: // hostname [: port] / path [; parameters] [? query]
School of Information Technologies
Web Standards
• Separate standards for
– Representation
– Transfer
School of Information Technologies
Representation
• HyperText Markup Language (HTML)
• Document contains text plus embedded
links and formatting
• HTML gives guidelines for display, not
details
• Consequence: two browsers may choose to
display same document differently
School of Information Technologies
Transfer
• Used between browser and web server
• Protocol is HyperText Transfer Protocol
(HTTP)
• Runs over TCP
HTTP vs SMTP:
Pull vs push paradigm
School of Information Technologies
HTTP Characteristics
•
•
•
•
•
•
•
Application level
Request / response paradigm
Stateless
Permits bi-directional transfer
Offers capability negotiation
Support for caching
Support for intermediaries (proxy)
School of Information Technologies
HTTP Operation
• Browser sends requests to which server replies
• Typical request: GET used to fetch document
• Example
GET http://www.it.usyd.edu/current_students/ HTTP/1.1
• Relative URL also permitted
GET /current_students/ HTTP/1.1
School of Information Technologies
Error Messages
• HTTP includes set of error responses
• Server can format error as HTML message
for user or use internal form and allow
browser to format message
School of Information Technologies
Persistent Connections
• HTTP version 1.0 uses one TCP connection per
transfer
–
–
–
–
–
Browser forms TCP connection to server
Browser sends GET request
Server returns header describing item
Server returns item
Server closes connection
• HTTP version 1.1 permits connection to persist
across multiple requests
School of Information Technologies
HTTP Headers
• HTTP uses MIME-like headers to carry
meta information
• Both browsers and servers send headers that
allow them to negotiate agreement on the
document representation and encoding to
be used
School of Information Technologies
Headers And Length Encoding
• HTTP headers use same syntax as email headers
– Lines of text followed by blank line
– Lines of text have form keyword:information
• For persistent connection, header specifies length
(in octets) of data item that follows
Header
Content-Length
Content-Type
Content-Encoding
Content-Language
School of Information Technologies
Meaning
Size of item in octets
Type of item
Encoding used for item
Language(s) used in item
Example Of Header
Content-Length: 34
Content-Language: english
Content-Encoding: ascii
Blank line!
<HTML> A trivial example. </HTML>
• Note: if length is not known in advance, server can inform
browser that connection will close following transfer
Connection: close
School of Information Technologies
Conditional Request
• Allows browser to check cached copy for
freshness
• Eliminates useless latency
• Sends If-Modified-Since in header of GET request
• Example:
If-Modified-Since: Wed, 28 May 2007 08:00:01 GMT
– Avoids transfer if item older than 28 May 2007
School of Information Technologies
Proxy Servers
• Browser can be configured to contact proxy
• Permits caching for entire organization
• Server can specify maximum number of
proxies along path (including none)
School of Information Technologies
Caching Of Web Pages
• Caching essential to efficiency
• Server specifies
– Whether page can be cached
– Maximum time page can be kept
• Intermediate caches and browser cache web pages
• Browser can specify maximum age of page (forces
intermediate caches to revalidate)
School of Information Technologies
P2P file sharing
(Gnutella)
School of Information Technologies
P2P file sharing
Example
• Alice runs P2P client
application on her
notebook computer
• Intermittently connects to
Internet; gets new IP
address for each
connection
• Asks for “Hey Jude”
• Application displays other
peers that have copy of
Hey Jude.
School of Information Technologies
• Alice chooses one of the
peers, Bob.
• File is copied from Bob’s
PC to Alice’s notebook:
HTTP
• While Alice downloads,
other users download from
Alice
• Alice’s peer is both a Web
client and a transient Web
server
All peers are servers = highly
scalable!
Query flooding: Gnutella
• fully distributed
– no central server
• public domain
protocol
• many Gnutella clients
implementing protocol
School of Information Technologies
overlay network: graph
• edge between peer X and
Y if there’s a TCP
connection
• all active peers and edges
is overlay net
• Edge is not a physical link
• Given peer will typically
be connected with < 10
overlay neighbors
Gnutella: protocol
 Query message
sent over existing TCP
connections
 peers forward
Query message
 QueryHit
sent over
reverse
Query
path
QueryHit
Scalability:
limited scope
flooding
School of Information Technologies
File transfer:
HTTP
Query
QueryHit
Gnutella: Peer joining
1.
2.
3.
4.
5.
Joining peer X must find some other peer in Gnutella
network: use list of candidate peers
X sequentially attempts to make TCP with peers on list
until connection setup with Y
X sends Ping message to Y; Y forwards Ping message.
All peers receiving Ping message respond with Pong
message
X receives many Pong messages. It can then setup
additional TCP connections
School of Information Technologies
Summary
• typical request/reply
message exchange:
– client requests info or
service
– server responds with data,
status code
• message formats:
– headers: fields giving info
about data
– data: info being
communicated
School of Information Technologies
• control vs. data msgs
– in-band, out-of-band
• centralized vs.
decentralized
• stateless vs. stateful
• The End!