Electronic Mail
Electronic Mail
• Most heavily used application on any network
• Uses client-server architecture
– Electronic mail client accepts mail from user and
delivers to server on destination computer
– Many variations and styles of delivery
• Simple Mail Transfer Protocol (SMTP)
– TCP/IP
– Delivery of simple text messages
• Multi-purpose Internet Mail Extension (MIME)
– Delivery of other types of data
– Voice, images, video clips
SMTP Concept
User Agents and Mail Transfer Agents
Relay MTAs
Mail Gateways
The Entire E-mail System
SMTP
• Simple Mail Transfer Protocol is standard
application protocol for delivery of mail from
source to destination
• Built on top of TCP: provides reliable delivery of
data
• SMTP does not normally use intermediate mail
servers for sending mail
• SMTP uses persistent connections
• Other functions:
– E-mail address lookup
– E-mail address verification
• Example protocol exchange
Sample SMTP interaction
C:
S:
C:
S:
C:
S:
C:
S:
C:
S:
C:
C:
C:
S:
C:
S:
c:\Telnet hamburger.edu 25
220 hamburger.edu
HELO ce.sharif.edu
250 Hello ce.sharif.edu, pleased to meet you
MAIL FROM: <ali@ce.sharif.edu >
250 ali@ce.sharif.edu... Sender ok
RCPT TO: <bob@hamburger.edu>
250 bob@hamburger.edu ... Recipient ok
DATA
354 Enter mail, end with "." on a line by itself
Do you like ketchup?
How about pickles?
.
250 Message accepted for delivery
QUIT
221 hamburger.edu closing connection
SMTP
• RFC 821
• Not concerned with format of messages or data
– Covered in RFC 822 (see later)
• SMTP uses info written on envelope of mail
– Message header
• Does not look at contents
– Message body
• Except:
– Standardize message character set to 7 bit ASCII
Basic Operation
• Mail created by user agent program (mail client)
– Message consists of:
• Header containing recipient’s address and other info
• Body containing user data
• Messages queued and sent as input to SMTP
sender program
– Typically a server process (daemon on UNIX)
Mail Message Contents
• Each queued message has:
– Message text
• RFC 822 header with message envelope and list of recipients
• Message body, composed by user
– A list of mail destinations
•
•
•
•
Derived by user agent from header
May be listed in header
May require expansion of mailing lists
May need replacement of mnemonic names with mailbox
names
• If BCCs indicated, user agent needs to prepare
correct message format
SMTP Sender
• Takes message from queue
• Transmits to proper destination host
– Via SMTP transaction
– Over one or more TCP connections to port 25
• Host may have multiple senders active
• When delivery complete, sender deletes
destination from list for that message
• When all destinations processed, message is
deleted
Optimization
• If message destined for multiple users on a given
host, it is sent only once
– Delivery to users handled at destination host
• If multiple messages ready for given host, a single
TCP connection can be used
– Saves overhead of setting up and dropping connection
Possible Errors
•
•
•
•
Host unreachable
Host out of operation
TCP connection fail during transfer
Sender can re-queue mail
– Give up after a period
• Faulty destination address
–
–
–
–
User error
Target user changed address
Redirect if possible
Inform user if not
SMTP Protocol - Reliability
• Used to transfer messages from sender to receiver
over TCP connection
• Attempts to provide reliable service
• No guarantee to recover lost messages
• No end to end acknowledgement to originator
• Error indication delivery not guaranteed
• Generally considered reliable
SMTP Receiver
• Accepts arriving message
• Places in user mailbox or copies to outgoing queue
for forwarding
• Receiver must:
– Verify local mail destinations
– Deal with errors
• Transmission
• Lack of disk space
• Sender responsible for message until receiver
confirm complete transfer
– Indicates mail has arrived at host, not user
SMTP Forwarding
• Mostly direct transfer from sender host to receiver
host
• May go through intermediate machine via
forwarding capability
– Sender can specify route
– Target user may have moved
Conversation
• SMTP limited to conversation between sender and
receiver
• Main function is to transfer messages
• Rest of mail handling beyond scope of SMTP
– May differ between systems
Figure 3.9 SMTP Mail Flow
SMTP System Overview
• Commands and responses between sender and
receiver
• Initiative with sender
– Establishes TCP connection
•
•
•
•
Sender sends commands to receiver
e.g. HELO<SP><domain><CRLF>
Each command generates exactly one reply
e.g. 250 requested mail action ok; completed
SMTP Replies
• Leading digit indicates category
–
–
–
–
Positive completion reply (2xx)
Positive intermediate reply (3xx)
Transient negative completion reply (4xx)
Permanent negative completion reply (5xx)
Operation Phases
• Connection setup
• Exchange of command-response pairs
• Connection termination
Connection Setup
• Sender opens TCP connection with receiver
• Once connected, receiver identifies itself
– 220 <domain> service ready
• Sender identifies itself
– HELO
• Receiver accepts sender’s identification
– 250 OK
• If mail service not available, step 2 above
becomes:
– 421 service not available
Mail Transfer
• Sender may send one or more messages to
receiver
• MAIL command identifies originator
– Gives reverse path to used for error reporting
– Receiver returns 250 OK or appropriate fail/error
message
• One or more RCPT commands identifies
recipients for the message
– Separate reply for each recipient
• DATA command transfers message text
– End of message indicated by line containing just period
(.)
Closing Connection
•
•
•
•
Two steps
Sender sends QUIT and waits for reply
Then initiate TCP close operation
Receiver initiates TCP close after sending reply to
QUIT
Electronic Mailboxes and Addresses
• E-mail users have an electronic mailbox into
which mail is deposited
• Usually associated with computer account; one
user may have different electronic mailboxes
• User accesses mail with a mail reader program
• Electronic mailbox is identified by an e-mail
address
mailbox@computer
Internet Mail Addressing
• User portion or mailbox portion is site specific
– Joe
– Seyyed_hamid_1221
– 1234.5678
• Host portion or computer portion is domain name
• Source mail client
– Resolves destination name using DNS
– Contacts mail delivery server at destination
– Copies mail to server
• Destination mail server
– Interprets user name according to local mailbox addresses
– Places mail in appropriate mailbox
E-mail message format
• Simple two-part format
– Header includes delivery information
– Body carries text of message
• Header and body separated by blank line
E-mail Headers
• Lines of text in format keyword: information
• keyword identifies information; information can
appear in any order
• Essential information:
– To: list of recipients
– From: sender
– Cc: list of copy recipients
• Useful information
– Reply-to: different address than From:
– Received-by: for debugging
• Frivolous information:
– Favorite drink: lemonade
– Phase-of-the-moon: gibbous
Data in E-mail
• Original Internet mail carried only 7-bit ASCII
data
• Couldn’t contain arbitrary binary values; e.g.,
executable program
• Techniques for encoding binary data allowed
transport of binary data
• Uuencode: three 8-bit binary values as four ASCII
characters (6 bits each)
– Also carries file name and protection information
– Incurs 33% overhead
– Requires manual intervention
Multipurpose Internet Mail Extension
(MIME)
• Extension to RFC822
• SMTP can not transmit executables
– Uuencode and other schemes are available
• Not standardized
• Can not transmit text including international characters
(e.g. â, å, ä, è, é, ê, ë)
– Need 8 bit ASCII
• Servers may reject mail over certain size
• Translation between ASCII and EBCDIC not standard
• SMTP gateways to X.400 can not handle none text data in
X.400 messages
• Some SMTP implementations do not adhere to standard
– CRLF, truncate or wrap long lines, removal of white space, etc.
MIME
• Multipart Internet Mail Extensions - Extends and
automates encoding mechanisms
• Allows inclusion of separate components – programs,
pictures, audio clips – in a single mail message
• Sending program identifies the components so
receiving program can automatically extract and
inform mail recipient
– Header includes
MIME-Version: 1.0
Content-Type: Multipart/Mixed; Boundary=Mime_separator
– Separator line gives information about specific encoding
Content-Type: text/plain
MIME (cont.)
• MIME is extensible – sender and receiver agree on
encoding scheme
• MIME is compatible with existing mail systems
– Everything encoded as ASCII
– Headers and separators ignored by non-MIME mail
systems
• MIME encapsulates binary data in ASCII mail
envelope
MIME
Overview of MIME
• Five new message header fields
–
–
–
–
–
MIME version
Content type
Content transfer encoding
Content Id
Content Description
• Number of content formats defines
• Transfer encoding defined
Content Types
• Text body
• Multipart
– Mixed, Parallel, Alternative, Digest
• Message
– RFC 822, Partial, External-body
• Image
– jpeg, gif
• Video
– mpeg
• Audio
– Basic
• Application
– Postscript
– octet stream
MIME Transfer Encodings
• Reliable delivery across wide largest range of
environments
• Content transfer encoding field
– Six values
– Three (7bit, 8bit, binary) no encoding done
• Provide info about nature of data
• Quoted-printable
– Data largely printable ASCII characters
– Non-printing characters represented by hex code
• Base64
– Maps arbitrary binary input onto printable output
• X-token
– Named nonstandard encoding
Figure 3.10 Printable Encoding of
Binary Data into Radix-64 Format
Programs as mail recipients
• Can arrange for e-mailbox to be associated with a
program rather than a user’s mail reader
• Incoming mail automatically processed as input to
program
• Example – mailing list subscription administration
• Can be used to implement client-server processing
– Client request in incoming mail message
– Server response in returned mail reply
Mailing Lists and Forwarders
• E-mail addresses can be attached to programs as
well as electronic mailboxes
• Mail exploder or mail forwarder resends copies of
message to e-mail addresses in mailing list
– UNIX mail program sendmail provides mail aliases
– Mailing list processor, e.g. listserv, can also interpret
subscription management commands
Mail Gateways
• Mailing list processing may take significant
resources in large organizations
• May be segregated to a dedicated server computer:
mail gateway
– Provides single mail destination point for all incoming
mail
– Can use MX records in DNS to cause all mail to be
delivered to gateway
Mail gateways and forwarding
• Users within an organization may want to read
mail on local or departmental computer
• Can arrange to have mail forwarded from mail
gateway
• Message now makes multiple hops for delivery
• Hops may be recorded in header
• Forwarded mail may use proprietary (non-SMTP)
mail system
Mail Gateways and E-mail Addresses
• Organization may want to use uniform naming for
external mail
• Internally, may be delivered to many different
systems with different naming conventions
• Mail gateways can translate e-mail addresses
Mailbox Access
• Where should mailbox be located?
• Users want to access mail from most commonly
used computer
• Cannot always use desktop computer as mail
server
– Not always running
– Requires multitasking OS
– Requires local disk storage
• Can TELNET to remote computer with mail
server
Mail Access Protocols
• Typically users do NOT run a server on their local
PC
• User agent accesses a mailbox from a shared mail
server
• Mail access protocol transfers mail from server to
local PC
• 2 popular protocols:
– POP3 (Post Office Protocol v3)
– IMAP (Internet Mail Access Protocol)
• Can’t use SMTP because getting messages is a
pull operation & SMTP is a push protocol.
Mail Access Protocol
• Instead of TELNEt, use protocol that accesses
mail on remote computer directly
• TCP/IP suite includes Post Office Protocol (POP)
for remote mailbox access
– Computer with mailboxes runs POP server
– User runs POP client on local computer
– POP client can access and retrieve messages from
mailbox
– Requires authentication (password)
– Local computer uses SMTP for outgoing mail
Email Protocols
POP3 and SMTP
POP3
• Three phases
– Authorization
– Transaction
– Update
• During a POP3 session, the server maintains state
information
• POP3 downloads mail messages to the local
machine – not good for “nomadic user”
Dialup Access and POP
• POP useful for dialup connection
– User’s computer not always connected
– Can download all mail at once and read off-line
– Can compose mail off-line and mail in one connection
POP3 protocol
authorization phase
• client commands:
– user: declare username
– pass: password
• server responses
– +OK
– -ERR
transaction phase, client:
• list: list message numbers
• retr: retrieve message by
number
• dele: delete
• quit
S:
C:
S:
C:
S:
+OK POP3 server ready
user bob
+OK
pass hungry
+OK user successfully logged
C:
S:
S:
S:
C:
S:
S:
C:
C:
S:
S:
C:
C:
S:
list
1 498
2 912
.
retr 1
<message 1 contents>
.
dele 1
retr 2
<message 1 contents>
.
dele 2
quit
+OK POP3 server signing off
on
POP3
• Stands for Post Office Protocol – Version 3
• RFC in 1993
• POP3 “is intended to permit a workstation to
dynamically access a maildrop on a server host in
a useful fashion. Usually, this means that the
POP3 protocol is used to allow a workstation to
retrieve mail that the server is holding for it.“
POP3
• Doesn’t provide extensive mail operations on the
server
• Normally mail is downloaded, then deleted
• Deletion doesn’t really occur until QUIT
• Uses port 110
Minimal POP3 Commands
•
•
•
•
•
•
•
•
•
USER name – user’s name
PASS string – user’s password
QUIT – quits, and updates the mailbox
STAT – status
LIST [msg] – lists the mail messages (#’s only)
RETR msg – retrieves a mail message
DELE msg – deletes a message
NOOP – No operation
RSET – resets the mail to its original state
Example POP3 Transaction
Server
Establish the Socket
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
Server is OK
+OK POP3 server ready
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
Establish the Socket
USER bob
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
This user is OK
+OK
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
Send the Password
PASS ITSD4304
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
The Password is OK
+OK
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
Ask how the Server is Feeling…
STAT
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
I’m Fine, Thanks!
+OK 2 320
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
Got Any Mail?
LIST
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
Yup…
+OK 2 messages (320 octets)
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
Message One
1 120
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
Message Two
2 200
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
End
.
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
Get Message One
RETR 1
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
Message One Coming!
+OK 120 octets
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
Message One Coming!
<the POP3 server sends message 1>
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
End of Message 1
.
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
Get Message One
DELE 1
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
Message 1 Gone!
+OK message 1 deleted
Primary
Thread
Socket (110)
Primary
Thread
Client
Server
Note: it’s not really gone…
Get Message One
QUIT
Primary
Thread
Server
Socket (110)
Primary
Thread
Client
QUIT – Commits/Updates
+OK bob POP3 server signing off
Primary
Thread
Socket (110)
Primary
Thread
Client
Server
Note: now it’s gone
Socket Disposed
Primary
Thread
Server
Primary
Thread
Client
IMAP
• Internet Mail Access Protocol
• Allows users to manipulate remote mailboxes
• Must maintain info about user (e.g., folder
hierarchy) even after the user quits
• IMAP server is always in 1 of 4 states:
–
–
–
–
Nonauthenticated
Authenticated
Selected
Logout
POP3 (more) and IMAP
More about POP3
• Previous example uses
“download and delete”
mode.
• Bob cannot re-read e-mail
if he changes client
• “Download-and-keep”:
copies of messages on
different clients
• POP3 is stateless across
sessions
IMAP
• Keep all messages in one
place: the server
• Allows user to organize
messages in folders
• IMAP keeps user state
across sessions:
– names of folders and
mappings between message
IDs and folder name
Security
• SMTP, POP3 and IMAP are plain text protocols
• Secured version available to protect the user
– SMTPS
– POP3s
– Need to be configured both on server and client
Stunnel
• Name of an application/service
– Even if the mail server application does not implement
secure version
• What does it do?
– gathers the encrypted message from client
– Decrypts the data
– Sends plain text data using a local connection to
insecure port
• Server application considers it a normal client
STunneling
Send decrypted data to port 25 of the server
Stunnel
Encrypted data
(Command/mail body)
(on server)
Listening to port 995
Server
Client
Web interface
• Nowadays there are many web interface user
agents
• They allow accessing messages from any client
with only a web browser
• However the interface is not as sophisticated as
special mail agents
• But it is very useful if you need to check your
emails from different computers
Summary for Email
• Electronic mail allows quick, asynchronous
communication across entire Internet
• Can attach e-mail addresses to programs for processing
– Mailing lists
– Other client-server applications
• Simple Mail Transfer Protocol (SMTP) is Internet standard
for mail delivery
• Mail gateways
– Provide uniform user addressing outside organizations
– Translate from Internet mail (e.g. SMTP) to proprietary systems
• Post Office Protocol (POP) and Internet Mail Access
Protocol (IMAP) allow remote access to electronic
mailboxes.