Mail Services
Pre-survey: How many server/client programs (or
services) are needed to send and receive email?
0%
0%
0%
or
e
0%
or
m
0%
4
E.
3
D.
2
C.
1
B.
0
1
2
3
4 or more
0
A.
Mail Services

Three major mail service protocols:

Simple Mail Transfer Protocol


Post Office Protocol


SMTP
POP or POP3
Internet Mail Access Protocol

IMAP or IMAP4
Which one to use?

Outbound Mail

SMTP for outbound email



Port 25 or 2525
Server/repository
Inbound Mail (clients)

POP3 for inbound email


Port 110
IMAP for inbound email

Port 143
What is SMTP?

Simple Mail Transfer Protocol (SMTP) is the
standard protocol for sending emails across the
Internet


SMTP uses TCP port 25 or 2525
Sometimes you may have problems sending messages


ISP may have closed port 25
To determine the SMTP server for a given domain name

MX (Mail eXchange) DNS record is used
IMAP or POP3?


Both protocols receive email
Main differences between IMAP and POP3:

POP3 protocol assumes there is only one client
(computer terminal) that ever connects to the mailbox





Connect
Get/send mail
Disconnect
Read and write email while offline
IMAP protocol allows simultaneous access by multiple
clients (computer terminals)


Allows the same user on the clients
IMAP is also suitable if the mailbox is used by multiple users
Material from: http://en.wikipedia.org/wiki/Smtp
SMTP
SMTP

Simple Mail Transfer Protocol


The de facto standard for e-mail transmissions across the
Internet
Defined in RFC 821 (STD 10)


amended by RFC 1123 (STD 3) chapter 5.
Protocol used today is usually ESMTP


Extended SMTP
Defined in RFC 2821
SMTP

Relatively simple text-based protocol

One or more recipients of a message are specified



Most cases verified to exist
The message text is transferred
Client-server protocol


The client transmits an email message to the server
Either


an end-user's email client, a.k.a. MUA (Mail User Agent),
- or a relaying server's MTA (Mail Transfer Agents)

Which then, in turn, acts as an SMTP client
SMTP


An email client knows the outgoing mail SMTP server from
its configuration
A relaying server typically determines which SMTP server
to connect to by looking up the MX (Mail eXchange) DNS
record for each recipient's domain name


Conformant MTAs (not all) fall back to a simple A record
in the case of no MX


A Record: address record
Some current mail transfer agents will also use SRV records




The part of the email address to the right of the at (@) sign
SRV Records: SeRVice records
More general form of MX
These are not widely adopted
Relaying servers can also be configured to use a smart host
SMTP

The SMTP client initiates a TCP connection

Server's port 25


Unless overridden by configuration
Easy to test an SMTP server using telnet

see following example
SMTP

SMTP is a "push" protocol

Does not allow one to "pull" messages from a
remote server on demand



It only sends to the next or end destination
To “pull” (i.e. receive) a mail client must use
POP3 or IMAP
Another SMTP server can trigger a delivery in
SMTP

Using ETRN
Outgoing mail SMTP server




Email client requires the name or IP address
of an SMTP server as part of its configuration
Server will deliver messages on behalf of the
user
Setting allows for various policies and
network designs
End users connected to the Internet can use
the services of an e-mail provider that is not
necessarily the same as their connection
provider
Outgoing mail SMTP server


Network topology, or the location of a client
within a network or outside of a network, is
no longer a limiting factor for email
submission or delivery
Modern SMTP servers typically use a client's
credentials (authentication) rather than a
client's location (IP address), to determine
whether it is eligible to relay email
Outgoing mail SMTP server

Can use either TCP port 25 (SMTP) or port 587
(Submission) for relaying outbound mail to a mail
server



Established by RFC 2476
Many servers support both
Some servers still support port 465 for legacy secure
SMTP

Preferable to use encryption on standard ports according
to RFC 2487
Outgoing mail SMTP server

Some servers are set to reject all relaying on port 25


Valid users authenticating on port 587 are allowed to
relay mail to any valid address.
Open Relay:


Server that relays all email for all destinations for all
clients connecting to port 25
Generally considered a bad practice worthy of blacklisting
Sample communication of a legal
SMPT Session

After establishing a connection between the sender
(the client) and the receiver (the server)

In the following conversation



everything sent by the client is prefaced with C:
everything sent by the server is prefaced with S:
On most computer systems, a connection can be tested
using the telnet command on the client machine
For example:

telnet www.example.com 25

Opens a TCP connection from the sending machine to the MTA listening on
port 25 on host www.example.com
Sample communication (after connect to server)
S:
C:
S:
C:
S:
C:
S:
C:
S:
C:
C:
C:
C:
C:
C:
C:
C:
S:
C:
S:
220 www.example.com ESMTP Postfix
HELO mydomain.com
250 Hello mydomain.com
MAIL FROM:<sender@mydomain.com>
250 Ok
RCPT TO:<friend@example.com>
250 Ok
DATA
354 End data with <CR><LF>.<CR><LF>
Subject: test message
From: sender@mydomain.com
To: friend@example.com
Hello,
This is a test.
Goodbye.
.
250 Ok: queued as 12345
QUIT
221 Bye
Sample communications

Note: the data the client sends in the HELO and
MAIL FROM commands can be retrieved in
additional headers that the server adds to the
message:


Received and Return-Path respectively.
Optional: (and not shown above) nearly all clients
ask the server which SMTP extensions the server
supports


Uses the EHLO greeting to invoke Extended SMTP
(ESMTP)
These clients use HELO only if the server does not
respond to EHLO
Sample communications


Contemporary clients will use the ESMTP
extension keyword SIZE to inquire of the
server the maximum message size that will be
accepted
Older clients and servers

Try to transfer huge messages that will be
rejected


After wasting the network resources
Including a lot of connect time to dialup ISPs

paid by the minute
Sample communications

Users can determine the maximum size accepted by ESMTP servers


For the sending of giant files or sending with older clients
The user telnets as above, but substitutes "EHLO mydomain.com" for the
HELO command line:








S: 220-serverdomain.com ESMTP {postfix version and
date}
S: 220 NO UCE. {etc., terms of service}
C: EHLO mydomain.com
S: 250-serverdomain.com Hello mydomain.com [127.0.0.1]
S: 250-SIZE 14680064
S: 250-PIPELINING
S: 250 HELP
Here, serverdomain.com declares that it:

Will accept a fixed maximum message size no larger than 14,680,064 octets



Depending on the server's actual resource usage, it may be currently unable to accept a
message this large
Can pipeline
Can do help
Sample communications


In the simplest case, an ESMTP server will declare a
maximum SIZE with only the EHLO user
interaction.
If no number appears after the SIZE keyword, or if
the current message limit must be exactly
determined:


User can further interact by simulating the ESMTP header
of a message with an estimated size.
See External Link RFC 1870
Material From: http://en.wikipedia.org/wiki/Post_Office_Protocol
POP
POP

Local e-mail clients use the Post Office Protocol
version 3 (POP3)



An application-layer Internet standard protocol
Retrieves e-mail from a remote server over a TCP/IP
connection
Many subscribers to individual Internet service
provider e-mail accounts access their e-mail with
client software that uses POP3
POP Overview

POP3 has made earlier versions of the
protocol obsolete


POP (formally referred to POP1 and POP2)
Now, the term POP almost always means POP3
in the context of e-mail protocols
POP Overview

Design of POP3 and its procedures supports endusers with intermittent connections




(e.g. dial-up connection)
Allows users to retrieve e-mail when connected
View and manipulate the retrieved messages without
needing to stay connected
Although most clients have an option to leave mail
on server, e-mail clients using POP3 generally:



Connect
Send all new messages composed on the client
Retrieve all messages stored on the mail server



Save the messages on the user's PC as new unread messages
Delete messages from the server
Disconnect
POP Overview

Clients with a leave mail on server option generally use
POP3 UIDL command




Unique IDentification Listing
Most POP3 commands identify specific messages by their
ordinal number on the mail server
Creates a problem for a client intending to leave messages on
the server
Message numbers may change from one connection to the
server to another

For example if a mailbox contains five messages at last connect, and a different
client then deletes message #3, the next connecting user will find the last two
messages' numbers decremented by one
POP Overview

UIDL provides a mechanism to avoid these numbering
issues


Server assigns a string of characters as a permanent and unique
ID for the message
When a POP3-compatible e-mail client connects to the server


Can use the UIDL command to get the current mapping from these
message IDs to the ordinal message numbers
Client can then use this mapping to determine which messages
it has yet to download


Saves time when downloading
Note: IMAP has a similar mechanism, using a 32-bit UID (Unique IDentifier)
that is required to be strictly ascending.
POP Overview

Advantage of the numeric UID is with large mailboxes


A client can request just the UIDs greater than its previously
stored "highest UID"
In POP, the client must fetch the entire UIDL map
POP Overview

Whether POP3 or IMAP is used to retrieve messages:



E-mail clients typically use the SMTP_Submit profile of
the SMTP protocol to send messages
E-mail clients are commonly categorized as either POP or
IMAP clients, but in both cases the clients also use SMTP
There are extensions to POP3 that allow some clients to
transmit outbound mail via POP3

Known as "XTND XMIT" extensions.

The Qualcomm qpopper and CommuniGate Pro servers and
Eudora clients are examples of systems that optionally utilize the
XTND XMIT methods of authenticated client-to-server e-mail
transmission.
POP Overview

MIME serves as the standard for attachments and
non-ASCII text in e-mail

Although neither POP3 nor SMTP require MIMEformatted e-mail, essentially all Internet e-mail comes
MIME-formatted


POP clients must also understand and use MIME
IMAP, by design, assumes MIME-formatted e-mail
POP Overview

POP3 originally supported only an unencrypted login
mechanism

Plain text transmission of passwords in POP3 still
commonly occurs

POP3 currently supports several authentication methods to provide varying
levels of protection against illegitimate access to a user's e-mail

APOP: Authenticated POP




Uses the MD5 hash function
Attempt to avoid replay attacks and disclosure of the shared secret
Clients implementing APOP include Mozilla Thunderbird, Opera,
Eudora, KMail and Novell Evolution
POP3 clients can also support SASL authentication methods
via the AUTH extension. MIT Project Athena also produced
a Kerberized version
POP Overview

POP3 works over a TCP/IP connection




Uses TCP on network port 110
E-mail clients can encrypt POP3 traffic using TLS or SSL
A TLS or SSL connection is negotiated using the STLS
command
Some clients and servers, like Google Gmail, instead use the
deprecated alternate-port method, which uses TCP port 995
POP Example (APOP)
S: <wait for connection on TCP port 110>
C: <open connection>
S: +OK POP3 server ready <1896.697170952@dbc.mtview.ca.us>
C: APOP mrose c4c9334bac560ecc979e58001b3e22fb
S: +OK mrose's maildrop has 2 messages (320 octets)
C: STAT
S: +OK 2 320
C: LIST
S: +OK 2 messages (320 octets)
S: 1 120
S: 2 200
S: .
C: RETR 1
S: +OK 120 octets
S: <the POP3 server sends message 1>
S: .
C: DELE 1
S: +OK message 1 deleted
C: RETR 2
S: +OK 200 octets
S: <the POP3 server sends message 2>
S: .
C: DELE 2
S: +OK message 2 deleted
C: QUIT
S: +OK dewey POP3 server signing off (maildrop empty)
C: <close connection>
S: <wait for next connection>
POP Example (no auth)

POP3 servers without the optional APOP
command expect you to log in with the
USER and PASS commands:
C: USER mrose
S: +OK User accepted
C: PASS mrosepass
S: +OK Pass accepted
POP3
0%
n
th
a
o
xi
bl
e
en
tt
ef
le
c li
Is
m
or
he
st
lo
w
Al
IM
sc
...
di
se
th
e
lo
n
ai
0%
AP
0%
rv
e.
..
t2
5
sp
or
em
4.
0%
Us
e
3.
av
es
2.
Uses port 25
Leaves email on the
server by default
Allows the client to
disconnect from the
server, then work with
the individual emails
Is more flexible than
IMAP
Le
1.
IMAP
IMAP

Internet Message Access Protocol


Commonly known as IMAP or IMAP4
An application layer Internet protocol




Previously called Internet Mail Access Protocol, Interactive
Mail Access Protocol , and Interim Mail Access Protocol
Current version is IMAP version 4 revision 1


Operates on port 143
Allows a local client to access e-mail on a remote server
IMAP4rev1: defined by RFC 3501
IMAP4 and POP3 are the two most prevalent Internet
standard protocols for e-mail retrieval

Virtually all modern e-mail clients and servers support both
IMAP

IMAP supports both connected and disconnected modes of
operation

E-mail clients using IMAP generally leave messages on the server
until the user explicitly deletes them



IMAP is often used in large networks



Allows multiple clients to access the same mailbox
Clients in this case are different workstations
For example, a college campus mail system
Few Internet Service Providers (ISPs) support IMAP
IMAP4 offers access to the mail “store”

Client may store local copies of the messages


Considered to be a temporary cache
Server's store is authoritative
IMAP

Proprietary protocols:

Microsoft Outlook client


IBM Notes client



When communicating with an Exchange server
When communicating with a Domino server
Both products also support SMTP, POP3, and IMAP4.
Support for the Internet standard protocols



Allows other e-mail clients to access these servers
Allows the clients to be used with other servers
E.g. Qualcomm's Eudora or Mozilla Thunderbird
IMAP

IMAP allows users to access new messages instantly
on their computers


The mail is stored on the network
With POP3, users either




download the e-mail to their computer
access it via the web
Both methods take longer than IMAP
User must either download any new mail or "refresh" the
page to see the new messages
POP – IMAP COMPARISONS
Overview

Most e-mail clients support either POP3 or IMAP to retrieve
messages


Few Internet Service Providers (ISPs) support IMAP
Fundamental differences between POP3 and IMAP4:

POP3 offers access to a mail drop


Mail starts on the server
Downloaded to client when accessed


If the client leaves some or all messages on the server


Deleted from server
The client's message store is considered authoritative
IMAP4 offers access to a mail store


Mail stays on the server after accessed
Client may store local copies of the messages


These are considered to be a temporary cache
The server's store is authoritative
IMAP Advantages over POP3

Connected and disconnected modes of
operation

When using POP3, clients typically connect to
the e-mail server briefly


When using IMAP4, clients often stay connected
as long as the user interface is active


Only as long as it takes to download new messages
Download message content on demand
For users with many or large messages

IMAP4 usage pattern can result in faster response
times.
IMAP Advantages over POP3

Multiple clients simultaneously connected
to the same mailbox


The POP3 protocol requires the currently
connected client to be the only client connected to
the mailbox.
The IMAP protocol specifically allows
simultaneous access by multiple clients

Provides mechanisms for clients to detect changes
made to the mailbox by other, concurrently connected,
clients
IMAP Advantages over POP3

Access to MIME message parts and partial
fetch

Nearly all internet e-mail is transmitted in MIME
format

Allows messages to have a tree structure where



The IMAP4 protocol allows clients to



leaf nodes are any of a variety of single part content types
non-leaf nodes are any of a variety of multipart types
Separately retrieve any of the individual MIME parts
Retrieve portions of either individual parts or the entire
message
Mechanisms allow clients to retrieve the text portion
of a message without retrieving attached files or to
stream content as it is being fetched
IMAP Advantages over POP3

Message state information

flags can keep track of message state


Flags are stored on the server


For example: whether or not the message has been read, replied to, or deleted
Different clients accessing the same mailbox at different times can detect state
changes made by other clients
POP3 provides no mechanism for clients to store such state information on
the server

If a single user accesses a mailbox with two different POP3 clients state
information cannot be synchronized between the clients


IMAP4 protocol supports both pre-defined system flags and client defined
keywords


System flags indicate state information such as whether a message has been read
Keywords allow messages to be given one or more tags whose meaning is up
to the client


E.g. whether a message has been accessed
(not supported by all IMAP servers)
Adding user created tags to messages is an operation supported by some
Webmail services, such as Gmail

Generally not using IMAP
IMAP Advantages over POP3

Multiple mailboxes on the server

IMAP4 clients can create, rename, and/or delete mailboxes on the server,
and move messages between mailboxes



Multiple mailbox support also allows servers to provide access to shared
and public folders
Server-side searches



Usually presented to the user as folders
IMAP4 provides a mechanism for a client to ask the server to search for
messages meeting a variety of criteria
Avoids requiring clients to download every message in the mailbox in
order to perform searches
Built-in extension mechanism


IMAP4 defines an explicit mechanism by which it may be extended
Many extensions to the base protocol have been proposed and are in
common use


IMAP2 did not have an extension mechanism
POP3 now has one defined by RFC 2449
IMAP Disadvantages v. POP

IMAP remedies many of the shortcomings of POP


 inherently additional complexity
Much of this complexity is compensated for by
server-side workarounds



e.g., multiple clients accessing the same mailbox at the
same time
Maildir
Database backends
Disadvantages of IMAP

If the mail store and searching algorithms on
the server are not carefully implemented:

Client can potentially consume large amounts of
server resources when searching massive
mailboxes
Disadvantages of IMAP

IMAP4 clients need to explicitly request new email
message content



Potentially causing additional delays on slow connections
such as those commonly used by mobile devices.
A private proposal, push IMAP, would extend IMAP
to implement push e-mail by sending the entire
message instead of just a notification
However, push IMAP has not been generally
accepted

Current IETF work has addressed the problem in other
ways
Disadvantages of IMAP

Unlike some proprietary protocols which combine sending
and retrieval operations, sending a message and saving a copy
in a server-side folder with a base-level IMAP client requires
transmitting the message content twice, once to SMTP for
delivery and a second time to IMAP to store in a sent mail
folder




Remedied by a set of extensions defined by the IETF LEMONADE
Working Group for mobile devices
POP3 servers don't support server-side folders so clients have no
choice but to store sent items on the client
Many IMAP clients can be configured to store sent mail in a clientside folder
Courier Mail Server offers a non-standard method of sending using
IMAP by copying an outgoing message to a dedicated outbox folder
IMAP
0%
..
lc
lie
ot
al
st
em
ai
us
es
Is
t
he
be
fa
ul
t,
de
By
nt
of
t
on
i
m
to
nt
cli
e
0%
se
...
0%
t..
t2
5
sp
or
sa
4.
0%
Us
e
3.
lo
w
2.
Uses port 25
Allows a client to monitor
email from multiple
workstations at the same
time
By default, uses a lot of the
client’s local disk storage
to hold email
Is the best email client
service since it is a newer
protocol
Al
1.
POP Overview

In contrast, the newer, more capable Internet
Message Access Protocol (IMAP) supports both
connected and disconnected modes of operation.


E-mail clients using IMAP generally leave messages on
the server until the user explicitly deletes them
This and other facets of IMAP operation allow
multiple clients to access the same mailbox
Sidebar

The mail on the server is kept in files in a
directory until read and/or deleted

POP: until “read”


E.g. until downloaded to client
IMAP: until deleted
SMTP Authentication
SMTP-AUTH
SMTP Authentication

SMTP-AUTH

Extension of the Simple Mail Transfer Protocol (SMTP)




Includes an authentication step
Client effectively logs in to the mail server during the
process of sending mail
Servers which support SMTP-AUTH can usually be
configured to require clients to use this extension,
ensuring the true identity of the sender is known.
SMTP-AUTH is defined in RFC 2554
SMTP Authentication

SMTP-AUTH provides an access control
mechanism


Can be used to allow legitimate users to relay
mail while denying relay service to unauthorized
users, such as spammers
Does not guarantee the authenticity of either the
SMTP envelope sender or the "From:" header

Spoofing is possible even with SMTP-AUTH

(When one sender masquerades as someone else)
SMTP Authentication

SMTP-AUTH extension also allows one mail
server to indicate to another that the sender has
been authenticated when relaying mail

This requires the recipient server to trust the
sending server


This aspect of SMTP-AUTH is rarely used on the
Internet
The recipient of an e-mail message cannot tell
whether the sender was authenticated

Use of SMTP-AUTH is only a partial solution to spam
SMTP Authentication

While SMTP-AUTH is generally a security
improvement over unauthenticated SMTP, it can
also introduce a weakness

If authenticated users are allowed to submit messages
from IP addresses where unauthenticated users are not —
that is, if authenticated users are allowed to relay mail —
then an attacker who subverts one user's account is then
able to use the authenticated server as an open mail relay



In such a configuration every user's password becomes a key to
the mail system's security.
Spammers have attacked SMTP-AUTH mail servers by
brute forcing common usernames and passwords
A good password policy can effectively prevent such an
attack
Pop quiz: How many server/client programs (or
services) does an IT pro need to send and receive email?
0%
0%
0%
0%
3
D.
2
C.
1
B.
0
1
2
3
0
A.
Acronyms
Acronym summary









ESMTP - Extended SMTP
ETRN - Extended Turn
IMAP - Internet Mail Access Protocol
MTA - Mail Transfer Agents
MUA - Mail User Agent
MX
- Mail eXchange
POP
- Post Office Protocol
SMTP - Simple Mail Transfer Protocol
UIDL - Unique IDentification Listing