Workbook 10
Network Applications
Pace Center for Business and Technology
1
Chapter 1. An Introduction to TCP/IP Networking
Key Concepts
• Most Linux networking services are designed around a client-server relationship.
• Network server applications are generally designed to be "always running",
starting automatically as a system boots, and only shutting down when the system
does. Generally, only the root user may manage server processes.
• Network client applications are generally running only when in use, and may be
started by any user.
• Most Linux network servers and clients communicate using the TCP/IP protocol.
• The TCP/IP address of both the client process and the server process consists of an
IP address and a port.
• Network servers usually use assigned, "well known" ports, as cataloged in the
/etc/services file. Network clients generally use randomly assigned ports. Often,
well know ports reside in the range of privileged ports, below port number 1024.
• The hostname command can be to examine a machine's current IP address, while
the netstat -tuna command can be used to examine all open ports.
2
Clients, Servers, and the TCP/IP Protocol
Most networking applications today are designed around a client-server relationship.
The networking client is usually an application acting on behalf of a person trying to
accomplish a particular task, such as a web browser accessing a URL, or the rdate
command asking a time server for the current time. The networking server is generally
an application that is providing some service, such as supplying the content of web
pages, or supplying the current time.
The design of (applications acting as) network clients and (applications acting as)
network servers differs dramatically. In order to appreciate the differences, we will
compare them to the agents in a more familiar client-server relationship, a customer
buying a candybar from a salesperson.
3
The Server
Servers are Highly Available
Just as a salesperson must always be running the register, even when customers are
not around, processes implementing networking services need to be running, ready to
supply services upon request. Usually, processes implementing network services are
started at boottime, and continue to run until the machine is shutdown. In Linux (and
Unix), such processes are often referred to as daemons. Generally, only the root user
may start or shutdown processes acting as network servers.
Servers have Well Known Locations
In addition to being available when a customer wants service, salespeople stay where
they know customers can find them. Just as customers can look up the locations of
unfamiliar candybar salesmen in telephone books and find them by street address,
networking clients can look up the locations of unfamiliar network servers using a
hostname, which is converted into the IP Address used to access the service.
4
The Server
The Client
In contrast, the candybar customer needs to be neither highly available nor well
known. A customer doesn't hang around a store from dawn until dusk just in case he
decides he wants a candybar. Likewise, clients generally do not need to to stay at well
known locations. Our customer doesn't stay at home all day, just in case somebody
wants to come by to sell him a candybar.
Processes that implement network clients are started by normal users, and generally
are only running as long a necessary to complete a task. When someone breaks for
lunch, he closes his web browser. Also, client applications do not need to have fixed
addresses, but can move from place to place. More on this next.
5
The Server
TCP/IP Addresses
Every process which is participating in a TCP/IP conversation must have an IP Address,
just as every participant in a phone conversation must have a phone number.
Additionally, every process in a TCP/IP conversation must have port number, whose
closest analogy might be a telephone extension associated with a phone number.
Computers on a network are identified by IP address. The IP address comes in the
form of four integers, each ranging from 0 to 255 (not coincidentally, the amount of
information that can be encoded in one byte of memory), and traditionally written
separated by periods, such as 192.168.0.3. This representation is often informally
referred to as a dotted quad.
Processes on computers are identified by a port number, which is an integer ranging
from 1 to 65535 (not coincidentally, the amount of information that can be encoded in
two bytes of memory). Whenever a process wants to participate in a TCP/IP
conversation with another process, it must first be assigned a port number by the
kernel.
The TCP/IP protocol allows two processes, each identified by a combination of an IP
address and port number, to locate one another. The IP address is used to locate the
machine that the process is running on (this is the "IP" part of the protocol), and the
port number is used to locate the correct process on the machine (this is the "TCP"
6
part).
Sockets
In order to illustrate a typical TCP/IP transaction, we will examine the conversation
between a fictitious student's mozilla web browser, running on the machine
station3.example.com, which translates into an IP address of 123.45.67.89, and the
httpd web server running at academy.redhat.com, which translates into an IP address
of 66.187.232.51. The process usually resembles the following.
As the machine academy.redhat.com is booted, the httpd process is started. It first
allocates a socket, binds it to the port 80, and begins listening for connections.
At some point later, perhaps measured in minutes, perhaps days, the mozilla process
is started on the machine station3.example.com. It also allocates a socket, and
requests to connect to port 80 of the machine 66.187.232.51. Because it did not
request a particular port number, the kernel provides a random one, say 12345. As it
requests the connection, it provides its own IP address and (randomly assigned) port
number to the server.
The server chooses to accept the connection. The established socket is now identified
by the combined IP address and port number of both the client and server.
7
Sockets
Once the socket is established, the mozilla process and the httpd process can read
information from and write information to one another as easily as reading and
writing from a file. (Remember... "everything is a file", even network connections! For
most practical purposes, a socket is just another file descriptor.) The highlighted verbs
in this section, bind, listen, connect, accept, and even read and write, are well defined
terms in Linux (and Unix). They also are the names of the programming system calls
that implement each step.
8
Sockets
In the TCP/IP protocol, a socket is defined by the combined IP address and port
number of both the server and the client. For example, what if our student was
running multiple versions of Mozilla, each making requests of academy.redhat.com, or
what if multiple users were using the machine station3.example.com simultaneously,
all accessing academy.redhat.com? How would the web server keep straight which
conversation it was having with which client? Each client process would be assigned a
distinct port number, and therefore converse with the httpd daemon using a distinct
socket.
9
More about Ports
Well Known Services and the /etc/services File
In our example, we mentioned that the httpd process requested to bind to port 80,
and in turn the mozilla process requested to connect to port 80 of the server. How did
each agree that port 80 was the appropriate port for the web server? Traditional
Internet services, such as a web server, or ftp server, or SMTP (email) server, are
referred to as well known services. On Linux (and Unix) machines, a catalog of well
known services, and the ports assigned to them, is maintained in the file /etc/services.
Notice that both the client and server need to agree on the appropriate port number,
so the /etc/services file is just as important on the client's machine as on the server's.
Just because a service is listed in the /etc/services file does not mean that you are
implementing (or even capable of implementing) that well known service.
10
Privileged Ports
Unlike clients, processes implementing network servers generally request which port
they would like to bind to. Only one process may bind to a port at any given time (Why
must this be?). Otherwise, how would a client distinguish which process it would like to
connect to? Ports less than 1024 are referred to as privileged ports, and treated
specially by the kernel. Only processes running as the root user may bind to privileged
ports. (This helps ensure that, if elvis had an account on the machine
academy.redhat.com, he couldn't start up a rogue version of a web server which might
hand out false information.) Originally, the well known ports and the privileged ports
were meant to coincide, but in practice there are more well known ports than
privileged ones.
11
Determining Current TCP/IP Networking Services
Using hostname to Display the Current IP Address
The hostname command, without arguments, displays a machine's current hostname.
With the -i command line switch, machine's IP address is displayed instead.
What if there are multiple IP addresses?
The design of the hostname command is a little misguided, because machines can
easily have more than one IP address (one for each of multiple network interface
cards, for instance.) In such situations, there is no reason why any one IP address
should be privileged over the others. For historical reasons, however, the kernel keeps
track of a parameter it refers to as its hostname, and the hostname -i command
displays the IP address associated with it.
12
Using the netstat Command to Display Open Ports
When a port is used by a socket, it is referred to as an open port. The netstat
command can be used to display a variety of networking information, including open
ports.
Unfortunately, when called with no command line switches, the netstat command's
output is overwhelmed by the least interesting information, local "Unix" sockets that
are used to communicate between processes on the same machine. When called with
the following command line switches, however, more interesting information is
displayed.
13
Using the netstat Command to Display Open Ports
Many more switches are available, as a quick look at the netstat(8) man page will
reveal. The above switches were chosen from the many, not only because when
combined, they produce interesting output, but also because they are easy to
remember: just think "tuna".
When invoked with the above switches, netstat's output is akin to the following.
14
Using the netstat Command to Display Open Ports
Of the many lines, we may weed away those associated with the udp protocol, and
focus instead on columns 4, 5, and 6 of the rows associated with tcp.
15
Special IP Addresses
16
Understanding IPv6 Addresses
The IP protocol which we discussed above, with friendly IP addresses such as
"66.187.232.51", is known as Internet Protocol version 4, or "IPv4", and is by far the
dominant IP protocol in use today. With the IPv4 protocol, you can have around 4
billion distinct IP addresses. When the IPv4 protocol was developed in the early
1970's, this was plenty of addresses, and the IPv4 protocol has served us well. By
learning a few tricks such as masquerading, where many machines can "hide behind"
a single public IP address, we still manage to live with it well. As appliances become
more intelligent, however, and networking (particularly wireless networking) becomes
cheaper, we are approaching a day where every toaster is going to want its own IP
address, and 4 billion address wont' be enough. Changes are on the horizon.
These changes are coming in the form of Internet Protocol Version 6, or "IPv6". Among
many other changes, the IPv6 protocol takes the obvious step, and makes IP addresses
bigger. Four times bigger. An IPv6 address, in full form, looks like
fe80:0000:0000:0000:0211:22ff:fe33:4411. Instead of writing addresses as familiar
decimal numbers (like "127"), they are written in hexadecimal numbers (like "fe33").
17
Understanding IPv6 Addresses
Some merciful conventions in representing IPv6 addresses help.
Once within an address, a series of zero segments, such as 0000:0000:0000, can be
replaced with a double colon (::).
For any segment which begins with leading zeros, such as 0211, the leading zeros can
be dropped.
With these two shortcuts in mind, the above address could be written a little more
friendly as fe80::211:22ff:fe33:4411.
18
Understanding IPv6 Addresses
Red Hat Enterprise Linux is getting ready for a future transition to IPv6, and many
applications can already handle either IPv4 or IPv6 connections. For our purposes,
because it is not yet widely adopted, we're going to ignore IPv6 as much as possible.
Fortunately, there's just a few forms of IPv6 addresses we need to be able to
recognize, summarized in the table below.
Whenever these forms of IPv6 address are encountered ( ::ffff:192.168.0.1, ::1, and ::),
you can simply think of them as their IPv4 equivalents ( 192.168.0.1, 127.0.0.1, and
19
0.0.0.0, respectively).
Understanding IPv6 Addresses
Listening Sockets
Listening are sockets are sockets owned by a server, before any clients have come
along. For example, at the end of step one of our sample TCP/IP connection above, the
httpd process would have a socket open in the listening state. Notice in the above
output that for listening sockets, only the local half of the address is defined.
Established Sockets
As the name implies, established sockets have both a client process and a server
process with established communication.
Pulling it All Together
20
Understanding IPv6 Addresses
These two sockets are listening for connections, but only on the loopback address.
They must belong to services expecting to receive connections from other processes
on the local machine, but not from other machines. To determine what services these
ports belong to, we do some greping from the /etc/services file.
Apparently, whatever process has claimed port 25 is listening for email clients. This is
probably the sendmail daemon. The process listening on port 631 is listening for print
clients. This is probably the cupsd printing daemon. Both of these services are
discussed in more detail in this Workbook.
21
Understanding IPv6 Addresses
These lines reflect both halves of an established connection between two processes,
both on the local machine (notice the loopback IP address for both of them). The first
is bound to port 59330 (probably a randomly assigned client port), and the second to
the port 631. Some process on the local machine must be communicating with the
cupsd daemon.
Our final extracted lines represent established connections between clients on remote
machines, and services on our local machine's external interface (192.168.122.156).
The first is a connection to an IPv6 aware service on port 22. Again, we try a little
grepping to look up the well known service associated with port 22.
Apparently, this line represents an active connection between a ssh client on a remote
machine with IP address 192.168.122.1, and a sshd daemon on our local machine.
The latter is a connection to an IPv4 only service bound to port 653, probably an NFS
related service.
22
Online Exercises
Chapter 1. An Introduction to TCP/IP Networking
Online Exercises
Lab Exercise
Objective: Gain familiarity with TCP/IP configuration and activity.
Estimated Time: 10 mins.
Specification
1. Create the file ~/lab11.1/ipaddr, which contains your machine's IP address, as
reported by the hostname command, as its single word.
2. Create the file ~/lab11.1/listening_ports, which contains a list of all ports less then
1024 on your current machine which are open in the listening state, one port per
line.
Deliverables
1. The file ~/lab11.1/ipaddr, which contains your machine's current IP address (as
reported by the hostname command) as its single word.
2. The file ~/lab11.1/listening_ports, which contains a list of all ports less then 1024
on your current machine which are open in the listening state, one port per line.
23
Using Email
The Simple Solution
The simplest solution is appropriate for people using computers which have
permanent, well known Internet connections. At the time that many of the
protocols defining how email delivery should be handled were developed,
this was more of the standard instead of the exception. The solution involves
two separate applications whose roles are identified by TLA's (Three Letter
Acronyms). The first application is referred to as the MTA, or Mail Transport
Agent; the second is known as the MUA, or Mail User Agent.
24
The MTA (Mail Transport Agent)
The MTA generally operates in the background, performing the work of the
local post office. The MTA receives email to be delivered from programs on
the local machine, determines from the recipient's address the appropriate
machine to contact, and attempts to connect to a complementary MTA
running on the recipients machine, who should be listening to port 25. If the
sender's MTA cannot contact the receiver's MTA, the mail is spooled on the
sender's machine, and the sender's MTA tries again at a later time.
The MTA also binds to the local port 25, where it receives connections from
other MTA's. When it receives mail from a remote MTA which is destined for a
user on the local machine, it receives the mail, and stores it in a mail spool
which is referred to the user's inbox. In Linux (and Unix), the default inbox for
a user is /var/spool/mail/username, so that mail awaiting the user elvis would
be stored in the file /var/spool/mail/elvis.
The default MTA in Red Hat Enterprise Linux is a daemon called sendmail.
25
The MUA (Mail User Agent)
The MUA is the application most user's think about when they think about
email. The MUA retrieves delivered mail from a user's mail spool (inbox), and
presents it to the user as "New Mail". The MUA allows users to compose
responses or new mail messages, and passes these messages to the local MTA
for delivery.
Red Hat Enterprise Linux ships with a wide selection of MUAs, several of
which will be introduced in this and the next lesson.
26
Mailbox Servers
While simple, the previous solution requires that, first of all, users are
receiving email on a machine that maintains a persistent Internet connection,
and secondly, the machine has a well known host name and is accessible to
others. Many users, such as people using a "dial up" or "High Speed"
connection from an ISP (Internet Service Provider), or people using machines
behind an institution's firewall, are not in this situation. Another solution has
evolved to serve people in these situations: Mailbox servers.
27
Mailbox Servers
In the diagram above, we assume that elvis is using a "high speed" Internet
connection which he has subscribed to from the company "ISP.Net". When he
connects to the Internet, his ISP issues his machine an IP address, but elvis cannot
predict ahead of time which IP address he will receive. The hostname which the ISP
has assigned to the IP address is probably unattractive, such as dhcp-19193.isx.isp.net, so even if elvis were guaranteed to receive the same IP address each
time, he would not want to advertise his machine's hostname as his email address.
Instead, elvis takes advantage of an "email account" which his ISP offers him. Very
likely, this account exists on a Linux or Unix machine owned by the ISP, which has a
permanent connection to the Internet, and is assigned a hostname such as
pop.isp.net. The ISP has arranged all mail destined to addresses of the form
user@isp.net to be delivered to the MTA running on this machine. When the MTA
running on pop.isp.net receives mail for elvis, it stores the delivered mail into a mail
spool dedicated to elvis (very likely in the file /var/spool/mail/elvis), and the email is
considered delivered.
Because the ISP's machine pop.isp.net has a permanent, well known connection to the
Internet, it is a much better candidate for receiving email than elvis's machine at
home. Sitting at home, however, elvis still needs access to the email sitting in his
"inbox" on pop.isp.net. Usually, this access is provided in the from of a POP (Post
Office Protocol) or IMAP (Internet Mail Access Protocol) server.
28
Mail Servers
POP servers
POP servers perform a very simple service. They allow users to access the user's single
mail spool, and transfer its contents to their local MUA. POP servers generally bind to
port 110, and require that any POP client authenticate itself using a
username/password pair. Most modern MUA's act as POP clients, and can be
configured to retrieve mail from a specified POP server.
If a mailbox server is implementing the POP service, it usually implies that the mailbox
server is not intending to provide permanent storage for a user's email, but instead
just cache it temporarily until the user can "pop" it down to a local machine.
IMAP servers
IMAP servers generally provide clients with more sophisticated mailbox management.
IMAP users may maintain several folders on the mailbox server, not just their
individual inbox. In general, an IMAP server implies that a user's email is intended to
be permanently stored on the mailbox server, and users will occasionally connect with
a MUA from a remote machine to "browse" their mail. Generally, IMAP servers are
found in institutional and corporate environments. IMAP daemons bind to port 143.
29
Mail Servers
Sending Mail
Because elvis's machine maintains an almost continuous connection to the Internet,
he is willing to still use his local MUA to deliver mail. If his local MUA is temporarily
unable to connect to the recipient's machine, the MUA will spool the mail locally and
try again later.
Red Hat Enterprise Linux Default Configuration
By default, Red Hat Enterprise Linux is configured appropriately for this scenario. The
local MTA is started, but it will not accept connections over port 25 (except for from
the loopback address, 127.0.0.1). It serves merely to deliver outgoing email. Users are
assumed to be accessing their email from a POP or IMAP server.
The default configuration can be changed, but the necessary configuration is beyond
the scope of this course.
30
Outgoing Mail Servers
For machines that have only transient connections to the Internet, attempting to
deliver email using the local MTA may not be appropriate. Instead, users of "dial up"
connections and the like often use what is referred to as an SMTP (Simple Mail
Transfer Protocol) Server, or outgoing mail server. Many ISPs and institutions provide
outgoing SMTP servers, often with names like smtp.isp.net.
The MTA on the SMTP server is willing to accept mail from "local" machines, even though it is not
the final recipient for the email. Instead, the SMTP server relays the mail, forwarding it on to its
destination. If any temporary problems occur, the spooling and redelivery attempts of the
pending mail is now the responsibility of the SMTP server.
Many MUA's allow users the option of specifying a remote host to act as the user's SMTP server
(as opposed to forwarding mail to the local MTA for delivery). [1]
31
Local Delivery
All of the above scenarios assume that a user is sending email from his machine to a
recipient on a remote machine. Linux (and Unix) also allows users on a local machine
to deliver email to one another, where email is addressed simply to a username, such
as blondie.
For local delivery, no POP servers or SMTP servers are required, as the email is
instantly delivered by the local MTA.
32
Local Delivery
Sending Email with mail
The mail command can be used to deliver mail to recipients who are specified as
arguments on the command line. The body of the message is read from standard in
(which may be read from a pipe, a redirected file, or from the keyboard directly, where
CTRL+D ("EOF") is used to indicate the end of the message).
The command line switches in the following table can be used to specify a subject line,
recipients to "Cc:", etc.
As an example, in the following, elvis mails to blondie the contents of the file lyrics.txt.
33
Using mail to Read Mail
The mail command can also be used to read mail from a user's local mail spool. The
interface is primitive, however, and usually other MUA's are used instead. Details can
be found in the mail man page. These days, the most common use of the mail
command is as a quick and easy way to send mail.
In the following example, blondie is using the mail command, without arguments, to
view the mail in her inbox, and then delete the message and quit.
34
Chapter 6. Network Diagnostic Applications
Key Concepts
• /sbin/ifconfig displays local IP configuration.
• ping confirms low level network connectivity
between hosts.
• host makes direct DNS queries.
• The netstat -tuna command lists currently active
network services and connections.
• /sbin/traceroute can diagnose routing problems.
35
Required Configuration for the Internet Protocol
IP Address
Linux (and Unix) represents every networking device attached to a machine (such as
an Ethernet card, a Token Ring card, a modem used for dialup connections, etc...) as a
network interface. Before an interface can be used to send or receive traffic, it must be
configured with an IP address which serves as the interface's identity.
Default Gateway
The mechanics of the IP protocol organizes machines into sub networks, or
subnets. All machines on a given subnet may exchange information directly. IP
subnets are in turn linked to other subnets by machines acting as routers. A
router has multiple network interfaces, usually each participating in a distinct
subnet. In order to communicate with a host on another subnet, the data
must be passed to a router, which (with the help of other routers) routes the
information to the appropriate subnet, and from there to the appropriate
host.
In order to communicate with machines outside of your local subnet, your
machine must know the identity of a nearby router. The router used to route
packets outside of your local subnet is commonly referred to as a default
gateway.
36
Required Configuration for the Internet Protocol
Nameserver
Other machines on the Internet are in turn identified by their IP address.
People tend to prefer to think in terms of names instead of numbers,
however, so a protocol has been developed to assign names to IP addresses.
The protocol is called Domain Name Service, or more commonly DNS.
The DNS protocol requires that every machine have available one or more
DNS servers (commonly called nameservers), which can serve as both a
database for assigning name to local IP addresses, and also a starting point for
determining IP addresses for domain names for which the server does not
have direct knowledge.
37
Determining Your IP Address(es): /sbin/ifconfig
A previous Lesson introduced the hostname -i command, which displays the
IP address of your local machine. In reality, a "machine" does not have an IP
address, a machine's network interfaces do. This Lesson discusses the topic of
IP addresses and network interfaces in more detail.
In Linux (and Unix), every network device is represented as a network
interface. (For once, we encounter something that is not a file!) Linux names
interfaces according to the type of device it represents. The following table
lists some of the more commonly encountered interface names used in Linux.
In each case, n is replaced with a distinct integer for each instance of a given
device attached to a machine.
38
Confirming Network Interface Configuration
The ifconfig command displays the configuration of all active network
interfaces. Because the command is generally used by root to configure
interfaces, it lives within the /sbin directory, which is outside of the default
PATH for standard users. Standard users can use the command to view
interface configuration information, however, by using an absolute reference,
as in the following example.
39
Confirming Network Interface Configuration
The ifconfig command displays a stanza of IP configuration information and
usage statistics for each active network interface. In most situations, users
should expect to see two stanzas reported. One stanza contains the
configuration for an attached Ethernet card, while the other shows the
configuration for the virtual loopback device. The important line is the second
line, which displays the IP address assigned to the interface. If the line
containing the IP address is missing, or if the IP address does not look
reasonable for your local network configuration, you can expect to have
40
trouble accessing the network.
Determining Your Default Gateway: /sbin/route
As mentioned in this Lesson's introduction, communicating with hosts on
your local subnet uses different procedures than communicating with hosts
on a separate subnet. The Linux kernel, like all Unix kernels, maintains an
internal table which defines which machines should be considered local, and
what gateway should be used to help communicate with those machines
which are not. This table is called the routing table.
If you are a standard user, the route command can be used to display the
system's routing table. If you are the root user, the command can be used to
manipulate the table as well. Like the ifconfig command, the route command
lives in the /sbin directory, so standard users should invoke it using an
absolute reference.
41
Determining Your Default Gateway: /sbin/route
A standard routing table displays two types entries. The first type defines
which subnets should be considered local subnets. In general, there should be
one line specifying a subnet for each active interface. In the output above, the
first line defines the subnet associated with the Ethernet interface (with an IP
address of 192.168.0.51), and the second line defines the subnet associated
with the loopback interface (with an IP address of 127.0.0.1).
The second type of entry, which is used to define gateways, is more relevant
to our discussion. Gateway entries can be distinguished because a host is
defined in the second column ("Gateway"), and the fourth column ("Flags")
contains a "G". Every routing table should contain an entry for a "default"
gateway, an the second column should contain the gateway's hostname.
42
Determining Your Default Gateway: /sbin/route
The same information can be displayed using IP addresses instead of
hostnames using /sbin/route -n.
43
Determining Your Nameserver(s)
Domain Name Service allows users to refer to networked computers using
hostnames instead of IP addresses. Unlike the other two aspects of network
configuration, a nameserver is to some extent optional. In order to
communicate with other machines at all, your host must have an IP address.
In order to communicate with machines outside of your subnet, you must
have a default gateway. If users are willing to refer to every machine by IP
address instead of hostname, however, your machine can communicate using
the IP protocol without ever consulting a nameserver.
In practice, however, nameservers seem more of a necessity. (Do you have an
easier time remembering academy.redhat.com, or 66.187.232.51?)
Converting a hostname into an IP address is often referred to as resolving an
address, and the library which implements nameservice is called the resolv
library. When the library attempts to resolves an address, there are generally
two resources available.
44
Static DNS Configuration: /etc/hosts
The first resource is a simple text file called the /etc/hosts file. While only
root can edit the file, any user can observe it.
The format of the file is as simple as it looks. The first token on a line should
be an IP address, and subsequent tokens are hostnames which should resolve
to the IP address. The standard Unix comment character (“#”) is also
supported.
If a machine is only communicating with a few machines, or if an
administrator wants to create shortcut hostnames (such as “s”), or if she
would like to override the local nameserver, entries can be added to the
/etc/hosts file using a simple text editor.
Obviously, this technique does not scale well. You cannot expect the local
/etc/hosts file to provide every answer.
45
Dynamic DNS Configuration: /etc/resolv.conf
When the local /etc/hosts cannot provide the answer, the resolv library consults a
nameserver. In order to determine which machine running a nameserver to consult, it
examines the resolv library's configuration file, /etc/resolv.conf.
The /etc/resolv.conf configuration file uses lines which begin with the keyword
nameserver to specify the IP addresses of machines that are running nameservers. If
multiple nameservers are specified, the first one will be used by default. If it is
unavailable, the second will be used, and so on. Accordingly, the first listed
nameserver is sometimes called the primary nameserver, the second listed server the
secondary nameserver, and so on.
Notice that a nameserver does not need to be a nearby machine. Any machine which
is implementing nameservice and allows you to query it can be used as a nameserver.
In practice, using a local nameserver leads to better performance.
46
Network Diagnostic Utilities
Diagnostic utilities that can be used to ensure that the configurations are
working appropriately with your local network environment.
Confirming IP Connectivity: ping
The ping command can be used to confirm IP connectivity between two
hosts. The first argument to ping can be either the hostname or the IP
address of the machine you would like to contact.
The ping command will continue to run until canceled with the CTRL+C control sequence.
If no response is returned from the ping command, you cannot expect any higher level
networking communication to occur. One common culprit is an Ethernet cable which has become
physically disconnected from the machine or from the wall.
Inconsistencies in the rate at which lines are displayed or discontinuities in the sequence
numbers are generally indicative of a congested network, or a noisy connection, and you can
47
generally expect poor network performance of higher level protocols.
Examining Routing: /usr/sbin/traceroute
When connecting to a machine outside of your subnet, your packet is passed
from router to router as it traverses various subnets, until finally the packet is
delivered to the subnet which contains the destination machine. The path of
the packet, as it is passed from router to router, can be traced with the
/usr/sbin/traceroute command.
The traceroute command is generally called with a single argument, the
hostname or IP address of the destination machine.
48
Examining Routing: /usr/sbin/traceroute
Often, you will find that packets take surprising paths to get from one place to
another. The number of routers that your packet passes through is generally
referred to as the number of hops the packet has made. The packet above
took 17 hops to reach its destination.
If your packet cannot complete the first hop, your machine's default gateway
has probably not been correctly defined. If your traceroute terminates within
the first couple of hops (without reaching the final destination), the problem
is a misconfigured local router, and your local network administrator can
probably help solve the problem. If the traceroute peters out more than four
or five hops away, the problem is probably outside of your local network
administrator's control.
49
Performing DNS Queries Manually: host
The host command can be used to perform DNS queries directly. With a single
argument, the host command will simply report the requested DNS
resolution.
If the -a command line switch is included, the host command displays
detailed information about the query performed, and the response received,
in "resource record" format. Additionally, the final line identifies the
nameserver who resolved the request, and the amount of time the resolution
required.
50
Chapter 7 Terminal Based Web and FTP Clients
Key Concepts
• The Elinks terminal HTTP client, invoked as the command links, can be
used to browse web pages from a non graphical environment.
• The curl application is a non-interactive multi-protocol client (including
HTTP, HTTPS, and FTP) which can be used to transfer files in batch mode.
• The wget application is a non-interactive multi-protocol client (including
HTTP, HTTPS, and FTP) which can be used to recursively transfer web
pages, recursing through all pages the web page references.
51
Chapter 7 Terminal Based Web and FTP Clients
Why Terminal Based Clients?
Users are probably familiar with sophisticated graphical applications such a
mozilla for browsing the web, or nautilus for accessing FTP servers. In
addition to these applications, Linux provides a collection of robust command
line base clients as well. Why would someone be tempted to use command
line based clients instead of point and click applications?
The obvious reason: If you are not running an X graphical environment, they
are the only utilities available.
Command line utilities are scriptable. If you are repeatedly pulling
information from the same location, you might consider writing a script to do
the repetitive work for you.
Command line utilities require less resources. If you are trying to perform a
long download on an otherwise busy machines, a curl process which
consumes 4 megabytes of memory is much more attractive than a mozilla
process which consumes 40 megabytes of memory.
Simplicity tends to encourage stability. When downloading large files, such as
700 Megabyte ISO images, many of the large graphical applications can
become unstable. (The list of culprits includes Internet Explorer). Command
52
line clients such as curl tend to produce more stable results.
Starting Elinks
The Elinks text HTTP client (web browser) is designed to render web pages in
a terminal, emulating the layout of a graphical web browser. As much as
possible, tables and forms are rendered to scale. Note that in the following
display of a web page associated with Red Hat Network, frames and tables
are rendered appropriately.
Elinks is started as the links command, specifying a URL to load as an
argument, as in the following.
53
Browsing Web Pages with Elinks
When using links, the screen is active, and links may be navigated using arrow
keys or the mouse. By using the ESC key, a menu is created which allows a
user to access bookmarks, history, and customization dialogs.
Although the -dump command line switch can be used to non-interactively
render a page to standard out, and the -source command line switch will do
the same to the "RAW" content of the URL (often dumping HTML), the
support for scripted interactions is fairly unsophisticated when compared to
other utilities such as curl. The links browser's strength is its ability to provide
a comfortable web browsing experience in a text environment.
54
Configuring Elinks to use a Proxy Server
If the HTTP_PROXY environment variable is set, Elinks will use the specified
proxy server, where (like the Elinks browser) the proxy server can be specified
using the following syntax.
Fetching Web Resources with curl
In contrast the the links command, the curl command line web client makes
no effort to render or interactively browse web pages. Instead, the
development of curl has focused on a robust collection of command line
switches that makes it well suited for the automated retrieval of files which
are published using a web or ftp server. This text would have a hard time
describing curl any more effectively or succinctly than the first two
paragraphs of the curl(1) man page:
55
Starting curl
When called with a URL as its argument, the curl command retrieves the URL,
and dumps its contents to standard out. Additionally, some timing
information is written to standard error, which is discarded in the following
example.
56
Command Line Switches for the curl Command
The following table lists a few of the command line switches which can be
used to modify curl's behavior. Consult the curl(1) man page for a complete
list.
57
Command Line Switches for the curl Command
As an example, the following command line could be used to download the
ISO images for the Fedora Core 1 release:
The following details should be noted about the curl command line.
Notice the use of "brace expansion syntax" in the referenced URL. The curl command
will attempt to download three files, yarrow-i386-disc1.iso, yarrow-i386-disc2.iso, and
yarrow-i386-disc3.iso.
Because of the -O command line switch, the files will be downloaded to the local
directory with the same names.
Notice the time left at the current throughput: 13 hours! If the command needs to be
aborted and later restarted, the -C - switch tells curl to examine any already existing
partial downloads, and pick up the transfer where it left off.
The -L command line switch tells curl to follow any redirects issued by the web server.
(Although not evident from the output, curl was redirected to a FTP server.)
58
Configuring curl to use a Proxy Server
If the http_proxy environment variable is set, curl will use the specified proxy
server, where the proxy server can be specified using the following syntax.
Recursively Downloading Web Resources with wget
The wget command line web client is designed to non-interactively use the
HTTP, HTTPS, and FTP protocols to download resources, possibly recursing to
pages referenced from downloaded pages.
Starting wget
When called with a URL as its argument, the wget command retrieves the
contents of the URL and stores it in a local file in the current working
directory of the same name (or index.html if the URL specified a directory).
The following table lists a few of the command line switches which can be
used to modify wget's behavior. Consult the wget(1) man page for a complete
list.
59
Command Line Switches for the wget Command
wget
60
Command Line Switches for the wget Command
As an example, the following command line could be used to mirror the
content of the http://www.redhat.com/training site.
61
Command Line Switches for the wget Command
And off it goes. After a little observation, it should become obvious that when
called recursively, the wget command often recurses beyond expectations.
After canceling the previous command with a CTRL+C, the following
directories demonstrate that wget strayed beyond the bounds of the
www.redhat.com/training website.
62
Configuring wget to use a Proxy Server
Like the curl command, wget uses the http_proxy environment variable to
specify a proxy server, again using the following syntax.
63
Chapter 8. Remote Shell Commands
Key Concepts
• Remote shell applications allow users to execute arbitrary commands on
remote machines, and have standard out returned locally. Alternately, an
interactive shell can be started.
• The Secure Shell application provides a remote shell, where all
transactions are encrypted, and users can be authenticated by traditional
passwords or using a public key authentication scheme.
• In order to use the public key authentication scheme, a public- private key
pair must be generated with the ssh-keygen command.
• Because Secure shell servers have their own public-private key pairs,
servers can be authenticated to clients as well as clients authenticated to
servers.
64
The Original Remote Shell: rsh and Rhosts Authentication
Remote Shells with rsh
Linux (and Unix) shells are intentionally designed with simple interfaces: they
read input from the standard in stream, and deliver output to the standard
out stream. As a result, the interfaces are easily implemented over network
connections. By simply substituting a TCP socket for a terminal device, a shell
can operate on a remote machine as easily as the local machine. In Linux (and
Unix), applications which provide this functionality are referred to as remote
shells.
The first commonly used Unix remote shell was the simple rsh application. If a
remote machine is properly configured (i.e., it is running the RSH server),
users can use a rsh command line akin to the following to invoke a remote
shell.
65
The Original Remote Shell: rsh and Rhosts Authentication
Translating, this command says "as the user elvis on the host server1, run the
command ls /tmp". The command executes on the remote machine (server1),
but standard out is delivered to the local machine (station). When the
command completes, elvis's prompt implies that he is still on the host station.
If elvis does not specify a command to run, the rsh utility would opens an
interactive shell on the remote host server1. By paying close attention to the
bash prompt in the following excerpt, note which commands execute on
which machine.
66
Rhosts Authentication: ~/.rhosts
In each case, elvis did not need issue a password. Before he could access his
remote account using rsh, however, elvis needed to configure the account to
allow him access from his local machine. For rsh, access control configuration
is as trivial as adding a line to a file. On the remote account, elvis created the
file ~/.rhosts, and added one line containing the hostname and username for
each external account which he wanted to grant access. Additionally, the RSH
server requires that the file's permissions prohibit anyone but the user owner
from reading the file. As the following commands illustrate, elvis has already
configured his .rhosts file on the remote machine.
Authentication which relies on a properly configured ~/.rhosts configuration
file is commonly called rhosts authentication.
67
The Secure Shell
The rhosts authentication method is pitiful. At its essence, it relies on DNS
(Domain Name Service) to authenticate a user. In order to exploit elvis's
~/.rhosts configuration, all someone would have to do is detach the real host
station from the network, and bring up another machine configured with
station's IP address. The fault is not elvis's, but the design of the rhost
authentication protocol.
Additionally, rsh is a plaintext protocol. Exchanging data over a network in
plaintext is essentially the equivalent to sending mail on postcards: anyone
handling the data between here and there is privy to its contents.
The Secure Shell was developed to address both of these shortcomings of the
rsh command, and add significant new capabilities, while still providing all of
rsh's convenience. Assuming the remote machine is running the ssh service
(i.e., the sshd daemon), elvis could invoke a shell on the remote machine with
the following.
68
The Secure Shell
The ssh command's new syntax for specifying the username of the remote
account is slightly easier than is rsh's, although the -l command line switch is
also implemented (in order to be fully backwards compatible).
In the above example, elvis is authenticated by providing a password instead
of configuring a ~/.rhosts file. In the next section, we find that the Secure
Shell can use a more mature public key technique to grant users "password
free" access to an account. When public key authentication is not
implemented, however, ssh falls back to traditional password authentication.
[2]
69
Secure Shell Public Key Authentication
In addition to traditional password authentication, the Secure Shell
application can use public key cryptography to authenticate users. Public key
encryption algorithms relate two large numbers, referred to as "keys", so that
information encrypted with one key can only be decrypted with the other.
Anyone who wants to use public key cryptography must first generate a pair
of keys. Most public key protocols call one key a public key, and the
complementary key a private key. Your public key you treat like your phone
number: you share it with anyone with whom you are willing to
communicate, and may choose to list it in public directories. Your private key,
on the other hand, you share with no one. All of the security provided by
public key protocols relies on the fact that only you know the contents of your
private key.
70
Generating a Public-Private Key Pair: ssh-keygen
When using ssh, a user's public-private key pair can be generated with the
ssh-keygen command. In the following example, elvis uses ssh-keygen to
generate a ssh public-private key pair.
The user elvis was first prompted for the new (private) key's filename, to
which elvis simply hit RETURN to accept the default filename: ~/.ssh/id_rsa.
Next, elvis was given the opportunity to attach a passphrase to his private key.
By hitting RETURN again (twice), elvis chose not to. (We will discuss
passphrases in more detail later.)
71
Generating a Public-Private Key Pair: ssh-keygen
When the command returns, elvis has two new files in his (perhaps newly
created) ~/.ssh directory. The first is his private key, which he shares with no
one. (He certainly doesn't publish it in an online text).
He is free to share his second key (the public key) with anyone whom asks.
72
Allowing Account Access: ~/.ssh/authorized_keys
SSH access to an account is granted by obtaining a copy of the public key of
the person who is to be granted access, and storing it in the account's
~/.ssh/authorized_keys file. Like the ~/.rhosts file, the ~/.ssh/authorized_keys
file, and the whole ~/.ssh directory, must only be readable by the user. How
the copy of the public key is obtained does not matter. It could be emailed,
scped (as discussed in a moment), or transferred from one terminal to
another using the mouse's cut and paste buffer.
73
Allowing Account Access: ~/.ssh/authorized_keys
When handling public keys, however, care must be taken to ensure that the
key is placed in the file with no embedded whitespace, including newlines.
Although too long to be displayed as such, SSH public keys are always stored
as a single line of text. More people can be granted access to an account by
simply appending their public keys to the ~/.ssh/authorized_keys files, one
public key per line.
74
Allowing Account Access: ~/.ssh/authorized_keys
In the following example, elvis uses ssh, redirection, and some carefully
placed quotes to append his public key (on the host station) to the
authorized_keys file in his account on the host serer1.
Okay, so we need to make the .ssh directory first.
Why the quotes? With no quotes, the output of the cat command would have
been appended to the file .ssh/authorized_keys on the local machine. The
quotes serve to pass the redirection syntax "into" the remote shell.
75
Allowing Account Access: ~/.ssh/authorized_keys
Having placed his public key in the ~/.ssh/authorized_keys file on the remote
machine, elvis now expects to be able to examine the contents of the file
without having to issue a password.
Something is amiss, because elvis was again prompted for his password.
Recalling that only the user should be able to access the ~/.ssh directory and
read the ~/.ssh/authorized_keys file, elvis implements the appropriate chmod
command on the remote machine. Afterwords, he is able to observe the new
permissions without having to issue a password.
Success.
76
Public Key Authentication Details
In order to develop an appreciation for the robustness of public key
authentication, we will spend a few moments discussing the protocol. When
the secure shell application implements public key authentication, it uses a
procedure similar to the following. In our discussion, the following symbols
will be used.
77
Public Key Authentication Details
First, the ssh client on the host station requests a connection to the sshd
daemon on the host server1. Upon receiving the connection request, the sshd
daemon looks for a registered public key in the destination account's
~/.ssh/authorized_keys file.
If a relevant public key is discovered, the sshd daemon initiates public key
authentication by generating a random string R. It then encrypts the random
string with elvis's public key P (which it obtains from the
~/.ssh/authorized_keys file), and delivers the encrypted random string P(R)
over the network to the ssh client.
78
Public Key Authentication Details
Upon receiving the encrypted random string P(R), the ssh client uses elvis's
private key S to decrypt it. Once the original random string R is recovered, the
ssh client returns it to the sshd daemon.
79
Public Key Authentication Details
If the sshd daemon receives from the ssh client the same random string with
which it started, the client is authenticated, and the connection is allowed to
continue.
80
Public Key Authentication Details
A couple of aspects of this algorithm deserve mentioning.
The ssh client is authenticated not by a hostname or IP address, and not by a
password, but solely by the possession of the private key. (If the client could
not access the appropriate private key, it would not have been able to decrypt
the encrypted random string passed to it.)
The only information passed over the network is an encrypted random string,
and a random string (the symbols colored red in the accompanying figures).
Anyone eavesdropping on the conversation would not learn anything useful.
In practice, the actual algorithm used is more complicated. But the protocol
outlined above illustrates the most important features of the public key
authentication protocol
81
Transferring Files Securely and Easily: scp
As the previous discussion illustrates, files can be transferred from one
machine to another using ssh with the cat command and careful redirection.
Fortunately, there is an easier and less error prone way: scp.
The scp command uses a syntax almost identical to the cp command, but
either the source file(s) or the destination file can be on a remote machine,
accessed through a specified account. [3] When referring to a file on a remote
machine, the following syntax is used.
The user and host are simply enough the host where the file resides, and the
user whose account is used to access the file. If the file's path begins with a
“/”, it is considered an absolute reference. If not, it is considered relative to
the user's home directory. If no path is supplied, the user's home directory is
assumed.
As an example, the following command line would transfer the /etc/services
file from server1 into the ~/cfg/server1/etc/ directory in elvis's home
directory.
82
Transferring Files Securely and Easily: scp
Because elvis has a properly configured public key authentication with his
account on server1, he is able to transfer the file without issuing a password.
What happens if he tries to transfer the file /etc/shadow?
The user elvis on the host server1 does not have permissions to read the file
/etc/shadow, so the file can naturally not be transferred. If the user elvis
knows the password to the root account on the remote machine, however,
the file could be accessed through it.
Because elvis does not have public key authenticated access to the root
account on server1, ssh used traditional password authentication.
83
Transferring Files Securely and Easily: scp
The -r command line switch (for "recursive") must be specified when copying
an entire directory (and its subdirectories). In the following, elvis recursively
copies the /etc/sysconfig directory from his local machine (station) to the
machine server1's /tmp directory.
As the scp command performs the transfer, it displays transfer timing
information for each file.
84
Secure Shell Host Authentication
The first time the ssh (or scp) client is used to connect to a sshd Secure Shell
server, a message similar to the following is displayed.
If the user answers yes (the only answer which will allow the connection to
continue), the connection proceeds, with the following warning.
On subsequent connections, the message is no longer seen.
85
Secure Shell Host Authentication
The Secure Shell not only authenticates clients to servers, but also servers to
clients, using public key authentication. Just as users can create public-private
key pairs with the ssh-keygen command, the sshd daemon maintains its own
public-private key pair, known as its host key. The first time a ssh client
connects to a sshd daemon, it appends a copy of the remote daemon's public
host key to the local file ~/.ssh/known_hosts.
As the client is used to connect to various machines, the ~/.ssh/known_hosts
file grows, recording one public key for each machine contacted. The next
time the client connects to a host, it silently uses the same public key protocol
used to authenticate users, reversed, to authenticate the host to which it is
connecting.
86
Secure Shell Host Authentication
What if the remote host does not posses the same identity which
complements the public key stored in the client's ~/.ssh/known_hosts file?
The client refuses to connect, and instead issues the following warning.
87
Secure Shell Host Authentication
Often, there is a very reasonable explanation why the server has changed
identity. For example, the server might have been upgraded with a more
recent version of its operating system, and as a result, generated a new host
public-private key pair. If a reasonable explanation for the change in identity is
available, the ssh client can be convinced to connect by removing the
offending line from the ~/.ssh/known_hosts file, and "starting over" by
collecting a new key for the host.
88
Secure Shell Host Authentication
LAB
http://csis.pace.edu/adelgado/rha-030/labs/workbook10/chapter-8/ssh-no-keys.htm
Class Evaluation
\\sfs1\classroom\Evaluations
89