Part Workbook 10.
Network Applications
Table of Contents
1. An Introduction to TCP/IP Networking .............................................................................. 5
Discussion .............................................................................................................. 5
Clients, Servers, and the TCP/IP Protocol ............................................................. 5
The Server .............................................................................................. 5
Sockets ........................................................................................................... 6
More about Ports ............................................................................................. 8
Well Known Services and the /etc/services File .............................................. 8
Privileged Ports ....................................................................................... 8
Determining Current TCP/IP Networking Services .................................................. 8
Using hostname to Display the Current IP Address ......................................... 8
Using the netstat Command to Display Open Ports ......................................... 8
Online Exercises .................................................................................................... 12
Specification .................................................................................................. 13
Deliverables ................................................................................................... 13
Questions .............................................................................................................. 13
2. Linux Printing ............................................................................................................. 16
Discussion ............................................................................................................. 16
Introducing CUPS .......................................................................................... 16
Print Queues .................................................................................................. 17
Browsing Available Print Queues: lpstat ............................................................. 18
Submitting and Managing Jobs: lpr, lpq,and lprm ................................................. 18
Submitting Jobs with lpr .......................................................................... 18
Monitoring Jobs with lpq ......................................................................... 19
Removing Jobs with lprm ........................................................................ 19
Alternate Front End Commands: lp and cancel ............................................. 19
The CUPS Web Interface ................................................................................. 19
Online Exercises .................................................................................................... 20
Setup ............................................................................................................ 20
Specification .................................................................................................. 20
Deliverables ................................................................................................... 21
Questions .............................................................................................................. 21
3. Managing Print Files .................................................................................................... 24
Discussion ............................................................................................................. 24
PostScript ...................................................................................................... 24
Viewing PostScript Documents with the evince Document Viewer. .......................... 24
Decorating Text for Printing with enscript ........................................................... 26
Rearranging PostScript with mpage .................................................................... 27
Converting PostScript to PDF and PDF to PostScript ............................................. 28
Rearranging PostScript with mpage .................................................................... 29
Online Exercises .................................................................................................... 30
Specification .................................................................................................. 31
Deliverables ................................................................................................... 31
Questions .............................................................................................................. 31
4. Email Overview ........................................................................................................... 34
Discussion ............................................................................................................. 34
Using Email .................................................................................................. 34
The Simple Solution ....................................................................................... 34
The MTA (Mail Transport Agent) ............................................................. 35
The MUA (Mail User Agent) .................................................................... 35
Mailbox Servers ............................................................................................. 36
POP servers ........................................................................................... 36
2
Network Applications
IMAP servers ......................................................................................... 37
Sending Mail ......................................................................................... 37
Red Hat Enterprise Linux Default Configuration ........................................... 37
Outgoing Mail Servers .................................................................................... 37
Local Delivery ............................................................................................... 38
The mail MUA .............................................................................................. 38
Sending Email with mail .......................................................................... 39
Using mail to Read Mail .......................................................................... 39
Examples .............................................................................................................. 40
Example 1. Sending mail .................................................................................. 40
Example 2. Simple Mail Transport Protocol .......................................................... 40
Online Exercises .................................................................................................... 40
Specification .................................................................................................. 40
Deliverables ................................................................................................... 41
Questions .............................................................................................................. 41
6. Network Diagnostic Applications .................................................................................... 44
Discussion ............................................................................................................. 44
Required Configuration for the Internet Protocol ................................................... 44
IP Address ............................................................................................ 44
Default Gateway ..................................................................................... 44
Nameserver ........................................................................................... 44
Determining Your IP Address(es): /sbin/ifconfig ................................................... 45
Confirming Network Interface Configuration ............................................... 45
Determining Your Default Gateway: /sbin/route .................................................... 46
Determining Your Nameserver(s) ...................................................................... 46
Static DNS Configuration: /etc/hosts ........................................................... 47
Dynamic DNS Configuration: /etc/resolv.conf .............................................. 47
Network Diagnostic Utilities ............................................................................. 47
Confirming IP Connectivity: ping .............................................................. 48
Examining Routing: /usr/sbin/traceroute ...................................................... 48
Performing DNS Queries Manually: host .................................................... 49
Examples .............................................................................................................. 49
Example 1. Diagnosing Network Difficulties ........................................................ 49
Online Exercises .................................................................................................... 51
Specification .................................................................................................. 51
Deliverables ................................................................................................... 51
Questions .............................................................................................................. 51
7. Terminal Based Web and FTP Clients ............................................................................. 55
Discussion ............................................................................................................. 55
Why Terminal Based Clients? ........................................................................... 55
Browsing Web Pages with Elinks ...................................................................... 55
Starting Elinks ....................................................................................... 55
Configuring Elinks to use a Proxy Server .................................................... 57
Fetching Web Resources with curl ..................................................................... 57
Starting curl ........................................................................................... 57
Configuring curl to use a Proxy Server ....................................................... 58
Recursively Downloading Web Resources with wget ............................................. 58
Starting wget ......................................................................................... 59
Configuring wget to use a Proxy Server ...................................................... 60
Examples .............................................................................................................. 60
Example 1. Using terminal based file transfer utilities ............................................. 60
Online Exercises .................................................................................................... 61
Set Up .......................................................................................................... 61
Specification .................................................................................................. 61
3
Network Applications
Deliverables ................................................................................................... 62
Questions .............................................................................................................. 62
8. Remote Shell Commands .............................................................................................. 65
Discussion ............................................................................................................. 65
The Original Remote Shell: rsh and Rhosts Authentication ..................................... 65
Remote Shells with rsh ............................................................................ 65
Rhosts Authentication: ~/.rhosts ................................................................ 66
The Secure Shell ............................................................................................ 66
Secure Shell Public Key Authentication .............................................................. 67
Generating a Public-Private Key Pair: ssh-keygen ......................................... 67
Allowing Account Access: ~/.ssh/authorized_keys ........................................ 67
Public Key Authentication Details ............................................................. 68
Transferring Files Securely and Easily: scp .......................................................... 70
Secure Shell Host Authentication ....................................................................... 71
Examples .............................................................................................................. 72
Example 1. Accessing a Remote Account ............................................................. 72
Example 2. Configuring Public Key Authentication ................................................ 73
Example 3. Performing Remote Backups with SSH ................................................ 73
Online Exercises .................................................................................................... 74
Specification .................................................................................................. 74
Deliverables ................................................................................................... 74
Questions .............................................................................................................. 75
4
Chapter 1. An Introduction to TCP/IP
Networking
Key Concepts
• Most Linux networking services are designed around a client-server relationship.
• Network server applications are generally designed to be "always running", starting automatically as a
system boots, and only shutting down when the system does. Generally, only the root user may manage
server processes.
• Network client applications are generally running only when in use, and may be started by any user.
• Most Linux network servers and clients communicate using the TCP/IP protocol.
• The TCP/IP address of both the client process and the server process consists of an IP address and a port.
• Network servers usually use assigned, "well known" ports, as cataloged in the /etc/services file.
Network clients generally use randomly assigned ports. Often, well know ports reside in the range of
privileged ports, below port number 1024.
• The hostname command can be to examine a machine's current IP address, while the netstat -tuna
command can be used to examine all open ports.
Discussion
Clients, Servers, and the TCP/IP Protocol
We will start our Workbook on networking applications by introducing the concepts of clients and servers,
and the basics of the TCP/IP protocol. We will speak of the TCP/IP protocol as if it were the only protocol
available on the Internet. This is not the case, but it is the most commonly used protocol, and the concepts
necessary to understand it extend (or reduce) naturally to other protocols as well.
Most networking applications today are designed around a client-server relationship. The networking client
is usually an application acting on behalf of a person trying to accomplish a particular task, such as a web
browser accessing a URL, or the rdate command asking a time server for the current time. The networking
server is generally an application that is providing some service, such as supplying the content of web
pages, or supplying the current time.
The design of (applications acting as) network clients and (applications acting as) network servers differs
dramatically. In order to appreciate the differences, we will compare them to the agents in a more familiar
client-server relationship, a customer buying a candybar from a salesperson.
The Server
Effective salespeople and effective network services share the following characteristics.
Servers are Highly Available
Just as a salesperson must always be running the register, even when customers are not around, processes
implementing networking services need to be running, ready to supply services upon request. Usually,
processes implementing network services are started at boottime, and continue to run until the machine is
5
An Introduction to
TCP/IP Networking
shutdown. In Linux (and Unix), such processes are often referred to as daemons. Generally, only the root
user may start or shutdown processes acting as network servers.
Servers have Well Known Locations
In addition to being available when a customer wants service, salespeople stay where they know customers
can find them. Just as customers can look up the locations of unfamiliar candybar salesmen in telephone
books and find them by street address, networking clients can look up the locations of unfamiliar network
servers using a hostname, which is converted into the IP Address used to access the service.
The Client
In contrast, the candybar customer needs to be neither highly available nor well known. A customer doesn't
hang around a store from dawn until dusk just in case he decides he wants a candybar. Likewise, clients
generally do not need to to stay at well known locations. Our customer doesn't stay at home all day, just
in case somebody wants to come by to sell him a candybar.
Processes that implement network clients are started by normal users, and generally are only running as
long a necessary to complete a task. When someone breaks for lunch, he closes his web browser. Also,
client applications do not need to have fixed addresses, but can move from place to place. More on this next.
TCP/IP Addresses
Every process which is participating in a TCP/IP conversation must have an IP Address, just as every
participant in a phone conversation must have a phone number. Additionally, every process in a TCP/IP
conversation must have port number, whose closest analogy might be a telephone extension associated
with a phone number.
Computers on a network are identified by IP address. The IP address comes in the form of four integers,
each ranging from 0 to 255 (not coincidentally, the amount of information that can be encoded in one byte
of memory), and traditionally written separated by periods, such as 192.168.0.3. This representation is
often informally referred to as a dotted quad.
Processes on computers are identified by a port number, which is an integer ranging from 1 to 65535
(not coincidentally, the amount of information that can be encoded in two bytes of memory). Whenever
a process wants to participate in a TCP/IP conversation with another process, it must first be assigned a
port number by the kernel.
The TCP/IP protocol allows two processes, each identified by a combination of an IP address and port
number, to locate one another. The IP address is used to locate the machine that the process is running
on (this is the "IP" part of the protocol), and the port number is used to locate the correct process on the
machine (this is the "TCP" part).
Sockets
In order to illustrate a typical TCP/IP transaction, we will examine the conversation between a fictitious
student's mozilla web browser, running on the machine station3.example.com, which translates into an IP
address of 123.45.67.89, and the httpd web server running at academy.redhat.com, which translates into
an IP address of 66.187.232.51. The process usually resembles the following.
1. As the machine academy.redhat.com is booted, the httpd process is started. It first allocates a socket,
binds it to the port 80, and begins listening for connections.
2. At some point later, perhaps measured in minutes, perhaps days, the mozilla process is started on
the machine station3.example.com. It also allocates a socket, and requests to connect to port 80 of
the machine 66.187.232.51. Because it did not request a particular port number, the kernel provides a
6
An Introduction to
TCP/IP Networking
random one, say 12345. As it requests the connection, it provides its own IP address and (randomly
assigned) port number to the server.
3. The server chooses to accept the connection. The established socket is now identified by the combined
IP address and port number of both the client and server.
Once the socket is established, the mozilla process and the httpd process can read information from and
write information to one another as easily as reading and writing from a file. (Remember... "everything is
a file", even network connections! For most practical purposes, a socket is just another file descriptor.) The
highlighted verbs in this section, bind, listen, connect, accept, and even read and write, are well defined
terms in Linux (and Unix). They also are the names of the programming system calls that implement each
step.
Figure 1.1. A TCP/IP Socket Between a Mozilla Web Browser and an Apache Server
In the TCP/IP protocol, a socket is defined by the combined IP address and port number of both the server
and the client. For example, what if our student was running multiple versions of Mozilla, each making
requests of academy.redhat.com, or what if multiple users were using the machine station3.example.com
simultaneously, all accessing academy.redhat.com? How would the web server keep straight which
conversation it was having with which client? Each client process would be assigned a distinct port number,
and therefore converse with the httpd daemon using a distinct socket.
Figure 1.2. Multiple TCP/IP Sockets
7
An Introduction to
TCP/IP Networking
More about Ports
Well Known Services and the /etc/services File
In our example, we mentioned that the httpd process requested to bind to port 80, and in turn the mozilla
process requested to connect to port 80 of the server. How did each agree that port 80 was the appropriate
port for the web server? Traditional Internet services, such as a web server, or ftp server, or SMTP (email)
server, are referred to as well known services. On Linux (and Unix) machines, a catalog of well known
services, and the ports assigned to them, is maintained in the file /etc/services.
Notice that both the client and server need to agree on the appropriate port number, so the /etc/
services file is just as important on the client's machine as on the server's. Just because a service is listed
in the /etc/services file does not mean that you are implementing (or even capable of implementing)
that well known service.
Privileged Ports
Unlike clients, processes implementing network servers generally request which port they would like to
bind to. Only one process may bind to a port at any given time (Why must this be?). Otherwise, how
would a client distinguish which process it would like to connect to? Ports less than 1024 are referred to
as privileged ports, and treated specially by the kernel. Only processes running as the root user may bind
to privileged ports. (This helps ensure that, if elvis had an account on the machine academy.redhat.com,
he couldn't start up a rogue version of a web server which might hand out false information.) Originally,
the well known ports and the privileged ports were meant to coincide, but in practice there are more well
known ports than privileged ones.
Determining Current TCP/IP Networking Services
Using hostname to Display the Current IP Address
The hostname command, without arguments, displays a machine's current hostname. With the -i command
line switch, machine's IP address is displayed instead.
[elvis@station elvis]$ hostname
station.example.com
[elvis@station elvis]$ hostname -i
172.16.62.9
What if there are multiple IP addresses?
The design of the hostname command is a little misguided, because machines can easily have
more than one IP address (one for each of multiple network interface cards, for instance.) In such
situations, there is no reason why any one IP address should be privileged over the others. For
historical reasons, however, the kernel keeps track of a parameter it refers to as its hostname, and
the hostname -i command displays the IP address associated with it.
Using the netstat Command to Display Open Ports
When a port is used by a socket, it is referred to as an open port. The netstat command can be used to
display a variety of networking information, including open ports.
Unfortunately, when called with no command line switches, the netstat command's output is overwhelmed
by the least interesting information, local "Unix" sockets that are used to communicate between processes
8
An Introduction to
TCP/IP Networking
on the same machine. When called with the following command line switches, however, more interesting
information is displayed.
Table 1.1. Command Line Switches for the netstat Command
Switch
Effect
-t
Display TCP sockets
-u
Display UDP sockets
-n
Display IP address instead of hostname
-a
Display all sockets, even those in LISTENing state
Many more switches are available, as a quick look at the netstat(8) man page will reveal. The above
switches were chosen from the many, not only because when combined, they produce interesting output,
but also because they are easy to remember: just think "tuna".
When invoked with the above switches, netstat's output is akin to the following.
[elvis@station elvis]$ netstat -tuna
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address
Foreign Address
tcp
0
0 127.0.0.1:2208
0.0.0.0:*
tcp
0
0 0.0.0.0:653
0.0.0.0:*
tcp
0
0 0.0.0.0:111
0.0.0.0:*
tcp
0
0 127.0.0.1:631
0.0.0.0:*
tcp
0
0 127.0.0.1:25
0.0.0.0:*
tcp
0
0 127.0.0.1:6010
0.0.0.0:*
tcp
0
0 127.0.0.1:6011
0.0.0.0:*
tcp
0
0 127.0.0.1:2207
0.0.0.0:*
tcp
0
0 192.168.122.156:653
192.168.122.1:43099
tcp
0
0 :::80
:::*
tcp
0
0 :::22
:::*
tcp
0
0 ::1:6010
:::*
tcp
0
0 ::1:6011
:::*
tcp
0
0 127.0.0.1:631
127.0.0.1:59330
tcp
0
0 127.0.0.1:59330
127.0.0.1:631
tcp
0
0 ::ffff:192.168.122.156:22
::ffff:192.168.122.1:39543
tcp
0
0 ::ffff:192.168.122.156:22
::ffff:192.168.122.1:54577
udp
0
0 0.0.0.0:32768
0.0.0.0:*
udp
0
0 0.0.0.0:647
0.0.0.0:*
udp
0
0 0.0.0.0:650
0.0.0.0:*
udp
0
0 0.0.0.0:68
0.0.0.0:*
udp
0
0 0.0.0.0:5353
0.0.0.0:*
udp
0
0 0.0.0.0:111
0.0.0.0:*
udp
0
0 0.0.0.0:631
0.0.0.0:*
udp
0
0 :::32769
:::*
udp
0
0 :::5353
:::*
State
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
ESTABLISHED
LISTEN
LISTEN
LISTEN
LISTEN
ESTABLISHED
ESTABLISHED
ESTABLISHED
ESTABLISHED
Of the many lines, we may weed away those associated with the udp protocol, and focus instead on columns
4, 5, and 6 of the rows associated with tcp.
Table 1.2. Relevant Columns from netstat's Output
Column
Title
Role
1
Protocol
The protocol of the socket. We will only concern ourselves
with tcp sockets.
4
Local Address
The IP address and port number of the local half of the socket.
5
Foreign Address
The IP address and port number of the remote half of the
socket.
9
An Introduction to
TCP/IP Networking
Column
Title
Role
6
State
The state of the TCP connection. We will only concern
ourselves with the two most interesting states, LISTEN and
ESTABLISHED.
Special IP Addresses
The following types of IP addresses can be found in the Local Address column of the above output.
Role
IPv4 Address
IPv6 Address
Role
Externally
192.168.122.156 ::ffff:192.168.122.156
The external IP address of the machine, to which
Facing Address
remote clients can connect.
Internally
Facing
("loopback")
Address
127.0.0.1
"Any" Address ::
::1
The loopback address. Every machine that uses
TCP/IP networking supports the loopback address.
Connections to and from this address always loop
back to the local machine, so that communication
can occur between two process on the same machine
as if they were on different machines. Conversations
which occur over the loopback address occur
internally; they are not exposed to the network.
0.0.0.0
This special IP address is used to refer to "all
available" IP addresses. When a process binds to a
particular port, it may choose to use only a particular
external address, only the loopback address, or all
available IP addresses.
Understanding IPv6 Addresses
The IP protocol which we discussed above, with friendly IP addresses such as "66.187.232.51", is known
as Internet Protocol version 4, or "IPv4", and is by far the dominant IP protocol in use today. With the IPv4
protocol, you can have around 4 billion distinct IP addresses. When the IPv4 protocol was developed in
the early 1970's, this was plenty of addresses, and the IPv4 protocol has served us well. By learning a few
tricks such as masquerading, where many machines can "hide behind" a single public IP address, we still
manage to live with it well. As appliances become more intelligent, however, and networking (particularly
wireless networking) becomes cheaper, we are are approaching a day where every toaster is going to want
its own IP address, and 4 billion address wont' be enough. Changes are on the horizon.
These changes are coming in the form of Internet Protocol Version 6, or "IPv6". Among many other
changes, the IPv6 protocol takes the obvious step, and makes IP addresses bigger. Four times bigger. An
IPv6 address, in full form, looks like fe80:0000:0000:0000:0211:22ff:fe33:4411. Instead
of writing addresses as familiar decimal numbers (like "127"), they are written in hexadecimal numbers
(like "fe33").
Some merciful conventions in representing IPv6 addresses help.
1. Once within an address, a series of zero segments, such as 0000:0000:0000, can be replaced with
a double colon (::).
2. For any segment which begins with leading zeros, such as 0211, the leading zeros can be dropped.
With these two shortcuts in mind, the above address could be written a little more friendly as
fe80::211:22ff:fe33:4411.
10
An Introduction to
TCP/IP Networking
Red Hat Enterprise Linux is getting ready for a future transition to IPv6, and many applications can already
handle either IPv4 or IPv6 connections. For our purposes, because it is not yet widely adopted, we're going
to ignore IPv6 as much as possible. Fortunately, there's just a few forms of IPv6 addresses we need to be
able to recognize, summarized in the table below.
Table 1.3. Commonly encountered IPv6 Addresses in the IPv4 World
Role
IPv6 Form
IPv6 Convention
IPv4 Form
Notes
IPv4
Mapped ::ffff:192.168.0.1
An IPv6 address 192.168.0.1
Addressed
which starts with
a bunch of zeros,
then "ffff", then
an IPv4 address
(written in decimal,
not hexadecimal).
When an IPv6 ready
application accepts
a IPv4 connection,
it represents the
IPv4 address using
this special form of
IPv6 address.
Loopback Address ::1
Essentially,
the 127.0.0.1
address "1". All
leading zeros are
replaced with ::.
This address is
the IPv6 equivalent
of the loopback
address.
"Any" Address
Essentially,
the 0.0.0.0
address "0". All
leading zeros (the
whole thing) is
replaced with ::.
This address is
only
used
by
applications which
are listening for
new connections,
and
it
implies
that
they
will
receive connections
over any of the
addresses which are
assigned to the local
machine.
::
Whenever these forms of IPv6 address are encountered ( ::ffff:192.168.0.1, ::1, and ::), you
can simply think of them as their IPv4 equivalents ( 192.168.0.1, 127.0.0.1, and 0.0.0.0,
respectively).
Listening Sockets
Listening are sockets are sockets owned by a server, before any clients have come along. For example,
at the end of step one of our sample TCP/IP connection above, the httpd process would have a socket
open in the listening state. Notice in the above output that for listening sockets, only the local half of the
address is defined.
Established Sockets
As the name implies, established sockets have both a client process and a server process with established
communication.
Pulling it All Together
We now pull some of the pieces together to analyze a few extracted lines from the above output.
tcp
tcp
0
0
0 :::80
0 0.0.0.0:111
:::*
0.0.0.0:*
11
LISTEN
LISTEN
An Introduction to
TCP/IP Networking
These two sockets are bound to all interfaces in the LISTENING state, one on port 80 ( with the IPv6
and IPv4 protocol), the other on port 111 (with the IPv4 protocol only). With a little experience, these
can be be recognized as the httpd web server actively listening for client connections, and the portmap
NFS related daemon.
tcp
tcp
0
0
0 127.0.0.1:631
0 127.0.0.1:25
0.0.0.0:*
0.0.0.0:*
LISTEN
LISTEN
These two sockets are listening for connections, but only on the loopback address. They must belong to
services expecting to receive connections from other processes on the local machine, but not from other
machines. To determine what services these ports belong to, we do some greping from the /etc/services
file.
[elvis@station elvis]$ grep 25 /etc/services
smtp
25/tcp
mail
smtp
25/udp
mail
timed
525/tcp
timeserver
timed
525/udp
timeserver
prospero-np
1525/tcp
prospero-np
1525/udp
[elvis@station elvis]$ grep 631 /etc/services
ipp
631/tcp
ipp
631/ucp
# Prospero non-privileged
# Internet Printing Protocol
# Internet Printing Protocol
Apparently, whatever process has claimed port 25 is listening for email clients. This is probably the
sendmail daemon. The process listening on port 631 is listening for print clients. This is probably the
cupsd printing daemon. Both of these services are discussed in more detail in this Workbook.
tcp
tcp
0
0
0 127.0.0.1:631
0 127.0.0.1:59330
127.0.0.1:59330
127.0.0.1:631
ESTABLISHED
ESTABLISHED
These lines reflect both halves of an established connection between two processes, both on the local
machine (notice the loopback IP address for both of them). The first is bound to port 59330 (probably a
randomly assigned client port), and the second to the port 631. Some process on the local machine must
be communicating with the cupsd daemon.
tcp
tcp
0
0
0 ::ffff:192.168.122.156:22
0 192.168.122.156:653
::ffff:192.168.122.1:39543
192.168.122.1:43099
ESTABLISHED
ESTABLISHED
Our final extracted lines represent established connections between clients on remote machines, and
services on our local machine's external interface (192.168.122.156). The first is a connection to an IPv6
aware service on port 22. Again, we try a little grepping to look up the well known service associated
with port 22.
[elvis@station elvis]$ grep 22 /etc/services
ssh
22/tcp
ssh
22/udp
imap3
220/tcp
imap3
220/udp
...
#
#
#
#
SSH Remote Login Protocol
SSH Remote Login Protocol
Interactive Mail Access
Protocol v3
Apparently, this line represents an active connection between a ssh client on a remote machine with IP
address 192.168.122.1, and a sshd daemon on our local machine.
The latter is a connection to an IPv4 only service bound to port 653, probably an NFS related service.
Online Exercises
Lab Exercise
Objective: Gain familiarity with TCP/IP configuration and activity.
12
An Introduction to
TCP/IP Networking
Estimated Time: 10 mins.
Specification
1. Create the file ~/lab11.1/ipaddr, which contains your machine's IP address, as reported by the
hostname command, as its single word.
2. Create the file ~/lab11.1/listening_ports, which contains a list of all ports less then 1024
on your current machine which are open in the listening state, one port per line.
Deliverables
1.
1. The file ~/lab11.1/ipaddr, which contains your machine's current IP address (as reported
by the hostname command) as its single word.
2. The file ~/lab11.1/listening_ports, which contains a list of all ports less then 1024
on your current machine which are open in the listening state, one port per line.
Questions
1.
In TCP/IP networking, what parameter is used to specify a particular process on a machine?
a.
IP address
b.
window size
c.
port number
d.
protocol
e.
None of the above
2.
What file contains a catalog of well known services?
a.
/etc/services
b.
/etc/protocols
c.
/usr/share/net/services
d.
/usr/share/net/protocols
e.
None of the above
3.
Which port number serves as a boundary between privileged and non-privileged ports?
a.
25
b.
991
c.
1024
d.
6000
e.
65535
13
An Introduction to
TCP/IP Networking
4.
What parameter serves to uniquely identify an established TCP/IP connection?
a.
the client IP Address
b.
the server IP Address
c.
the client port number
d.
the server port number
e.
All of the above
5.
If on one machine, elvis was using both the firefox and epiphany web browsers to access the same
website, what parameter would differ for the two involved sockets?
a.
The client IP Address
b.
The server port number
c.
The client port number
d.
The server IP Address
e.
None of the above
6.
When a socket is opened by a server and awaiting client connections, what state is the socket said
to be in?
a.
Waiting
b.
Listening
c.
Established
d.
Initialized
e.
None of the above
7.
What is special about privileged ports?
a.
Only well known services may bind to them.
b.
Only networking servers may bind to them.
c.
Multiple processes may bind to them simultaneously.
d.
They are only available as the system boots.
e.
None of the above.
Use the following transcript to answer the next 3 questions. Assume that the output to the netstat command
displays all ESTABLISHED connections.
[elvis@station elvis]$ netstat -tuna
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address
Foreign Address
tcp
0
0 127.0.0.1:32768
0.0.0.0:*
tcp
0
0 0.0.0.0:111
0.0.0.0:*
tcp
0
0 0.0.0.0:6000
0.0.0.0:*
tcp
0
0 0.0.0.0:22
0.0.0.0:*
14
State
LISTEN
LISTEN
LISTEN
LISTEN
An Introduction to
TCP/IP Networking
tcp
0
0 69.57.79.162:53
0.0.0.0:*
LISTEN
tcp
0
0 192.168.0.254:53
0.0.0.0:*
LISTEN
tcp
0
0 127.0.0.1:53
0.0.0.0:*
LISTEN
tcp
0
0 0.0.0.0:631
0.0.0.0:*
LISTEN
tcp
0
0 127.0.0.1:25
0.0.0.0:*
LISTEN
tcp
0
0 127.0.0.1:953
0.0.0.0:*
LISTEN
tcp
0
0 127.0.0.1:6010
0.0.0.0:*
LISTEN
tcp
0
0 127.0.0.1:631
127.0.0.1:32773
ESTABLISHED
tcp
0
48 69.57.79.162:22
66.187.233.200:35954
ESTABLISHED
...
[elvis@station elvis]$ grep 53 /etc/services
...
domain
53/tcp
# name-domain server
domain
53/udp
gdomap
538/tcp
# GNUstep distributed objects
gdomap
538/udp
# GNUstep distributed objects
...
cfengine
5308/tcp
# CFengine
cfengine
5308/udp
# CFengine
knetd
2053/tcp
# Kerberos de-multiplexor
rndc
953/tcp
# rndc control sockets (BIND 9)
rndc
953/udp
# rndc control sockets (BIND 9)
...
8.
How many TCP/IP clients are currently connected to services on the local machine?
a.
13
b.
0
c.
2
d.
11
e.
None of the above
9.
Of these, how many are on remote machines? (Assume all local interfaces are listed in the fourth
column of output from the netstat command).
a.
1
b.
0
c.
11
d.
2
e.
None of the above
10.
The three sockets bound to port 53 most likely belong to what service?
a.
SSH (Secure Shell)
b.
HTTP (Hypertext Transfer Protocol)
c.
FTP (File Transfer Protocol)
d.
DNS (Domain Name Service)
e.
TELNET (Telnet Service)
15
Chapter 2. Linux Printing
Key Concepts
• Red Hat Enterprise Linux uses the CUPS printing system for managing printers.
• The CUPS printing system is designed around the concept of a print queue, which combines a spooling
directory, a filter, and a print device.
• The lpstat command browses available print queues.
• The lpr, lpq, and lprm commands are used to submit ("request") print jobs, query for outstanding jobs,
and remove pending print jobs, respectively.
• Print submission applications examine the PRINTER environment variable to determine the default
print queue.
• The lp and cancel commands behave similarly to the lpr and lprm commands.
• The cupsd daemon supports a web interface, which can be accessed at http://localhost:631.
Discussion
Introducing CUPS
Red Hat Enterprise Linux uses the Common Unix Printing System (CUPS) for managing printers. Rather
than interacting with a printer directly, users submit print requests to print queues which are managed by
the cupsd daemon. Print requests which are pending in a print queue are referred to as print jobs. Once a
job has been submitted to the queue, users may return immediately to whatever tasks are at hand. If the
printer is busy with another document, or out of paper, or unreachable over the network, the cupsd daemon
will monitor the situation, and send (or resend) the print job to the printer as it becomes available.
The cupsd daemon uses the Internet Printing Protocol (IPP), which is a direct extension of the HTTP
protocol designed to allow print queue management in an operating system independent manner. As a
result, CUPS management has much in common with web server management.
The following figure identifies the elements which participate in Linux printing. The various elements are
discussed in more detail below.
16
Linux Printing
Figure 2.1. Linux Printing Infrastructure
Print Queues
A print queue is a combination of the following elements.
• A spooling directory, where pending jobs may be temporarily stored.
• A series of filters which reformat various formats of input files into a format appropriate for whatever
back end device is connected to the queue.
• A back end device, such as a locally attached printer, or a print queue defined on a remote machine. The
following lists some of the various back end devices supported by CUPS.
• locally attached parallel port printers
• locally attached USB printers
• networked printers using the LPD interface
• networked printers using the JetDirect interface
• IPP based print queues on remote machines
• LPD based print queues on remote machines
• SMB (Microsoft) network print services
When configuring Red Hat Enterprise Linux to use a particular printer or networked print service, the
fundamental task facing the administrator is to define and name an appropriate print queue for the resource.
This generally involves identifying which of the above back ends is used to access the device, choosing
an appropriate filter for the device, and naming and activating the print queue.
In this course, we will assume that all of the hard work has been done, and that any available print queues
for your system have already been defined.
17
Linux Printing
Browsing Available Print Queues: lpstat
Print queues are either available because they have been defined on the local machine, or discovered using
CUPS's ability to browse the local network for published printers.
The lpstat command can be used to scan available print queues from the command line. The following
command line switches may be used to qualify the lpstat command.
Table 2.1. Command Line Switches for the lpstat Command
Switch
Effect
-a
List accepting state of all print queues.
-o
List all outstanding jobs
-p
List all print queues, and if they are idle or busy.
-s
List printer status, including default queue, and back end associated with each queue.
See the lpstat(1) man page for more details. In the following, elvis is discovering that his system's default
print queue is named simply "printer", and that he has several print queues available to him, which seem
to all refer to IPP based print queues on a local print server.
[elvis@station elvis]$ lpstat -s
system default destination: printer
device for acct2_ire: ipp://printsrv.example.com:631/printers/acct2_ire
device for acct_ire: ipp://printsrv.example.com:631/printers/acct_ire
device for ba: ipp://printsrv.example.com:631/printers/ba
device for checks: ipp://printsrv.example.com:631/printers/checks
device for coms: ipp://printsrv.example.com:631/printers/coms
device for exec: ipp://printsrv.example.com:631/printers/exec
...
Submitting and Managing Jobs: lpr, lpq,and lprm
CUPS uses traditional UNIX commands to interface with the print system: lpr for submitting files and
data to be printed, lpq for examining the status of outstanding print jobs, and lprm for removing pending
print jobs from the queue. All three commands use the following techniques for specifying which print
queue to use, in the specified order.
1. If the -P command line switch is found, its argument is used to specify the print queue.
2. If -P is not used, then if the PRINTER environment variable exists, it is used to define the default print
queue.
3. Otherwise, the system default print queue is used.
Submitting Jobs with lpr
Jobs may be submitted with the lpr command. Any arguments are interpreted as files to submit. If no
arguments are specified, standard in is read instead. The following options may be used to qualify the
lpr command.
Table 2.2. Command Line Switches for the lpr Command
Switch
Effect
-P printer
Use print queue printer.
-#
Print # copies
18
Linux Printing
Switch
Effect
-p
Decorate text files with a header that contains filename, job name, and timestamp.
-r
Remove the named print files after printing.
As an example, in the following, blondie uses the lpr command to print the file README, using the sales
print queue.
[blondie@station blondie]$ lpr -P sales README
Monitoring Jobs with lpq
The lpq command lists pending jobs in a queue. In the following example, blondie will submit the output
of the df command to the printer legal, and then examine the contents of the queue.
[blondie@station blondie]$ df | lpr -P legal
[blondie@station blondie]$ lpq -P legal
legal is ready and printing
Rank
Owner
Job
File(s)
active elvis
1
services
1st
blondie 5
(stdin)
Total Size
20480 bytes
1024 bytes
Removing Jobs with lprm
Blondie suspects that something is wrong with the legal printer, and decides to start using the sales printer
as her default printer. She first sets up the PRINTER environment variable to reflect her new preferences,
the uses the lprm command to remove her job from the legal queue.
[blondie@station blondie]$ export PRINTER=sales
[blondie@station blondie]$ lpq
sales is ready
no entries
[blondie@station blondie]$ lpq -P legal
legal is ready and printing
Rank
Owner
Job
File(s)
active elvis
1
services
1st
blondie 5
(stdin)
[blondie@station blondie]$ lprm 5
[blondie@station blondie]$ lpq -P legal
legal is ready and printing
Rank
Owner
Job
File(s)
active elvis
1
services
[blondie@station blondie]$ df | lpr
Total Size
20480 bytes
1024 bytes
Total Size
20480 bytes
Notice in the first lpq command, and the last lpr command, the PRINTER environment variable implicitly
specified the sales print queue.
Alternate Front End Commands: lp and cancel
In Unix, in the days preceding CUPS, two parallel implementations of printing infrastructure were
implemented. The first used the three commands introduced above, namely lpr, lpq, and lprm. The other
used lp, lpstat, and cancel for analogous roles.
We have already seen that lpstat is supported, and is the preferred tool for discovering available print
queues. The commands lp and cancel are also available as slight variations of the lpr and lprm commands.
Consult the shared lp(1) man page for details.
The CUPS Web Interface
Lastly, we would be remiss to leave the topic of CUPS without mentioning the native web interface
provided by the cupsd daemon. As mentioned, most clients interact with the cupsd daemon using the
19
Linux Printing
IPP protocol, which is an extension of the HTTP protocol. Because of the similarities, the cupsd daemon
behaves in may ways like a web daemon, including the serving of CGI style management pages.
In order to view CUPS's management pages, point a web browser to the localhost address, but override
the default port 80 with the CUPS daemons well known service port, 631.
http://localhost:631
The cupsd daemon will return with a CUPS "homepage", from where printers and print jobs can be
browsed, and copious online documentation is available.
Figure 2.2. The CUPS Management Homepage
Online Exercises
Lab Exercise
Objective: Manage print jobs effectively.
Estimated Time: 10 mins.
Setup
Your station should be configured with a print queue named rha_faux. This print queue is attached to a
virtual printer running on the classroom server. You should be able to view the first page of any print job
sent to this printer on the classroom server [http://rha-server/rha/].
If the lpstat -s command does not mention a print queue called rha_faux, or you are not able to access the
classroom server's web site, consult your instructor.
Specification
1. Print the file /etc/services using the print queue rha_faux, and observe its output [http://rhaserver/cgi-bin/rha/printer].
20
Linux Printing
2. Print the file /usr/share/backgrounds/tiles/floral.png using the print queue rha_faux,
and observe its output [http://rha-server/cgi-bin/rha/printer].
3. From with your web browser, print the current page directly to the rha_faux print queue.
Deliverables
1.
1. Three print jobs submitted to the rha_faux print queue.
Questions
1.
What print system does Red Hat Enterprise Linux use by default?
a.
BSD
b.
LPD
c.
LPRng
d.
CUPS
e.
None of the above
2.
Which of the following commands would print all known print queues to standard out?
a.
lpstat -a
b.
lslp
c.
cupsd -l
d.
gnome-print-manager
e.
None of the above
3.
How can a default print queue be specified to the lpr command?
a.
With the -P command line switch
b.
With the -Q command line switch
c.
With the PRINTER environment variable
d.
A and C
e.
A, B, and C
4.
Which of the following command lines would view all jobs pending in the queue snail?
a.
lpview -P snail
b.
lpstat -a
c.
lpq -P snail
d.
lsps -d snail
21
Linux Printing
e.
5.
None of the above
Which two paired commands have near identical behavior?
a.
lprm and cancel
b.
cancel and killjob
c.
killjob and lpdel
d.
lpdel and flush
e.
flush and lprm
6.
Which of the following is the well known port associated with the cupsd daemon?
a.
631
b.
1024
c.
25
d.
534
e.
None of the above
7.
Which of the following commands can be used to (directly) determine available print queues?
a.
lpr
b.
lpq
c.
lpstat
d.
B and C
8.
Which of the following is not an element of a CUPS print queue?
a.
A spooling directory
b.
A spell checker
c.
A print filter
d.
A back end device
e.
None of the above
9.
What is the name of the network printing protocol used extensively by CUPS?
a.
Network Printing Protocol (NPP)
b.
Network Printing Service (NPS)
c.
Print Device Queuing (PDQ)
d.
Internet Printing Protocol (IPP)
e.
None of the above
22
Linux Printing
10.
Which of the following command lines would print the file /tmp/out.dat to the queue sales,
and remove the file when completed?
a.
lpr -P sales -r /tmp/out.dat
b.
lpr -Pd sales /tmp/out.dat
c.
lpr -P sales -r < /tmp/out.dat
d.
lpr -d sales -z /tmp/out.dat
e.
None of the above
23
Chapter 3. Managing Print Files
Key Concepts
• The primary printing format in Linux is PostScript.
• Utilities such as ps2pdf and pdf2ps can convert PostScript to PDF and back.
• evince previews PostScript and PDF documents.
• enscript converts text file into decorated PostScript.
• mpage can rearrange individual pages from a PostScript document.
Discussion
PostScript
In Linux, most printers expect to receive either ASCII text, or graphics using the PostScript format. Unlike
most graphics formats, PostScript is actually a scripting language which has been tailored to the task of
rendering graphics on the printed page. The PostScript Language's sophistication allows it to perform
powerful tasks, but a PostScript interpreter must be used to render PostScript files as images.
Many printers implement native PostScript interpreters, and are referred to as PostScript printers.
Whenever a PostScript printer receives a text file which begins with the characters %!PS, the remainder of
the file is interpreted as a PostScript script, rather than printed as ASCII text directly. (Note the similarity
to Unix's #!/bin/bash scripting mechanism).
In Linux (and Unix), an application called Ghostscript, or simply gs, implements a PostScript interpreter.
Implementing a PostScript interpreter is a significant task, and although several applications in Linux can
be used to view or manipulate PostScript files, almost all use Ghostscript as the back end to perform the
actual rendering of PostScript into more accessible graphics formats.
Rather than using the low level Ghostscript interpreter directly, the easy to use evince application is usually
used to view PostScript documents.
Viewing PostScript Documents with the evince
Document Viewer.
Many applications will print PostScript directly to files instead of delivering them to a print queue. For
example, in the following dialog, by selecting "Print to File", the firefox web browser is being asked to
print the current web page not to a print queue, but to a PostScript file titled example.ps.
24
Managing Print Files
Figure 3.1. Printing to a PostScript File
As the head command illustrates, a PostScript file is a simple text file beginning with the PostScript
"magic" %!PS.
[elvis@station elvis]$ head example.ps
%!PS-Adobe-3.0
%%BoundingBox: 0 0 612 792
%%HiResBoundingBox: 0 0 612 792
%%Creator: Mozilla PostScript module (rv:1.8.0.9/0)
%%DocumentData: Clean8Bit
%%DocumentPaperSizes: Letter
...
The PostScript file can be viewed with the evince document viewer.
[elvis@station ~]$ evince example.ps
Figure 3.2. Using Evince to view PostScript Documents
25
Managing Print Files
Decorating Text for Printing with enscript
The enscript utility converts text files into PostScript, often decorating the text with syntax highlighting
(referred to as pretty printing), a header, or formatting multiple text pages per printed page (referred to
as printing 2-up, 4-up, etc..).
A little awkwardly, the enscript command sends a text file directly to the lpr command by default, with the
result that enscripted files are immediately printed. The -o command line switch is often used to specify
an output PostScript file instead.
As an example, elvis could easily create a PostScript version of the GPL license.
[elvis@station ~]$ enscript -o gpl.ps /usr/share/doc/redhat-release-5Server/GPL
[ 7 pages * 1 copy ] left in gpl.ps
[elvis@station ~]$ evince gpl.ps
Figure 3.3. A simply formatted PostScript file generated by enscript
So far, the formatting has been fairly minimal: A title, a date, and a page number. The enscript command
comes with a host of more sophisticated formatting options, however. Some of the more commonly used
command line switches are found in the following table.
Table 3.1. Common Command Line Switches for enscript
Switch
Effect
-o filename
Generate output to the file filename.
-2, --columns=num Format text into 2 (or num) columns.
-a pages
Only print pages pages. pages can be of the form begin-end, or the word odd
or even.
-b header
Use text header as a page header. An elaborate format for specifying filename,
page number, etc., is provided.
-d printer
Send output to print queue printer.
-E [lang]
Decorate text with syntax highlighting appropriate for the lang programming
language. Use --help-pretty-print to list supported languages.
26
Managing Print Files
Switch
Effect
-G
Decorate pages with a fancy header.
-r
Rotate page 90 degrees.
-W lang
Generate output in language lang, which may be one of PostScript, html,
overstrike, or rtf.
Many more options exist. Consult the enscript(1) man page for more information. As an example, the
following command line would render the C header file malloc.h as PostScript with 2 columns per
page, rotated, and decorated with a fancy header and syntax highlighting.
[elvis@station elvis]$ enscript -r2 -E -G -o malloc.ps /usr/include/malloc.h
[ 2 pages * 1 copy ] left in malloc.ps
[elvis@station elvis]$ evince malloc.ps
Figure 3.4. An elaborately formatted PostScript file generated by enscript
Rearranging PostScript with mpage
The mpage command can be used to extract pages from the middle of a multi-page PostScript document,
or reformat the document to be printed with multiple pages per printed sheet.
While designed to operate on PostScript, mpage can be fed plain text files as well, which it will simply
render as PostScript before handling. Arguments are considered input files, with the output is directed to
standard out.
Table 3.2. Common Command Line Switches for mpage
Switch
Effect
-o filename
Generate output to the file filename.
-1, -2, -4, -8
Print specified number of pages per printed page (default is 4).
-j
first[-last] Only print specified range of pages, including only every interval pages,
[%interval]
if specified.
-l
Toggle printing landscape or portrait mode.
-o
Toggle printing border around every page (default is on).
-P [printer]
Send PostScript output to print queue printer.
-G
Decorate pages with a fancy header.
-r
Rotate page 90 degrees.
-G
Decorate pages with a fancy header.
-W lang
Generate output in language lang, which may be one of PostScript,
html, overstrike, or rtf.
27
Managing Print Files
As an example, mpage could be used to convert the layout of the gpl.ps file generated earlier to "4 UP".
[elvis@station ~]$ mpage -4 gpl.ps > gpl_4up.ps
[elvis@station ~]$ evince gpl_4up.ps
Figure 3.5. A PostScript file formatted 4-up with mpage
Converting PostScript to PDF and PDF to PostScript
On Linux (and Unix) systems, PostScript tends to be the dominant format for printable documents. Because
its formatting primitives are plain text, a Unix design principle, it's comparatively easy to apply basic
reformatting filters to convert from one format to another.
Unfortunately, on operating systems which are not Unix based, PostScript documents are pretty useless.
The rest of the world deals with the alternate PDF format. Fortunately, Red Hat Enterprise Linux contains
easy to use utilities that allow PostScript documents to be converted to PDF documents (ps2pdf), and vice
versa (pdf2ps).
The syntax of the ps2pdf command is trivial, where the first argument is the input PostScript filename (or
a “-” to imply standard in), and the second argument is the output PDF filename (or a “-” to imply standard
out). The pdf2ps command works similarly.
As a quick example, in order to share his work with a friend who prefers PDF documents, elvis now uses
the ps2pdf command to convert his malloc.ps file into PDF format, which is of comparable quality,
but dramatically more compact.
[elvis@station ~]$ ps2pdf malloc.ps malloc.pdf
[elvis@station ~]$ file malloc.p*
malloc.pdf: PDF document, version 1.2
malloc.ps: PostScript document text conforming at level 3.0
[elvis@station ~]$ evince malloc.pdf
28
Managing Print Files
Figure 3.6. Using evince to view a PDF document
Note that as a nice benefit, the PDF format allows evince to display document thumbnails in a side pane.
Lastly, consider the Samba by Example text found as the PDF file /usr/share/doc/samba-3*/
Samba3-ByExample.pdf, which weighs in at about 500 pages. Suppose elvis wanted a hardcopy for
quick skimming, but was willing to print 4up to conserve paper.
[elvis@station samba-3.0.23c]$ pdf2ps Samba3-ByExample.pdf - | mpage -4 | lpr
Rearranging PostScript with mpage
The mpage command can be used to extract pages from the middle of a multi-page PostScript document,
or reformat the document to be printed with multiple pages per printed sheet.
The mpage accepts as input either PostScript or text. Text will be rendered into PostScript before
managing. Arguments are considered input files, with the output is directed to standard out.
Table 3.3. Common Command Line Switches for mpage
Switch
Effect
-o filename
Generate output to the file filename.
-1, -2, -4, -8
Print specified number of pages per printed page (default is 4).
-j
first[-last] Only print specified range of pages, including only every interval pages,
[%interval]
if specified.
-l
Toggle printing landscape or portrait mode.
-o
Toggle printing border around every page (default is on).
-P [printer]
Send PostScript output to print queue printer.
-G
Decorate pages with a fancy header.
-r
Rotate page 90 degrees.
-G
Decorate pages with a fancy header.
-W lang
Generate output in language lang, which may be one of PostScript,
html, overstrike, or rtf.
29
Managing Print Files
The following command line converts the PDF file RHEL3FamOverWPPdf.pdf into a PostScript file,
printed four pages per printed sheet.
[elvis@station elvis]$ pdf2ps RHEL3FamOverWPPdf.pdf - | mpage > rhel3.ps
[elvis@station elvis]$ ggv rhel3.ps
Figure 3.7. A PDF Document Converted into PostScript (4 Up)
Online Exercises
Lab Exercise
Objective: Practice preparing output for printing.
Estimated Time: 20 mins.
30
Managing Print Files
Specification
1. Use the enscript command to generate a PostScript document of the file /usr/share/doc/bash*/loadables/getconf.c name getconf.ps, with two pages per printed sheet. Optionally,
have the enscript command "pretty print" the text (by adding syntax highlighting).
2. Use the mpage command to extract pages 8 and 9 from the getconf.ps file created above, storing
them in a new document called getconf89.ps.
(By the default, the mpage command will print 4 input pages per output pages, confusing the page count.
Include the appropriate command line switch so that only one input page is printed per output page.)
3. Use the ps2pdf command to convert the file getconf89.ps into a PDF document titled
getconf89.pdf. Compare the sizes of the two documents.
Deliverables
1.
There is no grading element for this exercise.
1. A multi-page PostScript document getconf.ps which contains the text of the file /usr/
share/doc/bash-*/loadables/getconf.c, printed 2 pages per output page.
2. A two page PostScript document getconf89.ps, which contains the extracted pages 8 and
9 from the document getconf.ps.
3. The PDF document getconf89.pdf, which is the contents of the PostScript document
getconf89.ps converted into PDF format.
Questions
1.
In Linux, which of the following is the scripting language used as a printing format for documents?
a.
PostScript
b.
PDF
c.
JPEG
d.
PNG
e.
None of the above
2.
3.
Which of the following applications is used to view PostScript documents?
a.
eog
b.
lpr
c.
evince
d.
mpage
If you chose "print to file" from within a Linux application, what would be the resulting file's
format?
31
Managing Print Files
a.
PostScript
b.
PNG
c.
JPEG
d.
PDF
e.
None of the above
4.
Which of the following applications can extract pages from a multi-page PostScript document, and
place them in a new PostScript document?
a.
ps2pdf
b.
evince
c.
mpage
d.
B and C
5.
Which of the following commands can be used to convert a PDF document to PostScript?
a.
eog
b.
pdf2ps
c.
mpage
d.
A and C
6.
Which of the following would reformat the multi-page PostScript document input.ps into a
PostScript Document with 2 pages per printed sheet?
a.
enscript -2 -o output.ps input.ps
b.
mpage -2 input.ps > output.ps
c.
psnup -r2 input.ps output.ps
d.
mpage input.ps output.ps
e.
None of the above
7.
8.
Which of the following command lines could be used to convert the PDF document report.pdf
to PostScript?
a.
pdf2ps report.pdf report.ps
b.
eog -sps report.pdf
c.
mpage report.pdf > report.ps
d.
enscript -o report.ps report.pdf
e.
None of the above
Which of the following applications can be used to render PDF documents?
32
Managing Print Files
a.
cat
b.
eog
c.
evince
d.
mpage
9.
By default, where does the enscript command send its output?
a.
The file enscripot.ps
b.
The default printer
c.
Standard Out
d.
The file /dev/null
10.
Which of the following command lines would convert the text file README into a PostScript
document with two pages per printed sheet?
a.
txt2ps -2 README README.ps
b.
enscript -r2 -o README.ps README
c.
mpage -2 README > README.ps
d.
B or C
e.
None of the above
33
Chapter 4. Email Overview
Key Concepts
• Email Management involves an MUA (Mail User Agent), which is used to present newly delivered mail
to a user, and allow the user to compose new responses, and an MTA (Mail Transport Agent), which
manages the background task of exchanging email with remote machines.
• Depending on the details of a computer's Internet access, the job of receiving email may be delegated
to a mailbox server, which would then allow user's to access their delivered email using the POP or
IMAP protocols.
• Again depending on the details of a computer's Internet access, the job of delivering email may be
delegated to a remote outgoing SMTP server.
• Generally, locally delivered but unread mail is spooled in the file /var/spool/mail/$USER, where
USER is the username of the recipient.
• One of the simplest MUA's is the mail command.
Discussion
Using Email
Arguable the most popular service the Internet provides is email. Email is simple in concept: Alice
composes a text file, and specifies to send it to Bob. When Bob checks his mail, the text file will be
waiting for him, labeled as delivered from Alice. In practice, however, a reliable email delivery system
must provide solutions to some fairly complicated problems.
The software which Alice and Bob use must solve the following issues.
1. Alice's email application must somehow determine which computer is Bob's computer.
2. Once determined, Alice's application must connect to Bob's machine, and transfer the message. If Bob's
machine cannot be contacted, Alice's application must spool the message which is pending delivery
and have some strategy for trying to contact Bob's machine again at a later time.
3. As the recipient, Bob's email application must have high availability, so that whenever someone (Alice
or otherwise) wants to send him an email message, his application is available to receive it.
In many instances, the email recipient is using a machine which does not have a permanent Internet
connection, or is behind a firewall, which further complicates issues. In this Lesson, we will discuss various
arrangements which provide solutions to these problems.
The Simple Solution
The simplest solution is appropriate for people using computers which have permanent, well known
Internet connections. At the time that many of the protocols defining how email delivery should be handled
were developed, this was more of the standard instead of the exception. The solution involves two separate
applications whose roles are identified by TLA's (Three Letter Acronyms). The first application is referred
to as the MTA, or Mail Transport Agent; the second is known as the MUA, or Mail User Agent.
34
Email Overview
Figure 4.1. Email Delivery on Computers with Permanent, Well Known Internet
Connections
The MTA (Mail Transport Agent)
The MTA generally operates in the background, performing the work of the local post office. The MTA
receives email to be delivered from programs on the local machine, determines from the recipient's address
the appropriate machine to contact, and attempts to connect to a complementary MTA running on the
recipients machine, who should be listening to port 25. If the sender's MTA cannot contact the receiver's
MTA, the mail is spooled on the sender's machine, and the sender's MTA tries again at a later time.
The MTA also binds to the local port 25, where it receives connections from other MTA's. When it receives
mail from a remote MTA which is destined for a user on the local machine, it receives the mail, and stores
it in a mail spool which is referred to the user's inbox. In Linux (and Unix), the default inbox for a user
is /var/spool/mail/username, so that mail awaiting the user elvis would be stored in the file /
var/spool/mail/elvis.
The default MTA in Red Hat Enterprise Linux is a daemon called sendmail.
The MUA (Mail User Agent)
The MUA is the application most user's think about when they think about email. The MUA retrieves
delivered mail from a user's mail spool (inbox), and presents it to the user as "New Mail". The MUA
allows users to compose responses or new mail messages, and passes these messages to the local MTA
for delivery.
Red Hat Enterprise Linux ships with a wide selection of MUAs, several of which will be introduced in
this and the next lesson.
35
Email Overview
Mailbox Servers
While simple, the previous solution requires that, first of all, users are receiving email on a machine that
maintains a persistent Internet connection, and secondly, the machine has a well known host name and is
accessible to others. Many users, such as people using a "dial up" or "High Speed" connection from an
ISP (Internet Service Provider), or people using machines behind an institution's firewall, are not in this
situation. Another solution has evolved to serve people in these situations: Mailbox servers.
Figure 4.2. Mailbox Servers
In the diagram above, we assume that elvis is using a "high speed" Internet connection which he has
subscribed to from the company "ISP.Net". When he connects to the Internet, his ISP issues his machine
an IP address, but elvis cannot predict ahead of time which IP address he will receive. The hostname which
the ISP has assigned to the IP address is probably unattractive, such as dhcp-191-93.isx.isp.net, so even
if elvis were guaranteed to receive the same IP address each time, he would not want to advertise his
machine's hostname as his email address.
Instead, elvis takes advantage of an "email account" which his ISP offers him. Very likely, this account
exists on a Linux or Unix machine owned by the ISP, which has a permanent connection to the Internet,
and is assigned a hostname such as pop.isp.net. The ISP has arranged all mail destined to addresses of
the form user@isp.net to be delivered to the MTA running on this machine. When the MTA running on
pop.isp.net receives mail for elvis, it stores the delivered mail into a mail spool dedicated to elvis (very
likely in the file /var/spool/mail/elvis), and the email is considered delivered.
Because the ISP's machine pop.isp.net has a permanent, well known connection to the Internet, it is a much
better candidate for receiving email than elvis's machine at home. Sitting at home, however, elvis still
needs access to the email sitting in his "inbox" on pop.isp.net. Usually, this access is provided in the from
of a POP (Post Office Protocol) or IMAP (Internet Mail Access Protocol) server.
POP servers
POP servers perform a very simple service. They allow users to access the user's single mail spool, and
transfer its contents to their local MUA. POP servers generally bind to port 110, and require that any POP
client authenticate itself using a username/password pair. Most modern MUA's act as POP clients, and can
be configured to retrieve mail from a specified POP server.
If a mailbox server is implementing the POP service, it usually implies that the mailbox server is not
intending to provide permanent storage for a user's email, but instead just cache it temporarily until the
user can "pop" it down to a local machine.
36
Email Overview
IMAP servers
IMAP servers generally provide clients with more sophisticated mailbox management. IMAP users may
maintain several folders on the mailbox server, not just their individual inbox. In general, an IMAP server
implies that a user's email is intended to be permanently stored on the mailbox server, and users will
occasionally connect with a MUA from a remote machine to "browse" their mail. Generally, IMAP servers
are found in institutional and corporate environments. IMAP daemons bind to port 143.
Sending Mail
Because elvis's machine maintains an almost continuous connection to the Internet, he is willing to still
use his local MUA to deliver mail. If his local MUA is temporarily unable to connect to the recipient's
machine, the MUA will spool the mail locally and try again later.
Red Hat Enterprise Linux Default Configuration
By default, Red Hat Enterprise Linux is configured appropriately for this scenario. The local MTA is
started, but it will not accept connections over port 25 (except for from the loopback address, 127.0.0.1).
It serves merely to deliver outgoing email. Users are assumed to be accessing their email from a POP or
IMAP server.
The default configuration can be changed, but the necessary configuration is beyond the scope of this
course.
Outgoing Mail Servers
For machines that have only transient connections to the Internet, attempting to deliver email using the
local MTA may not be appropriate. Instead, users of "dial up" connections and the like often use what is
referred to as an SMTP (Simple Mail Transfer Protocol) Server, or outgoing mail server. Many ISPs and
institutions provide outgoing SMTP servers, often with names like smtp.isp.net.
37
Email Overview
Figure 4.3. Outgoing Mail Server
The MTA on the SMTP server is willing to accept mail from "local" machines, even though it is not the
final recipient for the email. Instead, the SMTP server relays the mail, forwarding it on to its destination.
If any temporary problems occur, the spooling and redelivery attempts of the pending mail is now the
responsibility of the SMTP server.
Many MUA's allow users the option of specifying a remote host to act as the user's SMTP server (as
opposed to forwarding mail to the local MTA for delivery). 1
Local Delivery
All of the above scenarios assume that a user is sending email from his machine to a recipient on a remote
machine. Linux (and Unix) also allows users on a local machine to deliver email to one another, where
email is addressed simply to a username, such as blondie.
For local delivery, no POP servers or SMTP servers are required, as the email is instantly delivered by
the local MTA.
The mail MUA
Perhaps the simplest MUA is a command called simply mail. The mail command dates from the earliest
days of Unix, and therefor relies heavily on the local MTA. The mail command cannot be configured to
access mail from a POP or IMAP server, nor can it be configured to use a SMTP server for outgoing mail.
Instead, it expect mail to be delivered to the local mail spool, and passes mail to the local MTA for delivery.
1
Additionally, the local MTA can be configured to pass all mail to a SMTP server, rather than attempting to deliver the mail directly. If so configured,
local MUAs do not need to be configured to use a SMTP server explicitly. Instead, they can pass mail to the local MTA, which will then pass the
mail to the specified SMTP server.
Configuring the local MTA requires root privileges, and is beyond the scope of this course.
38
Email Overview
For situations where this is appropriate, however, such as sending email to local users, the mail command
can be very efficient.
Sending Email with mail
The mail command can be used to deliver mail to recipients who are specified as arguments on the
command line. The body of the message is read from standard in (which may be read from a pipe, a
redirected file, or from the keyboard directly, where CTRL+D ("EOF") is used to indicate the end of the
message).
The command line switches in the following table can be used to specify a subject line, recipients to "Cc:",
etc.
Table 4.1. Command Line Switches for the mail Command
Switch
Effect
-v
Verbose mode. Prints communication between the mail command an the local (or
remote) MTA.
-s subject
Specify a subject line.
-c addrs
Send carbon copies to the specified addresses.
-b addrs
Send blind carbon copies to the specified addresses.
As an example, in the following, elvis mails to blondie the contents of the file lyrics.txt.
[elvis@station elvis]$ mail -s "here they are" blondie < lyrics.txt
Using mail to Read Mail
The mail command can also be used to read mail from a user's local mail spool. The interface is primitive,
however, and usually other MUA's are used instead. Details can be found in the mail man page. These
days, the most common use of the mail command is as a quick and easy way to send mail.
In the following example, blondie is using the mail command, without arguments, to view the mail in her
inbox, and then delete the message and quit.
[blondie@station blondie]$ mail
Mail version 8.1 6/6/93. Type ? for help.
"/var/spool/mail/blondie": 1 message 1 new
>N 1 elvis@redhat.com
Thu Nov 13 00:08 16/653
"here they are"
& RETURN
Message 1:
From elvis@station.example.com Thu Nov 13 00:08:45 2003
Date: Thu, 13 Nov 2003 00:08:44 -0500
From: elvis@station.example.com
To: blondie@station.example.com
Subject: here they are
100 bottles of ink on the wall.
100 bottles of ink.
Take one down,
Pass it around.
& d
& q
39
Email Overview
Examples
Sending mail
A quick an easy way to send a text file to another user is with the mail
[julius@station julius]$ mail -s "here's the file" nero < somefile
It is also easy with mail tosend the output of a command to another user
[julius@station julius]$ find . -name "*.old" | mail -s "find output" nero
Simple Mail Transport Protocol
Using a simple mail command with a verbose option, a user can watch the process that sendmail goes
through to deliver a message.
[julius@station julius]$ mail -v -s hello elvis@localhost
this is only a test
.
elvis@localhost... Connecting to [127.0.0.1] via relay...
220 station.example.com ESMTP Sendmail 8.12.8/8.12.8; Sat, 12 Apr 2003 13:08:30 -0400
>>> EHLO station.example.com
250-station.example.com Hello localhost.localdomain [127.0.0.1], pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-8BITMIME
250-SIZE
250-DSN
250-ETRN
250-DELIVERBY
250 HELP
>>> MAIL From:julius@station.example.com SIZE=60
250 2.1.0 julius@station.example.com... Sender ok
>>> RCPT To:elvis@station.example.com
>>> DATA
250 2.1.5 elvis@station.example.com... Recipient ok
354 Enter mail, end with "." on a line by itself
>>> .
250 2.0.0 h3CH8U3j002250 Message accepted for delivery
elvis@localhost... Sent (h3CH8U3j002250 Message accepted for delivery)
Closing connection to [127.0.0.1]
>>> QUIT
221 2.0.0 station.example.com closing connection
Online Exercises
Lab Exercise
Objective: Send email to local users.
Estimated Time: 10 mins.
Specification
1. Use the mail command to mail the contents of the file /proc/cpuinfo to your first and second
alternate accounts (i.e, the users username_a and username_b, where username is the name of
your primary account.
40
Email Overview
2. Create the file ~/you_have_mail, which contains a sorted list of all users on your local system
which currently have mail in their "Inboxes" (i.e., their spool of delivered but no yet read email). You
should include "system" users in your list, and list one user per line.
Deliverables
1.
1. Your first and second alternate accounts should have the contents of the file /proc/cpuinfo
in their spool of delivered email.
2. The file ~/you_have_mail should contain a sorted list of users how have unread mail
waiting in their mail spool. (Obviously, your first and second alternate account names should
be included in this list.)
Questions
1.
Which of the following utilities provides authenticated users remote access to their individual queue
of delivered (but not other mail folders)?
a.
MTA
b.
IMAP server
c.
MUA
d.
POP server
e.
Outgoing SMTP server
2.
Which of the following utilities may retrieve mail from a local (and often remote) mail queue and
present it to the user for reading?
a.
MTA
b.
IMAP server
c.
MUA
d.
Outgoing SMTP server
e.
POP server
3.
4.
Which of the following utilities provides authenticated users remote access to their individual queue
of delivered mail, and remote mail folder management?
a.
POP server
b.
Outgoing SMTP sever
c.
MUA
d.
MTA
e.
IMAP server
Which of the following utilities receives mail from remote MTA's, and accepts the mail for local
delivery?
41
Email Overview
a.
POP server
b.
MUA
c.
IMAP server
d.
MTA
e.
None of the above
5.
Which of the following utilities receive mail from local applications, and relays the mail to its
destination?
a.
Outgoing SMTP server
b.
POP server
c.
MUA
d.
IMAP server
e.
None of the above
6.
What is the well know port for an IMAP server?
a.
110
b.
25
c.
80
d.
22
e.
None of the above
7.
What is the well known port for an MTA?
a.
25
b.
110
c.
22
d.
80
e.
None of the above
Use the following transcript to answer the next question.
[blondie@station blondie]$ netstat -tuna | grep LISTEN
tcp
0
0 0.0.0.0:110
0.0.0.0:*
tcp
0
0 0.0.0.0:6000
0.0.0.0:*
tcp
0
0 0.0.0.0:80
0.0.0.0:*
tcp
0
0 0.0.0.0:22
0.0.0.0:*
tcp
0
0 127.0.0.1:631
0.0.0.0:*
tcp
0
0 127.0.0.1:25
0.0.0.0:*
tcp
0
0 127.0.0.1:33018
0.0.0.0:*
tcp
0
0 0.0.0.0:443
0.0.0.0:*
8.
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
LISTEN
The machine blondie is using will currently accept remote connections for which of the following
email related services?
42
Email Overview
a.
MTA
b.
IMAP Server
c.
POP Server
d.
both A and C
e.
All of the above
9.
Which of the following command lines would deliver mail to the user blondie with a subject line
of current disk usage and whose body contain the output of the df command?
a.
mail -s current disk usage blondie < df
b.
df | mail -s "current disk usage" blondie
c.
df | mail -c blondie "current disk usage"
d.
echo blondie | mail -s "current disk usage" df
e.
None of the above
10.
Which of the following directories is the default spooling directory for locally delivered mail?
a.
/var/mail/
b.
$HOME, where HOME refers to the relevant user's home directory.
c.
/etc/mail/
d.
/var/spool/mail/
e.
None of the above
43
Chapter 6. Network Diagnostic
Applications
Key Concepts
• /sbin/ifconfig displays local IP configuration.
• ping confirms low level network connectivity between hosts.
• host makes direct DNS queries.
• The netstat -tuna command lists currently active network services and connections.
• /sbin/traceroute can diagnose routing problems.
Discussion
Required Configuration for the Internet Protocol
The Internet Protocol, or more commonly (and a bit redundantly), the IP protocol, is the adopted standard
for delivering information between machines which are attached by a network. In order to use the IP
protocol, a machine must be configured with the following information.
While configuring a system with its appropriate IP configuration information requires root privileges, this
Workbook will demonstrate how you can determine the networking configuration of your local machine,
and introduce utilities that can be used to confirm that a machine's networking infrastructure is functioning
properly.
IP Address
Linux (and Unix) represents every networking device attached to a machine (such as an Ethernet card, a
Token Ring card, a modem used for dialup connections, etc...) as a network interface. Before an interface
can be used to send or receive traffic, it must be configured with an IP address which serves as the interface's
identity.
Default Gateway
The mechanics of the IP protocol organizes machines into sub networks, or subnets. All machines on a
given subnet may exchange information directly. IP subnets are in turn linked to other subnets by machines
acting as routers. A router has multiple network interfaces, usually each participating in a distinct subnet.
In order to communicate with a host on another subnet, the data must be passed to a router, which (with the
help of other routers) routes the information to the appropriate subnet, and from there to the appropriate
host.
In order to communicate with machines outside of your local subnet, your machine must know the identity
of a nearby router. The router used to route packets outside of your local subnet is commonly referred to
as a default gateway.
Nameserver
Other machines on the Internet are in turn identified by their IP address. People tend to prefer to think
in terms of names instead of numbers, however, so a protocol has been developed to assign names to IP
addresses. The protocol is called Domain Name Service, or more commonly DNS.
44
Network Diagnostic Applications
The DNS protocol requires that every machine have available one or more DNS servers (commonly called
nameservers), which can serve as both a database for assigning name to local IP addresses, and also a
starting point for determining IP addresses for domain names for which the server does not have direct
knowledge.
Determining Your IP Address(es): /sbin/ifconfig
A previous Lesson introduced the hostname -i command, which displays the IP address of your local
machine. In reality, a "machine" does not have an IP address, a machine's network interfaces do. This
Lesson discusses the topic of IP addresses and network interfaces in more detail.
In Linux (and Unix), every network device is represented as a network interface. (For once, we encounter
something that is not a file!) Linux names interfaces according to the type of device it represents. The
following table lists some of the more commonly encountered interface names used in Linux.
Table 6.1. Linux Network Interfaces
Interface
Device
ethn
Ethernet Card
trn
Token Ring Card
fddin
Fiber Optic Card
pppn
Modem Dialup Connection
lo
The Loopback Device
In each case, n is replaced with a distinct integer for each instance of a given device attached to a machine.
Confirming Network Interface Configuration
The ifconfig command displays the configuration of all active network interfaces. Because the command
is generally used by root to configure interfaces, it lives within the /sbin directory, which is outside of
the default PATH for standard users. Standard users can use the command to view interface configuration
information, however, by using an absolute reference, as in the following example.
[blondie@station blondie]$ /sbin/ifconfig
eth0
Link encap:Ethernet HWaddr 00:00:86:4D:F0:0C
inet addr:192.168.0.51 Bcast:192.168.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1716503 errors:0 dropped:0 overruns:2 frame:0
TX packets:2146415 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:193489123 (184.5 Mb) TX bytes:1299754457 (1239.5 Mb)
Interrupt:11 Base address:0xd400
lo
Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:206054 errors:0 dropped:0 overruns:0 frame:0
TX packets:206054 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:22911068 (21.8 Mb) TX bytes:22911068 (21.8 Mb)
The interface eth0 is assigned an IP address of 192.168.0.254.
The ifconfig command displays a stanza of IP configuration information and usage statistics for each
active network interface. In most situations, users should expect to see two stanzas reported. One stanza
contains the configuration for an attached Ethernet card, while the other shows the configuration for the
45
Network Diagnostic Applications
virtual loopback device. The important line is the second line, which displays the IP address assigned to
the interface. If the line containing the IP address is missing, or if the IP address does not look reasonable
for your local network configuration, you can expect to have trouble accessing the network.
Determining Your Default Gateway: /sbin/route
As mentioned in this Lesson's introduction, communicating with hosts on your local subnet uses different
procedures than communicating with hosts on a separate subnet. The Linux kernel, like all Unix kernels,
maintains an internal table which defines which machines should be considered local, and what gateway
should be used to help communicate with those machines which are not. This table is called the routing
table.
If you are a standard user, the route command can be used to display the system's routing table. If you
are the root user, the command can be used to manipulate the table as well. Like the ifconfig command,
the route command lives in the /sbin directory, so standard users should invoke it using an absolute
reference.
[blondie@station blondie]$ /sbin/route
Kernel IP routing table
Destination
Gateway
Genmask
192.168.0.0
*
255.255.255.0
127.0.0.0
*
255.0.0.0
default
server1.example 0.0.0.0
Flags
U
U
UG
Metric
0
0
0
Ref
0
0
0
Use
0
0
0
Iface
eth0
lo
eth0
A standard routing table displays two types entries. The first type defines which subnets should be
considered local subnets. In general, there should be one line specifying a subnet for each active interface.
In the output above, the first line defines the subnet associated with the Ethernet interface (with an IP
address of 192.168.0.51), and the second line defines the subnet associated with the loopback interface
(with an IP address of 127.0.0.1).
The second type of entry, which is used to define gateways, is more relevant to our discussion. Gateway
entries can be distinguished because a host is defined in the second column ("Gateway"), and the fourth
column ("Flags") contains a "G". Every routing table should contain an entry for a "default" gateway, an
the second column should contain the gateway's hostname.
The same information can be displayed using IP addresses instead of hostnames using /sbin/route -n.
[blondie@station blondie]$ /sbin/route -n
Kernel IP routing table
Destination
Gateway
Genmask
192.168.0.0
0.0.0.0
255.255.255.0
127.0.0.0
0.0.0.0
255.0.0.0
0.0.0.0
192.168.0.254
0.0.0.0
Flags Metric Ref
U
0
0
U
0
0
UG
0
0
Use Iface
0 eth0
0 lo
0 eth0
Here, blondie determines that her machine is using the host 192.168.0.254 as its default gateway.
The "G" flag indicates that this line is used to define a gateway, not a local subnet.
Determining Your Nameserver(s)
Domain Name Service allows users to refer to networked computers using hostnames instead of IP
addresses. Unlike the other two aspects of network configuration, a nameserver is to some extent optional.
In order to communicate with other machines at all, your host must have an IP address. In order to
communicate with machines outside of your subnet, you must have a default gateway. If users are willing
to refer to every machine by IP address instead of hostname, however, your machine can communicate
using the IP protocol without ever consulting a nameserver.
In practice, however, nameservers seem more of a necessity. (Do you have an easier time remembering
academy.redhat.com, or 66.187.232.51?) Converting a hostname into an IP address is often referred to as
46
Network Diagnostic Applications
resolving an address, and the library which implements nameservice is called the resolv library. When the
library attempts to resolves an address, there are generally two resources available.
Static DNS Configuration: /etc/hosts
The first resource is a simple text file called the /etc/hosts file. While only root can edit the file, any
user can observe it.
[blondie@station blondie]$ cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1
localhost.localdomain
localhost rha-server
192.168.0.254
server1.example.com server s
192.168.0.1
station1 station1.example.com www1 www1.example.com
192.168.0.51
station51 station51.example.com
192.168.129.201 z
160.168.170.24 rosemont.example.com dhcp-1116-114 r
The format of the file is as simple as it looks. The first token on a line should be an IP address, and
subsequent tokens are hostnames which should resolve to the IP address. The standard Unix comment
character (“#”) is also supported.
If a machine is only communicating with a few machines, or if an administrator wants to create shortcut
hostnames (such as “s”), or if she would like to override the local nameserver, entries can be added to the
/etc/hosts file using a simple text editor.
Obviously, this technique does not scale well. You cannot expect the local /etc/hosts file to provide
every answer.
Dynamic DNS Configuration: /etc/resolv.conf
When the local /etc/hosts cannot provide the answer, the resolv library consults a nameserver. In order
to determine which machine running a nameserver to consult, it examines the resolv library's configuration
file, /etc/resolv.conf.
[blondie@station blondie]$ cat /etc/resolv.conf
; generated by /sbin/dhclient-script
search example.com
nameserver 192.168.0.254
nameserver 207.175.42.153
The host 192.168.0.254 is acting as the primary nameserver.
The host 207.175.42.153 is acting as the secondary nameserver.
The /etc/resolv.conf configuration file uses lines which begin with the keyword nameserver to
specify the IP addresses of machines that are running nameservers. If multiple nameservers are specified,
the first one will be used by default. If it is unavailable, the second will be used, and so on. Accordingly, the
first listed nameserver is sometimes called the primary nameserver, the second listed server the secondary
nameserver, and so on.
Notice that a nameserver does not need to be a nearby machine. Any machine which is implementing
nameservice and allows you to query it can be used as a nameserver. In practice, using a local nameserver
leads to better performance.
Network Diagnostic Utilities
In the previous sections, we have outlined the most direct techniques for determining your machine's local
IP networking configuration. In this section, we introduce diagnostic utilities that can be used to ensure
that the configurations are working appropriately with your local network environment.
47
Network Diagnostic Applications
Confirming IP Connectivity: ping
The ping command can be used to confirm IP connectivity between two hosts. The first argument to ping
can be either the hostname or the IP address of the machine you would like to contact.
[blondie@station blondie]$ ping 192.168.0.254
PING 192.168.0.254 (192.168.0.254) 56(84) bytes of data.
64 bytes from 192.168.0.254: icmp_seq=1 ttl=64 time=0.245 ms
64 bytes from 192.168.0.254: icmp_seq=2 ttl=64 time=0.180 ms
64 bytes from 192.168.0.254: icmp_seq=3 ttl=64 time=0.186 ms
CTRL+C
--- 192.168.0.254 ping statistics --3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.180/0.203/0.245/0.033 ms
When successful, the ping command should display one line per second, including a sequence number and
round trip timing information which reports how long it took to receive a reply from the remote machine.
The ping command will continue to run until canceled with the CTRL+C control sequence.
If no response is returned from the ping command, you cannot expect any higher level networking
communication to occur. One common culprit is an Ethernet cable which has become physically
disconnected from the machine or from the wall.
Inconsistencies in the rate at which lines are displayed or discontinuities in the sequence numbers are
generally indicative of a congested network, or a noisy connection, and you can generally expect poor
network performance of higher level protocols.
Examining Routing: /usr/sbin/traceroute
When connecting to a machine outside of your subnet, your packet is passed from router to router as it
traverses various subnets, until finally the packet is delivered to the subnet which contains the destination
machine. The path of the packet, as it is passed from router to router, can be traced with the /usr/sbin/
traceroute command.
The traceroute command is generally called with a single argument, the hostname or IP address of the
destination machine.
[blondie@station blondie]$ /usr/sbin/traceroute www.whitehouse.gov
traceroute: Warning: www.whitehouse.gov has multiple addresses; using 12.129.72.144
traceroute to a1289.g.akamai.net (12.129.72.144), 30 hops max, 38 byte packets
1 server1 (192.168.0.254) 0.243 ms 0.162 ms 0.252 ms
2 10.44.160.1 (10.44.160.1) 8.563 ms 10.488 ms 7.642 ms
3 srp2-0.rlghncg-rtr1.nc.rr.com (24.25.1.99) 7.382 ms 10.162 ms 0.874 ms
4 srp4-0.rlghnca-rtr2.nc.rr.com (24.25.2.146) 10.008 ms 8.798 ms 15.931 ms
5 srp13-0.rlghncrdc-rtr2.nc.rr.com (66.26.33.178) 4.259 ms 10.079 ms 10.031 ms
6 son0-1-1.chrlncsa-rtr6.carolina.rr.com (24.93.64.81) 29.675 ms 9.950 ms 29.665 ms
7 pop1-cha-P4-0.atdn.net (66.185.132.45) 10.087 ms 21.045 ms 8.202 ms
8 bb2-cha-P2-0.atdn.net (66.185.132.42) 11.392 ms 20.371 ms 9.712 ms
9 bb2-ash-P13-0.atdn.net (66.185.152.50) 29.322 ms 26.192 ms 25.901 ms
10 pop2-ash-P1-0.atdn.net (66.185.139.211) 26.506 ms 26.485 ms 26.208 ms
11 ATT.atdn.net (66.185.140.250) 26.704 ms 27.127 ms 21.004 ms
12 tbr1-p014001.wswdc.ip.att.net (12.123.9.82) 38.888 ms 22.006 ms 28.123 ms13 gbr6-p20.wswdc.ip.a
14 gar3-p370.wswdc.ip.att.net (12.123.9.69) 24.231 ms 29.829 ms 20.827 ms
15 mdf1-gsr12-2-pos-7-0.atl1.attens.net (12.122.255.154) 40.184 ms 50.815 ms
38.903 ms
16 mdf1-bi8k-2-eth-1-1.atl1.attens.net (12.129.64.246) 39.845 ms 39.803 ms 39.887 ms
17 12.129.72.144 (12.129.72.144) 39.856 ms 39.934 ms 39.898 ms
Often, you will find that packets take surprising paths to get from one place to another. The number of
routers that your packet passes through is generally referred to as the number of hops the packet has made.
The packet above took 17 hops to reach its destination.
48
Network Diagnostic Applications
If your packet cannot complete the first hop, your machine's default gateway has probably not been
correctly defined. If your traceroute terminates within the first couple of hops (without reaching the
final destination), the problem is a misconfigured local router, and your local network administrator can
probably help solve the problem. If the traceroute peters out more than four or five hops away, the problem
is probably outside of your local network administrator's control.
Performing DNS Queries Manually: host
The host command can be used to perform DNS queries directly. With a single argument, the host
command will simply report the requested DNS resolution.
[blondie@station rha030]$ host academy.redhat.com
academy.redhat.com has address 66.187.232.51
If the -a command line switch is included, the host command displays detailed information about the query
performed, and the response received, in "resource record" format. Additionally, the final line identifies
the nameserver who resolved the request, and the amount of time the resolution required.
[elvis@station rha030]$ host -a academy.redhat.com
Trying "academy.redhat.com"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 53870
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 0
;; QUESTION SECTION:
;academy.redhat.com.
IN
ANY
;; ANSWER SECTION:
academy.redhat.com.
284
IN
A
66.187.232.51
;; AUTHORITY SECTION:
redhat.com.
redhat.com.
redhat.com.
584
584
584
IN
IN
IN
NS
NS
NS
ns1.redhat.com.
ns2.redhat.com.
ns3.redhat.com.
Received 106 bytes from 192.168.0.254#53 in 30 ms
In this example, the nameserver who provided the answer has an IP address of 192.168.0.254.
Examples
Diagnosing Network Difficulties
The user madonna is having trouble getting her mozilla web browser to connect to the site www.yahoo.com,
and suspects that her local machine might be misconfigured. She proceeds to examine her local network
settings, and confirm that they seem to be functional.
Following the advice to start with the basics, she first ensures that her machine has an IP address by
examining her local network interface configuration.
[madonna@staton madonna]# /sbin/ifconfig
eth0
Link encap:Ethernet HWaddr 00:50:8B:0C:B1:D5
inet addr:109.125.90.86 Bcast:109.125.90.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:513364 errors:0 dropped:0 overruns:0 frame:0
TX packets:319118 errors:0 dropped:0 overruns:0 carrier:0
collisions:5068 txqueuelen:100
RX bytes:78770024 (75.1 Mb) TX bytes:356094835 (339.5 Mb)
Interrupt:11
49
Network Diagnostic Applications
lo
Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:63432 errors:0 dropped:0 overruns:0 frame:0
TX packets:63432 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:6124991 (5.8 Mb) TX bytes:6124991 (5.8 Mb)
Convinced that her interface is active, and using the IP address 109.125.90.86, she next examines her
routing information to determine her default gateway.
[madonna@station madonna]# /sbin/route -n
Kernel IP routing table
Destination
Gateway
Genmask
109.125.90.0
*
255.255.255.0
127.0.0.0
*
255.0.0.0
0.0.0.0
109.125.90.1
0.0.0.0
Flags
U
U
UG
Metric
0
0
0
Ref
0
0
0
Use
0
0
0
Iface
eth0
lo
eth0
She then uses the ping command to confirm that she can contact the gateway.
[madonna@station madonna]# ping 109.125.90.1
PING 109.125.90.1 (109.125.90.1) from 109.125.90.86 : 56(84) bytes of data.
64 bytes from 109.125.90.1: icmp_seq=1 ttl=255 time=2.92 ms
64 bytes from 109.125.90.1: icmp_seq=2 ttl=255 time=2.65 ms
CTRL+C
--- 109.125.90.1 ping statistics --2 packets transmitted, 2 received, 0% loss, time 1004ms
rtt min/avg/max/mdev = 2.652/2.788/2.924/0.136 ms
Satisfied, she cancels the command, and next examines her DNS configuration.
[madonna@station madonna]# cat /etc/resolv.conf
search example.com
nameserver 109.125.90.75
nameserver 109.125.90.2
nameserver 66.218.71.63
Because she has not yet attempted to ping a host outside of her local subnet, she attempts to ping the
tertiary nameserver.
[madonna@station madonna]# ping 66.218.71.63
PING 66.218.71.63 (66.218.71.63) from 109.125.90.86 : 56(84) bytes of data.
64 bytes from 66.218.71.63: icmp_seq=3 ttl=54 time=11.2 ms
64 bytes from 66.218.71.63: icmp_seq=7 ttl=54 time=20.3 ms
64 bytes from 66.218.71.63: icmp_seq=11 ttl=54 time=27.7 ms
CTRL+C
--- 66.218.71.63 ping statistics --13 packets transmitted, 3 received, 76% loss, time 12045ms
rtt min/avg/max/mdev = 11.275/19.766/27.702/6.717 ms
The results of the ping command are a little troubling, because most of her packets were being dropped.
She attempts to traceroute to the machine instead.
[madonna@station madonna]# /usr/sbin/traceroute -n 66.218.71.63
traceroute to 66.218.71.63 (66.218.71.63), 30 hops max, 38 byte packets
1 209.125.90.1 3.473 ms 2.276 ms 2.281 ms
2 10.252.86.221 8.836 ms 15.761 ms 16.423 ms
3 216.217.3.193 11.293 ms 20.227 ms 15.257 ms
4 216.217.3.153 13.123 ms 10.306 ms 7.699 ms
5 63.215.192.173 10.835 ms 6.679 ms 6.881 ms
6 64.159.2.169 6.789 ms 7.117 ms 7.520 ms
7 64.152.69.30 12.358 ms 7.087 ms 6.877 ms
8 66.218.71.63 9.680 ms 11.787 ms 9.963 ms
50
Network Diagnostic Applications
Because traceroute returned such prompt output, madonna assumes that what ever problems she saw with
the previous ping command must have been transient.
Lastly, she confirms that her name service is operational by performing a few DNS queries with the host
command.
[madonna@station madonna]$ host www.redhat.com
www.redhat.com has address 66.187.232.50
[madonna@station madonna]$ host hardware.redhat.com
hardware.redhat.com is an alias for enterprise.redhat.com.
enterprise.redhat.com has address 66.187.233.193
[madonna@station madonna]$ host www.yahoo.com
www.yahoo.com is an alias for www.yahoo.akadns.net.
www.yahoo.akadns.net has address 66.218.71.95
www.yahoo.akadns.net has address 66.218.70.48
...
Convinced that all seems in working order, she tries the mozilla web browser again, and succeeds.
Whatever problems she was having must have been related to the transient network congestion she
witnessed with the ping command.
Online Exercises
Lab Exercise
Objective: Determine Local Networking Configuration
Estimated Time: 15 mins.
Specification
Create the following files, each of which should contain the specified IP address as a "dotted quad".
File
Contents
~/net_ipaddr
The IP address assigned to the interface eth0.
~/net_gw
The IP address of your default gateway.
~/net_ns
The IP address of your primary nameserver.
Deliverables
1.
1. The three files tabled above, each of which contains the specified IP address as its only word.
Questions
1.
Which of the following commands displays the IP addresses assigned to network interfaces?
a.
ping
b.
ifconfig
c.
lsip
d.
ipdump
51
Network Diagnostic Applications
e.
2.
None of the above
Which of the following commands can be used to examine a system's routing table?
a.
ifconfig
b.
lsroute
c.
ipdump
d.
route
e.
None of the above
3.
Which of the following files is used to specify a system's nameserver(s)?
a.
/etc/sysconfig/named
b.
/etc/hosts
c.
/etc/ns.conf
d.
/etc/sysconfig/network
e.
None of the above
4.
Which of the following commands can be used to perform manual DNS queries?
a.
ns
b.
ifconfig
c.
host
d.
route
e.
None of the above
Use the following transcript to answer the next 3 questions.
[madonna@station madonna]$ /sbin/ifconfig
eth0
Link encap:Ethernet HWaddr 00:00:86:4D:F0:0C
inet addr:118.45.92.51 Bcast:118.45.92.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:242939 errors:0 dropped:0 overruns:0 frame:0
TX packets:302515 errors:0 dropped:0 overruns:1 carrier:1
collisions:0 txqueuelen:100
RX bytes:24308852 (23.1 Mb) TX bytes:166603272 (158.8 Mb)
Interrupt:11 Base address:0xd400
lo
Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:29291 errors:0 dropped:0 overruns:0 frame:0
TX packets:29291 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2661822 (2.5 Mb) TX bytes:2661822 (2.5 Mb)
[madonna@station madonna]$ /sbin/route -n
Kernel IP routing table
Destination
Gateway
Genmask
Flags Metric Ref
118.45.92.0
0.0.0.0
255.255.255.0
U
0
0
127.0.0.0
0.0.0.0
255.0.0.0
U
0
0
52
Use Iface
0 eth0
0 lo
Network Diagnostic Applications
0.0.0.0
127.0.0.1
0.0.0.0
UG
[madonna@station madonna]$ cat /etc/resolv.conf
; generated by /sbin/dhclient-script
search example.com
nameserver 66.23.35.53
nameserver 118.45.92.254
5.
0
0
0 lo
What is the IP address of madonna's secondary nameserver?
a.
127.0.0.1
b.
118.45.92.51
c.
66.23.35.53
d.
118.45.92.254
e.
Not enough information is provided.
6.
What is the IP address of madonna's machine?
a.
118.45.92.0
b.
127.0.0.1
c.
118.45.92.51
d.
66.23.35.53
e.
Not enough information is provided.
7.
madonna is having trouble accessing the site www.redhat.com from her web browser. What is most
likely her problem?
a.
Her primary nameserver does not reside on her same subnet.
b.
Her network interface is not active.
c.
Her routing table does not contain entries defining her local subnet.
d.
Her nameserver does not contain the information for www.redhat.com in its database.
e.
Her default gateway is incorrect.
8.
Which of the following command lines would reveal which machine is acting as your nameserver?
a.
ping www.redhat.com
b.
host www.redhat.com
c.
host -a www.redhat.com
d.
ping -a www.redhat.com
e.
None of the above
9.
Which of the following commands can be used to examine how information is routed to a remote
machine?
a.
host
53
Network Diagnostic Applications
b.
ping
c.
route
d.
traceroute
e.
None of the above
10.
In the default Red Hat Enterprise Linux configuration, which file is used when resolving the
hostname localhost?
a.
/etc/inittab
b.
/etc/hosts
c.
/etc/resolv.conf
d.
/etc/sysconfig/localhost
e.
None of the above
54
Chapter 7. Terminal Based Web and
FTP Clients
Key Concepts
• The Elinks terminal HTTP client, invoked as the command links, can be used to browse web pages
from a non graphical environment.
• The curl application is a non-interactive multi-protocol client (including HTTP, HTTPS, and FTP)
which can be used to transfer files in batch mode.
• The wget application is a non-interactive multi-protocol client (including HTTP, HTTPS, and FTP)
which can be used to recursively transfer web pages, recursing through all pages the web page references.
Discussion
Why Terminal Based Clients?
Users are probably familiar with sophisticated graphical applications such a mozilla for browsing the web,
or nautilus for accessing FTP servers. In addition to these applications, Linux provides a collection of
robust command line base clients as well. Why would someone be tempted to use command line based
clients instead of point and click applications?
• The obvious reason: If you are not running an X graphical environment, they are the only utilities
available.
• Command line utilities are scriptable. If you are repeatedly pulling information from the same location,
you might consider writing a script to do the repetitive work for you.
• Command line utilities require less resources. If you are trying to perform a long download on an
otherwise busy machines, a curl process which consumes 4 megabytes of memory is much more
attractive than a mozilla process which consumes 40 megabytes of memory.
• Simplicity tends to encourage stability. When downloading large files, such as 700 Megabyte ISO
images, many of the large graphical applications can become unstable. (The list of culprits includes
Internet Explorer). Command line clients such as curl tend to produce more stable results.
Browsing Web Pages with Elinks
The Elinks text HTTP client (web browser) is designed to render web pages in a terminal, emulating the
layout of a graphical web browser. As much as possible, tables and forms are rendered to scale. Note that
in the following display of a web page associated with Red Hat Network, frames and tables are rendered
appropriately.
Starting Elinks
Elinks is started as the links command, specifying a URL to load as an argument, as in the following.
[elvis@station elvis]$ links http://www.redhat.com
55
Terminal Based Web and FTP Clients
Figure 7.1. The Elinks Text Web Browser
56
Terminal Based Web and FTP Clients
When using links, the screen is active, and links may be navigated using arrow keys or the mouse. By
using the ESC key, a menu is created which allows a user to access bookmarks, history, and customization
dialogs.
Although the -dump command line switch can be used to non-interactively render a page to standard out,
and the -source command line switch will do the same to the "RAW" content of the URL (often dumping
HTML), the support for scripted interactions is fairly unsophisticated when compared to other utilities
such as curl. The links browser's strength is its ability to provide a comfortable web browsing experience
in a text environment.
Configuring Elinks to use a Proxy Server
If the HTTP_PROXY environment variable is set, Elinks will use the specified proxy server, where (like
the Elinks browser) the proxy server can be specified using the following syntax.
http://servername:port
Fetching Web Resources with curl
In contrast the the links command, the curl command line web client makes no effort to render or
interactively browse web pages. Instead, the development of curl has focused on a robust collection of
command line switches that makes it well suited for the automated retrieval of files which are published
using a web or ftp server. This text would have a hard time describing curl any more effectively or
succinctly than the first two paragraphs of the curl(1) man page:
curl is a client to get documents/files from or send documents to a
server, using any of the supported protocols (HTTP, HTTPS, FTP, GOPHER,
DICT, TELNET, LDAP or FILE). The command is designed to work without
user interaction or any kind of interactivity.
curl offers a busload of useful tricks like proxy support, user authentication, ftp upload, HTTP post, SSL (https:) connections, cookies,
file transfer resume and more.
Starting curl
When called with a URL as its argument, the curl command retrieves the URL, and dumps its contents
to standard out. Additionally, some timing information is written to standard error, which is discarded in
the following example.
[elvis@station rha030]$ curl http://www.redhat.com 2>/dev/null | head
<html>
<head>
<title>Red Hat -- Linux, Embedded Linux and Open Source Solutions</title>
<meta name="MSSmartTagsPreventParsing" content="TRUE">
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" media="all">@import "/css/non_ns4.css";</style>
<link rel="stylesheet" TYPE="text/css" HREF="/css/homepage.css">
The following table lists a few of the command line switches which can be used to modify curl's behavior.
Consult the curl(1) man page for a complete list.
Table 7.1. Command Line Switches for the curl Command
Switch
Effect
-b, --cookie filename
Supply cookie values from the specified file.
57
Terminal Based Web and FTP Clients
Switch
Effect
-c, --cookie-jar filename
Store retrieved cookies in the specified file.
-C, --continue-at position
Resume the transfer of a large file at the specified position. If the offset
is given as “-”, curl will try to automatically figure out the appropriate
position.
-d data
Supply data as if submitting a POST request.
-i
Include HTTP headers in the output.
-L, --location
Follow redirects to a new location.
-m, --max-time seconds
Kill connection after the specified number of seconds has elapsed.
-O, --remote-name
Write output to file of same name as file referenced in the URL.
-u,
username:password
--user Authenticate using the supplied username/password pair.
As an example, the following command line could be used to download the ISO images for the Fedora
Core 1 release:
[elvis@station elvis]$ curl -C - -O -L "http://download.fedora.redhat.com/pub/fe
dora/linux/core/1/i386/iso/yarrow-i386-disc{1,2,3}.iso"
[1/3]: http://download.fedora.redhat.com/pub/fedora/linux/core/1/i386/iso/yarrow
-i386-disc1.iso --> yarrow-i386-disc1.iso
% Total
% Received % Xferd Average Speed
Time
Curr.
Dload Upload Total
Current Left
Speed
0 629M
0 536k
0
0 13375
0 13:42:49 0:00:41 13:42:08 75883
The following details should be noted about the curl command line.
• Notice the use of "brace expansion syntax" in the referenced URL. The curl command will attempt to
download three files, yarrow-i386-disc1.iso, yarrow-i386-disc2.iso, and yarrowi386-disc3.iso.
• Because of the -O command line switch, the files will be downloaded to the local directory with the
same names.
• Notice the time left at the current throughput: 13 hours! If the command needs to be aborted and later
restarted, the -C - switch tells curl to examine any already existing partial downloads, and pick up the
transfer where it left off.
• The -L command line switch tells curl to follow any redirects issued by the web server. (Although not
evident from the output, curl was redirected to a FTP server.)
Configuring curl to use a Proxy Server
If the http_proxy environment variable is set, curl will use the specified proxy server, where the proxy
server can be specified using the following syntax.
http://servername:port
Recursively Downloading Web Resources with wget
The wget command line web client is designed to non-interactively use the HTTP, HTTPS, and FTP
protocols to download resources, possibly recursing to pages referenced from downloaded pages.
58
Terminal Based Web and FTP Clients
Starting wget
When called with a URL as its argument, the wget command retrieves the contents of the URL and stores
it in a local file in the current working directory of the same name (or index.html if the URL specified
a directory).
The following table lists a few of the command line switches which can be used to modify wget's behavior.
Consult the wget(1) man page for a complete list.
Table 7.2. Command Line Switches for the wget Command
Switch
Effect
-i, --input-file filename
Read URL's to fetch from the file filename, which may either be a
HTML file, or list the URL's sequentially as text.
-B, --base URL
Prepend URL to all relative links.
--spider
Don't download pages, just confirm their presence.
--http-user=user
passwd=passwd
--http- Authenticate using the specified username and password.
-o filename
Concatenate the contents of all downloaded files to the file
filename. The special filename - implies standard out.
-r, --recursive
Enable recursive retrieving.
-l, --level depth
Do not recurse beyond depth levels of recursion.
-np, --no-parent
When recursing, only include links underneath the parent URL.
As an example, the following command line could be used to mirror the content of the http://
www.redhat.com/training site.
[elvis@station mirror]$ wget -r http://www.redhat.com/training
[elvis@station mirror]$ wget -r http://www.redhat.com/training
--02:53:01-- http://www.redhat.com/training
=> `www.redhat.com/training.1'
Resolving www.redhat.com... done.
Connecting to www.redhat.com[66.187.232.50]:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://www.redhat.com/training/ [following]
--02:53:01-- http://www.redhat.com/training/
=> `www.redhat.com/training/index.html'
Connecting to www.redhat.com[66.187.232.50]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 29,575 [text/html]
100%[====================================>] 29,575
175.04K/s
ETA 00:00
02:53:02 (175.04 KB/s) - `www.redhat.com/training/index.html' saved [29575/29575]
Loading robots.txt; please ignore errors.
--02:53:02-- http://www.redhat.com/robots.txt
=> `www.redhat.com/robots.txt'
Reusing connection to www.redhat.com:80.
HTTP request sent, awaiting response... 200 OK
Length: 262 [text/plain]
100%[====================================>] 262
...
255.86K/s
ETA 00:00
And off it goes. After a little observation, it should become obvious that when called recursively, the wget
command often recurses beyond expectations. After canceling the previous command with a CTRL+C, the
59
Terminal Based Web and FTP Clients
following directories demonstrate that wget strayed beyond the bounds of the www.redhat.com/training
website.
.
|-|
|-|
|-|
|-|
|-|
|-|
`--
certcities.com/
`-- robots.txt
rhn.redhat.com/
`-- index.html
secure.safaribooksonline.com/
`-- promo.asp?code=ORA14&portal=oreilly
www.europe.redhat.com/
`-- robots.txt
www.google.com/
`-- robots.txt
www.oreillynet.com/
`-- robots.txt
www.redhat.com/
|-- about/
|
|-- careers/
|
|
`-- index.html
|
|-- contact/
|
|
`-- index.html
|
|-- corporate/
|
|
`-- wwoffices/
|
|
`-- index.html
|
|-- index.html
|
`-- presscenter/
|
|-- 2002/
|
|
|-- press_bluepoint.html
|
|
|-- press_rhct.html
|
|
|-- press_training.html
|
|
`-- press_veterans.html
|
`-- 2003/
|
`-- press_rhacademy.html
|-- apps/
|
|-- commerce/
|
|
`-- index.html
...
Use with care.
Configuring wget to use a Proxy Server
Like the curl command, wget uses the http_proxy environment variable to specify a proxy server,
again using the following syntax.
http://servername:port
Examples
Using terminal based file transfer utilities
wget with retry options can be useful for fetching files from a busy ftp site.
[einstein@station einstein]$ wget --tries=50 --wait=30 ftp://updates.redhat.com/
8.0/en/os/i386/wget-1.8.2-5.i386.rpm
--16:19:24-- ftp://updates.redhat.com/8.0/en/os/i386/wget-1.8.2-5.i386.rpm
=> `wget-1.8.2-5.i386.rpm'
Resolving updates.redhat.com... done.
Connecting to updates.redhat.com[66.187.232.52]:21... connected.
Logging in as anonymous ... Logged in!
60
Terminal Based Web and FTP Clients
==> SYST ... done.
==> PWD ... done.
==> TYPE I ... done. ==> CWD /8.0/en/os/i386 ... done.
==> PORT ... done.
==> RETR wget-1.8.2-5.i386.rpm ... done.
Length: 365,737 (unauthoritative)
100%[====================================>] 365,737
4.83K/s
ETA 00:00
16:20:40 (4.83 KB/s) - `wget-1.8.2-5.i386.rpm' saved [365737]
When using wget to mirror a web site, you can limit number of levels of recursion
[einstein@station einstein]$ wget --recursive --level=1 --convert-links http://w
ww.redhat.com/training/
Online Exercises
Lab Exercise
Objective: Gain familiarity with terminal based web clients.
Estimated Time: 20 mins.
Set Up
If you use a proxy server to access the Internet, you will need to configure your web clients by setting the
http_proxy and/or HTTP_PROXY environment variables. Your instructor will give you guidance.
If this is the case, place your configuration in the file ~/.bash_profile, so that the configuration
takes place automatically at shell startup.
Specification
Extracting bits of text from web pages using crude tools such as grep is known as screen scraping. Screen
scraping is tricky, as the format of web pages can easily change out from under you. Still, there are times
when it's useful, and it's a good trick to know.
1. Create the directory ~/wget, and cd into it. Use the wget command to recursively download
the site www.redhat.com, but use the -Q 256k command line switch to limit your download to a
maximum of 256 kilobytes. When completed, your ~/wget directory should contain the directory
www.redhat.com, and possibly others.
2. The CIA publishes a factbook on countries at the site https://www.cia.gov/library/publications/theworld-factbook/index.html More specifically, URLs such as https://www.cia.gov/library/publications/
the-world-factbook/geos/ca.html refer to country specific pages, where countries are specified using a
two letter abbreviation in the filename portion of the URL.
Use the Elinks browser to examine the factsheets on the following countries, specifying the URL to the
country's page directly from the command line.
Abbreviation
Country
as
Australia
ch
China
ni
Nigeria
61
Terminal Based Web and FTP Clients
After you are familiar with the format of each page, write a short script called ~/bin/
get_birthrate which combines the curl and grep commands. The script should expect as its single
argument a country's two letter abbreviation. The script should download the relevant country page,
and reduce the output to only the line which contains information about the country's birth rate.
When finished, the script should produce output akin to the following.
[student@station student]$ get_birthrate au
8.69 births/1,000 population (2007 est.)
[student@station student]$ get_birthrate ch
13.45 births/1,000 population (2007 est.)
[student@station student]$ get_birthrate ni
40.2 births/1,000 population (2007 est.)
The following script would meet the specification.
[student@station student]$ cat bin/get_birthrate
#!/bin/bash
URLBASE=https://www.cia.gov/library/publications/the-world-factbook/geos
COUNTRY=$1
curl $URLBASE/$COUNTRY.html 2>/dev/null | grep "births/1,000"
Deliverables
1.
1. The directory ~/wget, which contains the first 256 Kbytes (or so) of a recursive download of
the website http://www.redhat.com
2. The script ~/get_birthrate, which when invoked with a two letter country abbreviation,
extracts the appropriate line from the URL https://www.cia.gov/library/publications/the-worldfactbook/geos/ca.html (where “ca.html” should be replaced with the appropriate two letter
abbreviation).
Questions
1.
Which of the following commands could be used to interactively browse a web site from a terminal?
a.
links
b.
curl
c.
wget
d.
elinks
e.
None of the above
2.
Which of the following commands could be used to non interactively download a web page and
all of the pages to which that web page to referred?
a.
elinks
b.
wget
c.
curl
62
Terminal Based Web and FTP Clients
d.
links
e.
None of the above
Use the following command line to answer the next 3 questions.
[madonna@station madonna]$ curl -C - -O -L "http://download.fedora.redhat.com/pu
b/fedora/yarrow-i386-disc1.iso"
3.
What will be the name of the resulting file?
a.
index.html
b.
download.fedora.redhat.com/pub/fedora/yarrow-i386-disc1.iso
c.
yarrow-i386-disc1.iso
d.
The user will be prompted for the filename.
e.
The file will be transferred directly to standard out.
4.
What is the purpose of the -C - command line switch?
a.
The file should be transferred directly to standard out.
b.
All cookies should be accepted.
c.
If evidence of a partial previous download exists, pick up the current download where the
previous left off.
d.
Configuration arguments should be read from standard in.
e.
None of the above.
5.
What is the purpose of the -L command line switch?
a.
Any local proxy server settings should be ignored.
b.
All cookies should be accepted.
c.
If curl is redirected to another location, it should silently follow the redirect.
d.
Display timing information associated with the download.
e.
None of the above.
6.
7.
What is the purpose of the -O command line switch?
a.
Output should be transferred directly to standard out.
b.
The output file should be named according to the file portion of the requested URL path.
c.
The download should be optimized for a high speed connection.
d.
Options should be read directly from the standard in stream.
e.
None of the above.
Which of the following environment variables, if set, will cause curl to use a HTTP proxy server?
63
Terminal Based Web and FTP Clients
a.
PROXY
b.
http_proxy
c.
HTTP_PROXY
d.
HTTP_proxy
e.
None of the above
8.
Which of the following command lines could be used to transfer the contents of the file http://
server1/pub/README to standard out?
a.
links -source http://server1/pub/README
b.
curl http://server1/pub/README
c.
wget -O - http://server1/pub/README
d.
A and C only
e.
A, B, and C
9.
Which of the following commands can download files from a FTP server?
a.
curl
b.
wget
c.
links
d.
A and B only
e.
A, B, and C
An administrator has prepared the following file to automatically initialize relevant proxy server
environment variables on startup.
[root@station root]# cat proxy.sh
if [ -z "$HTTP_PROXY" ]; then
HTTP_PROXY=http://proxy.example.com:8080
fi
if [ -z "$http_proxy" ]; then
http_proxy=$HTTP_PROXY
fi
10.
In what directory should the administrator install the file?
a.
/etc/profile.d
b.
/etc/rc.d/init.d
c.
/etc/sysconfig
d.
/etc/skel
e.
None of the above
64
Chapter 8. Remote Shell Commands
Key Concepts
• Remote shell applications allow users to execute arbitrary commands on remote machines, and have
standard out returned locally. Alternately, an interactive shell can be started.
• The Secure Shell application provides a remote shell, where all transactions are encrypted, and users
can be authenticated by traditional passwords or using a public key authentication scheme.
• In order to use the public key authentication scheme, a public- private key pair must be generated with
the ssh-keygen command.
• Because Secure shell servers have their own public-private key pairs, servers can be authenticated to
clients as well as clients authenticated to servers.
Discussion
The Original Remote Shell: rsh and Rhosts
Authentication
Remote Shells with rsh
Linux (and Unix) shells are intentionally designed with simple interfaces: they read input from the standard
in stream, and deliver output to the standard out stream. As a result, the interfaces are easily implemented
over network connections. By simply substituting a TCP socket for a terminal device, a shell can operate
on a remote machine as easily as the local machine. In Linux (and Unix), applications which provide this
functionality are referred to as remote shells.
The first commonly used Unix remote shell was the simple rsh application. If a remote machine is properly
configured (i.e., it is running the RSH server), users can use a rsh command line akin to the following
to invoke a remote shell.
[elvis@station elvis]$ rsh -l elvis server1 ls /tmp
jd_sockV4
lost+found
orbit-root
ssh-WjMO1585
[elvis@station elvis]$
Translating, this command says "as the user elvis on the host server1, run the command ls /tmp". The
command executes on the remote machine (server1), but standard out is delivered to the local machine
(station). When the command completes, elvis's prompt implies that he is still on the host station.
If elvis does not specify a command to run, the rsh utility would opens an interactive shell on the remote
host server1. By paying close attention to the bash prompt in the following excerpt, note which commands
execute on which machine.
[elvis@station elvis]$ rsh -l elvis server1
Last login: Sat Nov 8 18:23:49 from station.example.com
[elvis@server1 elvis]$ hostname
server1.example.com
[elvis@server1 elvis]$ who
root
tty1
Nov 8 16:56
65
Remote Shell Commands
root
tty2
Nov 8 16:56
root
pts/0
Nov 8 16:57 (:0.0)
elvis
pts/4
Nov 8 18:28 (station)
[elvis@server1 elvis]$ exit
rlogin: connection closed.
[elvis@station elvis]$
Rhosts Authentication: ~/.rhosts
In each case, elvis did not need issue a password. Before he could access his remote account using rsh,
however, elvis needed to configure the account to allow him access from his local machine. For rsh,
access control configuration is as trivial as adding a line to a file. On the remote account, elvis created the
file ~/.rhosts, and added one line containing the hostname and username for each external account
which he wanted to grant access. Additionally, the RSH server requires that the file's permissions prohibit
anyone but the user owner from reading the file. As the following commands illustrate, elvis has already
configured his .rhosts file on the remote machine.
[elvis@station elvis]$ rsh -l elvis server1 cat .rhosts
station elvis
[elvis@station elvis]$ rsh -l elvis server1 ls -l .rhosts
-rw------1 elvis
elvis
16 Nov 8 18:23 .rhosts
Authentication which relies on a properly configured ~/.rhosts configuration file is commonly called
rhosts authentication.
The Secure Shell
The rhosts authentication method is pitiful. At its essence, it relies on DNS (Domain Name Service) to
authenticate a user. In order to exploit elvis's ~/.rhosts configuration, all someone would have to do
is detach the real host station from the network, and bring up another machine configured with station's
IP address. The fault is not elvis's, but the design of the rhost authentication protocol.
Additionally, rsh is a plaintext protocol. Exchanging data over a network in plaintext is essentially the
equivalent to sending mail on postcards: anyone handling the data between here and there is privy to its
contents.
The Secure Shell was developed to address both of these shortcomings of the rsh command, and add
significant new capabilities, while still providing all of rsh's convenience. Assuming the remote machine
is running the ssh service (i.e., the sshd daemon), elvis could invoke a shell on the remote machine with
the following.
[elvis@station elvis]$ ssh elvis@server1 ls /tmp
elvis@server1's password:
jd_sockV4
lost+found
orbit-root
[elvis@station elvis]$
The ssh command's new syntax for specifying the username of the remote account is slightly easier than is
rsh's, although the -l command line switch is also implemented (in order to be fully backwards compatible).
In the above example, elvis is authenticated by providing a password instead of configuring a ~/.rhosts
file. In the next section, we find that the Secure Shell can use a more mature public key technique to grant
users "password free" access to an account. When public key authentication is not implemented, however,
ssh falls back to traditional password authentication. 1
1
Most versions of rsh, when rhosts authentication fails, will also fall back to traditional password authentication. Because rsh is a plaintext protocol,
however, any supplied passwords would be exposed to anyone eavesdropping on the conversation. As a security precaution, in Red Hat Enterprise
Linux, rsh password authentication has been disabled.
66
Remote Shell Commands
Secure Shell Public Key Authentication
In addition to traditional password authentication, the Secure Shell application can use public key
cryptography to authenticate users. Public key encryption algorithms relate two large numbers, referred
to as "keys", so that information encrypted with one key can only be decrypted with the other. Anyone
who wants to use public key cryptography must first generate a pair of keys. Most public key protocols
call one key a public key, and the complementary key a private key. Your public key you treat like your
phone number: you share it with anyone with whom you are willing to communicate, and may choose to
list it in public directories. Your private key, on the other hand, you share with no one. All of the security
provided by public key protocols relies on the fact that only you know the contents of your private key.
Generating a Public-Private Key Pair: ssh-keygen
When using ssh, a user's public-private key pair can be generated with the ssh-keygen command. In the
following example, elvis uses ssh-keygen to generate a ssh public-private key pair.
[elvis@station elvis]$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/elvis/.ssh/id_rsa): RETURN
Enter passphrase (empty for no passphrase): RETURN
Enter same passphrase again: RETURN
Your identification has been saved in /home/elvis/.ssh/id_rsa.
Your public key has been saved in /home/elvis/.ssh/id_rsa.pub.
The key fingerprint is:
e0:71:43:df:ed:40:01:0b:44:54:db:c2:80:f2:33:aa elvis@station
The user elvis was first prompted for the new (private) key's filename, to which elvis simply hit RETURN
to accept the default filename: ~/.ssh/id_rsa. Next, elvis was given the opportunity to attach a
passphrase to his private key. By hitting RETURN again (twice), elvis chose not to. (We will discuss
passphrases in more detail later.)
When the command returns, elvis has two new files in his (perhaps newly created) ~/.ssh directory.
The first is his private key, which he shares with no one. (He certainly doesn't publish it in an online text).
[elvis@station elvis]$ cat .ssh/id_rsa
-----BEGIN RSA PRIVATE KEY----MIICWwIBAAKBgQClJnymgdK0myP41/DcIyR9aam0DZQJUT20RLfqQb8ptk90jXSL
FrcIR2Ia59W/kJVLo4pqwJDsEJetWdhYiKUVJTANxbV2Pv21OACMlYcM316YLTOm
...
qigTMYAxoBKwPVnpAkEAvHl24SepSlAuSIwgtbluJApOfaDTizIAHh/G8PPFvH1e
p0J+MM7d/qFjg9gpcqZN34LOW8lD7Ab/GTQGl/XsWw==
-----END RSA PRIVATE KEY-----
He is free to share his second key (the public key) with anyone whom asks.
[elvis@station elvis]$ cat .ssh/id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEApSZ8poHStJsj+Nfw3CMkfWmptA2UCVE9tES36kG/KbZP
dI10ixa3CEdiGufVv5CVS6OKasCQ7BCXrVnYWIilFSUwDcW1dj79tTgAjJWHDN9emC0zpiHqBGY9dvMo
2XHobNmgQYTRFVv2NBNTA5/Zpt+Ml0+M9+uzlpxl03PcjFk= elvis@station
Allowing Account Access: ~/.ssh/authorized_keys
SSH access to an account is granted by obtaining a copy of the public key of the person who is to be
granted access, and storing it in the account's ~/.ssh/authorized_keys file. Like the ~/.rhosts
file, the ~/.ssh/authorized_keys file, and the whole ~/.ssh directory, must only be readable
by the user. How the copy of the public key is obtained does not matter. It could be emailed, scped (as
discussed in a moment), or transferred from one terminal to another using the mouse's cut and paste buffer.
When handling public keys, however, care must be taken to ensure that the key is placed in the file with
no embedded whitespace, including newlines. Although too long to be displayed as such, SSH public keys
67
Remote Shell Commands
are always stored as a single line of text. More people can be granted access to an account by simply
appending their public keys to the ~/.ssh/authorized_keys files, one public key per line.
Figure 8.1. SSH Public Key Authentication
In the following example, elvis uses ssh, redirection, and some carefully placed quotes to append his public
key (on the host station) to the authorized_keys file in his account on the host serer1.
[elvis@station elvis]$ ssh elvis@server1 "cat >> .ssh/authorized_keys" < .ssh/id_rsa.pub
elvis@server1's password:
bash: .ssh/authorized_keys: No such file or directory
Okay, so we need to make the .ssh directory first.
[elvis@station ~]$ ssh rha-server mkdir .ssh
elvis@server1's password:
[elvis@station elvis]$ ssh elvis@server1 "cat >> .ssh/authorized_keys" < .ssh/id_rsa.pub
elvis@server1's password:
Why the quotes? With no quotes, the output of the cat command would have been appended to the file
.ssh/authorized_keys on the local machine. The quotes serve to pass the redirection syntax "into"
the remote shell.
Having placed his public key in the ~/.ssh/authorized_keys file on the remote machine, elvis
now expects to be able to examine the contents of the file without having to issue a password.
[elvis@station elvis]$ ssh elvis@server1 "cat .ssh/authorized_keys"
elvis@server1's password:
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEApSZ8poHStJsj+Nfw3CMkfWmptA2UCVE9tES36kG/KbZP
dI10ixa3CEdiGufVv5CVS6OKasCQ7BCXrVnYWIilFSUwDcW1dj79tTgAjJWHDN9emC0zpiHqBGY9dvMo
2XHobNmgQYTRFVv2NBNTA5/Zpt+Ml0+M9+uzlpxl03PcjFk= elvis@station
Something is amiss, because elvis was again prompted for his password. Recalling that only the user
should be able to access the ~/.ssh directory and read the ~/.ssh/authorized_keys file, elvis
implements the appropriate chmod command on the remote machine. Afterwords, he is able to observe
the new permissions without having to issue a password.
[elvis@station elvis]$ ssh elvis@server1 chmod -R go-rwx .ssh
elvis@server1's password:
[elvis@station elvis]$ ssh elvis@server1 ls -l .ssh/authorized_keys
-rw------1 elvis
elvis
225 Nov 22 22:29 .ssh/authorized_keys
[elvis@station elvis]$
Success.
Public Key Authentication Details
In order to develop an appreciation for the robustness of public key authentication, we will spend a few
moments discussing the protocol. When the secure shell application implements public key authentication,
it uses a procedure similar to the following. In our discussion, the following symbols will be used.
68
Remote Shell Commands
Symbol
Interpretation
S
The contents of elvis's private ("secret") key
P
The contents of elvis's public key
R
A random string
P(R)
The random string encrypted by elvis's public key
S(P(R)) = R
The random string first encrypted by elvis's public key, and then decrypted by elvis's
private key.
First, the ssh client on the host station requests a connection to the sshd daemon on the host server1. Upon
receiving the connection request, the sshd daemon looks for a registered public key in the destination
account's ~/.ssh/authorized_keys file.
If a relevant public key is discovered, the sshd daemon initiates public key authentication by generating
a random string R. It then encrypts the random string with elvis's public key P (which it obtains from the
~/.ssh/authorized_keys file), and delivers the encrypted random string P(R) over the network
to the ssh client.
Figure 8.2. Public Key Authentication Algorithm (1 of 3)
Upon receiving the encrypted random string P(R), the ssh client uses elvis's private key S to decrypt it.
Once the original random string R is recovered, the ssh client returns it to the sshd daemon.
Figure 8.3. Public Key Authentication Algorithm (2 of 3)
If the sshd daemon receives from the ssh client the same random string with which it started, the client is
authenticated, and the connection is allowed to continue.
69
Remote Shell Commands
Figure 8.4. Public Key Authentication Algorithm (3 of 3)
A couple of aspects of this algorithm deserve mentioning.
• The ssh client is authenticated not by a hostname or IP address, and not by a password, but solely by
the possession of the private key. (If the client could not access the appropriate private key, it would not
have been able to decrypt the encrypted random string passed to it.)
• The only information passed over the network is an encrypted random string, and a random string (the
symbols colored red in the accompanying figures). Anyone eavesdropping on the conversation would
not learn anything useful.
In practice, the actual algorithm used is more complicated. But the protocol outlined above illustrates the
most important features of the public key authentication protocol.
Transferring Files Securely and Easily: scp
As the previous discussion illustrates, files can be transferred from one machine to another using ssh with
the cat command and careful redirection. Fortunately, there is an easier and less error prone way: scp.
The scp command uses a syntax almost identical to the cp command, but either the source file(s) or the
destination file can be on a remote machine, accessed through a specified account. 2 When referring to a
file on a remote machine, the following syntax is used.
user@host:path
The user and host are simply enough the host where the file resides, and the user whose account is
used to access the file. If the file's path begins with a “/”, it is considered an absolute reference. If not,
it is considered relative to the user's home directory. If no path is supplied, the user's home directory
is assumed.
As an example, the following command line would transfer the /etc/services file from server1 into
the ~/cfg/server1/etc/ directory in elvis's home directory.
[elvis@station elvis]$ scp elvis@server1:/etc/services cfg/server/etc/
services
100% |*****************************| 19936
00:00
Because elvis has a properly configured public key authentication with his account on server1, he is able to
transfer the file without issuing a password. What happens if he tries to transfer the file /etc/shadow?
[elvis@station elvis]$ scp elvis@server1:/etc/shadow cfg/server/etc/
scp: /etc/shadow: Permission denied
2
In fact, the source and destination files can be (different) remote machines, though in practice, managing the streams appropriately is problematic.
70
Remote Shell Commands
The user elvis on the host server1 does not have permissions to read the file /etc/shadow, so the file
can naturally not be transferred. If the user elvis knows the password to the root account on the remote
machine, however, the file could be accessed through it.
[elvis@station elvis]$ scp root@server1:/etc/shadow cfg/server/etc/
root@server1's password:
shadow
100% |*****************************| 2588
00:00
Because elvis does not have public key authenticated access to the root account on server1, ssh used
traditional password authentication.
The -r command line switch (for "recursive") must be specified when copying an entire directory (and its
subdirectories). In the following, elvis recursively copies the /etc/sysconfig directory from his local
machine (station) to the machine server1's /tmp directory.
[elvis@station elvis]$ scp -r /etc/sysconfig elvis@server1:/tmp
ifup-aliases
100% |*****************************| 13137
ifcfg-lo
100% |*****************************|
254
ifdown
100% |*****************************| 3676
ifdown-ippp
100% |*****************************|
820
ifdown-ipv6
100% |*****************************| 4076
...
00:00
00:00
00:00
00:00
00:00
As the scp command performs the transfer, it displays transfer timing information for each file.
Secure Shell Host Authentication
The first time the ssh (or scp) client is used to connect to a sshd Secure Shell server, a message similar
to the following is displayed.
[elvis@station elvis]$ ssh elvis@server1 who
The authenticity of host 'server1 (192.168.0.254)' can't be established.
RSA key fingerprint is fc:c8:87:90:f0:39:af:4f:de:99:cc:30:ce:64:b2:8e.
Are you sure you want to continue connecting (yes/no)?
If the user answers yes (the only answer which will allow the connection to continue), the connection
proceeds, with the following warning.
Warning: Permanently added 'server1,192.168.0.254' (RSA) to the list of known hosts.
root
tty6
Nov 8 22:03
root
pts/1
Nov 8 22:17 (:0.0)
On subsequent connections, the message is no longer seen.
[elvis@station elvis]$ ssh elvis@server1 who
root
tty6
Nov 8 22:03
root
pts/1
Nov 8 22:17 (:0.0)
[elvis@station elvis]$
The Secure Shell not only authenticates clients to servers, but also servers to clients, using public key
authentication. Just as users can create public-private key pairs with the ssh-keygen command, the sshd
daemon maintains its own public-private key pair, known as its host key. The first time a ssh client connects
to a sshd daemon, it appends a copy of the remote daemon's public host key to the local file ~/.ssh/
known_hosts.
[elvis@station elvis]$ cat .ssh/known_hosts
server1,192.168.0.254 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEAvaQ4ILVi9lceyBuGo9KUFY
ksKtPT8BsLPkLZYLIRVxmXDtG1+W+qxiAgw6qCROX8fAvBkdGS4gegt06NVpKItW87K5Wq6OMIlUfwfX
OmPNOWrdA+1Wym0LXYnkUDEOV8xvTUtGzy4MRIl0Phi92uJYEJkKsHMOCWGazN/DclBZk=
As the client is used to connect to various machines, the ~/.ssh/known_hosts file grows, recording
one public key for each machine contacted. The next time the client connects to a host, it silently uses
71
Remote Shell Commands
the same public key protocol used to authenticate users, reversed, to authenticate the host to which it is
connecting.
What if the remote host does not posses the same identity which complements the public key stored in
the client's ~/.ssh/known_hosts file? The client refuses to connect, and instead issues the following
warning.
[elvis@station elvis]$ ssh elvis@server1 ls /tmp
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@
WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!
@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
c8:96:41:69:84:38:57:4c:18:55:7a:16:04:33:4d:f1.
Please contact your system administrator.
Add correct host key in /home/elvis/.ssh/known_hosts to get rid of this message.
Offending key in /home/elvis/.ssh/known_hosts:1
RSA host key for server1 has changed and you have requested strict checking.
Host key verification failed.
[elvis@station elvis]$
Here, the ssh client is identifying the line from the ~/.ssh/known_hosts file which contains
the key which no longer complements the server.
Often, there is a very reasonable explanation why the server has changed identity. For example, the server
might have been upgraded with a more recent version of its operating system, and as a result, generated a
new host public-private key pair. If a reasonable explanation for the change in identity is available, the ssh
client can be convinced to connect by removing the offending line from the ~/.ssh/known_hosts
file, and "starting over" by collecting a new key for the host.
[elvis@station elvis]$ rm ~/.ssh/known_hosts
[elvis@station elvis]$ ssh elvis@server1 ls /tmp
The authenticity of host 'server1 (192.168.0.254)' can't be established.
RSA key fingerprint is c8:96:41:69:84:38:57:4c:18:55:7a:16:04:33:4d:f1.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'server1,192.168.0.254' (RSA) to the list of known hosts.
jd_sockV4
lost+found
orbit-root
ssh-ayfI2751
sysconfig
Examples
Accessing a Remote Account
The emperor julius has recently subscribed to a DSL line at home, and would like to be able to access
information on his home machine from work. On his home machine, he added an account for himself
using his nickname, jules. He does not know the hostname of his home machine, but before leaving for
work, he jotted down its current IP address: 69.57.97.126. At work, he uses a machine with a hostname
of emporer.rome.gov, and his account name is julius.
He first confirms that he can connect to his home machine with the ssh client.
[julius@emperor julius]$ ssh jules@69.57.97.126
The authenticity of host '69.57.97.126 (69.57.97.126)' can't be established.
RSA key fingerprint is 89:01:b1:55:7e:6f:da:0c:bc:fc:19:62:af:84:d6:7f.
Are you sure you want to continue connecting (yes/no)?
72
Remote Shell Commands
As this is the first time he has accessed his home machine using ssh, he accepts the host key by answering
yes. At the following password prompt, he enters the password for his home account.
Warning: Permanently added '69.57.97.126' (RSA) to the list of known hosts.
jules@69.57.97.126's password:
/usr/X11R6/bin/xauth: creating new authority file /home/jules/.Xauthority
[jules@localhost jules]$
Julius is now satisfied that he can access his home account.
Configuring Public Key Authentication
Now that Julius knows that he can access his home account, he would like to configure public key
authentication, so that he can shell directly to his home machine from work, without having to issue a
password. He starts by generating a public-private key pair on his work machine. Because the default
configuration is appropriate for him, he merely hits the RETURN key for all of the associated questions.
[julius@emperor julius]$ ssh-keygen
...
After confirming that his public-private key files exist, he uses the scp command to copy his public key
from his work machine to the file .ssh/authorized_keys on his home machine, again authenticating
himself with his password.
[julius@emperor julius]$ ls .ssh/
id_rsa id_rsa.pub known_hosts
[julius@emperor julius]$ scp .ssh/id_rsa.pub jules@69.57.97.126:.ssh/authorized_keys
jules@69.57.97.126's password:
scp: .ssh/authorized_keys: No such file or directory
Apparently, the ~/.ssh does not yet exist. Julius remedies the problem, and then transfers the file
successfully. Lastly, he sets the permissions on the remote ~/.ssh directory, and its contents, so that
they are only available to the user owner.
[julius@emperor julius]$ ssh jules@69.57.97.126 mkdir .ssh
jules@69.57.97.126's password:
[julius@emperor julius]$ scp .ssh/id_rsa.pub jules@69.57.97.126:.ssh/authorized_keys
jules@69.57.97.126's password:
id_rsa.pub
100% |*****************************|
226
00:00
[julius@emperor julius]$ ssh jules@69.57.97.126 chmod -R go-rwx .ssh
jules@69.57.97.126's password:
With his public key authentication now configured, Julius can easily access his home machine.
[julius@emperor julius]$ ssh jules@69.57.97.126 uptime
11:01:12 up 7:55, 1 user, load average: 0.08, 0.12, 0.09
Performing Remote Backups with SSH
On his home machine, Julius has a directory called archives, which contains important documents. He
would like to make a backup copy of the archive on his work machine. From his work machine, he issues
the following command line.
[julius@emporer julius]$ ssh jules@69.57.97.126 "tar czf - archive" > jules.arch
ive.tgz
Where the filename for the archive to create would normally be specified, Julius places a “-”. This instructs
the tar command to throw the archive to standard out, rather than into a file. Locally, standard out was
redirected into the file jules.archive.tgz. Julius has now created a local archive of the remote file
~/archive, as confirmed with the following command.
[julius@emporer julius]$ tar tvzf jules.archive.tgz
73
Remote Shell Commands
-rw-r--r-- jules/jules
-rw-r--r-- jules/jules
-rw-r--r-- jules/jules
...
112 2003-10-24 15:19:04 archive/named
18 2003-10-26 08:00:11 archive/java
204 2003-10-30 21:24:33 archive/quagga
Online Exercises
Lab Exercise
Objective: Effectively use the Secure Shell Application.
Estimated Time: 30 mins.
Specification
1. Configure your first and third alternate accounts (username_a and username_c, respectively) so
that you can access the accounts from your primary account using ssh, without having to supply a
password. Recall that networking clients can connect to local network services using the hostname
localhost.
2. Using your primary account, create a script called ~/bin/send_to_a, which will copy a regular
file listed as its only argument into the home directory of your first alternate account.
Assuming public key authentication is appropriately configured, the following script would meet the
specifications.
#!/bin/bash
USERA=$(whoami)_a
scp $1 $USERA@localhost:
3. Using your primary account, create a script called ~/bin/backup_c, which creates a gzipped
tar archive of the contents of your third alternate account's home directory into the file /tmp/
backup_c.tgz.
Assuming public key authentication is appropriately configured, the following script would meet the
specifications.
#!/bin/bash
USERC=$(whoami)_c
ssh $USERC@localhost "tar czf - ." > /tmp/backup_c.tgz
Deliverables
1.
1. Appropriate configuration so that your primary account may access your first and third alternate
accounts using ssh without specifying a password.
2. In your primary account, a script called ~/bin/send_to_a which will copy a regular file
specified as its only argument into your first primary accounts home directory.
3. In your primary account, a script called ~/bin/backup_c, which creates a gzipped tar
archive of the contents of your third alternate account's home directory into the file /tmp/
backup_c.tgz.
74
Remote Shell Commands
Questions
1.
What is the name of the Red Hat Enterprise Linux remote shell client which provides encrypted
communications?
a.
ssh
b.
sshd
c.
rsh
d.
rlogin
e.
None of the above
2.
In which file does the Secure Shell store a user's private RSA key, by default?
a.
~/.key.private
b.
~/.ssh/rsa
c.
~/.ssh/id_rsa
d.
~/.sshrc
e.
None of the above
3.
Which of the following command lines could be used to generate a Secure Shell public-private
key pair?
a.
mkssh
b.
ssh-keygen
c.
sshinit -t rsa
d.
rsa.mkkey
e.
None of the above
4.
When using Secure Shell public key authentication, which file must exist on the remote machine?
a.
~/.ssh/id_dsa.pub
b.
~/.ssh/known_hosts
c.
~/.ssh/authorized_keys
d.
A and C
e.
All of the above
Use the following transcript to answer the next 2 questions.
[julius@station julius]$ ssh jules@69.57.97.126 "find /etc/sysconfig -size -100k
| xargs grep GATEWAY > /tmp/gateways"
5.
On the remote machine, which of the following processes was not executed as a result of this
command?
75
Remote Shell Commands
a.
find
b.
xargs
c.
ssh
d.
grep
e.
All of the above processes executed on the remote machine.
6.
On which machine was the file /tmp/gateways created?
a.
The local machine
b.
The remote machine
c.
Not enough information is provided.
7.
Which of the following is not a feature of Secure Shell public key authentication?
a.
Clients can access remote accounts without having to issue a password.
b.
Multiple clients can be configured to access a single account.
c.
No sensitive information is exchanged over the network during authentication negotiation.
d.
The same configuration can be used to authenticate ssh and scp connections.
e.
All of the above are features of Secure Shell public key authentication.
Use the following transcript to answer the next question.
[julius@station julius]$ ssh jules@69.57.97.126 mkdir tmp
[julius@station julius]$ scp .bashrc jules@69.57.97.126:tmp
.bashrc
100% |*****************************|
124
8.
00:00
Which file on the remote machine contains a copy of the local file ~/.bashrc?
a.
~/.bashrc
b.
~/tmp/.bashrc
c.
/tmp/.bashrc
d.
~/tmp
e.
Not enough information is provided.
9.
Which file is automatically updated whenever the first connection to a new host is accepted?
a.
~/.ssh/id_dsa.pub
b.
~/.ssh/known_hosts
c.
~/.ssh/authorized_keys
d.
~/.ssh/authorized_hosts
e.
None of the above
76
Remote Shell Commands
10.
Which of the following command lines could be used to recursively copy the ~/backups
directory to a remote machine's /tmp directory?
a.
scp backups jules@69.57.97.126:tmp
b.
scp -r backups jules@69.57.97.126:/tmp
c.
scp -R /tmp/backups jules@69.57.97.126:
d.
scp backups jules@69.57.97.126:
e.
None of the above
77