84-01-19
DATA SECURITY MANAGEMENT
INTRODUCTION TO UNIX
SECURITY FOR SECURITY
PRACTITIONERS
Jeffery J. Lowder
INSIDE
Operating System Security Services; Identification and Authentication; Access Control;
Availability and Integrity; Audit; Security Facilities for Users; Vulnerabilities in Traditional UNIX;
TCP Wrappers; Login or Warning Banners
In an age of increasingly sophisticated security tools (e.g., firewalls, virtual private networks, intrusion detection systems, etc.), many people do
not consider operating system security a very sexy topic. Indeed, given
that the UNIX operating system was originally developed in 1969 and
that multiple full-length books have been written on protecting UNIX
machines, one might be tempted to dismiss the entire topic as “old hat.”
Nevertheless, operating system security is a crucial component of an
overall security program. In the words of Anup Ghosh, the operating system is “the foundation for any software that runs on a machine,” and this
is just as true in the era of E-commerce as it was in the past. Thus, security practitioners who are even indirectly responsible for the protection
of UNIX machines need to have at least a basic understanding of UNIX
security. This article attempts to address that need by providing an overview of security services common to all flavors of UNIX; security mechanisms available in trusted UNIX are beyond the scope of this article (but
see Exhibit 1).
OPERATING SYSTEM SECURITY SERVICES
Summers1 (1997) lists the following
security services that operating systems in general can provide:
1. Identification and authentication: A secure operating system
PAYOFF
IDEA
Operating system security is a crucial component
of an overall security program. Thus, security
practitioners who are even indirectly responsible
for the protection of UNIX machines need to have
at least a basic understanding of UNIX security.
This article addresses that need by providing an
overview of security services common to all flavors of UNIX.
Auerbach Publications
© 2000 CRC Press LLC
EXHIBIT 1 — Versions of Trusted or Secure UNIX
A1 (Verified Design)
B3 (Security Domains)
B2 (Structured Protection)
B1 (Labeled Security Protection)
C2 (Controlled Access Protection)
C1 (Discretionary Access Protection)
D1 (Minimal Protection)
No operating systems have been
evaluated in class A1
Wang Government Services, Inc., XTS-300
STOP 4.4.2
Trusted Information Systems, Inc. Trusted
XENIX 4.0
Digital Equipment Corporation ULTRIX
MLS+ Version 2.1 on VAX Station 3100
Hewlett Packard Corporation HP-UX BLS
Release 9.09+
Silicon Graphics Inc. Trusted IRIX/B
Release 4.0.5EPL
No UNIX operating systems have been
evaluated in class C2
Products are no longer evaluated at this
class
No operating systems have been
evaluated in class D1
Note: Various versions of UNIX have been evaluated by the U.S. Government’s National
Security Agency (NSA) according to the Trusted Computer System Evaluation
Criteria. (By way of comparison, Microsoft Corporation’s Windows NT Workstation and Windows NT Server, version 4.0, have both been evaluated at class
C2.) The above chart is taken from the NSA’s Evaluated Product List.
must be able to distinguish between different users (identification); it
also needs some assurance that users are who they say they are (authentication). Identification and authentication are crucial to the other operating system security services. There are typically three ways
to authenticate users: something the user knows (e.g., a password),
something the user has (e.g., a smart card), or something the user is
(e.g., a retinal pattern). Passwords are by far the most common authentication method; this method is also extremely vulnerable to
compromise. Passwords can be null, easily guessed, cracked, written
down and then discovered, or “sniffed.”
2. Access control: An operating system is responsible for providing logical access control through the use of subjects, objects, access rights,
and access validation. A subject includes a userID, password, group
memberships, privileges, etc. for each user. Object security information includes the owner, group, access restrictions, etc. Basic access
rights include read, write, and execute. Finally, an operating system
evaluates an access request (consisting of a subject, an object, and
the requested access) according to access validation rules.
3. Availability and integrity: Does the system start up in a secure fashion? Does the system behave according to expectations during an attack? Is the data on the system internally consistent? Does the data
correspond with the real-world entities that it represents?
Auerbach Publications
© 2000 CRC Press LLC
4. Audit: An audit trail contains a chronological record of events. Audit
trails can be useful as a deterrent; they are even more useful in investigating incidents (e.g., Who did it? How?). Audit trails have even
been used as legal evidence in criminal trials. However, for an audit
trail to be useful in any of these contexts, the operating system must
record all security-relevant events, protect the confidentiality and integrity of the audit trail, and ensure that the data is available in a
timely manner.
5. Security facilities for users: Non-privileged users need some method
for granting rights to their files and changing their passwords. Privileged users need additional facilities, including the ability to lock accounts, gain access to other users’ files, configure auditing options,
change ownership of files, change users’ memberships in groups, etc.
The remainder of this article explores how these services are implemented in the UNIX family of operating systems.
IDENTIFICATION AND AUTHENTICATION
UNIX identifies users according to usernames and authenticates them
with passwords. In many implementations of UNIX, both usernames and
passwords are limited to eight characters. As a security measure, UNIX
does not store passwords in plaintext. Instead, it stores the password as
ciphertext, using a modified Digital Encryption Standard (DES) algorithm
(crypt) for encryption. The encrypted password, along with other pertinent account information (see Exhibit 2), is stored in the /etc/passwd file
according to the following format:
username:encrypted password:UserID:GroupID:user’s full name:home
directory:login shell
Unfortunately, the /etc/passwd file is world-readable, which can place
standard, “out-of-the-box” configurations of UNIX at risk for a brute-force
password-guessing attack by anyone with system access. Given enough
computing resources and readily available tools like Alec Muffet’s crack
EXHIBIT 2 — Sample /etc/passwd Entries
keith::1001:15:Keith Smith:/usr/keith:/bin/csh
greg:Qf@14pL1aqzqB:Greg Jones:/usr/greg/:/bin/csh
cathy:*:1003:15:Cathy Jones:/usr/cathy:/bin/csh
Note: In this example, user keith has no password, user greg has an encrypted
password, and user cathy has a shadowed password.
Auerbach Publications
© 2000 CRC Press LLC
utility, an attacker can eventually guess every password on the system. In
light of this vulnerability, all current implementations of UNIX now provide support for so-called “shadow” passwords. The basic idea is to store
the encrypted passwords in a separate file (/etc/shadow to be exact) that
is only readable by the privileged “root” account. Also, although vanilla
UNIX does not provide support for proactive password checking, add-on
tools are available. Finally, password aging is not part of standard UNIX
but is supported by many proprietary implementations.
UserIDs (UIDs) are typically 16-bit integers, meaning that they can
have any value between 0 and 65,535. The operating system uses UIDs,
not usernames, to track users. Thus, it is entirely possible in UNIX for two
or more usernames to share the same UID. In general, it is a bad idea to
give two usernames the same UID. Also, certain UIDs are reserved. (For
example, any username with an UID of zero is considered root by the
operating system.) Finally, UNIX requires that certain programs like
/bin/passwd (used by users to change their passwords) and /bin/login
(executed when a user initiates a login sequence) run as root; however,
users should not be able to arbitrarily gain root permissions on the system. UNIX solves this problem by allowing certain programs to run under the permissions of another UID. Such programs are called Set UserID
(SUID) programs. Of course, such programs can also be risky: if attackers
are able to interrupt an SUID program, they may be able to gain root access and ensure that they are able to regain such access in the future.
Group IDs (GIDs) are also typically 16-bit integers. The GID listed in
a user’s entry in /etc/passwd is that user’s primary GID; however, in some
versions of UNIX, a user can belong to more than one group. A complete
listing of all groups, including name, GID, and members (users), can be
found in the file /etc/group.
Once a user successfully logs in, UNIX executes the global file
/etc/profile along with the .profile file in the user’s home directory using
the user’s shell specified in /etc/passwd. If the permissions on these files
are not restricted properly, an attacker could modify these files and cause
unauthorized commands to be executed each time the user logs in. UNIX
also updates the file /usr/adm/lastlog, which stores the date and time of
the latest login for each account. This information can be obtained via
the finger command and creates another vulnerability: systems with the
finger command enabled may unwittingly provide attackers with useful
information in planning an attack.
ACCESS CONTROL
Standard UNIX systems prevent the unauthorized use of system resources (e.g., files, memory, devices, etc.) by promoting discretionary access
control. Permissions are divided into three categories: owner, group, and
other. However, privileged accounts can bypass this access control. UNIX
Auerbach Publications
© 2000 CRC Press LLC
treats all system resources consistently by making no distinction between
files, memory, and devices; all resources are treated as files for access
control purposes.
The UNIX filesystem has a tree structure, with the top-level directory
designated as /. Some of the second-level directories are standards. For
example, /bin contains system executables, /dev contains devices, /usr
contains user files, etc. Each directory contains a pointer to itself (the ‘.’
file) and a pointer to its parent directory (the ‘..’ file). (In the top-level directory, the ‘..’ file points to the top-level directory.) Every file (and directory) has an owner, a group, and a set of permissions. This information
can be obtained using the ls -l command:
drwxr-xr-x
drwxr-xr-x
-rw-------rw-rw----rw-rw-rw-rws------
1
2
3
4
5
6
jlowder
jlowder
jlowder
jlowder
jlowder
jlowder
staff 1024
staff 1024
staff 2048
staff 2048
staff 2048
staff 18495
Feb 21
Oct 28
Feb 21
Feb 21
Feb 21
Feb 21
18:30
1996
18:31
18:31
18:31
18:31
./
../
file1
file2
file3
file4
In the above example, file1 is readable and writable only by the owner; file2 is readable and writable by both the owner and members of the
‘staff’ group; file3 is readable and writable by everyone; and file4 is readable and writable by the owner and is a SetUID program.
Devices are displayed a bit differently. The following is the output of
the command ls -l /dev/cdrom /dev/tty02:
br-------crw-------
1 root
2 root
root
root
1024
1024
Oct 28
Oct 28
1996
1996
/dev/cdrom
/dev/tty02
UNIX identifies block devices (e.g., disks) with the letter ‘b’ and character
devices (e.g., modems, printers) with the letter ‘c.’
When a user or process creates a new file, the file is given default permissions. For a process-created file (e.g., a file created by a text editor),
the process specifies the default permissions. For user-created files, the
default permissions are specified in the startup file for the user’s shell
program. File owners can change the permissions (or mode) of a file by
using the chmod (change mode) command.
UNIX operating systems treat directories as files, but as a special type
of file. Directory “files” have a specified structure, consisting of filenameinode number pairs. Inode numbers refer to a given inode, a sort of
record containing information about where parts of the file are stored,
file permissions, ownership, group, etc. The important thing to note
about the filename-inode number pairs is that inode numbers need not be
unique. Multiple filenames can (and often do) refer to the same inode
number. This is significant from a security perspective, because the rm
Auerbach Publications
© 2000 CRC Press LLC
command only removes the directory entry for a file, not the file itself.
Thus, to remove a file, one must remove all of the links to that file.
AVAILABILITY AND INTEGRITY
One aspect of availability is whether a system restarts securely after failure. Traditional UNIX systems boot in single-user mode, usually as root.
And, unfortunately, single-user mode allows literally anyone sitting at the
system console to execute privileged commands. Thus, single-user mode
represents a security vulnerability in traditional UNIX. Depending on the
flavor of UNIX, the security administrator has one or two options for
closing this hole. First, if the operating system supports it, the security
practitioner should configure the system to require a password before
booting in single-user mode. Second, tight physical controls should be
implemented to prevent physical access to the system console.
System restarts are also relevant to system integrity. After an improper
shutdown or system crash, the UNIX fsck command will check filesystems for inconsistencies and repair them (either automatically or with administrator interaction). Using the fsck command, an administrator can
detect unreferenced inodes, used disk blocks listed as free blocks, etc.
Although there are many ways to supplement UNIX filesystem integrity, one method has become so popular that it deserves to be mentioned
here. Developed by Gene Kim and Gene Spafford of Purdue University,
Tripwire is an add-on utility that provides additional filesystem integrity
by creating a signature or message digest for each file to be monitored.
Tripwire allows administrators to specify what files or directories to monitor, which attributes of an object to monitor, and which message digest
algorithm (e.g., MD5, SHA, etc.) to use in generating signatures. When
executed, Tripwire reports on changed, added, or deleted files. Thus, not
only can Tripwire detect Trojan horses, but it can also detect changes that
violate organizational policy.
AUDIT
Different flavors of UNIX use different directories to hold their log files
(e.g., /usr/adm, /var/adm, or /var/log). But wherever the directory is located, traditional UNIX records security-relevant events in the following
log files:
• lastlog: records the last time a user logged in
• utmp: records accounting information used by the who command
• wtmp: records every time a user logs in or out; this information can
be retrieved using the last command
• acct: records all executed commands; this information can be obtained using the lastcomm command.
Auerbach Publications
© 2000 CRC Press LLC
Unfortunately, there is no way to select events or users to record; thus,
this log can consume an enormous amount of disk space if implemented.
Furthermore, most versions of UNIX support the following logfiles:
• sulog: logs all su attempts, and indicates whether they were successful
• messages: records a copy of all the messages sent to the console and
other syslog messages
Additionally, most versions of UNIX provide a generic logging utility
called syslog. Originally designed for the sendmail program, syslog accepts messages from literally any program. (This also creates an interesting audit vulnerability: any user can create false log entries.) Messages
consist of the program name, facility, priority, and the log message itself;
the system prepends each message with the system date, time, and
name. For example:
Nov 7
Nov 7
Nov 7
04:02:00 alvin syslogd: restart
04:10:15 alvin login: ROOT LOGIN REFUSED on ttya
04:10:21 alvin login: ROOT LOGIN on console
The syslog facility is highly configurable; administrators specify in
/etc/syslog.conf what to log and how to log it. syslog recognizes multiple
security states or priorities, including emerg (emergency), alert (immediate action required), crit (critical condition), err (ordinary error), warning,
notice, info, and debug. Furthermore, syslog allows messages to be
stored in (or sent to) multiple locations, including files, devices (e.g.,
console, printer, etc.), and even other machines. These last two options
make it much more difficult for intruders to hide their tracks. (Of course,
if intruders have superuser privileges, they can change the logging configuration or even stop logging altogether.)
SECURITY FACILITIES FOR USERS
Traditional UNIX supports one privileged administrative role (the “root”
account). The root account can create, modify, suspend, and delete user
accounts; configure auditing options; administer group memberships;
add or remove filesystems; execute any program on the system; shut the
system down; etc. In short, root accounts have all possible privileges.
This violates both the principle of separation of duties (by not having a
separate role for operators, security administrators, etc.) and the principle
of complete mediation (by exempting root from access control).
Non-privileged users can change their passwords using the passwd
command, and they can modify the permissions of their files and directories using the chmod program.
Auerbach Publications
© 2000 CRC Press LLC
MISCELLANEOUS TOPICS
Finally, there are a few miscellaneous topics that pertain to UNIX security
but do not neatly fall into one of the categories of operating system security listed at the beginning of this article. These miscellaneous topics
include tcpwrapper and fundamental operating system holes.
Vulnerabilities in Traditional UNIX
Many (but by no means all) of UNIX’s security vulnerabilities result from
flaws in its original design. Consider the following examples:
1. Insecure defaults. Traditional UNIX was designed for developers; it is
shipped with insecure defaults. Out-of-the-box UNIX configurations
include enabled default accounts with known default passwords.
Traditional UNIX also ships with several services open by default,
password shadowing not enabled, etc. Administrators should immediately disable unnecessary accounts and ports. If a default account
is necessary, the administrator should change the password.
2. Superuser and SUID attacks. Given that UNIX does not have different
privileged roles, anyone who compromises the root account has
compromised the entire system. When combined with SUID programs, the combination can be disastrous. An attacker need simply
“trick” the SUID program into executing an attack, either by modifying the SUID program or by supplying bogus inputs. If the SUID program runs as root, then the attack is likewise executed as root. Given
this vulnerability, SUID programs should be prohibited if at all feasible; if not, the system administrator must continually monitor SUID
programs to ensure they have not been tampered with.
3. PATH and Trojan horse attacks. When a user requests a file, the
PATH environment variable specifies the directories that will be
searched and the order in which they will be searched. By positioning a Trojan horse version of a command in a directory listed in the
search path, such that the Trojan horse directory appears prior to the
real program’s directory, an attacker could get a user to execute the
Trojan horse. Therefore, to avoid this vulnerability in the PATH variable, administrators can specify absolute filepaths and place the user’s home directory last.
4. Trust relationships. UNIX allows both administrators and users to
specify trusted hosts. Administrators can specify trusted hosts in the
/etc/hosts.equiv file and users in a file named .rhosts in their home directory. When a trust relationship exists, a user on a trusted (remote)
machine can log into the local machine without entering a password.
Furthermore, when the trust relationship is defined by an administrator in the /etc/hosts.equiv file, the remote user can log into the local
machine as any user on the local system, again without entering a
Auerbach Publications
© 2000 CRC Press LLC
password. Clearly, this is extremely risky. Even if one trusts the users
on the remote machine, there are still two significant risks. First, the
trust relationships are transitive. If one trusts person A, then one implicitly trusts everyone who person A trusts. Second, if the remote
machine is compromised, the local machine is at risk. For these reasons, trust relationships are extremely risky and should almost always be avoided.
TCP Wrapper
Written by Wietse Venema, tcpwrapper allows one to filter, monitor, and
log incoming requests for various Internet services (systat, finger, ftp, telnet, rlogin, rsh, exec, tftp, talk, etc.). The utility is highly transparent; it
does not require any changes to existing software. The chief advantage
of tcpwrapper is that it provides a decent access control mechanism for
network services. For example, an administrator might want to allow incoming FTP connections, but only from a specific network. tcpwrapper
provides a convenient, consistent method for implementing this type of
access control. Depending on the implementation of UNIX, tcpwrapper
might also provide superior audit trails for the services it supports.
Login or Warning Banner
UNIX can be configured to display a “message of the day,” specified in
the file /etc/motd, to all users upon login. At least part of this message
should be a so-called login or warning banner, advising would-be attackers that access to system resources constitutes consent to monitoring and
that unauthorized use could lead to criminal prosecution (see Exhibit 3).
CONCLUSION
Traditional UNIX implements some of the components of operating systems security to varying extents. It has many well-known vulnerabilities;
out-of-the-box configurations should not be trusted. Furthermore, addon security tools can supplement core UNIX services. With proper configuration, a UNIX system can be reasonably protected from would-be intruders or attackers.
EXHIBIT 3 — Sample Warning Banner
WARNING: THIS SYSTEM FOR AUTHORIZED USE ONLY. USE OF THIS
SYSTEM CONSTITUTES CONSENT TO MONITORING; UNAUTHORIZED USE
COULD RESULT IN CRIMINAL PROSECUTION. IF YOU DO NOT AGREE
TO THESE CONDITIONS, DO NOT LOG IN!
Auerbach Publications
© 2000 CRC Press LLC
Notes
1. Summers, Rita C., Secure Computing: Threats and Safeguards, McGraw-Hill, New York, 1997.
Jeffrey J. Lowder is chief of the network security element at the U.S. Air Force Academy, Colorado Springs, Colorado. He is a contributing author of Information Security Management Handbook, 4th ed., Vol. 2, chap. 21, Tipton,
H.F. and Krause, M., Eds., Auerbach Publications, Boca Raton, FL, 2001, from which this article is extracted.
Auerbach Publications
© 2000 CRC Press LLC