Basic System Administration
Your daily commands as root
Becoming root
•
Avoid login as root over network (denied by default)
•
Use "/bin/su -" command from your regular account
- “-” runs user LOGIN scripts
- root can su to any userid without password.
- Note utility for NOLOGIN ID’s.
•
$HOME is sometimes not “/root” but instead “/”; so watch what you delete!
•
Remove the current working directory (".") from your PATH
•
Never execute any regular user's program as root (possible Trojan Horse)
•
Use SSH, not TELNET over network to avoid sniffers
root access - sudo
•
visudo (as root) creates /etc/sudoers file in the following format:
usernames/group servername = (usernames command can be run as) command
•
•
To implement: sudo <command>
Groups are the same as user groups and are differentiated from regular users by a % at
the beginning. The Linux user group "users" would be represented by %users.
You can have multiple usernames per line separated by commas.
Multiple commands also can be separated by commas. Spaces are considered part of the
command.
The keyword ALL can mean all usernames, groups, commands and servers.
If you run out of space on a line, you can end it with a back slash (\) and continue on the
next line.
sudo assumes that the sudoers file will be used network wide, and therefore offers the
option to specify the names of servers which will be using it in the servername position
in Table 9-1. In most cases, the file is used by only one server and the keyword ALL
suffices for the server name.
The NOPASSWD keyword provides access without prompting for your password.
Same concept as the SUID bit in permissions (4000).
•
•
•
•
•
•
•
sudo examples
•
sudo command
… to run the command as USER
Examples:
- user1 localhost=/sbin/halt
… user1 can halt local system
[user1@student1]$ sudo /sbin/halt
password:
[user1@student1]$ System going down now!
- user2 ALL= NOPASSWD: /sbin/halt
- user3 instructor = /usr/sbin/*
… user2 can halt any system w/o password
… user 3 can run any command in /usr/sbin
System Administration tools
•
•
•
•
•
•
•
•
•
•
•
•
man: Sections - 1 commands, 2 system calls, 3 C library routines, 4 devices
and networks, 5 file formats, 6 games and demos, 7 miscellaneous, 8 system
administration
info – textinfo man page
vi editor (front-end to a lot of utilities)
su, sudo
df/du, mount
dump/restore, dd, cpio, tar, rmt, find, rsync
ps, at, batch, crontab, anacron, watch, kill, nice, nohup, killall,
useradd, usermod, userdel. passwd
groupadd, groupmod, groupdel
who, whoami, w, id
syslog
system configuration files – /etc
System information
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
hostname
uname –a
dmesg
who, whoami, w, id
last (reboot)
which, where/whereis, apropos
hwclock
date
ulimit (user limits)
sysctl (system limits/settings)
cgroups
/etc/sysconfig
/etc/security
/proc
ps, pstree,
System monitoring
•
•
•
•
•
•
•
•
•
•
•
•
•
•
sar
pmap
vmstat,
mpstat
iostat
nstat (network),
pidstat
dstat
free
lsof
top, ntop, iftop, latencytop
ulimit –a (view), ulimit –n (set) ulimits –Hn, ulimits -Sn … per user limits,
/etc/security/limits.*
See /etc/security/limits.conf: sysctl –q (view), sysctl –w (set) … system
limits, /etc/sysctl.conf
strace (debugging)
/etc/sysconfig
•
The /etc/sysconfig directory is where many of the files that control the system
configuration are stored for daemon processes or system services like networking.
Contents vary depending on products installed.
/etc/sysconfig is usually “sourced” (. /etc/sysconfig) in SYSV startup scripts.
•
Files in the /etc/sysconfig/ Directory
amd , apmd ‘ authconfig , clock , desktop , devlabel, dhcpd, firstboot, gpm, harddisks,
hwconf, i18n, init, ip6tables-config , iptables-config, irda, keyboard , kudzu, mouse,
named, netdump, network, network-scripts, ifup-xxxx, ntpd, pcmcia, radvd, rawdevices,
selinux, logrotate, samba , sendmail, spamassassin, squid , tux , vncservers, xinetd
/proc
•
/proc is a virtual filesystem. It's sometimes referred to as a process information pseudofile system. It doesn't contain 'real' files but runtime system information (e.g. system
memory, devices mounted, hardware configuration, etc) for all processes started by init
including PID and startup commands. /proc was developed as a LINUX extension to
keep track of all the complex processes started in the system
•
For this reason it can be regarded as a control and information centre for the kernel. In
fact, quite a lot of system utilities are simply calls to files in this directory. For example,
'lsmod' is the same as 'cat /proc/modules' while 'lspci' is a synonym for 'cat /proc/pci'. By
altering files located in this directory you can even read/change kernel parameters
(sysctl) while the system is running.
•
The most distinctive thing about files in this directory is the fact that all of them have a
file size of 0, with the exception of kcore, mtrr and self.
/etc/security
•
Central directory for system defaults
•
The limits.conf file defines process resource limits for users. (see ulimit)
•
opasswd - Store old passwords.
•
access.conf used to allow or restrict access to the system.
•
chroot.conf used to restrict users to there home directories
•
console.apps contains files which are same as service names.
•
console.perms and console.perms.d directory determine the permissions that
will be given to
•
The rest are PAM (Program Authentication Module) related.
sysctl – system limits
•
•
sysctl –q, sysctl –w, sysctl –p file, sysctl -A
/etc/sysctl.conf
For network:
# Enable IP spoofing protection
• net.ipv4.conf.all.rp_filter=1
# Disable IP source routing
• net.ipv4.conf.all.accept_source_route=0
# Ignoring broadcasts request
• net.ipv4.icmp_echo_ignore_broadcasts=1
net.ipv4.icmp_ignore_bogus_error_messages=1
• # Make sure spoofed packets get logged
net.ipv4.conf.all.log_martians = 1
• # disable IPv6
net.ipv6.conf.all.disable_ipv6=1
Kernel isolation (test carefully on test system):
• Turn on execshield
kernel.exec-shield=1
kernel.randomize_va_space=1
ulimit – user limits
•
ulimit - set user limits
•
•
•
•
•
•
•
•
•
•
-c
maximum core file size (in 512-byte blocks)
-d
maximum size of data segment or heap (in kbytes)
-f
maximum file size (in 512-byte blocks)
-n
maximum file descriptor plus 1
-s
maximum size of stack segment (in kbytes)
-t
maximum CPU time (in seconds)
-v
maximum size of virtual memory (in kbytes)
-S
soft limit
-H
hard limit
/etc/security/limits.conf
c(ontrol)groups
•
Cgroups allow you to allocate resources—such as CPU time, system memory, network bandwidth, or
combinations of these resources—among user-defined groups of tasks (processes) running on a
system.
•
A *cgroup* associates a set of tasks with a set of parameters for one or more subsystems. A
*subsystem* is a module that makes use of the task grouping facilities provided by cgroups to treat
groups of tasks in particular ways. A subsystem is typically a "resource controller“ in a hierarchy of
processes.
•
A cgroup is mounted as a virtual filesystem and can be modified to re-alllocate kernel resources.
Each cgroup is represented by a directory in the cgroup file system containing the following files
describing that cgroup:
- tasks: list of tasks (by pid) attached to that cgroup
- releasable flag: cgroup currently removeable?
- notify_on_release flag: run the release agent on exit?
- release_agent: the path to use for release notifications (this file exists in the top cgroup only) Other
subsystems such as cpusets may add additional files in each cgroup dir.
PAM
•
•
•
•
Pluggable Authentication Module
Centralized authentication mechanism
“Plug in” different authentication methods
Different services can have different
authentication policies
• Highly secure systems can require multiple
passwords to authenticate
PAM Framework
•
ftp
login
ssh
PAM Library
libpam
conf
Applications
Modules
pam_unix
pam_ldap
pam_securetty
PAM Stack
• Modules are stacked (order is important)
• Sample PAM configuration in /etc/pam.d:
interface
auth
auth
auth
auth
control flag
required
required
sufficient
required
module name
pam_nologin.so
pam_securetty.so
pam_unix.so
pam_ldap.so
Security Enhanced LINUX
•
•
•
•
•
•
•
•
•
•
Kernel level security included since 2.60 kernel
Not an application interface. But sends access return codes to applications.
Can be combined with ACLs
/etc/selinux directory
Old gui system-config-selinux, new gui: policycureutils-gui
BE CAREFUL with changes. Especially deleting files.
See “enforcing=0” or “selinux=0” on Grub edit menu
Protects, files, processes, applications
Based on security “context”
An SE Linux security context is comprised of three parts: an "identity", a
"role", and a "type" for users and files or "domain" for processes. Default
context for root, the role is sysadm_r, and the domain is sysadm_t.
Configuration directory: /etc/selinux directory
SELINUX Modes/Types
• Enforcing: enable and enforce the SELinux
security policy on the system, denying access
and logging actions in /var/log/audit/audit.log
• Permissive: enabled but will not enforce the
security policy, only warn and log actions.
Used for troubleshooting SELinux issues
• Disabled: SELinux is turned off
• Targeted: Specific processes
• MLS/STRICT: VERY secure - systemwide
SELINUX Policy
• Policy: a set of rules the SELinux security engine that
defines types for file objects and domains for processes,
and user defined (identities) roles to limit the domains that
can be entered.
• Strict - minimum access
• Targeted – specific processes
• Unconfined – not under SELINUX control, outside
SELINUX context
SELINUX Access Control
• Type Enforcement (TE): Type Enforcement is the
primary mechanism of access control used in
the targeted policy
• Role-Based Access Control (RBAC): Based around
SELinux users (not necessarily the same as the Linux
user), but not used in the default targeted policy
• Multi-Level Security (MLS): Not commonly used and
often hidden in the default targeted policy.
• Shown with –Z option (ls –Z, ps –Z etc)
SELINUX Commands
•
•
•
•
•
•
•
sestatus: show SELINUX status
getenforce: show SELINIX status
setenforce: set SELINUX status
semanage: command line policy management
chcon: change SELINUX context
restorecon: restore default SELINIX context
audit2allow: Generate SELINUX policy from
/var/log/audit/audit.log
• sealert: troubleshooting tool
• ls –Z, ps –Z: show SELINUX context for files, processes
• id: show the current user id context.
Process info: ps -aux(BSD)
• Common options:
-a print all processes involving terminals
-e print environment and arguments
-l long listing
-u print user information
-xi nclude processes with no terminals
• Meaning of user information columns:
%CPU percentage use of CPU
SZ total size (in 1024 byte pages) of the process
RSS total resident size (in pages) of the process
STAT state of the process
TIME time, including both user and system time
Process info: ps –ef (System V)
•
Common options:
-e print all processes
-f print full listing
-l long listing (more info than -f)
•
pstree
•
Meaning of full listing columns:
S state
PRI priority
SZ total size (in 4096 byte pages) of the process
RSS total resident size (in pages) of the process
TIME starting time
TIME cumulative execution time
Process Management
•
at – schedule onetime batch job (scripts or commands).
Example: at now +1 minutes –f somecommandfile.txt
“batch” interactive at command on some systems.
atq, atrm etc
•
anacron – (Linux) workstation scheduler. See /etc/anacrontab
•
watch - execute a program periodically, display results fullscreen
•
cron – scheduler routine must be a started process. See /etc/crontab/, /etc/cron.d,
/etc/cron.hourly, /etc/cron.daily, /etc/cron.weekly
- crontab [-e] [-l] [-r] [–u user] [filename]. Creates cron table “-e”uses vi syntax.
- Each line contains:
mm(0-59) hh(0-23) dd(1-31) mm(1-12) day (0-6, 0=Sunday**) command
* - Is treated as a wild card. Meaning any possible value.
*/5 - Is treated as ever 5 minutes, hours, days, or months.
2,4,6 - Treated as an OR, if placed in the hours, this could mean at 2, 4, or 6 o-clock.
9-17 - Treats for any value between 9 and 17. If placed in day of month, days 9 -17.
** Note 1-7 on some UNIX systems
Example */5 * * * * echo “hi there” > /dev/tty2 2>&1
Process Management
•
kill pid– stop a process. -9 kills absolutely. killall
•
nice pid or command – set process priority
•
nohup command – run a process after logging off. Nohup.out contains job
output.
•
“sighup” option on some commands for forcing process to reinitialize.
•
“zombie” processes – “kill”ed or abended processes with no parent.
Usually requires a reboot to reclaim resources. Can cause system instability.
See also: Ctrl Z, Ctrl C, fg, bg, &
Runlevels
Run Level
Generic
Fedora Core
Slackware
Debian
0
Halt
Halt
Halt
Halt
1
Single-user mode
Single-user mode
Single-user mode
Single-user mode
2
Basic multi-user
mode (without
networking)
User definable
(Unused)
User definable configured the same
as runlevel 3
Multi-user mode
3
Full (text based)
multi-user mode
Multi-user mode
Multi-user mode default Slackware
runlevel
4
Not used
Not used
X11 with
KDM/GDM/XDM
(session managers)
Multi-user mode
5
Full (GUI based)
multi-user mode
Full multi-user mode
(with an X-based
login screen) default runlevel
User definable configured the same
as runlevel 3
Multi-user mode
6
Reboot
Reboot
Reboot
Reboot
service/chkconfig
• See systemd (RHEL/Fedora/Centos)
• See upstart (Debian/Ubuntu)
• service <service name> start/stop/restart/status
•
• chkconfig --list [name]
chkconfig --add name
chkconfig --del name
chkconfig [--level levels] name <on|off|reset>
chkconfig [--level levels] name
Backup/Restore
•
dump/restore - backs up file systems, has interactive mode, can do
incremental backups, maintains "sparse files", is most commonly used utility
•
cpio - can back up individual files/directories, handles special files, packs data
tighter than tar, skips bad spots on media on restore, use with find (some
versions of find have -cpio option for this purpose)
•
tar - backs up directory trees, does not back up special files, poor error
handling with media errors, does not pack blocks (GNU tar solves some of
these problems). Some LINUX/UNIX systems have built-in compress with –z
flag.
•
dd - copies/converts files, can go from one medium to another, processes
whole entity or select blocks, can swap bytes and do ASCII/EBCDIC
conversions. Performs physical backup of raw devices.
•
rmt - used for remote tape operations. Varies by OS.
•
rsync – used for directory synchronization, e.g. “hot” folders
Backup Strategy
•
Physical (dd, cpio) – usually devices (as root)
dd if=devicefile if=outputfile bs=blocksize count=#blocks
•
Logical (rmt, tar, dump/restore). Backup marker.
create: tar –cvf tarfilename.tar [directory list] | compress
list: tar –tvf tarfilename.tar
extract: tar –xvf tarfilename.tar
•
Can use logical backups in conjunction with find command –exec option (next panel)
for differential or incremental backups
•
Backup types: Full (everything)
Incremental (Difference since last backup)
Differential (Difference since last full backup)
Full + Incremental or Differential = Backup set
find
•
Syntax: find starting-dir(s) matching-criteria-and-actions Matching criteria
-atime n file was accessed n days ago
-mtime n file was modified n days ago
-size n file is exactly n 512-byte blocks
-type c file type (e.g., f=plain, d=dir)
-name nam file name (e.g., `*.c')
-user usr file's owner is usr
-perm p file's access mode is p
-print display pathname
-exec cmd execute command ({} expands to file)
•
find examples
find . -name \*.c -print
find / -size +1000 -mtime +30 \ -exec ls -l {} \;
find / \( -name a.out -o -name core \ -o -name' \) -type f -atime +14 \ -exec rm -f {} \;
find / \( -perm 2000 -o -perm 4000 \) \ -print | diff - files.secure
Disk management
•
df
•
mount / umount
•
du | sort –rn | more
•
find / -name core -exec rm -f {} \;
•
Filesystems: /home, /var, /tmp (noexec), / (never full!)
•
mkdev, mkfs, fdisk
User management
•
Set system account parameters (e.g., password aging, account expiration,
quotas, login scripts - /etc/profile, /etc/bashrc etc)
•
Determine login name, user ID (UID), group ID (GID)
•
Assign password (passwd)
•
/etc/passwd - logname:passwd:uid:gid:user info:home:shell
•
Passwords stored in /etc/shadow (pwconv)
•
Commands: useradd, usermod, userdel, chage, passwd
•
/etc/group: group:passwd:gid:members
•
Commands: groupadd, groupmod, groupdel
User Security
•
ALWAYS use /etc/shadow (pwconv command)
•
Password aging:
get: chage -l userid
set: chage -M 60 -m 7 -W 7 userid
•
Lock/unlock and account
passwd –l userid
passwd –u userid
•
Limit password reuse:
vi /etc/pam.d/system-auth (RHEL/Fedora)
vi /etc/pam.d/common-password (Ubuntu)
Add: password sufficient pam_unix.so use_authtok md5 shadow remember=10
•
Verify root ID’s: awk -F: '($3 == "0") {print}' /etc/passwd
•
Verify no password ID’s: awk -F: '($2 == "") {print $1 }' /etc/shadow | grep /etc/passwd
Make sure they are /bin/nologin
•
No ROOT LOGIN, use su or sudo.
•
Disable ROOT login under FTP., SSH, GUI (procedures vary)
•
Configure password policy (LINUX: see pam_cracklib.so, other vary).
•
See defaults in /etc/login.defs for /etc/shadow values.
SYSLOG
• (r)syslog is a utility for tracking and logging all manner of system
messages from the merely informational to the extremely critical.
Sysogs stored in
• In LINUX, system logs are stored in /var/log. System messages are
recorded in /var/log/messages. Other OS’es may use different files in
different directories (e.g. /var/adm).
• Each system message sent to the syslog server has two descriptive
labels associated with it that makes the message easier to handle.
- The first describes the function (facility) of the application that
generated it. For example, applications such as mail and cron generate
messages with easily identifiable facilities named mail and cron.
- The second describes the degree of severity of the message.
SYSLOG
•
Severity:
•
•
•
•
•
•
•
•
0 emergencies System unusable
1 alerts
Immediate action required
2 critical
Critical condition
3 errors
Error conditions
4 warnings
Warning conditions
5 notifications Normal but significant conditions
6 informational Informational messages
7 debugging
Debugging messages
Level: Keyword: Description
SYSLOG
•
configuration file /etc/rsyslog.conf or/etc/syslog.conf.
•
File consists of two columns.
- First lists the facilities and severities of messages to expect
- Second lists the files to which they should be logged.
- LINUX default directory is /var/log
•
Example:
*.info;mail.none;authpriv.none;cron.none /var/log/messages
•
Note other services may record messages in other files (e.g. sendmail)
•
Syslog is also a network service. A common implementation is to forward
system info to a common syslog server. TCP or UDP can be used.
•
Logs compressed, ,stored and optionally e-mailed by the logrotate function.
Definitions stored in /etc/logrotate.conf and /etc/logrotate.d
System shutdown
•
Shutdown will run SysV K* scripts.
•
shutdown {–h|-r} {time in minutes|now}
•
See also wall command
•
Other commands: halt, reboot, Ctrl-Alt-Del may bypass some processing. Not
recommended for production systems.
•
Reboots recorded in /var/log/wtmp or utmp
•
last (reboot) displays info