Unix System Administration Chuck Hauser 2007-10-19 Cfengine Automated suite of programs for configuring and maintaining Unix-like computers Developed by Mark Burgess of Oslo University College Started in 1993; wanted to replace shell scripts with a declarative language that documented configurations. Some Cfengine Capabilities Check or set file ownership and permissions Edit configuration files Remove unwanted files (“tidy”) Check integrity of important files Process management Implementing Cfengine Primary Documentation: www.cfengine.org Secondary Documentation: Wikipedia lists several Cfengine links Software: required packages are at www.sunfreeware.com Additional Useful Info Luke A. Kanies: Introducing Cfengine http://www.onlamp.com/pub/a/onlamp/ 2004/04/15/cfengine.html Chapter 6 from Kirk Bauer: Automating UNIX and Linux Administration http://www.apress.com/book/download file/1169 AEleen Frisch: Top Five Open Source Packages for System Administrators http://www.onlamp.com/pub/a/onlamp/ 2003/05/29/essentialsysadmin.html System configuration with CFEngine http://sial.org/howto/cfengine/ Mark Burgess and AEleen Frisch: A System Engineer’s Guide to Host Configuration and Maintenance Using Cfengine http://www.sage.org Cfengine Software Packages Follow the required packages list on Sunfreeware.com. File names consist of package-version-os_versionarchitecture-default_directory; e.g. cfengine-2.1.20-sol9sparc-local.gz. Cfengine GNU cfengine program suite libgcc GCC libraries db Berkley embedded database openssl SSL/TSL cryptography library Installing Packages Put packages in /var/spool/pkg. Install in this order: libgcc, db, openssl, cfengine. Unzip each package: # gunzip libgcc-3.3-sol9-sparc-local.gz Then install as root: # pkgadd –d libgcc-3.3-sol9-sparc-local Cfengine Directory Structure After package installation, libraries are in /usr/local/lib and binaries are in /usr/local/sbin. Cfengine’s production location is /var/cfengine: /var/cfengine/bin stores programs, ./inputs stores configuration files, and ./outputs stores output from cfagent runs in timestamped files. Additional /var/cfengine directories are created as needed by the Cfengine programs. Setup Script for Cfengine #!/usr/bin/ksh # if [ ! -f /usr/local/sbin/cfagent ] then echo "Quitting, no cfengine programs on this machine!" exit fi if [ ! -d /var/cfengine/bin ] then mkdir -p /var/cfengine/bin chown root:other /var/cfengine/bin fi Setup Script continued cd /usr/local/sbin cp cfagent cfenvgraph cfrun cfdoc cfexecd \ cfservd cfenvd cfkey cfshow vicf \ /var/cfengine/bin if [ ! -d /var/cfengine/inputs ] then mkdir /var/cfengine/inputs fi if [ ! -d /var/cfengine/outputs ] then mkdir /var/cfengine/outputs fi Some Cfengine Programs The configuration agent; implements a cfagent machine’s configuration Generates public/private key pairs; cfkey usually run only once. Execute and reporting daemon (for cfexecd cfagent). For file transfer and remote execution cfservd (runs on a central configuration server) Run from server; contacts cfservd to run cfrun cfagent (rarely used) How It Works A configuration file describes the state a system should be in Descriptive, not procedural; does not describe explicitly how to achieve that state A single cfengine run may result in multiple passes (“convergence”) Single host setup only requires the cfagent program and a cfagent.conf file that describes the desired configuration The Configuration File A configuration file consists of actions and classes (also called groups) Actions either tell the program how to behave or what to do. Actions are often followed by statements in this form: name = ( list ) Classes may be used to restrict a particular action to a host that is only a member of that class (group) May have variables: these may be special internal variables, user-defined strings, or shell environment variables Configuration File Structure File consists of action sections, which are reserved words followed by a colon Some sections are for initial settings and definitions: acl, alerts, binservers, broadcast, control, defaultroute, filters, groups, homeservers, ignore, import, strategies, etc. Other sections perform specific actions: alerts, copy, disks, disable, editfiles, files, links, netconfig, resolve, packages, processes, shellcommands, and tidy It is not necessary to have or use all sections A cfagent.conf Section links: Actions end with a colon and start a section easyspooler_fix:: Classes within an action end with a double-colon /usr/bin/lp -> /usr/bin/llp syslog=true inform=true A Very Simple Configuration File # cfagent.hello control: actionsequence = ( shellcommands ) shellcommands: “/bin/echo Hello world!” useshell=false To execute: # /var/cfengine/bin/cfagent –f cfagent.hello cfengine:cis:/bin/echo Hello: Hello world! Action Sequence Types 1 alerts Display messages Copy files to or update files on the local copy system; source files can be local or remote Verify presence of or free space on disk disks partitions Deactivate system features by renaming disable configuration files; can also perform log rotation Action Sequence Types 2 editfiles Modify test in configuration files files Verify or correct file attributes links Verify/create/correct symbolic links netconfig Configure the network interfaces resolve Specify name servers etc. in resolv.conf Action Sequence Types 3 packages Verify presence of or install packages processes Monitor and manage processes Execute external shell shellcommands commands Delete unwanted files and tidy directories Classes (Groups) Classes may be predefined (also referred to as “fixed” or “hard” classes) or defined in the configuration file Custom classes are usually defined in the groups section “Feedback classes:” a class may also be defined using the define statement when actions are performed in other sections (for example, when disable actions are performed): define=boot_server_disabled Predefined Classes Operating systems: sunos_5_8, sunos_5_9 Architecture or hardware: sparc, SUNW_Sun_Fire_480R Host name or IP address: cis, 10_1_12_23 Date and time stuff: Yr2007, March, Day12, Monday, Hr00, Min45 Time intervals in minutes or quarter hours: Min00-05, Min05-10, Q1, Q3, HR00_Q1 Custom Classes Can use explicit host name: no_samba = ( cis entityclient ) Use command that returns true/false: easyspooler_fix = ( `/usr/bin/test –x /usr/bin/llp -a ! –L /usr/bin/lp` ) Use built-in functions: easyspooler = ( FileExists(/usr/bin/llp) ) Built-in Functions for Classes Several built-in functions are available for class evaluation, including: True if f2 was modified more IsNewerThan(f1,f2) recently than f1 FileExists(file) True if file exists IsDir(dir) True if dir is a directory IsLink(file) True if file is a symbolic link Built-in functions execute more quickly than using the test command. Using Compound Classes Dot (.) is a logical AND: nfs.sunos_5_8:: Later cfengine versions also support ‘&’ for logical AND Vertical bar (|) is a logical Or: Hr00|Hr12:: Exclamation point (!) is logical NOT: !Hr00:: Parentheses override order: dbservers.(sunos_5_8|sunos_5_9):: Precedence is () – NOT – AND – OR Additional Class Info The any class is a generic all-inclusive group (same as not specifying a class) To find all defined classes using the default configuration file: /var/cfengine/bin/cfagent –p –v To find all defined classes using a configuration file other than cfagent.conf: /var/cfengine/bin/cfagent –p –v –f cfagent.test Sample groups Section groups: datatel = ( IsDir(/datatel) ) # Perform MD5 checksumming on these systems do_checksum = ( cis ) # Defines an EasySpooler system that needs # to have the EasySpooler llp binary used # instead of the default lp command. easyspooler_fix = ( `/usr/bin/test -x /usr/bin/llp -a ! -L /usr/bin/lp` ) Sample groups Section continued # Place machines in edit_password_defaults # to edit /etc/default/passwd file edit_password_defaults = ( cis entityclient ) # If at.allow or cron.allow exist, don't need # the .deny files no_atdeny = ( IsFile(/etc/cron.d/at.allow) ) no_crondeny = ( IsFile(/etc/cron.d/cron.allow) ) Control Section A configuration file must have a control section, otherwise nothing will be done Sets default variables Can also be used to define new variables Defines which actions are carried out and in what order Cfengine Variables Used for string substitution, similar to a macro processor Can be defined in the control section for use in other sections: datatel_age_hold = ( 30 ) May be defined within a specific group, but this must be used carefully – some must be defined globally to avoid runtime errors in the tidy section. Using Variables Variables are dereferenced either using curly braces or parentheses preceded by a dollar sign: exclude=${unidata_log_files} $(unidata_mnt)/bin Using undefined variables causes syntax errors. Control: Default Variables The control section can be used to set numerous variables that control execution Use access to list who can run cfengine: access = ( root ) Syslog activates syslog logging when an inform statement is encountered: syslog = ( on ) Defining Variables control: cfengine_note = ( "# Note: this file managed under cfengine" ) datatel:: unidata_mnt = ( /usr/ud71 ) datatel_owner = ( datatel ) # Database locations datatel_production = ( /datatel/coll18/production ) List Variables Variables may consist of multiple items separated by a colon: datatel_hold_dirs = ( ${datatel_production}/apphome/_HOLD_: ${datatel_development}/apphome/_HOLD_: ${datatel_test}/apphome/_HOLD_ ) unidata_log_files = ( ${unidata_mnt}/bin/udt.errlog: ${unidata_mnt}/bin/udtlatch.log: ${unidata_mnt} /bin/saved_logs/udtlatch.log ) Control Section: actionsequence The actionsequence variable specifies which actions are carried out and in what order: actionsequence = ( disable links ) Action sections in the configuration file that are not included in the actionsequence list are not performed actionsequence continued Classes may be used for control in the actionsequence statement: actionsequence = ( tidy.Hr03 disable links.ThisClass editfiles links.ThatClass ) The import Section The import section is used for reading additional configuration files: import: piopen:: cf.app_piopen For breaking large configuration files into smaller files or for using separate files for special processing Inheritance and import Files The main (or parent) file is completely parsed before the import file is read Variables and groups in the parent file are inherited in the imported file, but variables and groups in the imported file are not visible in the parent file The disable Section Cfengine will disable files (and directories) by renaming them instead of deleting them (as opposed to the tidy action). If no destination name is specified, the file will be renamed by appending the suffix .cfdisabled to the file name. disable can also be used to rotate files such as logs. disable syntax disable: class:: /filename dest=filename define=classlist syslog=true/on/false/off inform=true/on/false/off action=disable/warn … A disable Example disable: easyspooler_fix:: /usr/bin/lp syslog=true inform=true no_boot_server.(sunos_5_8|sunos_5_9):: # Don't run boot services /etc/rc3.d/S16boot.server dest=cfdisabled.S16boot.server define=boot_server_disabled syslog=true Feedback class The editfiles Section Performs line-based editing on text files (or limited binary editing) after making a backup of the file to be edited Supports simple regular-expressions Syntax different from other actions: editfiles: class:: { file-to-be-edited action “quoted-string…” } Sample editfiles Section editfiles: sunos_5_8|sunos_5_9:: # # # # IIPS Baseline 4.5 Set TCP initial sequence number generation to RFC 1948 unique-per-connection-ID { /etc/default/inetinit ReplaceAll "TCP_STRONG_ISS=[01]“ With "TCP_STRONG_ISS=2" } Sample editfiles Section continued # # # # { } IIPS Baseline 5.1 Enable TCP connection tracing by inetd (this is independent of any TCP Wrappers logging). /etc/default/inetd PrependIfNoSuchLine "$(cfengine_note)" UnCommentLinesContaining "LOGGING=" ReplaceAll "LOGGING=NO“ With "LOGGING=YES" DefineClasses "modified_inetd_conf" The filters Section The filters section does not perform actions, instead it is used for defining selection criteria that may be used in the files or processes sections. filters: { root_owned_files Owner: "root" Result: "Owner" } The files section The files section can be used for File creation Checking the existence, ownership, and permssions of files Changing the ownership and permissions of files Testing for setuid root programs Syntax for files files: classes:: /file-object mode=mode owner=uid-list group=gid-list action=fixall/other-options/warnall links=false/stop/traverse/follow/tidy ignore=pattern include=pattern exclude=pattern … Correcting File Permissions files: datatel:: ${datatel_production}/apphome mode=o+rw,g+rw,o-rwx owner=datatel group=users action=fixall ignore=_HOLD_ ignore=_PH_ ignore=BP recurse=inf Sample report of correcting file permissions Checking file(s) in /datatel/coll18/production/apphome cfengine:cis: Owner of /datatel/coll18/production/apphome/DATA/DATA_P/ PAYROLL.EXPORTS/200710MO was 1010, setting to 100 cfengine:cis: Owner of /datatel/coll18/production/apphome/DATA/DATA_P/ PAYROLL.EXPORTS/200710PT was 1010, setting to 100 cfengine:cis: Owner of /datatel/coll18/production/apphome/DATA/DATA_X/ XCSD.DIRECTORY/DCA*804*071*14536.SEQ was 1006, setting to 100 Creating Files # IIPS Baseline 6.5 # Make sure the machine tracks # failed login attempts /var/adm/loginlog owner=root group=sys mode=600 action=create File Monitoring Cfengine provides a file monitoring facility similar to the Tripwire program. Any file flagged for file monitoring in the files section will have its md5 checksum registered in a checksums database. On subsequent cfengine passes the file will have its md5 checksum computed and compared with the previously stored value; a warning will be issued if the values do not match. Configuring File Monitoring A file that stores the checksums must be defined in the control section: CheckSumDatabase = ( /var/cfengine/checksum.db ) Any files specified in the files section with the statement checksum=md5 will be monitored: ${unidata_mnt}/bin/udt_signal checksum=md5 inform=true File Monitoring Example files: (sunos_5_8|sunos_5_9):: /sbin/* checksum=md5 action=warnall /usr/bin checksum=md5 action=warnall include=cancel include=login … include=passwd include=su Controlling Updates To The Checksum Database The control section’s ChecksumUpdates variable controls updating the stored checksums The default value of no means the database will not be updated when a file’s checksum changes. If ChecksumUpdates is set to yes, when a file’s checksum changes a warning is issued once and then the new checksum is stored in the database. Maintaining the Checksum Database If a patch cluster has been installed, switch ChecksumUpdates to yes to store the checksums of new binaries in the database, then return ChecksumUpdates to off. Periodically set the CheckumPurge variable to on to remove files that no longer exist from the checksum database. The cfengine.hostname.log As cfagent searches file systems, it builds a log file of all root-owned setuid and setgid programs that are found. This log is stored in /var/cfengine; the file name consists of the string ‘cfengine.’, the system’s hostname, and the suffix ‘.log’ – e.g. cfengine.cis.log. Cfagent issues warnings on subsequent searches if a new root-owned setuid/setgid program is found that is not in the log file. The links Section Used to either check or create links: linkname -> object_to_link_to Symbolic links are the default unless type=hard is specfied. If the link exists but points to a different object, a warning is issued If the link is specified using the ‘!’ operator (linkname ->! object_to_link_to), an existing link that points incorrectly is changed to point to the correct object. The tidy Section The tidy action removes (deletes) files from the system tidy: /directory pattern/include=wildcard ignore=pattern exclude=pattern age=days syslog=true/on/false/off inform=/true/on/false/off A tidy Example tidy: datatel.tidy_hold:: $(datatel_hold_dirs)/ pattern=* ignore=*.txt ignore=*W2REPORT* age=${datatel_age_hold} The processes Section The process action is used to test for processes, signal processes, or restart processes A regular expression is used to search output from the ps command to find the process to be acted on A processes Example processes: Feedback class modified_inetd_conf:: "inetd" signal=hup no_snmp:: # Stop SNMP daemon "snmpdx" signal=kill inform=true syslog=true The shellcommands Section Executes system commands or external scripts Must specify full-path for security reasons Can specify owner, group, umask, etc. of command A shellcommands Example shellcommands: sunos_5_8|sunos_5_9:: # # # # Fix tape device permissions. Use a shell command because 'files' section doesn't work very well with symbolic links. "/usr/bin/chmod 0770 /dev/rmt/*" "/usr/bin/chown root:sys /dev/rmt/*” Some cfagent Runtime Options -f -h -n -p -v Use the file name after this switch Help – display version banner and options summary “All talk and no action.” Only print what has to be done without actually doing it. Parse the configuration file to check syntax and then stop. Verbose mode: print detail information cfagent Debugging Levels -d Enable debugging output -d1 Show only parsing output -d2 Show only runtime action output -d0 Both d1 and d2 levels output Test, Test, Test Modify actionsequence to test individual sections. Use –p and –n options Run in verbose (-v) mode and save output Use –d options when desperate Production Simplest approach uses cron to call a script that runs cfagent instead of using cfexecd Use a source-code control system for cfagent.conf file. Be sure you have a good backup ….