Colleague System
Maintenance
File System Administration &
Maintenance – Part One
Cinda Goff
Chuck Hauser
2005-10-26
Presentation Conventions

Names (files, users, daemons) are usually in bold:
/etc/syslog.conf
System dependent or variable items are usually in italics:
/var/sadm/patch/patchnumber/log
File entries and output are in mono-spaced type:
> root 8036 c Tue Apr 26 23:59:00 2005
< root 8036 c Tue Apr 26 23:59:59 2005
 marks a line wrapped to fit on the slide:

mv Solaris_8_Recommended_log
Solaris_8_Recommended_log.yyyymmdd
 marks a horizontal tab (09 hex)





Reference OE is Solaris 8
Reference UniData is 6.0
Introduction
Why maintain files
 UniData File Maintenance and Utilities
 Account File Cleanup
 Transaction Log Cleanup
 Status File Purging
 File Rebuilds

UniData Maintenance Utilities
 Datatel’s
WUFA - Weekly Udt
File Analysis (free)
 Fitzgerald & Long’s FAST File Analysis and Sizing Tool
(license required)
WUFA & FAST Primary Functions
 Report
file storage
 Recommend optimum block size and
modulo selection
 Automate file resizing
WUFA
Reports on and resizes both static and
dynamic files
 Support recently added for dictionaries

WUFA Recommendations
Leave all files static upon initial installation
 Use weekly WUFA run to evaluate
whether some files should be redefined as
dynamic
 For more information see AnswerNet
document 147.992: Whether to use Static
vs Dynamic UniData Files in Colleague.

WUFA Processing
Performs thorough system analysis on
both static and dynamic files
 Provides data file, UDT_GUIDE, for
statistical queries
 Creates DATATEL.RESIZE.FILES
paragraph for file resizing

Running WUFA
WUFA can be run with users on system
 DATATEL.RESIZE.FILES must be run in a
quiet account
 Use cron to automate WUFA execution

Automating WUFA with cron
Create for each account:
 shell script for cron execution
 Datatel crontab entry to run shell script
 VOC paragraphs to run WUFA
Datatel wufascript crontab entry


Login as datatel
If the EDITOR environment variable is not set or
exported, give the command:
EDITOR=vi ; export EDITOR

Giving this command to edit crontab:
crontab –e

Add the following line to run the wufa script every
night, Monday through Thursday:
0 22 * * 1-4
 /export/home/datatel/wufascript_collive
Unix Script to run WUFA


Use vi (or another editor) to create a unix shell script that runs a UniData
paragraph for the live remote account.
cd /export/home/datatel
vi wufascript_collive
Make sure the cd command in the script points to the appropriate account:
#!/usr/bin/ksh
umask 007
cd /datatel/live/collive
UDTBIN=/usr/ud60/bin
UDTHOME=/usr/ud60
export UDTBIN UDTHOME
$UDTBIN/udt<<ineof
WUFA
LO
ineof
#EOF
Variant WUFA Shell Script (part 1)
This version will accept an argument specifying the account to
run in; the default account is /datatel/live/collive. Example:
# wufa.ksh colconv
#!/usr/bin/ksh
# @(#)wufa.ksh
1.5 @(#)
# @(#) Source: cis:/opt/local/sbin/SCCS/s.wufa.ksh 1.5
 05/10/20 11:42:54 @(#)
# @(#) Last update: 05/10/20 11:42:56 @(#)
#
# Local Variables
PROG="`basename $0`"
DEFAULT_DATATELDIR="/datatel/live/collive"
DATATELDIR=""
Variant WUFA Shell Script (part 2)
# Get UniData Variables
. /usr/local/udtparams
Use the dot command to load the current
UniData value for UDTBIN, umask, etc.
# Only datatel should run this. UID is probably 100,
# but to be safe check user name.
UNAME=`id | sed -e 's/).*$//' -e 's/^uid=.*(//'`
if [ $UNAME != "datatel" ]
then
printf "$PROG: stopping, should be run by user
 datatel.\n"
exit 1
fi
Variant WUFA Shell Script (part 3)
if [ "$1" ]
Test for a script argument. If there is one, choose the correct account.
then
If there is no argument, use the default account (collive).
case "$1"
in
*live
) DATATELDIR="/datatel/live/collive" ;;
*test
) DATATELDIR="/datatel/work/coltest" ;;
*conv
) DATATELDIR="/datatel/work/colconv" ;;
*LIVE17 ) DATATELDIR="/datatel/release/LIVE17/INSTALL" ;;
*TEST17 ) DATATELDIR="/datatel/release/TEST17/INSTALL" ;;
*DEBUG17 ) DATATELDIR="/datatel/release/DEBUG17/INSTALL" ;;
*
) printf "$PROG: stopping, bad datatel directory
($1) specified!\n" ; exit 2 ;;
esac
else
DATATELDIR="$DEFAULT_DATATELDIR"
fi
Variant WUFA Shell Script (part 4)
if [ ! -d $DATATELDIR ]
then
printf "$PROG: stopping, directory $DATATELDIR does not
 exist!\n"
exit 3
fi
cd $DATATELDIR
$UDTBIN/udt<<INEOF
WUFA
LO
INEOF
logger -p user.info "WUFA run on $DATATELDIR"
# End /opt/local/sbin/SCCS/s.wufa.ksh
WUFA VOC paragraphs
Create a VOC paragraph in each account
that will run WUFA and send the results to
a hold file.
 Create additional VOC paragraphs for two
more hold files that will list any damaged
files and any files in level 2 overflow.

VOC paragraph to run WUFA
Top of "WUFA" in "VOC", 11 lines,
 167 characters.
001: PA
002: SETPTR ,,,,,3,BANNER UNIQUE WUFA,BRIEF
003: WEEKLY.UDT.FILE.ANALYSIS
004: SETPTR ,,,,,3,BANNER UNIQUE DAMAGED,BRIEF
005: XNC.DAMAGED.FILES
006: SETPTR ,,,,,3,BANNER UNIQUE OVER2,BRIEF
007: XNC.OVERFLOW.FILES
Bottom
VOC Entry for Damaged Files
Top of "XNC.DAMAGED.FILES" in "VOC",
 2 lines,103 characters.
001: S
002: LIST UDT_GUIDE BY FILE_NAME WITH
 DATATEL.DAMAGED GT "" FILE_NAME
 FMT "25" STATUS DATATEL.DAMAGED
LPTR
Bottom.
VOC Entry for Overflow 2 Files
Top of "XNC.OVERFLOW.FILES" in "VOC",
 2 lines, 177 characters.
001: S
002: LIST UDT_GUIDE BY FILE_NAME
FILE_NAME FMT "25L" STATUS OVERFLOW2
DATATEL.DAMAGED ID.SUP WITH
DATATEL.DAMAGED GT "" OR WITH
OVERFLOW2 GT 1 BLOCKSIZE
NEW_BLOCKSIZE FILETYPE LPTR
Bottom.
Resizing Files – MEMRESIZE



WUFA creates a DATATEL.RESIZE.FILES paragraph
that assumes a VOC entry called MEMRESIZE.
MEMRESIZE executes the unix level memresize
command.
Create a MEMRESIZE entry in both remote and install
accounts:
AE VOC MEMRESIZE
Top of "MEMRESIZE" in "VOC", 2 lines, 77 characters.
*--: P
001: PA
002: !/usr/ud60/bin/memresize <<I2,file>>
<<I3,modulo>> <<C4,ARG>> memory 32678
Bottom.
Resizing Files
Run the DATATAL.RESIZE.FILES
paragraph on a weekly basis.
 Exclude files from being resized by putting
them in a RESIZE.EXCLUDE savedlist.
 The VOC file is always excluded from
DATATEL.RESIZE.FILES.

Datatel resizescript crontab entry


Login as datatel
If the EDITOR environment variable is not set or
exported, give the command:
EDITOR=vi ; export EDITOR

Giving this command to edit crontab:
crontab –e

Add the following line to run the script every
Saturday morning:
0 2 * * 6 /export/home/datatel/resizescript_collive
Unix Script to run MEMRESIZE

Use vi (or another editor) to create a unix shell script that runs a UniData
paragraph for the live remote account.
cd /export/home/datatel
vi resizescript_collive

Make sure the cd command in the script points to the appropriate account:
#!/usr/bin/ksh
umask 007
cd /datatel/live/collive
UDTBIN=/usr/ud60/bin
UDTHOME=/usr/ud60
export UDTBIN UDTHOME
$UDTBIN/udt<<ineof
DATATEL.RESIZE.FILES
LO
ineof
#EOF
Resizing the VOC File




The DATATEL.RESIZE.FILES paragraph will not
resize the VOC file
VOC files should always be resized manually.
The account should always be quiet with no user
having access to the account.
The steps are performed either from an account
colon prompt or at the Unix prompt in the
account directory.
Steps for Resizing the VOC (Part 1)

From the colon/SHEL prompt, create a
new file with the proper modulo and
blocksize as recommended by WUFA:
CREATE.FILE VOC.NEW 179,2

Copy records from the original VOC file to
the new VOC:
COPY FROM VOC TO VOC.NEW ALL
Steps for Resizing the VOC (Part 2)

From the Unix prompt in the account directory,
rename the VOC files and remove the dictionary
of the new VOC:
# mv VOC VOC.OLD
# mv VOC.NEW VOC
# rm D_VOC.NEW

From the colon/SHEL prompt, delete the VOC
entry for VOC.NEW:
DELETE VOC VOC.NEW
Application/Account File Cleanup
appl.PPROCESS purge (UTJP)
 Transaction Log Cleanup
 EDX.STATUS purge
 ELF.TRANSACTIONS purge
 Express Load Purge Repository (ELPR)
 appl.PRINTERS

Transaction Log Cleanup - UTTP


UTTP can only be run
against files for which
transaction logging has
been set up locally.
If no transaction logging
has been set up, this
process does not need to
be run on a regular basis.
appl.PPROCESS Purge (UTJP)


Access the UTJP screen
from the appropriate
application in a remote
account.
UTJP allows selection by
date and time of process,
by username, or by
mnemonic.
EDX Status Archive Purge (EDSP)


The EDX phantom
creates a record in the
EDX archive (the
EDX.STATUS file) for
each DMI transaction.
The records stay in this
file until you delete them.
Use the EDX Status
Archive Purge (EDSP)
screen to delete records
from the EDX archive .
ELF.TRANSACTIONS Purge (EPRG)



The ELF.TRANSACTIONS
file is purged with the EPRG
screen.
This screen allows you to
purge batches that were
imported into Colleague
through the ELF routines.
These records are not
archived, so once they are
purged, they are gone.
Express Load File Purge (ELPR)
The Express Load Purge Repository (ELPR)
process purges:
 unneeded IMAGE directories
 VOC paragraphs
 data backup repositories for successfully
installed software updates
ELPR Screen


You can specify whether
or not to purge each item.
In the bottom part of the
screen, you can select
the software update
batches for purging by
entering Yes in the Purge
field (the default is “No”).
ELPR Comments


Remember that the software updates repository
may be shared by more than one installation
account.
If it is shared by more than one installation
account, make sure you don't purge the IMAGE
directories from this software updates repository
until you have loaded those updates into the
other installation accounts sharing this
repository.
ELPR Issue
There is a known issue with ELPR if you
have many items to purge.
 See AnswerNet document 8870.89 S_MIO_OPEN errors on ELPR for a
workaround.

Files Purged at the Colon Prompt






appl.PRINTERS
VOC RUN.#### files
_PH_ COMO Output
_HOLD_ Records
SAVEDLISTS
PARAMETERS




JSPARAMS
X.SECURITY.LOG
AE_COMS
WWW.STATE &
WWW.TOKENS
appl.PRINTERS



Records in appl.PRINTERS beginning with
SETPTR can be safely purged.
This should be done in a quiet account.
Run this command for each application:
SELECT appl.PRINTERS WITH @ID
 LIKE ‘SETPTR…’

If the above command returns records, execute
this command:
DELETE appl.PRINTERS
VOC Process Entries



Some Colleague processes create entries in the VOC
that can be cleaned up.
The entries should be removed in a quiet account.
Run this command to remove RUN.#### files:
SELECT VOC @ID LIKE “’RUN.’…”
 AND WITH F1 UNLIKE “C…”

If the above command returns records, execute this
command:
DELETE VOC
Express Load VOC Entries



The Express Load process creates various
entries in the VOC of both the install and remote
accounts.
Because multiple naming conventions were
used in the past for patches, there are multiple
steps for safely removing the various VOC
entries.
After the initial cleanup, all of the following steps
may not be needed.
Steps to Remove Express Load Entries

Select the _17014_UNIX records:
SELECT VOC WITH @ID LIKE …_17014_UNIX…

If the above command returns records, execute this
command: DELETE VOC

Select the _170_U records:
SELECT VOC WITH @ID LIKE …_170_U…

If the above command returns records, execute this
command: DELETE VOC
Steps to Remove Express Load Entries
continued

Select the _4178_U records:
SELECT VOC WITH @ID LIKE …_4178_U_…

If the above command returns records, execute
this command: DELETE VOC
_PH_ / _HOLD_ / SAVEDLISTS Directory Files
These are directory files in each account
that are used by various UniData and
Colleague processes.
 Most of the data in these files can be
purged on a periodic basis, but each
college should set local naming
conventions and retention standards.

_PH_ Records
The _PH_ file stores records of all
processes executed in background mode
and the output for COMO commands.
 COMO output records start with an O_
and the name assigned when starting the
como.

_PH_ Background Record Keys
Each background record has a multi-part key:
 ID of the VOC record executed in background
mode
 The internal representation of the time the
process was executed
 The internal representation of the data on which
the process was executed
The _HOLD_ File
The HOLD file is the database file in which
Envision Run Time writes report output
for processing by the BROWSE utility.
 Some records are keyed by the date and
time the user sent report output to the
HOLD file.
 Other records have IDs that are strings
users entered.

SAVEDLISTS File
The SAVEDLISTS file stores any created
list the user has saved using the SAVE
LIST command.
 Envision Run Time also temporarily
stores lists of records ids for use in
procedures in the SAVEDLISTS file

savedlists File
The savedlists file (lower case) is a
workfile that is used when selecting on
multivalues.
 All data in this file is transient and can be
deleted in a quiet account.

PH, HOLD, and SAVEDLISTS Clean Up
Multiple ways to clear data in these files.
 Method used depends on a college’s
standards and procedures.

The CLEAR.FILE method for PH, HOLD,
and SAVEDLISTS Accounts

The UniData CLEAR.FILE command can
be used to clear all records in the file:
CLEAR.FILE filename

The CLEAR.ACCOUNT command issues
a CLEAR.FILE command for the _PH_
and _HOLD_ files.
The SELECT and DELETE method for PH,
HOLD, and SAVEDLISTS Accounts



Use the UniData SELECT and DELETE
commands to selectively delete records in the
files.
This method uses local naming conventions to
build a list of records to be deleted.
The example assumes that users append
‘PERM’ to the end of any records to be kept.
SELECT and DELETE Example for PH,
HOLD, and SAVEDLISTS Accounts
SELECT fname WITH @ID
 UNLIKE …PERM
SAVE.LIST fname.DEL
EDIT.LIST fname.DEL 
GET.LIST fname.DEL
DELETE fname
To review and manually
remove anything that
should not be deleted.
The Unix find and rm method for PH,
HOLD, and SAVEDLISTS Accounts
Because these files are UniData DIR files,
the records can be manipulated using
standard Unix commands.
 This example uses find and rm to remove
HOLD files that are older than 30 days:

find /datatel/live/collive/_HOLD_
 -mtime +30 –exec rm {} \;
Clearing the PARAMETERS File
The PARAMETERS file is a work file that
can grow rapidly.
 It can be safely cleared in a quiet account:
CLEAR.FILE PARAMETERS

Clearing the JSPARAMS File




JSPARAMS is a Runtime Variable file.
The records result from the Application
Peripheral Options Default Screen.
The file holds ‘remembered’ user responses.
Records can be purged in a quiet account, but
do not use CLEAR.FILE to purge records!
Example Clearing of JSPARAMS

The key structure for JSPARAMS is:
PROCESS_FILENAME_OPERATOR_DATE_TIME

Select the JSPARAMS records:
SELECT JSPARAMS WITH @ID LIKE
‘..._ ..._..._...’

If the above command returns records, execute
this command: DELETE JSPARAMS
The X.SECURITY.LOG File


The NCCCS modification for verb security logs
access to verbs in the X.SECURITY.LOG file.
This file can grow large if there are many users
with colon level or Query Builder access.

Purge this file by selecting records by date:
SELECT X.SECURITY.LOG WITH XSL.DATE
 LE ’01/01/04’

If the above command returns records, execute this
command: DELETE X.SECURITY.LOG
Clearing the AE_COMS File


The AE_COMS file should be deleted monthly in
a quiet account.
Purge this file by selecting records by date:
SELECT AE_COMS WITH @ID LIKE ’com…’’pre…’

If the above command returns records, execute
this command: DELETE AE_COMS
WWW.STATE and WWW.TOKENS Files




WWW.STATE and WWW.TOKENS files are selfmaintaining as of DMI release 3.3 and higher.
If data is found in these files can be safely removed in a
quiet account.
If WebAdvisor is used in the account, it must be
inaccessible during the file purge.
Use CLEAR.FILE:
CLEAR.FILE WWW.STATE
CLEAR.FILE WWW.TOKENS
GUIDE_ (Guide Files)
When guide is run to fix damaged files, a
series of files beginning with GUIDE_ are
created in the account directory (ex.
/datatel/live/collive).
 These files need to be deleted regularly:
cd /datatel/live/collive
rm GUIDE*

Security File Maintenance
Occasionally the appl.SECLASS and
appl.PRCS.CTL files will get out of sync in
regard to security classes
 The BUILD.SECURITY functions will clear
out the appropriate fields in the
appl.PRCS.CTL file and rebuild based on
the appl.SECLASS definitions

Rebuilding appl.PRCS.CTRL

All applications should be rebuilt in the this order:
BUILD.SECURITY
BUILD.SECURITY
BUILD.SECURITY
BUILD.SECURITY
BUILD.SECURITY

UT
CORE
CF
ST
HR
These commands are issued at the colon, or as
an alternative, you may run BSEC from each
application.
Colleague System Maintenance Schedule
Frequency
Daily
Weekly
Monthly
Level Task
OS
System backup, either full or incremental
UniData File system maintenance using WUFA;
maintain any files that need manual
intervention due to damage or level 2 overflow
OS
Cleanup operating system logs and document
as required by state auditors. Check sulog,
wtmpx, utmpx, etc.
Monthly
UniData Remove any RUN.#### files in the VOC of
each remote account
Monthly
UniData Cleanup the _PH_, _HOLD_, SAVEDLISTS,
and savedlists Files
Colleague System Maintenance Schedule
Frequency
Level
Task
Monthly
UniData
Clean up the PARAMETERS file
Monthly
Colleague Clean up the appl.PPROCESS file using
the UTJP screen in each application
Monthly
Colleague Clean up the transaction logging files in
each application using the UTTP screen
Monthly
Colleague Clean up EDX.STATUS in the UT
application using the EDSP screen
Quarterly
Colleague Clean up ELF.TRANSACTIONS in each
application using the EPRG screen
Quarterly
Colleague Clean up Express Load files in each install
account using the ELPR screen
Colleague System Maintenance Schedule
Frequency
Level
Task
Quarterly
Colleague Remove any patch files in the VOC of each
remote and install account
Quarterly
Colleague Unix backup to supplement a third-party
backup strategy
Quarterly
Colleague Clean up the appl.PRINTERS file by
deleting all records with @ID beginning
with SETPTR
Quarterly
UniData
Clean up the AE_COMS file
Qaurterly
UniData
Clean up the JSPARAMS file
Quarterly
UniData
Clean up the X.SECURITY.LOG file
Colleague System Maintenance Schedule
Frequency
Level
Task
Quarterly
UniData
Clean up the WWW.STATE and
WWW.TOKENS files
Quarterly
UniData
Remove any guide GUIDE_ files form the
Unix directory of each remote account
Quarterly
Colleague Rebuild security classes for each
application using the BUILD.SECURITY
function