Siebel File System Cleanup
1
Introduction .......................................................................................................... 2
1.1
The Siebel File System ............................................................................... 2
1.3
Using the SFSMKDIR Utility........................................................................ 5
1.2
2
Using the Siebel File System Cleanup Utility ............................................ 2
Procedure ............................................................................................................ 6
2.1
Stop Siebel Server ....................................................................................... 7
2.3
Report ........................................................................................................... 7
2.2
2.4
2.5
2.6
Backup .......................................................................................................... 7
Repeated Cleanup ....................................................................................... 7
Dividing ......................................................................................................... 8
Start Siebel Server....................................................................................... 8
1
1.1
Introduction
The Siebel File System
The Siebel File System consists of a shared directory, which is network-
accessible to the Siebel Server that contains the physical files used by Siebel
clients. To gain access to files, Web clients connect to the appropriate Siebel
Server to request file uploads or downloads. The Siebel Server then accesses
the Siebel File System using the File System Manager (FSM) component. File
System Manager processes these requests through interaction with the Siebel
File System directory.
Files stored in the Siebel File System are compressed at the Siebel Server-level
and named according to the format
<TABLENAME>_<ROW_ID>_<REVISION#>.SAF. (Siebel Attachment File size
displayed in the GUI represents the size of the compressed .saf file, not the
actual file size.) The Siebel File System storage location of the compressed files
is set by the Siebel Enterprise Server parameter Siebel File System (alias
FileSystem). The files stored in the Siebel File System are not directly
accessible by users and must be decompressed and returned to the user
through the Web client.
1.2
Using the Siebel File System Cleanup Utility
Sometimes, for example due to bad synchronizing, an expected file does not
exist for a database record. On the other hand the File System can contain files
that are not used (anymore) by Siebel, for example files with an old revision
number. These files do not correspond with a database record and are called
orphan files.
The Siebel File System Cleanup Utility can be used to verify the integrity
between database records and files in the Siebel File System, or to remove files
from the Siebel File System that no longer have a corresponding database
record. It is a command-line utility, named sfsutl.exe in Siebel 6 or
sfscleanup.exe in Siebel 7, located in the bin subdirectory within the Siebel
Server root directory. The sfsutl.exe utility processes every file in the file
attachment directory and performs one of several operations to each file
depending on the file type and the parameters that you set. For descriptions of
the run-time parameters that you can set when running sfsutl.exe, see Table 1.
For descriptions of the file types and the associated operation that will be
performed by sfsutl.exe during processing, see Table 2. Run sfsutl.exe after
running siebenv.bat (for setting the Siebel environment variables) and use the
parameters listed in Table 1 as shown in the following example:
sfscleanup /u sadmin /p password /f \\server\files /x \\server\sfscleanup.log
Tabel 1.
sfsutl.exe Parameters
Parameter Value
Description
Required?
/u
Username
Username ID.
Y
/p
Password
Username password.
Y
/c
ODBC data
Set this value to the ODBC data
N
source
source. Default value is set to the
environment variable,
SIEBEL_DATA_SOURCE.
/d
/f
/x
/m
/n
(since
Siebel 7)
Siebel table
owner
Path for file
directory
Set this value to the Siebel table owner. N
Default value is set to the environment
variable, SIEBEL_TABLE_OWNER.
Set this value to the path for the file
attachment directory. Do not append att
to the file attachment directory path.
P ath for output
N
file
Set this value to the path for the output
Path for move
Set this value to the path for the
directory
Remove old
revisions
Y
file.
directory where discarded files will be
N
moved.
Determines if old versions of file
attachments will be removed. To remove
old versions, set this value to Y. Default
N
value is N.
/r
(since
Siebel 7)
/g
(since
Generate report
file only
Set this value to Y to generate only a
report file. If set to Y, the report file will
N
contain only the columns File Name and
File Type. Default value is N.
Garbage files
Siebel 7)
Set this value to remove garbage or
non-Siebel files. Default value is N.
N
If you specified an output file using the /x parameter, sfsutl.exe generates a log
file listing the operations that were performed. The output file is a tab-delimited
text file that contains the following columns:
 File Name
This column lists the name of each file that was processed.
 File Type
This column lists the type of each file that was processed. Table 2 lists the
possible file types and the associated operation that will be performed by
sfsutl.exe during processing.
Tabel 2. File Types and Associated Operation
File Type
Description
Operation
CURRENT The file has a corresponding record in the file attachment KEPT
database table.
NEW
The file is less than one hour old. Sfsutl.exe will not
KEPT
check for the file in the file attachment database table.
ORPHAN
The file does not have a corresponding record in the file
I NVALID
The file (or directory) is not a file attachment. If sfsutl.exe KEPT
attachment database table.
is attempting to delete a subdirectory that is not empty,
the operation will error out. This gives you an opportunity
to review the files contained within the directory before
DELETED
deletion.
ANCIENT
The file has an associated record in the database with a
different revision number.
KEPT
 Operation
This column lists the type of operation that was performed during processing.
Table 3 lists the types of operation that sfsutl.exe may have performed during
processing.
Tabel 3. Operations
Operation
Description
KEPT
The file was kept.
DELETED
The file was deleted.
MOVED
The file was moved to the directory specified by the /m
parameter. Files will only be moved if you used the /m
parameter.
KEPT_DIR
The item was kept because it was a directory and requires
manual processing.
KEPT_ERROR The file was kept because an error occurred while trying to
move or delete the file.
Note that the sfsutl.exe utility removes only a few files at a time, and needs to
be re-run several times in order to remove all orphaned files. This behavior is
being addressed in Product Defect 12-4OI6JS. (Please, refer to Service Request
38-276688359 on Siebel SupportWeb.)
1.3
Using the SFSMKDIR Utility
If the Siebel File System contains a large number of files, a decrease in
performance can be expected. NTFS supports volumes with large numbers of
files and folders, but Microsoft warns us to avoid putting a large number of files
into a folder if you use programs that create, delete, open, or close files quickly
or frequently. They suggest as solution to logically separate the files into folders
so that the workload can be distributed on multiple folders at a time. (See
http://www.microsoft.com/technet/treeview/default.asp?url=/TechNet/prodtechnol/winx
ppro/reskit/prkc_fil_punq.asp.) Microsoft Research and third-party experts were
contacted and they considered 20,000 files per subdirectory as good middle
ground (SupportWeb SR 38-258916951, "Sizing Siebfile Subdirectories".)
The SFSMKDIR utility allows you to create sub-directories in the Siebel File
System (SiebFS) in order that you can avoid the Windows limitations on the
maximium number of files in a given directory. The subdirectories which are
added will also increase the speed of access to files. If your file system
currently consists of a single directory, the SFSMKDIR utility will help you create
a set number of directories underneath this main file system root directory. For
example, if the current file system directory is d:/siebel/FS, running "sfsmkdir
d:/siebel/FS 10" will create 10 sub-directories: d:/siebel/FS/A, d:/siebel/FS/B...
d:/siebel/FS/J. The attachment files will be added across the subdirectories to
keep the sizes balanced.
There is no maximum size for each subdirectory that is set by Siebel - the
directories can grow as large as the operating system allows. The customer
currently does not have any control over which subdirectory is used for which
file. This is strictly controlled by the algorithm generated. The algorithm is stored
in a binary file called siebfile.cfg and is placed in the root directory of the Siebel
File System. This algorithm governs where files are stored and how they are
located, and is solely based on the file names and ROW_IDs contained in the
name. Statistically, each subdirectory will contain about the same number of
files, but there may be some fluctuations.
Having re-configured your file system in this way you can revert to using the
single root directory of the filesystem simply by re-running the sfsmkdir utility but
instead of specifying a number of sub-directories, you set the command
parameter to 0 instead, i.e., "sfsmkdir d:/siebel/FS 0", which will move all of the
files back to the root directory.
2
Procedure
I suggest to use a Siebel File System Cleanup procedure as follows:
2.1
Stop Siebel Server
Before starting the cleanup action it is wise to make the Siebel Application static
by stopping te Workflows and the Siebel Server. This way there is no contact
with the database, which could interfere the cleanup action. Furthermore, you
can note down the Row_ID of some existing Siebel Attachments as test
candidates for later testing.
2.2
Backup
You can consider making a backup of your Siebel File System by copying the
shared directory. A restore can be done by simply copying the shared directory
back. Because this backup step works the same way as the sfsutl.exe with the
Move parameter, it seems unnessessary to me if you trust the sfsutl.exe Utility.
2.3
Report
A report step can be considered, but seems to me unnessessary, because you
can make a report and a cleanup at the same time if you enable logging of the
cleanup. The report step is run as follows:
sfsutl /u sadmin /p password /c Srvr_ent /d siebel /f f:\siebfile
/x sfsutl_report.txt
2.4
Repeated Cleanup
Only the orphan files can be cleaned by moving them to another directory, for
example \siebfile_ discards. Thereby, the number of files decreases which could
increase System performance. In case of mistakes, a restore can be done by
putting back the missing Siebel File. The Cleanup is performed by the following
command:
sfsutl /u sadmin /p password /c Srvr_ ent /d siebel /f \siebfile
/m \siebfile_discards /x sfsutl_move_<move#>.txt
The Cleanup is repeated until the number of removed Siebel Files does not
increase anymore. This can be monitored in the logs by looking at the number
of files with Status "DISCARD" and/or by counting the removed Siebel Files in
their directory.
2.5
Dividing
This step is only needed when the number of files is so high, that it decreases
System performance. (For details, see 1.3 "Using the SFSMKDIR Utility".) When
performed, you can check if the SFSMKDIR Utility did create the file siebfile.cfg
in the root of the Siebel File System. I suggest to divide the Siebel File
Systems in as many subdirectories, that each subdirectory contain about 20,000
files.
2.6
Start Siebel Server
After checking the logs of the previous steps for errors, the Siebel Server and
the Workflows can be restarted. A test that the noted test candidates are still
accessible from the Siebel Server is highly recommended, even as is a test that
a new Siebel Attachment can be made correctly. The latter can be tested by
looking for a new file in the Siebel File System with the format
<TABLENAME>_<ROW_ID>_<REVISION#>.SAF, where the <ROW_ID> is equal
to the Row_ID of the record of the new Siebel Attachment in the Siebel Client.