SAS: Managing Memory and
Optimizing System Performance
Jacek Czajkowski
09/29/2008
Optimizing System Performance

Optimizing System Performance consists of managing the interplay of the
following three critical computer resources:

I/O

Memory

CPU time
2
Definitions

Performance Statistics
are measurements of the total input and output operations (I/O), memory, and
CPU time used to process individual DATA or PROC steps. You can obtain these
statistics by using SAS system options that can help you measure your job's initial
performance and to determine how to improve performance.

System Performance
is measured by the overall amount of I/O, memory, and CPU time that your
system uses to process SAS programs. By using certain techniques and SAS
system options you can reduce or reallocate your usage of these three critical
resources to improve system performance. While you may not be able to take
advantage of every technique for every situation, you can choose the ones that
are best suited for a particular situation.
3
System Options

Options STIMER;
NOTE: DATA statement used:
real time 1.16 seconds
cpu time 0.09 seconds

Options FULLSTIMER;
NOTE: The SAS System used:
real time
0.16 seconds
user cpu time
0.01 seconds
system cpu time
0.02 seconds
Memory
1162k
Page Faults
0
Page Reclaims
2619
Page Swaps
0
Voluntary Context Switches
81
Involuntary Context Switches
6
Block Input Operations
0
Block Output Operations
0
4
Interpreting the Performance Statistics

Real time
represents the clock time it took to execute a job or step; it is heavily dependent on the
capacity of the system and the current load. As more users share a particular resource,
less of that resource is available to you.

CPU time
represents the actual processing time required by the CPU to execute the job, exclusive
of capacity and load factors. If you must wait longer for a resource, your CPU time will
not increase, but your real time will increase. It is not advisable to use real time as the
only criterion for the efficiency of your program because you cannot always control the
capacity and load demands on your system. A more accurate assessment of system
performance is CPU time, which decreases more predictably as you modify your
program to become more efficient.
5
Interpreting the Performance Statistics
Description of FULLSTIMER Statistics
Statistic
Real Time
Description
the amount of time spent to process the SAS job. Real
time is also referred to as elapsed time.
User CPU
System CPU
the CPU time spent to execute your SAS code.
the CPU time spent to perform system overhead tasks
on behalf of the SAS process.
the amount of memory required to run a step.
the number of pages that SAS tried to access but
were not in main memory and required I/O activity.
Memory
Page Faults
Page Reclaims
Page Swaps
Voluntary Context Switches
Involuntary Context Switches
Block Input Operations
Block Output Operations
the number of pages that can be accessed without I/O
activity.
the number of times a process was swapped out of
main memory.
the number of times that the SAS process had to give
up on the CPU because of a resource constraint such
as a disk drive.
the number of times that the operating system forced
a process into an inactive state.
the number of I/O operations performed to read the
data into memory.
the number of I/O operations performed to write the
data to file.
6
Optimizing I/O



I/O is one of the most important factors for optimizing performance. Most SAS jobs
consist of repeated cycles of reading a particular set of data to perform various data
analysis and data manipulation tasks. To improve the performance of a SAS job, you
must reduce the number of times SAS accesses disk or tape devices.
Process only the necessary variables and observations by:
 using WHERE processing
 using DROP and KEEP statements
 using LENGTH statements
 using the OBS= and FIRSTOBS= data set options.
Reduce the number of times it processes the data internally by:
 creating SAS data sets
 using indexes
 accessing data through views
7
Optimizing I/O
Process more data each time a device is accessed by:
 BUFNO=
SAS uses the BUFNO= option to adjust the number of open page buffers when it
processes a SAS data set. Increasing this option's value can improve your
application's performance by allowing SAS to read more data with fewer passes;
however, your memory usage increases. Experiment with different values for this
option to determine the optimal value for your needs.
 BUFSIZE=
When the Base SAS engine creates a data set, it uses the BUFSIZE= option to set the
permanent page size for the data set. The page size is the amount of data that can be
transferred for an I/O operation to one buffer. The default value for BUFSIZE= is
determined by your operating environment. Note that the default is set to optimize
the sequential access method. To improve performance for direct (random) access,
you should change the value for BUFSIZE=.
Whether you use your operating environment's default value or specify a value, the
engine always writes complete pages regardless of how full or empty those pages are.
If you know that the total amount of data is going to be small, you can set a small
page size with the BUFSIZE= option, so that the total data set size remains small and
you minimize the amount of wasted space on a page. In contrast, if you know that
you are going to have many observations in a data set, you should optimize
BUFSIZE= so that as little overhead as possible is needed. Note that each page
requires some additional overhead. Large data sets that are accessed sequentially
benefit from larger page sizes because sequential access reduces the number of
system calls that are required to read the data set. Note that because observations
cannot span pages, typically there is unused space on a page.
8
Optimizing I/O

SASFILE global statement
The SASFILE global statement opens a SAS data set and allocates enough buffers to
hold the entire data set in memory. Once it is read, data is held in memory, available to
subsequent DATA and PROC steps, until either a second SASFILE statement closes the
file and frees the buffers or the program ends, which automatically closes the file and
frees the buffers. Using the SASFILE statement can improve performance by reducing
multiple open/close operations (including allocation and freeing of memory for buffers)
to process a SAS data set to one open/close operation reducing I/O processing by
holding the data in memory.
9
Optimizing I/O

COMPRESS=
One further technique that can reduce I/O processing is to store your data as
compressed data sets by using the COMPRESS= data set option. However, storing your
data this way means that more CPU time is needed to decompress the observations as
they are made available to SAS. But if your concern is I/O, and not CPU usage,
compressing your data may improve the I/O performance of your application.
Long Character Values Dataset
Resource
Uncompressed
Compressed
Change
CPU
4.27 sec
27.46 sec
+23.19 sec
Space
235 MB
54 MB
-181 MB
Mostly Numeric Values Dataset
Resource
Uncompressed
Compressed
Change
CPU
1.17 sec
14.68 sec
+13.51 sec
Space
52 MB
39 MB
-13 MB
10
Optimizing Memory Usage



If memory is a critical resource, several techniques can reduce your dependence on
increased memory. However, most of them also increase I/O processing or CPU usage.
However, by increasing memory available to SAS, you can decrease processing time
because the amount of time that is spent on paging, or reading pages of data into
memory, is reduced.
MEMSIZE=
Specifies the limit on the total amount of memory to be used by the SAS System
SAS does not automatically reserve or allocate the amount of memory that you specify
in the MEMSIZE option. SAS will only use as much memory as it needs to complete a
process. For example, a DATA step might only require 20M of memory, so even though
MEMSIZE is set to 500M, SAS will use only 20M of memory.
Setting MEMSIZE to 0 is not recommended except for debugging and testing purposes.
To determine this optimal value, run the SAS procedure or DATA step with MEMSIZE set
to 0 and the FULLSTIMER option. Note the amount of memory used by the process and
then set MEMSIZE to a larger amount.
11
Optimizing Memory Usage



SORTSIZE=
Specifies the amount of memory that is available to the SORT procedure
SUMSIZE=
Specifies a limit on the amount of memory that is available for data summarization
procedures such as the MEANS, OLAP, REPORT, SUMMARY, SURVEYFREQ,
SURVEYLOGISTIC, SURVEYMEANS, and TABULATE procedures.
MVARSIZE=
Specifies the maximum size for in-memory macro variable values
12
Optimizing Memory Usage



MEXECSIZE=
Specifies the maximum macro size that can be executed in memory. Use the
MEXECSIZE option to control the maximum size macro that will be executed in memory
as opposed to being executed from a file. The MEXECSIZE option value is the compiled
size of the macro. Memory is allocated only when the macro is executed. After the
macro completes, the memory is released. If memory is not available to execute the
macro, an out-of-memory message is written to the SAS log. Use the MCOMPILENOTE
option to write to the SAS log the size of the compiled macro. The MEMSIZE option
does not affect the MEXECSIZE option.
MSYMTABMAX=
Specifies the maximum amount of memory available to the macro variable symbol
table. Once the maximum value is reached, additional macro variables are written out
to disk. A value of 0 causes all macro symbol tables to be written to disk. If this option
is set too low and the application frequently reaches the specified memory limit, then
disk I/O increases. If this option is set too high (on some operating environments) and
the application frequently reaches the specified memory limit, then less memory is
available for the application, and CPU usage increases.
REALMEMSIZE=
Indicates the amount of real memory SAS can expect to allocate. Use the
REALMEMSIZE system option to optimize the performance of SAS procedures that alter
their algorithms and memory usage. Setting the value of REALMEMSIZE too low might
result in less than optimal performance. For better performance, set REALMEMSIZE to
the amount of memory (excluding swap space) that is available to the SAS session at
invocation.
13
SAS System Options Lisiting

PROC OPTIONS;
Lists the current values of all SAS system options.
For example on our system:
MEMSIZE=67108864 (64Mb) Specifies the limit on the total amount of memory to be used by SAS
(maximum was found to be 4GB)
SUMSIZE=0
Upper limit for data-dependent memory usage during summarization
SORTSIZE=50331648 (48Mb) Specifies the amount of memory that is available to the SORT procedure
MEXECSIZE=65536 (64KB) Maximum size for a macro to execute in memory
MSYMTABMAX=4194304 (4MB) Maximum amount of memory allocated for the macro table
MVARSIZE=32768 (32KB) Maximum length of value of macro variable
REALMEMSIZE=0
Limit on the total amount of real memory to be used by the SAS System
BUFNO=1
Number of buffers for each SAS data set
BUFSIZE=0
Size of buffer for page of SAS data set

Setting value of 0 for some of the options causes the maximum allowable memory to be set
14
SAS Typical Memory/CPU Usage

Creating a copy of a dataset
data basic;
set indat.ganon3_1;run;
NOTE: There were 982 observations read from the data set GANON3_1
NOTE: The data set WORK.BASIC has 982 observations and 42068 variables.
NOTE: DATA statement used (Total process time):
real time
1.86 seconds
user cpu time
0.56 seconds
system cpu time
0.34 seconds
Memory
15820k
Page Faults
0
Page Reclaims
7215
Page Swaps
0
Voluntary Context Switches
1275
Involuntary Context Switches
4
Block Input Operations
0
Block Output Operations
0
15
Optimizing CPU Performance

One can Reduce CPU Time by:



Using More Memory
For example, you might be able to reduce CPU time by using more memory,
because more information can be read and stored in one operation, but less
memory is available to other processes.
Reducing I/O
Because the CPU performs all the processing that is needed to perform an I/O
operation, an option or technique that reduces the number of I/O operations can
also have a positive effect on CPU usage.
Executing a single stream of code takes approximately the same amount of CPU time
each time that code is executed. Optimizing CPU performance in these instances is
usually a tradeoff.
16
Optimizing CPU Performance

Other Techniques to improve CPU performance:

Storing a Compiled Program for Computation-Intensive DATA Steps
Another technique that can improve CPU performance is to store a DATA step that
is executed repeatedly as a compiled program rather than as SAS statements.
This is especially true for large DATA step jobs that are not I/O-intensive.
A stored compiled DATA step program is a SAS file that contains a DATA step
program that has been compiled and then stored in a SAS data library. You can
execute stored compiled programs as needed, without having to recompile them.
Stored compiled DATA step programs are of member type PROGRAM.
To store a compiled DATA step:
DATA data-set-name(s) / PGM=stored-program-name;
To load a compiled DATA step:
DATA PGM=stored-program-name ;
17