UKCMG 2011
MXG BOF
May 16, 2011
Thame, Oxfordshire
MXG Version 29.04 dated May 17, 2011.
UPDATES TO MXG
Z/OS 1.12 Support
CICS/TS 4.2 Support
Z196 Support
DB2 V10 Support
EXITCICS COMPRESSED CICS AND DB2
TYPE113 Support
VMXGSRCH
VMXGFIND
SAS 9.2
WINDOWS SEVEN AND MSE
ANALDUPE
JCLSIMPL/JCLSPSMx/BLDSIMPL/BLDSPSMx
1. New stuff in MXG 28.01, Mar 9, 2010:
VMXGFIND 28.012
Kewl tool, find all obs in all datasets meeting,
a test (all obs with JOB=‘CICS’ in all PDB.
TYPESTC 28.005 Support for Sun StorageTek VSM Version 6.2/7.0
TYPE89
28.015 Support for z/TPM SMF 89 record, subtype wrong.
TYPENTSM 28.042 New Sentry VM 3.1.4.3 adds VMWARE objects/metrics.
TYPE30
28.031 z/OS 1.11 GA added variables to SMF 30 and SMF 71.
TYPEZTPF 28.043* zTPF has major revisions in Performance Data
TYPETMS 28.040* CA-1 Retention and VMRECORD extensions.
Changes:
TYPE74
28.039 R7451RID now one byte, R7451FLG/TYPE74CA overlays.
BUILDPDB 28.037 PDB.SMFINTRV have EXCP/IOTM counts for FLUSHED.
TYPE103 28.036 TYPE1032 deaccum needed PORTNR, label changed.
VGETOBS 28.034 %TRIM() references here removed, still in VMXGSUM.
IMACICMR 28.032 Protect 200-byte CMRDATA on CICS/TS 3.2 (s/b 256).
VGETDDS 28.014 Colon in DDNAMES= worked only with DDNAMES=PDB:)
TYPEDB2 28.010 Variable SHIFT (from QWHSSTCK, END) kept.
2. Major enhancements added in MXG 28.02, dated Apr 14, 2010
VMXGINIT
VMXGINIT
TYPE84
ASMIMSL6
TYPEIMS7
TYPEMVCI
TYPEDB2
TYPETMNT
TYPEDB2
TYPE42
BUILDPDB
MXGSAS92
TYPERMVV
28.081
28.057
28.077
28.066
28.066
28.065
28.051
28.079
28.073
28.072
28.071
28.070
28.048
VARLENCHK=NOWARN eliminates SAS V9.2 WARNINGS
ERROR MACRO %ABORT IS NOT IN SAS 8.2, ONLY.
Support for all JES3 JMF TYPE84 subtypes.
Support for IMS Version 11 (INCOMPATIBLE).
Support for IMS Version 11 (INCOMPATIBLE).
Support for BMC CICS CMRDETL C660 for CICS/TS 4.1
Support for DB2 APAR PK62161 new SQL Counters.
TOTAL LENGTH OF VARIABLES MUST BE LT 32760.
DB2STATS had missing values for QW0225 variables.
TYPE42X4 Above the BAR LRU variables wrong.
PDB.STEPS/PDB.JOBS duplicates if FLUSHED steps.
SAS 9.2 TS2M2 DSNAMES may have changed
RMFV CPUG3 was misaligned in z/OS 1.1122.
3.
BLDSMPDB
CONFIGVx
VMXGSRCH
BUILDPDB
28.125
28.128
28.147
28.139
Support for Week/Month if 1st-day-of-week NOT MON.
ERROR APPARENT MACRO TRIM NOT RESOVED: MORE DOC.
Kewl Tool. Find all instances of VARIABLE='VALUE'.
Recently added SMF30xxx vars kept in PDB.STEPS.
Major enhancements added in MXG 28.03, dated May 25, 2010
TYPEWPMO
TYPECTCP
TYPE80A
VMXGSUM
TYPE7072
VMXGSET
SMFRECNT
TYPE110
TYPECIMS
28.086
28.108
28.107
28.105
28.099
28.098
28.089
28.087
28.084
Support for Windows Performance Monitor PERFMON.
Support for CleverView for TCP/IP TN3270 SMF.
TOKDANAM BINDDN,BINDPW,APPLNAM,UTYPE,JPNUM.
Optional KEEPWEEK/MNTH/YEAR/DAYS/ keeps TRENDs.
Variable CPULHKTM, CPU TIME Lock Promoted, 72GO.
DSETOPT= optional argument for data set options.
BUILDPDBs PDB.SMFRECNT now has bytes and counts.
Internal Decompression Algorithm use now ERROR:'d.
BMC IMF INPUTCLS and LASTCLAS variables restored.
4.
Many
28.175
TYPE70
28.175
TYPE113
28.166
TYPE119
TYPEITRF
TYPECTCP
TYPE42
TYPEVM
TYPE102
TYPENMON
IMACCADI
TYPETPFC
TYPEZCOS
UTILPDSL
IMACZDAT
28.175
28.162
28.160
28.158
28.157
28.156
28.176
28.172
28.152
28.151
28.179
28.174
Support for z/OS 1.12 (REQUIRES MXG 28.05).
DO NOT USE MXG 28.04 WITH z/OS 1.12 DATA INCOMPAT.
Support z196: REQUIRED ONLY WITH GT 64 ENGINES.
(Lots of new data added compatibly.)
Major revision - TYPE113 - John Burg's SHARE 2010.
Calculation of RNI, new z196 fields, new metrics.
Support for SMF 119 new subtypes 32-37 and 48-52.
Support for ITRF V420 IF2 (COMPATIBLE).
Support for CleverView GMT offset, CTCPIPAD fixed.
Support for APAR OA31648 TYPE42D1/D3 buff gets
Support for VM ACCOUNT ID='09' ISFC record.
Support for IFCID=27 specific variables.
Support for SARMON - Solaris SAR in NMON format.
Support for CA-Dispatch V11 SMF 6 INCOMPATIBLE.
Support for zTPFC TPF Continuous Monitoring .
Support for zCost AutoSoftcapping V 1.5.00 SMF.
Utility to read PDS/PDSE directories of a concat.
Example to set ZDATE when you rebuild a past PDB.
5. Major enhancements added in MXG 28.05, dated Aug 18, 2010
The z196 processor with more than 64 engines REQUIRES MXG 28.05.
A z196 with LESS THAN 64 engines DOES NOT require MXG 28.05, as
long as the operating system is z/OS 1.11 or earlier.
IBM Maintenance APARs OA30563,OA33976 REQUIRES MXG 28.05.
Major enhancements added in MXG 28.06, dated Oct
TYPEWSMQ
TYPEDB2
ASUMDB2EXITCICS
DFH$MOLS
TYPEITRF
TYPEIMFS
TYPE113
TYPE74
28.233
28.222
28.220
28.223
28.223
28.227
28.193
28.226
28.212
7, 2010
Support for WebSphere MQ Version 7 Accounting
ITRM only, DB2STAT4 NOT SORTED ERROR.
DB2 Summary ASUMDBxx and Trending TRNDDBxx.
Support for DB2 V10 Compressed SMF records.
JCL example to use IBM CICS decomp utility.
INPUT STATEMENT EXCEEDED RECORD LENGTH type=17x.
Full support for IMF records in SMF format.
Variable LPBUSY,LPARBUSY replaced LPARCPU. m
TYPE74ID (small) created, saves pass TYPE74CA.
6. MXG Version 28.07 dated Nov
5, 2010.
Major enhancements added in MXG 28.07, dated Nov
TYPEDB2
28.264
TYPEOMMQ
TYPEMIM
GRAFCEC
TYPETPX
UTILEXCL
TYPE120
ANALID
READDB2
VMXGSUM
TYPE110
TYPE30
TYPESTC
28.263
28.262
28.261
28.260
28.259
28.258
28.257
28.250
28.249
28.247
28.246
28.244
5, 2010
Support for DB2 Version 10. COMPLETELY INCOMPAT:
MXG 28.06 was required to process the V10 data,
now, MXG 28.07 has full support.
Support for IBM/OMEGAMON XW MQ file (INCOMPAT)
Support for CA MIM RESOURCE SHARE R11.7 (COMPAT)
SAS/GRAPH example charts CEC Util by engine
IP address and Port Number now decoded in TPX
Spurious "WRONG LENGTH OF 200 FOR CMRDATA"
Support for WebSphere ID=120 SUBTYPE=20 records
ERROR: VARIABLE IDANDSUM ... with PDB,DISP=OLD
COPYONLY logic now works.
VMXGSUM enhanced with MODE and MEDIAN statistics
Example using _Kdddddd to create new datasets
New CPITCxTM/CPISRxTM wrong in MXG 28.06.
STC/STK/Oracle VSM user SMF records support
7. MXG Version 28.08 dated Dec 13, 2010, thru Change 28.288.
TYPE113
TYPE89
TYPENMON
TYPETNG
TYPETMVT
TYPE110
ASUMUOWT
ASUMCICR
DB2ACCT
TYPE89
WEEKBLD
UTILNPRT
28.279
28,282
28.275
28.273
28.287
28.285
28.284
28.281
28.277
28.272
28.269
28.268
"Near duplicate" ASUM113 intervals corrected.
Support for APAR OA31615, zIIP/zAAP CPU times.
Support for NMON FCREAD/FCWRITE/XFERIN/XFEROUT
Support for more than 9999 instances in CA NSM.
Support for ASG-TMON for VTAM subtype 'SX'.
CICS Statistics Subtype 2 STID=143 corrected.
ASUMUOWT (for ASG-TMON MRO) uses VMXGUOW.
Count/avg response time by DATE for each APPLID.
NETSNAME/UOWTIME only created QWHCATYP=4 (CICS).
SMF89HOF/SMF89DTO SCRT don't use last 3 nybbles.
TYPE72DL NOT SORTED after Clock Set Back.
Identify non-print chars, SAS Enterprise Guide
8. MXG Version 28.28 dated Jan 18, 2011, thru Change 28.331.
TYPE89
TYPE111
JCLINSTT
TYPENDM
TYPE102
TYPEIMSA
TYPEIMS7
TYPE0
BUILDPDB
ANALDUPE
TYPEVMXA
TYPEVMXA
UTILGETM
TYPE89
TYPE30
28.331
28.329
28.328
28.327
28.325
28.311
28.310
28.313
28.305
28.308
28.315
28.307
28.312
28.304
28.302
INVALID DATA FOR SMF89CZT if APAR OA31615 on.
CTG records had zero obs in TY111CXI "IPIC".
JCL example to ftp/unterse/install on z/OS.
Connect Direct/NDM 'RT' record INCOMPAT change.
DB2 SQL-text vars only 100 bytes COMPRESS=NO.
Support for IMS/DBCTL transactions in IMSTRAN.
Support for IMS/DBCTL transactions in IMS0708.
Variable CVTTZ in TYPE0 could be one sec wrong.
PDB.NJEPURGE did not contain all NJE-variables.
Removal of Duplicate SMF (or any) records.
PFXCPUAD in VXSYTCUM is the LCPUADDR, no CPUID.
Short LINUXKRNL MONWRITE record caused errors.
No Reporting Class data in SMFSMALL NRECORD=10.
SMF 89 with no usage segment INPUT EXCEEDED.
TYPE30MU duplicate records exist, non-dupes.
9. MXG Version 29.01 dated Feb
4, 2011, thru Change 29.022.
BLDSMPDB
WEEKBLD
MONTHBLD
These two impacted MONTHLY build:
29.017 SERIOUS ERROR CORRECTED: last day's PDB skipped.
29.017 LIBNAME WEEK1 not found corrected.
These two eliminate possibility of NOTSORTED errors:
29.008 SORTEDBY=NO default to eliminate NOTSORTED.
29.008 MXGNOBY default to eliminate NOTSORTED exposure.
29.008 MXGNOBY default to eliminate NOTSORTED exposure.
TYPEENDV
TYPE111
TYPE115
TYPE89
29.012
29.001
29.015
29.002
MONTHBLD
BLDSMPDB
Support for Endeavor Version 14 (INCOMPATIBLE).
Support for IPIC creates obs in TY111CXI.
Support for MQ Version 7 compression statistics.
Support for APAR OA31615, zIIP/zAAP times added,
and false error messages are eliminated..
10. MXG Version 29.02 dated Mar
1, 2011, thru Change 29.050.
VSETMNTH 29.041
POSSIBLE LOSS OF MON DATA IN FEB MONTHLY PDB.
(Unfortunately, EVEN with the newest MXG 29.01).
ONLY IF USING NEW VSETMTH LOGIC IN MONTHBLD.
Future MONTHly's can also be missing one or more
day's PDBs in your MONTH PDB.
Complete details are in Change 28.041, below.
TYPENDM
VMACDB2H
TYPE30
TYPECIMS
TYPE0
TYPEDB2
TYPEVMXA
TYPE30
TYPE26J2
Support for NDM-CDI Version 5 records (COMPAT).
DB2 V9.1 false "INVALID DISTRIBUTED HEADER" msg.
Invalid data for SMF30RGT is true, circumvented.
Support for IMF Version 4.5 is in place.
PDB.IPLS, now, DOES always report a SYSTEM IPL.
DB2 V9.2 only, QBGxxx variables DB2GBPST wrong.
Support for zVM APAR VM64794 (COMPATIBLE).
Small negative CPUUNITS now set to zero.
Cosmetic: INREASON NOT DECODED msgs corrected.
29.042
29.037
29.034
29.033
29.032
29.031
29.026
29.025
29.024
11. MXG Version 29.03 dated Apr 19, 2011, thru Change 29.094.
TYPE110
TYPE116
TYPE115
TYPEBBMQ
TYPEQACS
TYPE110
TYPE120
TYPETPMX
TYPENTSM
UTILVREF
BUILDPDB
TYPERACF
TYPEBETA
TYPE30
MONTHxxx
TYPE85
ASUM70PR
TYPEVMXA
29.094
29.057
29.057
29.056
29.078
29.076
29.081
29.071
29.075
29.075
29.068
29.067
29.059
29.058
29.052
29.093
29.092
29.092
1st MXG 29.03 ONLY. CICSTRAN CPUTM fields WRONG.
Support for Websphere for z/OS MQ Version 7.0.1.
Support for Websphere for z/OS MQ Version 7.0.1.
Support for MainView MQ (MVMQ) Version 4.4.
Support for OS/400, AS/400 Version 7.1 (INCOMPAT)
CICS CPUTM exceeds ELAPSTM, zAAP/zIIP Equivalent
Support for User Field in SMF 120 Subtype 9.
Support for Throughput Manager subtype 10 and 16.
Support for 62 new objects and 1425 new metrics.
MXG creates DATASET names up to 32 characters.
28.28-29.02. ABEND=JCL obs missing in PDB.JOBS.
RACF UNLOAD dataset RACF0270 UID limit variables.
Support for Beta 93 Version 4.2 subtypes 25/50.
Variable CPUCEPTM always a missing value.
SAS 9.1.3 Only. %QCMPRES needed versus %CMPRES.
INPUT STATEMENT EXCEEDED st 79, z/OS 1.12.
ZIPCPUS/IFACPUS included parked time.
z/VM new PDB.VXINTUSR sums all engines each VM.
12.
MXG Version 29.04 dated May 17, 2011, thru Change 29.115.
TYPE105
DB2ACCT
TYPEIMSA
BUIL3005
VMXGSRCH
TYPE70PR
TYPE110
29.100
29.111
29.110
29.106
29.103
29.098
29.097
Support for GDPS Global Mirror V3R8 SMF 105.
DB2 CICS TRAN name wrong, now from QWHCCV.
the exit _IMSTRN was accidentally removed.
JES3 PDB.JOBS variable JOBCLAS8 after change.
RESULTS=FINDVAR finds all datasets with a var.
Counts ICFCPUS/IFLCPUS/IFACPUS/ZIPCPUS too high.
INPUT EXCEEDED 110-2 MNSEGLC=5 with DPL segment
13. VMXGFIND
EXAMPLES:
1. FIND ALL OBSERVATIONS FOR A JOB NAME:
%VMXGFIND(PDB=PDB SPIN,
PDBOUT=WORK,
KEEPIN=JOB,
FIND= IF JOB=:'CICS'; ,
PRINT=99);
2. FIND ALL OBSERVATIONS FROM/FOR A SPECIFIC USER FROM PDB1-PDBN
AND SPIN LIBRARIES:
%VMXGFIND(PDB=PDB: SPIN,
PDBOUT=WORK,
KEEPIN=FSRUID
JOB
RACFUSER QWHCAID,
FIND=IF RACFUSER='XYZ12345' OR
QWHCAID ='XYZ12345' OR
FSRUID ='XYZ12345' OR
JOB
='XYZ12345';,
PRINT=YES);
14. VMXGSRCH EXAMPLES:
TO FIND ALL OBSERVATIONS IN ALL SAS DATASETS IN THE PDB LIBRARY
THAT HAVE ANY VARIABLE THAT CONTAINS "SYSD", AND PRINT THE FIRST
TEN OBSERVATIONS OF EVERY FOUND DATASET:
%VMXGSRCH(LIBNAME=PDB,RESULTS=PRINT,VALUE=SYSD,NOBS=10);
TO FIND ALL VARIABLES IN ALL ALLOCATED DATA LIBRARIES WITH THE
TEXT "CPU" IN THE LABEL
%VMXGSRCH(VALUE=CPU,RESULTS=LABEL);
WILL REPORT EVERY SAS DATASET AND VARIABLE IN THOSE "PDBS" THAT
HAVE CPU IN THEIR LABEL.
TO FIND ALL VARIABLES WITH TIME IN THE FORMAT (WILL ALSO FIND
DATETIME)
%VMXGSRCH(VALUE=TIME,RESULTS=FORMAT);
WILL REPORT EVERY SAS DATASETS/VARIABLES WITH TIME IN THE FORMAT
15. CICS Record Size Growth
TS
1.1
1.2
1.3
2.1
2.2
2.3
3.1
3.2
4.1
4.1
NUMBERCICS
DATA-LENGTH FIELDS
MXG
RELEASE MCTSSDRL
MCTSSDCN
VERSION
1.6.1
176
54
1.7
312
60
2.1
320
61
3.1
380
70
3.2.1
388
71
9.9
3.3.0
444
83
10.01
4.1.0
572
107
13.09
4.1.1
852
165
13.09
5.1.0
664
124
14.07
5.2.0
732
134
15.06
5.3.0
1288
203
16.09
6.1.0
1564
230
18.11
6.2.0
1620
239
19.08
6.3.0
1692
252
21.04
7.1.0
1848
286
22.12
7.2.0.5
2352
307
25.03
7.3.0.4
2512
317
26.1
7.3.0.5
2640
330
27.01
16. Assemble and Install the EXITCICS
THIS ASM PROGRAM IS ASSEMBLED AND LINK-EDITED INTO A LOAD MODULE
THAT IS NAMED CICSIFUE THAT IS STORED IN A LOAD LIBRARY DATASET
THAT IS CONCATENATED TO THE //STEPLIB DD STATEMENT IN YOUR
MXGSASVN
JCL PROCEDURE.
THE LOAD MODULE IS A SAS INFILE EXIT.
1)
2)
3)
%LET SMFEXIT=CICS;
FILENAME SMF 'dsname of dataset' ENGINE=CICS;
DATA X; INFILE SMF; INPUT ...
DATA X; INFILE SMF CICS; INPUT ....
THAT FIRST %LET STATMENT CAN BE PLACED IN THE "IMACKEEP" MEMBER
IN YOUR USERID.SOURCLIB TAILORING LIBRARY, OR IT CAN BE PLACED IN
THE //SYSIN DD STREAM AS THE FIRST STATEMENT..
EXITCICS decompresses CICS 110, 112, and DB2 100, 101, and 102
records.
17. The INTERNAL SAS algorithm on z/OS is VERY CPU intensive
(and it takes a long time, too!). DFH$MOLS and read UNCOMP
only slightly slower than reading COMPRESSED+EXIT, BUT
uncompressed file needs nearly 5 times the disk space.
I/O activity with DFH$MOLS (read comp, write uncomp, read
took six times the EXCPs and four times the IOTM (Connect Time)
Reading of the compressed file with the EXITCICS exit is best.
On Windows/ascii platforms, SAS Infile Exits do not exist.
Only two ways to process compressed CICS data on ASCII:
a. Use DFH$MOLS first to uncomp file and read UNCOMP.
c. Use MXG's internal algorithm to read COMP NO EXIT.
Elapsed
User
SYS
Size
a. DFH$MOLS
.4
.07
.00
212/970
ftp download
2.0
.04
.00
970 MB
UNCOMPRESSED
.4
.23
.05
970 MB
total
2.8
.34
.05
c. ftp download
0.5
.01
.00
212 MB
INTERNAL SAS
3.8
2.71
.05
212 MB
total
4.3
2.72
.05
18. SAS Version Requirements information:
MXG 29.04 executes best with SAS V9.2, or with SAS V9.1.3 with
Service Pack 4, on any supported SAS platform.
And, only for z/OS 1.10 with SAS V9.1.3 with ANY version of MXG,
the SAS Hot Fix for SN-35332 is REQUIRED (to be completely safe).
Old MXG code may continue to execute with SAS V8.2, but V8 is now
"Level B" support from SAS Institute, and there are known errors
in V8.2 that are only fixed in SAS V9. I no longer QA with V8.2;
While many MXG programs (accidentally) will still execute under
V8.2, I can not guarantee that all of MXG executes error free.
19. Install of SAS V9.2 for z/OS is rumored to be difficult,
BUT BY following the SAS Install Instructions at:
http://preview.tinyurl.com/443mlpd
upload and install of the (7.5 GB) z/OS SAS/BASE took 2 hours
BUT ONLY if everything needed is already in place!).
Size of the depot between 5GB and 17GB
depending on products
SAS Depot will be uploaded to a ZFS filesystem
z/OS restriction of 4GB for size of normal zFS mount point
Need authority to define a larger zFS mount point
Either must be created with the SMS Data Class EXTENDED attribute.
Need two zFS directories of 5GB to 17GB because 2nd SASHOME
But, one last note, do NOT use that EXTENDED attribute for the Data
Class of your SAS Data Libraries on z/OS; it is not supported due
to SAS's use of the EXCP access method.
20. Using MicroSoft Security Essential, MSE, causes various errors
the MXG QA job is run, never at the same place in the job:
ERROR: File Deletion Failed For MXGSUM1 (after 293 succeeded)
ERROR: An I/O error occurred on file WORK._tf6737.UTILITY
ERROR: Rename of temporary member WORK.OPTVAR.DATA failed.
BUT BY disabling MSE Settings to:
-exclude the SAS.EXE process, and
-exclude files *.sas7bdat and
-exclude the c:\qa directory, where all output is written, &
-exclude the c:\sastemp "WORK" directory.
were both errors were eliminated, with WINDOWS 7 64 bit.
“SAS NOTE: Do not run your antivirus software in real-time mode.
OR be sure to exclude the following file types from your
virus scan:
SAS*
LCK*
UTL* “
21. Issue open with SAS Institute and Microsoft since Feb, 2010;
In October, a new MicroSoft "Senior Escalation Engineer" attempted
resolution, providing instructions to install several MicroSoft
diagnostics tools that either failed to initialize or failed to
capture the event data, including runs with TTTracer that generated
over 85 GigaBytes of trace (how do you send a file that big??)
that still did not capture anything of use to MS.
Nov 2010: 14 MicroSoft updates auto installed, error went away.
MS engineer was unable to identify why the error disappeared.
Moral: Disable MSE for SAS.
MAY 2011: WINDOWS 7 ULTIMATE AUTOINSTALL SP1 REINSTATED ERRORS.
DISABLING MSE REAL TIME SCAN WAS REQUIRED TO RUN QA.
USING MSE SETTINGS (process/dir/file) DID NOT CIRCUMVENT.
ADDITIONALLY:
RUN SAS AS ASMINISTRATOR MAY BE REQUIRED WITH SP1.
ERROR: User does not have appropriate authorization level for
library C:\PROGRAM FILES\SAS\SASFOUNDATION\9.2
(to write qasas.log and qasas.lst files).
22. Removal of duplicate (SMF) records on z/OS - new ANALDUPE.
500,000 SMF Records Processed
Several techniques for removal of duplicate SMF records on z/OS
are provided in the new ANALDUPE member. Two approaches are both
sort-based but are limited by requirements for MASSIVE amounts of
disk space or tape drives and require more CPU time than the two
elegant solutions created by MP Welch, who imagined a solution and
discovered that the SAS V9 MD5 (digital signature) function could
be used to create a unique Hash Value for each record, and the Hash
Values are then sorted (instead of the full record), to MASSIVELY
reduce the disk and CPU requirements. A one-pass solution using
a hash table works fine, but could rapidly exhaust virtual storage,
so the recommended solution creates the MD5 Hash Value, but then
uses a second step (freeing temp space of the first step) and a SAS
Format for the look up table to remove duplicates.
23.
SORT ONE
DISK BASED
29 SORTWRK
1000 CYL
CPU
SRB
EXT
SYS
EXCP
CONN
CLOCK
41.40
3.60
31,780
11,860
484,000
27.052
648.00
SEC
SEC
K
K
SEC
SEC
SORT TWO
TAPE BASED
7 TAPE DRIVES
REQUIRED
49.80
6.60
31,776
11,864
463,000
20.40
1380.00
SEC
SEC
K
K
SEC
SEC
SORT THREE
MD5 HASH FUNCTION
HASH TABLE
ONE PASS
16.2
0.6
43,848
11,884
84,000
15.00
18.00
SEC
SEC
K
K
SEC
SEC
SORT FOUR
MD5 HASH
SAS FORMAT
TWO STEP
16.2 SEC
1.2 SEC
50,284 K
12,060 K
126,000
18.00 SEC
42.00 SEC
Clearly it's much more efficient to hash a record and operate on a
shorter value than operating on the full record itself. In this
case, it works particularly well because there is no expectation
nor requirement to reorder the records. The Hash table filled 2GB
of memory at 3.5 million unique records. But the two pass hash will
handle hundreds of millions of records in most shops.
24.
JCLSIMPL and JCLSPxxx examples use UTILBLDP/BLDSMPDB and
are THE now-recommended z/OS jobs for a "SIMPLE" BUILDPDB
or the "SPLIT SMF" family of "BUILDPDB" jobs.
JCLSIMPL creates a "simple", PDB library, with one job
that reads the SMF file, showing how to add an SMF record
and invoking all of the default ASUMxxxx members to build
a "single", default PDB data library from raw SMF data.
You could do the same with BUILDPDB and the EXPDBxxx exit
members, but these more recent utility macros are now the
recommended way to build/tailor a simple BUILDPDB:
UTILBLDP - defines what data is to be created in a PDB
you can add, subtract, or change what's kept
by each of these jobs using UTILBLDP to create
a specific suit of MXG datasets in a PDB
built from SMF data records.
BLDSMPDB - flexible job manager creates day/week/etc
PDBs using the UTILBLDP execution preceding
its invocation to define the PDB contents.
Will process non-SMF data records.
25.
JCLSPxxx is a family of jobs to read "split" subsets of
SMF and other data records to parallelize the BUILDPDB,
using the above+ UTILBLDP and BLDSMPDB members:
JCLSPGDG - run once to create GDGs, and then never again
unless there is a need to alter a GDG base or
to change dataset names..
JCLSPLIT - first job in daily stream - standalone splits the daily SMF into pieces for
subsequent processing
SMF.ALL
- All SMF for archive
SMF.CICS
- SMF 110.1
SMF.DB2
- SMF 101/102
SMF.IO
- SMF 14/15/42/61/65/66/74/240/241
SMF.MQ
- SMF 115/116
SMF.SPLITPDB - All other SMF records
JCLSPSMA/JCLSPSMB/JCLSPSMC/JCLSPSMD/JCLSPSME can be run
concurrently to process the split SMF files:
26. Family of jobs to read the SPLIT SMF files:
JCLSPSMA - Read only CICS SMF 110.1, create:
JCLSPSMB - Read only DB2 SMF 101/102, create:
JCLSPSMC - Read only I/O records, create:
JCLSPSMD - Read only MQ records, create:
JCLSPSME - Read all remaining SMF, create:
JCLSPOTH - DCOLLECT, TMC.
JCLSPUOW - after JCLSPLTA and JCLSPLTB have run,
build PDB.ASUMUOW from CICSTRAN and DB2ACCT,
build PDB.CICS from PDB.ASUMUOW.
JCLSPCPY - Copies these datasets into PDB library:
ASUMCACH CICS
ASUMUOW ASUMDB:
27. SAS Paper
Configuration and Tuning Guidelines for SAS®9 in the
Microsoft Windows Server 2008 Operating Environment
The monitoring tool that SAS recommends is the Performance Monitor
(PerfMon) tool from Microsoft that comes with the Windows operating
system. Hundreds of performance counters are associated with this tool,
but from a SAS perspective, only the following subset of PerfMon objects
and their associated counters is of primary interest:
Cache
– o Data Flush Pages/sec
– o Dirty Pages
– o Fast Read Not Possible/sec
– o Fast Read Resource Misses/sec
– o Lazy Write Pages/sec
– o Read Aheads/sec
Client-Side Caching
– o Application Bytes Read from Cache
– o Application Bytes Read from Server (Not Cached)
And Many more in the paper at:
http://support.sas.com/resources/papers/
WindowsServer2008ConfigurationandTuning.pdf
Questions
• Contact Info
– www.mxg.com
– support@mxg.com
– Listserv MXG-L
– 214-351-1966 x7