OS/390 & ISV RESVOL
(Installation and) Maintenance
Strategies
Putting It All Together
on a RESVOL-set
Session 2829 - SHARE 93 Technical Conference
Steve Schunk
Schunk and Associates, Inc. (SNK)
s.schunk@schunk-associates.com
www.schunk-associates.com
Copyright 1999 Schunk and Associates, Inc. (Permission granted to SHARE, Inc. to distribute.)
08/1999 - Page 1
Disclaimer:
This material is provided for your information, critical analysis and
discussion. You need to evaluate whether the methods and
techniques presented are useful and appropriate for your particular
installation and operation environment.
Trademarks:
IBM (International Business Machines Corporation):
OS/390, MVS/SP, MVS/XA, MVS/ESA, ServerPac, CBIPO, CBPDO, CICS, DB2.
CA (Computer Associates International, Inc.):
CA-APCDOC, CA-1, CA-7, CA-11, CA-90s, CA-Dispatch, CA-DNAM/TLMS, CA-EARL,
CA-Easytrieve, CA-Examine, CA-Intertest, CA-JARS, CA-JCLCheck, CA-LIBRARIAN,
CA-Mutil-Image Manager (MIM), CA-Optimizer/II, CA-Optimizer, CA-Panvalet,
CA-Scheduler, CA-View, CA-Ramis, CA-NetMan.
Team Contributions:
K. Tomiak, H. Sherman, G. Nikovits, B. Sheinin, J. Perlman
08/1999 - Page 2
Agenda
Objectives for this Session
This is a Systems Management Session
Definition of Terms
Goals, Background and Assumptions
Implementation
Summary
Examples
Questions
08/1999 - Page 3
Why are you here?
Are you managing more OS/390 images?
Has the number of installed and supported ISV products in your
shop increased?
Has the IT technical support staff remained at the same (or
reduced) levels?
Are you being asked to do more with less?
Do you repeatedly install the same ISV product only for deployment
on multiple images?
Were the last ISV software deployments less than perfect?
Have you been asked to install or maintain an ISV product with little
notice?
Is Disaster Recovery or Contingency Planning an important
function?
08/1999 - Page 4
Definition of Terms
ISV:
An acronym for Independent Software Vendor, a.k.a. OEM (Original
Equipment Manufacturer), or other non-IBM system software.
RESVOL-set:
Used to logically reference the collection of physical DASD volumes that
collectively contain the OS/390 operating libraries. Sometimes called
the extended RESVOLs.
OS:
Generic term for complete Operating System - OS/390 with all its
elements and ISV products.
VANILLA:
Unmodified code delivered by a software vendor.
BASE:
The starting point for a product's related target and distribution libraries,
where the vanilla code is installed (both SMP/E and non-SMP/E
methods). The base is never modified by the customer.
08/1999 - Page 5
Library Types
Target:
Contains executable load modules and ISPF libraries.
Parm/Control:
Parameter and other statements providing control data for regular
production operation (i.e., PARMLIB, PPOPTION).
Source:
Macros used to specify operating (run-time) information that are
assembled and/or linked into modules and load modules.
Run-Time Data:
Used in normal production operation (like CA-TMS TMC,
Netview/AOC Logs, HSM CDS, XCF CDS, APPC SI and TP
cluster, and various SMS control files).
08/1999 - Page 6
Goals: What We Want to Achieve
Gain process efficiencies (do more with less)
Skilled systems programming and technical resources are
limited
Reduce the total OS install and maintenance cycle time
Improve OS production deployment quality
Reduce inter-product errors among all OS/390 elements and
ISV products
Reduce intra-product errors among ISV product suites
Contained, IPLable system
A limited, contained DASD set for Disaster Recovery
08/1999 - Page 7
Goals: What We Want to Achieve
(continued)
Simplify ISV product installs
Build (install) vanilla product base only once
Design allows systems specific customization isolated from the
vanilla base
Base cloned out to other systems for production deployment
Supports SMP/E and non-SMP/E installation and maintenance
processes
Complete, integrated testing (on a dedicated SYSPROG
development system)
Simple, accurate cloning to other target systems from the base
08/1999 - Page 8
Goals: What We Want to Achieve (continued)
Simplify ISV customization process
Allow distinct, separate product customization by target system
Select and use standard options (whenever possible)
Customized system specific libraries are concatenated before
the vanilla base
Integrated component testing (to improve quality)
ISV products thoroughly tested with all OS/390 elements to
make sure they work and play well with others
08/1999 - Page 9
CONS - the downside considerations
Staff coordination for installation, customization, test and
deployment is increased
Technicians must know ISV product function and operation
Technicians must be proficient with SMP/E and non-SMP/E
processes:
Standard RECEIVE, APPLY and ACCEPT
USERMOD builds
ZONE EDIT, JCLIN, UCLIN
DASD requirements may be increased
A profound culture change for the technical staff is likely
Some ISV products are not sufficiently critical to business to be
included in this process standardization
Reevaluation of the "emergency service" process is suggested
08/1999 - Page 10
Assumptions
OS/390 was installed using ServerPac (modified system
replacement)
Operating target datasets now span more than 1 RESVOL
System symbolics are being used in the target dataset catalog
entries (&SYSR1)
A defined deployment process currently exists
The technical support group owns the complete software
installation process
08/1999 - Page 11
Implementation Strategy Recommendations
Define (review - adjust) dataset naming conventions
No system specific identification is included in a HLQ
Consider a single, unique HLQ for ISV product dataset names
(Example: ISV1 - similar to SYS1)
Use or incorporate vendor supplied DSN HLQ whenever
possible (Example: ISV1.CAI...)
After selecting a convention, build or adjust GLOBAL, TARGET
and DLIB DDDEF entries
Create batch ZONE EDITs from JCL supplied with the
product
DDDEF VOLSER entry uses the hard-coded (not symbolic)
volume serial
Catalog entries in the MCAT
Review and validate security constructs, and access rules
Consider publishing a "SYSPROG" User Guide
08/1999 - Page 12
Implementation Strategy Recommend (continued)
Get more DASD to extend the RESVOL-set
Need sufficient DASD space to contain all ISV products
Recommendations:
1 additional volume per RESVOL-set for target libraries and
SMP/E zone
1 DLIB volume
1 Global volume
Use the same volume naming convention as the existing
RESVOLs by adding a volume sequence identification
(more on naming conventions in a moment...)
08/1999 - Page 13
Implementation Strategy Recommend (continued)
Review and adjust system parameters
Define and adjust system symbolic parameters:
&SYSR1, &SYSR2, &SYSR3...
Master Catalog Entries:
DEF NVSAM(NAME(isv1.product.linklib) VOL(&SYSRSn) DEVT(0000))
Review SYS1.IPLPARM(LOADxx)
Validate the IEASYM statement
SYS1.PARMLIB (IEASYMxx) and (IEASYM00)
Define additional system symbol variables to suit your needs
Segregate library types
Identify libraries that will be modified and determine the correct
volume placement (usually OFF RESVOL)
08/1999 - Page 14
Rules of the Road
BE WARNED: If you don't follow them, you WILL have a
serious accident.
RESVOLs are "Read-only" and contain only:
Target operating libraries, and
SMP/E target zones (CSI) and their related SMP/E control files
Target libraries residing on any volume in the RESVOL-set are
cloned
Read-only: No datasets are updated during normal system
operation
Backup process usually captures CSI and related target libraries
together
Consolidated SMP/E CSIs are not used
Global and DLIB zones need to be stationary, and are not
cloned
08/1999 - Page 15
Rules of the Road (continued)
Use DDDEFs for ALL SMP/E supported products
No instream DD statements allowed in SMP/E process JCL
APPLY, ACCEPT, and BUILDMCS function considerations
Always want SMP/E to select the correct target library copy
SMP/E datasets are cataloged in a UCAT
Allows access and use among systems for query purposes
APPLY and ACCEPT are done only from the driver system
Customized modules MUST be segregated to system specific
libraries
Parm/Control (manual text) go to the CATVOL or UTLVOL,
Target (UMOD, assemble or link process) on the RESVOL-set,
Data (end-user or operating) on other data volumes.
08/1999 - Page 16
Rules of the Road (continued)
Customized modules - library placement example:
ISV1.CAI.sysid.CAILIB
= RESVOL
ISV1.CAI.CAILIB
= RESVOL
ISV1.CAI.sysid.PPOPTION
= CATVOL
userdept.CAI.sysid.CA7.CALENDAR
= UTLVOL
08/1999 - Page 17
Naming Conventions and Examples
Purpose: Common understanding only - "Red means Stop", "Green
means Go". Examples are shown for illustrative purposes - choose what
is meaningful for your environment!
RESVOL (DASD VOLSER convention for RESVOL-set)
scRSlq
sc = 2 letter &SYSCLONE value (also used in PARMLIB member suffix)
RS = constant "RS" for (RESVOL-set)
l = single numeric maintenance level indicator (1, 2, 3, ...)
q = single alphabetic volume sequence (A, B, C, ...)
RSscl#
RS = constant "RS" (for RESVOL-set)
sc = 2 letter &SYSCLONE value
l = single alphabetic maintenance "level" indicator
# = single number volume sequence (1, 2, 3,...)
08/1999 - Page 18
Naming Conventions and Examples (continued)
SMP/E Environment:
CSI DSNs:
SMPE.vendor.[product.]Tresvol.CSI
ZONENAMEs:
Tresvol (Target), Dvolser (Distribution), GLOBAL (Global Zone)
(lower case indicates a variable)
vendor: 3 to 8 character vendor name or acronym
product: 3 to 8 character sub-product name or acronym
Whenever possible, use dataset names supplied by the vendor.
The SMP/E CSI DSN contains the SMP/E zonename providing an
internal to external link.
08/1999 - Page 19
Naming Conventions and Examples (continued)
SMP/E CSI and Related Dataset Placement:
Target
RESVOLn:
CSI, SMPLDS, SMPLOG, SMPLOGA,
SMPMTS, SMPSCDS, SMPSTS.
DLIB
DLIB VOLn:
CSI, SMPLOG, SMPLOGA, and product
DLIB datasets.
Global
GLOBAL VOL:
CSI, SMPLOG, SMPLOGA, SMPPTS (TLIB),
perhaps RELFILEs.
Objective:
Related SMP/E datasets kept on the same volume as the CSI,
Target and DLIB CSI's on a volume with related target or DLIB files
HFS DFDSS Backup:
Is placed on the last ServerPac RESVOL..
08/1999 - Page 20
CAI Product Examples
This vendor selected for concept illustration only because:
most products are installed using SMP/E
most products are easily contained in a single or multiple
GLOBAL,
operating options are selected using different methods
(assembled loadmods, PARMLIB)
(We do not offer or imply any endorsement, positive or negative, nor have
these techniques been approved or disapproved by CAI.)
08/1999 - Page 21
CAI Product Examples (continued)
Standard target library set (HLQ: ISV1.CAI. or CAI.)
...CAICLIB
...CAIHELP
...CAIISPL
...CAIISPM
...CAIISPP
...CAIISPS
...CAIISPT
...CAIOLIBE
...CAILIB
...CAILPA
...CAIMAC
...CAISRC
...CAIPROC
...CAICICS
...SAMPJCL
...PPOPTION
TSO CLIST
TSO Help
ISPF Load
ISPF Menu
ISPF Panel
ISPF Skeleton
ISPF Table
SAR Panels & JCL Skeletons
(Utility to Load into SAR DB)
LINKlib
LPAlib
Macro
Source
PROCLIB
CICS Loadlib
Sample JCL & Installation Procedures
PARMLIB
08/1999 - Page 22
CAI Product Examples (continued)
System specific target library set (HLQ: CAI.sysid)
...CAILIB
...CAIPROC
...CAISRC
...PPOPTION
Not necessarily limited to these, usage determines volume
placement.
08/1999 - Page 23
CAI Product Examples (continued)
DSN Samples (with exceptions):
HLQ:
Panvalet
Optimizer
Ramis
ISV1.CAI.**
ISV1.CAI.PAN.**
ISV1.CAI.OPT.**
ISV1.CAI.RAM.**
Panvalet
Optimizer
Ramis
SMPE.CAI.GLOBAL.CSI
SMPE.CAI.PAN.resvol.CSI
SMPE.CAI.OPT.resvol.CSI
SMPE.CAI.RAM.resvol.CSI
SMP/E:
for target and DLIBs
tzn: Presvol
tzn: Oresvol
tzn: Rresvol
lower case indicates
variable
08/1999 - Page 24
Implementation Checklist
1. Select and prioritize ISV products for rebuild or reinstall.
For each product or vendor product suite:
Review vendor installation documentation
Select ISV products to combine or separate
Review USERMOD application and other customization techniques
2.
3.
4.
5.
6.
7.
8.
Establish additional RESVOL(s) - minimally 1 to start.
Establish additional DLIBVOL(s).
Verify GLOBAL volume free space.
Establish (or adjust) dataset naming conventions.
Validate security.
Allocate vanilla base libraries for the primary install.
Allocate system specific (SYSID) target libraries on appropriate
volume(s) to accommodate system specific customization.
9. Segregate SMP/E CSIs.
10. Update SMP/E target and DLIB zone DDDEFs.
11. Update system cloning procedure for additional RESVOLs.
08/1999 - Page 25
Implementation Checklist (continued)
12. Update (alter) master catalog entries for all target libraries:
Add new convention names using system symbolic VOLSER (&SYSRS3),
Delete old, existing name (only after new setup if fully deployed).
13. Update SYS1.PARMLIB (PROGxx) entries - DSN & VOLSER
parameters:
Remove duplicate entries for APF authorization,
Order/Pair system specific libraries before vanilla,
Add VOLSER to use appropriate system symbolic (like &SYSRS3).
14. Review and update all existing DSNs for ISPF allocation CLISTs.
15. Review and update target DSNs in system started procedures (STCs).
16. Test and correct, and re-test.
08/1999 - Page 26
Summary
Know and understand your ISV product and its associated library
types
Develop or enhance DSN conventions
Only vanilla code is allocated on the RESVOL-set
Modified code must be located off the RESVOL-set, or installed in
system specific libraries
Keep associated SMP/E CSIs (and related control datasets)
together with the libraries they represent
Think logical container
Develop scheduled release cycles
Plan for emergency service process
08/1999 - Page 27
EXAMPLES
08/1999 - Page 28
BEFORE
Dataset Copy #3
ISV-VOL#1
Product "A"
ISV-VOL#1
Product "A"
ISV-VOL#2
Product "B"
Product "B" Reinstall #2
ISV-VOL#3
Product "C"
Datatset
Copy#1
ISV-VOL#2
Product "B"
ISV-VOL#3
Product "C"
RES-VOL#2
RES-VOL#2
OS/390 "CLONE"
RES-VOL#1
RES-VOL#1
08/1999 - Page 29
(TRANSITION)
CAT-VOL
ISV-VOL#1
Product "A"
UTL-VOL
ISV-VOL#2
Product "B"
RES-VOL#2
RES-VOL#1
ISV "CLONE"
OS/390
"CLONE"
RES-VOL#4
Product "C" & "D"
RES-VOL#3
Product "A" & "B"
RES-VOL#2
RES-VOL#1
08/1999 - Page 30
AFTER
CAT-VOL
RES-VOL#4
Product "C" &
"D"
UTL-VOL
RES-VOL#3
Product "A" & "B"
RES-VOL#4
Product "C" &
"D"
RES-VOL#3
Product "A" & "B"
OS/390 "CLONE"
RES-VOL#2
RES-VOL#2
RES-VOL#1
RES-VOL#1
08/1999 - Page 31
The System Volume - Logical Container
RESVOL: Read-only target operating libraries (RESVOL-Set)
CATVOL: MCAT, with minimal page datasets, JES2 Spool, and
critical customized product libraries (ISPF, SDSF) for emergency use
IPLVOL: Required run-time data files for major subsystems (i.e.,
TMC, HSM CDS)
HFSVOL: Operating libraries for UNIX System Services
UTLVOL: Unique product (or subsystem) data files that are system
specific -- utility
08/1999 - Page 32
USERMOD Customization Example
++USERMOD(UMTLMS1) REWORK(9902182)
/*
LIB: CAIBASE.USERMODS(UMTLMS1)
DOC: THIS USERMOD CREATES 2 CA-TLMS V5.4 OPTION LMODS (TLTPOPTS)
FOR SEPARATE SYSTEMS.
*/ .
++VER (Z038)
FMID(CD15400).
++JCLIN
.
//UMTLMS1 JOB
//*
//LKED1
EXEC PGM=IEWL,PARM='LIST,XREF,LET,NORENT'
//SYSPRINT DD SYSOUT=*
//CDO53LLD DD DISP=SHR,DSN=CAI.OMS.CDO53LLD,DISP=SHR
//SYSLMOD DD DISP=SHR,DSN=CAIBASE.CAILIB(TLTPOPT1),DISP=SHR
//SYSLIN
DD *
INCLUDE CDO53LLD(TLTPOPT1)
ENTRY
TLTPOPTS
NAME
TLTPOPT1(R)
/*
08/1999 - Page 33
USERMOD Customization Example (continued)
//LKED2
//SYSPRINT
//CDO53LLD
//SYSLMOD
//SYSLIN
INCLUDE
ENTRY
NAME
/*
//*
EXEC PGM=IEWL,PARM='LIST,XREF,LET,NORENT'
DD SYSOUT=*
DD DISP=SHR,DSN=CAI.OMS.CDO53LLD,DISP=SHR
DD DISP=SHR,DSN=CAIBASE.CAILIB(TLTPOPT2),DISP=SHR
DD *
CDO53LLD(TLTPOPT2)
TLTPOPTS
TLTPOPT2(R)
08/1999 - Page 34
USERMOD Customization Example (continued)
++SRC(TLTPOPT1)
DISTLIB(CDO53SLD)
DISTMOD(CDO53LLD) .
*
*
CA-TLMS OPTIONS FOR SYSTEM #1
*
TLTPOPTS START
CAIDTOPT USER=DEFAULT,PRODUCT=TLMS,
PFKEYS=(HELP,,END,RETURN,,,BACKWARD,FORWARD,,,,CANCEL)
CAIDTOPT USER=LIBRMGR,PASSWRD=MGR,ACCESS=UPDATE
CAIDTOPT USER=LIBR1,PASSWRD=TAPE1,ACCESS=UPDATE
CAIDTOPT USER=LIBR2,PASSWRD=TAPE2,ACCESS=UPDATE
CAIDTOPT USER=LIBR3,PASSWRD=TAPE3,ACCESS=UPDATE
CAIDTOPT USER=APPL1,PASSWRD=APPL1,ACCESS=INQUIRY
CAIDTOPT USER=APPL2,PASSWRD=APPL2,ACCESS=INQUIRY
CAIDTOPT USER=APPL3,PASSWRD=APPL3,ACCESS=INQUIRY
CAIDTOPT USER=PC01AN,PASSWRD=APPL1,ACCESS=INQUIRY
CAIDTOPT USER=TS01LW,PASSWRD=GOD2,ACCESS=UPDATE
CAIDTOPT USER=INQR,PASSWRD=INQR,ACCESS=INQUIRY
CAIDTOPT
END
X
08/1999 - Page 35
USERMOD Customization Example (continued)
++SRC(TLTPOPT2)
DISTLIB(CDO53SLD)
DISTMOD(CDO53LLD) .
*
*
CA-TLMS OPTIONS FOR SYSTEM #2
*
TLTPOPTS START
CAIDTOPT USER=DEFAULT,PRODUCT=TLMS,
PFKEYS=(HELP,,END,RETURN,,,BACKWARD,FORWARD,,,,CANCEL)
CAIDTOPT USER=TAPELB1,PASSWRD=LIBRARY1,ACCESS=UPDATE
CAIDTOPT USER=TAPELB2,PASSWRD=LIBRARY2,ACCESS=UPDATE
CAIDTOPT USER=MANAGER,PASSWRD=CSCUPDT,ACCESS=UPDATE
CAIDTOPT USER=INQR,PASSWRD=INQR,ACCESS=INQUIRY
CAIDTOPT
END
X
08/1999 - Page 36
USERMOD Customization Example (continued)
//*
//*
//SMPE1
//SMPPTFIN
//SMPCNTL
SET
RECEIVE AND APPLY TLMS CUSTOM USERMODS
INVOKE CA-PRODUCT SMP PROCEDURE
EXEC CAISMPEU
DD DISP=SHR,DSN=CAIBASE.USERMODS(UMTLMS1)
DD *
BDY
(GLOBAL).
RECEIVE S
(UMTLMS1)
LIST
SYSMODS.
RESETRC .
SET
BDY
(CAIBASE).
APPLY
S
(UMTLMS1)
/*
CHECK
*/
/*
REDO
*/
ASSEM.
/*
*
08/1999 - Page 37
USERMOD Customization Example (continued)
//*
//*
//
//STEP2
//SYSPRINT
//IPFILE
//OPFILE1
//OPFILE2
//SYSIN
COPYMOD
SELECT
COPYMOD
SELECT
/*
//*
COMMENT JCL TERMINATOR (//) FOR APPLY
EXEC PGM=IEBCOPY
DD SYSOUT=*
DD DISP=SHR,DSN=CAIBASE.CAILIB
DD DISP=SHR,DSN=CAIBASE.MVS1.CAILIB
DD DISP=SHR,DSN=CAIBASE.MVS2.CAILIB
DD *
I=((IPFILE,R)),O=OPFILE1,LIST=YES
M=((TLTPOPT1,TLTPOPTS,R))
I=((IPFILE,R)),O=OPFILE2,LIST=YES
M=((TLTPOPT2,TLTPOPTS,R))
I/P COMMON LIB
O/P SYSTEM #1
O/P SYSTEM #2
08/1999 - Page 38
PARMLIB Customization Example
SYS1.IPLPARM(LOADP1)
************************************************************************
*---+----1----+----2----+----3----+----4----+----5----+----6----+----7-IODF
13 SYS3
OS390
IEASYM
(P1,00,L)
NUCLST
P1
NUCLEUS 1
SYSCAT
CTLGP1113CCATALOG.MASTER.NPP1
SYSPLEX PLEXP
PARMLIB SYS1.PARMLIB
PARMLIB SYS1.IBM.PARMLIB
SYSPARM 00
SYS1.PARMLIB(IEASYMP1)
/*----------------------HISTORY--------------------------------------*/
SYSDEF SYSNAME(NPP1)
SYMDEF(&NPJESPLX='JPLX1')
SYMDEF(&HSMID='E')
SYMDEF(&HSMASTER='Y')
SYMDEF(&NPSUBA='02')
08/1999 - Page 39
PARMLIB Customization Example (continued)
SYS1.PARMLIB(IEASYM00)
/*-------------------------------------------------------------------*/
SYSDEF SYMDEF(&NPSYSID='&SYSNAME(-1:1)')
SYMDEF(&NPSUFFIX='&SYSNAME(-2:2)')
SYMDEF(&NPSYSPLX='&SYSPLEX')
SYMDEF(&SYSR2='&SYSR1(1:3)2&SYSR1(-2:2)')
SYMDEF(&SYSR3='&SYSR1(1:3)3&SYSR1(-2:2)')
SYMDEF(&SYSR4='&SYSR1(1:3)4&SYSR1(-2:2)')
SYMDEF(&RESVSET='&SYSR1(1:3)&SYSR1(-2:2)')
SYMDEF(&RESVTGT='T&SYSR1(1:3)&SYSR1(-2:2)')
SYSDEF LPARNAME(PRBLP1)
SYMDEF(&NPLPAR='B1')
SYSDEF LPARNAME(PRBLP2)
SYMDEF(&NPLPAR='B2')
SYSDEF LPARNAME(PRBLP3)
SYMDEF(&NPLPAR='B3')
/*-------------------------------------------------------------------*/
08/1999 - Page 40