SYSTEM
STRATEGIES
BY TOM BRYANT
MVS/XA/ESA Problem
Solving: Part III —
Advanced SVCDUMP
Analysis
This article will take us
beyond the “one-page
MVS/XA/abend analysis”
to gather the relevant
information and quickly
determine where to go
for further help with
problems other than the
simple abend.
T
o recap this series so far, I’ve examined using IPCS for MVS/ESA
Version 5 (Part I, Technical Support, October 1996); and Part II
(November 1996) examined a technique called one-page
MVS/XA/ESA abend analysis. This article will take us beyond
the “one-page MVS/XA/abend analysis” to gather the relevant information
and quickly determine where to go for further help with problems other than
the simple abend. Future articles will examine slip traps/GTF tracing and
standalone dump analysis.
Once upon a time, I believed that
MVS/XA/ESA debugging (what I referred to
it as then) was about nailing down the exact
problem to the instruction level. With the
increasing complexity of MVS/ESA Version 5
and the many other platforms supported,
this methodology had to change. This article
will examine:
■ displaying/correcting problems with
system contention;
■ how to know when you must use
the MVS/ESA FORCE command;
■ how to dump an address space
and get just the data you need;
■ the two types of MVS/ESA SVCDUMPs
and how to tell them apart;
■ SVCDUMP structures (ASCB, ASXB,
TCB, RB, CDE, LLE, and XL);
■ the IBM contents directory entries (CDE)
you should know;
■ using the trace table when you are ready
for it; and
■ a case study.
IDENTIFYING SYSTEMS
CONTENTION CONDITIONS
The essence of MVS/ESA problem solving
is identifying system contention conditions.
If you remember nothing else from this article,
TECHNICAL SUPPORT DECEMBER 1996
I ask that you commit to memory the commands to list system contention. The payback
will be immense. You will save a great deal of
time solving problems and avoid that churning
feeling in your stomach when a problem does
occur. The most important command to know
is the one that lets you display MVS/ESA
ENQ contention. In IBM systems with Global
Resource Serialization (GRS) or equivalent
third-party software, this command is the
DISPLAY GRS,CONTENTION or D GRS,C.
The normal output of this command should be
the following two messages:
NO ENQ RESOURCE CONTENTION EXISTS
NO LATCH CONTENTION EXISTS
If you see these two messages, you can relax.
Besides ENQ contention, GRS will also
report any latch contention, if found.
The GRS latch manager is new with
MVS/ESA Version 5 and is used to serialize
resources within an address space. The GRS
latch manager provides a more specific
resource serialization mechanism and incurs
much less overhead for address space serialization than using the MVS/ESA LOCAL
LOCK. Many of the new MVS/ESA Version 5
components, such as Workload Manager, use
GRS latches for low-overhead serialization.
SYSTEM STRATEGIES
Figure 1: D GRS,C Output
D GRS,C
ISG020I 10.03.18 GRS STATUS 169
S=SYSTEM SYSVTOC TEMP02
SYSNAME
JOBNAME ASID
LPAR1
USER1
0041
LPAR2
USER2
003D
NO LATCH CONTENTION EXISTS
TCBADDR EXC/SHR OWN/WAIT
008BAD90 EXCLUSIVE OWN
008F69D8 EXCLUSIVE WAIT
Figure 2: Output Message From Issuing the D GRS,RES=???? Command
D GRS,DEV=????
ISG020I 12.10.33 GRS STATUS 404
DEVICE:???? VOLUME:TEMP02 RESERVED BY SYSTEM LPAR1
NO RESERVE RESOURCE REQUEST EXISTS
I have noticed that the GRS latch names
a re hyphenated (i.e., SRM_SA_SAMPLING_LATCH_SET), which should make
them easier to spot in the D GRS,C display.
Many times you will issue the D GRS,C
command and see an ENQ (or RESERVE)
lockout. If you have a VTOC RESERVE
condition, the D GRS,C output shown in
Figure 1 will be displayed. In this situation,
the owning user has the resource and the other
users wait. In addition, because a hardware
reserve is placed on the volume, on LPAR2
you should see IOS071I start pending messages
which will display the DASD device and the
JOBNAME of the user waiting for that device.
In MVS/ESA Version 5 there is a wonderful
new command called D GRS,DEV=????
(where ???? is the device address) that shows
the system issuing the hardware reserve. We
could have issued that command knowing
the SYSVTOC ENQ name by D
GRS,RES=(SYSVTOC,*). Our output message
would look like Figure 2.
Once you know the reserving system and
the system receiving the action, you can take
the appropriate steps without having various
jobs (or the system itself) in strife. The onl y
caveat to the D GRS,RES=???? command is
that it must be issued on the reserving system.
If you are running in parallel sysplex mode,
just issue the command as:
RO *ALL,D GRS,RES=????
and it will be routed to every system in
the sysplex.
When you issue the D GRS,C command it
will take some time before you are familiar
with all of the jobnames in the system and
what they do. If you have a conflict with
multiple instances of the master address space
ASID 1 (*MASTER*), you may have a master
catalog conflict or another situation in which
the master address space had to get involved.
In this case, check for outstanding operator
reply messages. If you have more than one
instance of the dump services jobname
DUMPSRV, this means that one SVCDUMP
is in progress and another one is waiting.
Later, you may want to examine EREP to
make sure your dump is one of the oldest
problems observed.
If the catalog address space (jobname of
CATALOG) ever appears when you issue the
D GRS,C command, you should immediately
issue a modify catalog,list (F CATALOG,LIST)
command to see if there is any contention in
the catalog address space. The IEC347I
message will list jobnames, stepnames and
flag fields to indicate the status of the job
while its catalog entries are buffered in the
CATALOG address space. The flag entries are:
■ O - OLDEST: oldest task in catalog
address space;
■ W - WAIT: wait for dataset enqueue
or tape mount;
■ A - ABEND: task is abending;
■ E - ENQ: dataset enqueue; and
■ R - RECALL: task is suspended.
The OLDEST indicator provides a timeframe
as to the oldest jobname in the CATALOG
address space. The WAIT and ENQ flags are
self explanatory. The RECALL indicator is
the most misleading flag; this is not a DFHSM
recall request but rather it indicates the task
was suspended and should be further examined
with other MVS/ESA display commands
(such as D A,jobname).
There is a handy ID field in the IEC347I
message output that can be used by a F CATALOG command to selectively abend requests
hung up in catalog processing. Make sure to
spend at least a few minutes and look over
the situation before issuing the F CATALOG,ABEND(??) command (where ?? is the
task ID).
Another helpful command is the modify
catalog,open command (F CATALOG,OPEN)
which will display all open catalogs and
important indicators such as:
■ Y/N: allocated to catalog;
■ S: SMS-managed catalog;
■ V: catalog is under VLF caching;
■
■
■
■
■
I: ISC;
C: catalog was closed;
D: catalog was deleted;
R: catalog is shared; and
A:ALT.
A very useful, and potentially dangerous,
command is the modify catalog, restart (F
CATALOG,RESTART). This command will
restart the CATALOG address space if internal
catalog pointers become corrupted. I’ve had
only 50/50 luck with this command. If you see
a message from the F CATALOG,OPEN
about DAMAGE TO THE CATALOG
CAWXA CHAIN (the catalog auxiliary work
area), do not issue the F CATALOG,RESTART?
command. The catalog auxiliary workarea is
one of the few catalog structures that can’t be
dynamically rebuilt in the event of a problem
(since it is the list of catalogs you see from the
F CATALOG,OPEN).
H ave you ever issued the MVS/ESA
CANCEL command when a batch job or TSO
user locked up? The IEE301I jobname
CANCEL COMMAND ACCEPTED message
appears but the job does not leave the system!
In this situat i o n , p e r fo rm a display
active,jobname (D A,jobname) and look at
the SMC keyword value in your output. If you
see a value greater than SMC=000 (e.g.,
SMC=001), then the job you are cancelling is
running in a special system state called stepmust-complete. See Figure 3.
USING THE MVS FORCE COMMAND
The authorized version of the ENQ macro
has a field called SMC which refers to the
step-must-complete state. If you wish to get
rid of this job (and risk creating partially-freed
resources), use the MVS FORCE command
with the ARM (Address Space Resource
Manager) parameter. Using FORCE
U=tsouser,ARM will drive the normal task
termination routines without causing address
space destruction. To use FORCE ARM the
cancel command must have been previously
issued. If FORCE ARM doesn’t do it, then
you must issue the FORCE U=tsouser which
terminates the address space (the final step of
address space termination occurs in the master
address space ASID 1). See Figure 4.
Once a FORCE has been issued for an
address space , this does not absolutely guarantee that the address space is gone. It only
guarantees that the address space identifier is
removed from the dispatching queue (nothing
more will run there). Additional recovery
termination management (RTM) processing
occurs in the master address space (ASID 1),
and if things get hung up there, a TSO user may
not be able to logon again. The system will
think the user is active, when in fact, he is not.
IBM gave me a nifty little program that
will issue the CALLRTM macro which allows
SYSTEM STRATEGIES
Figure 3: Display Active With a Non-Zero ENQ SMC Value
D A,TSOUSER
IEE115I 14.55.03 96.131 ACTIVITY 633
JOBS
M/S TS USERS
SYSAS
INITS
ACTIVE/MAX VTAM
00015
00085 00148
00023
00050
00148/00300
TSOUSER IN PR A=00C8 PER=YES SMC=001 PGN=N/A DMN=N/A
CT=099.775S ET=05.07.18
WUID=TSU21757
WKL=TSO
SCL=TSO
P=1
SYSTEM ID PT11 DATE 5/10/96
96.131
LINE
90,313
RGP=N/A
SRVR=NO QSC=NO
ADDR SPACE ASTE=1FD20200
Figure 4: FORCE Out a TSO User With Address Space Destruction
FORCE U=TSOUSER
IEE301I TSOUSER
FORCE COMMAND ACCEPTED
*ISG033E UNIDENTIFIED TASK JS FAILED WHILE IN MUST COMPLETE STATUS
ISG032E RESOURCE NAMED BLXDAS, Q % MAY BE DAMAGED
ISG032E RESOURCE NAMED SYSIKJUA,TSOUSER MAY BE DAMAGED
ISG032E RESOURCE NAMED SPFEDIT,TSOUSER.ISPPROF MAY BE DAMAGED
ISG032E RESOURCE NAMED SPFEDIT,TSOUSER.ISPPROF MAY BE DAMAGED
ISG032E RESOURCE NAMED SPFEDIT,TSOUSER.ISPPROF MAY BE DAMAGED
ISG032E RESOURCE NAMED SYSDSN,TSOUSER.SPFTEMP1.CNTL MAY BE DAMAGED
IST804I CLOSE IN PROGRESS FOR TSO0012 OPENED BY TSOUSER
ISG032E RESOURCE NAMED SPFEDIT,TSOUSER.ISPPROF MAY BE DAMAGED
IST400I TERMINATION IN PROGRESS FOR APPLID TSO0012
ISG032E RESOURCE NAMED SPFEDIT,TSOUSER.ISPPROF MAY BE DAMAGED
$HASP310 TSOUSER TERMINATED AT END OF MEMORY
IEF743I IKJACCNT.TSOUSER FORCED - CODE SA22 IN ADDRESS SPACE 00C8
IST805I VTAM CLOSE COMPLETE FOR TSO0012
$HASP250 TSOUSER PURGED — (JOB KEY WAS ACD86C3F)
me to select a TCB for recovery termination.
TSO uses the CALLRTM macro when you
reply FSTOP to the TSO termination message.
This program can be beneficial, but it can
also bring your system down in a heartbeat if
used improperly. See IBM APAR II05723 for
more details.
CONSOLE DUMPS
There are certain circumstances where you
will need to take a console dump of one or
more address spaces for IBM or another vendor.
It’s important to make sure that you have all
the pertinent information in the dump. The
IBM defaults for the areas to be dumped are
on the lean side. To be sure, I thought it would
be best to write down the steps on how to
dump an address space. Let’s say we want to
dump the address space of the batch job called
HOGJOB. To do this, issue the MVS command
DUMP COMM=(HOGJOB NOT CANCELLING). The COMM value will be our
dump title and should reflect something
meaningful. MVS will prompt you for more
information with the following message:
*?? IEE094D SPECIFY OPERAND(S) FOR DUMP
COMMAND
where ?? is the outstanding reply number. We
wish to indicate that HOGJOB is to be
dumped and also want to specify our SDATA
dumped areas options. Since the SDSF line
may not be long enough, we reply as follows:
R ??,JOBNAME=HOGJOB,CONT
to make sure MVS will prompt us again with
the IEE094D message (the CONT parameter
does this). Our next reply to the IEE094D
message will tell MVS what areas we want
dumped. I usually reply as follows:
R
??,SDATA=(LPA,LSQA,PSA,RGN,SQA,SUM,TRT,GR
SQ,CSA)
which means I want just about every data area
except all of the DAT-on nucleus (which I
haven’t needed as yet). MVS will first capture
the dump in a dataspace and issue the message:
IEA794I SVC DUMP HAS CAPTURED:
DUMPID=??? REQUESTED BY JOB (*MASTER*)
DUMP TITLE=HOGJOB NOT CANCELLING
where ??? is a generated number for the dump
identifier. If you issue the DISPLAY
DUMP,TITLES (D D,T) command you
should see the dump in an appropriate
DUMP?? dataset (where ?? is from 00 to 99).
If you’re using MVS dynamic dump datasets,
then issue D D,T,AUTODSN=3 to list the last
three dynamic dump datasets created.
Identifying SVCDUMPs
This section examines the sch e d u l e d
(asynchronous) dump where a system routine
detects an abnormal situation that would
c o m p romise processing and sch e d u l e s
abnormal termination of the task via the
SDUMP BRANCH=YES or ABEND macro.
The current task and request block information in a scheduled dump is for the dump
task. The dump must be analyzed to find the
failing TCB and RB.
To find out what kind of dump you have,
execute the IPCS STATUS SYSTEM command. In the field PROGRAM REQUESTED
SVCDUMP you will see the name IEAVTSDT,
which indicates a scheduled SVCDUMP.
In most situations with a scheduled SVCDUMP you will have to look at the current
and/or error address space(s) followed by the
task control blocks (TCBs) and request blocks
(RBs) to get a clear indication of what was
going on. Remember, your job is to narrow
the scope of the problem and first determine
what vendor owns the code (not always IBM).
For an IBM problem, the next goal is to be
able to provide a good search argument so you
can narrow down the number of possibilities.
The IPCS CBSTAT and IPCS VERBEXIT
SRMDATA commands are good ways to
check the health indicators of our current
and/or error address space. The IPCS
CBSTAT command takes an ASCB address as
input. For example, you would enter IPCS
CBSTAT XXXXXX. (where XXXXXX is the
ASCB address (make sure to include the
period). You can find out if a address space is
swapped in or out, and the reasons why such
as input terminal wait (user hasn’t pressed
enter) or detected wait (e.g., ENQ or DFHSM
recall). You will also get information on
address space status such as if the address
space was forced out (S0A22 abend). The
ASCB fields ASCBJBNI (jobname field for
initiated programs) and ASCBJBNS (jobname
field for start/mount/logon) will indicate what
type of job you are looking at. If you look at
the eight bytes off of the address in ASCBJBNI
and see INIT, and look at ASCBJBNS and see
YOURJOB, then you know that the system
initiator just started job YOURJOB. I will
cover more on types of jobnames/recognition
of dumps later in this article.
The IPCS VERBEXIT SRMDATA command was primarily designed to let us check
health indicators for the System Resources
Manager (SRM) responsible for performance
management decisions in MVS/XA/ESA.
By using the IPCS VERBEXIT SRMDATA
command, we can see much more information
than by using the IPCS CBSTAT command
which predominantly checks indicators in
the ASCB. Since the output from IPCS
VERBEXIT SRMDATA is quite large, I just
perform a FIND command for ASID in column
two and continue the finds while I look at
the data. See macro IRAOUCB in
SYS1.MODGEN for more information on the
SRM user control block (OUCB) fields.
SYSTEM STRATEGIES
Once you are done looking at the address
space, it’s time to look over the task control
block (TCB) and request block (RB) structures
to see what was going on. Practice this first on
small problems so you can get a feel for what
types of work (and loaded programs) are
present. The IPCS SUMMARY KEYFIELD
command shows you key fields in the ASCB,
TCB and RBs. For the TCB you get the
completion code (COMP) where (for the most
part) any non-zero completion code should be
reviewed. The IPCS SUMMARY KEYFIELD
command will automatically issue the IPCS
CBSTAT command so you see TCB status
along with the TCB. See Figure 5.
Request blocks are simply the representation
of programs in stor age. There are some types
of RBs which need to be clarified. When you
execute your program, a program request
block (PRB) is created. On the other hand, if
MVS has to execute a program on your
behalf, a supervisor request block (SVRB) is
created. The RB WLIC field is very important,
since it shows the wait count and interrupt
code (which is usually, but not always, the
abend code). When the interrupt code (the IC
of WLIC) is from a PRB, your program issued
the SVC. When the IC is from a SVRB, the
system issued a SVC on your behalf.
Another type of RB is an interrupt request
block. This block is created by the create
interrupt request block (CIRB) authorized
macro. The CIRB creates an interrupt queue
element which is high priority work tacked on
to a TCB. I will provide an example of an IRB
problem later in the case study. The contents
directory entry (CDE) is the physical representation of a module, its entry point and use
count. If you see multiple copies of the same
CDE in a dump, you can be sure that the load
module does not have the re-entrant and
reusable (RENT,REUS) linkage editor indicators on. The CDE is pointed to by the
RBCDE field, which you will have to format
with IPCS CBFORMAT since it is not listed
in IPCS SUMMARY KEYFIELD. Two other
fields of note are the load list element (LLE),
which is used to control load and delete references to a load module, and the extent list
(XL), which contains the load module address
and size. I will not cover them further since I
have not had to use them very often.
When you execute the IPCS SUMMARY
KEYFIELD command , your RB information
is presented backwards. For a given TCB, if
you want the last (most recent) RB in the
dump you will go to the last RB for this TCB.
For the times you use IPCS SUMMARY
FORMAT (hopefully not too often due to the
amount of output) this order is the other way
around in that the most recent RB is presented
immediately after the TCB. I thought I was
going crazy until I realized this simple fact
and verified it with IBM Level 2. When using
Figure 5: IPCS SUMMARY KEYFIELD Output (A Section)
TCB: 008FE240
CMP. . . . . 00000000 PKF. . . . . 00
LMP. . . . FF
DSP. . . . . FF
TSFLG. . . 00
STAB. . . . 008FF6E0 NDSP. . . 00000000
JSCB. . . . 008FFDFC BITS . . . . 00000000 DAR. . . . 00
RTWA . . . 00000000 FBYT1. . . 00
Task non-dispatchability flags from TCBFLGS4:
Top RB is in a wait
Figure 6: Common IBM CDE Entries (Field FLCDE)
IEAVAR00 - Region Control Task RCT initialization and termination.
IEAVTSDT - SVC Dump Task.
IEFSD060 - Initiator Control Initialization.
IEESB605 - Job Scheduling routine. (If the next CDE NAME is an application name then this
jobname must be a started task.)
IEFIIC
- Initiator Interface Control (batch jobs only)
IKJEFT01 - Terminal Monitor Program Initialization routine (TSO usage only). If previous
cdename is IEFIIC then TSO was run in batch.
IKJEFT02 - Terminal Monitor Program Mainline routine (TSO usage only). You may see this name
twice due to parallel tmp!
IKJEFT09 - Terminal Monitor Program Second Level routine (TSO usage only). Unauthorized routines
only.
the IPCS SUMMARY KEYFIELD command,
only the first CDE is shown for a given RB in
the dump. If you want to see all of the CDEs for
a given RB then execute:
RUNCHAIN ADDRESS(XXXXXX.) NAME(CDE)
LINK(0) +
CHAIN(999) LENGTH(32)
EXEC((CBFORMAT X STRUCTURE(CDE)))
I would recommend getting used to IPCS
SUMMARY KEYFIELD, since it has less
output and is easier to use. If this is your
first time in an MVS/XA/ESA dump, the
amount of data presented even using IPCS
S U M M A RY KEYFIELD can be ove rwhelming. I have seen many first- and second-time dump readers looking over the
TCB/RB fields for CDE IEAVAR00. This is
the MVS/XA/ESA region control task program and in no way is where their problems
are. I have found that most dumps contain
TCB/RB information you need to be aware
of when locating your application or system
problem. See Figure 6. Once you learn these
names, it will be easier to determine the type
of job you are looking at.
Figure 7 presents an example of four
types of jobs and their CDE entries. The one
that is somewhat different is the batch job
running a DB2 command under the TSO
terminal monitor program. The documentation for IEAVAR00, IEAVTSDT, IEFSD060
and IEESB605 is presented in the
MVS/ESA Version 3 logic manuals. Logic
manuals were discontinued in MVS/ESA
Version 4 and higher (to my dismay), so if
you have access to earlier logic manuals,
keep them handy.
COMPONENT ANALYSIS
The last section I want to cover in this article
before the case study is component analysis.
The IPCS VERBEXIT TRACE command
or SYSTRACE TIME(LOCAL) for IPCS
Version 5 and higher will give you the instorage trace table. I find the trace table
invaluable when, for example, I know the area
where the problem is and want to see the SVC
instruction. From a SAS regression of many
trace tables, I have determined the frequency
of trace table entries and the ones you should
be able to recognize. See Figure 8.
In life there are always exceptions, and the
recovery event (RCVY) is one of them. A
RCVY PROG means that recovery termination
management phase 1 (RTM1) was entered for a
program check. RCVY PERC means that a
given recovery routine could not correct the
abend and is passing the error to the next recovery routine (percolation) from RTM1 to RTM2.
RCVY FRR means that RTM1 is invoking a
SETFRR macro instruction which places the
address of a recovery routine in the FRR stack.
RCVY ABT is a request for abnormal end-oftask by the CALLRTM TYPE=ABTERM
macro. If you’ve ever replied FSTOP to a P
TSO request and pulled the rug out from many
TSO users, then you initiated a CALLRTM
TYPE=ABTERM routine.
SYSTEM STRATEGIES
A CASE STUDY
Figure 7: Examples of IBM CDE Entries
This case study will hopefully pull together some of the concepts
covered in this article. This problem occurred on an MVS/ESA Version
4.2.0. system a few years ago, although the debugging techniques are still
valid. The symptoms occur when your co-worker gets a SVCDUMP
under his userid with the following SVCDUMP title:
Major CDE entries
for various types of work
in IPCS SUMMARY KEYFIELD order
BATCHJOB
————
TSOUSER
————
MYSTC
————
BATCHDB2
————
IEAVAR00
IEAVAR00
IEAVAR00
IEAVAR00
IEAVTSDT
IEAVTSDT
IEAVTSDT
IEAVTSDT
IEFSD060
IEFSD060
IEFSD060
IEFSD060
IEESB605
IEESB605
IEESB605
IEESB605
COMPID=5665-28502,ISSUER=IKJEFT05.
SDUMP FROM THE TMP ESTAE ROUTINE
—————————————————————————————
IEFIIC
IKJEFT01
TPX1
IEFIIC
DFSSBT00
IKJEFT02
TPXMON
IKJEFT01
DFSPCC30
IKJEFT09
TPXCONS
IKJEFT02
DFSRRC00
ISRUMC
TPXWRITR
IKJEFT09
ISPTASK
DSN
ISPMAIN
ISPF
Figure 8: Frequency Distribution of IBM Trace Table Entries
ENTRY PERCENT
————————SVC
SVCR
SSAR
SSRV
PC
PT
DSP
SRB
WAIT
I/O
PGM
SSCH
PR
CALL
EXT
EMS
CLKC
RCVY
SUSP
RSCH
EXPL
SSRB
18.5
18.3
13.1 (LOWER PERCENTAGE IN SADUMP)
12.9
8.0
5.4
4.8 (HIGHER PERCENTAGE IN SADUMP)
3.8
3.2
2.5
2.4
2.4
2.2
1.0
0.6
0.3
0.2
0.2
0.2
0.1
0.1
0.1
The first thing we want to do is find out what kind of SVCDUMP we
are dealing with. Issue the IPCS STATUS SYSTEM command and
look at the program requesting dump, which in this case is a SVCDUMP (a synchronous dump).
I will always issue a IPCS STATUS FAILDATA command to get the
failure data and the registers. See Figure 9. Since we have a synchronous
dump (SVCDUMP) you may think that all we do is start looking
backward from the address in register 14 to get a CSECT (the one in
error). Getting the CSECT moving back from register 14 may pr ovide
our answer. If you want to reduce the number of hits on a search, it’s
better to understand exactly what was going on. We issue the IPCS
SUMMARY KEYFIELD command and look over the output. See
Figure 10.
We have passed control synchronously (the SYNCH SVC X’0C’)
to a routine (PRB) with a IRB in front of it. Seeing IRBs in any
SVCDUMP is unusual enough. Where does the EPA (entry point
address) of the IRB point to? I got the IEAVAR05 name from the
MVS/ESA Version 3 logic manuals. IEAVAR05 is the TMP attention
exit routine. Figure 11 is the output of the SYNCH SVRB registers so
we can see what was passed. If you check the previous IPCS STATUS
FAILDATA REGISTERS output you will notice the SVRB registers
are identical. Now we want to work backward from register 14’s
address until we see a TSO CSECT eyecatcher. We list the storage
area 0260F430. from register 15 to get the module name. From Figure
12 we see the words “TMP EXIT ATTENTION ROUTINE”in the text.
You may also notice that register 11 in our IPCS STATUS FAILDATA
register area is the entry point address of the IKJEFT03 TMP attention exit routine. The actual fix for this was APAR OY33820.
This is not the only way to solve this problem. My intention in this
case study was to make sure the debugger is aware of all conditions
before searching. It is too easy to get confused in the search when all
that is needed is a little up-front analysis to save a great deal of time
later.
Part IV will examine SLIP traps and GTF tracing. You will need the
SLIP trap knowledge to get the right information in the dump or if the
system is not providing a dump. GTF tracing will allow you to see system events as they unfold. Please stay tuned! ts
Figure 9: Case Study Dump of IPCS STATUS FAILDATA Command
TIME OF ERROR INFORMATION
PSW: 078C0000 80000004 INSTRUCTION LENGTH: 02 INTERRUPT CODE: 0001
FAILING INSTRUCTION TEXT: 0000040C 00008113 5DF80000
REGISTERS 0-7
GR: 00000000 000084B8
AR: 00000000 00000000
000084D4 00008904 00008918
00000000 00000000 00000000
000084D0 005D1230 005FF510
00000000 00000000 00000000
REGISTERS 8-15
GR: 005E8AF8 00008770
AR: 00000000 00000000
000084D8 8260F530 00008438
00000000 00000000 00000000
00008438
00000000
RECOVERY ENVIRONMENT
RECOVERY ROUTINE TYPE: ESTAE RECOVERY ROUTINE
RECOVERY ROUTINE ENTRY POINT: 82611F58
AN IRB PRECEDED THE RB THAT IS ASSOCIATED WITH THIS TASK.
AN SVC DUMP WAS SCHEDULED BY A PREVIOUS RECOVERY ROUTINE.
8260FAC6 80000000
00000000 00000000
SYSTEM STRATEGIES
Figure 10: Case Study Dump of IPCS SUMMARY KEYFIELD Command
FROM IPCS SUMMARY KEYFIELD command:
TCB: 005F99F0
CMP . . . . . . . 940C1000
NDSP. . . . . . . 00000000
PKF. . . . . . . . 80
JSCB. . . . . . . 005EE6A4
LMP. . . . . . . . FF
BITS. . . . . . . . 00000000
SVRB: 005FD330
WLIC. . . . . . . 0002008A
FLCDE. . . . . . 00000000
OPSW . . . . . . 070C1000
8217C906
PRB: 005911F8
WLIC. . . . . . . 00020033
FLCDE. . . . . . 14000000
OPSW . . . . . . 070C1000
82613B42
SVRB: 005FD4A0
WLIC. . . . . . . 0002000C
FLCDE. . . . . . 00000000
OPSW . . . . . . 070C1000
8222FD50
IRB: 005D1230
WLIC . . . . . . . 00020001
EPA . . . . . . . . 829916C1
OPSW . . . . . . 078C0000
80000004
PRB: 005FF510
WLIC . . . . . . . 00020001
FLCDE. . . . . . 00C29D40
OPSW . . . . . . 078C1000
82608954
CDE: 00C29D40
NAME. . . . . . . IKJEFT02
ENTPT. . . . . . 82608000
Figure 11: Case Study Dump Inspection of SYNCH SVRB
SVRB: 005FD4A0
-0020
-0008
+0018
+0020
+0030
+0040
+0050
XSB . . . . . . . 005FD570
FLAGS1. . . . . 20000000
Q . . . . . . . . . 00000000
GPR0-3. . . . . 00000000
GPR4-7. . . . . 00008918
GPR8-11. . . . 005E8AF8
GPR12-15. . . 00008438
FLAGS2 . . . . .
WLIC . . . . . . .
LINK. . . . . . . .
000084B8 . . .
000084D0 . . .
00008770. . . .
00008438. . . .
00000000
0002000C
005D1230
000084D4
005D1230
000084D8
8260FAC6
00008904
005FF510
8260F530
80000000
Figure 12: Case Study Dump List of Register 15 Area
LIST 0260F430
ASID(X’0020’)
ASID(X’0020’) LENGTH(300)
ADDRESS(0260F430) KEY(00)
CHARACTER
COMMON
0260F430
. . . . . . &. . . . . . . .|
0260F440 | B4&. . . . .C.&. . . . .C%&. . . . . . . &. . . . }. . . \. . .0\.. . 0H. . . . B>.0B. .-4. . . . . |
0260F480 | IRXTVARS. . .0B@. . . . . . . . . 0.H. . . . . . . . . 0. . . . . . BQ. .B0.0. . . . . . . . PATC |
0260F4C0 | H AREA - IKJEFTP8 91.077B. B . B . B . B{BBBDBFBHB.B.B.B}BKBMBOBQB.B.B. |
0260F500 | B\BSBUBWBY. . . . . . . . }. . . . . . . .QSYSIKJUA. . . . . . . . . . . 00. . IKJEFT03-TM |
0260F540 | P EXIT ATTENTION ROUTINE 910
|
IRB: 005D1230
WLIC.. . . . 00020001 EPA. . . . . . 829916C1 OPSW. . . . . 078C0000 80000004
029916C1 | 00. . IEAVAR05 91038 HBB4420. . . }. . .3. . /. . . . J.-. . .B. .
|
NaSPA board member
Tom Bryant has more
than 18 years of experi ence in data processing,
with 15 of those years
as an MVS/XA/ESA
systems programmer.
©1996 Technical Enterprises, Inc. Reprinted
with permission of Technical Support
magazine. For subscription information, email
mbrship@naspa.net or call 414-768-8000,
Ext. 116.
TECHNICAL SUPPORT DECEMBER 1996