DB2z LOG RBA Reset
Jay Seiler(Systems Engineer)
Laurie Pratt(DBA)
Jseiler@LLBean.com
Lpratt@LLBean.com
L.L. Bean, Inc
1
Agenda
•
•
•
•
•
•
•
•
LLBean overview
LLBean DB2 history
LLBean’s DB2z current environment
The problem
Planning
Testing
Implementation
Questions
L.L. Bean, Inc
2
LLBean Overview
•
•
•
•
Founded in 1912 by Leon Leonwood Bean
Our 100th anniversary this year !
Headquartered in Freeport, Maine
L.L.Bean began as a one-room operation selling a
single product, the Maine Hunting Shoe
• We have grown to a global, multi-channel
organization
L.L. Bean, Inc
3
LLBean Overview
• L.L.Bean products are rigorously tested,
guaranteed to last and always shipped free;
trusted source for quality apparel, reliable
outdoor equipment and expert advice for 100
years
• Our 220,000 sq. ft. Flagship campus of stores in
Freeport is open 24 hours a day, 365 days a year
and welcomes more than three million visitors
each year
L.L. Bean, Inc
4
LLBean Overview
• Winner of National Retail Federation’s “Top 10
Customers' Choice Award”; rated in the top 3 for
past 4 years; nominated once again this year
• L.L.Bean products are recognized around the
world for their practical design and high quality;
many items have been in the product line for
several decades
L.L. Bean, Inc
5
LLBean DB2 History
•
•
•
•
1991
Brought in DB2z v2.3
1996
Launched LLBean website (DB2 LUW)
1996
Implemented Data Sharing
2003
Implemented a distributed front end
gui client accessing DB2z data via stored
procedures for our customer service reps to
take orders on the phones and handle
telephone inquiries
• 2006
Eliminated Data Sharing
• 2010
migrated to v9.1
L.L. Bean, Inc
6
LLBean’s DB2z
Current Environment
• Two subsystems in our Production environment
– DBQ1: originally intended as decision support
and it is small.
– DBP1: large, 3TB
• V9.1 nfm/non data sharing
• Virtual tape for image copies
• Had a z9 (2674 mips) at start of planning,
but a z10 (same mips) by the time
we did the RBA Reset
L.L. Bean, Inc
7
The Problem
• On March 8, 2011, from the console:
*DSNJ032I -DBP1 DSNJW307 WARNING - APPROACHING END OF 761
THE LOG RBA RANGE OF FFFFFFFFFFFF. CURRENT LOG RBA IS F00035BCE000.
DSNJ033E -DBP1 DSNJW307 THE LOG RBA MUST BE RESET 762
BEFORE THE END OF THE LOG RBA RANGE IS REACHED
• This was the beginning of our journey
L.L. Bean, Inc
8
How long do we have before the end
of the log RBA is reached?
• Looks like more than a year based on our processing rates
in March:
FFFF00000000 (high log RBA)
minus F065441C3F42 (current RBA)
equals F99BBE3C0BE
• Divided by the average RBA increase per a day in March
equals 397 DAYS
• Divided by the average RBA increase per a day in our
peak (busiest) season equals 78 DAYS
• The decision was made to start planning the RBA Reset
immediately
L.L. Bean, Inc
9
Planning
L.L. Bean, Inc
10
LOG RBA Reset process for non
data sharing environment
IBM link to their documentation on the process:
http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/index.jsp?topic=/com
.ibm.db29.doc.admin/db2z_subsystemendrba.htm
In a nutshell:
• Use IDCAMS to delete and redefine the table spaces SYSUTILX, SYSCOPY,
and SYSLGRNX and their corresponding indexes. Then, re-initialize these
page sets.
• APAR PK28576 – new DSNZPARM, SPRMRRBA, to enable special processing
to reset the RBAs in the DB2 data for non data sharing DB2 subsystems
using DB2’s COPY utility.
• Cold Start the subsystem back to an RBA value of 0
• Image copy all tablespaces and indexes with zparm SPRMRRBA on
• Stop DB2, Turn off zparm SPRMRRBA, Start DB2 for normal access L.L. Bean, Inc
11
RBA vs. LRSN
• RBA
– Relative Byte Address
– Used by non data sharing subsystem
• LRSN
– Log Record Sequence Number
– Used by data sharing subsystems
L.L. Bean, Inc
12
Planning Continued
• IBM was contacted and provided great support:
• information from other companies that had done an RBA
Reset
• Bank of Italy’s plan
• Using DIAGNOSE TYPE(576) in the image copy for indexes
which reduces what is written for an image copy dataset
• Formalized as a project with a project manager, systems
programmer, DBA and a DBA who knew DPROP
• Attended March NEDB2UG. John Iczkovits talked about LBI
(large block interface) for image copy datasets. Never
benchmarked, but did use this for the image copies.
L.L. Bean, Inc
13
Planning Continued
• Looked for obsolete tables to drop
• Looked for tables where purges could be more aggressive
• Used real time stats to determine how to break up image
copy jobs.
• Turned COPY YES on for all indexes
• Created validation queries to determine if any
tablespaces or indexes had been missed during the image
copy process
• Planned backout for RBA implementation; this would use
SRDF (mirrored data)
L.L. Bean, Inc
14
DataPropagator(DPROP)
aka: Infosphere Replication Server
• DataPropagator captures
inserts/updates/deletes from the log to be able
to propagate the data changes to other
systems. DataPropagator keeps track of the last
RBA propagated to know where to start in the
log when DPROP is stopped and started.
• This added time to our outage to insure that
DPROP was caught up with the log when we
shut down DB2 and making sure that DPROP
worked appropriately when DB2 was restarted
for normal access after the RBA Reset.
L.L. Bean, Inc
15
Testing
• Tested on 2 non prod subsystems
• Only one non prod subsystem had DataPropagator; needed
to test with DPROP twice
• Did a dry run just image copying all page sets in production to
help determine how many jobs could run concurrently
L.L. Bean, Inc
16
Implementation
•
•
•
•
•
Sat
11pm
11:45pm
midnight
• Sun
• 1:30am
• 4:30am
• 4:45am
• 6:30am
• 8am
Brought down prod systems
Once everything was down, broke the SRDF links
DBA made sure DPROP was completely caught up
Started image copies
It took 1.5 hours to do the image copies
Restarted DB2 for normal access,
Validation, cold start of DPROP and DPROP steps
Developers ran batch jobs to insert/delete/update
Go decision was made, Reconnect SRDF,
Bring up prod servers & apps
Call centers online
Batch started
L.L. Bean, Inc
17
Questions?
Thank you.
L.L. Bean, Inc
18