Anthony Reina - Accelerated Value Specialist aereina@ca.ibm.com
Information Management
© 2010 IBM Corporation

2
Information Management
•The IBM Software Accelerated Value Program delivers a proactive, cost-reducing, and productivity enhancing advisory service. The program pairs you with an assigned team who build a foundational understanding of your overall environment. Through that understanding, the trusted partner works to facilitate faster deployment, lifecycle leadership, risk mitigation, and more by identifying ways to improve your environment, staff skill set, and processes.
http://www-01.ibm.com/software/support/acceleratedvalue/
© 2010 IBM Corporation

3
Information Management
• Introduction – Why, What, and How?
• Using db2fodc for v9.7 for hang/performance issue
• Enabling and setting db2cos (DB2 call out script)
• The db2support
• Collecting db2support with OPT data
• The db2exfmt (access plan)
• Sending files to IBM.
• Related links and documents
• Gotchas
• Question/Answer
© 2010 IBM Corporation

4
Information Management
Why there is a need for an action plan when issues occur?
 Avoid confusion due to a standard response procedure.
 Problem definition is immediately defined.
 Correct diagnostic information is collected based on the issue reported.
 Reduce overhead time, improve time to resolution.
© 2010 IBM Corporation

5
Information Management
What information to collect?
 Background Information :
- Time when the problem occurred.
- Conditions that currently exist on the system.
- Any recent changes to the system (ie. HW, SW, others)
- Operation/steps leading to the issue.
- Steps taken to recover from the issue.
- Time when DB2 or system return to normal.
- Diagnostic data
 Diagnostic Data
© 2010 IBM Corporation

6
Information Management
How to collect diagnostic data?
Type of Issue db2fodc
(V9.7) db2cos db2support OPTIMIZER db2exfmt
All type of issue
Query performance
Query nonperformance
Performance
Hang
Timeout/Deadlock
© 2010 IBM Corporation

7
Information Management
• db2fodc o DB2 First Occurrence Data Capture process.
• Can be used to collect hang and performance issues.
• Diagnostic data collection usage : db2fodc –db <dbname> -<perf, basic> <full, basic>
• Output directory will be in the DIAGPATH directory.
• Can be customized as needed.
• Output directory will have a prefix of
“FODC_Perf_<timestamp>_<nodeno>”
© 2010 IBM Corporation

Information Management
• db2fodc -perf : refer to TOOLS
OPTION of /bin/db2cos_perf
• db2fodc -hang : refer to TOOLS
OPTION of /bin/db2cos_hang
8
Data to be collected
OS Iteration db2_config basic_db2_config more_db2_config os_monitor_info call_stacks db2pd_info db2pd_dump db2trc ostrc db2_monitoring extra_info
On
On
On
BASIC
On
On
FULL
On
On
On
On
On
On
On
Data to be collected
Snapshot
Stacktrace aka call stacks vmstat
Iostat db2perfcount db2trc latchtrace
4
3
BASIC
0
0
200
200
0
6
5
FULL
1
0
200
200
2
Data to be collected os_iterations os_sleep db2pdstack_iterations db2pdstack_sleep db2monitor_sleep fullstack_sleep db2trc_size ostrc_size ostrc_sleep announce_sleep no_wait
Value
2
30
2
120
60
300
8mb
8000000
30
15
OFF/ON (Default=OFF)
© 2010 IBM Corporation

Information Management
• Customize the db2fodc –perf full data collection
Before After
9
Actual Execution
© 2010 IBM Corporation

10
Information Management
• Customize the db2fodc –hang basic to collect db2look.
© 2010 IBM Corporation

Information Management
• Execute the db2fodc –hang basic command.
db2fodc output
11
Additional db2look output.
© 2010 IBM Corporation

12
Information Management
• DB2 Callout Script invoked by default when DB2 cannot continue processing.
• Can be used when diagnostic data is hard to capture
(ie. locktimeout, deadlock, sqlcode, or trap).
• The db2cos script can be modified as needed.
• By default it only collects only DB2PD Instance related info.
• The default db2cos scripts is under /bin directory.
• Place modified db2cos scripts under /adm directory. This will be executed first if exist.
© 2010 IBM Corporation

13
Information Management
• db2cos example usage :
 Copy the /sqllib/bin/db2cos to /sqllib/adm/db2cos
 Modify db2cos to capture snapshot and db2pd info.
 Enable –catch flag to trigger db2cos script db2pdcfg –catch -911,2 locktype=V –alldbp db2pdcfg –catch -911,68 locktype=V –alldbp
 Wait for problem reoccurence
 To disable : db2pd –catch clear
 To verify state : db2pd –catch status
© 2010 IBM Corporation

Information Management
The db2support
• Collects environment data from client or server machine.
• Most important utility in collecting information for a DB2 issue.
• The following information is collected :
 db2diag.log
 All trap file
 Locklist files
 Dump files
 Buffer pool and table space
 Various system related files
 Output from various system commands
 db2cli.ini
• Command : “db2support . –d <dbname>”
• Output is created on the directory where command is run.
• Output filename is db2support.zip
• Db2support will collect all the files in the DIAGPATH incl. old db2diag.logs.
© 2010 IBM Corporation 14

Information Management
15
• Collects additional optimizer (OPT) data.
• Used to investigate optimizer issues.
• Can be invoked via –sf and/or –cl option
 -sf <SQL file> : SQL filename
 -cl <0, 1, 2, 3> : Level of info to collect
• Level of information collected :
 0 = collects only catalogs and db2look
 1 = collect 0 plus db2exfmt
 2 = collect 1 plus .db2service
 3 = collect 2 plus db2batch
• Commands :
 db2support . –d <dbname> -sf <SQL file> -cl 1
 db2support . –d <dbname> -cl 0
• Output will be part of the db2support.zip called db2supp_opt.zip -
/OPTIMIZER.
© 2010 IBM Corporation

16
Information Management
1. Collect optimizer data for different OPTLEVEL including db2batch information.
 db2support <output dir> -d <dbname> -c –cl 3 –sf <sql file>
-ol 0,1,2,3,5,7 –extenddb2batch
2. The
–basic option collects only optimizer-related diagnostic information.
3. The –extenddb2batch collects db2batch information when –ol and
–cl options are used.
4. No db2look and catalog information collected when
“-nl | -nodb2look” and “-nc | -nocatalog” options are used.
© 2010 IBM Corporation

17
Information Management
• Captures information about the access plan, optimizer inputs, and environment of an SQL query.
• Helps to understand how the SQL query is compiled and executed.
• Shows how configuration parameter changes impact query performance.
• Indispensible tool for query problem determination.
• Explain information includes
 Sequence of operations to process the query
 Cost information
 Predicates and selectivity estimates for each predicate
 Statistics for all objects referenced in the SQL query at the time the explain information is captured.
© 2010 IBM Corporation

18
Information Management
• Explain tables must be created before the tool can be used.
 SYSINSTALLOBJECTS() procedure – call sysproc.sysinstallobjects(‘EXPLAIN’,’C’,NULL,<schema name>)
 EXPLAIN.DDL – db2 –tvf …/sqllib/misc/EXPLAIN.DDL
• Using the db2exfmt tool :
 db2 set current explain mode explain
 db2 –tvf <SQL filename>
 db2 set current explain mode no
 db2exfmt –d <dbname> -1 –o <output filename> or db2exfmt –d <dbname> -g TIC –w -1 –n –s % -# 0 –o <output filename>
© 2010 IBM Corporation

19
Information Management
• Submit files to IBM by using IBM Enhanced Customer Data
Repository (ECuRep) Service site.
• Provides secure and central repository of submitting files to IBM.
• Important Notes :
 Files must be in a compressed format.
 Submit files only if a problem record is already opened.
 IBM may provide information back via ECuRep as well.
© 2010 IBM Corporation

20
Information Management
• Filename convention : xxxx.bbb.ccc.yyy.yyy
Where : xxxxx = PMR Number bbb = Branch ccc = Country Code yyy.yyy = Description of the file type.
Example: 34143.055.724.zip
• How to submit files to the ftp server :
1. Use a FTP client to connect to ftp.ecurep.ibm.com
2. Login as anonymous and enter your email address as the password.
3. Change the directory to /toibm/im
4. Change to binary transfer mode for example for a FTP connection from a Windows command line, enter: bin.
5. Upload your file.
6. Close your FTP client.
© 2010 IBM Corporation

21
Information Management
• Submitting files to IBM via ECuRep
• Collecting Data for DB2 Compiler Issues
• IBM developerWorks
• IBM Support Portal
© 2010 IBM Corporation

22
Information Management
 Db2support will also collect db2fodc diagnostic data as well.
 Collect db2cos data if stored outside the DIAGPATH directory.
 Use “db2support…-sf <SQL file> -cl 1” for issues involving one query.
 Use “db2support…-cl 0” for issues involviong more than one query.
 Use db2exfmt tool to collect query access plan as needed or if more than one query is involved.
 Advice support analyst via email once diagnostic files have been sent via
ECuRep.
© 2010 IBM Corporation

23
Information Management
© 2010 IBM Corporation