October 14, 2013
DB2 for z/OS: Tuning
Dynamic SQL
Randy Ebersole
IBM DB2 for z/OS Lab Services
November 20, 2013
© 2013 IBM Corporation
© 2013 IBM Corporation
1
Disclaimer
© Copyright IBM Corporation 2013. All rights reserved. U.S. Government Users Restricted
Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM
Corp.
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL
PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF
THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS” WITHOUT WARRANTY
OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT
PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM
SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE
RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS
PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR
REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND
CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR
SOFTWARE.
IBM, the IBM logo, ibm.com, DB2 are trademarks or registered trademarks of International Business Machines
Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked
on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S.
registered or common law trademarks owned by IBM at the time this information was published. Such trademarks
may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available
on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml
1
© 2013 IBM Corporation
2
Agenda
▪ Dynamic SQL Review
▪ Dynamic SQL Performance Considerations
▪ Identifying Dynamic SQL for Tuning
▪ Tuning Tools
© 2013 IBM Corporation
3
But First.....
© 2013 IBM Corporation
TCP/IP
TCP/IP
TCP/IP
Clients
App
Server
DS
Driver
DB2
z/OS
4
The Transaction Manager, Connections, Threads
• DDF is DB2’s transaction manager for distributed database
connections
• With DRDA, connections can come from literally anywhere
• DDF has very mature thread management strategies
• DDF runs as an additional address space in the DB2 subsystem
– The address space name is xxxxDIST
– DDF uses SRBs• instead
of TCBs,
which reduces CPU time.
Click to
add text
MVS enclaves are used in exchanging
• Enclaves are used across address spaces for
communication
© 2013 IBM Corporation
Dynamic SQL
▪ Dynamic SQL is becoming more prevalent
 Programming models
 Vended applications
▪ Dynamic SQL is evaluated for cost-based
optimization at execution
 “Full Prepare” - Code path for executing dynamic SQL
includes DB2 Optimizer access path decision
 “Short Prepare” – Dynamic SQL statement matches
statement found in global Dynamic Statement Cache
– Access path reused
– DSNZPARM CACHEDYN=YES
 CACHEDYN=NO only appropriate if no dynamic SQL at all
© 2013 IBM Corporation
6
Dynamic SQL, continued
▪ “Prepare Avoided” – Prepared dynamic SQL
statement string saved in local thread storage
 If statement reused, no need to call Dynamic Statement
Cache (DSC)
– Locally cached statement reused unless new PREPARE for
same statement
 DSNZPARM MAXKEEPD limits number of statements
cached in local thread storage
– Virtual storage considerations, especially prior to DB2 10
 BIND option KEEPDYNAMIC YES keeps prepared
dynamic SQL statement in local thread storage across
COMMIT
– Avoid cost of subsequent PREPARE after each COMMIT
© 2013 IBM Corporation
7
Dynamic SQL Performance Considerations
▪ Full prepare can cost many times that of
equivalent static SQL statements
 E.g: in one case of typical transaction workload, 3x CPU
when code change made short prepare impossible
▪ Short prepare (using dynamic statement cache, or
DSC) will still be more expensive than equivalent
static SQL statement
 Short prepare requires character by character match of
statement in DSC
▪ Local statement cache can approach, or in some
cases exceed, performance of equivalent static
SQL
© 2013 IBM Corporation
8
Dynamic Statement Cache
▪ On execution of dynamic SQL, DB2 searches DSC for
matching SQL statement
 If no match, new statement added to DSC
 Older statements evicted from DSC
▪ Increasing hit ratio in DSC improves performance of
dynamic SQL
 No match in DSC if literals in predicates change
▪ Use parameter markers, “?”, for literal values that change
frequently
 Parameter markers allow match in DSC
 Do NOT use parameter markers for ALL predicates;
– Better for predicates for which values change
© 2013 IBM Corporation
9
Concentrate Literals
▪ Some applications do not use parameter markers
 3rd party SW, Code generators, programmers just won’t
▪ DB2 10 for z/OS provides CONCENTRATE
LITERALS
 Uses “&” instead of “?”
▪ BIND option and statement option





Allows DB2 to replace all literals with “&”
May improve match in DSC
May reduce efficiency for literals that do not change
Better for transactional type work
You may not have access to statement level !
© 2013 IBM Corporation
10
Dynamic SQL Performance Hierarchy
Not to scale
BEST
▪ SQL statement string in local statement cache
▪ SQL statements with intelligent use of parameter
markers
 E.g. if predicate on “LastName” use ‘?’
 If predicate for status code with 10 values, use literal
– Only 10 statements in DSC
▪ SQL statements with Concentrate Literals
▪ Packages with Concentrate Literals
▪ SQL statements with frequently changing literals
▪ All full prepares (e.g. CACHEDYN = NO)
WORST
© 2013 IBM Corporation
11
Dynamic SQL: Costs of Full and Short Prepare
SQL Complexity
SQL Execution Cost
b
Full Prepare
a
Short Prepare
Cost
Total cost of SQL = prepare cost + execution cost
In ‘a’, short + execution far less than full + execution, and ratio of full to execution is very high
In ‘b’, short + execution less than full + execution, but ratio of full compared to execution is lower, and
potential better access path with full prepare may be worthwhile
© 2013 IBM Corporation
12
Tuning Dynamic SQL - Challenges
▪ Even with excellent hit ratio in DSC, there may be bad
access paths that are reused
 ‘Excellent’ >= 95%
▪ How to identify problem dynamic SQL statements?
 Infrequently used dynamic SQL will not stay in DSC long
▪ Once identified, how to tune?
▪ Generally RUNSTATS on table or index will evict
statements from DSC that depend on that table or index
 Subsequent execution requires full prepare
▪ Plan Management does not apply to dynamic SQL
 But Optimizer Hints can
© 2013 IBM Corporation
13
Identifying Dynamic SQL for Tuning
▪ EXPLAIN Dynamic Statement Cache
▪ Sort resulting statements by total CPU consumption
▪ For top consumers, evaluate Getpage Ratio
 Ratio of Getpages to Rows Returned
 High Getpage Ratio suggests tuning opportunity
▪ Use tools to find tuning problems
 Use acquired skills, intuition, etc.
 Few DB2 professionals make best use of tools
▪ Test performance of hypothetical new path
▪ Evaluate results
▪ Repeat
© 2013 IBM Corporation
14
EXPLAIN Dynamic Statement Cache
▪ Sample procedure
 Create tables (DSNTESC in SDSNSAMP)
–
–
–
–
DSN_STATEMENT_CACHE_TABLE for execution stats data
DSN_FUNCTION_TABLE
DSN_STATEMENT_TABLE
PLAN_TABLE
 Start collection of DSC performance statistics
– -START TRACE(P) CLASS(30) IFCID(316 317, 318)
 Use EXPLAIN STMTCACHE ALL to extract SQL
statements from DSC and dump statistics to
DSN_STATEMENT_CACHE_TABLE
© 2013 IBM Corporation
15
Changes in Dynamic SQL Performance
▪ How do you determine if the performance of your dynamic
SQL has changed?
 Monitors?
– If so, which one?
 User calls to Help Desk?
– “I ran the VERY SAME query on Friday, but today it still has not
responded?”
 Except on Friday ‘STATE’ = ‘AK’ and today ‘STATE’ = ‘CA’
 Other?
▪ If you do not have historical information about your SQL it
will be difficult to tune effectively
 Note: just because it has always run this way does NOT mean it is
good!
▪ Monitor tuned statements as well as largest consumers
 If cannot validate value of tuning, nobody will report the benefit
© 2013 IBM Corporation
16
Tuning the Problem SQL Statements
▪ So, now you have found some problem SQL
statements….
▪ What do you do?
▪ Tuning dynamic SQL statements is different from
tuning static SQL statements, because…
 NOT !
 Same issues, same approaches, same tools
– With a couple exceptions
© 2013 IBM Corporation
17
SQL Tuning Tools
▪ Different tools have different approaches and offer different
benefits
 But same tools can be used for static or dynamic SQL
▪ IBM tools include:
 Query Monitor
– Take advantage of IFCID 316, 317, 318 data
 InfoSphere Optim Query Workload Tuner
– Rather than analyzing a single statement, analyze a set of statements
that make up a workload
 Data Studio
– Access Path Graphs, RUNSTATS Advisor, Query Capture Environment
 DB2 Connect Unlimited Advanced Edition for System z
– Includes: InfoSphere Optim pureQuery for LUW
 Capture dynamic SQL, convert to static SQL, increase security
© 2013 IBM Corporation
18
Data Studio
© 2013 IBM Corporation
19
Query Hints
▪ Tell the Optimizer what you want it to do…
 As a last ditch effort, assisted by level 2, use access
path hints in DB2
 Determine if the 80/20 rule applies and which access
path is acceptable for the majority of the time
 Determine if BIND(QUERY) is reasonable
 Try Selectivity Override (DB2 11)
– DSN_PREDICATE_SELECTIVITY
 Skew your own stats
▪ Put access path hint in DSN_USERQUERY, BIND
query, then flush user query table
© 2013 IBM Corporation
20
What if Tuning Efforts Do Not Work?
▪ If you suspect Optimizer problem
 Gather data for Level 2
– https://www.ibm.com/support/docview.wss?uid=swg21206998
– http://www.ibm.com/developerworks/data/library/techarticle/dm1012capturequery/index.html
– SYSPROC.ADMIN_INFO_SQL supports V8 -> V11 (Required)
 Data Studio V4.1 incorporates this procedure into a GUI
(Best Practice)
–
–
–
–
–
http://www.ibm.com/developerworks/downloads/im/data/
No charge product (replacement for OSC and Visual Explain)
Incorporates Statistics Advisor
Query Environment Capture used to collect doc
FTP doc directly to DB2 Level 2 in the tool
© 2013 IBM Corporation
21
High Performance DBATs
•
Re-introducing RELEASE(DEALLOCATE) in distributed package
– Could not break in to do system maintenance - Utility or DDL
– V6 PQ63185 to disable RELEASE(DEALLOACTE) on DRDA DBATs
•
High Performance DBATs reduce CPU consumption by
– RELEASE(DEALLOCATE) to avoid repeated package allocation/deallocation
– Avoids processing to go inactive and then back to active
– Bigger CPU reduction for short transactions that commit often
•
Using High Performance DBATs
– Stay active if there is at least one RELEASE(DEALLOCATE) package exists
• Click to add text
– Connections will turn inactive after 200 commits to free up resources
– Normal idle thread time-out detection will be applied to these DBATs
– Good match with JCC packages (handful packages)
– No benefit and not support for ACTIVE threads (CMSTATS=ACTIVE)
– No benefit for KEEPDYNAMIC YES users
© 2013 IBM Corporation
Enabling High Performance DBATs


Two steps to enable High Performance DBAT
1. REBIND with RELEASE(DEALLOCATE)


Default BIND option in JCC driver will be RELEASE (DEALLOCATE) for JCC
Driver matching with DB2 10 (JCC 9.7 FP3a)
2. Then command –MODIFY DDF PKGREL (BNDOPT)

• DISPLAY DDF shows the option currently used

To disable - MODIFY DDF PKGREL (COMMIT) to overlay BNDOPT option

To monitor - Statistics GLOBAL DDF Activity section
• DDF
Click
to add text
GLOBAL
ACTIVITY
CUR ACTIVE DBATS-BND DEALLC
HWM ACTIVE DBATS-BND DEALLC
QUANTITY
5.3
10.00
© 2013 IBM Corporation
Summary
▪
▪
▪
▪
Dynamic SQL use is growing and growing
Many options in place to manage
Take advantage of all of the functionality
It's the “same” as static SQL
© 2013 IBM Corporation
24
Questions?
THANK YOU !
ebersole@us.ibm.com
© 2013 IBM Corporation
25