1
®
Tuning with Optimization Service Center
Part I
Patrick Bossman
IBM Silicon Valley Lab
Columbia, MD
September 12, 2007
© IBM Corporation 2007
2
Agenda
• Overview of Optimization Service Center
• Connect and configure OSC
• Workload (application) Tuning
• Query tuning (Part II)
© IBM Corporation 2007
3
Overview
•
Optimization Service Center (OSC)
– New no charge product supported for connections
to DB2 9.
•
Support for DB2 for z/OS V8 connections in open beta
– Workstation tool for monitoring and tuning of
queries
– Facilitates the identification and tuning of
workloads (sets of queries) as well as individual
queries
– New powerful query diagnostic tools enable faster
deep analysis of queries
© IBM Corporation 2007
4
Overview
•
OSC feature list
– Support for workload tuning
•
•
–
–
–
–
–
Stand-alone workload creation
Push-out model (monitoring)
Visual Explain query graphing capabilities
Query formatting and annotation
Query report
Visual Plan Hint
Statistics Advisor - query and workload
© IBM Corporation 2007
5
Connect and configure
• Launch OSC
– Initial screen is connection and configuration
– Review look and feel
– Review layout of screen
© IBM Corporation 2007
6
Launch OSC
© IBM Corporation 2007
7
Project navigator area
© IBM Corporation 2007
8
Project navigator
• Navigate through open projects
© IBM Corporation 2007
9
Top navigation tabs
© IBM Corporation 2007
10
Top navigator tabs close-up
© IBM Corporation 2007
11
Connection / configuration
© IBM Corporation 2007
12
Connection close-up
• Connection
– Subsystems cataloged in DB2 Connect listed
– Use subsystem menu option to add / remove
subsystems
© IBM Corporation 2007
13
Add Subsystem
• Add DB2 subsystem connection
– Select subsystem button
– Choose Add
© IBM Corporation 2007
14
Connection information
• Add DB2 subsystem connection
– Subsystem alias is a meaningful name to you
– Location, hostname, port
• Can be found in <ssid>MSTR address space DDF startup
message DSNL004I
© IBM Corporation 2007
15
DSNL004I
• DSNL004I message
– Location = location
– Domain = hostname
– TCPPORT = port
© IBM Corporation 2007
16
Jeez Pat, Connect already!
• Connect to subsystem
– Choose Connection, connect
– You can connect to more than one subsystem and
perform activities on more than one subsystem at a
time.
© IBM Corporation 2007
17
Login information…
© IBM Corporation 2007
18
Bind packages
© IBM Corporation 2007
19
Create explain tables
© IBM Corporation 2007
20
Create explain tables
© IBM Corporation 2007
21
Create explain tables
© IBM Corporation 2007
22
Create alias to explain tables
• Create aliases
– You can use OSC to create ALIASes to the explain
tables also.
© IBM Corporation 2007
23
Create alias to explain tables
© IBM Corporation 2007
24
Subsystem status
© IBM Corporation 2007
25
Subsystem status
© IBM Corporation 2007
26
Explain-enabled Authid’s
© IBM Corporation 2007
27
EXPLAIN enabled authid’s
© IBM Corporation 2007
28
Connection complete.
Now what?
(Welcome menu…)
© IBM Corporation 2007
29
Go to welcome page…
© IBM Corporation 2007
30
Welcome page
© IBM Corporation 2007
31
Welcome page close-ups
© IBM Corporation 2007
32
Welcome page close-ups
© IBM Corporation 2007
33
Welcome page close-ups
© IBM Corporation 2007
34
Application tuning
• Application tuning
– Creating workloads
– Workload options
– Workload tuning features
© IBM Corporation 2007
35
Application tuning process
• What is application tuning?
– Identify what the application workload is
• Individual SQL statements
– Get an understanding of the applications behavior
• How often are individual SQL statements executing?
• What is the performance of individual SQL statements?
– Determine statistics for workload
• Workload statistics advisor
• More workload analysis features coming… (?)
– Remeasure workload
• Identify top tuning candidates
• Use query based tools to further analyze (next session)
© IBM Corporation 2007
36
Creating workloads
• Creating a workload
– Tune a workload
– View workloads
– Monitoring functions
© IBM Corporation 2007
37
Tune a workload
© IBM Corporation 2007
38
Workload sources
• Workload sources
–
–
–
–
–
–
Snap statement cache
Catalog (static SQL)
QMF / QMF HPO
File
Categories
Other workloads
© IBM Corporation 2007
39
Workload sources
© IBM Corporation 2007
40
Catalog source…
© IBM Corporation 2007
41
Catalog source
© IBM Corporation 2007
42
Package filter options
• Package filter options
– Collection id, name, owner, …
– Can use equals, like, in, etc.
© IBM Corporation 2007
43
Package filter options
© IBM Corporation 2007
44
Plan filter options
• Plan filter options
– Plan name, plan creator, etc.
– Can use equals, like, in, etc.
© IBM Corporation 2007
45
Plan filter options
© IBM Corporation 2007
46
Cost & Object filter options
• Cost and Object filters
– Filters SQL within package / plan filter
– Requires the static SQL be bound with explain yes.
– DSN_STATEMNT_TABLE must already be
populated.
– Show only SQL with PROCSU > …
– Show SQL which uses table…
– Show SQL which uses index…
• Choice to return rows which qualify for ANY
or ALL of the cost / object filter conditions
© IBM Corporation 2007
47
Cost & Object filter options
© IBM Corporation 2007
48
Access path filters
• Access path filters
– Filters SQL within package / plan filter
– Requires the static SQL be bound with explain yes.
– PLAN_TABLE must already be populated.
• Show SQL which performs…
–
–
–
–
–
–
Tablespace scan
Sort
Non-matching index scan
List prefetch
Outer join
….
© IBM Corporation 2007
49
Cost & Object filter options
© IBM Corporation 2007
50
Capture the workload
• After selecting packages / plans / filters…
– Click finish to capture workload
– OSC goes about the business of collecting the
statements…
• Queue Jeopardy music, get some coffee.
© IBM Corporation 2007
51
Workload captured…
© IBM Corporation 2007
52
Review workload screen
• Workload header
• Workload options
• Project Navigator
© IBM Corporation 2007
53
Workload header
© IBM Corporation 2007
54
Workload header
© IBM Corporation 2007
55
Workload options
© IBM Corporation 2007
56
Workload options
© IBM Corporation 2007
57
Workload option close-up (1)
© IBM Corporation 2007
58
Workload options close-up (2)
© IBM Corporation 2007
59
Workload statements…
© IBM Corporation 2007
60
Workload statements
• Workload statements
– Show the SQL within the workload
– Refine execution runtime information
• For catalog capture, runtime information would have to be
manually obtained.
• You don’t really need to change it for statistics advisor…
– Let’s walk through it…
© IBM Corporation 2007
61
Workload statements view
© IBM Corporation 2007
62
Capture more statements…
© IBM Corporation 2007
63
Description
© IBM Corporation 2007
64
Schedule workload activity
© IBM Corporation 2007
65
Remove statements from workload
© IBM Corporation 2007
66
Execute query tools on selected SQL
© IBM Corporation 2007
67
Select “edit runtime information…”
© IBM Corporation 2007
68
Edit runtime information
© IBM Corporation 2007
69
Run workload advisors (SA)
© IBM Corporation 2007
70
Why Statistics Advisor?
• Importance of statistics
• Difficulty of determining what statistics to
collect
• Solution: Workload Statistics Advisor!!!
© IBM Corporation 2007
71
Estimating query cost
• Accurately estimating the number of rows is
important for to accurately cost a query
– Index matching
• Accurately estimate index cost
– Total index filtering
• Estimate table access cost via index(es)
• Choose how to use index (prefetch?)
– Total table level filtering
• Efficient join order
• Efficient join method
• Appropriate sorts
© IBM Corporation 2007
Symptoms of inaccurate
estimation
• Sometimes - great performance!
– Few competitive choices
– Good performance possible, but not a "winning" strategy
• Inefficient access path
– Query never performs at optimal level
• Unstable access path
– Slight changes in input result in dramatic swings in actual
performance
– Efficient and inefficient access paths can end up with close cost
estimates
– More accurate selectivity estimates can improve optimizer
"differentiation" of efficient
and inefficient access
© IBM Corporation 2007
72
73
Some observations
• Several candidate indexes?
– Need sufficient statistics to differentiate the candidates
• Local filtering spread across many tables?
– The more "potentially" desirable outer tables there are, the more
important it is to accurately estimate the true qualified rows
• Lot's of predicates which don't filter much?
– Important to make sure DB2 knows that
– When many rows are processed, a single bad choice can be magnified
• The necessity for accurate estimation is related to the
number of competitive choices.
© IBM Corporation 2007
74
Why statistics advisor?
• Statistics are collected to help optimizer in
costing available access paths
• Determination of statistics required can be
daunting
• Automate the determination of statistics
required
© IBM Corporation 2007
75
What are the right statistics?
• Identify interesting objects
– Tables
– Indexes
• Identify interesting columns, column groups
– Predicate analysis
• Which column (groups) have predicates?
• Which column (groups) are indexed?
• Which statistics should I collect?
–
–
–
–
Column statistics
Correlation statistics
Frequencies
Histograms
– Statistic needed depends on predicate types (range, equal)
© IBM Corporation 2007
76
Determining the "right" statistics
• Manual analysis very time consuming, requires
significant skill
– Format the query to make it readable
– Recognize predicate transformations
•
•
•
•
Predicate transitive closure
Between to equal opens new possibilities
Or to IN creates new boolean term predicates
Has DB2 rewritten the query (predicate transitive closure)
– Some columns used as predicates hidden in views
– Recognize which statistics are appropriate in which
circumstances
• Significant effort even for expert
• Error prone
© IBM Corporation 2007
77
Run workload advisors (WSA)
© IBM Corporation 2007
78
Complete recollection, or repair?
© IBM Corporation 2007
79
RUNSTATS command
© IBM Corporation 2007
80
Table portion of command
RUNSTATS TABLESPACE DB2OSC.WSATS0
TABLE(DB2OSC.DSN_WSA_DATABASES)
COLUMN(NAME,SESSIONID)
TABLE(DB2OSC.DSN_WSA_CGFREQS)
COLUMN(TBCREATOR,TBNAME,COLGROUPCOLNO,SESSIONID)
TABLE(DB2OSC.DSN_WSA_KTGFREQS)
COLUMN(IXCREATOR,IXNAME,SESSIONID,KEYGROUPKEYNO)
TABLE(DB2OSC.DSN_WSA_SESSIONS)
COLUMN(SESSIONID)
TABLE(DB2OSC.DSN_WSA_COLGROUPS)
COLUMN(TBCREATOR,TBNAME,COLGROUPCOLNO,SESSIONID)
COLGROUP(COLGROUPCOLNO,SESSIONID,TBCREATOR,TBNAME)
TABLE(DB2OSC.DSN_WSA_TSS)
COLUMN(NAME,DBNAME,SESSIONID)
COLGROUP(DBNAME,SESSIONID)
COLGROUP(DBNAME,NAME,SESSIONID)
TABLE(DB2OSC.DSN_WSA_KTGS)
COLUMN(IXCREATOR,IXNAME,SESSIONID)
TABLE(DB2OSC.DSN_WSA_KEYS)
COLUMN(IXCREATOR,IXNAME,SESSIONID)
TABLE(DB2OSC.DSN_WSA_CGHISTS)
COLUMN(TBCREATOR,TBNAME,COLGROUPCOLNO,SESSIONID)
TABLE(DB2OSC.DSN_WSA_KTGHISTS)
COLUMN(IXCREATOR,IXNAME,SESSIONID,KEYGROUPKEYNO)
TABLE(DB2OSC.DSN_WSA_INDEXES)
COLUMN(TBCREATOR,TBNAME,SESSIONID)
TABLE(DB2OSC.DSN_WSA_TABLES)
COLUMN(CREATOR,NAME,TSNAME,DBNAME,SESSIONID)
COLGROUP(CREATOR,NAME,SESSIONID)
TABLE(DB2OSC.DSN_WSA_LITERALS)
COLUMN(TBCREATOR,TBNAME,SESSIONID,COLNO,VALUE)
COLGROUP(COLNO,SESSIONID,TBCREATOR,TBNAME,VALUE)
TABLE(DB2OSC.DSN_WSA_KEYTARGETS)
COLUMN(IXCREATOR,IXNAME,SESSIONID)
TABLE(DB2OSC.DSN_WSA_COLUMNS)
COLUMN(TBCREATOR,TBNAME,SESSIONID)
SORTDEVT SYSDA
SHRLEVEL CHANGE REPORT YES
© IBM Corporation 2007
81
Index portion of suggestion
RUNSTATS TABLESPACE DB2OSC.WSATS0
…
INDEX(DB2OSC.DSN_WSA_IDX_DBS,
DB2OSC.DSN_WSA_IDX_CGFS,
DB2OSC.DSN_WSA_IDX_KTGFS,
DB2OSC.DSN_WSA_IDX_SSNS,
DB2OSC.DSN_WSA_IDX_CGS,
DB2OSC.DSN_WSA_IDX_TSS,
DB2OSC.DSN_WSA_IDX_KTGS,
DB2OSC.DSN_WSA_IDX_KEYS KEYCARD,
DB2OSC.DSN_WSA_IDX_CGHS KEYCARD,
DB2OSC.DSN_WSA_IDX_KTGHS KEYCARD,
DB2OSC.DSN_WSA_IDX_IDXS,
DB2OSC.DSN_WSA_IDX_TABLES,
DB2OSC.DSN_WSA_IDX_LTRS KEYCARD,
DB2OSC.DSN_WSA_IDX_KTS KEYCARD,
DB2OSC.DSN_WSA_IDX_COLS KEYCARD)
SHRLEVEL CHANGE REPORT YES
© IBM Corporation 2007
82
Output looks complex…
• RUNSTATS commands are more targeted
– Only collect on tables used in queries
– Only collect column statistics on columns used as
predicates
– Collect frequency and histogram statistics based on
predicates which would benefit
© IBM Corporation 2007
83
Pause for questions on WSA?
© IBM Corporation 2007
84
Back to our regularly scheduled
programming…
© IBM Corporation 2007
85
More OSC GUI review
© IBM Corporation 2007
86
Open projects as tabs on top…
© IBM Corporation 2007
87
Project Navigator
© IBM Corporation 2007
88
Project navigator
• Navigate across projects
– As multiple projects opened, you can switch between
projects you’re working on
• Navigate within a project
– As you perform different tasks within a project, you
can navigate through the different tuning sections
© IBM Corporation 2007
89
Project Navigator
© IBM Corporation 2007
90
Project tabs
© IBM Corporation 2007
91
Project tabs
© IBM Corporation 2007
92
Project tabs
• Navigate within project
– As multiple projects opened, you can switch between
projects you’re working on
• Close sections of project you’re done with
– Click x on the tab to close selected tab
© IBM Corporation 2007
93
More OSC GUI review
© IBM Corporation 2007
94
Workload sources - cache
© IBM Corporation 2007
95
Cache statement view
• Cache statement view
–
–
–
–
Number of executions
Accumulated & average CPU time
Accumulated & average elapsed time
SQL text
• Cache as input provides more visibility into
how often SQL executing, cost of statements
© IBM Corporation 2007
96
Workload – cache statements view
© IBM Corporation 2007
97
Monitor workload
• Normal monitor workload
– All SQL based on certain criteria
– Useful to profile an application
• Which SQL running most frequently?
• Which queries consuming most elapsed / execution time?
• Exception monitor
– Workload generated based on exception criteria
• All SQL for package > x seconds elapsed
• All SQL for package > x seconds elapsed
© IBM Corporation 2007
Application tuning process
(revisit)
• What is application tuning?
– Identify what the application workload is
• Individual SQL statements
– Get an understanding of the applications behavior
• How often are individual SQL statements executing?
• What is the performance of individual SQL statements?
– Determine statistics for workload
• Workload statistics advisor
• More workload analysis features coming… (?)
– Remeasure workload
• Identify top tuning candidates
• Use query based tools to further analyze (next session)
© IBM Corporation 2007
98
99
Create normal monitor
© IBM Corporation 2007
100
Add monitor sources
© IBM Corporation 2007
101
Set monitor options
© IBM Corporation 2007
102
Monitor statement view
© IBM Corporation 2007
103
Exception monitor options
© IBM Corporation 2007
104
Application tuning
• Normal monitor allows for capture of all SQL
– Specify list of packages
• Prevents workload “pollution” which can occur with
statement cache
– Capture valuable execution information
– Ideally initially used in QA environment to profile /
tune application before production implementation
• “How often does this execute?”
• “How long does this run?”
• … you can have answers to these questions…
– Selectively use in production to develop application
profile and tune application
© IBM Corporation 2007
Application tuning process
(review)
• What is application tuning?
– Identify what the application workload is
• Individual SQL statements
– Get an understanding of the applications behavior
• How often are individual SQL statements executing?
• What is the performance of individual SQL statements?
– Determine statistics for workload
• Workload statistics advisor
• More workload analysis features coming… (?)
– Remeasure workload
• Identify top tuning candidates
• Use query based tools to further analyze (next session)
© IBM Corporation 2007
105
106
Which workload source to use?
• Which workload source provides…
–
–
–
–
–
Identification of all SQL statements executed
Accurate number of execution counts
Accurate accumulated and average execution performance
Provides runtime information for any source
Can re-measure and expect to see the same SQL again,
compare results
– Is not polluted with dynamic SQL which are not relevant to the
application?
• Workload monitoring (normal)
© IBM Corporation 2007
107
Tuning with OSC I
• End of OSC tuning Part I
– Show some demo if time permits
• Next up, Query tuning with OSC
–
–
–
–
–
Statistics Advisor
Query annotation
Query report
Visual Explain
Visual plan hint
© IBM Corporation 2007
108
®
Tuning with Optimization Service Center I
Patrick Bossman
bossman@us.ibm.com
© IBM Corporation 2007