IBM Software Group
DB2 10 and 11 for z/OS
Overview - SQL
October 15, 2015
UDUG
John Iczkovits
DB2 ATS (Advanced Technical Support)
iczkovit@us.ibm.com
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Disclaimer and Trademarks
Information contained in this material has not been submitted to any formal IBM review and is
distributed on "as is" basis without any warranty either expressed or implied. Measurements data
have been obtained in laboratory environment. Information in this presentation about IBM's
future plans reflect current thinking and is subject to change at IBM's business discretion. You
should not rely on such information to make business plans. The use of this information is a
customer responsibility.
IBM MAY HAVE PATENTS OR PENDING PATENT APPLICATIONS COVERING SUBJECT
MATTER IN THIS DOCUMENT. THE FURNISHING OF THIS DOCUMENT DOES NOT IMPLY
GIVING LICENSE TO THESE PATENTS.
TRADEMARKS: THE FOLLOWING TERMS ARE TRADEMARKS OR ® REGISTERED
TRADEMARKS OF THE IBM CORPORATION IN THE UNITED STATES AND/OR OTHER
COUNTRIES: AIX, DATABASE 2, DB2, Enterprise Storage Server, FICON, FlashCopy, Netfinity,
RISC, RISC SYSTEM/6000, System i, System p, System x, System z, IBM, Lotus, NOTES,
WebSphere, z/Architecture, z/OS, zSeries
The FOLLOWING TERMS ARE TRADEMARKS OR REGISTERED TRADEMARKS OF THE
MICROSOFT CORPORATION IN THE UNITED STATES AND/OR OTHER COUNTRIES:
MICROSOFT, WINDOWS, WINDOWS NT, ODBC, WINDOWS 95
For additional information see ibm.com/legal/copytrade.phtml
2
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
DB2 z/OS Availability Summary
3
Version
PID
General
Availability
Marketing
Withdrawal
End of
Service
5
5655-DB2
June 1997
December
2001
December
2002
6
5645-DB2
June 1999
June 2002
June 2005
7
5675-DB2
March 2001
March 2007
June 2008
8
5625-DB2
March 2004
September
2009
April 2012
9
5635-DB2
March 2007
December
2012
June 2014
10
5605-DB2
October 2010
July 2015
September
2017
11
5615-DB2
October 2013
http://www.ibm.com/software/data/support/lifecycle/
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
DB2 10 SQL Enhancements
Access Currently Committed
Greater Timestamp Precision
Extended SQL Procedure
Language (SQL PL)
Timestamp with Time Zone
– Enhanced Native SQL
Procedure
– SQL Scalar Functions
– SQL Table Functions
Function Modifications
Stored Procedure
Enhancements
64-bit ODBC driver for z/OS
Extended Indicator Variables
More OLAP Specifications
4
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
SQL Enhancements Business Values
Possibility of enhanced concurrency using the new Access
Currently Committed feature.
Expanded use of the easy to use SQL Procedure for scalar
functions and table functions.
Simplified coding of INSERT and UPDATE statements by using
the new enhanced indicator variables. Eliminates/reduces need
to code multiple statements when dealing with multiple values
that may be provided or not.
Several new rolling OLAP functions.
Greater timestamp precision and inclusion of time zone with
timestamp.
Ability to run 64-bit ODBC applications on z/OS.
5
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Access Currently Committed
Pre-DB2 10 Problem/Requirements:
– Reads acquire locks on data
– Overhead, contention, concurrency issues
– ISOLATION(UR) avoids contention, but does not return committed
data
DB2 10 Solution:
– Return currently committed data without waiting for locks.
– Ability can be specified on:
• BIND/REBIND PACKAGE or PLAN
• SQL PREPARE statement
• CREATE/ALTER PROCEDURE (native SQL) or
FUNCTION (SQL Scalar)
6
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Access Currently Committed…
DB2 10 Solution Restrictions:
– Support for uncommitted INSERT and DELETE
– Reader will still wait for uncommitted UPDATE
– Not applicable when:
• LOCK TABLE IN EXCLUSIVE used
• Lock holder is performing mass delete
• If lock holder escalates
DB2 10 Solution Requirements:
– Universal Table Space
– Row or Page-level locking
• IRLM will remember first 8 locked rows when using page-level locking.
– NFM
7
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Access Currently Committed…
SELECT bound with ISOLATION(CS) or ISOLATION(RS) and
CONCURRENTACCESSRESOLUTION(USECURRENTLYCOMMITED)
Syntax specifications:
– BIND/REBIND PACKAGE and PLAN:
• Specify option CONCURRENTACCESSRESOLUTION
(USECURRENTLYCOMMITTED) or (WAITFOROUTCOME)
– PREPARE attribute string:
• Specify USE CURRENTLY COMMITTED or WAIT FOR OUTCOME
– CREATE/ALTER PROCEDURE or FUNCTION
• Specify option CONCURRENTACCESSRESOLUTION
(USECURRENTLYCOMMITTED) or (WAITFOROUTCOME)
• Can be specified for new and existing Native SQL Procedures
– DRDA supports use of CONCURRENTACCESSRESOLUTION bind
option, and can be used for SQLJ applications.
8
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Access Currently Committed…
SELECT not blocked by INSERT:
– Pre-DB2 10 SELECT waits and eventually a row returned or times out.
– DB2 10 can see row being inserted is not committed, and would
immediately skip row.
– Relation to SKIPUNCI ZParm for uncommitted INSERTs:
• Bind option or PREPARE attribute value controls outcome when specified
• SKIPUNCI value controls outcome when not defined
SELECT not blocked by DELETE:
– Pre-DB2 10 SELECT waits and eventually no row is returned or times out.
– Currently committed row, including any LOB or XML data is returned, until
DELETE is committed.
9
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Access Currently Committed…
Consider as second generation lock avoidance.
– DB2 may revert to unconditional locking.
– Alternative to DB2 9 “SKIP LOCKED DATA” option, which
ignores row.
– Alternative to ISOLATION(UR)
QMF global variable DSQEC_CON_ACC_RES can be
used to support feature.
10
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Extended SQL Procedure Language (SQL PL)
Enhanced Native SQL Procedures capabilities:
– Can define an SQL parameter or SQL variable as a distinct type or
XML data type XML
– Makes it easier to port applications from other DBMS
– CREATE statement for a native SQL procedure can be embedded
in an application program
– ALTER statement for a native SQL procedure can be embedded in
an application program, even if the ADD or REPLACE VERSION
clause is included. This was not available in DB2 9.
– DB2 10 NFM
11
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Extended SQL Procedure Language (SQL PL)…
SQL Scalar Function:
– DB2 10 NFM introduces two types of SQL Scalar Functions
• Inline and
• Non-inline
– Inline SQL scalar functions are similar in structure to SQL scalar
functions prior to DB2 10 with a single RETURN statement, and
can use the XML data type in DB2 10 NFM.
– Non-inline SQL scalar functions have added benefits:
• Can contain logic using SQL statements and SQL control statements like
those used in native SQL procedures. Will create a package.
• Allow for multiple versions, where one is considered the “active” version.
– Initial version defined by CREATE FUNCTION … VERSION id
– Subsequent versions defined by ALTER FUNCTION… ADD VERSION id
– Activate version using ALTER FUNCTION… ACTIVATE VERSION id
• Versions can easily be replaced or dropped.
12
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Extended SQL Procedure Language (SQL PL)…
SQL Scalar Function… additional Non-inline benefits:
– Packages can be rebound with:
• ALTER FUNCTION … REGENERATE – includes the SQL control
statements and SQL statements in the function; mainly used in conjunction
with DB2 maintenance.
• REBIND PACKAGE – rebinds only the SQL statements in the function;
usually used to generate new access paths.
– Allow for debugging using the Unified Debugger, like what is used for
debugging native SQL procedures.
– Can replace the use of external scalar functions by:
• Simplifying development
• Address performance considerations
• Providing a simpler change management capability that can be used to maintain
multiple versions.
– Can deploy the function by using the BIND PACKAGE… DEPLOY
– Makes it easier to port applications from other DBMS.
– DB2_LINE_NUMBER from GET DIAGNOSTICS, introduced in DB2 9, will
return the line number where an error is encountered with the function.
13
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Extended SQL Procedure Language (SQL PL)…
SQL Scalar Function… Examples:
Inline: Defines a scalar function that returns a duration in years.
Reference Date
14
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Extended SQL Procedure Language (SQL PL)…
SQL Scalar Function… Examples:
Non-inline: Defines a scalar function that reverses a string.
15
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Extended SQL Procedure Language (SQL PL)…
SQL Table Function:
– DB2 10 NFM introduces ability to create SQL table functions.
– Written using SQL statements and includes a single RETURN
statement that returns a single result table (a set of rows).
– Parameters can be defined as:
• Built-in or distinct data types
• TABLE LIKE table/view-name AS LOCATOR (transition table)
– Can be defined to behave as parameterized views
– CREATE and ALTER statements can be embedded in an
application program or issued interactively.
16
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Extended SQL Procedure Language (SQL PL)…
SQL Table Function… Example:
Defines a table function that returns a list of employees for a department.
17
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
SQL Procedure Language Cross Releases
PM13525 enables DB2 to be more tolerant of SQL
Stored Procedures created across DB2 Versions
This helps with the following conditions:
– A procedure developed on DB2 10 and DEPLOYed on DB2 9
– When a DB2 10 created or regenerated procedure is run on
DB2 9 and is rebound or regenerated
18
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Extended Indicator Variables
Pre-DB2 10 Problem/Requirements:
– No way to tell DB2 what value to use in an INSERT, UPDATE or
MERGE when values for all the columns are not specified.
– Need to define all possible SQL combinations… very tedious
– Very low use of dynamic statement cache.
DB2 10 Solution:
– DB2 10 NFM introduces extended indicator variables to specify
what value should be used for columns when a value is not
provided in an INSERT, UPDATE or MERGE.
• New values used for existing indicator variables.
– Simplifies application development… no need to define all possible
SQL combinations
– Improves overall performance
19
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Extended Indicator Variables
Implementation:
– EXTENDEDINDICATOR YES/NO option on BIND/REBIND PACKAGE
• NO is the default for BIND
• Existing setting is default for REBIND
• New EXTENDEDINDICATOR column added to SYSPACKAGE
– WITH EXTENDED INDICATORS or WITHOUT EXTENDED
INDICATORS attribute string on PREPARE
• Default - WITHOUT EXTENDED INDICATORS
– Various methods available for implementing support with JDBC, SQLJ
and ODBC applications.
– Extended indicator variable values:
• -5 specifies the DEFAULT value – set column to the default value
• -7 specifies the UNASSIGNED value - treat as column was not specified
20
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Extended Indicator Variables … Examples
Assume the following table definition:
Request DB2 to use a DEFAULT value be used for an INSERT:
– DB2 will use the default value for C1.
Request DB2 to consider a value UNASSIGNED for an UPDATE:
– DB2 will leave the value of C1 untouched and update C2.
– UPDATE with all SET columns using -7 results in -20434.
21
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
More OLAP Specifications
Moving Sum and Moving Average
– New DB2 10 NFM specifications (aggregate functions) that compute a
single value for the current row based on some or all the rows in a defined
group.
– Supports cumulative sums and moving averages by using a window.
– A window can specify three optional components:
•
•
•
•
Partitioning of the result table using PARTITION BY clause (in DB2 9).
Ordering of rows within a partition using ORDER BY clause (in DB2 9).
Aggregation group by using ROW or RANGE clause
Two mandatory components need to be specified for an aggregation group.
– Keyword ROWS to indicate a physical group … or
– Keyword RANGE to indicate a logical group … and
– The starting and ending row of the aggregation group
– Cannot be used within an XMLQUERY, XMLEXISTS, or as an argument of
an aggregate function.
– New -20117 SQL code to indicate an invalid window
22
SQL
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
More OLAP Specifications… Examples
Assume a SALES_HISTORY table containing the following.
– Each TERRITORY has 5 rows.
TERRITORY
EAST
WEST
EAST
WEST
EAST
WEST
EAST
WEST
EAST
WEST
23
SQL
MONTH
200910
200910
200911
200911
200912
200912
201001
201001
201002
201002
SALES
10.00
8.00
4.00
12.00
10.00
7.00
7.00
11.00
9.00
7.00
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
More OLAP Specifications… Examples
TERRITORY
EAST
EAST
EAST
EAST
EAST
WEST
WEST
WEST
WEST
WEST
24
SQL
MONTH
200910
200911
200912
201001
201002
200910
200911
200912
201001
201002
SALES
10.00
4.00
10.00
7.00
9.00
8.00
12.00
7.00
11.00
7.00
MOVING_AVG
10.00
7.00
8.00
7.00
8.66
8.00
10.00
9.00
10.00
8.33
Results:
1. Grouped by TERRITORY
2. Ordered by MONTH
3. MOVING_AVG values are
computed on the average
SALES value of the current
row with the prior 2 rows
within the TERRITORY.
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
More OLAP Specifications… Examples
“ROWS
UNBOUNDED
PRECEDING” is
optional. Results
are the same
without it.
TERRITORY
EAST
EAST
EAST
EAST
EAST
WEST
WEST
WEST
WEST
WEST
25
SQL
MONTH
200910
200911
200912
201001
201002
200910
200911
200912
201001
201002
SALES
10.00
4.00
10.00
7.00
9.00
8.00
12.00
7.00
11.00
7.00
CUMULATIVE_SUM
10.00
14.00
24.00
31.00
40.00
8.00
20.00
27.00
38.00
45.00
Results:
1. Grouped by TERRITORY
2. Ordered by MONTH
3. CUMULATIVE_SUM
values are computed on
the sum of the SALES
value of the current row
with the prior row(s) within
each TERRITORY.
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Greater Timestamp Precision
Number of digits for the fractional second in
a timestamp extended
– The DB2 9 default of 6 digits remains
•
•
TIMESTAMP is the same as TIMESTAMP(6)
String representation:
yyyy-mm-dd-hh.mm.ss.nnnnnn
– Range supported in DB2 10 NFM is 0 to 12 digits
•
•
E.g. TIMESTAMP(12) is the maximum
String representation:
yyyy-mm-dd-hh.mm.ss.nnnnnnnnnnnn
– Other capabilities like timestamp duration and CURRENT TIMESTAMP
extended to support new precision capability
– Can be altered to higher precision.
•
26
Will fail with SQLCODE -190 if lowered.
SQL
Example: TS(6) to TS(10)
Example: TS(10) to TS(6)
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Timestamp with Time Zone
New data type TIMESTAMP WITH TIME ZONE
– SET SESSION TIME ZONE or rely on IMPLICIT_TIMEZONE (DECP)
– Year.month.day.hour.minutes.seconds +-timezone
CREATE TABLE TABLE1 (C1 TIMESTAMP WITH TIME ZONE, C2 INTEGER);
INSERT INTO TABLE1 VALUES ( '2009-11-05-08.00.00-08:00', 1);
INSERT INTO TABLE1 VALUES ( '2009-11-05-11.00.00-05:00', 2);
INSERT INTO TABLE1 VALUES ( '2009-11-05-10.00.00', 3);
Comparison takes place after converting internally to UTC
– '2009-11-05-08.00.00 -08:00‘ is converted to ‘2009-11-05-16.00.000000’
– ‘2009-11-05-11.00.00 -05.00’ is converted to ‘2009-11-05-16.00.000000’
• SELECT C2 FROM TABLE1 WHERE C1 = '2009-11-05-10.00.00 -06:00';
• would retrieve all 3 of the above
– ‘-6:00’ is the timezone, US Central which is UTC -6, except during daylight savings time
27
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Timestamp with Time Zone…
New DSNHDECP parameter IMPLICIT_TIMEZONE.
– Determines implicit time zone to be associated with DB2 table columns
and routine parameters declared with TIMESTAMP WITH TIME ZONE,
when a time zone is not provided.
– Defined in new IMPLICT TIME ZONE field on installation panel
DSNTIP4 (Application programming defaults panel 2)
– Values:
• CURRENT (default) = uses setting of the CURRENT TIME ZONE special
register
• SESSION = uses setting of the SESSION TIME ZONE special register
• -12:59 to +14:00 = uses the value of the string. Format of value is th:tm,
where th is the time zone hour (-12 to +14), and tm is the time zone minute
(00 to 59). Value is an offset of hours and minutes from UTC.
28
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Timestamp with Time Zone
– Definitions:
• CURRENT TIME ZONE special register: ( data type = DECIMAL(6,0) )
– Contains the difference between UTC and local time at the current server. The
difference is represented by a decimal number in the format ttmmss.
> tt = number of hours between -24 and 24 exclusive
> mm = number of minutes
> ss = number of seconds
– Example:
> CURRENT TIME = 08.30.00 US Central time
> CURRENT TIME ZONE = -60000.
(Except during daylight savings time, then -50000.)
• SESSION TIME ZONE special register: ( data type = VARCHAR(128) )
–
–
–
–
29
Identifies the time zone of the application process in the format tt:tm
tt = time zone hour offset between -12 and +14
tm = time zone minutes offset between 0 and 59
Example (using above current time) = -06:00 (Follows same DST exception.)
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Timestamp Considerations
Some timestamp formats are no longer accepted in DB2 10
– These were allowed in previous release
– However they were not documented as valid
– One case (YYYY-MM-DD-HH:MM:SS.nnnnnn) is noted in the MPW
Consolidated Checklist
PM29124 added the BIF_COMPATIBILITY ZParm
– See APAR for settings to restore DB2 9 function behavior
– Also see closing Hold Data text for option to revert timestamp
behavior
PM48741 adds IFCID 366 support
– While using BIF_COMPATIBILITY options to revert behavior
– This IFCID reports application incompatible behavior
30
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Functions Compatibility
Functions Modified for Competitive Compatibility
– See PM40724
• Enabling APAR PM38326
– Functions added / modified
• Modified
–
–
–
–
–
LTRIM
RTRIM
REPLACE
ROUND
TRUNCATE or TRUNC
• Added
– TRIM
PM70455 available for DB2 9
– Begins using IFCID 366 to report functions & timestamp usage
considered incompatible in DB2 10
31
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Stored Procedure Enhancements
PM37668
– Stored Procedure RETURN TO CLIENT cursor support
• Returns result set to client from a nested stored procedure
• Result set is invisible to intermediate stored procedures
– For NSPs RETURN TO CLIENT cursors
• Duplicate cursor instance support (see APAR for details)
• Becomes a result set cursor
PM53243
– Stored procedure monitoring improvements
– Begin / End for each SP or UDF invocation (IFCID 233)
– Execution details (IFCID 380 & 381)
– Statement level details (IFCID 497, 498, 499)
32
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
64-bit ODBC driver for z/OS
Provides the ability for 64-bit ODBC applications to run on z/OS, and take advantage of the new
addressability.
– In DB2 9 with PK83072.
New driver benefits:
• runs in AMODE(64)
• reduces virtual storage constraint
• can accept 64-bit user data pointers
• access user data above the 2GB bar in the application address space
• XPLINK only
• Shipped in addition to the 31-bit ODBC driver.
31-bit ODBC driver is XPLINK and non-XPLINK. The APIs for existing 31-bit applications have not
been changed, and continue to work using the 31-bit ODBC driver.
Consider migrating 31-bit applications to 64-bit, if the application can take advantage of more than
2GB of memory.
– This is a migration
Migrating an ODBC 31-bit application to a 64-bit application
The XPLINK driver is recommended to enhance performance, but only if your ODBC application
uses XPLINK code exclusively.
For more information: DB2 ODBC application requirements
33
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
DB2 10 pureXML, The Sequel
DB2 9 pureXML Recap
Multi-Versioning Support
Subdocument Update
Validation Improvements
CHECK & REPAIR
Native SQL Procedures &
UDFs
XML Date and Time
Binary XML
34
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
pureXML Business Value
Enable / support modern application requirements
Support for the W3C standard for information
exchange
Store native XML or publish XML from relational
Performance based architecture to minimize parsing
Cost effective exploitation of specialty processors
Familiar SQL and Utilities for use and administration
XML Schema support built in
35
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
DB2 9 pureXML Recap...
Extends XML Publishing (introduced in DB2 V8)
– Create XML documents from relational information
DB2 for z/OS became a hybrid relational and XML engine.
New XML storage architecture using auxiliary XML spaces.
Native XML data type
– XML stored in parsed node format
– DB2 ensures XML is well formed
– UTS, RRF, 16KB page
– Stored in Unicode
XPath introduced into SQL to administer and manipulate XML
– Use familiar CREATE, ALTER, DROP DDL for administration
• XPath used in CREATE INDEX to identify the Element or Attribute
– Use familiar SELECT, INSERT, DELETE, UPDATE DML
XML Schema Repository (DSNXSR)
36
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
DB2 9 pureXML Recap...
Key Functions
– XMLQUERY to SELECT specific elements or attributes from a document
– XMLEXISTS to apply a WHERE predicate against elements or attributes
– XMLSERIALIZE to construct the document in a external string format
– XMLPARSE to parse an XML document into the internal node format
– XMLTABLE to externalize an XML document in relational row / column format with
SQL data types
• Used for XML Decomposition
XML
XML
INTEGER
TIMESTAMP
CHAR
37
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Multi-Versioning (MV) Support...
Requires the base table to be a Universal Tablespace (UTS)
Existing tables with XML columns must be dropped and
recreated after DB2 10 NFM for MV format.
MV format required for several V10 features:
– XMLMODIFY
BTD
– Temporal data (“AS OF”)
SQL
– Currently Committed
– SELECT FROM OLD TABLE (V9 feature)
The MV format reduces locking
– V9
• XML data is not kept with base row data in the work files
• Therefore a lock is maintained on the XML row until it is processed after
work file
• MV eliminates the need for this lock except for UR Readers (DOCID
lock). Avoids a UR Reader from getting an incomplete document.
38
XML
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Multi-Versioning (MV) Support...
DB2 9 pureXML Table Formats
Base Table
Relational DOCID
Columns
XML Table(s)
DOCID MIN_
NODEID
XMLDATA
XML
Indicator
XML
Indicator
DOCID MIN_
NODEID
XMLDATA
DOCID:
DOCID:
Colname: DOCID
Colname: DB2_GENERATED_DOCID_FOR_XML
Data Type: BIGINT
Data Type: BIGINT
One per base table that has 1 or more XML column(s) MIN_NODEID:
Colname: MIN_NODEID
XML:
Data Type: VARBIN(128)
Colname: DDL XML column name
XMLDATA:
Data Type: VARBIN(6)
Colname: XMLDATA
One per XML column associated with the base table
Data Type: VARBIN(15850)
39
XML
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Multi-Versioning (MV) Support...
DB2 10 pureXML Table Formats
Base Table
XML Table(s)
Relational DOCID
Columns
DOCID MIN_
NODEID
XMLDATA
XML
Indicator
START_
TS
END_
TS
AAAAAAAA FFFFFFFF
XML
Indicator
DOCID MIN_
NODEID
XMLDATA
AAAAAAAA
BBBBBBBB
START_
TS
END_
TS
AAAAAAAA FFFFFFFF
BBBBBBBB
BBBBBBBB FFFFFFFF
DOCID:
DOCID:
Colname: DB2_GENERATED_DOCID_FOR_XML
Colname: DOCID
Data Type: BIGINT
Data Type: BIGINT
One per base table that has 1 or more XML column(s) MIN_NODEID:
XML:
Colname: MIN_NODEID
Colname: DDL XML column name
Data Type: VARBIN(128)
Data Type: VARBIN(14). Now includes 8 byte TS.
XMLDATA:
One per XML column associated with the base table
Colname: XMLDATA
Data Type: VARBIN(15850)
40
XML
START_TS:
Colname: START_TS
Data Type: BINARY(8)
END_TS:
Colname: END_TS
Data Type: BINARY(8)
RRF makes physical column order:
DOCID, START_TS, END_TS,
MIN_NODEID, XMLDATA
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Subdocument Update with XMLMODIFY...
DB2 9 pureXML supported the updating of an XML column
– But the entire column needed to be replaced
DB2 10 delivers the XMLMODIFY function to:
– Insert nodes within an existing document
– Replace existing nodes of a document
– Delete nodes from an existing document
Invoked via UPDATE....SET SQL
Requires MV format
One updater at a time for a document
– Concurrency control by the base table
– Document level lock to prevent UR reader
41
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Subdocument Update with XMLMODIFY...
Only changed rows in XML table are updated
UPDATE...SET...XMLMODIFY with “insert node”
– insert node (must be in lower case)
• Adds a node to an existing document
• Example: Insert a new PROJ element node (case matters)
– With a PROJNO attribute of “z02” and NAME attribute of “pureXML
V10 Updates”
– After an existing PROJ element with a PROJNO attribute of “z01”
– A -16085 will be returned if a referenced after / before node cannot be
located or multiple nodes qualify the XPath expression
42
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Subdocument Update with XMLMODIFY...
UPDATE...SET...XMLMODIFY “insert node” (continued)
– Positioning options:
•
•
•
•
after <XPath identifying an existing node>
before <XPath identifying an existing node>
as first into
as last into (default when “into” is coded)
UPDATE...SET...XMLMODIFY “replace value of node”
• Replaces the value of an existing element (text node) or attribute
• Example: The value of an existing DEPTNO attribute of a DEPT element
is updated to a value of “R32”
• A -16085 will be returned if multiple nodes qualify the XPath expression
43
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Subdocument Update with XMLMODIFY
UPDATE...SET...XMLMODIFY “delete node”
– Removes an existing node from a document
– Example: The PROJ element with a PROJNO attribute
of “z01” is removed from the document
– The returned SQLCODE may not be a good indication
of what has taken place
• If the WHERE clause qualifies a row
• But the delete expression does not match any nodes of a document
• A SQLCODE of +0 is returned and there is no indication of the number
of nodes removed
• Also, there is no reporting of the number of nodes impacted
• If multiple nodes qualify, they will be deleted
44
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Validation Improvements...
XML Column Type Modifier
– Allows a “constraint” type of definition on an XML column via CREATE or
ALTER TABLE
•
Several schemas may be specified in the type modifier
–
–
This supports the evolution of a schema
See the XML Schema Versioning topic
– All documents INSERTed, UPDATEd, or LOADed in the column are
validated to conform to the specified schema(s)
– Adding an XML Schema into XML Column Type Modifier via an ALTER
may place the XML Table Space into CHKP status
•
When ALTERing the Type Modifier, include all Schemas to be used
–
–
•
Including those already in use
Let DB2 figure out the delta
For CHKP status
–
>
–
45
XML
Run CHECK DATA to validate schema conformance
Validates that existing document conform to specified schema(s)
Note: Removing an XML Column Type Modifier will not remove a CHKP status
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Validation Improvements...
XML Column Type Modifier (continued)
– Add
– Remove
Change from
Type Modifier
To Type
Modifier
XML Table Space Impact
No modifier
XMLPOT1
Check pending
XMLPOT1
XMLPOT1,
XMLPOT2
None
XMLPOT1
XMLPOT2
Check pending
XMLPOT1,
XMLPOT1
Check pending
No modifier
None
XMLPOT2
XMLPOT1
46
XML
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Validation Improvements...
XML Schema Versioning
– Multiple versions of a schema can be stored in the XML Schema Repository
– An XML Schema is identified by:
See “How DB2 chooses
•
•
•
Target Namespace
Schema Location (also known as Schema Location Hint)
Schema Name
an XML schema from an
XML type modifier” in
the XML Guide
– When there is no Schema Location and multiple schemas are found then the latest
registered schema is chosen
– Administration
•
•
•
Use the same stored procedures provided in DB2 9 to administer the XSR
Use Data Studio
CLP
– If setting up the XML Schema Repository under DB2 10, see installation job
DSNTIJNX & DSNTIJRT if migrating from V8. Only DSNTIJRT from V9.
47
SYSIBM.XSROBJECTS
One row per registered XML Schema
SYSIBM.XSROBJECTCOMPONENTS
One row per document in an XML Schema
SYSIBM.XSROBJECTHIERARCHIES
Records the XML Schema document relationships
SYSIBM.SYSXMLSTRINGS
(DSNDB06)
Not part of the XSR, but contains the decode
information for schema namespace & location
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Validation Improvements...
XML Schema Versioning (continued)
– V9 XML Schemas still work for V10
– Validation process changes
•
–
–
–
–
•
–
–
•
SYSFUN.DSN_XMLVALIDATE V9 UDF is replaced by V10 BIF SYSIBM.DSN_XMLVALIDATE
in CM9 or NFM
Increases validation document size from 50MB to 2GB – 1.
No application changes are needed if DSN_XMLVALIDATE is not qualified.
SYSFUN.DSN_XMLVALIDATE is ALTERed in CM9, invalidating packages
Available in DB2 9 with PK90032 & PK90040 (recommended for migration from DB2 9)
If DSN_XMLVALIDATE is used for a column with a XML type modifier
If the BIF specified schema is also defined in the type modifier, then this directs DB2 to
use the BIF specified schema
If the BIF specified schema is not defined in the type modifier, then the document must
conform to the BIF schema AND the selected schema from the type modifier
100% zIIP / zAAP redirectable, if available
– z/OS XML System Services is used for validation
•
•
48
This produces different error messages from those in DB2 9.
“LOCATION x”, points to the byte offset location in the document
XML
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Validation Improvements...
What XML Schema was used to validate a document?
– The XMLXSROBJECTID function will return the internal identifier for the
XML Schema from the XML column
•
XMLXSROBJECTID(xml_column)
– The text of the schema name can be retrieved from XSR
– The IDs of the XML Namespace and Schema Location Hint can also be
retrieved from the XSR (SYSIBM.XSROBJECTS)
•
The text of the XML Namespace and Schema Location Hint is available in the
catalog (SYSIBM.SYSXMLSTRINGS)
The XMLXSROBJECTID function retrieves the
XML Schema ID from the documents in the
XML_TIME_ACCOUNT column.
These IDs are used to find the XML Schema
Names (XSROBJECTNAME) from the
SYSIBM.XSROBJECTS table.
The Namespace and Location IDs are
retrieved from the XSROBJECTS table and
the description (STRING) retrieved from the
SYSIBM.SYSXMLSTRINGS table.
(LOJ is used because Namespace and
Location are optional)
49
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
CHECK DATA & REPAIR...
CHECK DATA
– DB2 9
• CHECK DATA utility does not cover inconsistency inside XML
data
• NODEID index consistency with XML TS (CHECK INDEX)
• Base table consistency with NODEID index (CHECK DATA)
– DB2 10 CHECK DATA does:
• Base table consistency with NODEID index (CHECK DATA)
• NODEID index consistency with XML TS (V9 CHECK INDEX)
• Check document structure consistency for each document
• Schema validation if column(s) have a type modifier
– SHRLEVEL REFERENCE moves invalid documents to an
automatically created exception table
> With XMLERROR INVALIDATE or AUXERROR INVALIDATE
– SHRLEVEL CHANGE will generate REPAIR statements
50
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
CHECK DATA & REPAIR...
CHECK DATA (continued)
– DB2 10 CHECK DATA options:
• INCLUDE XML TABLESPACES (ALL | specify XML table space or column)
– XMLSCHEMA as part of the XML table space or column spec does schema checking
– XMLERROR INVALIDATE (SHRLEVEL REFERENCE) removes nonconforming XML
documents to an exception table
> The base table space is places in AUXW status
– SHRLEVEL CHANGE generates REPAIR LOCATE..DOCID...DELETE statements
> The base table space status is not changed
• SCOPE XMLSCHEMAONLY for schema validation only
• SCOPE PENDING (default)
– Will verify the NODEID index
– If INCLUDE XML TABLESPACES is specified, all XML Schema validation is also done
• EXCEPTIONS <count> does not apply to XML
– For MV format, only the latest version of a document is checked
– Use CHECK INDEX to verify XML Value Indexes (user created XML
indexes)
– When table version is involved, the REPAIR statements from SHRLEVEL
CHANGE must be used
• This avoids versioning
51
XML
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Native SQL Procedure & UDF Enhancements
XML is now valid as a passed variable
XML can be a data type of a declared variable
With these enhancements parsed XML can be processed within
the routine without repetitively reparsing the serialized XML
string
This snippet of code shows an example of a
DECLARE statement for the XML data type for a
case when XML is received as a CLOB (DB2 10
could pass the document in to the procedure as
type XML)
The body of the Native SQL Procedure then uses
the XMLPARSE function to store a document in
the variable.
Subsequent use of the Parsed_XMLDOC variable
does not require parsing.
52
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
XML Date & Time Support
DATE and TIME are not supported types in DB2 9 XPath
DB2 10 provides
–
–
–
–
–
–
–
xs:dateTime
xs:time
xs:date
xs:duration
xs:yearMonthDuration
xs:dayTimeDuration
Several functions (fn) for comparison, value extraction, arithmetic operations
Implicit time zone will be UTC if not specified
These types can be used in the creation of XML Value Indexes
53
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Binary XML & XQuery
DB2 9 supported textual XML as input / output
– This may cause the application to perform some additional
parsing
DB2 10 also supports binary XML as input / output
– Smaller in size (up to 46%)
– More efficient INSERT
• CPU (up to 30%)
• Elapsed (up to 50%)
Binary XML used by default in the V10 JDBC Driver
UNLOAD / LOAD support
Validation supported with PM53282 /
– See UA63422 / UA65591 for performance enhancement
PM47618 added Embedded XQuery support
54
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
DB2 10 LOB Enhancements
Inline LOBs
Benefits
Inline LOB DDL
Inline LOB Support
Streaming LOB Data
55
LOB
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Business value statements
The use of inline LOBs reduces I/O & CPU
consumption since LOB data can now be stored in
the base table
This enables new workload that employs LOB data
Eases LOB handling and management
56
LOB
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Benefits of Inline LOBs
Reduces auxiliary table storage, CPU, and virtual
storage potential
Inline portion of LOB compressible even though
LOBs are not compressed
Inline portion of LOB can be used in index on
expression
– For example, when using spatial data the index portion of the
spatial object can reside in the row
Inline LOBs stored in the base table can achieve
similar performance characteristics as VARCHAR
columns
57
LOB
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Inline LOBs...
LOB data can also be split
AUX table
– Entirely or partially
• Split LOB
Prefix Fixed Cols
Varchar
Indicators
InLine
Varying Length Cols Length
AUX
ID
CLOB
– DASD storage associated with small LOBs may be reduced
– Avoids I/O’s associated with LOB table spaces
– Base table can become larger.
• SQL limited to non-LOB columns may be impacted
58
LOB
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Inline LOB DDL…
Pre-requisites
– DB2 10 NFM
– RRF
– Default value of other than NULL or empty string is supported
– Define length of inline portion of LOB
• DDL
– Default to ZPARM
– Valid range 0 – 32680 bytes
• New ZParm LOB_INLINE_LENGTH
– Default 0 bytes
> No inline length. Stored entirely in AUX
– Range 0 – 32680 bytes
– Restrictions
• Cannot alter shorter if used in index on expression or shorter than
default data value
59
LOB
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Inline LOB DDL…
Create table with LOB column & Inline portion
Alter table INCREMENT length of Inline LOB portion
Alter to larger size is an immediate change. Base
table space is put in AREO* status. If rows exist at
time of change, length change is materialized upon
REORG
60
LOB
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Inline LOB DDL…
Alter table DECREMENT length of Inline LOB portion
– Hard REORG pending status (REORP)
Alter add a lob column with inline portion
– AREO* status for table space
61
LOB
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Additional LOB Flexibility
AUX (and XML) table spaces & dependent indexes can be
created DEFINE NO
– Defers create time to first INSERT operation
– Improves application installation time
– Simplifies/improves backup as empty data sets don’t have to be
backed up
LOAD/UNLOAD LOBs along with non-LOB columns
– Spanned record support UTL
62
LOB
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Streaming LOB (& XML)…
LOB streaming first introduced in DB2 V8
– Allowed programs to read LOB objects from applications
without knowing their size first
– Streaming occurs to DDF (out), not to the engine (in)
– Had to materialize it entirely before sending to DB layer
In DB2 10 LOBs will be streamed to the engine from DDF without
needing to materialize it in memory
Customers who use LOB & XML objects in DB2 10 will use fewer
resources
– DB2 10 eliminates materialization for LOB (>2MB) and XML
objects (>32KB)
– Class 2 CPU time reduced
– Virtual storage reduction
63
LOB
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Streaming LOB (& XML)
Streaming between DDF and DBM1 for LOB/XML
objects occurs in the following situations
– INSERT, UPDATE
– Limited to max 1 LOB or 1 XML column per row
– LOAD utility using file reference variables
– When CCSID conversion is required by the LOAD utility
– When inserting XML data in a DRDA® environment at the
remote server
64
LOB
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
DB2 11 Serviceability
Work File Space
Instrumentation
Maximum Query Sort Memory
Data Set Maximum
Profile Expansion
Directory SELECT Access
zIIP Expansion
IFCID Changes
NOTE: Only listing items not
previously covered
ACCESS DB
Client Special Registers
Global Variable
DB2 Supplied Procedure
Changes
RBA / LRSN Expansion
65
SVC
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Serviceability Business Value
Controls are added to manage, report, and preserve
critical DB2 resources to further support expanded
and mixed workloads.
66
SVC
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Work File Space Instrumentation ...
New System Parameters for Work File alerts
– See Migration presentation
– WFSTGUSE_AGENT_THRESHOLD
• Agent level warning
• Percentage of work file space in use by an agent
• Message DSNI052I issued when exceeded
– Once per unit of work
– WFSTGUSE_SYSTEM_THRESHOLD
• System level warning
• Percentage of work file space in use by the system
• Message DSNI053I issues when exceeded
– With PM94722, this message is not repeated for 5 minutes
– If WFDBSEP=YES
• The percentages are of the segregated temporary spaces
• Messages indicate if due to DGTTs, Work Files, or Both
67
SVC
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Work File Space Instrumentation
New work file ZParms for thread & system usage warnings
MIG
For WF the space map pages are updated under an S Latch
– Reducing space map contention
IFCIDs
–2
• Statistics Class 1
• MXDTCACH storage use / shortages resulting in work file use
– 27
• Types of Sparse Indexes used and storage consumed
– 311
• Performance Class 8
• DGTT Information
• See DSNDQW04 member of SDSNMACS for more information
68
SVC
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Maximum Query Sort Memory & DSMAX
MAXSORT_IN_MEMORY
OPT
– Max sort storage for ORDER BY or GROUP BY in KB for a query
DSMAX increase to 200,000
– Retrofitted to DB2 10 via APAR PM88166
• See PM88165 & PM87969 for enabling DB2 maintenance
• Internal DB2 changes to support expanded DSMAX
– Ex. Allow Buffer Manager control block above the bar
– See the APARs for z/OS configuration changes
– The practical limit may be much lower
• Especially if the Scheduler Work Area is below the line
69
SVC
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Profile Expansion
System Profiles Support
– Post-GA in V11 (PM93658)
– Ability to use SPECIAL_REGISTER Keyword to establish several
Special Registers for remote connections
• Client
• Application
• User
– Including CURRENT APPLICATION COMPATIBILITY
– Other special registers:
•
•
•
•
•
•
•
•
•
•
CURRENT DEBUG MODE
CURRENT DECFLOAT ROUNDING MODE
CURRENT DEGREE
CURRENT EXPLAIN MODE
CURRENT GET_ACCEL_ARCHIVE
CURRENT LOCALE LC_CTYPE
CURRENT MAINTAINED TABLE TYPES FOR
OPTIMIZATION and CURRENT MAINTAINED TABLE
TYPES
CURRENT OPTIMIZATION HINT
CURRENT PACKAGE PATH
CURRENT PATH, PATH, CURRENT FUNCTION PATH
70
SVC
•
•
•
•
•
•
•
•
•
•
•
CURRENT PRECISION
CURRENT QUERY ACCELERATION
CURRENT REFRESH AGE
CURRENT ROUTINE VERSION
CURRENT RULES
CURRENT SCHEMA and SCHEMA
CURRENT SQLID
CURRENT TEMPORAL BUSINESS_TIME
CURRENT TEMPORAL SYSTEM_TIME
ENCRYPTION PASSWORD
SESSION TIME ZONE and TIME ZONE
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
ACCESS DB
Externalizes Real Time and Externalized Statistics
– ACCESS DB(*) SP(*) MODE(STATS)
OPT
• Data Sharing Group scope
• Wildcarding is supported, but can impact performance
• Best wildcard options:
–
–
–
–
DB(*) SP(*) for Subsystem / group level
DB(db-name) SP(*) for Database level
DB(db-name) SP(space-name) for Table Space level
DB(db-name) SP(space-name) PART(part-#) for Partition level
– Parallelism for performance
• Internal threads now execute in parallel instead of serially
• Retrofitted to DB2 10 via APAR PM80779
71
SVC
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Client Special Register & Global Variables
CLIENT_CORR_TOKEN Special register
CON
– Client specified correlation information for the connection
SQL
– 255 bytes
• Note: Most other Client registers are also expanded to 255
– Defaults to the generated client Logical Unit of Work ID (LUWID)
– Reported in DISPLAY THREAD
• DSNV402I message V442 section
– Can be SET via various APIs:
•
•
•
•
sqleseti
JDBC
SET_CLIENT_ID Function
SIGNONs
CLIENT_IPADDR Global Variable
– Set to the local IP Address or set DB2 as obtained from the network
– Currently requires the 10.5 level driver and local returns NULL
72
SVC
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
DB2 Supplied Stored Procedures Changes
ADMIN_COMMAND_MVS
– QUERY COPYPOOL
– LIST COPYPOOL
– DB2 START
– DB2 STOP
– DUMP
– DISPLAY WLM
– DB2 commands are routed to specified DB2 via command-prefix
• Can be routed to another LPAR via remote_system
– E.g. Issue a START DB2 command for DB2B from DB2A
– Must have EXECUTE or Ownership of the stored procedure, or SYSADM
• Caller must have READ access of the RACF MVS.MCSOPER.DSNADMCM (or
higher) profile in RACF OPERCMDS class
– Retrofitted to DB2 10 via APAR PM93773
DSNWZP & ADMIN_INFO_SYSPARM
– Includes all new System Parameters
73
SVC
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
RBA / LRSN Expansion
See the Migration presentation for more detail
– DSNTIJCB for BSDS expansion
– DSNTIJCV for Catalog expansion
Catalog columns are created / expanded during
migration
DB2 internals are 10 bytes in CM
– Expect to see DISPLAY changes
– Log externalization (IFCID 306 & Exit) are 10 bytes in CM
Expect changes to IFCIDs that report RBAs / LRSNs
74
SVC
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Directory SELECT Access
DB2 10 introduced directory SQL access via:
– PM35190
• SYSLGRNX
• SYSUTIL
• SYSUTILX
– PM80685
• DBDR
• SPTR
DB2 11 ENFM and Beyond
– SCTR (SCT02)
75
SVC
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
zIIP Expansion
zIIP for all asynchronous processing executed as SRB mode system
agents
Not expected to assist as much as V10, but can provide increase use
Up to 100% of processing for asynchronous enclave service request
blocks that execute in the DB2 ssnmMSTR, ssnmDIST, ssnmDBM1 and
ssnmDIST address spaces.
– Prefetch (V10)
P Lock
– Deferred write (V10)
Negotiation NOT
– Log write and read
zIIP Eligible
– Index pseudo delete / XML clean up
Expanded RUNSTATS and LOAD redirect
UTL
– Enclave SRB tasks for LOAD, REORG, and REBUILD INDEX Inline Stats
– Enclave SRB tasks for RUNSTATS Column Group Distribution Stats
– Enclave SRB tasks for LOAD REPLACE index management
IIPHONORPRIORITY=NO
– Disables system agent (including prefetch) redirect
76
SVC
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
SQL Enhancements
Application Compatibility
Global Variables
Big Data & JSON Integration
Preview
Native SQL PL Enhancements
– Arrays
– Autonomous Transactions
Aggregate Functions
– GROUP BY Expansion
• Grouping Sets
• Super Groups
– MEDIAN
Miscellaneous Topics
77
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
SQL Business Value
Applications can decide when to experience new SQL (DML)
behavior and new functions
Simplified sharing of information across applications and
routines within SQL without writing additional code
Improved application development with easier management of
tabular information within Native SQL Language
Expanded Business Intelligence and OLAP capabilities
DB2 for z/OS integrates with Big Data & supports JSON
development
78
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Application Compatibility ...
Occasionally SQL functions change behavior
– Usually in support of family compatibility and SQL standards
– This introduces an application incompatibility which can
• Delay version migrations
• Potentially create a single version charge issue
V9
App
– These are documented in the Release Incompatibilities
V10
App
Example DB2 10 incompatibilities & resolutions
– CHAR function results (also for VARCHAR and CAST of these data types)
• Leading zeroes no longer returned when there is a decimal point
• ZParm BIF_COMPATIBILITY was introduced to reverse the V10 behavior
• IFCID 366 introduced to report on use of V9 code path while in V10 ZParm
enabled
– Strong data typing for .NET stored procedures
• ZParm DDF_COMPATIBILITY reversed this behavior
– Acceptance of unsupported Timestamp formats was also reversed by
BIF_COMPATIBILITY
– Others required application changes starting in Conversion Mode
79
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Application Compatibility ...
DB2 11 fences DML behavior change beginning in CM with
APPLCOMPAT
– Does not fence DDL or DCL
– Separates the application migration from the system migration
• Application migration can begin after the system migration is complete
• Or the application migration can be delayed for up to 2 future DB2 versions
APPLCOMPAT ZParm and Bind Parameter
– V10R1
• DML behaves as it did for DB2 10
• Must use V10R1 until NFM
• Attempting to use new features under V10R1 results in SQLCODE -4743
– V11R1
• Requires the subsystem / group to be in NFM
• New DML behavior is introduced
• Also required for new features of V11
80
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Application Compatibility ...
ZParm APPLCOMPAT is the default for the BIND / REBIND parameter
– Has no effect on existing Packages OR REBIND PACKAGE with APPLCOMPAT set
– Defaults to V11R1 for installations and V10R1 for migrations
BIND / REBIND parameter
– Defaults to ZParm APPLCOMPAT
• After the previous setting for REBIND
– V11R1 will cause an error before NFM
• Once NFM, can BIND / REBIND with V10R1 or V11R1 irrespective of the ZParm
– REBIND existing application when ready for new behavior
– REBIND existing application when wanting to use new features
• WARNING: Must also be ready for potential behavioral changes
CURRENT APPLICATION COMPATIBILITY special register for dynamic SQL
– Defaults to Package APPLCOMPAT
– Cannot be set to V11R1 until NFM
• Once NFM, can bet set to V10R1 or V11R1 irrespective of the Package or ZParm settings
– Can SET via a System Profile with PM93658
81
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Application Compatibility
Instrumentation
– When using the V10R1 compatibility level, 3 IFCIDs are produced to
identify application at risk at the V11R1 level
• IFCID 239 enhanced to report on Packages that use the V10 code path
• IFCID 366 (also in V10) identifies Statements in Packages that use V10
code path
• IFCID 376 reports the same as 366, but attempts to eliminate duplicates
Considerations
– Behavioral change is possible in the down level compatibility level
when needing to conform to SQL Standards
– This does not fence DDL or DCL, in NFM (system) / V10R1 (package)
• A Global Variable can be created (DDL) and have authority granted (DCL)
• But applications cannot SET or reference the Global Variable until V11R1
– Build your NFM plan to adopt V11R1 for applications
82
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Global Variables ...
Enable the sharing of data between SQL statements (including
stored procedures) without the need for application logic
Memory variables maintained by DB2 and are available
throughout a session
Similar to special registers
– Settings last for the connection
– In thread storage
Have access controlled by GRANT and REVOKE statements
There are also Built In Global Variables for DB2 11
– Archive Tables
– Client IP Address information
83
SQL
VER
CON
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Global Variables ... vs DGTTs
Comparison between DGTTs and Global Variables
DGTT
Global Variable
Stores multiple rows
Stores a value
Internal workfile created
In memory storage allocated
Due to in memory storage can be used to
pass parameters to stored procedure or
UDF
Should be faster than a DGTT due to less
overheads
84
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Global Variables ...
A new object in DB2 11 NFM ...
– CREATE a variable with a data type and optional default value
• As with a Table, the Schema can be implicit or explicit
– Can be created and referenced in the same unit of work
• Not available globally until the UOW COMMITs
– No LOB, ROWID, or XML data types
85
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Global Variables ...
A new object in DB2 11 NFM
– Defaults to NULL
• A user defined default can be specified
• Can default to a special register except CURRENT:
–
–
–
–
GET_ACCEL_ARCHIVE
QUERY_ACCELERATION
TEMPORAL BUSINESS_TIME
TEMPORAL SYSTEM_TIME
VER
– A Variable can be DROPped or COMMENTed, but not ALTERed
• Built In Global Variables cannot be DROPped
– GRANT / REVOKE - ALL, READ, or WRITE - ON VARIABLE
• Can also be managed via the Security Server
• REVOKE
– With RESTRICT prevents operation when Revoker owns a UDF, Trigger, or View
that references Global Variable
– Dependent Dynamic Statement Cache statements are invalidated
86
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Global Variables ...
Compatibility level must be V11R1 use the Global Variable
Automatically set to DEFAULT upon instantiation
Can be SET
– Implicit qualification is via the CURRENT PATH order
– As of NFM, SYSIBMADM is added to CURRENT PATH
Can be referenced in DML statements
The Variable can be used by all sessions at the current server
The values are private to each session
Value is preserved beyond COMMIT / ROLLBACK
Cannot be set in an Autonomous Transaction
– SQLCODE -20430
87
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Global Variable Example …
88
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Global Variables Usage ...
Can appear in expression, predicates, and select list
Global Variables cannot be used in
– Check Constraints
– MQTs
– Definition expressions for indexes
– View definitions WITH CHECK OPTION
The value can be changed by
– SET, SELECT INTO, or VALUES INTO
– An argument of an OUT or INOUT parameter in a CALL statement
The content is not affected by COMMIT or ROLLBACK
statements.
89
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Global Variables in Routines ...
The content of the Global Variables are always
inherited from the caller
Stored Procedures can make changes to the Global
Variables, but not if the SP was called by a UDF or
trigger
– Except for Autonomous Transactions may not change global
variables.
The changes always propagate to the caller
90
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Global Variables with DRDA ...
For implicit connection via three-part name
– System directed
– The values at the requester are used at the server for the
processing of the SQL statement
– Global Variables updated at the server will be sync back to the
requester
– The Global Variables definitions must be same in both requester
and server sides. Error checking will be performed
For explicit connection via CONNECT statement
– The Global Variable must be created at the server with needed
authorizations
– Modified values are not returned to the requester
91
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Global Variable with DRDA and Data Sharing
When Sysplex Workload Balancing is used
This requires drivers at the V10.5 FP2 level
Global variables are sent back to the client
– The logical connection could be moved to a different
transport to the Data Sharing Group
92
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Native SQL Arrays ...
Native SQL Procedures & Functions can now use an Array as a:
– SQL Variable or
– SQL Parameter within the appropriate context
• SQL PL context (Ex. Native SQL Procedure) or
• Java program using Type 4 JDBC (10.5 FP2) connectivity
A set of elements of the same Data Type
CREATE TYPE to establish the array User Defined Data Type
– Based on built in data types
– Not ROWID or XML
Declare a SQL Variable or Parameter using the Array UDT
Can be passed between Java & Native SQL Procedures and Functions
Array Built In Functions are provided to
– Construct Arrays
– Derive Tables from Arrays
– Obtain information about the Array
– Navigate the Array elements
93
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Native SQL Arrays ...
There are 2 types Arrays
– Ordinary
• Has a user defined upper bounds (Max Cardinality)
– Defaults to INTEGER high value
• Elements are referenced / indexed via an integer value
– Associative
• Have no upper bounds
• Elements are referenced via a specified (named) index value
– The Array Index is an INTEGER, CHAR or VARCHAR data type
– When instantiated, they are stored above the bar (HVCOMMON) if > 32K
• Limited to 4GB per subsystem
The examples below are both Array UDTs
– DEPT_LIST is for an Ordinary Array with elements of type VARCHAR(10)
• Explicit upper bounds placed in the brackets [100]
– DEPT_ATTR is for an Associative Array with elements of type INTEGER
• The Array Index has a data type of VARCHAR(10)
94
SQL
Ordinary
Associative
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Native SQL Arrays ...
Define Variable(s) or Parameter(s)
Built In Functions & Constructors ...
– Array Construction
• ARRAY Constructor
• ARRAY_AGG
– Can SET or SELECT from one Array or Table to an Ordinary Array
– Collection derived table
• UNNEST
– Converts the element of one or more Arrays into Column values of
rows in an intermediate table
– WITH ORDINALITY
> An extra INTEGER column is added last with
> Position of the element in the Array
• Can UNNEST multiple Arrays producing columns
• One Associative Array produces 2 columns
– Index
– Element
95
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Native SQL Arrays ...
Built In Functions ...
– Obtain Array information
• CARDINALITY
– Returns the number of elements in the Array
– Returns a BIGINT value
• MAX_CARDINALITY
– Returns the maximum entries possible for an Ordinary Array
– Returns NULL for an Associative Array
– Array Navigation
•
•
•
•
•
96
Returns the index values for the Array
ARRAY_FIRST
ARRAY_NEXT
ARRAY_PRIOR
ARRAY_LAST
SQL
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Native SQL Arrays ...
Dept_Attr1
Built In Functions examples
– Options to populate Associative
Array elements
6
EMP
4
PROJ
– Get the number of elements in the
DEPT_LIST1 Associative array
• Also shows an Array element
assignment & use in a predicate
– Use the ARRAY_FIRST &
ARRAY_NEXT functions to get the EMP
Array index value
EMP=6,
• EMP
• PROJ
PROJ
– Use the index value to reference
the element value
97
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Native SQL Arrays
More Built In Functions
– Array maintenance
• ARRAY_DELETE
– Erases an Ordinary Array
> The index inputs to the function must be NULL
> This is the same as setting the Array variable = ARRAY[ ]
– For an Associative Array
> Specifying a single index value, deletes that entry
> The 2 index value input represent a range that are deleted
> The removed elements can be assigned to another Array
• TRIM_ARRAY
– Deletes elements from the end of an Ordinary Array
– Returns the remaining Cardinality
98
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Native SQL Autonomous Transactions ...
An Autonomous Transaction is a native SQL
Procedure that can commit work outside the commit
scope of the calling program
– If there are multiple versions, they must all be
AUTONOMOUS
The AUTONOMOUS keyword is an option on
– CREATE & ALTER PROCEDURE
– As an alternative to COMMIT ON RETURN syntax
99
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Native SQL Autonomous Transactions ...
An Autonomous procedure always commits before returning to
the caller
– Can issue COMMITs or ROLLBACKs for the nested processes
– Will not COMMIT or ROLLBACK changes from the calling application
– It runs independently
• DISPLAY THREAD will show the Autonomous Transaction
• Does not share locks with the caller and may contend with the caller
• Can be canceled by terminating the invoker
100
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Native SQL Autonomous Transactions
Can be called by other Procedures, UDFs, or Triggers
Can invoke other nested routines (Procedures, Triggers,
UDFs)
– Up to 64 levels
– However, another autonomous procedure may not be invoked
Parallelism is disabled for the Autonomous Transaction
and nested levels
No LOB parameters or UDTs based on LOB or XML
101
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Grouping Sets & Super Groups ...
More in SQL BI / OLAP function with 3 GROUP BY Super Group
options:
– GROUPING SETS
– ROLLUP
– CUBE
GROUPING SETS
– Provides for multiple GROUP BY results in a single query
ROLLUP
– Rolls up the results by the expressions specified in the GROUP BY
CUBE
– Calculates the results for the intersection of all GROUP BY expressions
102
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
GROUPING SETS
Determine the Average total compensation for the
WorkDept, Job, and EdLevel sets
24 Rows
103
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Super Sets ... ROLLUP
Determine the Average total compensation for the
various hierarchies of WorkDept, Job, EdLevel and
overall sets
– Produces sub & grand total rows
– Column order matters
– Made readable with ORDER BY
49 Rows
104
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Super Sets ... CUBE
Determine the Average total compensation for the various
intersections of WorkDept, Job, EdLevel dimensions
– Produces subtotal rows
– Produces cross tabulations
– Super Set Combines
• ROLLUP
• Cross tabulations
– Made readable with ORDER BY
– Column sequence doesn’t matter
107 Rows
105
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
New Function
MEDIAN(numeric-expression)
– DB2 Analytics Accelerator only function
– Returns the median of a set of numbers
– For Static SQL bound for acceleration
– EXPLAIN on a stmt containing MEDIAN that cannot run on an
accelerator sets PLAN_TABLE columns
• SELECT stmt
– QBLOCK_TYPE set to PRUNED and ACCESSTYPE set to blank
• INSERT stmt
– QBLOCK_TYPE set to NCOSUB and ACCESSTYPE set to blank
– Introduced via APAR PI11026
– Retrofitted to DB2 10 via APAR PI10167
106
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Miscellaneous Topics
DB2 Version 11 adds support for running Java™ stored procedures in a
64-bit JVM
– Previous versions of DB2 supported 31-bit Java virtual machines (JVMs) only,
• Each JVM can run only one Java stored procedure at a time
– DB2 Version 11 can concurrently run multiple Java stored procedures in 64-bit
JVMs.
• More Java stored procedures can run in a single stored procedure address space
than in previous DB2 versions
– The recommended NUMTCB for WLM environments that run 64-bit JVMs is
25.
LIKE predicates trimmed before the comparison
– With the LIKE_BLANK_INSIGNIFICANT ZParm set to YES
• Defaults to NO
– Strips trailing blanks before comparison
– Requested by SAP
Change order for applying aggregate functions with Column Masks and SEC
DISTINCT
DB2 11 NFM / V11R1 introduced CHAR9 and VARCHAR9 functions to
maintain pre-V10 behavior without BIF_COMPATIBILITY ZParm
107
SQL
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Big Data is coming to DB2 for z/OS
DB2 is providing the connectors and the DB capability to
allow DB2 applications to easily and efficiently access data
in Hadoop
•New user-defined
functions
•New generic table
UDF capability
IBM Big Insights
JAQL
{
}
108
SQL
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
JSON Technology Preview
Providing the best of both worlds
JSON
API
Tunable
Consistency
Performance
& Scalability
109
SQL
JSON
{
“Product”: {
“SKU”: 11213,
“Name”: “Google
Glass”,
“Category”: {
},
“Size”: [ “S”, “M”,
“L” ]
}
}
SQL + JSON
API
Referential Integrity
Check constraints
Transactions
Geo-spatial
Scalability
Temporal
Security
Joins
SQL
API
Relational
Tools for higher
Productivity
Established
Security
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
JSON Support
PI05250 & PI10521 delivers JSON in DB2 11
Available via IBM DB2 Accessories Suite for z/OS, V3.2
– PID 5697-Q04, Announcement 214-148
See Informational APAR II14727
– 10.5 FP3 Data Server Client
Provides NoSQL Java API to JSON documents in a
DB2 store
Until the DB2 for z/OS Information Center is updated
– The DB2 LUW implementation is expected to be equivalent
developerWorks documentation
110
SQL
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
JSON Basics
{
"firstName“
"lastName"
"age"
“active”
“freqflyer_num”
JavaScript Object Notation
– Lightweight data interchange format
:
:
:
:
:
"John",
"Smith",
25,
true,
null,
String
Number
Boolean
– Specified in IETF RFC 4627
– http://www.JSON.org
Designed to be minimal,
portable, textual and a subset of
JavaScript
– Only 6 kinds of values!
– Easy to implement and easy to use
Text format, so readable by
humans and machines
Language independent, most
languages have features that
map easily to JSON
Used to exchange data between
programs written in all modern
programming languages
111
SQL
}
"address“ :
Null
{
"streetAddress“ : "21 2nd Street",
"city“
: "New York",
"state"
: "NY",
"postalCode"
: "10021"
},
"phoneNumber“
:
[
{
"type"
: "home",
"number“ : "212 555-1234"
},
Array
{
"type"
: “mobile",
"number“ : "646 555-4567"
}
]
Object
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
DB2 11 Specific JSON Functions
JSON2BSON convert JSON string into BSON format
BSON2JSON convert BSON LOB into JSON string
JSON_VAL retrieve specific value from inside a BSON object
INSERT INTO EMPLOYEE(data) VALUES (SYSTOOLS.JSON2BSON
(‘{ name: "Joe", age:28, isManager: false, jobs :[“QA”, “Developer”] } ’))
SELECT SYSTOOLS.BSON2JSON(data) FROM EMPLOYEE
UPDATE EMPLOYEE SET DATA =
SYSTOOLS.JSON2BSON('{ name: "Jane", age:18, isManager: false, jobs
:["Developer", "Team Lead"] } ')
Definition
JSON_VAL(DATA,'PO.customer.@cid','i:na')
112
SQL
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Optimization Evolution
APREUSE
EXPLAIN changes
Statistics Feedback
BIND / REBIND changes
Extended Optimization
Real Time Stat changes
Query Transformations
RDS / Runtime Optimizations
DPSI improvements
Parallelism enhancements
Results Set Management
Virtual Index changes
113
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Optimization Business Value
DB2 11 is extending query stability options through features
introduced in DB2 9 & 10
Stability will also be enhanced via a direct feedback mechanism
from the optimizer
Query transformation, internal management, and
externalizations continue to offer enhanced ways to achieve
lower administration costs and high performance
114
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
APREUSE
DB2 10 APCOMPARE / APREUSE stability across BIND & REBIND
– APREUSE options were NONE/NO or ERROR
DB2 11 provides APREUSE(WARN)
– Attempts to hint the same access path for matched statements
– When the access path cannot be maintained, a new one is calculated for those
specific matching statements
– Effectively operates at the statement level
– The same considerations from DB2 10 carry forward
• Uses the Explain Data Block, as does APREUSE(ERROR) and APCOMPARE(WARN /
ERROR)
• Some PLAN_TABLE columns are not hint-able (MATCHCOLS)
– APREUSE(WARN) will allow the BIND / REBIND to continue
DB2 10 APREUSE(ERROR) ... EXPLAIN(ONLY) may represent invalid plan
– In DB2 11, this will rolled backed
115
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Statistics Enhancements
New RESET ACCESSPATH option on RUNSTATS
UTL
– Full details in Utility Section
– RESET ACCESSPATH resets the access path statistics back to default
values, for all the tables in the specified table space and their respective
indexes
– “Vanilla” RUNSTATS does not fully resolve this situation
• Does not collect skewed distribution statistics on anything but the
leading column of an index
– Ideally, in a QA environment…
1.
2.
3.
4.
116
Reset statistics
Collect "vanilla" statistics (RUNSTATS TABLE ALL, INDEX ALL)
Explain / run the workload in QA populating statistics feedback objects
Use the statistics feedback to create more robust RUNSTATS statement
including recommended COLGROUP statistics
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Statistics Feedback ...
The Optimizer can provide feedback on missing statistics during
– Access path selection
– Explain processing
Access path selection (BIND / REBIND / PREPARE)
– Externalized to new Catalog table SYSSTATFEEDBACK
• Table added in CM
• Feedback is externalized asynchronously starting in NFM
– On STATSINT interval, or
– With ACCESS DB command
– At the Table level
– Controlled by ZParm STATFDBK_SCOPE (ALL)
• ALL, DYNAMIC, NONE, STATIC
– Additional control at the table level
• SYSTABLES STATS_FEEDBACK which defaults to Y
– Redundant recommendations are not created
– RUNSTATS for an object removes these associated recommendations
117
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Statistics Feedback ...
Explain processing
– Externalized to new EXPLAIN table DSN_STAT_FEEDBACK (DSNTESC)
– Controlled by the existence of this table
• SYSTABLES STATS_FEEDBACK does not control these recommendations
– Statement level recommendations written synchronously with EXPLAIN
– History is maintained
– User managed clean up
Optim Query
Does not apply to VOLATILE, DGTT, or CGTT
Workload Tuner
Not seeing expected feedback in SYSSTATFEEDBACK? could avoid
– Make sure it has been externalized
Stats Advisor
• STATSINT interval
• ACCESS DB(*) SPACENAM(*) MODE(STATS) will externalize on command
OPTHINTS, APREUSE, APCOMPARE
– Recommendations limited to costing performed
Not directly consumable by RUNSTATS
118
OPT
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Statistics Feedback ...
SYSSTATFEEDBACK
Table/Index level
In NFM, governed by
ZParm and table control
DSN_STAT_FEEDBACK
Statement level
BIND
REBIND
PREPARE
SQL
EXPLAIN
Yes
Yes
Yes
Yes
w/ EXPLAIN
(YES or ONLY)
w/ CURRENT
EXPLAIN
MODE YES or
EXPLAIN
Yes
Including
SQLERROR(CHECK)
EXPLAIN(ONLY)
w/ EXPLAIN
(YES or ONLY)
In CM, governed by
existence of the table
COLLID for dynamic SQL
– Identifies source of dynamic SQL (Cache, CURRENT EXPLAIN MODE
YES/EXPLAIN)
Use common sense when considering feedback
Consider the use of Statistics Profiles (SYSTABLES_PROFILE)
– Not used via Inline Statistics
119
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Statistics Feedback ...
NUMCOLUMNS
– # of columns in COLGROUP
COLGROUPCOLNO
– Hex array of column numbers for tooling use
TYPE
– ‘C’ardinality
• Use the COLUMN or COLGROUP option
– ‘F’requency
• Use FREQVAL option (COUNT = Floor(10, COLCARF-1))
– ‘H’istogram
• Use HISTOGRAM option
– ‘I’ndex
• Collect basic index stats
– ‘T’able
• Collect basic table stats
BLOCK_RUNSTATS
– For use by tooling
– Not used by DB2
120
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Statistics Feedback
REASON
– BASIC
• TABLE(ALL) INDEX(ALL) will resolve, currently -1
– KEYCARD
• Index column cardinalities missing
– LOWCARD
• Low cardinality, likely skew
– NULLABLE
• No distribution stats available, likely skew
– DEFAULT
• Predicate likely on a default value
– RANGEPRD
• No histogram on a range predicate
– PARALLEL
• Could be improved by balancing data
– CONFLICT
• Collect consistent stats at the same point in time
– COMPFFIX
• Multi-column cardinality stats are needed
121
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Extended Optimization ...
DB2 10 introduced the Access Path Repository
– SYSQUERY
– SYSQUERYPLAN
– SYSQUERYOPT
For
– Statement Level Optimization Hints
– Statement Level Optimization Parameters (Options)
Static and dynamic statements
Does not apply to statements in CREATE FUNCTION, TRIGGER,
and PROCEDURE
122
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Extended Optimization ...
DB2 11 extends the APR and BIND QUERY with predicate selectivity
enhancements
– Allows for overriding the selectivity of predicates within a statement
– Addresses conditions where the filter factors cannot be accurately
determined and defaults are not representative of the data
• Host Variables or Parameter Markers
• Expressions
• Subqueries
– Uses the existing input table:
• DSN_USERQUERY_TABLE (DSNTESH)
– Adds two new input tables to this process (DSNTESC)
• DSN_PREDICAT_TABLE (existed in DB2 10, but not used with BIND QUERY)
• DSN_PREDICATE_SELECTIVITY
123
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Extended Optimization ... Access Path Repository
New Tables
in CM
New
Columns
in CM
124
OPT
© 2015 IBM Corporation
Migration Planning Workshop
Extended Optimization ... Input Tables
DB2 11 for z/OS
Added
Table
DSN_USERQUERY_TABLE QUERY_TEXT
–Should be populated from SYSPACKSTMT
for Static
–From Dynamic Statement Cache for
Dynamic
DSN_PREDICAT_TABLE and
DSN_PREDICATE_SELECTIVITY (new in V11)
–Are populated from EXPLAIN
–Use EXPLAIN rows as a model
Profile =
Statement
1:1-20
Instances
New
Columns
1:1-# Predicates
Overrides
125
OPT
New
Table
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Extended Optimization ... Definitions
Selectivity Profiles are stored in DSN_PREDICATE_SELECTIVITY
– Set of all selectivity instances for a single statement
Selectivity Instance
– Represents up to 20 Override Sets per Profile / Statement
– Defines different filtering characteristics (Overrides) at different times
– Frequency is identified by WEIGHT
• All Instance WEIGHTs for a statement must sum to <= 1.0
• If sum of WEIGHTs is < 1.0
– The max # of Instances is 19
– DB2 assumes an Instance with default selectivity for the balance of the total WEIGHT
• WEIGHT values must be the same # for the same Instance
– Recommend creating explicit rows with WEIGHT values totaling 1.0
Selectivity Override
– A specific predicate override identified by the PREDNO
– Indicate the INSTANCE (not 0) & SELECTIVITY override value
– Set ASSUMPTION to OVERRIDE
126
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Extended Optimization ... Considerations
EXPLAIN output tables considerations
– Use one set of EXPLAIN/Input tables for output of EXPLAIN
BIND QUERY input tables considerations
– Use another set under a different schema to copy only the needed rows for
overrides
– Only override predicates that need extended optimization
– Remember to create required indexes for BIND QUERY
• DSN_PREDSEL_IX1 on DSN_PREDICATE_SELECTIVITY
• DSN_PREDICAT_TABLE_IDX1 on DSN_PREDICAT_TABLE
• DSN_PREDICAT_TABLE_IDX2 on DSN_PREDICAT_TABLE
Use the EXPLAININPUTSCHEMA option of BIND QUERY to specify the
schema of the input tables
Access Path Hints (from DB2 10) are mutually exclusive with
– Options Overrides (from DB2 10) and
– Selectivity Overrides
127
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Extended Optimization
An example from the Managing Performance Guide
This is an excerpt of columns from DSN_PREDICATE_SELECTIVITY
– Rows for the same QUERYNO would be needed in the
• DSN_USERQUERY_TABLE
• DSN_PREDICAT_TABLE
Shows 2 Selectivity Instances for a Profile (Statement)
– One occurs 45% of the time
– The other 25% of the time
– A implied Instance with default selectivity occurs 30%
3 predicates have Selectivity Overrides for this Profile
– There could be more predicates in the statement taking defaults
128
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Real Time Statistics & Resetting Stats
Catalog changes
– The two tables are fanned out into separate table spaces in ENFM
– New SYSTABLESPACESTATS columns
• UPDATESIZE (BIGINT): bytes added / removed via UPDATE operations since
creation, REORG, or LOAD REPLACE
• LASTDATACHANGE (TIMESTAMP): last update time for this row in RTS
ACCESS DB ... MODE(STATS)
SVC
– Externalizes in memory statistics to
• Real Time Statistics and
• Optimizer Feedback
– Group wide scope
– Wildcarding (*) is supported
• More efficient to avoid range (A:B) and partial-* (dbname*) qualifications
RUNSTATS...RESET ACCESSPATH
UTL
– Removes collected statistics
– Excludes LOB & XML
129
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Query Transformation ...
Common stage 2 predicates transformed to indexable...
– Requires REBIND
– Index on Expression cases where no IOE exists
• If IOE exist, this transformation will not be attempted
– YEAR(date column) op :hv or ? or literal
– DATE(timestamp column)
– SUBSTR(left most portion)
130
OPT
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Query Transformation ...
Common stage 2 predicates transformed to indexable...
– x BETWEEN col1 AND col2
• Externalized as a new
DSN_PREDICATE_TABLE.TYPE=“COMPOUND”
– OR predicates involving NULLS
• Internally rewritten to a IN list that cannot be coded directly
Internally rewritten to:
EDLEVEL>*12
NULL tolerant > predicate
But not rewritten when externalized
131
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
RDS / Runtime Optimizations ...
CASE expression is indexable
– For a local predicate
– For a join predicate
• If the CASE expression is evaluated before the join
Non-column expressions processed once
– Before V11 these were assessed for each row
– Examples:
• HV + 5
• CURRENT TIMESTAMP – 1 MONTH
132
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Query Transformation ...
Common stage 2 predicates transformed to indexable
– OR and IN on different columns
• Rewritten to ORs (if IN list < INLISTP) for multiple index access
• WHERE C1 = ? OR C2 IN (1,2)
– becomes WHERE C1 = ? OR C2 = 1 OR C2 = 2
Pruning always TRUE / FALSE predicates
– Tools / ISV Apps use them (Ex. turn predicates “on” and “off”)
– In DB2 10 they are evaluated for each row
– DB2 11 prunes them to avoid this redundant checking
• DB2 10 already prunes some always false predicates
– “=“, or “IN” with ORs
– IS NULL checking of NOT NULL columns with ANDs
– WHERE, HAVING, and ON clauses
– Must be literals (no Host Variables or Parameter Markers)
• REOPT not support
– OR 0=1 is not pruned
133
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Query Transformation ... Pruning
Always True Predicates
– Example WHERE 1=1
– So what’s the problem with this harmless predicate?
• DB2 will execute the WHERE 1=1 predicate for every qualified row
SELECT * FROM TABLE
WHERE 1=1
AND CUSTNO = ?
– Prune always true predicate to become
SELECT * FROM TABLE
WHERE CUSTNO = ?
134
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Query Transformation ... Pruning
Always False Predicates
– DB2 10 already prunes “always false” equal / IN under OR
• WHERE C1 = ? OR ‘A’ = ‘B’
– DB2 11 extends to “always false” underneath parent “AND”
SELECT * FROM TABLE1 T1, TABLE2 T2
WHERE (1=1 AND T1.C1 = T2.C1)
OR (1=2 AND T1.C2 = T2.C2)
– Prune always true/false predicates to become
SELECT * FROM TABLE1 T1, TABLE2 T2
WHERE T1.C1 = T2.C1
– “OR 0=1” is NOT pruned
– Literals only. No host variables or parameter markers. No REOPT.
135
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Query Transformation ... Pruning and EXPLAIN
PLAN_TABLE & DSN_STATEMNT_TABLE indicates
statement pruned
DSN_STATEMNT_TABLE
PLAN_TABLE
DSN_PREDICAT_TABLE reflects the pruned predicate
136
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Query Transformation ...
More Correlated to Non-Correlated Subquery
Transformations
– Example:
SELECT *
FROM T1 A
WHERE A.C1 = 5
AND (A.C2 IS NULL OR
A.C2 = (SELECT MAX(B.C2) FROM T1 B WHERE B.C1 = A.C1))
Enhanced predicate pushdown for
– Materialized Views
– Table Expressions
137
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
RDS / Runtime Optimizations ...
Expression merge for multiple references
– A View has an expression to compute a resultant column
– A SELECT of that View column then uses this expression for:
• SUM, AVG, MIN, etc.
– DB2 11 uses a shared result of the expression across functions
Subquery 100 element array
– Been in DB2 since V2
• most recent 100 results from subquery
– DB2 11 adds optimizer recognition
• Optimize order of entries to match the outer query
138
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
RDS / Runtime Optimizations ...
Index Skipping / Early Out
– Duplicate removal
– Benefits GROUP BY and DISTINCT functions
– Upward probe to non-leaf page to find the next unique value
– DSN_DETCOST_TABLE
• IXSCAN_SKIP_DUPS
• EARLY_OUT
DECFLOAT
– EC12 improvements
• CPU reductions, primarily for INSERT & FETCH
– Still no indexing or partitioning
RID Pool
– Overflow to work file expanded to COUNT, MAX, MIN
– Restrict Hybrid Join to 80% of total RID Pool
• Before V11, this could take 100% of the RID Pool
• Other queries (needing > 1 block) could then run into RID limits
139
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
RDS / Runtime Optimizations
Sort Improvements
– MAXSORT_IN_MEMORY ZParm
• Defaults to 1MB / query
• Fences query sort resources
– Avoid work files for final merge
• Application fetches from the tournament tree
NOT LOGGED DGTTs
DSN
Less Overhead for Decompression
DSN
– Column Level
140
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Data Partitioned Secondary Indexes (DPSI) ...
A secondary index is an index which is not partitioning
DPSIs
delivered in
V8
141
OPT
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Data Partitioned Secondary Indexes (DPSI) ...
Join probing
– Before DB2 11
YEAR
C1
2009
1
2010
2011
2
3
• Each composite row access all parts
– DB2 11 can eliminate parts when
2009 2010 2011 2012 2013
• Qualified on the DPSI columns
• The partitioning scheme (PI) is qualified in the join predicates
– Must be Equal (“=“) with same data type / length
• EXPLAIN
– PLAN_TABLE.PAGERANGE=Y
– DSN_FILTER_TABLE.STAGE=PAGERANGE
– If there is no qualification to eliminate parts (PI columns)
• The composite table can be shared for local joins to each part for
each Child task
142
OPT
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Data Partitioned Secondary Indexes (DPSI) ...
Part Level Nested Loop Join
– Parallelism cut on inner table on DPSI boundaries
• A Child task process a DPSI part
– Outer table shared in work file for child tasks
• If less than 32K, uses In Memory Data Cache
– Controlled by ZParm PARAMDEG_DPSI
– EXPLAIN
• PLAN_TABLE.PRIMARY_ACCESSTYPE=P
• Various indications in DSN_PGROUP_TABLE
– Can be hinted
PARAMDEG_DPSI
– Defaults to 0
• PARAMDEG controls parallelism
C1
C1
C1
C1
C1
1
1
1
1
1
2
3
2009
2
2010
3
2
3
2011
2
3
2012
2
2013
3
– 1; DPSI parallel task may be used but are run serially
– 2 – 254; DPSI may drive parallel tasks up to this value
• May be larger or smaller than PARAMDEG
• Other portions of the query would be governed by PARAMDEG
143
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Data Partitioned Secondary Indexes (DPSI)
DPSI Return In Order
– Keys are ordered within each partition of a DPSI
• Ordering across has typically involved a sort to return keys like an NPSI
– DB2 11 can merge the keys without a sort, managed by:
•
•
•
•
The Parent task when parallelism is used (1 Child task per partition)
Data Manager when there is no parallelism
Supported with Index on Expression
May use Index Lookaside
– May require more leaf pages in the Buffer Pool
• This is an access path choice
– Larger results are often more efficient to sort
– Smaller results benefit by avoiding the sort
• EXPLAIN
– METHOD=3 row will not be present
144
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Parallelism Improvements ...
Degrees of parallelism is determined at BIND /
PREPARE
May be adjusted during Host Variable / Parameter
Marker substitution
May be reduced / degraded at run time
– Buffer Pool negotiation
DB2 11 introduced System Negotiation
– LPAR storage usage may reduce / degrade parallelism
– New DSNDQXST instrumentation fields
145
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Result Set Management
In Memory Data Cache, Sparse Index, Work File
– Runtime chooses the appropriate method based on storage available and can
reduce the memory request if the system is in a critical state
• In memory hash (added in V10)
– Uses the most memory but provides the best performance
• In memory binary search
• Hash both
– some rows in memory, some keys in memory with data in a Work File
• Sparse Index
– Keys in memory, data in Work File
DB2 11
– Each Sparse Index / In Memory Data Cache is given a portion of MXDTCACH
(defaults to 20 MB per thread)
Note:
– When a sparse index is used it competes with sort for Buffer Pool resources
– Buffer Pool access is not sequential and won’t benefit from VPSEQT=100
Instrumentation
– IFCIDs 2 & 3 show storage shortages resulting in work file (increase MXDTCACH)
– IFCID 27 show the type of sparse index used and storage consumed
146
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Virtual Indexes ...
DSN_VIRTUAL_INDEXES enhanced with more index information
and types
– UNIQUE_COUNT
– SPARSE
• For NULL Suppressed Indexes
– DATAREPEATFACTOR
– KEY_TARGET_COUNT & IX_EXTENSION
• To support Index on Expression and XML Valued Indexes
DSN_VIRTUAL_KEYTARGETS
– Detail support for Index on Expression and XML Valued
Indexes
To create / update tables
– See DSNTESC for DDL
– Look at DSNTIJXA / ADMIN_EXPLAIN_MAINT (PI05200) to
migrate existing tables
147
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
EXPLAIN Changes ...
EXPANSION_REASON
– Applies to statements that reference archive or temporal tables where DB2 adds implicit
syntax for
VER
• Use of the SYSIBMADM.GET_ARCHIVE Global Variable with ARCHIVESENSITIVE bind option
• Setting CURRENT TEMPORAL BUSINESS_TIME special register with the BUSTIMESENSITIVE
bind option
• Setting CURRENT TEMPORAL SYSTEM_TIME special register with the SYSTIMESENSITIVE
bind option
– Added to the PLAN_TABLE and 17 other EXPLAIN tables
PLAN_TABLE
– SCAN_DIRECTION for index scans
– EXPANSION_REASON
DSN_DETCOST_TABLE
– IXSCAN_SKIP_DUPS
– IXSCAN_SKIP_SCREEN
– EARLY_OUT
– EXPANSION_REASON
ADMIN_INFO_SQL
– Service SQL Stored Procedure enhanced for DB2 11
148
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Temporal Update and Archive Transparency
V10 Temporal Review
Archive Transparency
Temporal Special Registers
Temporal View Enhancements
149
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Versioning Business Objectives
Remove some restrictions from implementing Temporal
Tables
Easier development with temporal tables
Ease manageability by reducing the size of the active
database
150
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Temporal High Level Review
Application Period Table
End
Time
Begin
Time
Business Time
– Begin & End business time columns
– Set by the application
– Modifications to SQL
– Update/Delete modify periods
• Could split rows to preserve Business Time
• DB2 populates the History Table
– DML changes for SELECT only
• Implicit UNION ALL to query History
System Period Table
Trans
ID
End
Time
Begin
Time
System Time
– Begin, End, and Trans time columns
– Maintained by DB2
– History Table for previous row versions
– UPDATE/DELETE
UPDATE/DELETE
History Table
Trans
ID
End
Time
Begin
Time
151
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
System Time Enhanced Auditing Capabilities
Integrated auditing support for automatic tracking of some audit
information including
– Who modified the data in the table
– What SQL operation modified the data in the table
System-period temporal tables can be defined with non-deterministic
expression columns
– Could be useful for non-temporal applications that want to record auditing
data
APARs
– PM99683 preconditioning
– PI15298 enablement
– PI15666 LOAD OVERRIDE support
152
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
System Time Enhanced Auditing Capabilities
Create system-period temporal table (STT) with extra columns
SQLID VARCHAR(8) GENERATED ALWAYS AS (CURRENT SQLID),
DCOP CHAR(1) GENERATED ALWAYS AS (DATA CHANGE OPERATION),
Create history table (STT_HIST)
Enable system versioning with new option
ALTER TABLE STT ADD VERSIONING USE HISTORY
TABLE STT_HIST ON DELETE ADD EXTRA ROW
An additional row is written to STT_HIST table when a row is deleted from STT
– New values are generated for each column that corresponds to a non-deterministic
generated expression column
– The row-begin column is set to the same value as the row-end column
– The other columns are set to the same value as in the row inserted into the history table for
the delete.
SYSTABLES.CLUSTERID set to 1 indicating that enhanced auditing is enabled
153
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
System Time Enhanced Auditing Capabilities
LOAD OVERRIDE(option)
– Allows unloaded data to be reloaded into the specified types of
GENERATED ALWAYS columns
– Allow unloaded data to be reloaded into a GENERATED ALWAYS …
• SYSTEMPERIOD
– … row-begin or row-end column
• IDENTITY
– … identity column.
– Specify OVERRIDE(IDENTITY) when you want to run LOAD with the INTO TABLE
PART clause and a generated identity column is part of the partitioning index
• TRANSID
– … column that is defined as a transaction-start-ID column
• NONDETERMINISTIC
– … column that is defined by a non-deterministic expression.
154
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Archive Transparency
What is the purpose of archiving?
– When you want to delete rows from the table, but need to keep the deleted rows for legal or
business purposes
– To move data to a cheaper storage medium
– When you do not need to access the old data often, but need to be able to retrieve the data
quickly
– When you do not care about the lineage of a row
• This means that you do not care about the changes to a row over time
Do we add extra columns for archiving like we do for system time
tables?
– You do not need extra columns to enable Archive Transparency
Temporal and Archive Tables are mutually exclusive
– Can not build an Archive Table on a table that has either Business Time or System Time
Archive a large amount of data using REORG DISCARD to facilitate
migration
– User would be responsible for loading data from the DISCARD file into the archive table
155
VER
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Archive Transparency Compared to System Time
System Time
Archive Transparency
Base table & History table
Base table & Archive table
same column #, column name,
column attributes (data type,
etc)
same column #, column name,
column attributes (data type, etc)
Additional columns
ROW BEGIN/ROW END/TRANS
ID columns
No additional columns required
Compatible with
Period enabled tables
Compatible with Business
Time
No
Data propagation to
history/archive table
UPDATE and DELETE
DELETE
SYSIBMADM.MOVE_TO_ARCHIVE
Tables
Utilities can be used
(REORG...DISCARD,LOAD...RESUME)
Implicit UNION ALL
query transformation
Controlled by:
CURRENT TEMPORAL
SYSTEM_TIME special register
&
SYSTIMESENTIVE RE/BIND
option
Controlled by built-in global variable
SYSIBMADM.GET_ARCHIVE
EXPANSION_REASON=A
EXPANSION_REASON=S
156
VER
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Archive Transparency Management ...
Base Table
Activate archiving
Archive Table
OR
Create the base table
Create the archive table
Tell DB2 to associate the base table with the archive table
ALTER ADD COLUMN to the Base Table propagates the column to
the Archive Table
157
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Archive Transparency Management
Use the ALTER TABLE ... DISABLE clause to remove relationship between base and
archive tables
– This may be required for Table schema ALTERs other than ADD COLUMN
When Archive relationship is enabled
– Archive table is TYPE is ‘R’
– ARCHIVING_TABLE column is populated
Before Disable Archive
After Disable Archive
158
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Archive Transparency Global Variables ...
Built-in Global Variables that impact archival tables &
processing
– Defined as CHAR(1) NOT NULL DEFAULT ‘N’
– READ authority granted to PUBLIC
– SYSIBMADM.GET_ARCHIVE
• Determines if SELECTs against Archive Enabled (Base) Tables automatically
UNION ALL the associated Archive Table
• 'Y‘ includes the UNION ALL to Archive Tables
• Packages must be bound ARCHIVESENSITVE(YES)
– SYSIBMADM.MOVE_TO_ARCHIVE
• Determines if deleted rows of Archive Enabled Tables are inserted into
associated Archive Tables
• ‘Y’: INSERT and UPDATE not allowed against the Archive Enabled (Base)
Tables
• ‘E’: INSERT and UPDATE allowed against the Base Tables
159
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Archive Transparency
These settings for BIND will control the sensitivity of the
SYSIBMADM.GET_ARCHIVE global variable:
– ARCHIVESENSITIVE (default YES) – packages (No space between ARCHIVE and
SENSITIVE))
• BIND PACKAGE
• REBIND PACKAGE
• REBIND TRIGGER PACKAGE
• CREATE TRIGGER (implicit trigger package)
– ARCHIVE SENSITIVE (default YES) – UDFs and Stored Procedures (space
between ARCHIVE & SENSITIVE)
• CREATE FUNCTION (SQL scalar)
• ALTER FUNCTION (SQL scalar)
• CREATE PROCEDURE (SQL native)
• ALTER PROCEDURE (SQL native)
If you REBIND a package and change ARCHIVESENSITIVE, all copies of the
package will be purged
APREUSE and APCOMPARE are valid options
You can set the EXPANSION_REASON in the Access Path repository
DB2I Panels support ARCHIVESENSITIVE
160
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Archive Transparency Example ...
Archive all rows where START_DATE less than December 31, 2010
Archive-enabled
table has 6 rows
We set the Global variable MOVE_TO_ARCHIVE to ‘Y’
and then issue the DELETE command where the
START_DATE is prior to December 31, 2010
The rows that were deleted from the base table are inserted into the archive table
The Timestamp in the Archive Table has the time the row was archived, not the time in the base table
Microseconds are
greater in the archive
table than the base
(archive-enabled)
table
161
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Archive Transparency Example ...
To select data from both the base and archive tables
Set GET_ARCHIVE global variable to ‘Y’ before the select statement
̶ Can add an indicator column (ex. BASE_ARCHIVE CHAR(01)) to both tables
to indicate from where the row was sourced
̶ Base column defaults to one value (Ex. “B”)
̶ INSERT Trigger on the Archive column to set a different value (Ex. “A”)
To select data from only the base table
Set GET_ARCHIVE global variable to ‘N’ before the select statement
162
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Archive Transparency Example ...
Searched UPDATEs will only update base table rows whether
the GET_ARCHIVE is set to Y or N
In the first example, we set the GET_ARCHIVE to ‘Y’ so that the
SELECT will retrieve rows from both
– The Base and Archive tables, and you can see that only the base
rows were updated
163
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Archive Transparency Restrictions
Restrictions
– LOAD utility will not allow REPLACE option if the table is archived-enabled
– Columns can not be ALTERed, RENAMEd or DROPped in either a base or archive table
– If MOVE_TO_ARCHIVE = ‘Y’, INSERT, UPDATE & MERGE against the base
– PERIOD can not be added to a base or archive table
– A foreign key can not be defined on an archive table
– ROTATE partitions
– MQT
– Row Permissions and Column Masks
– CLONE table
– A data change statement cannot reference temporal table in same statement as reference to
archive table
– Positioned UPDATEs and Positioned/Searched DELETEs against base tables not allowed
when
• GET_ARCHIVE = ‘Y’ and
• Bound ARCHIVESENSITIVE YES
– If the target of the INSERT is a view that is defined with the WITH CHECK OPTION, the
definition of the view must not reference an archive-enabled table
– If you use a DYNAMIC SENSITIVE SCROLLable cursor, you can run with
GET_ARCHIVE=‘N’, but if you run with GET_ARCHIVE=‘Y’ you get -525 SQL error
164
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Archive Transparency ...
When the BIND is performed with an Archive Enabled Table
– We create two sections in the package when ARCHIVESENSITIVE
is YES
• First section - Base table only
• Second section - Base table and archive table UNION’ed ALL together
– ARCHIVESENSITIVE only refers to GET_ARCHIVE sensitivity
• When the GET_ARCHIVE global variable is set to ‘N’
– DB2 will use the base table only section
• When the GET_ARCHIVE global variable is set to ‘Y’
– DB2 will use the base and archive table section
– When MOVE_TO_ARCHIVE is set to ‘Y’
• DB2 will move rows to archive table on a DELETE even if ARCHIVESENSITIVE
BIND option is set to NO
• This prevents data not being archived if the program tells it to archive
– When archiving data, you would usually set GET_ARCHIVE to ‘N’
and MOVE_TO_ARCHIVE as ‘Y’
165
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Archive Transparency EXPLAIN ...
In our example, Data in SYSPACKSTMT has
–SECTNO = 1 for archive-enabled table only and
–SECTNO = 3 for archive-enabled table UNIONed with archive table
– STATEMENT stored as the original statement with EXPANSION_REASON of ‘A’
166
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Archive Transparency EXPLAIN ...
Here you can see that there are two sections in the package in PLAN_TABLE
Section 1 is the base section and will be used when GET_ARCHIVE=‘N’
Section 3 is the expanded section and will be used when GET_ARCHIVE = ‘Y’
More information is available in
DSN_STATEMNT_TABLE
DSN_STAT_FEEDBACK
DSN_STRUCT_TABLE
DSN_DETCOST_TABLE
167
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Archive Transparency EXPLAIN
EXPANSION_REASON added to the following tables
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
168
DSN_COLDIST_TABLE
DSN_DETCOST_TABLE
DSN_FILTER_TABLE
DSN_FUNCTION_TABLE
DSN_KEYTGTDIST_TABLE
DSN_PGRANGE_TABLE
DSN_PGROUP_TABLE
DSN_PREDICATE_SELECTIVITY
DSN_PREDICAT_TABLE
DSN_PTASK_TABLE
DSN_QUERYINFO_TABLE
DSN_QUERY_TABLE
DSN_SORTKEY_TABLE
DSN_SORT_TABLE
DSN_STATEMENT_CACHE_TABLE
DSN_STATEMNT_TABLE
DSN_STRUCT_TABLE
DSN_VIEWREF_TABLE
PLAN_TABLE
VER
OPT
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Temporal Special Registers …
Enable customers to be able to code applications using temporal data
and to be able to test the system, possibly “without changing code”
DB2 will be able to run the same query for different times by changing
the special registers
Have the ability to run AS OF any date by changing the special
register
Provides “Time Machine” capability
– Setting the temporal special registers to a specific point in time
– Works for all subsequent SQL statements
– Including those in invoked functions, stored procedures, and
triggers
– This allows the application to see data from a different point in
time without modifying the SQL statements
169
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Temporal Special Registers …
CURRENT TEMPORAL SYSTEM_TIME
– TIMESTAMP(12), nullable
CURRENT TEMPORAL BUSINESS_TIME
– TIMESTAMP(12), nullable
SET Temporal Registers
– For DRDA the value of the special register is sent to remote side for implicit connect
• When using a 3-part name
– If you use the special registers, they continue to be used for that session until you turn
them off by setting them to NULL
– We add these predicates when there are direct or indirect references to
Business Time or System Time tables
170
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Temporal Special Registers …
Bind parameters determine if the Special Register will be honored when set
– SYSTIMESENSITIVE (YES / NO)
– BUSTIMESENSITIVE (YES / NO)
– BIND PACKAGE
• Default Value - YES
– REBIND PACKAGE
• Default Value – Existing Option
– REBIND TRIGGER PACKAGE
• Default Value – Existing Option
SYSTEM_TIME SENSITIVE and BUSINESS_TIME SENSITIVE for Routines
– Options on CREATE / ALTER SQL Scalar Procedure or Function
– INHERIT SPECIAL REGISTERS passes set values from invoker by default
• DEFAULT SPECIAL REGISTERS will reset to NULL
DB2I support
– Set CHANGE DEFAULTS to YES to find these options
171
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Temporal Special Registers ...
Bind / Rebind two section implementation
– Bind/Rebind keywords cause System Time / Bitemporal tables to bind SQL
twice
– Original section
• Bind the original SQL for non-temporal access / no temporal predicates
• The majority of System Time applications request for current data only
– There is no performance degradation caused by UNION ALL query
transformation
– Extended section
• System Time SQL uses UNION ALL to the associated History Table
• Business Time & System Time temporal predicates, as appropriate
– SQL accessing temporal tables does not have to be changed
At execution time
– If the Temporal Special Register is not set (the default), Original Section
used
– If the Temporal Special Register is set, Extended Section used
EXPANSION_REASON
– In several Catalog & EXPLAIN tables to indentify extended SQL
172
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Temporal Special Registers ...
In PLAN_TABLE, there is a new column called
EXPANSION_REASON, which is populated when statements
reference temporal or archive tables
– A: Query has implicit query transformation as a result of the
SYSIBMADM.GET_ARCHIVE built-in global variable
– B: Query has implicit query transformation as a result of the
CURRENT TEMPORAL BUSINESS_TIME special register
– S: Query has implicit query transformation as a result of the
CURRENT TEMPORAL SYSTEM_TIME special register
– SB: Query has implicit query transformation as a result of BOTH
the CURRENT TEMPORAL SYSTEM_TIME special register and
the CURRENT TEMPORAL BUSINESS_TIME special register
– Blank: The query does not contain implicit query transformation
173
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Temporal Special Registers System Time ...
If the register is set to a valid not null value DB2 will add the clause
– FOR SYSTEM TIME AS OF CURRENT TEMPORAL SYSTEM_TIME
• To static and dynamic SQL statements referencing System time and bi-temporal
tables
• Including indirect references in a View or via a Trigger
An explicit FOR SYSTEM_TIME period specification produces SQLCODE
-20524
– You can not use FOR or AS OF SYSTEM_TIME with the special register
Any INSERT, UPDATE, DELETE, or MERGE (data modification)
statements against System Time tables produces SQLCODE -20535
Data modification statements are allowed against Regular Tables
174
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Temporal Special Registers System Time ...
Base table
– All Copays are currently $15
– All were UPDATEd on 9/24/2013
History table
– Copays were different values in the
past
– Recorded in History by the
9/24/2013 UPDATE
175
VER
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Temporal Special Registers System Time ...
Set CURRENT TEMPORAL SYSTEM_TIME register to before the UPDATE
SELECT all rows that were in effect at that time
Explain shows UNION ALL
Rows all have the before update occurred
176
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Temporal Special Registers System Time ...
Reset CURRENT TEMPORAL SYSTEM_TIME register to NULL
SELECT rows from base table (Same SELECT statement)
Explain shows no access to history table
All rows all have COPAY value of $15 which is the current value in the table
177
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Temporal Special Registers Business Time ...
If the register is set to a valid not null value DB2 will add the clause
– FOR BUSINESS TIME AS OF CURRENT TEMPORAL BUSINESS_TIME
– Static and dynamic SQL statements referencing business time and bi-temporal tables
– Including indirect references in a View or via a Trigger
DB2 will cast the CURRENT TEMPORAL BUSINESS_TIME to the column
definition of either DATE or TIMESTAMP(6)
– start_date <= CAST(CURRENT TEMPORAL BUSINESS_TIME AS DATE/TIMESTAMP(6))
– end_date > CAST(CURRENT TEMPORAL BUSINESS_TIME AS DATE/TIMESTAMP(6))
An explicit FOR BUSINESS_TIME period specification produces an error
For UPDATE or DELETE
– Where CURRENT TEMPORAL BUSINESS_TIME register is not NULL and
BUSTIMESENSITIVE(YES)
– Predicates are generated as shown above
FOR PORTION OF can also be included
– Normal Temporal UPDATE or DELETE logic will be performed
– Potentially splitting existing rows
178
VER
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Temporal Special Registers Business Time ...
CURRENT TEMPORAL BUSINESS_TIME special register example
̶ The left side shows that when we set the CURRENT BUSINESS_TIME register to NULL
• All qualifying rows returned
̶ The right side shows that when we set the CURRENT BUSINESS_TIME register to a value
• Rows returned AS OF the specified time
• Row D054 is not included because the BUS_END date is the same as the CURRENT
BUSINESS_TIME register, and the BUS_END is exclusive, meaning that 2012-12-30 is
not part of the row
179
VER
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Versioning & Views …
DB2 11 - You can use temporal predicates when referring to a view
DB2 10 & DB2 11 - You can not use temporal predicates in a view
Base Table
VIEW
SQLCODE -4736
Temporal predicates can now be used in DML on statements
referencing views
SQL perspective
View 2012
Table
180
VER
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Versioning & Views Example …
Show how the date predicates on the view work with the FOR BUSINESS_TIME
predicate in the SQL statement
Rows in Base Table (POLICY_BITEMPORAL)
Remember the view
Select Rows from view using AS OF business time
Row is not in the view because
BUS_BEG IS > 12/31/2012
Row is not in the view because
BUS_END <=01/01/2012
Row is in the view, but not returned
because Business End time is
exclusive
BUS_END = 12/30/2012
181
VER
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Versioning & Views Temporal Modifications ...
UPDATE or DELETE with the FOR PORTION OF clause can be
applied to Views
Temporal modifications can cause rows to be split
– Rows that are created by splitting a row through a VIEW update
may not be visible in the view after the update
Symmetric Views are Views WITH CHECK OPTION
– Temporal modifications are not constrained by the check option
– Split rows that disappear from the View definition are still allowed
for a complete temporal modification
1/1/2004
Not visible
via View
A054
P667
A054
P667
182
VER
A054
12/31/9999
10/31/2012
5/1/2011
PPO
A054
P667
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
XML
XQUERY Prolog
New XQUERY Constructors
Castable Expressions
Implicit Document Node
Cross loader support
High Volume XML INSERTs
Validation Improvements
XML Query Rewrite
Conditional Support
XMLTABLE Performance
183
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
XML Business Value
Make using XML even easier for developers
Align XML usage across DB2 for z/OS and DB2 for
Linux, Unix and Windows (LUW)
Supports the most requested features
Improve performance of XML
184
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
XQUERY ...
FLOWR expressions
– Retrofitted to DB2 10
– Give developers more power to write XML applications in a familiar language
1) Create table
3) Select items, part numbers and amount
for orders whose items are ordered in an
amount greater than $100
2) Insert XML document into table
4) Output from FLOWR expression
185
XML
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
XQUERY Prolog ...
The prolog contains a series of optional declarations that define the
processing environment for the expression
– Boundary space
• Controls if whitespace is preserved or stripped in XQuery
– Copy-namespaces declaration
• Controls how namespace bindings are assigned when copying elements or nodes
– Namespace declaration
• Declares a namespace prefix and associates the prefix with a namespace URI
– Default namespace declaration
• Declares namespace to use for unprefixed qualified names
Prolog
186
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Additional XQuery constructors ...
Document node constructor
– Construct the root node of an XML document .
– Equivalent to the XMLDOCUMENT function, but can be used in XQuery
expression
Comment node constructor
– Construct an XML comment node.
– Equivalent to the XMLCOMMENT function, but can be used in XQuery
expression
Processing Instruction constructor
– Construct an XML processing instruction (PI) node.
• PIs are instructions for applications,
• PI begins with a target used to identify the application to which the
instruction is directed.
– Equivalent to the XMLPI function, but can be used in XQuery expression
See the pureXML Guide section on XQuery constructors
187
XML
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
XQUERY Castable Expressions ...
CASTABLE tests whether a given value is castable to a given target
type
CASTABLE can be used to avoid errors at evaluation time
2) Use Castable to return invalid price as 0
1) INSERT XML into table
3) Results
In this example, the XML has a price of AAAAAA for partno AAA-AA
Using the Castable function returns a 0 instead of an invalid decimal number
188
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Implicit Document Node
Pre DB2 11 SQL requires an explicit XMLDOCUMENT function
during insert/update.
– XML requires a “root node”. Pre DB2 11, you could not insert a document using
XMLELEMENT, because it would not create the “root node”.
– In V11, you can use XMLELEMENT to insert an XML document
SQLCODE -20345 in V10
SQLCODE of 0 in V11
Required V10 syntax
189
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Crossloader support
Tables with XML columns can now be the target of
the Crossloader utility
190
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
High Volume XML INSERTs (retrofit 9/10)
Problem is that the DOCID values are created in sequential order
– This leads to hotspots in these indexes during high volume INSERT
processing
Solution is to use Randomized DOCID
ZParm XML_RANDOMIZE_DOCID to control
Does not apply to existing XML columns, must drop and
recreate to apply
Field QWP1XRDI is added in IFCID 106 to trace the
XML_RANDOMIZE_DOCID internal setting
191
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
Validation Improvements Binary XML
PRE DB2 11, to validate binary XML, we needed to serialize the
XML before we could validate
IN DB2 11, we can validate binary XML in both LOAD and
INSERT processing without serializing
– XML is always stored in parsed format
– Eliminate the need to serialize before storing
– Dependency on z/OS 1.13 PTF OA36712 or 1.12 PTF OA38633
Performance improvement
– INSERT into XML column with type modifier using binary XML
• 30~40% CPU reduction
• 15~18% CPU reduction vs. string XML
– LOAD into XML column with type modifier using binary XML
• 41% CPU reduction vs. DB2 10,
• 18% improvement vs. string XML
192
XML
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
XML Query Rewrite
Transforming the XML query into a better performing query
Original Query
Original Query
Price > 100 predicate is
pushed down into the “for”
clause for better performance
Simple query is converted to
XPath expression for better
performance
193
XML
© 2015 IBM Corporation
DB2 11 for z/OS
Migration Planning Workshop
Conditional Support
Calculate the shipping cost of an order
– Total shipping cost = "Per Shipment" cost + "Per Item" cost * # of items
XML Document
Conditional XQUERY statement
Result
194
© 2015 IBM Corporation
Migration Planning Workshop
DB2 11 for z/OS
XMLTABLE Performance
Remove unreferenced column definitions.
Merge common column path expressions
Date and Timestamp predicates pushdown
Optimize index key range for Varchar predicates
Process large number of small documents much more
efficiently
Push cast into column XPath
195
© 2015 IBM Corporation