Interesting SQL Solutions to
Real Life Problems
Anthony Tichonoff
Florida Hospital Sr. DB2 DBA
Tampa Bay RUG Meeting
Fri, Feb 21, 2013 12:45 AM - 1:45 AM
Presentation Overview
• Using the full power of SQL as a programming language
• Fully exploiting common SQL features in creative ways
• Fun SQL –
•
•
•
•
•
•
•
Real Time Statistics Auditing DASD Growth
Probabilistic Matching
DB2 Catalog Logger
Auditing Temporal Data for Errors
Calendars
Dynamic Screens
DDL SQL Generation
• Implementing interesting SQL in different languages
2
The Challenge
I need to identify DB2
objects that grow rapidly.
3
The Steps
Build DB2 table to collect daily
RTS History Stats
- Columns for RTS History Table
Insert both tablespace and
indexspaces into RTS History table
- SQL Insert for RTS Tablespace
- SQL Insert for RTS Indexspace
SQL to determine DASD Growth
- SQL for DASD Growth
4
The Breakdown
Insert
Select
SQL
SQL
5
The
Insert
SQL
INSERT INTO {Your RTS History Table}
SELECT CURRENT TIMESTAMP
,A.DBNAME
,A.NAME
,'T'
,A.PARTITION
,COALESCE(INT(A.TOTALROWS),0)
,COALESCE(A.NACTIVE,0)
,COALESCE(A.SPACE,0)
,COALESCE(A.EXTENTS,0)
,SUBSTR(B.STORNAME,1,8)
AS T_STATS
AS I_DATB
AS I_OBJC
AS I_TYPE_OBJC
AS I_PART
AS Q_ROWS
AS Q_PAGE_ACTV
AS Q_SPAC_KB
AS Q_EXTN
AS I_N_STOR
FROM SYSIBM.SYSTABLESPACESTATS A
JOIN SYSIBM.SYSTABLEPART B
ON B.DBNAME
= A.DBNAME
AND B.TSNAME
= A.NAME
AND B.PARTITION
= A.PARTITION
JOIN SYSIBM.SYSTABLESPACE C
ON C.DBNAME
= A.DBNAME
AND C.NAME
= A.NAME
AND C.DBID
= A.DBID
AND C.PSID
= A.PSID
WHERE {Your Filter}
6
Tablespace
Growth SQL
Part 1
WITH RTS_HISTORY
( I_DATB
,I_OBJC
,T_STATS
,Q_ROWS
,Q_PAGES
,Q_SPAC_KB
,Q_EXTN
) AS
(
SELECT
I_DATB
,I_OBJC ,T_STATS
,SUM(Q_ROWS)
AS Q_ROWS
,SUM(Q_PAGE_ACTV)
AS Q_PAGES
,SUM(Q_SPAC_KB)
AS Q_SPAC_KB
,MAX(Q_EXTN)
AS Q_EXTN
FROM { Your RTS History Table }
WHERE I_TYPE_OBJC
= ‘T’
AND T_STATS BETWEEN CURRENT TIMESTAMP –
(&Q_DAYS + 1) DAYS
AND CURRENT TIMESTAMP
GROUP BY
I_DATB
,I_OBJC
,T_STATS
)
7
SELECT C.I_DATB,
FROM
JOIN
Tablespace
Growth SQL
ON
JOIN
Part 2
ON
JOIN
ON
C.I_OBJC
,&Q_DAYS
AS Q_DAYS
,Output : Row, Page, Space, Extent Data
(
) L
(
SELECT Minimum Values by Database & Object
FROM RTS_HISTORY
SELECT Maximum Values by Database & Object
FROM RTS_HISTORY
) H
H.I_DATB = L.I_DATB AND H.I_OBJC
(
SELECT Current Values by Database & Object
FROM RTS_HISTORY
WHERE T_STATS = SubSelect for MAX(T_STATS)
) C
C.I_DATB = L.I_DATB AND C.I_OBJC
(
= L.I_OBJC
= L.I_OBJC
SELECT Oldest Values by Database & Object
FROM RTS_HISTORY
WHERE T_STATS = SubSelect for MIN(T_STATS)
) O
O.I_DATB = L.I_DATB AND O.I_OBJC
= L.I_OBJC
8
The Output
Rows
Database
Object
# Days
Delta
Max
DKM97PRD
SKM9732
30
-162,082,402
248,311,973
56,125,070
DKM97PRD
SKM9733
30
-35,025,961
49,524,240
9,494,040
DIS97PRD
SIS9761
30
-28,795,566
201,123,146
124,806,402
DHC97PRD
SHC9708
30
22,001,796
-
62,233,272
DGL97PRD
SGL9749
30
18,007,738
-
229,901,672
Pages
Current
Space MB
Delta
Max
Current
Delta
Max
Current
-1,889,280
3,021,840
810,000
-8,136
12,528
3,240
-406,440
603,730
133,210
-1,620
2,628
720
-1,299,136
6,105,568
3,292,608
-5,231
24,496
13,232
769,625
-
2,282,476
3,008
-
9,180
67,841
-
3,160,942
272
-
12,962
9
The Challenge
I need DB2 to match people
and identify duplicates from
two separate systems.
10
The Breakdown & Steps
L
o
a
d
11
The SQL
EX
1
0/Many
IN
Select
Medical Record Id (Patient Key)
,Demographic Data (Compare Keys)
From Internal Demographic Table
Where
IN.SSN
= :EX.SSN
Union
IN.Last Name = :EX.Last Name
& IN.DOB
Union
IN.Driver Lic
Union
•
•
•
=
:EX.DOB
=
:EX.Driver Lic
Result Set outputs
0 to Many Keys
Match ?
Possible match,
more processing
needed to confirm.
No Match
No rows located in
the internal table.
12
The SQL
Check
Compare Keys
If EX.cKey1 = IN.cKey1 and
EX.cKey2 = IN.cKey2 and
…
Then Add to Score
If EX.cKey4 = IN.cKey4 and
EX.cKey5 = IN.cKey5 and
EX.cKey6 = IN.cKey6 and
Match
Ratio
Score
Full
1:1
100%
Partial
1:M
1% - 100%
No
1:0
0%
…
Then Add to Score
13
The Output
FULL
Match
Insert into Cross
Reference Table
PARTIAL
Match
Match
or
Reject
NO
Match
Insert into
Demographic &
Cross Ref Table
14
The Challenge
Can I audit the differences in
two different DB2 subsystems.
15
The Breakdown & Steps
•
•
•
•
•
Create Snap Shot Catalog Tables
Create Catalog Chronology Table
Program must utilize Declare Global Temp Tables to collect
all catalog data for all DB2 Subsystems
Use Full Outer Joins comparing data for changes
Differences are loaded into Catalog Chronology Table
DB2 Subsystem A
Catalog Chronology
DB2 Subsystem B
Databases -Tablespaces -Tablespace Parts
Tables -Table Columns-Indexes-Index Parts-Index Columns
Views-View Columns-Aliases
Routines -Triggers
16
The Breakdown
DB2 Catalog
Logger Program
17
The SQL
Processing DB2 Declared Temporary Tables
Objects exists in both tables but the
column values are different then
object was Altered.
Objects only exists in snapshot catalog then
object was Dropped.
Objects only exists in current catalog then
object was Created.
18
The Output
Work Date
Type
Object
Name
Creator
Work
Operator
Request
Type
Comment
9/12/2012
X
XRSD97321
PCTL
AAT89A
AO
ALTERED PRIMARY
OLD 36,000 NEW 108,000
8/2/2012
8/2/2012
8/2/2012
8/2/2012
8/2/2012
8/2/2012
T
R
R
X
X
T
SSD9732
RSD97321
RSD97320
XSD97321
XSD97320
SSD9732
PCTL
PCTL
PCTL
PCTL
PCTL
PCTL
AAT89A
AAT89A
AAT89A
AAT89A
AAT89A
AAT89A
CO
CO
CO
CO
CO
CO
CREATED NEW TABLESPACE
CREATED NEW TRIGGER
CREATED NEW TRIGGER
CREATED NEW INDEX
CREATED NEW INDEX
CREATED NEW TABLE
2/27/2012
T
SSD9732
TEST
RLB77C
AC
F_VALD_PDX_LOCK
"LOCK PRINCIPAL DIAGNOSIS"
1/4/2012
1/4/2012
1/4/2012
1/4/2012
1/4/2012
1/4/2012
R
R
X
T
X
T
RSD97321
RSD97320
XSD97321
SSD9732
XSD97320
SSD9732
TEST
TEST
TEST
TEST
TEST
TEST
RLB77C
RLB77C
RLB77C
RLB77C
RLB77C
RLB77C
CO
CO
CO
CO
CO
CO
CREATED NEW TRIGGER
CREATED NEW TRIGGER
CREATED NEW INDEX
CREATED NEW TABLESPACE
CREATED NEW INDEX
CREATED NEW TABLE
19
The Challenge
I need to audit the data in my user
maintained inclusive - inclusive
temporal tables.
20
The Breakdown & Steps
Inclusive - Inclusive Time Series
SQL
Same row Check
Start Overlapped 
End Overlapped 
Encapsulated 
SELECT
A.I_KEY
,Output Data
FROM SD00 A JOIN SD00 B
ON A.I_Key
= B.I_KEY
WHERE A.D_STAR
<>
AND A.D_END
<>
AND
( ( A.D_STAR
<=
AND A.D_STAR
<=
AND
( A.D_END
>=
OR A.D_END
>=
)
OR ( A.D_END
>=
AND A.D_END
>=
AND
( A.D_STAR
<=
OR A.D_STAR
<=
)
OR ( A.D_STAR
>=
AND A.D_END
<=
)
Join to Self
B.D_STAR
B.D_END
B.D_STAR
B.D_END
B.D_STAR
B.D_END )
B.D_STAR
B.D_END
B.D_STAR
B.D_END )
B.D_STAR
B.D_END )
22
SQL
UNION ALL
SELECT
A.I_KEY
,Output Data
FROM SD00 A
ON A.I_Key
Gap Check 
Find Next End Date 
Is Gap Span > 1 Day 
Join to Self
JOIN SD00 B
= B.I_KEY
WHERE &GAPS
= 'Y'
AND B.D_STAR
> A.D_STAR
AND B.D_END
=
(
SELECT MIN(C.D_END)
FROM SD00 C
WHERE C.I_KEY
= A.I_KEY
AND C.D_END > A.D_END
)
AND DAYS(B.D_STAR) – DAYS(A.D_END) > 1
ORDER BY
I_KEY
,D_STAR_A
WITH UR
23
The Output
Key
Error
Start -A
End - A
Start - B
End - B
KEY E
EMBEDDED
01/01/0001
12/31/9999
04/10/2008
12/31/2050
KEY E
EMBEDDED
04/10/2008
12/31/2050
01/01/0001
12/31/9999
KEY G
GAP
10/23/2002
12/12/2002
12/18/2002
01/08/2003
KEY G
GAP
12/18/2002
01/08/2003
09/05/2003
12/31/9999
KEY O
OVERLAP
07/01/2004
11/16/2005
10/02/2005
10/31/2009
KEY O
OVERLAP
11/01/2009
01/25/2010
01/02/2010
12/31/9999
24
The Challenge
I need to build a dynamic
calendar in my application.
25
The Breakdown & Steps
1. Calendar 4 dimensional array
a.
b.
c.
d.
Year
Month
Week of Year
Day of Week
2. Need row for each day of month
(Use Recursive SQL)
3. Use SQL Functions for dimensions
a. Week of the Year (Group by)
b. Day of the Week
4. Assign Day of Week to each column
(Sun, Mon, Tue, Wed, Thu, Fri, Sat)
26
SQL
Must build a result set with a row for each day of the month
WITH DAYTAB
(
Recursive SQL
D_DAY
)
AS
(
SELECT
LAST_DAY(DATE(&DATE) – 1 MONTH) + 1 DAY AS D_DAY
FROM SYSIBM.SYSDUMMY1
UNION ALL
SELECT
D_DAY + 1 DAY
FROM DAYTAB
WHERE D_DAY < LAST_DAY(&DATE)
)
AS D_DAY
27
SQL
Formats 
Output & Builds
Week Day Columns
Calculates 
Dimensions for
Week Of Year
& Day Of Week
SELECT
DAYOFYEAR(&DATE)
AS DY
,YEAR(&DATE)
AS YY
,UCASE(
(SUBSTR(CHAR(DATE(&DATE), LOCAL),1,
POSSTR(CHAR(DATE(&DATE), LOCAL),' '))
) )
AS MM_NAME
, WY_DEM
,MAX(CASE WHEN DW_DEM = 1 THEN DD ELSE '' END) AS SUN
,MAX(CASE WHEN DW_DEM = 2 THEN DD ELSE '' END) AS MON
,MAX(CASE WHEN DW_DEM = 3 THEN DD ELSE '' END) AS TUE
,MAX(CASE WHEN DW_DEM = 4 THEN DD ELSE '' END) AS WED
,MAX(CASE WHEN DW_DEM = 5 THEN DD ELSE '' END) AS THU
,MAX(CASE WHEN DW_DEM = 6 THEN DD ELSE '' END) AS FRI
,MAX(CASE WHEN DW_DEM = 7 THEN DD ELSE '' END) AS SAT
FROM
(
SELECT
WEEK(D_DAY)
AS WY_DEM
,DAYOFWEEK(D_DAY)
AS DW_DEM
,RIGHT(' ' || STRIP(CHAR(DAY(D_DAY))),2)
AS DD
FROM DAYTAB
)X
GROUP BY
WY_DEM
WITH UR
28
The Output
D_DAY
02/01/2013
02/02/2013
02/03/2013
02/04/2013
02/05/2013
02/06/2013
02/07/2013
02/08/2013
02/09/2013
02/10/2013
02/11/2013
02/12/2013
02/13/2013
02/14/2013
02/15/2013
02/16/2013
02/17/2013
02/18/2013
02/19/2013
02/20/2013
02/21/2013
02/22/2013
02/23/2013
02/24/2013
02/25/2013
02/26/2013
02/27/2013
02/28/2013
49
FEBRUARY
2013
S M T W T F S
-- -- -- -- -- -- -1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28
QMF Result Set
using a form
Program
Display
Recursive Table Result Set
29
The Challenge
I need to build dynamic report
screens in CICS with scrolling.
30
The Breakdown & Steps
Hospital Grand Totals (Counts & Totals)
$$$$
+ Operator
+ Campus
+ Accounts
$$$$
Errors to work
...
...
...
Campus Totals
$$$$
...
Operator Totals
$$$$
31
The SQL – Utilize Nested Common Table Expressions
Build common table expression for all
account error detail – AccnDetail
32
The SQL – Utilize Nested Common Table Expressions
With
Account Details
AccnDetail
(
Type Row
,Account
,Campus
,Operator
,Account Details
)
AS
(
Select
‘2D’ as Type Row
,Account
,Campus
,Operator
,Other Details
From { User Table_A }
Join { User Table_B }
Join { User Table_C }
)
Where {Account Detail Filtering}
,AccnSumm
,CampSumm
,OprtSumm
,HospSumm
( Continue … )
( Continue … )
( Continue … )
( Continue … )
33
The SQL – Utilize Nested Common Table Expressions
With
Account Summary
AccnDetail
( Previous … )
,AccnSumm
( Continue … )
(
Type Row
,Account
,Campus
,Operator
,Aggregated Account Details
)
AS
(
Select
‘1H’ as Type Row
,Account
,Campus
,Operator
, Aggregated Account Details
)
From { AccnDetail }
Group By
Operator
,Campus
,Account
,CampSumm
,OprtSumm
,HospSumm
( Continue … )
( Continue … )
( Continue … )
34
The SQL – Utilize Nested Common Table Expressions
With
Campus Summary
AccnDetail
( Previous … )
,AccnSumm
( Previous … )
,CampSumm
(
Type Row
,Account
,Campus
,Operator
,Aggregated Account Summary
)
AS
(
Select
‘3C’ as Type Row
,Null Account ,Campus
,Operator
, Aggregated Account Summary
)
From { AccnSumm }
Group By
Operator
,Campus
,OprtSumm
,HospSumm
( Continue … )
( Continue … )
35
The SQL – Utilize Nested Common Table Expressions
With
Operator Summary
AccnDetail
( Previous … )
,AccnSumm
( Previous … )
,CampSumm ( Previous … )
,OprtSumm
(
Type Row
,Account
,Campus
,Operator
,Aggregated Campus Summary
)
AS
(
Select
‘4O’ as Type Row
,Null Account ,Null Campus
,Operator
, Aggregated Campus Summary
)
From { CampSumm }
Group By
Operator
,HospSumm
( Continue … )
36
The SQL – Utilize Nested Common Table Expressions
With
Hospital Summary
AccnDetail
( Previous … )
,AccnSumm
( Previous … )
,CampSumm ( Previous … )
,OprtSumm
( Previous … )
,HospSumm
(
Type Row
,Account
,Campus
,Operator
,Aggregated Operator Summary
)
AS
(
Select
‘5T’ as Type Row
,Null Account ,Null Campus
,Null Operator
,Aggregated Operator Summary
From { OprtSumm }
)
37
The SQL – Utilize Nested Common Table Expressions
SELECT *
FROM
( SELECT Type Row , Account, Campus, Operator
, Account Details
FROM AccnDetail Where ShowDetail = ‘Y’ )
Union All
( SELECT Type Row , Account, Campus, Operator
,Aggregated Account Details
FROM AccnSumm Where ShowAccn = ‘Y’)
Union All
( SELECT Type Row , Account, Campus, Operator
,Aggregated Account Summary
FROM CampSumm Where ShowCampus = ‘Y’)
Union All
( SELECT Type Row , Account, Campus, Operator
,Aggregated Campus Summary
FROM OprtSumm )
Union All
( SELECT Type Row , Account, Campus, Operator
,Aggregated Operator Summary
FROM HospSumm )
Main SQL
ORDER BY Operator, Campus, Account, Type Row
38
Account Detail
Campus Summary
The Output
Row
Type
Operator
Campus Account
1H
#
Errors
3544
Account Summary
Operator Summary
Hospital Summary
Charges
$ 5,182,626.03
#
Accounts
Error Id
0
1760
1A
2D
2D
AGU2E0
01
22814252
1
$
213.00
0
183250672
183270843
1
0
0
1A
2D
AGU2E0
01
22814405
2
$
213.00
0
183250703
1
0
3C
AGU2E0
01
3
$
426.00
0
2
1A
2D
2D
AGU2E0
02
2
$
31,166.50
0
183249295
183270206
1
0
0
3C
AGU2E0
02
2
$
31,166.50
0
1
4O
AGU2E0
5
$
31,592.50
0
3
22799806
39
The Output
40
The Challenge
I need to be able to generate
current DDL for all of my objects.
41
The Breakdown & Steps
Florida Hospital DDL Generator
A collection of Rexx modules which
regenerates our DDL from the DB2 Catalog
Rexx
Modules
DDLAL01
DDLCT01
DDLDB01
DDLGR01
DDLGR02
DDLGR03
DDLGR04
DDLIX01
DDLLO01
DDLV
DB2 Object to Regenerate
Aliases
Create All Objects
Databases
Table Grants
Database Grants
Tablespace Grants
Sequence Grants
Indexes
LOBs
Online View of DDL
Rexx
Modules
DDLMQ01
DDLRT01
DDLSQ01
DDLST01
DDLTB01
DDLTR01
DDLTS01
DDLVW01
DDLWR01
DDLG
DB2 Object to Regenerate
MQTs
Routines
Sequences
Object Stats Data
Tables
Triggers
Tablespaces
Views
Output: File or Online View
Generates DDL to a File
42
The SQL
Uses
Recursive
SQL to
Locate All
Views
WITH VIEWLIST
( BTYPE
,LEVEL
,BCREATOR
,BNAME
,DCREATOR
,DNAME
) AS
(
SELECT
ROOT.BTYPE
,1
,ROOT.BCREATOR
,ROOT.BNAME
,ROOT.DCREATOR
,ROOT.DNAME
FROM SYSIBM.SYSVIEWDEP ROOT
WHERE BCREATOR
= :CREATOR
AND BNAME
= :TBNAME
UNION ALL
SELECT
CHILD.BTYPE
,PARENT.LEVEL + 1 ,CHILD.BCREATOR
,CHILD.BNAME
,CHILD.DCREATOR ,CHILD.DNAME
FROM VIEWLIST PARENT
JOIN SYSIBM.SYSVIEWDEP CHILD
ON PARENT.DCREATOR
= CHILD.BCREATOR
AND PARENT.DNAME
= CHILD.BNAME
WHERE PARENT.LEVEL < 6
)
SELECT
BTYPE
,BNAME
FROM VIEWLIST
WITH UR
,LEVEL
,DCREATOR
,BCREATOR
,DNAME
43
The Output
TSO DDLV
D B 2
D D L
13/02/21 21:30
DB2 ===> DSN2
Object ===> PCTL.TDA9700
DDL Options:
Entire DDL ===> _
Object Only ===> _
Tablespace
Indexes
Aliases
Grants
Table
Views
Triggers
LOB
===>
===>
===>
===>
X
_
_
_
===>
===>
===>
===>
_
_
_
_
Enter "X" to select option
PF1 for Help
COMMAND ===>
44
The Output
TSO DDLV
--- DDL EXEC GENERATION BY: AAT89A - 13/02/21
-------------------------------------------------------- DDL WAS BUILT USING TABLE PCTL.TDA9700 AS INPUT
------------------------------------------------------------------------------------------------------------- THIS MEMBER CONTAINS SQL STATEMENTS TO CREATE:
-TABLESPACE
-- FOR DDA97PRD.SDA9700
------------------------------------------------------SET CURRENT SQLID = 'DBSYSADM'
;
--- ========== S T A T I S T I C S ==========
--- PCTL.TDA9700 - DB2 APPLICATION TABLE RECOVERY PROFILE
-ROW COUNT
6,267
-SIZE
0.0021 GB
-ROW LENGTH
279
-INDEXES
5
-PACK DEPS
61
-45
The Output
TSO DDLV
-- ========== T A B L E S P A C E ==========
---DROP TABLESPACE DDA97PRD.SDA9700
--;
--COMMIT WORK
--;
-CREATE
TABLESPACE
SDA9700
IN
DDA97PRD
USING
STOGROUP
PCTL
PRIQTY
720
SECQTY
3600
ERASE
NO
FREEPAGE
30
PCTFREE
10
COMPRESS
NO
SEGSIZE
64
BUFFERPOOL
BP2
LOCKSIZE
PAGE
LOCKMAX
SYSTEM
MAXROWS
255
CCSID
EBCDIC
CLOSE
NO
;
--- ========== T S - G R A N T S ==========
--
46
Interesting SQL Solutions to
Real Life Problems
Anthony Tichonoff
anthony.tichonoff@flhosp.org
Florida Hospital Sr. DB2 DBA
Tampa Bay RUG Meeting