TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
DB 2
„Deep dive into DB2“
Bruno Friess
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
Agenda:
TSM/DB2 installation
Download, installation, and then: where is it?
DB2 instance creation
DB2 process overview
Multi­threaded engine
Process and threads
Settings for dbm and db
Tuning for performance, scalability
Memory and virtual memory manager
Database reorganization
DB2 logs
db2diaglog
FODC, trace...
HADR – high availability disaster recovery
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
Download and installation
Where to download?
http://ftp.software.ibm.com/storage/tivoli­storage­management/ is still there.
●
●
●
●
You can use IBM FixCentral as well.
Never use double­zeros (6.3.0.0 etc.)
There are maintenance and patch levels
Packages including the license files are available for paying customers
It's one file. Extract the file and start with installation over CLI or Java.
Installation logfiles are located in /var/tivoli/tsm
Default DB2 Install Path is /opt/IBM/db2/V9.7 but TSM uses /opt/tivoli/tsm
If the installation is not successful:
Use the un­installer located in /opt/tivoli/tsm/_uninst
If DE hangs: delete /var/ibm/common/acsi and /usr/ibm/common/acsi
TSM installation is using db2_install, which will install all needed packages, but will not perform user and group creation, instance creation, or configuration.
TSM un­installation is using db2_deinstall.
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
DB2? Where is it?
After a successful installation, there is no rpm or lpp inventory.
rpm -qa or lslpp -l doesn't show anything about DB2.
Useful: /usr/local/bin/db2ls
With version 6.2:
> db2ls
Install Path
Level
Fix Pack Install Date
Installer UID
--------------------------------------------------------------------------------/opt/tivoli/tsm/db2
9.7.0.2
2 Sun Feb 27 17:58 2011 CET
0
With version 6.3:
> db2ls
Install Path
Level
Fix Pack Install Date
Installer UID
--------------------------------------------------------------------------------/opt/tivoli/tsm/db2
9.7.0.5
5 Wed Mar 20 22:16 2013 CDT
0
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
DB2 instance creation
A DB2 instance is an environment in which you store data and run applications
(= a database manager, a virtual server environment or „container“). Run the db2icrt command. DB2DIR/instance/db2icrt ­a AuthType ­u FencedID InstName
Using the user tsm01 and instance tsm01:
> db2icrt -a SERVER -u tsm01 tsm01
fenced user ID
instance name
Three remaining commands...
> db2 update dbm cfg using diagpath <any_path>
> db2 update dbm cfg using dftdbpath <home_of_user>
> db2set -i tsm01 DB2CODEPAGE=819
...end of TSM installation guide.
(Other instance commands: db2idrop, db2ilist...)
→ Now? What is going on? 17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
DB2 process overview
Remote
Client
Agent
DB2 Data Server
db2sysc
Instance Level
db2start
Remote
Listeners
Local
Client
Application
db2wdog
db2vend
db2fmp
db2acd
F
I
R
E
W
A
L
L
dbsysc
db2ipccm
Idle
Agent
Pool
dbaiothr
Application Level
Database Level
per Request
db2agent
db2agent
db2agntp
db2agntp
…
db2agntp
db2loggw
db2pfchr
db2pclnr
db2loggr
db2dlock
tiny hole for the NSA
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
db2lmr
db2med
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
DB2 common threads
db2sysc
System controller thread, which is responsible for startup, shutdown
and management of the running instance
db2agent
Coordinator agent that performs database operations on behalf of applications
db2agntp
Active subagent spawned if INTRA_PARALLEL is set to YES
db2pfchr
Asynchronous bufferpool prefetcher (NUM_IOSERVERS)
db2pclnr
Asynchronous bufferpool cleaner/writer (NUM_IOCLEANERS)
db2loggr
Manipulates log files to handle transaction processing and recovery
db2loggw
Writes log records to the log files
db2dlock
Thread responsible for deadlock detection
db2panic
„panic agent“ writes dump/log files during a crash (FODC) or when a limit is reached
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
DB2 common process
db2sysc
Main DB2 system controller or engine
db2acd
Autonomic computing daemon. Performs client side autonomic tasks, such as health
Monitor, maintenance (reorg) and scheduling
db2wdog
DB2 watchdog daemon cleans up resources if db2sysc abnormally terminates
db2vend
Fenced vendor process runs outside of the engine
db2fmp
Fenced processes that run user code outside the firewall for stored procedures and user defined functions
> ps -ef
root
root
rlin01
rlin01
rlin01
rlin01
| grep db2
10193
1
12360
1
12373 12360
12389 12360
13287 12360
24860 12360
PID USER
12373 rlin01
17448 root
PR
18
15
0
0
0
0
0
0
Aug06
Aug06
Aug06
Aug06
Aug06
Aug06
?
?
?
?
?
?
00:07:59
00:00:00
02:27:38
00:10:22
00:11:20
00:00:08
/opt/tivoli/tsm/db2/bin/db2fmcd
db2wdog 0
db2sysc 0
db2acd 8a6678,14,1e014,2,0,1,11fc0,388009,2,6800c
db2fmp (8a6678,14,1e014,2,0,1,31fc0,388009,2,77802f
db2fmp (8a6678,14,1e014,2,0,1,91fc0,388009,2,ab68039
NI VIRT RES SHR S %CPU %MEM
TIME+ COMMAND
0 2164m 300m 258m S 0.3 7.9 147:40.70 db2sysc
0 12764 1148 824 R 0.3 0.0
0:00.20 top
Linux top
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
DB2 settings for dbm and db
Settings for the database manager
> db2 get dbm cfg
Settings for the database (you have to connect to TSMDB1)
> connect to TSMDB1
> db2 get db cfg
> disconnect TSMDB1
IBM: „No change needed“
Before changing, please consider the TSM 6.4 performance guide.
Read the DB2 manuals, especially the overview of the performance impact.
Some of them needs a stop/start.
How to change/update:
> db2 update dbm cfg using <parameter> <value>
> db2 update db cfg using <parameter> <value>
To reset all (!!) values to their default:
> db2 reset dbm cfg
> db2 reset db cfg
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
DB2 settings for dbm and db
Potentially parameters of dbm to change:
name
online
impact
explanation
dftdbpath
yes
none
Path to the database meta-data
diaglevel
yes
low
Sets the diag level, default is 3, range is 0-4
diagpath
yes
none
Path to the DB2 diag files
intra_parallel
no
high
Enables/Disables Intrapartition parallelism
IBM Lab recommends „NO“, default is -1
max_querydegree
yes
high
Max number of parallel threads in one SQL statement
IBM Lab recommends 1, default is ANY (or -1)
notifylevel
yes
medium
Admin notification level (file instance.nfy), default is
3, 1 is enough
Potentially parameters of db to change:
name
online
impact
explanation
auto_maint
yes
medium
Maintenance values (Reorg, runstats, auto db backup..)
dft_degree
yes
high
Switch off intrapartition parallelism, default is 1
IBM Lab recommends „1“
longindexbuild
yes
none
Needed for HADR with ON, default is OFF
and: newlogpath, mirrorlogpath, overflowlogpath, failarchpath...
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
DB2 memory STMM – Self tuning memory manager
Default setting for TSM is to configure DB2 memory usage to AUTOMATIC.
DBMemPercent Auto
Within „get db cfg“ you will see:
Self tuning memory
(SELF_TUNING_MEM)
Size of database shared memory (4KB) (DATABASE_MEMORY)
Database memory threshold
(DB_MEM_THRESH)
Max storage for lock list (4KB)
(LOCKLIST)
Percent. of lock lists per application
(MAXLOCKS)
Package cache size (4KB)
(PCKCACHESZ)
Sort heap thres for shared sorts (4KB) (SHEAPTHRES_SHR)
Sort list heap (4KB)
(SORTHEAP)
=
=
=
=
=
=
=
=
In the db2 diaglog you will see:
CHANGE
ON
AUTOMATIC(135180)
10
AUTOMATIC(4120)
AUTOMATIC(97)
AUTOMATIC(29586)
AUTOMATIC(268)
AUTOMATIC(53)
: CFG DBM: "Instance_memory" From: "20" <automatic>
Log information is written into:
To: "780850" <automatic>
[DIAGPATH]/stmmlog
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
DB2 memory What does this mean?
If you have one TSM server on one machine, then automatic is ok.
If not, then it is difficult: TSM allows only a range from 10 to 99% of the memory.
What if you have one machine with 32 GB RAM and three instances?
What happens when you have 512 GB RAM and one instance? Or 20 instances??
TSM is using:
> db2pd -dbptnmem
Database Partition 0 -- Active -- Up 1 days 17:07:21 -- Date 09/13/2013 08:25:21
Database Partition Memory Controller Statistics
Controller Automatic: Y
Total memory
Memory Limit:
119984660 KB
Current usage:
66347392 KB
HWM usage:
74097600 KB
TSM + DB2
Cached memory:
6702912 KB
Individual Memory Consumers:
Name
Mem Used (KB) HWM Used (KB) Cached (KB)
========================================================
APPL-TSMDB1
58048
160000
10624
DBMS-tsmp01
126464
126464
48448
FMP_RESOURCES
22528
22528
0
PRIVATE
157376
409280
32320
DB-TSMDB1
65967104
73406400
6611520
db2sysc
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
Database reorganization (see IBM technote 1452146 or the presentation of N.Pott)
Is reorg needed? db2 reorgchk current statistics on table all >db2reorgchk.out
Settings of the database:
Automatic maintenance
Automatic database backup
Automatic table maintenance
Automatic runstats
Automatic statement statistics
Automatic statistics profiling
Automatic profile updates
Automatic reorganization
(AUTO_MAINT)
(AUTO_DB_BACKUP)
(AUTO_TBL_MAINT)
(AUTO_RUNSTATS)
(AUTO_STMT_STATS)
(AUTO_STATS_PROF)
(AUTO_PROF_UPD)
(AUTO_REORG)
=
=
=
=
=
=
=
=
ON
OFF
ON
ON
ON
OFF
OFF
OFF
Is Reorg on?
TSM options:
(6.1.5/6.2.3 or higher)
all are online
changeable
(setopt)
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
Database reorganization ….now it's reorganized, but I still have no more space in the filesystem.
To get the free space which is at the end of the tables now, try this:
You can reduce the space of the database in the filsystem by „reducing“ it.
This is for DB2 9.7 and only if this feature is switched on.
> db2 connect to tsmdb1
> db2 set schema tsmdb1
> db2 "select reclaimable_space_enabled from \
table(mon_get_tablespace('',-1)) where tbsp_id in (2,4,5,6)"
If all tablespaces show, that reclaimable_space_enabled=1 then you can start with reducing:
>
>
>
>
db2
db2
db2
db2
ALTER
ALTER
ALTER
ALTER
TABLESPACE
TABLESPACE
TABLESPACE
TABLESPACE
USERSPACE1 REDUCE MAX
IDXSPACE1 REDUCE MAX
LARGESPACE1 REDUCE MAX
LARGEIDXSPACE1 REDUCE MAX
Remarks:
●
All those ALTER process are slow, please start one after another
●
Can be done during TSM operating (online)
●
But consider I/O on your disks / filesystems
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
DB2 log files Set the diaglog size and location:
> db2 update dbm cfg using diagpath <path_to_diaglog>
> db2 update dbm cfg using diagsize 1024
If you have more space, then >1024 is ok.
Inside this log, there are a lot of messages and most of them are difficult to read.
IBM needs this file for error analysis.
0 to 9
Content of the [DIAGPATH]
-rw-rw-rw-rw-rw-rw-rw-r----drwxrwxrwx
drwxr-x---rw-rw-rw-rw-rw-rwdrwxrwxr-t
1
1
1
2
3
1
1
2
rlin02
rlin02
rlin02
rlin02
rlin02
rlin02
rlin02
rlin02
Automatic
tsmsrv 28765146 Sep 13 14:52 db2diag.0.log
FODC
tsmsrv
58231 Oct 15 2011 db2diag.log
tsmsrv 6291316 Aug 6 15:29 db2eventlog.000
tsmsrv
8192 Sep 12 20:55 events
tsmsrv
8192 Aug 6 10:31 FODC_Panic_2013-08-06-10.30.49.236060
tsmsrv
67042 Sep 13 14:41 rlin02.0.nfy
Admin
tsmsrv
763 Oct 15 2011 rlin02.nfy
notification
tsmsrv
8192 Sep 12 08:27 stmmlog
FODC – First occurence data collection
(a kind of dump files collection)
Old FODC directories can be deleted / archived
DB2 memory logs
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
DB2 log files Strange files in [DIAGPATH] or in sqllib/db2dump:
Hundreds or thousands of *.dump.bin, *.stack.txt, *.apm.bin and *.trap.txt.
For more information see IC72708: „Dump files fills up filesystem“
TSM will report „ANR1545W Add more space to the file system.“
Check this place and delete the files (as a workaround)
Old files can be deleted while TSM / DB2 is running
Files are needed by manual FODC command
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
HADR
High availability disaster recovery.
It replicates data in real time from a primary database to one or more standby databases.
Multiple standbys are introduced with DB2 10.1 (see TSM 7.1).
To use HADR with TSM no additional licensing is necessary!!!
Easy solution
Is integrated into DB2
Only needs TCP (no additional software / hardware)
Setup only requires a few configuration parameters (primary/standby network address, and hadr_syncmode)
Only 3 commands to manage: start/stop/takeover HADR
Choose the best sync mode for your business: sync, nearSync, async, superAsync
Automatic reconfiguration after failover or role switch in multiple standbys
But...
Multiple standbys
Delayed replay starts with DB2 version 10
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
HADR
Quick installation...
1. Priming the database
●
●
●
Backup of the primary database to disk Drop database on second node
Restore database on second node
2. Use the same ports for this instance in /etc/services on both machines
3. Configure HADR
db2
db2
db2
db2
db2
db2
db2
update
update
update
update
update
update
update
db
db
db
db
db
db
db
cfg
cfg
cfg
cfg
cfg
cfg
cfg
for
for
for
for
for
for
for
tsmdb1
tsmdb1
tsmdb1
tsmdb1
tsmdb1
tsmdb1
tsmdb1
using
using
using
using
using
using
using
hadr_local_host
hadr_local_svc
hadr_remote_host
hadr_remote_inst
hadr_remote_svc
hadr_syncmode
hadr_timeout
exaix01.exstor.de
60010
exaix02.exstor.de
tsm01
60010
SYNC
120
4. Start HADR
db2 start hadr on db TSMDB1 as standby (on standby server)
db2 start hadr on db TSMDB1 as primary (on primary server)
5. Failover with HADR
Shutdown DB2 on primary server
db2 takeover hadr on db TSMDB1 by force (on standby server)
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany
Swap this on
primary and secondary
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors
Bruno Friess
bf@exstor.de
GPS: 52.511992,13.392902
17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany