Oracle 12c New Features Web Version 1 © 2013 - Julian Dyke juliandyke.com Agenda 2 Introduction Pluggable Database Partial Indexes Online Data File Move Online Partition Move Index Columns Invisible Columns Identity Clause Session Sequences Global Temporary Table Undo Temporal Validity Extended Columns Row Limiting Clause Histograms Application Continuity © 2013 - Julian Dyke juliandyke.com Introduction This presentation investigates a selection of Oracle 12c new features that I believe will be interesting to DBAs The presentation was originally delivered at the UKOUG Conference 2013 in Manchester, England I have added section headers containing comments and feedback from delegates 3 © 2013 - Julian Dyke juliandyke.com What History Tells Us.... 4 Oracle 9i , 10g and 11g R1 releases have been available for 18-24 months R2 releases have been available for several years R2 releases include terminal release Support has often been extended for terminal release CPU and PSU support is limited for R1 releases Longer and more comprehensive for R2 releases It is occasionally necessary to upgrade to a terminal release in order to migrate to new functionality In past releases there have been compatibility issues between new features Occasionally bugs.... Sometimes new features are documented but not released © 2013 - Julian Dyke juliandyke.com New Features Oracle Marketing concentrates on a limited subset of new features 5 Particularly new licensing options Product Managers and Pre Sales are usually a better source of information New features are often overlooked by everyone: Particularly additional features in Standard/Enterprise Editions Too many in each release to investigate them all Documentation and support is often limited at initial release © 2013 - Julian Dyke juliandyke.com Pluggable Database Other presenters will have discussed pluggable databases in more detail The concept was announced in September 2012 and I now believe it is time to consider how and where it is appropriate to deploy pluggable databases My example of a possible deployment was a container database with a large number of pluggable databases replacing SYBASE. I know that SYBASE replacement has been a goal at a few of the larger banks for many years. Something I missed is that pluggable databases can be cloned allowing test databases to be created rapidly from production databases. I have not investigated this feature yet, so have limited my comments to technical questions I would still like to answer 6 © 2013 - Julian Dyke juliandyke.com Pluggable Database 7 Definitely the most attractive marketing feature Easy to explain Impresses managers and technical staff Obvious benefits May be more efficient than virtualization for large numbers of similar databases For example migrations from SYBASE Potential reduction in resource consumption including: CPU memory background processes management costs © 2013 - Julian Dyke juliandyke.com Pluggable Database 8 Reduction in CPU might reduce processor license requirements Separately licensed as Oracle Multi Tenant option US list price is $17,500 per processor (EE is $47,500) All options will need to be licenced for all pluggable databases Possibly irrespective of usage e.g. Partitioning, Advanced Compression, Advanced Security © 2013 - Julian Dyke juliandyke.com Pluggable Database 9 Only one redo thread per container instance Online redo logs may be a bottleneck Data Guard Single configuration for container database Pluggable databases share redo thread May become difficult to manage if standby databases need to be rebuilt Single large SGA may increase size of kernel page tables area for each process (foreground / background or both) Will offset some of the savings in background process memory New In-Memory database may have same problem Pluggable databases may contend for resources e.g. RAC background processes © 2013 - Julian Dyke juliandyke.com Partial Indexes I believe this is one of the best features in Oracle 12c for sites using the Partitioning Option Most sites partition their tables based on time e.g. year, month, week, day etc. Most activity centres around the latest (hot) partitions where indexes are often required to optimize access paths. However, the cost of creating an index for the entire table often prevents creation of appropriate indexes as, in current versions, the index needs to be created for all partitions, requiring additional storage and increasing backup and restore times. Partial indexes will not reduce redo generation, but could significantly reduce overall database sizes as they will often affect the largest tables I think the current implementation is limited, but I still think this is a great feature 10 © 2013 - Julian Dyke juliandyke.com Partial Indexes 11 One of the most important new features in Oracle 12.1 Allows additional indexes to be created for performance tuning Potentially reduces amount of storage required for indexes May reduce backup and restore times Will probably not reduce redo / archive generation Functionality is limited For a specific table, only one set of table partitions can be enabled for index partitions © 2013 - Julian Dyke juliandyke.com Partial Indexes 12 Useful for range-based partitioned tables Create partial indexes on most recent (hot) partitions Alternatively create partial indexes on older (archived) partitions However cannot create partial indexes on both Partial indexing must specified on table partitions INDEXING ON – partial indexes enabled INDEXING OFF – partial indexes disabled If a table partition has INDEXING ON then all rows in that partition will be indexed in each partial index © 2013 - Julian Dyke juliandyke.com Partial Indexes Partial Local and Global Indexes CREATE TABLE pcar ( season_key NUMBER, race_key NUMBER, driver_key VARCHAR2(4), team_key VARCHAR2(3), position NUMBER, laps_completed NUMBER, race_points NUMBER ) PARTITION BY RANGE (season_key) ( PARTITION p2008 VALUES LESS THAN (2009) INDEXING OFF, PARTITION p2009 VALUES LESS THAN (2010) INDEXING OFF, PARTITION p2010 VALUES LESS THAN (2011) INDEXING OFF, PARTITION p2011 VALUES LESS THAN (2012) INDEXING ON, PARTITION p2012 VALUES LESS THAN (2013) INDEXING ON ); 13 © 2013 - Julian Dyke juliandyke.com Partial Indexes Partial Local and Global Indexes SELECT season_key, COUNT(*) FROM pcar GROUP BY season_key ORDER BY season_key; SEASON_KEY 2008 2009 2010 2011 2012 14 COUNT(*) 368 338 456 456 480 © 2013 - Julian Dyke juliandyke.com Partial Indexes Example - Partial Local Index CREATE INDEX pcar1 ON pcar (driver_key) LOCAL INDEXING PARTIAL; dbms_stats.gather_index_stats ( ownname => 'GP', indname => 'PCAR1', estimate_percent => NULL ); SELECT partition_name,num_rows FROM dba_ind_partitions WHERE index_name = 'PCAR1'; PARTITION_NAME P2008 P2009 P2010 P2011 P2012 15 NUM_ROWS 0 0 0 456 480 © 2013 - Julian Dyke juliandyke.com Partial Indexes Example - Partial Global Index CREATE INDEX pcar2 ON pcar (team_key) GLOBAL INDEXING PARTIAL; dbms_stats.gather_index_stats ( ownname => 'GP', indname => 'PCAR2', estimate_percent => NULL ); SELECT num_rows FROM dba_indexes WHERE index_name = 'PCAR2'; NUM_ROWS 936 16 © 2013 - Julian Dyke juliandyke.com Partial Indexes Execution Plans - Partial Local Index CREATE INDEX pcar3 ON pcar (season_key,race_key,position) LOCAL INDEXING PARTIAL; SELECT COUNT(*) FROM pcar WHERE season_key = '2010'; 0 1 2 3 -- Unindexed SELECT STATEMENT SORT AGGREGATE PARTITION RANGE SINGLE TABLE ACCESS FULL (PCAR) Predicate Information (identified by operation id): 3 - filter("SEASON_KEY"=2010) 17 Cost = 14 © 2013 - Julian Dyke juliandyke.com Partial Indexes Execution Plans - Partial Local Index CREATE INDEX pcar3 ON pcar (season_key,race_key,position) LOCAL INDEXING PARTIAL; SELECT COUNT(*) FROM pcar WHERE season_key = '2011'; 0 1 2 3 -- Indexed SELECT STATEMENT SORT AGGREGATE PARTITION RANGE SINGLE INDEX FAST FULL SCAN (PCAR3) Predicate Information (identified by operation id): 3 - filter("SEASON_KEY"=2011) 18 Cost = 2 © 2013 - Julian Dyke juliandyke.com Partial Indexes Execution Plans - Partial Local Index CREATE INDEX pcar3 ON pcar (season_key,race_key,position) LOCAL INDEXING PARTIAL; SELECT COUNT(*) FROM pcar WHERE season_key IN ('2010','2011'); 0 1 2 3 -- Combined SELECT STATEMENT SORT AGGREGATE PARTITION RANGE INLIST TABLE ACCESS FULL (PCAR) Predicate Information (identified by operation id): 3 - filter("SEASON_KEY"=2010 OR "SEASON_KEY"=2011) 19 Cost = 27 © 2013 - Julian Dyke juliandyke.com Online Data File Move This is a great new feature which I have already been using to resolve space issues in my own virtual machines I have successfully used this to move the data file containing the SYSAUX tablespace – not sure I would want to risk it with the SYS tablespace 20 © 2013 - Julian Dyke juliandyke.com Online Data File Move In Oracle 12.1 and above any data file can be moved online For example: ALTER DATABASE MOVE DATAFILE '/u01/app/oradata/PROD/users01.dbf‘ TO '/u02/app/oradata/PROD/users01.dbf'; 21 The database can be open and accessing the data file while the move is in progress Data files can be moved online: from file system to file system from file system to ASM from ASM to file system from ASM to ASM © 2013 - Julian Dyke juliandyke.com Online Partition Move This feature could be very useful for sites with partitioned tables on tiered storage. Most likely usage is migrating partitions from fast expensive storage (SSD) to slower cheaper storage (SAS or SATA) The Oracle documentation hints that there are a lot of places where this partition move can fail, and the DBMS_PART package contains some subroutines that allow recovery from failures. 22 © 2013 - Julian Dyke juliandyke.com Online Partition Move 23 In Oracle 12c partitions can be moved online Useful for tiered storage Move from SSD to SAS to SATA May be useful with OLTP compression Also works for sub-partitions Not supported in the following cases: For tables owned by SYS For IOTs For heap tables containing object types For heap tables containing bitmap join indexes or domain indexes If database-supplemental logging is enabled When parallel DML or direct path INSERTs are executing on the table © 2013 - Julian Dyke juliandyke.com Online Partition Move Consider the following example CREATE TABLE pcar ( season_key NUMBER, race_key NUMBER, driver_key VARCHAR2(4), team_key VARCHAR2(3), position NUMBER, laps_completed NUMBER, race_points NUMBER ) PARTITION BY RANGE (season_key) ( PARTITION p2010 VALUES LESS THAN (2011) TABLESPACE sas, PARTITION p2011 VALUES LESS THAN (2012) TABLESPACE sas, PARTITION p2012 VALUES LESS THAN (2013) TABLESPACE ssd, PARTITION p2013 VALUES LESS THAN (2014) TABLESPACE ssd ); ALTER TABLE pcar MOVE PARTITION P2012 TABLESPACE sas; 24 © 2013 - Julian Dyke juliandyke.com Online Partition Move If online partition move operation fails, it can be cleaned up manually using: DBMS_PART.CLEANUP_ONLINE_OP Clean up failed operations on <partition> DBMS_PART.CLEANUP_ONLINE_OP (<schema>,<table>,<partition>); Clean up failed operations on <table> DBMS_PART.CLEANUP_ONLINE_OP (<schema>,<table>); Clean up failed operations on <schema> DBMS_PART.CLEANUP_ONLINE_OP (<schema>); Clean up all failed operations in database DBMS_PART.CLEANUP_ONLINE_OP; 25 © 2013 - Julian Dyke juliandyke.com Index Columns This is a useful new feature that allows multiple indexes to be created with the same column list For any given column list, only one index can be visible at a time. However, this enhancement will allow new indexes to be created invisibly and then made visible at an appropriate time. 26 © 2013 - Julian Dyke juliandyke.com Index Columns Multiple indexes can be created on the same set of columns The following conditions must be met: The indexes must have different properties e.g. type, partitioning, uniqueness Only one of the indexes can be VISIBLE at any given time Recommendation: Check existing databases for indexes that have been made invisible and then forgotten. 27 © 2013 - Julian Dyke juliandyke.com Index Columns Consider the following table and global index CREATE TABLE pcar ( season_key NUMBER, race_key NUMBER, driver_key VARCHAR2(4), team_key VARCHAR2(3), position NUMBER, laps_completed NUMBER, race_points NUMBER ) PARTITION BY RANGE (season_key) ( PARTITION p2010 VALUES LESS THAN (2011), PARTITION p2011 VALUES LESS THAN (2012), PARTITION p2012 VALUES LESS THAN (2013), PARTITION p2013 VALUES LESS THAN (2014) ); CREATE INDEX pcar_global ON pcar (season_key,race_key,position); 28 © 2013 - Julian Dyke juliandyke.com Index Columns We realise the index should be local so we can drop partitions efficiently The following statement fails with ORA-01408 CREATE INDEX pcar_local ON pcar (season_key,race_key,position) LOCAL; * ERROR at line 1: ORA-01408: such column list already indexed Create the new index INVISIBLE CREATE INDEX pcar_local ON pcar (season_key,race_key,position) LOCAL INVISIBLE; Index created * Switch the indexes ALTER INDEX pcar_global INVISIBLE; ALTER INDEX pcar_local VISIBLE; 29 The new index (PCAR_LOCAL) is now visible The old index (PCAR_GLOBAL) can be dropped © 2013 - Julian Dyke juliandyke.com Invisible Columns I strongly believe this is a very dangerous feature. Whilst it achieves its objectives, it is open to both accidental and malicious misuse as shown in the example. Misuse of this feature could introduce data corruptions that may go unnoticed for months or years and prove to be extremely difficult to resolve 30 © 2013 - Julian Dyke juliandyke.com Invisible Columns Consider the following table: CREATE TABLE icar ( season_key race_key driver_key team_key position laps_completed race_points ); NUMBER, NUMBER, VARCHAR2(4), VARCHAR2(3), NUMBER, NUMBER, NUMBER DESCRIBE icar Name SEASON_KEY RACE_KEY DRIVER_KEY TEAM_KEY POSITION LAPS_COMPLETED RACE_POINTS 31 Null? Type NUMBER NUMBER VARCHAR2(4) VARCHAR2(3) NUMBER NUMBER NUMBER © 2013 - Julian Dyke juliandyke.com Invisible Columns In the data dictionary COL$ contains the following rows for the ICAR table SELECT c.name,c.type#,c.col#,c.intcol#,c.segcol#, TO_CHAR (c.property,'XXXXXXXXXXXX') AS property FROM sys.col$ c, sys.obj$ o, sys.user$ u WHERE c.obj# = o.obj# AND o.owner# = u.user# AND u.name = 'GP‘ AND o.name = 'ICAR'; NAME SEASON_KEY RACE_KEY DRIVER_KEY TEAM_KEY POSITION LAPS_COMPLETED RACE_POINTS 32 TYPE# 2 2 1 1 2 2 2 COL# INTCOL# SEGCOL# 1 2 3 4 5 6 7 © 2013 - Julian Dyke 1 2 3 4 5 6 7 1 2 3 4 5 6 7 PROPERTY 0 0 0 0 0 0 0 juliandyke.com Invisible Columns Make the LAPS_COMPLETED column invisible: ALTER TABLE icar MODIFY laps_completed INVISIBLE; Describe the table again DESCRIBE icar Name SEASON_KEY RACE_KEY DRIVER_KEY TEAM_KEY POSITION RACE_POINTS 33 Null? Type NUMBER NUMBER VARCHAR2(4) VARCHAR2(3) NUMBER NUMBER The LAPS_COMPLETED column is now invisible © 2013 - Julian Dyke juliandyke.com Invisible Columns In the data dictionary COL$ now contains the following rows for ICAR: SELECT c.name,c.type#,c.col#,c.intcol#,c.segcol#, TO_CHAR (c.property,'XXXXXXXXXXXX') AS property FROM sys.col$ c, sys.obj$ o, sys.user$ u WHERE c.obj# = o.obj# AND o.owner# = u.user# AND u.name = 'GP‘ AND o.name = 'ICAR'; NAME SEASON_KEY RACE_KEY DRIVER_KEY TEAM_KEY POSITION LAPS_COMPLETED RACE_POINTS 34 TYPE# 2 2 1 1 2 2 2 COL# INTCOL# SEGCOL# 1 2 3 4 5 0 6 1 2 3 4 5 6 7 1 2 3 4 5 6 7 PROPERTY 0 0 0 0 0 400000020 0 0x400000000 = Invisible Column? 0x20 = Hidden Column © 2013 - Julian Dyke juliandyke.com Invisible Columns Make the LAPS_COMPLETED column visible again: ALTER TABLE icar MODIFY laps_completed VISIBLE; Describe the table again: DESCRIBE icar Name SEASON_KEY RACE_KEY DRIVER_KEY TEAM_KEY POSITION RACE_POINTS LAPS_COMPLETED 35 Null? Type NUMBER NUMBER VARCHAR2(4) VARCHAR2(3) NUMBER NUMBER NUMBER The LAPS_COMPLETED column now appears at end of table © 2013 - Julian Dyke juliandyke.com Invisible Columns In the data dictionary COL$ now contains the following rows for ICAR: SELECT c.name,c.type#,c.col#,c.intcol#,c.segcol#, TO_CHAR (c.property,'XXXXXXXXXXXX') AS property FROM sys.col$ c, sys.obj$ o, sys.user$ u WHERE c.obj# = o.obj# AND o.owner# = u.user# AND u.name = 'GP‘ AND o.name = 'ICAR'; NAME SEASON_KEY RACE_KEY DRIVER_KEY TEAM_KEY POSITION LAPS_COMPLETED RACE_POINTS 36 TYPE# COL# INTCOL# SEGCOL# 2 2 1 1 2 2 2 1 2 3 4 5 7 6 1 2 3 4 5 6 7 1 2 3 4 5 6 7 PROPERTY 0 0 0 0 0 0 0 LAPS_COMPLETED is now COL# 7 © 2013 - Julian Dyke juliandyke.com Invisible Columns Why is this dangerous? Consider the following: INSERT INTO icar VALUES (2013,1,'KRAI','LOT',1,58,25); SELECT * FROM icar; SEASON_KEY 2013 RACE_KEY DRIVER_KEY TEAM_KEY 1 KRAI POSITION LAPS_COMPLETED RACE_POINTS 1 58 25 LOT ALTER TABLE icar MODIFY laps_completed INVISIBLE; ALTER TABLE icar MODIFY laps_completed VISIBLE; INSERT INTO icar VALUES (2013,1,'FALO','FER',2,58,18); SELECT * FROM icar; SEASON_KEY 2013 2013 37 RACE_KEY DRIVER_KEY TEAM_KEY 1 1 KRAI FALO POSITION RACE_POINTS LAPS_COMPLETED 1 2 25 58 58 18 LOT FER © 2013 - Julian Dyke juliandyke.com Invisible Columns Continued... ALTER TABLE icar MODIFY race_points INVISIBLE; ALTER TABLE icar MODIFY race_points VISIBLE; INSERT INTO icar VALUES (2013,1,‘SVET',‘RBR',3,58,15); SELECT * FROM icar; SEASON_KEY 2013 2013 2013 38 RACE_KEY DRIVER_KEY TEAM_KEY 1 1 1 KRAI FALO SVET POSITION LAPS_COMPLETED RACE_POINTS 1 2 3 58 18 58 25 58 15 LOT FER RBR Column order is restored, but Fernando Alonso now has 58 points © 2013 - Julian Dyke juliandyke.com Identity Clause This new feature simplifies management of sequences used as primary keys for tables. The identity clause allows an implicit index to be created for the specified column. If the table is truncated, the sequence is unaffected If the table is dropped and recreated the sequence will dropped and recreated and will restart at the minimum value for the next insertion 39 © 2013 - Julian Dyke juliandyke.com Identity Clause In Oracle 12.1 and above an identity clause can be used to specify a sequence column in CREATE TABLE and ALTER TABLE statements Syntax is: GENERATED [ ALWAYS | BY DEFAULT [ ON NULL ] ] AS IDENTITY [ ( identity_options ) ] where <identity_options> are: { START WITH ( integer | LIMIT VALUE ) | INCREMENT BY integer | ( MAXVALUE integer | NOMAXVALUE ) | ( MINVALUE integer | NOMINVALUE ) | ( CYCLE | NOCYCLE ) | ( CACHE integer | NOCACHE ) | ( ORDER | NOORDER ) } . . . 40 © 2013 - Julian Dyke juliandyke.com Identity Clause Example: CREATE TABLE driver2 ( driver_key NUMBER GENERATED AS IDENTITY, driver_name VARCHAR2(30), driver_dob DATE, country_key VARCHAR2(3) ); INSERT INTO driver2 (driver_name,driver_dob,country_key) VALUES ('Sebastian Vettel','03-JUL-1987','GER'); INSERT INTO driver2 (driver_name,driver_dob,country_key) VALUES ('Fernando Alonso',‘29-JUL-1981','SPA'); INSERT INTO driver2 (driver_name,driver_dob,country_key) VALUES ('Kimi Raikkonen','17-OCT-1979','FIN'); SELECT * FROM driver2; DRIVER_KEY 1 2 3 41 DRIVER_NAME DRIVER_DOB Sebastian Vettel 03-JUL-1987 Fernando Alonso 29-JUL-1981 Kimi Raikkonen 17-OCT-1979 © 2013 - Julian Dyke COUNTRY_KEY GER SPA FIN juliandyke.com Identity Clause DESCRIBE includes identity column DESCRIBE driver2 42 Name Null? Type DRIVER_KEY DRIVER_NAME DRIVER_DOB COUNTRY_KEY NOT NULL NUMBER VARCHAR2(30) DATE VARCHAR2(3) No additional indexes are created © 2013 - Julian Dyke juliandyke.com Identity Clause Columns are stored in the data dictionary as follows: SELECT c.name,c.type#,c.col#,c.intcol#,c.segcol#, TO_CHAR (c.property,'XXXXXXXXXX') AS property FROM sys.col$ c, sys.obj$ o, sys.user$ u WHERE c.obj# = o.obj# AND o.owner# = u.user# AND u.name = 'GP‘ AND o.name = 'DRIVER2‘ ORDER BY intcol#; NAME TYPE# DRIVER_KEY DRIVER_NAME DRIVER_DOB COUNTRY_KEY 43 2 1 12 1 COL# INTCOL# SEGCOL# PROPERTY 1 2 3 4 1 2 3 4 1 2 3 4 2800000000 0 0 0 0x800000000 = Default as Sequence 0x2000000000 = Generated ALWAYS identity column © 2013 - Julian Dyke juliandyke.com Identity Clause Default value for DRIVER_KEY column can be found in DBA_TAB_COLUMNS: SELECT data_default FROM dba_tab_columns WHERE owner = 'GP‘ AND table_name = 'DRIVER2‘ AND column_name = 'DRIVER_KEY'; "GP"."ISEQ$$_92584".nextval In this example 92584 is the object ID of the GP.DRIVER2 table SELECT sequence_owner AS owner,min_value,max_value,increment_by, cycle_flag,order_flag,cache_size FROM dba_sequences WHERE sequence_name = 'ISEQ$$_92584'; OWNER GP 44 MIN_VALUE 1 MAX_VALUE INCREMENT_BY 1.0000E+28 1 © 2013 - Julian Dyke C O CACHE_SIZE N N 20 juliandyke.com Session Sequences This new feature allows sequences to be created that exist for the lifetime of the current session only. Intended for use with global temporary tables, but possibly useful in other places and more flexible than the ROWNUM pseudo column 45 © 2013 - Julian Dyke juliandyke.com Session Sequences In Oracle 12.1 and above sequences can have session visibility Current value only visible to session For example: CREATE SEQUENCE seq1 SESSION; SQL> CONNECT gp/gp SQL> SELECT seq1.NEXTVAL FROM dual; NEXTVAL 1 SQL> SELECT seq1.NEXTVAL FROM dual; NEXTVAL 2 SQL> CONNECT gp/gp SQL> SELECT seq1.NEXTVAL FROM dual; NEXTVAL 1 46 © 2013 - Julian Dyke juliandyke.com Global Temporary Table Undo This new feature allows undo for global temporary tables to be written to the temporary table space It will not have much impact for insertions, but could have a significant impact on redo generation caused by GTT undo during updates I envisage this becoming the default in future versions 47 © 2013 - Julian Dyke juliandyke.com Global Temporary Table Undo 48 By default DML on Global Temporary Tables Does not generate redo directly Does generate undo and indirect redo Undo is required to rollback transactions Redo will be archived, backed up , propagated to standby etc In Oracle 12c Global Temporary Table undo can be stored in a temporary tablespace Set TEMP_UNDO_ENABLED = TRUE Will not have much impact for INSERT statements May have significant impact for UPDATE and DELETE statements Review whether DELETE statements are necessary © 2013 - Julian Dyke juliandyke.com Temporal Validity Temporal validity allows tables to be created where rows are valid for a specific period of time A major defect is that is not possible to create primary keys with temporal validity. This functionality may be added in a future release, until which time this feature may be of limited use. 49 © 2013 - Julian Dyke juliandyke.com Temporal Validity 50 Flashback Data Archive was introduced in Oracle 11.1 Originally known as Total Recall Allows historic data to be inspected at any point in time Was a separately licensed option Consequently not very popular Now available free in Enterprise Edition (at least) Including Oracle 11.2 In Oracle 12.1 and above Temporal Validity builds on these concepts © 2013 - Julian Dyke juliandyke.com Temporal Validity For example: CREATE TABLE driver ( driver_key VARCHAR2(4), team_key VARCHAR2(3), joining_date DATE, leaving_date DATE, PERIOD FOR team_member_valid_time (joining_date,leaving_date) ); Insert some data INSERT INTO driver VALUES ('FALO','FER','01-JAN-2010',NULL); INSERT INTO driver VALUES ('FMAS','FER','01-JAN-2006','31-DEC-2013'); INSERT INTO driver VALUES ('KRAI','FER','01-JAN-2007','31-DEC-2009'); INSERT INTO driver VALUES ('RBAR','FER','01-JAN-2000','31-DEC-2007'); INSERT INTO driver VALUES ('MSCH','FER','01-JAN-1996','31-DEC-2006'); 51 Note: the above data is inaccurate © 2013 - Julian Dyke juliandyke.com Temporal Validity For example: SELECT * FROM driver; DRIVER_KEY FALO FMAS KRAI RBAR MSCH TEAM_KEY FER FER FER FER FER JOINING_DATE 01-JAN-2010 01-JAN-2006 01-JAN-2007 01-JAN-2000 01-JAN-1996 LEAVING_DATE 31-DEC-2013 31-DEC-2009 31-DEC-2007 31-DEC-2006 Who was in the team for the 2009 British Grand Prix qualifying? SELECT * FROM driver AS OF PERIOD FOR team_member_valid_time TO_DATE (‘20-JUN-2009’); DRIVER_KEY FMAS KRAI 52 TEAM_KEY FER FER JOINING_DATE 01-JAN-2006 01-JAN-2007 © 2013 - Julian Dyke LEAVING_DATE 31-DEC-2013 31-DEC-2009 juliandyke.com Temporal Validity Describe the driver table DESCRIBE driver Name DRIVER_KEY TEAM_KEY JOINING_DATE LEAVING_DATE 53 Null? Type VARCHAR2(4) VARCHAR2(3) DATE DATE © 2013 - Julian Dyke juliandyke.com Temporal Validity List the columns in COL$ SELECT c.name,c.col#,c.intcol#,c.segcol#,c.type#,TO_CHAR (c.property,'XXXXX') FROM sys.col$ c, sys.obj$ o WHERE c.obj# = o.obj# AND o.name = 'DRIVER3‘ ORDER BY c.intcol# NAME COL# INTCOL# SEGCOL# TEAM_MEMBER_VALID_TIME 0 1 0 DRIVER_KEY 1 2 1 TEAM_KEY 2 3 2 JOINING_DATE 3 4 3 LEAVING_DATE 4 5 4 54 TYPE# 2 1 1 12 12 PROPERTY 10028 0 0 0 0 0x10000 = Virtual Column 0x20 = Hidden Column 0x8 = Virtual Column © 2013 - Julian Dyke juliandyke.com Extended Columns This feature allows the size of VARCHAR2, NVARCHAR2 and RAW columns stored in the database to be increased to 32767 bytes. If the value is longer than 4000 bytes it is stored as an out of line LOB Built-in functions appear to work correctly with the longer column sizes This feature needs to be enabled by setting MAX_STRING_SIZE to EXTENDED. This parameter is not set by default. You may want to set this parameter before creating a database, otherwise you will need an outage as the parameter must be set when the database is in UPGRADE mode 55 © 2013 - Julian Dyke juliandyke.com Extended Columns In Oracle 12c and above maximum column length has increased Data Type 56 Oracle 11.2 and below Oracle 12.1 and above VARCHAR2 4000 32767 NVARCHAR2 2000 16383 RAW 2000 32767 Note that NVARCHAR2 limits assume two bytes per character Maximum length of CHAR and NCHAR remains at 2000 and 1000 respectively Extended columns are stored as SECUREFILE LOBs Stored in line if <= 4K Stored out of line if > 4K COMPATIBLE parameter must be 12.0.0.0.0 or above © 2013 - Julian Dyke juliandyke.com Extended Columns By default attempts to create an extended column will fail: ALTER TABLE car MODIFY notes VARCHAR2(32767); * ERROR at line 1: ORA-00910: specified length too long for its datatype MAX_STRING_SIZE parameter must be set to EXTENDED Default is value is STANDARD MAX_STRING_SIZE parameter cannot be updated when database is open: ALTER SYSTEM SET max_string_size = 'EXTENDED' * ERROR at line 1: ORA-02097: parameter cannot be modified because specified value is invalid ORA-14694: database must in UPGRADE mode to begin MAX_STRING_SIZE migration 57 © 2013 - Julian Dyke juliandyke.com Extended Columns To change the MAX_STRING_SIZE parameter, restart the database in UPGRADE mode SQL> SHUTDOWN IMMEDIATE SQL> STARTUP MIGRATE Set the parameter value to EXTENDED: SQL> ALTER SYSTEM SET max_string_size = EXTENDED; Run the utl32k.sql script SQL> @$ORACLE_HOME/rdbms/admin/utl32k.sql; Restart the database SQL> SHUTDOWN IMMEDIATE SQL> STARTUP 58 It is not possible to convert the MAX_STRING_SIZE parameter back from EXTENDED to STANDARD © 2013 - Julian Dyke juliandyke.com Extended Columns When MAX_STRING_SIZE is set to EXTENDED then tables can be created with extended columns: CREATE TABLE ecar ( season_key race_key driver_key team_key position laps_completed notes race_points ); NUMBER, NUMBER, VARCHAR2(4), VARCHAR2(3), NUMBER, NUMBER, VARCHAR2(32767), NUMBER Alternatively maximum size of columns in existing tables can be increased: ALTER TABLE car MODIFY notes VARCHAR2(32767); 59 © 2013 - Julian Dyke juliandyke.com Extended Columns Extended columns are implemented as SECUREFILE LOBs SELECT column_name,segment_name,securefile FROM dba_lobs WHERE owner = 'GP‘ AND table_name = 'ECAR'; COLUMN_NAME SEGMENT_NAME SECUREFILE NOTES SYS_LOB0000092626C00007$$ YES SECUREFILE LOBs have a system-created index SELECT column_name,index_name FROM dba_lobs WHERE owner = 'GP‘ AND table_name = 'ECAR'; 60 COLUMN_NAME INDEX_NAME NOTES SYS_IL0000092626C00007$$ © 2013 - Julian Dyke juliandyke.com Row Limiting Clause This feature provides a more comprehensive syntax for Top-N queries The new syntax uses analytic query operations as opposed to regular sort options It is probably worth doing comparative performance tests before adopting the new syntax Beware with the OFFSET clause – each invocation will require a full sort of the data before returning any rows 61 © 2013 - Julian Dyke juliandyke.com Row Limiting Clause In Oracle 12c and above SELECT statements can include the FETCH FIRST clause Limits rows returned by query Optional replacement syntax for TOP-N queries Syntax is: [ OFFSET offset { ROW | ROWS } ] [ FETCH { FIRST | NEXT } [ { rowcount | percent PERCENT } ] { ROW | ROWS } { ONLY | WITH TIES } ] 62 OFFSET specifies number of rows to skip before row limiting begins FETCH specifies number of rows or percentage of rows to return ONLY return exactly the number of rows specified WITH TIES return additional rows with same sort key as last row fetched © 2013 - Julian Dyke juliandyke.com Row Limiting Clause 63 An ORDER BY clause is normally required to ensure that sort order is deterministic Restrictions Cannot be specified in SELECT FOR UPDATE statements Cannot be used with CURRVAL or NEXTVAL pseudo-columns Cannot be used with materialized view incremental refresh © 2013 - Julian Dyke juliandyke.com Row Limiting Clause Example – top ten drivers in 2012 Driver Name Sebastian Vettel Fernando Alonso Kimi Raikkonen Lewis Hamilton Jenson Button Mark Webber Felipe Massa Romain Grosjean Nico Rosberg Sergio Perez 64 © 2013 - Julian Dyke Points 281 278 207 190 188 179 122 96 93 66 juliandyke.com Row Limiting Clause Example – Top N query SELECT * FROM ( SELECT d.driver_name,t.team_name,SUM(c.driver_points) FROM car c,driver d,team t WHERE c.season_key = 2012 AND c.driver_key = d.driver_key AND c.team_key = t.team_key GROUP BY d.driver_name,t.team_name ORDER BY SUM(c.driver_points) DESC ) WHERE ROWNUM <= 5; Driver Name Sebastian Vettel Fernando Alonso Kimi Raikkonen Lewis Hamilton Jenson Button Points 281 278 207 190 188 ----------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| ----------------------------------------------------------------------| 0 | SELECT STATEMENT | | 5 | 335 | 41 (0)| |* 1 | COUNT STOPKEY | | | | | | 2 | VIEW | | 480 | 32160 | 41 (0)| |* 3 | SORT ORDER BY STOPKEY| | 480 | 23520 | 41 (0)| | 4 | HASH GROUP BY | | 480 | 23520 | 41 (0)| |* 5 | HASH JOIN | | 480 | 23520 | 41 (0)| | 6 | TABLE ACCESS FULL | TEAM | 104 | 1248 | 2 (0)| |* 7 | HASH JOIN | | 480 | 17760 | 39 (0)| |* 8 | TABLE ACCESS FULL| CAR | 480 | 8160 | 36 (0)| | 9 | TABLE ACCESS FULL| DRIVER | 493 | 9860 | 3 (0)| ----------------------------------------------------------------------- 65 © 2013 - Julian Dyke juliandyke.com Row Limiting Clause Example – Fetch Only Clause SELECT d.driver_name,t.team_name,SUM(c.driver_points) FROM car c,driver d,team t WHERE c.season_key = 2012 AND c.driver_key = d.driver_key AND c.team_key = t.team_key GROUP BY d.driver_name,t.team_name ORDER BY SUM(c.driver_points) DESC FETCH FIRST 5 ROWS ONLY; Driver Name Sebastian Vettel Fernando Alonso Kimi Raikkonen Lewis Hamilton Jenson Button Points 281 278 207 190 188 ------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| ------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 480 | 44640 | 41 (0)| | 1 | SORT ORDER BY | | 480 | 44640 | 41 (0)| |* 2 | VIEW | | 480 | 44640 | 41 (0)| |* 3 | WINDOW SORT PUSHED RANK| | 480 | 23520 | 41 (0)| | 4 | HASH GROUP BY | | 480 | 23520 | 41 (0)| |* 5 | HASH JOIN | | 480 | 23520 | 41 (0)| | 6 | TABLE ACCESS FULL | TEAM | 104 | 1248 | 2 (0)| |* 7 | HASH JOIN | | 480 | 17760 | 39 (0)| |* 8 | TABLE ACCESS FULL | CAR | 480 | 8160 | 36 (0)| | 9 | TABLE ACCESS FULL | DRIVER | 493 | 9860 | 3 (0)| ------------------------------------------------------------------------- 66 © 2013 - Julian Dyke juliandyke.com Row Limiting Clause Example – Fetch Percent With Ties Clause SELECT d.driver_name,t.team_name,SUM(c.driver_points) FROM car c,driver d,team t WHERE c.season_key = 2012 AND c.driver_key = d.driver_key AND c.team_key = t.team_key GROUP BY d.driver_name,t.team_name ORDER BY SUM(c.driver_points) DESC FETCH FIRST 20 PERCENT ROWS WITH TIES; Driver Name Points Sebastian Vettel Fernando Alonso Kimi Raikkonen Lewis Hamilton Jenson Button 281 278 207 190 188 ----------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| ----------------------------------------------------------------------| 0 | SELECT STATEMENT | | 480 | 50880 | 41 (0)| | 1 | SORT ORDER BY | | 480 | 50880 | 41 (0)| |* 2 | VIEW | | 480 | 50880 | 41 (0)| | 3 | WINDOW SORT | | 480 | 23520 | 41 (0)| | 4 | HASH GROUP BY | | 480 | 23520 | 41 (0)| |* 5 | HASH JOIN | | 480 | 23520 | 41 (0)| | 6 | TABLE ACCESS FULL | TEAM | 104 | 1248 | 2 (0)| |* 7 | HASH JOIN | | 480 | 17760 | 39 (0)| |* 8 | TABLE ACCESS FULL| CAR | 480 | 8160 | 36 (0)| | 9 | TABLE ACCESS FULL| DRIVER | 493 | 9860 | 3 (0)| ----------------------------------------------------------------------- 67 © 2013 - Julian Dyke juliandyke.com Row Limiting Clause Example – Fetch with Offset Clause SELECT d.driver_name,t.team_name,SUM(c.driver_points) FROM car c,driver d,team t WHERE c.season_key = 2012 AND c.driver_key = d.driver_key AND c.team_key = t.team_key GROUP BY d.driver_name,t.team_name ORDER BY SUM(c.driver_points) DESC OFFSET 5 ROWS FETCH FIRST 5 ROWS ONLY; Driver Name Mark Webber Felipe Massa Romain Grosjean Nico Rosberg Sergio Perez Points 179 122 96 93 66 ------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| ------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 480 | 44640 | 41 (0)| | 1 | SORT ORDER BY | | 480 | 44640 | 41 (0)| |* 2 | VIEW | | 480 | 44640 | 41 (0)| |* 3 | WINDOW SORT PUSHED RANK| | 480 | 23520 | 41 (0)| | 4 | HASH GROUP BY | | 480 | 23520 | 41 (0)| |* 5 | HASH JOIN | | 480 | 23520 | 41 (0)| | 6 | TABLE ACCESS FULL | TEAM | 104 | 1248 | 2 (0)| |* 7 | HASH JOIN | | 480 | 17760 | 39 (0)| |* 8 | TABLE ACCESS FULL | CAR | 480 | 8160 | 36 (0)| | 9 | TABLE ACCESS FULL | DRIVER | 493 | 9860 | 3 (0)| ------------------------------------------------------------------------- 68 © 2013 - Julian Dyke juliandyke.com Histograms There are several enhancements to histograms in Oracle 12c. This section concentrates on the increase in maximum number of buckets from 254 to 2048. Increasing the number of buckets allows better cardinalities to be estimated by the optimization, potentially generating more efficient execution plans The increased bucket sizes work for both single column and multi column statistics This is particular useful with my Formula 1 database which (for the period 1961 to 2012) contains 492 drivers and 1289 driver/team combinations. 69 © 2013 - Julian Dyke juliandyke.com Histograms Maximum bucket size increased to 2048 Default bucket size is still 256 For example, an inefficient execution plan has been generated for a query We determine that the root cause is poor cardinality estimates for the DRIVER_KEY column in the CAR table The DRIVER_KEY column has 492 distinct values SELECT COUNT (DISTINCT (driver_key)) AS driver_key FROM car; DRIVER_KEY 492 70 © 2013 - Julian Dyke juliandyke.com Histograms Default statistics collection only gathers minimum and maximum values: dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR', estimate_percent => NULL ); SELECT COUNT (*) FROM dba_histograms WHERE owner = ‘GP’ AND table_name = ‘CAR’ AND column_name = ‘DRIVER_KEY’; COUNT (*) 2 71 © 2013 - Julian Dyke juliandyke.com Histograms Collect histograms on the DRIVER_KEY column dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR', estimate_percent => NULL method_opt => 'FOR COLUMNS driver_key' ); SELECT COUNT (*) FROM dba_histograms WHERE owner = ‘GP’ AND table_name = ‘CAR’ AND column_name = ‘DRIVER_KEY’; COUNT (*) 75 72 Default behaviour is to create a maximum of 256 buckets © 2013 - Julian Dyke juliandyke.com Histograms If more than 256 buckets are required, this must be specified explicitly: dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR', estimate_percent => NULL method_opt => 'FOR COLUMNS driver_key SIZE 2048' ); SELECT COUNT (*) FROM dba_histograms WHERE owner = ‘GP’ AND table_name = ‘CAR’ AND column_name = ‘DRIVER_KEY’; COUNT (*) 492 73 © 2013 - Julian Dyke juliandyke.com Histograms Multi-Column Statistics DECLARE l_extension_name VARCHAR2(30); BEGIN l_extension_name := dbms_stats.create_extended_stats ( ownname => 'GP', tabname => 'CAR6', extension => '(driver_key,team_key)‘ ); END; BEGIN dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR6', estimate_percent => NULL, method_opt => 'FOR COLUMNS (DRIVER_KEY,TEAM_KEY) SIZE 2048‘ ); END; 74 © 2013 - Julian Dyke juliandyke.com Histograms Multi-Column Statistics DECLARE l_extension_name VARCHAR2(30); BEGIN l_extension_name := dbms_stats.create_extended_stats ( ownname => 'GP', tabname => 'CAR6', extension => '(driver_key,team_key)‘ ); END; BEGIN dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR6', estimate_percent => NULL, method_opt => 'FOR COLUMNS (DRIVER_KEY,TEAM_KEY) SIZE 2048‘ ); END; 75 © 2013 - Julian Dyke juliandyke.com Histograms Multi-Column Statistics SELECT COUNT(*) FROM gp.car WHERE driver_key = ‘MSCH' AND team_key = 'FER'; COUNT(*) 181 Id Operation 0 1 2 Rows Bytes SELECT STATEMENT 1 9 SORT AGGREGATE 1 9 181 1629 TABLE ACCESS FULL Name CAR Cost (%CPU) Time 39 (0) 00:00:01 39 (0) 00:00:01 Correct Cardinality 776 © 2013 - Julian Dyke juliandyke.com Histograms Multi-Column Statistics SELECT COUNT(*) FROM gp.car WHERE driver_key = ‘MSCH' AND team_key = ‘JOR'; COUNT(*) 1 Id Operation 0 1 2 Rows Bytes SELECT STATEMENT 1 9 SORT AGGREGATE 1 9 1 1629 TABLE ACCESS FULL Name CAR Cost (%CPU) Time 39 (0) 00:00:01 39 (0) 00:00:01 Correct Cardinality 777 © 2013 - Julian Dyke juliandyke.com Application Continuity This is potentially a very important new feature which allows uncommitted transactions to be replayed in another instance following a RAC or Data Guard failover or session relocation I anticipate many sites will wish to take advantage of this new functionality. Initially I have attempted to create a simple test example of this functionality using a JDBC thin client application, but have so far been unsuccessful. I know that Trivadis have successful created a demonstration of Application Continuity using the Universal Connection Pool (UCP) so it does work. Further investigation is required for the JDBC Thin example. In the meantime this session contains the configuration that I have completed so far. 78 © 2013 - Julian Dyke juliandyke.com Application Continuity 79 Failed transactions are replayed on another instance / database Similar goals to TAF and FCF Better implementation Can be configured for: RAC Data Guard Single Instance Must use one of: Weblogic pool Universal Connection Pool (UCP) JDBC Thin OCI not currently supported Limitations may drive future development decisions e.g. connections pools © 2013 - Julian Dyke juliandyke.com Application Continuity 80 JDBC calls should handle events for the current session such as: Service shutdown Instance failure Network failure Node failure Session will attempt to reconnect again (same or different instance) Failed transactions will be rolled back and re-executed Similar (but not the same) as Database Replay Calls replayed with bind variables etc. Fewer synchronization issues – replay only includes last uncommitted transaction © 2013 - Julian Dyke juliandyke.com Application Continuity Must connect to a user-defined service Not the database service E.g. for single instance database DECLARE l_arr DBMS_SERVICE.SVC_PARAMETER_ARRAY; BEGIN l_arr ('FAILOVER_TYPE') := 'TRANSACTION'; l_arr ('REPLAY_INITIATION_TIMEOUT') := 600; l_arr ('FAILOVER_DELAY') := 3; l_arr ('FAILOVER_RETRIES') := 20; l_arr ('SESSION_STATE_CONSISTENCY') := 'DYNAMIC'; l_arr ('COMMIT_OUTCOME') := 'TRUE'; l_arr ('AQ_HA_NOTIFICATIONS') := 'TRUE'; DBMS_SERVICE.CREATE_SERVICE ( service_name => 'SERVICE1', network_name => 'SERVICE1‘, parameter_array => l_arr ); END; 81 © 2013 - Julian Dyke juliandyke.com Application Continuity E.g. for a RAC database srvctl add service -db TEST \ -service SERVICE1 \ -preferred TEST1 \ -available TEST2 \ -failovertype TRANSACTION \ -notification TRUE \ -commit_outcome TRUE \ -replay_init_time 600 \ -failoverretry 30 \ -failoverdelay 10 Remember to start the service... srvctl start service –d TEST –s SERVICE1 –i TEST1 82 © 2013 - Julian Dyke juliandyke.com Application Continuity Client connection string should include values for: TRANSPORT_CONNECT_TIMEOUT CONNECT_TIMEOUT RETRY_COUNT For example: jdbc:oracle:thin:gp/gp@(DESCRIPTION=(TRANSPORT_CONNECT_TIMEOUT=3) (CONNECT_TIMEOUT=60)(RETRY_COUNT=10)(FAILOVER=ON) (ADDRESS=(PROTOCOL=tcp)(PORT=1521)(HOST=vmcluster1-scan.juliandyke.com)) (CONNECT_DATA=(SERVICE_NAME=SERVICE1))) 83 REMOTE_LISTENER database parameter must include SCAN name if clients specify SCAN names Node names if clients specify address list © 2013 - Julian Dyke juliandyke.com Application Continuity Configure the Oracle JDBC 12c Replay Data Source in the property file or in the thin JDBC application e.g. import oracle.jdbc.replay.OracleDataSourceImpl; import oracle.jdbc.replay.ReplayableConnection; OracleDataSourceImpl ods = new OracleDataSourceImpl(); ods.setURL(url); connection = ods.getConnection(); connection.setAutoCommit (false); ... ((ReplayableConnection)connection).beginRequest(); # Application processing ((ReplayableConnection)connection).endRequest(); 84 Requires $ORACLE_HOME/jdbc/lib/ojdbc6.jar on CLASSPATH © 2013 - Julian Dyke juliandyke.com Application Continuity Debugging replayable connections Add $ORACLE_HOME/jdbc/lib/ojdbc6_g.jar to the CLASSPATH Add the following to the properties file oracle.jdbc.internal.replay.level = FINEST handlers = java.util.logging.FileHandler java.util.logging.FileHandler.pattern = /home/oracle/12c/appcon2/replay_%U.trc java.util.logging.FileHandler.limit = 500000000 java.util.logging.FileHandler.count = 1000 java.util.logging.FileHandler.formatter = java.util.logging.XMLFormatter Execute using: java -Djava.util.logging.config.file=/home/oracle/appcon/properties J7 85 Writes trace to replay_0.trc.0 © 2013 - Julian Dyke juliandyke.com Application Continuity 86 Potentially a very powerful feature Easier to implement, test and support than TAF Builds on FCF Applications need to be designed specifically for application continuity Very difficult to retrofit existing applications Special attention required for pseudo columns such as SYSDATE Sequences should use the new KEEP clause If you plan to use this feature in the future, I recommend DBAs become familiar with it in Oracle 12.1 so they can support developments New applications follow the development guidelines for this feature Expect to deploy the new applications in Oracle 12.2 © 2013 - Julian Dyke juliandyke.com Thank You For Your Interest info@juliandyke.com 87 © 2013 - Julian Dyke juliandyke.com