<Insert Picture Here> Introduction to Oracle Database 11g – The Innovation Continues Pat Shuff Solutions Architect Oracle Corporation BROUG May 17, 2007 Sample Oracle Innovations • Automatic Storage Management • Flashback technologies • XMLDB • Application Express • Real Application Clusters Oracle Database Innovation 30 years of sustained innovation … Audit Vault Database Vault Grid Computing Self Managing Database XML Database Oracle Data Guard Real Application Clusters Flashback Query Virtual Private Database Built in Java VM Partitioning Support Built in Messaging Object Relational Support Multimedia Support Data Warehousing Optimizations Parallel Operations Distributed SQL & Transaction Support Cluster and MPP Support Multi-version Read Consistency Client/Server Support Platform Portability Commercial SQL Implementation 1977 … continuing with Oracle Database 11g 2007 Oracle’s Mission Deliver the best information with the highest Quality of Service at the lowest cost Oracle’s Mission Deliver the best information with the highest Quality of Service at the lowest cost Oracle Database must be the fastest, most scaleable, most reliable, most secure, easiest to use, for all types of information… The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Managing Storage and Data Managing & Using Information Maintaining Availability, Security and Performance Managing Systems And Managing Change Managing Storage and Data Automatic Storage Management ASM Disk ASM Disk ASM Disk ASM Disk ASM Disk • The preferred and best storage manager for Oracle Databases • • • • Easier to manage than file systems Performance of raw volumes Built-in to Oracle database Shared storage pool for all databases • Free, and widely adopted • >65% of 10g RAC deployments on ASM • >25% of 10g customers already using ASM • Many VLDB over 10TB Automatic Storage Management ASM Disk ASM Disk ASM Disk ASM Disk ASM Disk • Spreads database files evenly across storage arrays • Storage arrays can be easily added or remove • transparent data redistribution • Data mirrored across arrays • Tolerates failure of disks or arrays New ASM features in Oracle 11g: • • • • ASM Fast Disk Resync ASM Preferred Mirror Read ASM Rolling Upgrade Larger extent, allocation unit sizes Data Compression for All Applications • Oracle 9i compresses data only during bulk load; useful for DW and ILM • Oracle 11g compresses w/ inserts, updates • Typical compression ratio of 2x to 3x • Database directly reads compressed data eliminating decompression overhead • Strategy: compress db’s 10 largest tables • Shrink table data by 50%, increase CPU by 5% • Savings cascade to all db copies: test, dev, standby, mirrors, archiving, backup, etc. Oracle SecureFiles High-Performance Large Objects • High-performance transactional access to large object data • documents, medical, CAD, imaging … • low-latency, high throughput, concurrent access • space-optimized storage • Protect your valuable data .. in the db! • • • • transactions transparent encryption compression and de-duplication database-quality security, reliability, and scalability • Better security, single view and management of data • Superset of LOB interfaces – easy migration SecureFiles Breaks the Performance Barrier! File Read Performance (MB/second) 120 100 80 60 40 20 0 SecureFiles Linux Files LOBs 0.1 1 10 File Size (MB) 100 • Innovative technology for high performance large object data •Smart buffering, write gathering, intelligent locking •Fast bulk data transfers, LOB prefetch • Much faster than LOBs with more capabilities • File system-like performance with database functionality! Database Encryption • Encrypt at a column level (10g) • New: encrypt entire tablespaces • Redo, undo, backups also encrypted • Key management done by database • Transparent to applications Growing Data Volumes 100 80 Database Size 60 (TB) 40 Size of the largest data warehouse in Winter Corp Survey 245% increase from 2003 to 2005! 20 0 1998 1999 2000 2001 2002 2003 2004 2005 Source: 2005 TopTen Program, November 2005 © Winter Corporation, Waltham, MA, USA Information Lifecycle Management Optimize storage cost and performance Active Less Active Historical Archive DIGITAL DATA STORAGE High Performance Storage Tier Low Cost Storage Tier Online Archive Storage Tier Offline Archive Storage Tier • Use Flashback Data Archive for long-term storage of “old” data • Use table, index partitioning to separate data into different tiers • Use new ILM assistant to establish policies, create scripts Flashback Data Archive Total Data Recall Select * from orders AS OF ‘Midnight 31-Dec-2004’ Archive Tables User Tablespaces Flashback Data Archive Oracle 11g Database • Tamper-proof data archive • Efficient storage and retrieval of undo • Keep data for months, years, decades! • Fast access to even very old data • View data, versions of rows as of any time • Control data retention time, purging of data Flashback Data Archive Total Data Recall Select * from orders AS OF ‘Midnight 31-Dec-2004’ Archive Tables User Tablespaces Flashback Data Archive Oracle 11g Database • Access Historical Data – ”AS OF” • Generate Reports – “ROW VERSIONS” • Information Lifecycle Management (ILM) • Auditing • Data Recovery • Enforce Data Retention Policies Oracle Partitioning 10 years of innovation Core functionality Oracle8 Range partitions, global range index Oracle8i Hash and composite range-hash partitioning Oracle9i List partitioning Oracle9i R2 Composite range-list partitioning Oracle 10g Global hash indexes Oracle 10g R2 1M partitions per table Oracle Partitioning 10 years of innovation Core functionality Oracle8 Range partitions, global range index Oracle8i Hash and composite range-hash partitioning Oracle9i List partitioning Oracle9i R2 Composite range-list partitioning Oracle 10g Global hash indexes Oracle 10g R2 1M partitions per table Partitioning by reference Virtual column partitioning New composite partitioning: range-range, list-range, list-list, list-hash New Partitioning Features • New composite partitioning schemes Range List • • • • Range List Hash 11g 9i 8i 11g 11g 11g Partition (or index) on virtual (computed) columns Partition advisor Automatic range partition creation Partition by REFERENCE (primary key of parent) Partitioning by REFERENCE Table ORDERS ... Jan 2006 Feb 2006 ... • RANGE(order_date) • Primary key order_id • Partitioning key inherited through PK-FK relationship • Avoids redundant storage, maintenance of order_date Table LINEITEMS ... Jan 2006 Feb 2006 • RANGE(order_date) ... • Foreign key order_id Partitioning Advisor Packaged Apps Custom Apps SQL Workload SQL Advisor SQL SQL Plan Access Structure Tuning Analysis Analysis SQL Profile SQL Advice Indexes & MVs Well-tuned SQL & Schema New! Partition Analysis Partition Advice • Considers entire query workload to improve query performance • Advises on partitioning methods • Range (equal-interval), range key and interval • Hash, hash key • Integrated, non-conflicting advice with Indexes, MVs Oracle ILM Assistant • Manages your ILM environment via a GUI interface • Define lifecycle definitions • Manage security & compliance • • • • Advises when data needs to be moved, generates scripts Requires Oracle Application Express Supports Oracle Database 9i and up Tool downloaded from OTN (available now!) Oracle’s Storage Strategy – Sustained Innovation • Oracle storage suite builtout over last decade • Each component continuously enhanced to add more value • Secure Files • Oracle Secure Backup, Encryption • ASM, RMAN Disk Backup • XML DB, ILM, Compression • Flashback • Data Guard • RMAN • Best of breed in each area 8.0 8i 9i 9.2 10g 10.2 11g Managing and Using Information Managing All Your Information • Oracle Strategy: evolve the Oracle database to manage all enterprise Information • Meld db and file metaphors • Enable integration of all enterprise information sources • Enable rich information retrieval capabilities • Provide solutions built on top of the database • Uniform management of content and metadata • Scalable, secure, highly available, integrated, robust, available on all platforms Secure Enterprise Search Enterprise Search Solution Content DB Enterprise Content Management XML DB Integrated Native XML Database Text & Ontology Text and Semantic Processing Location & Spatial Location and Proximity Searching Multimedia Multimedia management Relational Characters, Numbers, Dates, LOBs Oracle XML Support • SQL access to XML content and XML access to relational content • Flexible native XML storage delivers optimized application performance • Repository integration enables document centric access, security and integrity and development • Full support for key XML standards including the W3C XQuery recommendation and SQL/XML Extending Oracle’s XML Lead Performance Comprehensive XML storage and indexing and efficient end-to-end XML processing • Binary XML - Compact and efficient storage representation • • • • Complements existing object storage and text storage models XMLtype storage format transparent to developer Single, compact XML representation across client, mid-tier, db Plan to place our Binary XML format into open source • XML Path Indexing for schema-less XML documents Extending Oracle’s XML Lead Performance Performance improvements in many areas • Recursive schema handling • Scalable XSL output • XQuery and SQL/XML query optimization with schema-less XML • XML update optimizations • Asynchronous XML index updates Extending Oracle’s XML Lead XML application development Enhanced productivity and flexibility • Standards Support • • • • • XQuery 1.0 support XDB as HTTP Server – expose PL/SQL as Web Services Content Repository API for Java (JSR 170) support Support for JCR 1.0, SQL:2007 XLink/Xinclude support • XMLDB Repository Triggers Extending Oracle’s XML Lead Operational completeness Making XMLDB Mainstream • XML type support in streams & logical standby • In-place XML schema evolution • XML language translation support • Asynchronous XMLindex updates • Large text node handling Oracle XMLDB – Sustained Innovation Performance Binary XML Storage & Indexing XQuery XML API’s 1998 XML Storage & Repository 2001 2004 2007 Multimedia and Spatial Enhancements • New spatial features • 3-D geometry, surface, and point cloud storage and indexing • Spatial web services • Spatial routing engine enhancements • Scalability, manageability, reliability, usability enhancements • Multimedia and medical imaging • 3X performance improvement for common image processing operations • Large media handling (up to 128 TB) • DICOM medical imaging support • Java Advanced Imaging (JAI) support Text and Semantic Technology Oracle Text • • • • Enterprise Manager support Incremental, online indexing Web services Composite indexes speed text, relational queries • User-defined relevance ranking • More advanced new multilingual and linguistic support, including Arabic and Hebrew Semantic Technology • Query using concepts and terms related to a keyword • applications in life sciences, defense, health care, enterprise information integration • New in 11g: • Improved query/optimizer support • Improved bulk loading • native, lightweight OWL inferencing Fast OLAP via SQL Cubes as Materialized Views SQL Query • Detail data is stored in tables • Applications use SQL queries • Summary data is managed in cubes (like materialized view) • Databases manages cube refresh as data changes • Fast, incremental • Cost-based aggregation • SQL queries automatically rewritten to access the cube • Application is unchanged, but updates and queries are faster OLAP for Every Data Warehouse SQL Query • Any SQL application can leverage OLAP Option performance • Cube technology is optimized for business intelligence • Excellent performance for ad-hoc query loads • Highly optimized incremental refresh and aggregation • Content rich calculations are easily queried with SQL • Seamless fit with warehouse administration process Oracle Record of OLAP Innovation SQL Star Queries Cubes but no SQL access SQL Views over Cubes Transparent SQL Rewrite to Cubes Oracle Data Mining Enhanced, Automated, Simplified Find the top 5 stores that are the furthest below forecast SELECT store_id, loc, sales, forecast FROM (SELECT * FROM (SELECT s.*, PREDICTION (CREATE REGRESSION FOR sales USING *) OVER () as forecast FROM stores) ORDER BY forecast-sales) WHERE ROWNUM < 6; On-the-fly Data Mining in SQL • Enhanced data mining in SQL • Eases application use of models • Data Models as schema objects • simpler administration, security • Java API standards compliance • Automated and embedded data transformations • (aka “Supermodels”) • eases model building and scoring • New predictive analytic data mining procedures • Regressions, segmentation, profiling, grouping Enhanced SQL – PIVOT and UNPIVOT PROD QUARTER AMOUNT Shoes Q1 2000 Shoes Q2 1000 Jeans Q1 1000 Jeans Q3 500 Jeans Q3 100 Jeans Q4 1000 SELECT * FROM sales PIVOT (sum(amount) FOR quarter in ‘Q1’,’Q2’,’Q3’,’Q4’); PROD Q1 Q2 Q3 Q4 Shoes 2000 1000 Null Null Jeans 1000 Null 600 1000 • Rotate rows into columns and vice versa • Create aggregated cross-tabular result set • Use to combine multiple rows to one, enabling inter-row computations without self-join` ALL Your Data – Sustained Innovation Oracle8i Text Oracle8 Spatial VLDB Media LOB’s 1999 Objectrelational Extensibility 1977 Oracle 10g ULDB Location Services Oracle9i XQuery XML DB Repository 2004 SQL/XML 2001 Oracle 11g Secure Files Ontology DICOM Secure Search Binary XML XML Index 2007 Maintaining Availability, Security and Performance The Availability Challenge Unplanned Downtime Server Failures Storage Failure Data Failures Human Error Corruption Planned Downtime System Changes Data Changes Site Failure Server Scale-Out with Real Application Clusters • Great scalability & availability Active/passive • Pools standard low cost servers, clustered servers improves server utilization • Runs applications unchanged • 1000s of production customers Pool of database servers New: 11g fine tunes performance, scaling, fail-over, management Designed to Tolerate Server Failures Storage Scale-Out with Automated Storage Management Dedicated disks for database storage Pool of Storage for all databases • ASM pools storage from modular storage arrays • Automatically remirrors when disk or array fails New: 11g fine tunes recovery from corrupt blocks and crashed storage arrays Designed to Tolerate Storage Array Failure Error Investigation with Flashback • Flashback Query • Query all data at point in time select * from Emp AS OF ‘2:00 P.M.’ where … • Flashback Versions Query Tx 3 Tx 2 Tx 1 • See all versions of a row between times • See transactions that changed the row select * from emp VERSIONS BETWEEN ‘2:00 PM’ and ‘3:00 PM’ where … Error Correction with Flashback Correct errors at any level Database Customer • Flashback Database – restore database to time • Flashback Table - restore contents of tables to time Order New: Flashback Transaction – back out transaction and all subsequent conflicting transactions Data Recovery Advisor Investigation and Planning • Diagnoses persistent data failures • Presents appropriate repair actions • Intelligently determines plan for recovery, selecting repair option Recovery Time to Repair Uncertainty and confusion during an outage causes delays and errors • Data file restore/recovery, media recovery, Flashback database, etc. • Validates plan w.r.t. availability of media components required • Can automatically apply plan Reduces Downtime by Eliminating Confusion Customers Don’t Benefit from Disaster Recovery • Cost – choose no DR, or under-configure DR • Rarely used – so little confidence in fail-over • Data loss – leads to downstream problems • Slow – faster to fix problems than fail-over • Limited protection – site failures only Oracle Data Guard – Practical Disaster Protection Production Database • • • • • • Log Shipping Synchronous or asynchronous log shipping Corruptions don’t propagate Configurable for zero data loss Automatic fail over in seconds to standby (10.2) Uses far less bandwidth than remote mirroring Thousands of production customers Standby Database Snapshot Standby Physical Standby Apply Logs Open Database Back out Changes Snapshot Standby Perform Testing Continuous Redo Shipping • Use Standby Database for testing and development • Eliminates cost of DR • Preserves zero data loss while in test/dev mode • But no real time query or fast failover Real-Time Query with Physical Standby Production Database • • • • Continuous Log Shipping Real-time Queries Logical or Physical Standby Database Previously available with Logical Standby Continuous Available with Physical Standby in 11.1 Real-Time Query Handles all data types, very fast, simple operation Eliminates cost of DR: all hardware used for production Benefits of Data Guard for Disaster Recovery • No cost – hardware and software • Zero data loss - over long distances • Fast Automatic Failover • Covers all common failures – not just site failures • Works transparently for existing applications Bonus – Big Reduction in Planned Downtime Reducing Planned Downtime for System Changes • Scaling Servers on Demand Database Storage • Add RAC nodes online w/o data movement • Scaling storage on demand • Add ASM disks online w/ auto data rebalancing • Online patching • RAC rolling upgrades for complex patches, CPUs • Rolling upgrades w/ standby for patch sets, version changes New: simple one-off patches can be applied to a running Oracle instance Data Security: Oracle Products Access Control Identity Management • Database Vault • Label Security • Oracle Identity Management Monitoring • Audit Vault • EM Configuration Pack Core Platform Security Data Protection • Advanced Security • Secure Backup Oracle Advanced Security Transparent Data Encryption • Column level encryption (10gR2) • Tablespace level encryption • For encrypting entire application data • Supports foreign keys and range scan • LOB encryption • Master Key protection in hardware using PKCS #11 Oracle Database Vault • Controls privileged users, enforces separation of duties • Administrators can’t access application data • Site-specific controls limit access by any user Coming soon for Oracle9i, 10g, 11g DBA FIN DBA Fin Fin Realm SELECT … Unexpected IP address FIN user CREATE … HR DBA HR HR Realm Outside business hours HR DBA Oracle Audit Vault Coming soon Console Policies Monitor Report/Alert Secure Audit Warehouse Enterprise-wide Audit Solution for Compliance & Security • Collects audit info from multiple sources • SDK for customization 9i, 10gR1, 10gR2 Non Oracle DB OS App Server Audit Collectors Collector SDK Applications • Monitor security-sensitive activities • Tamper-evident repository • Alerting policies Core Database Security Enhancements • Secure configuration by default • Password management settings • Audit sensitive administrative operations • Stronger password verifier • PKI / Kerberos authentication for super privileged DBAs Oracle Database Security Sustained Innovation Oracle Audit Vault Oracle Database Vault DB Security Evaluation #19 Transparent Data Encryption EM Configuration Scanning Fine Grained Auditing (9i) Secure application roles Client Identifier / Identity propagation Oracle Label Security (2000) Proxy authentication Enterprise User Security Global roles Virtual Private Database (8i) Database Encryption API Strong authentication (PKI, Kerberos, RADIUS) Native Network Encryption (Oracle7) Database Auditing Government customer 1977 2007 Sustain Optimal Performance with Self-Managing Database Auto-Tuning Advisory Instrumentation Sustain Optimal Performance with Self-Managing Database Auto-Tuning Advisory Instrumentation Automatic SQL Tuning Packaged Apps Custom Apps SQL Workload Automatic SQL Tuning SQL Profiles SQL Analysis • Captures high-load SQL • Tunes SQL by creating SQL profiles • Optionally implements greatly improved SQL plans • Reports analysis Report Nightly Welltuned SQL Manually implement • Runs runs in maintenance window Automatic Memory Tuning OS Memory • Automatically adapts to workload changes OS Memory DB Shared Memory • Adjusts PGA, SGA, o/s memory DB Shared Memory DB Process Memory DB Process Memory • Maximizes memory use efficiency • Single dynamic memory parameter • Helps eliminate out-ofmemory errors Database Result Cache • Automatically caches results of queries, query blocks, or pl/sql function calls • Cache is shared across statements and sessions on server • Significant speed up for read-only / read-mostly data • Full consistency and proper semantics • Cache refreshed when any underlying table updated query 1 executes Group by join join Table 1 Table 2 Table 3 Database Result Cache • Automatically caches results of queries, query blocks, or pl/sql function calls • Cache is shared across statements and sessions on server • Significant speed up for read-only / read-mostly data • Full consistency and proper semantics • Cache refreshed when any underlying table updated result is cached cached Group resultby join join Table 1 Table 2 Table 3 Database Result Cache • Automatically caches results of queries, query blocks, or pl/sql function calls • Cache is shared across statements and sessions on server • Significant speed up for read-only / read-mostly data • Full consistency and proper semantics • Cache refreshed when any underlying table updated join cached Group resultby join Table 4 join query 2 uses cached result transparently Group by join Table 1 join Table 2 Table 3 Table 5 Table 5 Database Result Cache • Automatically caches results of queries, query blocks, or pl/sql function calls • Cache is shared across statements and sessions on server • Significant speed up for read-only / read-mostly data • Full consistency and proper semantics • Cache refreshed when any underlying table updated join cached Group resultby join Table 4 join query 2 uses cached result transparently Group by join Table 1 join Table 2 Table 3 Table 5 Table 5 Other Transparent Performance Improvements • Auto-compiled PL/SQL and Java • Compression for all data • XML performance improvements • Spatial, multimedia and semantic optimizations • Streams performance improvements • SecureFiles faster than files or LOBs • RAC and ASM optimizations Managing Systems and Managing Change <Insert Picture Here> Alfred North Whitehead: Principia Mathematica “The art of progress is to preserve order amid change and to preserve change amid order.” Lifecycle of Change Management Test Make Change Set Up Test Environments Patches & Workarounds Diagnose & Resolve Problems Provision for Production Diagnose Problems Preserve Order Amid Change Database Replay Realistic Testing • Recreate actual production database workload • Capture production workload incl. concurrency • Replay workload in test with production timing • Analyze & fix issues before production Middle Tier Production Environment Oracle DB servers Test (RAC) Environment` Capture DB Workload Replay DB Workload Storage SQL Performance Analyzer Realistic Testing • Test impact of change on SQL performance • Capture SQL incl incl. statistics & bind vars • Re-execute SQL in test environment • Use SQL Tuning Advisor to seed SQL plans Production Environment Oracle DB servers Capture SQL Storage Test (RAC) Environment` Execute SQL Queries Oracle Enterprise Manager 10g R3 • Configuration Management • Service Level Management Applications • Application Performance Mgmt • Lifecycle Management • Dashboards • Integrated view of applications and infrastructure • Manage service levels, diagnostics and remediation • Automate capacity on demand • Simplify deployment Middleware Database O/S Oracle 10g R3 Grid Control Stage Gold Image Create reference system 1 2 Reliable Deployment Automatic provisioning from gold image 4 Scale-out RAC nodes 3 Create production system • Greatly simplify RAC provisioning with automation • Single click RAC scale-out and scale-back • Provision full RAC and Clusterware systems • Configures entire stack • Pre-requisite checks & automatic fix-ups Patch Automation EM Support Workbench Reliable Deployment Proactively search Metalink for relevant patches Apply patch Download available patch Alert DBA Proactive advisories from Oracle Metalink • Patches acquired based on configurations and feature usage (new in 11g DB) • Stage once in library for multiple deployments • Best practice driven patching • Rolling patching support for RAC/ASM/Clusterware (new in 10gR3 Grid Control) Automatic Diagnostic Workflow Diagnostic Repository Alert DBA Run Health Checks Error! Auto Incident Creation First-Failure Capture Check Metalink If unknown issue Package incident & config Use Repair Advisor If known issue EM Support Workbench Apply patch / workaround Use Repair Advisors Reduce Time to Problem Resolution • Highest Quality of Service • Performance • Scalability • Availability • Security • Lowest Cost • Easier to Manage • Reduce risk of change The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remain at the sole discretion of Oracle.