May 2012 Oracle Spatial User Conference 1 The Oracle bug life cycle and support process January 29, 2013 Oracle, Blythe Valley Park, Solihull Presented by Rich Pitts - Senior Principal Support Engineer Oracle Support This talk came from “Making the most of a Spatial upgrade (The bug Life Cycle)” Ronald Reagan Building and International Trade Center Washington, DC USA Program Agenda • • • • • Oracle Support Overview Support Process – Bug Life Cycle A Quick look at Patch sets 11.2.0.3 interesting points to note & RAT Where to find help…. Oracle Premier Support Comprehensive Coverage for Your Oracle Database and other Oracle Products Service and Support Quickly diagnose and resolve issues 5 Tools and Resources Get the most of your Oracle products with proactive services Product Innovation Keep pace with change and capitalize on new opportunities My Oracle Support provides access to 30+ Database Communities • Spatial • Database Administration • Database Backup and Recovery • Database Install Unix and Linux • Database Security Products • Oracle Multimedia • Best Practices • Oracle Text • Tips and tricks • Patch Reviews - DB • PL/SQL • Pre-compilers and OCI • Scalability RAC • SQL Language • Managing DB Memory & 4030/4031 • Storage Management • Windows Oracle Data Access Components • XML Database • A real-time multi-channel personalized support environment to share ideas, connect with peers and Oracle experts • Rich Content: • Whitepapers, • Live and archived Webcasts and much more • Anyone with access to My Oracle Support portal is automatically a community member 6 Lifetime Support Policy – Database Jan2013 RDBMS OUG Conference Open World 2011 May 2012 Location Intelligence Conference 11.2 January 2018 Indefinite January 2015 11.1 Premier Support August 2015 Extended Support Indefinite August 2012 Sustaining Support 10.2 July 2013 R2 Indefinite July 2010 10.1 January 2012 Indefinite January 2009 9.2 July 2010 Indefinite July 2007 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2004 2003 2002 7 Program Agenda • • • • • 8 Oracle Support Overview Support Process – Bug Life Cycle A Quick look at Patch sets 11.2.0.3 interesting points to note & RAT Where to find help…. Customer> Support> BDE> Development (Bug Diagnosis & Escalation) • What BDE does… richard.pitts@oracle.com In Server Technology BDE team - Senior Principal Support Engineer Core RDBMS, Optimizer, RAC, ASM, Spatial began with MultiDimension at 8.1.7 Currently Global BDE Lead For Oracle Spatial and other complex content... - Bug triage - Bug escalations & Technical Management - Backport requests - Forum Participation - Requests For Assistance (RFAs) for Service Requests (SRs) - Knowledge Engineering - Severity 1 (P1) bug screening – always someone on duty 24x7 days a week 17 yrs Where are you ? Where should you be aiming? Open World 2012 -> Is the Future too cloudy? 11.2.0.4 Terminal due ?2013? Spatial User Group 2012 Hopefully you’re HERE! 11.2.0.3.x 11.2 January 2018 Jan2013 RDBMS OUG Conference Indefinite January 2015 11.1 Premier Support Extended Support August 2015 **** Are you here? **** Indefinite August 2012 Sustaining Support 10.2 July 2013 R2 Indefinite July 2010 10.1 January 2012 History (we hope) Indefinite January 2009 9.2 July 2010 Indefinite July 2007 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2004 2003 2002 10 The Bug Life Cycle – Fix in Main Customer has a problem with 10.2.0.5 BDE Tests Opens a Service Request via MOS (Which patch are you on) Problem defined Support Tests 10.2.0.5 & 11.2.0.3 12.1.0.0 – Reviews Passes to Development Bug Visible on MOS 11 Fixed on Main (Job Done) When a bug does not reproduce More work for BDE Problem at 11.2.0.3 Start to find the Fix 12.1.0.0 Does not reproduce... 12 Diagnosis…… Many spatial bugs are tagged to aid rediscovery. STACK Trace Searching knowledge base, You can do this as well... Bug Process: RFI’s, BLR’s & PSE’s Terms and definitions • In order to obtain a fix in the earlier release, i.e. 11.2.0.2 – • – – – 14 BDE files a Base Label Request (BLR) for 11.2.0.2 For the Version the customer has requested Automatically a Request For Inclusion (RFI) IE 11.2.0.4 is produced, (IF 12.1 was also production it would do a 12.1 RFI as well, we don't know where you will go next.... A Merge Label Request (MLR) may be required once the BLR is produced.... ”opatch lsinventory –detail” FINALLY, a Patch Set Exception (PSE) for Customer’s platform is produced.... The Bug Life Cycle – Getting the fix Fixed on Main (Job Done) Development produce fixes Customer Requests Patch for 11.2.0.2 BDE Files BLR For 11.2.0.2 & RFI For 11.2.0.4 BDE Files PSE for Customer Platform 15 Patch produced An MLR example…. A requested fix plus a fix applied affecting the same modules. Bug 5611025 is a Bug 5448688 is a MLR of bug 5181800 MLR of bug 5380055 4883174 4953737 5098050 4967266 4395779 5001975 & bug 5191972 1. rdbms/notes/backport.txt 1. rdbms/notes/backport.txt 2. rdbms/src/server/rcv/if/kcrr.h 2. rdbms/src/server/rcv/if/kcrr.h 3. rdbms/src/server/rcv/kcrf.c 3. rdbms/src/server/rcv/kcrf.c 4. rdbms/src/server/rcv/kcrfr.c 4. rdbms/src/server/rcv/krsl.c 5. rdbms/src/server/rcv/kcrr.c 5. rdbms/src/server/rcv/kcrr.c 6. rdbms/src/server/rcv/krsf.c 6. rdbms/src/server/rcv/krsf.c 7. rdbms/src/server/sqllang/parse/dbsdrv.c 7. rdbms/src/server/rcv/if/krsf.h 8. rdbms/src/server/txn/if/ktucts.h 8. rdbms/src/server/rcv/if/krsl.h 9. rdbms/src/server/txn/lcltx/ktb.c 16 Green is No Clash MLR to PSE Bug 5616780 is a MERGE OF THESE MERGES! FINISHED status 35 1. rdbms/notes/backport.txt Bug 5611025 is a MLR of bugs 5380055 4883174 4953737 5098050 4967266 4395779 5001975 2. rdbms/src/server/rcv/if/kcrr.h 3. rdbms/src/server/rcv/if/krsf.h 4. rdbms/src/server/rcv/if/krsl.h 5. rdbms/src/server/rcv/kcrf.c 6. rdbms/src/server/rcv/kcrfr.c 7. rdbms/src/server/rcv/kcrr.c Bug 5448688 is a MLR of bug 5181800 8. rdbms/src/server/rcv/krsf.c & bug 5191972 9. rdbms/src/server/rcv/krsl.c 10.rdbms/src/server/sqllang/parse/dbsdrv.c Finally PSE Filed to obtain a one-off fix for Customer’s version on their particular platform. PSE BUG 5616781 filed ended at status 93 Should be status 90 17 11.rdbms/src/server/txn/if/ktucts.h 12.rdbms/src/server/txn/lcltx/ktb.c Composite Patching – Prevention of requesting unnecessary BLR/MLR/PSE’s due to fact that there is NO conflict at all towards the PSU in question – The composite patch approach introduced starting from 11.2.0.2.7 + 11.2.0.3.2 – Conflict checking towards composite patches Attempting a more automated apporach Make supports life easy opatch lsinventory –detail 18 Why are we introducing Composite Patches? Currently Cumulative patches have several deficiencies. • Installing a new version of the patch requires a rollback of the currently installed cumulative patch • All overlay patches must be rolled back when a new version is installed • Any overlay patches not included in the new cumulative patch must be reapplied after they are recreated • It’s expensive and time consuming to recreate all the patches • Applying Cumulative patches is slow 19 When to use a base patch xx.x.0.x.0 • A PSU specific PSE (patch) should only be installed/made/ requested in case there is a conflict towards the PSU. • In case of no PSU conflict at all the base version of the patch should be installed/used (11.2.0.3.0/11.2.0.2.0/…). • KEEP IT SIMPLE 20 What are Composite Patches? • A packaging format used for Cumulative patches • A model to distinguish each new layer of a Cumulative patch • Replace only the parts of the patch that have changed and are not currently installed on the system • Rollback only the parts of the patch that were previously installed • Reduce the need to roll back overlay patches • Improve patch apply performance 21 Life gets complex 11.2.0.2.4 11.2.0.2.3 11.2.0.2.2 11.2.0.2.1 22 Complexity PSE A not conflicting with any PSU# => so base version of PSE 11.2.0.3.0 MLR BC for which B conflicts with C and both not conflicting with any PSU# => so base version of MLR 11.2.0.3.0 PSE D conflicting with PSU#1 => so PSU 11.2.0.3.1 needed PSE E conflicting with PSU#2 => so PSU 11.2.0.3.2 needed MLR FG for which F conflicts with G and G conflicts with PSU#4 implying that a FG MLR is needed on top of PSU#4 (which requires a PSU#4 BLR of F and G) => so PSU 11.2.0.3.4 version needed PSE H is an incorrect PSE, there is only a conflict with PSU 11.2.0.3.1 and no conflict with higher PSU’s, the correct version to install/request/… for this patch is a PSU 11.2.0.3.1 version 23 Windows does it differently But can be as complex Note 161549.1 11.2.0.2.4 11.2.0.2.3 11.2.0.2.2 11.2.0.2.1 24 Program Agenda • • • • • 25 Oracle Support Overview Support Process – Bug Life Cycle Quick look at Patch sets 11.2.0.3 interesting points to note & RAT Where to find help…. Terminal release of 10.2.0.5.x Bug 8202632 - 10.2.0.5.0 See – Note 161818.1 • Fixes many of the 10.2.0.4 issues • If your on 10.2 STILL then you need to be on 10.2.0.5 Premier Support Extended Support May 2012 Location Intelligence Conference Sustaining Support 10.2 July 2013 R2 Indefinite July 2010 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2004 2003 2002 26 Terminal release of 11.1.0.7.x Strongly advise move to 11.2.0.3 ASAP • Architecture changes in 11.2 make back ports tricky if not impossible [SDO_ANYINTERACT fixes wont backport....] A COMPLEX SDO_ANYINTERACT query performance • 11.1.0.7 Elapsed: 00:41:43.74 • 11.2.0.2, 16 times faster! Elapsed: 00:02:30.27 Jan2013 RDBMS OUG Conference Premier Support Extended Support May 2012 Location Intelligence Conference 11.1 Sustaining Support August 2015 August 2012 Indefinite 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2004 2003 2002 27 Current Release of 11.2.0.3.X To infinity and beyond? •Fixes much of the 10.2, 11.1, 11.2.0.1 and 11.2.0.2 issues…. •sdo_anyinteract 43 secs at 11.1.0.7 to 0.7 of a second •There are some important changes to Oracle Database Patch Sets starting with 11.2.0.2 (refer to Note 1189783.1 for details) •11.2.0.4 or 12.1 where will you go? Jan2013 RDBMS OUG Conference Premier Support May 2012 Location Intelligence Conference Extended Support 11.2 Sustaining Support Indefinite January 2015 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2004 2003 2002 28 Optimizer!!! Constantly evolving…. Mass of fixes each release. • Issues 10.2.0.5 and beyond testing will show • Slow Performance when using SDO_RELATE / SDO_FILTER (Doc ID 1268383.1) • Execution plans involving "bitmap conversion to rowids" and "table access full". • Cause Optimizer is not selecting the optimal plan. This is being addressed in a future release There is a workaround… 29 SDO_RELATE / SDO_FILTER (Doc ID 1268383.1) Workaround… SQL> connect /as sysdba SQL> alter session set current_schema=MDSYS; SQL> DISASSOCIATE STATISTICS FROM INDEXTYPES spatial_index FORCE; SQL> DISASSOCIATE STATISTICS FROM PACKAGES sdo_3gl FORCE; SQL> DISASSOCIATE STATISTICS FROM PACKAGES prvt_idx FORCE; Then be sure either flush the shared pool or bounce the database for the effects to take place 30 Consider Rapid Application Testing Difficult to test spatial systems…. • • • • • • Been around a while – mature enough to use…. “Record” production system as it runs… Great for peak loads… etc. Replay on test/upgraded system Patch, change statistics/base lines Replay again and again…. 31 Test results Comparison of DB Time by Platform 800.00 700.00 674.55 600.00 500.00 400.00 DB Time (hrs) 300.00 200.00 81.19 100.00 36.86 37.53 Exadata 11.2 Exadata 11.2 (HCC) - 32 GDMS 11.2 PROD 11.1 Test results Redo Generation by Platform 100.00 90.00 85.73 80.00 70.00 60.00 50.00 Redo Size (GB) 40.00 30.00 20.00 18.18 18.04 Exadata 11.2 (HCC) GDMS 11.2 14.66 10.00 Exadata 11.2 33 PROD 11.1 Program Agenda • • • • • Oracle Support Overview Support Process – Bug Life Cycle Quick look at Patch sets 11.2.0.3 interesting points to note & RAT Where to find help…. IF you do nothing you might end up here..... 34 We don’t want you coming for an ORA-paddle **** Use the MOS store *** 35 USEFUL Notes See – Note 161818.1 Oracle 11gR2 Upgrade Companion - 785351.1 36 THANK YOU 37 Q&A