Hello and welcome to this online, self-paced course entitled SQL Tuning New Features.
My name is Dimpi Sarmah. I am a curriculum developer at Oracle and work with the Server
Technologies Curriculum Group. I will be your guide for this course. The aim of this course is to
introduce you to the new and enhanced features of SQL Tuning in Oracle Database 12c,
Release 1.
Oracle Database 12c - SQL Tuning New Features - 1
Before we begin, take a look at some of the features of this Flash-based course player. If you’ve
attended a similar self-paced course in the past, feel free to skip this slide.
Let’s start with the outline. It’s set up so that you can go at your own pace. Feel free to skip
topics that you are familiar with, jump to a feature that interests you, or go back and review
topics that were already covered. Simply click a slide title in the outline to display its contents.
By default we will automatically walk you through the slides without requiring you to use the
outline.
Use the Transcript tab to view the audio transcript for each slide, and use the Search tab to find
specific information in the course.
Next, let’s look at the player controls. Use these controls to pause, play, or move to the
Previous or Next slides. You can also use the interactive progress bar to fast forward or rewind
the current slide. Some interactive slides in this course may contain additional navigation and
controls.
This icon in the bottom right corner allows you to change the view of this player. Some parts of
the course are programmed to open in full-screen mode, but you can control the view at any
time by using this button.
A downloadable print version of this course is available by clicking the Attachments button.
This course will now pause so that you can take some time to explore the interface. When
you’re ready to continue, click the Next button or click the next slide title in the outline at left.
Oracle Database 12c - SQL Tuning New Features - 2
At the end of this course, you should be able to understand:
•
Describe Adaptive SQL Plan Management
•
Describe enhancements to the SQL management base
•
Use Adaptive plan to improve query performance
•
Describe Enhancements to dynamic statistics
•
Describe Reoptimization for adaptive execution plans
•
Use SQL plan directives to generate better plan
•
Describe Statistics Gathering performance improvements
•
Use new histograms
•
Describe enhancements to extended statistics and detect useful column groups for a specific
workload
Oracle Database 12c - SQL Tuning New Features - 3
Oracle Database 12c - SQL Tuning New Features - 4
This course consists of three major modules:
• Adaptive SQL Plan Management
• Adaptive Query Optimization
• Optimizer Statistics Management
Examples, demonstrations, capabilities and processes will be reviewed in each module. Let's
begin with a discussion of Adaptive SQL Plan Management.
Oracle Database 12c - SQL Tuning New Features - 5
This module covers the following topics:
• SQL Plan Management: Overview
• Adaptive SQL Plan Management
- Automatically Evolving SQL Plan Baseline
• SQL Management Base Enhancements
Now, let's begin with SQL Plan Management overview.
Oracle Database 12c - SQL Tuning New Features - 6
SQL Plan Management (SPM) ensures that runtime performance never degrades due to the
change of an execution plan.
It has two main objectives:
• It prevents performance regressions resulting from sudden changes to the execution
plan of a SQL statement by providing components for capturing, selecting, and evolving
SQL plan information.
• It offers performance improvements by adapting to database system changes.
With SPM, the optimizer automatically manages execution plans and ensures that only known
or verified plans are used. When a new plan is found for a SQL statement, the plan is not
used until the database verifies that its performance is comparable to or better than the
current plans. Verification is a manual process performed either through Oracle Enterprise
Manager (EM) or by running DBMS_SPM.EVOLVE_SQL_PLAN_BASELINES in Oracle
Database.
The components of SPM are SQL plan baseline capture, SQL plan baseline selection and
SQL plan baseline evolution. There are two ways to seed or populate a SQL management
base (SMB) with execution plans:
• Automatic capture of execution plans (available starting with Oracle Database 11g)
• Bulk load execution plans or pre-existing SQL plan baselines
Oracle Database 12c - SQL Tuning New Features - 7
Starting in Oracle Database 12c, the database can use adaptive SPM. With adaptive SQL
plan management, DBAs no longer have to manually run the verification or evolve process for
non-accepted plans. They can go back days or weeks later and review what plans were
evolved during each of the nightly maintenance window. When automatic SQL tuning is in
COMPREHENSIVE mode, it runs a verification or evolve process for all SQL statements that
have non-accepted plans during the nightly maintenance window. All the non-accepted plans
are ranked and the evolve process is run for them. The newly found plans are ranked the
highest.
If the non-accepted plan performs better than the existing accepted plan (or plans) in the SQL
plan baseline, then the plan is automatically accepted and becomes usable by the optimizer.
After the verification is complete, a persistent report is generated detailing how the nonaccepted plan performs compared to the accepted plan performance.
As the evolve process is now an AUTOTASK, DBAs can also schedule their own evolve job at
end time and use DBMS_SPM.REPORT_AUTO_EVOLVE_TASK to view the results of auto job.
The Automatic Plan Evolution of SQL Plan Management is a feature of the Tuning Pack.
Oracle Database 12c - SQL Tuning New Features - 8
SPM Evolve Advisor is a SQL advisor that evolves plans that have recently been added to the
SQL plan baseline. The advisor simplifies plan evolution by eliminating the manual chore of
evolution.
New functions introduced in the DBMS_SPM package work on the advisory task infrastructure.
The new functions such DBMS_SPM.CREATE_EVOLVE_TASK,
DBMS_SPM.EXECUTE_EVOLVE_TASK, etc allow database administrators (DBAs) to create a
task, evolve one or more plans in the task, fetch the report of the task, and implement the
results of a task. The task can be executed multiple times, and the report of a given task can
be fetched multiple times. SPM can schedule an evolve task as well as rerun an evolve task.
There is also a function to implement the results of a task, which in this case would be to
accept the currently non-accepted plans.
The task infrastructure allows the report of the evolution to be persistently stored in the
advisory framework repository. It also allows Enterprise Manager to schedule SQL plan
baselines evolution and fetch reports on demand. The new function supports HTML and XML
reports in addition to TEXT. In Oracle Database 12c, the new task-based functions in
DBMS_SPM retain the time_limit specification as a task parameter. It has the same default
values as Oracle Database 11g. As for commit, the new evolve task does not accept any
plans by default. Users can view the report by using the appropriate function, and then
execute a new implement function to accept the successful plans.
Oracle Database 12c - SQL Tuning New Features - 9
The SPM Evolve Advisor is an AutoTask (SYS_AUTO_SPM_EVOLVE_TASK), which operates
during the nightly maintenance window and automatically runs the evolve process for
unaccepted plans in SPM. The AutoTask ranks all unaccepted plans in SPM (newly found
plans ranking highest) and then runs the evolve process for as many plans as possible before
the maintenance window ends.
The results of the nightly evolve task can be viewed using the
DBMS_SPM.REPORT_AUTO_EVOLVE_TASK function.
Oracle Database 12c - SQL Tuning New Features - 10
The SQL management base resides in the SYSAUX tablespace and store SQL plan baselines,
statement logs, plan histories, and SQL profiles.
In Oracle Database 11g, DBMS_XPLAN.DISPLAY_PLAN_BASELINE displayed the execution
plan by compiling the statement with the baseline.
In Oracle Database 12c, when a new plan is added to the plan history of a SQL statement, plan
rows are also stored in the SQL management base. When
DBMS_XPLAN.DISPLAY_PLAN_BASELINE is executed, you fetch the plan data from the SQL
management base. A new table is added to the SMB, which contains auxiliary information
about SQL management objects.
In Oracle Database 11g, the SQL Plan Management (SPM) plan rows are not stored in the
SMB. When customers upgrade to Oracle Database 12c, the plan rows are populated during
execution of the plans.
Oracle Database 12c - SQL Tuning New Features - 11
Answer:
1. EM or DBMS_SPM.EVOLVE_SQL_PLAN_BASELINES
2. SYS_AUTO_SPM_EVOLVE_TASK
3. create_evolve_task
4. True
Oracle Database 12c - SQL Tuning New Features - 12
Oracle Database 12c - SQL Tuning New Features - 1
Now let's move on to Adaptive Query Optimizations.
Oracle Database 12c - SQL Tuning New Features - 13
The module topics are:
• Adaptive Plans
• Dynamic Statistics
• Automatic Reoptimization
• SQL Plan Directives
Oracle Database 12c - SQL Tuning New Features - 14
Adaptive Query Optimization enables the optimizer to automatically adapt a poorly performing
execution plan at run time and prevent a bad plan from being chosen on subsequent
executions. It is extremely helpful when existing statistics are not sufficient to generate an
optimal plan. There are two distinct aspects in Adaptive Query Optimization, adaptive plans,
which focuses on improving the initial execution of a query and adaptive statistics, which
provide additional information to improve subsequent executions.
Oracle Database 12c - SQL Tuning New Features - 15
An adaptive plan is an execution plan that has different built-in plan options. During the first
execution, before a specific subplan becomes active, the optimizer makes a final decision
about which option to use. The optimizer bases its choice on observations made during the
execution up to this point. Adaptive plans enable the optimizer to defer the final plan decision
for a statement, until execution time, thereby potentially improving query performance. The
optimizer instruments its chosen plan (the default plan), with statistics at run time so that it can
detect if its cardinality estimates, differ greatly from the actual number of rows seen by the
operations in the plan. If there is a significant difference, then the plan or a portion of it can be
automatically adapted to avoid suboptimal performance on the first execution of a SQL
statement.
During statement execution, the statistics collector buffers a portion of rows. Portions of the
plan preceding the statistics collector can have alternate subplans, each of which is valid for a
subset of possible values returned by the collector.
Often the set of values for a subplan is a range. If a statistic falls into the valid values of a
subplan that was not the default plan, then the optimizer chooses the alternative subplan. After
the optimizer chooses a subplan, buffering is disabled. The statistics collector stops collecting
rows, and passes them through instead. On subsequent executions of the child cursor, the
optimizer disables buffering, and chooses the same final plan.
Oracle Database 12c - SQL Tuning New Features - 16
The code example shows a join of the SALES and CUSTOMERS tables. An adaptive plan for
this statement could show two possible plans, one with a nested loop and the other with a
hash join.
The following graphic shows the adaptive process. For the query given above, the adaptive
portion of the default plan contains two subplans, each of which uses a different join method.
The optimizer automatically determines when each join method is optimal, depending on
the cardinality of the left side of the join. The statistics collector buffers enough rows coming
from the sh.sales table to determine which join method to use.
In this example, the statistics collector retrieves row count statistics, and also buffers up to 10
rows. If the row count is below 10, then the optimizer chooses the nested loop; otherwise, the
optimizer chooses the hash join. Because the row count in SALES table is over 10, the
optimizer chooses a hash join, disables buffering, and makes the final plan.
Oracle Database 12c - SQL Tuning New Features - 17
The slide shows a join of the order_items and product_information tables. Explain plan
command always shows default plan. The first example shows a nested loops join as default
plan and no statistics collector shown in plan. In the second example, after the statement has
completed use DBMS_XPLAN.DISPLAY_CURSOR to see the final plan, the optimizer chooses
the hash join at execution time.
Oracle Database 12c - SQL Tuning New Features - 18
The full adaptive plan is displayed when format parameter ‘+adaptive’ is set.
Oracle Database 12c - SQL Tuning New Features - 19
In Oracle Database 12c dynamic sampling has been enhanced to become dynamic statistics.
Prior to Oracle Database 12c, dynamic sampling always occurred during optimization, and only
when a table in the query had no statistics. Starting in Oracle Database 12c, the optimizer can
use automatic dynamic statistics. Dynamic statistics allow the optimizer to augment existing
statistics to get more accurate cardinality estimates for not only single table accesses but also
joins and group-by predicates. Insights resulting from dynamic statistics can be shared and
persisted through SQL plan directives.
The primary factor in the decision is whether available statistics are sufficient to generate a
good plan. If the statistics are insufficient, the optimizer uses dynamic statistics. The statistics
gathered by dynamic statistics are now persistent and may be used by other queries to
eliminate overhead of repeated sampling. It is also helpful to compensate for stale statistics on
volatile tables. Automatic dynamic statistics is enabled when the
OPTIMIZER_DYNAMIC_SAMPLING initialization parameter uses its default value or it is set to
11. Level 11 enables the optimizer to automatically decide to use dynamic statistics for any SQL
statement, even if all basic table statistics exist. The optimizer bases its decision, to use
dynamic statistics, on the complexity of the predicates used, the existing base statistics, and the
total execution time expected for the SQL statement.
Oracle Database 12c - SQL Tuning New Features - 20
As shown in the figure, the optimizer automatically performs dynamic statistics in the following cases:
Missing Statistics
When tables in a query have no statistics, the optimizer gathers basic statistics on these tables before
optimization. Statistics can be missing because new objects are created by the application without a follow-up call
to DBMS_STATS to gather statistics, or because statistics were locked on an object before statistics were gathered.
In this case, the statistics are not as high quality or as complete as the statistics gathered by using the
DBMS_STATS package. This trade-off is made to limit the impact on the compile time of the statement.
Stale Statistics
Statistics gathered by DBMS_STATS can become out-of-date. Typically, statistics are stale when 10 percent or
more of the rows in the table have changed since the last time that statistics were gathered.
Insufficient Statistics
Statistics can be insufficient when the optimizer estimates the selectivity of predicates (filter or join) or the GROUP
BY clause without taking into account the correlation between columns, skew in the column data distribution, and
so on.
In the figure, the database issues a recursive SQL statement to scan a small random sample of the table blocks.
The database applies the relevant single-table predicates and joins to estimate predicate selectivities.
The database persists the results of dynamic statistics as sharable statistics. The database can share the results
during the SQL compilation of one query with recompilations of the same query. The database can also reuse the
results for queries that have the same patterns. If no rows have been inserted, deleted, or updated in the table
being sampled, then the use of dynamic statistics is repeatable.
Oracle Database 12c - SQL Tuning New Features - 21
This slide shows an example of dynamic statistics.
Oracle Database 12c - SQL Tuning New Features - 22
Oracle Database 12c - SQL Tuning New Features - 23
Statistics feedback is a form of reoptimization. This feature does a very limited form of
reoptimization, after the first execution is completed. It automatically improves plans for
repeated queries for which the optimizer estimate improper cardinalities in the plan. The
optimizer misestimates cardinalities for a variety of reasons, such as missing or inaccurate
statistics, or complex predicates.
The feature works as follows: During the first execution of a SQL statement, the optimizer
generates an execution plan. The optimizer may enable monitoring for statistics feedback for
the shared SQL area when there are tables with no statistics or multiple conjunctive or
disjunctive filter predicates are on a table or predicates containing complex operators for
which the optimizer cannot accurately compute selectivity estimates. At the end of execution,
the optimizer compares its initial cardinality estimates to the actual number of rows returned
by each operation in the plan during execution. If estimates differ significantly from actual
cardinalities, then the optimizer stores the correct estimates for subsequent use. The
optimizer also creates a SQL plan directive so that other SQL statements can benefit from the
information obtained during this initial execution. After the first execution, the optimizer
disables monitoring for statistics feedback.
If the query executes again, then the optimizer uses the corrected cardinality estimates
instead of its usual estimates.
Oracle Database 12c - SQL Tuning New Features - 24
Execution of the query given in the code box given in the slide is monitored. The query has a
combination of an equality filter predicate and an inequality filter predicate which causes the
misestimation in the cardinality of the order_items table after these filters are applied.
Querying the plan in the cursor shows that the estimated rows (E-Rows) is far fewer than the
actual rows (A-Rows).
Oracle Database 12c - SQL Tuning New Features - 25
Execution statistics are used to reparse the statement during the second execution of the query
in the code box. During the second execution, the optimizer used statistics feedback and the
actual cardinality is fed back to the optimizer.
And this time the optimizer gets the cardinality estimate right, and the optimizer displays a
different plan.
Oracle Database 12c - SQL Tuning New Features - 26
In reoptimization, the optimizer adapts a plan only on subsequent statement executions. Deferred
optimization can fix any suboptimal plan chosen because of incorrect optimizer estimates, such as a
suboptimal distribution method or an incorrect choice of degree of parallelism. At the end of the first
execution, the optimizer has a more complete set of statistics, and this information can improve plan
selection in the future. A query may be reoptimized several times, each time creating a larger and more
precise set of optimizer adjustments.
Reoptimization fixes plan problems that adaptive plans cannot solve. Whereas adaptive plans help
decide between multiple subplans, they are not feasible for all kinds of plan changes. For example, a
query with an inefficient join order might cause suboptimal performance, but it is not feasible to adapt
the join order during execution. In these cases, the optimizer automatically considers automatic
reoptimization . Automatic reoptimization changes a plan on subsequent executions after the initial
execution.
As of Oracle Database 12c, join statistics are also included. Statements are continually monitored to
verify if statistics fluctuate over different executions. This feature improves the ability of the query
processing engine (compilation and execution) to generate better execution plans.
The optimizer uses the statistics in the following ways:
-
The statistics collectors and their associated optimizer estimates determine if automatic
reoptimization is an option. If statistics differ from estimates by more than a threshold, the
optimizer looks for a replacement plan.
-
After the optimizer identifies a query as a reoptimization candidate, the database submits
all collected statistics to the optimizer.
Oracle Database 12c - SQL Tuning New Features - 27
In previous releases, the database stored compilation and execution statistics in the shared
SQL area, which is nonpersistent. Starting in Oracle Database 12c, the database can use a
SQL plan directive, which is additional information and instruction that the optimizer can use
to generate a more optimal plan. A SQL plan directive might instruct the optimizer to collect
missing statistics, create column group statistics, or perform dynamic statistics. For example,
when joining two tables that have a data skew in their join columns, a SQL plan directive can
direct the optimizer to use dynamic statistics to obtain a more accurate join cardinality
estimate. During SQL compilation or execution, the database analyzes the query expressions
that are missing statistics or that misestimate optimizer cardinality to create SQL plan
directives.
SQL plan directives are created on query expressions rather than at a statement or object level to
ensure they can be applied to multiple SQL statements. It is also possible to have multiple SQL
plan directives used for a SQL statement. For example, directives can help the optimizer with
queries that use similar patterns, such as web-based queries that are the same except for a select
list item. The database stores SQL plan directives persistently in the SYSAUX tablespace. Any
SQL plan directive that is not used after 53 weeks will be automatically purged.
Directives can be monitored using views DBA_SQL_PLAN_DIR_OBJECTS or
DBA_SQL_PLAN_DIRECTIVES. SQL plan directives improve plan accuracy by persisting both
compilation and execution statistics in the SYSAUX tablespace, allowing them to be used by
multiple SQL statements.
Oracle Database 12c - SQL Tuning New Features - 28
The graphic in the slide shows how the database automatically creates and uses SQL plan
directives for SQL statements. You submit a SQL statement and then the optimizer
determines the plan. After execution, when the optimizer determines that the actual number of
rows (A-Rows) returned by each operation in the plan varies greatly from the estimates (ERows) then the submitted statement becomes a suitable candidate for automatic
reoptimization that means it check whether the customers query can be reoptimized.
The IS_REOPTIMIZABLE column is marked Y, so the database will perform a hard parse of
the customers query on the next execution. The optimizer uses the execution statistics from
this initial execution to determine the plan. The database persists the information learned from
reoptimization as a SQL plan directive.
Initially, the database stores SQL plan directives in memory, and then writes them to disk
every 15 minutes. Three entries appear in the views, one for the customers table itself, and
one for each of the correlated columns. Because the customers query has
the IS_REOPTIMIZABLE value of Y, if you re-execute the statement, then the database will
hard parse it again, and then generate a plan based on the previous execution statistics.
Oracle Database 12c - SQL Tuning New Features - 29
Here the slide shows that the SQL statement is submitted again. When you query the plan in
the cursor, the Note section in the plan indicates that the database used reoptimization for this
statement. The estimated number of rows (E-Rows) is now correct. But the SQL plan directive
has not been used yet. When you query the cursors for the customers query, a new plan
exists for the customers query, and also a new child cursor.
Oracle Database 12c - SQL Tuning New Features - 30
To confirm that a SQL plan directive exists and is usable for other statements, you submit a
query which is similar but not identical to the original customers query.
The optimizer checks for the directive and finds one on the STATE and COUNTRY columns.
After the optimizer gets the directive, it determines the plan with the help of the directive.
When you the query the state of the directive to determine whether it has changed. You
observe that the SQL plan directive has a state of MISSING_STATS. This state means that
the database has automatically determined that extended statistics, in the form of column
group statistics, can address this problem. So, if you gather statistics on the CUSTOMER table,
then you should also see that a new column group is automatically created for the STATE and
COUNTRY columns.
Oracle Database 12c - SQL Tuning New Features - 31
This slide shows a SQL Directives example.
Oracle Database 12c - SQL Tuning New Features - 32
Answer:
1. True
2. OPTIMIZER_ADAPTIVE_REPORTING_ONLY
Oracle Database 12c - SQL Tuning New Features - 33
Answer:
1. SYSAUX tablespace
2. DBA_SQL_PLAN_DIR_OBJECTS
Oracle Database 12c - SQL Tuning New Features - 34
Now let's move on to optimizer statistics management.
Oracle Database 12c - SQL Tuning New Features - 35
The module topics are:
•
Statistics Gathering Performance Improvements
- Online Statistics Gathering for Bulk-Load
- Concurrent Statistics Gathering
- Incremental Statistics Gathering Improvements
- Session-Private Statistics for Global Temporary Tables
•
Histogram Enhancements
•
Enhancements to Extended Statistics
Detecting Useful Column Groups for a Specific Workload: Example
Oracle Database 12c - SQL Tuning New Features - 36
With online statistics gathering, statistics are automatically created as part of a bulk load
operation such as a CREATE TABLE AS SELECT operation or an INSERT INTO ...
SELECT operation on an empty table. Online statistic gathering eliminates the necessity to
manually gather statistics after a bulk data load has occurred. In a data warehouse, you
frequently need to load a large amount of data into the database. For example, a sales record
data warehouse might load sales data nightly where Online statistics gathering is useful.
Oracle Database 12c now gathers the statistics automatically, and the principal benefits are
improved performance and manageability of bulk load operations by eliminating user
intervention to gather statistics after the load. And also by removing an additional full table scan
required for separate statistics gathering operations. While gathering online statistics, the
database does not gather index statistics or histograms. If index statistics or histograms are
required, then Oracle recommends running DBMS_STATS.GATHER_TABLE_STATS procedure
with the new options parameter set to GATHER AUTO after the bulk load. By
default, GATHER_TABLE_STATS only gathers missing statistics. Thus, if you
invoke GATHER_TABLE_STATS after a bulk load, then the database only gathers index
statistics or histograms, and not table and column statistics collected during the bulk load. You
can disable the feature at the statement level by using the
NO_GATHER_OPTIMIZER_STATISTICS hint. You can also explicitly enable the feature at the
statement level by using the GATHER_OPTIMIZER_STATISTICS hint.
Oracle Database 12c - SQL Tuning New Features - 37
Concurrent Statistics Gathering was introduced in Oracle Database 11g Release 2. The
Concurrent Statistics Gathering mode enables the database to simultaneously gather optimizer
statistics for multiple tables in a schema, table partitions, or table subpartitions.
You can enable Concurrent Statistics Gathering by setting the CONCURRENT preference with
DBMS_STATS.SET_GLOBAL_PREF. When CONCURRENT is set to TRUE, Oracle uses Oracle
Job Scheduler and Advanced Queuing components to create and manage multiple Statistics
Gathering jobs concurrently. The scheduler decides how many jobs to execute concurrently,
and how many will be queued based on available system resources. The number of concurrent
gather operations is controlled by the job_queue_processes parameter.
Each gather operation can still operate in parallel. Concurrently gathering statistics on multiple
tables and subpartitions can reduce the overall time it takes to gather statistics by allowing
Oracle Database to fully utilize a multiprocessor environment.
Oracle Database 12c - SQL Tuning New Features - 38
The figure in the slide shows the creations of jobs at different levels, where the COSTS and
SALES tables are partitioned and other tables are nonpartitioned. Job 2 acts as a coordinator
job for the COSTS table and job 7 acts as a coordinator job for the SALES table. The respective
job creates a job for each partition in the tables, as well as a separate job for the global
statistics of the COSTS and SALES tables.
Oracle Database 12c - SQL Tuning New Features - 39
The number of concurrent gathers is controlled by the job_queue_processes parameter. In
this example, it is set to 3.
Each concurrent gather operation can execute in parallel. In this example, the parallel degree is
4.
Oracle Database 12c - SQL Tuning New Features - 40
Prior to Oracle Database 12c, we gathered statistics one segment at a time in the AutoJob.
The process included collecting statistics on partitions of a partitioned table or on different
tables in a schema, database, and dictionary.
In Oracle Database 12c, you can run multiple statistics gathering tasks concurrently which
may provide considerable improvements in statistics gathering time by increasing the
resource utilization rate, in particular, on multi-CPU systems. You can use concurrency to
reduce the overall time required to gather statistics by enabling the database to fully use
multiple CPUs. But running multiple concurrent jobs may quickly overwhelm the system, and
cause major slow down on the execution of user workload. So, in order to control the
resource usage of the concurrent statistics gathering, you can rely on the Resource Manager
which enforces resource plans, and cap the resources used by automatic statistics gathering.
The new field, cap-off limit will ensure that the statistics gathering activity should take place
only if the system load is below a certain threshold. And you can set the maximum utilization
limit and cap-off limit for ORA$AUTOTASK_STATS_GROUP as 75% in the default resource plan
and 25% and 90%, respectively, for the default resource plans in the maintenance windows.
Employ smart scheduling mechanisms to have maximum degree of concurrency (to the extent
that resources are available), e.g., allow gathering of statistics for multiple partitioned tables
concurrently, and while avoiding deadlock cases.
Oracle Database 12c - SQL Tuning New Features - 41
Global statistics are the most important statistics, but they also take the longest time to collect,
because a full table scan is required. However, in Oracle Database 11g this issue was resolved
with the introduction of Incremental Global Statistics. With partitioned tables, new partitions are
typically added and data is loaded into these new partitions. After the partition is fully loaded,
partition-level statistics need to be gathered and the global statistics need to be updated to
reflect the new data. If the INCREMENTAL preference for a partitioned table is set to TRUE, the
DBMS_STATS.GATHER_*_STATS parameter GRANULARITY includes GLOBAL, and
ESTIMATE_PERCENT is set to AUTO_SAMPLE_SIZE, Oracle will gather statistics on the new
partition, and accurately update all global level statistics by scanning only those partitions that
have been added or modified, and not the entire table.
Incremental Global Statistics works by storing a synopsis for each partition in the table. A
synopsis is statistical metadata for that partition and the columns in the partition. Each synopsis
is stored in the SYSAUX tablespace and takes approximately 10KB. Global statistics are
generated by aggregating the synopses from each partition, thus eliminating the need for the
full table scan.
The graphic in the slide shows how the database gathers statistics for the initial six partitions of
the SALES table, and then creates synopses for each partition (S1, S2, and so on). The
database creates global statistics by aggregating the partition-level statistics and synopses.
Oracle Database 12c - SQL Tuning New Features - 42
The following graphic in the slide shows a new partition, containing data for May 24, being
added to the SALES table. The database gathers statistics for the newly added partition,
retrieves synopses for the other partitions, and then aggregates the synopses to create global
statistics.
Oracle Database 12c - SQL Tuning New Features - 43
In previous releases, it was not possible to generate the necessary statistics on the nonpartitioned table to support incremental statistics during the partition exchange operation.
Instead statistics had to be gathered on the partition after the exchange had taken place, in
order to ensure the global statistics could be maintained incrementally.
In Oracle Database 12c, the necessary statistics (synopsis) can be created on the nonpartitioned table so that, statistics exchanged during a partition exchange load can
automatically be used to maintain incrementally global statistics. The new DBMS_STATS table
preference INCREMENTAL_LEVEL can be used to identify a non-partitioned table that will be
used in partition exchange load. By setting the INCREMENTAL_LEVEL to TABLE (default is
PARTITION), Oracle will automatically create a synopsis for the table when statistics are
gathered. This table level synopsis will then become the partition level synopsis after the load
exchange.
Oracle Database 12c - SQL Tuning New Features - 44
In Oracle Database 11g, when incremental statistics are enabled on a table and a single row
changes in one of the partitions, then statistics for that partition are considered stale and have
to be re-gathered before they can be used to generate global level statistics.
In Oracle Database 12c a new preference called INCREMENTAL_STALENESS allows you to
control when partition statistics are considered stale and not good enough to generate global
level statistics. By default, INCREMENTAL_STALENESS is set to NULL, which means partition
level statistics are considered stale as soon as a single row changes . It can be set to
USE_STALE_PERCENT which means the partition level statistics will be used as long as the
percentage of rows changed is less than the value of the preference STALE_PRECENTAGE
(10% by default) or USE_LOCKED_STATS which means if statistics on a partition are locked,
they will be used to generate global level statistics regardless of how many rows have
changed in that partition since statistics were last gathered.
Oracle Database 12c - SQL Tuning New Features - 45
A global temporary table(GTT) is a special table that stores intermediate session-private data for a
specific duration. Prior to Oracle Database 12c, it was difficult to gather statistics on global temporary
tables because only one set of statistics was available for all occurrences of global temporary tables.
When you create a global temporary table, you create a definition that is visible to all sessions. No
physical storage is allocated. When a session first puts data into the table, the database allocates storage
space. The data in a temporary table is visible only to the current session. The database did not maintain
statistics differently for global temporary tables and non-global temporary tables.
Starting in Oracle Database 12c, you can maintain session-private statistics. You can choose to gather
statistics for a global temporary table in one session and use the statistics only for this session.
Meanwhile, you can continue to maintain a shared version of the statistics that is usable by all other
sessions which have not yet gathered session-private statistics. Session level DML monitoring is also
provided to the global temporary tables. Session-specific statistics for global temporary tables allow
queries to be optimized more accurately based on your own data in a global temporary table.
Session-specific statistics have the following characteristics:
• Dictionary views that track statistics, such as USER_TAB_STATISTICS,
USER_IND_STATISTICS, and USER_TAB_HISTOGRAMS, and so on show global temporary
tables statistics only in the current session.
•
Session-specific statistics reside only in memory and are never persisted to disk. The database
automatically deletes the statistics when the session is exited.
Oracle Database 12c - SQL Tuning New Features - 46
Answer:
1. True
2. job_queue_processes parameter
3. False
4. Partition Maintenance Operations
Oracle Database 12c - SQL Tuning New Features - 47
Histograms represent the data distribution in a table column. Oracle Database creates
histograms on columns that have a data skew, to improve cardinality estimates. Based on
the NDV and the distribution of the data, the database chooses the type of histogram to create.
In Oracle Database 12c, Top Frequency and Hybrid are the two new types of histograms that
have been introduced for columns, which have more than 254 distinct values to improve the
cardinality estimates generated via histograms.
The new histograms have a more accurate frequency of endpoint values, and enable the
optimizer to choose better plans. An endpoint number is a number that uniquely identifies a
bucket. In frequency and hybrid histograms, the endpoint number is the cumulative frequency of
endpoints.
Before Oracle Database 12c, the database created a height-balanced histogram when the
number of distinct values was greater than the requested number of histogram buckets. If no
sampling percentage is specified, Oracle Database 12c no longer creates height-balanced
histograms. If you upgrade the database from Oracle Database11g to 12c, any height-based
histograms created before the upgrade remains in use. If Oracle Database 12c creates new
histograms, and if the sampling percentage is AUTO, they are either frequency or hybrid, but not
height-balanced.
Oracle Database 12c - SQL Tuning New Features - 48
Prior to Oracle Database 12c, a frequency histogram was created only if the number of distinct
values (NDV) was less than 254. Starting with Oracle Database 12c, if a column contains
more than 254 distinct values, and if a small number of distinct values occupy more than 99
percent of the data, the database creates a Top Frequency histogram by using the small
number of extremely popular distinct values. It is built using the same technique that is used
for frequency histograms.
A Top Frequency histogram can produce a better histogram for highly popular values by
ignoring the statistically insignificant unpopular values.
A Top Frequency histogram is indicated by the TOP-FREQUENCY value in the
USER_TAB_COL_STATISTICS.HISTOGRAM column.
In Oracle Database 12c, the database creates Top Frequency histograms when the following
conditions are true:
•
•
•
The data set has more than n distinct values.
The percentage of rows occupied by the top n frequent values is equal to or greater than
threshold p, where p is (1-(1/n))*100.
The estimate_percent parameter is set to AUTO_SAMPLE_SIZE in
the DBMS_STATS statistics gathering procedure.
Oracle Database 12c - SQL Tuning New Features - 49
In this example, you want to generate a top frequency histogram on
the sh.countries.country_subregion_id column. This table has 23 rows.
The first query shows that the country_subregion_id column contains 8 distinct values
that are unevenly distributed.
In the second example code, you gather statistics for sh.countries and
the country_subregion_id column, specifying fewer buckets than distinct values.
In the third example code, you query the histogram information for
the country_subregion_id column. Because you specified 7 buckets for the 8 distinct
values in the sh.countries.country_subregion_id column, n = 7. In
the country_subregion_id column, the top 7 most frequent values occupy 95.6% of the
rows, which exceeds the threshold of 85.7%, generating a top frequency histogram.
In the fourth example code, you query the endpoint number and endpoint value for the
column.
Oracle Database 12c - SQL Tuning New Features - 50
A hybrid histogram combines the characteristics of both height-based histograms and
frequency histograms. It is created if the number of distinct values in a column (NDV) is
greater than 254 values.
The height-based histogram sometimes produces inaccurate estimates for values that are
almost popular. For example, a value that occurs as an endpoint value of only one bucket but
almost occupies two buckets is not considered as popular.
To solve this problem, a hybrid histogram stores the endpoint repeat count value in a new
column endpoint_repeat_count, which is the number of times the endpoint value is
repeated, for each endpoint in the histogram. By using the repeat count, the optimizer can
obtain accurate estimates for almost popular values.
In Oracle Database 12c, the database creates hybrid histograms when the following
conditions are true:
• The sampling percentage is AUTO.
If you specify your own percentage, the database creates frequency or heightbalanced histograms.
The criteria for Top Frequency histograms do not apply.
n is less than NDV, where n is the user-specified number of buckets. If no number is
specified, n defaults to 254.
-
•
•
Oracle Database 12c - SQL Tuning New Features - 51
In the slide example, the oe.inventories.quantity_on_hand column has 237 distinct
values and 1112 rows. The histogram distributes the data into 10 buckets, each with roughly the
same number of rows. In query output, one row corresponds to each bucket in the histogram.
Oracle Database added a special 0 bucket to this histogram because the value in the first
bucket (29) is not the minimum value for the quantity_on_hand column. The 0 bucket holds
the minimum value of 0 for quantity_on_hand.
The graphic represents the hybrid histogram for inventories.quantity_on_hand. The xaxis shows the endpoint (bucket) numbers, which represent the cumulative frequency of the
associated values. The diagram shows that 63 is the value at the right border of the fourth
bucket, and that this value is repeated 13 times in the bucket.
Oracle Database 12c - SQL Tuning New Features - 52
Extended statistics enables the gathering of statistics on a group of columns within a table as
a whole providing the optimizer with more information about any correlation that may exist
between the columns. DBMS_STATS enables you to collect extended statistics, which are
statistics that can improve cardinality estimates when multiple predicates exist on different
columns of a table.
There are two types of extended statistics:
-
Column group statistics are useful when multiple columns from the same
table are used in WHERE clause predicates.
-
Expression statistics are useful when a column is used as part of a complex
expression in WHERE clause predicates.
Starting in this release, Oracle Database automatically determines which column groups are
required in a given workload or SQL tuning set (STS), and then creates the column groups.
Thus, for any given workload, you no longer need to know which columns from each table will
be used together as a workload. By monitoring a workload, Oracle Database 12c records the
necessary column groups. You can then create the column groups by executing
DBMS_STATS.CREATE_EXTENDED_STATS.
The next slides show you how to detect useful column groups for a specific workload.
Oracle Database 12c - SQL Tuning New Features - 53
Column groups are detected based on an SQL Tuning Sets(STS) or by monitoring a
workload. You can use DBMS_STATS.SEED_COL_USAGE and REPORT_COL_USAGE to
determine which column groups are required for a table based on a specified workload.
In an Oracle SQL*Plus session, connect as SYS and run the PL/SQL program shown in the
slide to enable monitoring for 300 seconds (the first argument refers to SQLSET_NAME and
the second to OWNER_NAME).
The new SEED_COL_USAGE procedure iterates over the SQL statements in the specified SQL
Tuning Set (here, all the statements that will be executed in the system for the next 300
seconds), compiles them, and then seeds column usage information in the data dictionary for
the columns that appear in these statements.
Oracle Database 12c - SQL Tuning New Features - 54
Following from the previous example, the database must observe a representative workload
to determine the appropriate column groups. You do not need to run the queries themselves
during the monitoring period. Instead, you can run EXPLAIN PLAN for some longer-running
queries in your workload to ensure that the database is recording column group information
for these queries.
The examples in the slide show the explain plans for two queries on the customer’s table.
Oracle Database 12c - SQL Tuning New Features - 55
The new REPORT_COL_USAGE procedure reports the recorded column group usage
information.
The example in the slide reviews the column group usage information recorded for the
CUSTOMERS table. All three columns appeared in the same WHERE clause, so the report
shows them as a group filter.
In the second query, two columns appeared in the GROUP BY clause, so the report labels
them as GROUP_BY. The sets of columns in the FILTER and GROUP_BY report are candidates
for column groups.
Oracle Database 12c - SQL Tuning New Features - 56
The example creates column groups for the CUSTOMERS table based on the usage
information captured during the monitoring window. After running the code, the database
created two column groups for customers: one column group for the filter predicate and one
group for the group by operation.
Column group statistics will now be automatically maintained every time you gather statistics
on this table.
Oracle Database 12c - SQL Tuning New Features - 57
Answer:
1. Top-frequency and Hybrid
2. True
3. True
Oracle Database 12c - SQL Tuning New Features - 58
Answer:
1. DBMS_STATS.CREATE_EXTENDED_STATS
2. True
Oracle Database 12c - SQL Tuning New Features - 59
In this course, we discussed:
•
Adaptive SQL Plan Management
•
Enhancements to the SQL management base
•
How to use Adaptive plan to improve query performance
•
Enhancements to dynamic statistics
•
Reoptimization for adaptive execution plans
•
How to use SQL plan directives to generate a better plan
•
Statistics Gathering performance improvements
•
How to use new histograms
•
Enhancements to extended statistics and how to detect useful column groups for a specific
workload
You should now be prepared to use the new and enhanced features.
Oracle Database 12c - SQL Tuning New Features - 60
Oracle offers a variety of additional channels from which you can learn more about SQL Tuning
or about any Oracle products.
We at Oracle Education know your time is valuable and limited, and so we thank you for
participating in this self-paced training. We hope this course has met your expectations and
learning objectives, and wish you the best of luck in all of your endeavors.
Please read through each of the tabs of this page to learn about all the Oracle resources
available to you.
Oracle Database 12c - SQL Tuning New Features - 61