Table of Contents
Introduction
1
What This Book Is About
1
Who Should Use This Book?
2
How This Book Is Structured
3
Understanding the Database
Database Structures and Utilities
Performance
Advanced SQL Features
Extensibility
Security
Appendices
3
4
5
5
6
7
7
Conventions
7
Source Code and Updates
8
Setting Up
11
Setting up the SCOTT/TIGER Schema
11
The SQL*PLUS Environment
12
Setting up AUTOTRACE in SQL*PLUS
14
Controlling the Execution Plan Report
Interpreting the Execution Plan
15
15
C Compilers
15
Coding Conventions
16
Other Issues
16
Chapter 1: Developing Successful Oracle Applications
19
My Approach
20
The Black Box Approach
21
How (and how not) to Develop Database Applications
24
Understanding Oracle Architecture
25
Do not run Long Transactions Under MTS
Use Bind Variables
5254TOCcmp3.pdf 1
25
27
3/7/2005 2:06:14 PM
Table of Contents
Understanding Concurrency Control
30
Implementing Locking
Multi-Versioning
30
33
Database Independence?
37
The Impact of Standards
Features and Functions
Solving Problems Simply
Openness
40
43
44
46
How Do I Make it Run Faster?
The DBA-Developer Relationship
47
49
Summary
50
Chapter 2: Architecture
53
The Server
54
The Files
60
Parameter Files
Data Files
Temp Files
Control Files
Redo Log Files
60
62
65
66
66
Online Redo Log
Archived Redo Log
67
69
Files Wrap-Up
70
The Memory Structures
70
PGA and UGA
SGA
70
74
Fixed SGA
Redo Buffer
Block Buffer Cache
Shared Pool
Large Pool
Java Pool
77
77
78
80
82
83
Memory Structures Wrap-Up
84
The Processes
84
Server Processes
85
Dedicated Server versus Shared Server
87
Background Processes
90
Focused Background Processes
Utility Background Processes
90
95
Slave Processes
97
I/O Slaves
Parallel Query Slaves
97
97
Summary
Chapter 3: Locking and Concurrency
What are Locks?
98
101
102
ii
5254TOCcmp3.pdf 2
3/7/2005 2:06:14 PM
Table of Contents
Locking Issues
104
Lost Updates
104
Pessimistic Locking
Optimistic Locking
105
106
Blocking
107
Blocked Inserts
Blocked Updates and Deletes
107
107
Deadlocks
Lock Escalation
108
111
Types of Lock
112
DML Locks
113
TX – (Transaction) Locks
TM – (DML Enqueue) Locks
113
118
DDL Locks
Latches and Internal Locks (Enqueues)
Manual Locking and User-Defined Locks
119
122
122
Manual Locking
Creating your own Locks
123
123
What is Concurrency Control?
123
Transaction Isolation Levels
READ UNCOMMITTED
READ COMMITTED
REPEATABLE READ
124
125
126
128
Getting a Consistent Answer
Lost Update Prevention
128
129
SERIALIZABLE
Read-Only Transactions
130
132
Summary
132
Chapter 4: Transactions
135
Transaction Control Statements
136
Integrity Constraints and Transactions
140
Bad Transaction Habits
143
Distributed Transactions
148
Redo and Rollback
150
Summary
154
Chapter 5: Redo and Rollback
157
Redo
157
What Does a COMMIT Do?
What Does a ROLLBACK Do?
How Much Redo Am I Generating?
Can I Turn Off Redo Log Generation?
Cannot Allocate a New Log?
158
163
164
173
176
iii
5254TOCcmp3.pdf 3
3/7/2005 2:06:14 PM
Table of Contents
Block Cleanout
Log Contention
Temporary Tables and Redo/Rollback
Analyzing Redo
Rollback
177
180
182
184
184
What Generates the Most/Least Undo?
SET TRANSACTION
'ORA-01555: snapshot too old'
185
185
185
Rollback Segments Are in Fact Too Small
You Fetch Across COMMITs
Delayed Block Cleanout
186
190
191
Summary
Chapter 6: Database Tables
195
197
Types of Tables
197
Terminology
199
High Water Mark
FREELISTS
PCTFREE and PCTUSED
199
200
202
Row Migration
Setting PCTFREE and PCTUSED values
202
204
INITIAL, NEXT, and PCTINCREASE
MINEXTENTS and MAXEXTENTS
LOGGING and NOLOGGING
INITRANS and MAXTRANS
208
208
208
209
Heap Organized Table
209
Index Organized Tables
212
Index Organized Tables Wrap-up
224
Index Clustered Tables
224
Index Clustered Tables Wrap-up
231
Hash Cluster Tables
231
Hash Clusters Wrap-up
239
Nested Tables
240
Nested Tables Syntax
Nested Table Storage
240
248
Nested Tables Wrap-up
250
Temporary Tables
251
Temporary Table Wrap-up
Object Tables
258
258
Object Table Wrap-up
Summary
265
265
iv
5254TOCcmp3.pdf 4
3/7/2005 2:06:14 PM
Table of Contents
Chapter 7: Indexes
269
An Overview of Oracle Indexes
270
B*Tree Indexes
271
Reverse Key Indexes
Descending Indexes
When should you use a B*Tree Index?
275
276
277
B*Trees Wrap-up
285
Bitmap Indexes
285
When Should you use a Bitmap Index?
286
Bitmap Indexes Wrap-up
288
Function-Based Indexes
288
Important Implementation Details
Function-Based Index Example
Caveat
288
289
296
Function-Based Index Wrap-up
297
Application Domain Indexes
297
Application Domain Indexes Wrap-up
298
Frequently Asked Questions About Indexes
299
Do Indexes Work On Views?
Indexes and Nulls
Indexes on Foreign Keys
Why isn't my Index Getting Used?
299
299
302
303
Case 1
Case 2
Case 3
Case 4
Case 5
Case 6
303
303
303
303
305
307
Are my Indexes Being Used?
Myth: Space is Never Reused in an Index
Myth: Most Discriminating Elements Should be First
308
308
311
Summary
314
Chapter 8: Import and Export
317
A Quick Example
318
Why You Might Use IMP and EXP
319
Detecting Corruption
Extracting DDL
Cloning Schemas
Transporting Tablespaces
Rebuilding Instances
Copying Data between Platforms
319
319
319
320
320
320
v
5254TOCcmp3.pdf 5
3/7/2005 2:06:14 PM
Table of Contents
How They Work
320
The Options
320
EXP Parameters
IMP Parameters
321
323
Large Exports
325
Use the FILESIZE Parameter
Export Smaller Pieces
Export to an OS Pipe
Export to a Device that does not Support Seeking
325
326
327
328
Subsetting Data
Transporting Data
Getting the DDL
328
329
334
Getting around the limitations with scripts
337
Backup and Recovery
IMP/EXP is not a Reorganization Tool (Any More)
Importing into Different Structures
Direct Path Exports
Caveats and Errors
340
340
341
345
345
Cloning
Using IMP/EXP Across Versions
Where did my Indexes go?
Named versus Default-Named Constraints
National Language Support (NLS) Issues
Tables Spanning Multiple Tablespaces
Summary
Chapter 9: Data Loading
346
353
353
356
359
360
365
367
An Introduction to SQL*LOADER
368
How to ...
374
Load Delimited Data
Load Fixed Format Data
Load Dates
Load Data Using Sequences and Other Functions
Update Existing Rows and Insert New Rows
Load Report-Style Input Data
Load a File into a LONG RAW or LONG Field
Load Data with Embedded Newlines
374
377
379
380
385
387
389
391
Use a Character Other than a Newline
Use the FIX Attribute
Use the VAR Attribute
Use the STR Attribute
Embedded Newlines Wrap-Up
391
392
397
398
399
Unload Data
Load LOBs
399
409
Load a LOB via PL/SQL
Load LOB Data via SQLLDR
409
412
Load VARRAYS/Nested Tables with SQLLDR
Call SQLLDR from a Stored Procedure
Caveats
418
421
426
You Cannot Pick a Rollback Segment to Use
TRUNCATE Appears to Work Differently
SQLLDR Defaults to CHAR(255)
Command Line Overrides Control File
Summary
426
426
426
426
427
vi
5254TOCcmp3.pdf 6
3/7/2005 2:06:14 PM
Table of Contents
Chapter 10: Tuning Strategies and Tools
429
Identifying the Problem
430
My Approach
431
Tuning is a Constant thing
432
Design for Performance
Try many Approaches
Program Defensively
Benchmark
432
433
434
434
Bind Variables and Parsing (Again)
436
Am I Using Bind Variables?
449
Bind Variables and Parsing Wrap-Up
451
SQL_TRACE, TIMED_STATISTICS, and TKPROF
451
Setting Up Tracing
Using and Interpreting TKPROF Output
Using and Interpreting Raw Trace Files
452
454
464
SQL_TRACE, TIMED_STATISTICS, and TKPROF Wrap-Up
474
DBMS_PROFILER
475
Instrumentation
475
StatsPack
477
Setting up StatsPack
StatsPack Wrap-Up
477
494
V$ Tables
494
V$EVENT_NAME
V$FILESTAT and V$TEMPSTAT
V$LOCK
V$MYSTAT
V$OPEN_CURSOR
V$PARAMETER
V$SESSION
V$SESSION_EVENT
V$SESSION_LONGOPS
V$SESSION_WAIT
V$SESSTAT
V$SESS_IO
V$SQL, V$SQLAREA
V$STATNAME
V$SYSSTAT
V$SYSTEM_EVENT
494
495
495
495
496
498
498
500
500
500
500
501
501
501
501
501
Summary
502
Chapter 11: Optimizer Plan Stability
505
An Overview of the Feature
506
vii
5254TOCcmp3.pdf 7
3/7/2005 2:06:14 PM
Table of Contents
Uses of Optimizer Plan Stability
509
A Method to Implement Tuning
A Development Tool
To See the Indexes Used
To See what SQL is Executed by an Application
How Optimizer Plan Stability Works
509
514
515
515
516
OUTLINES and OUTLINE_HINTS
516
The _OUTLINES Views
The _OUTLINE_HINTS Views
516
517
Creating Stored Outlines
518
Privileges Needed for Stored Outlines
Using DDL
Using ALTER SESSION
The OUTLN User
Moving Outlines from Database to Database
Getting Just the Right Outline
Managing Outlines
519
519
520
521
522
522
525
Via DDL
525
ALTER OUTLINE
DROP OUTLINE
525
527
The OUTLN_PKG Package
528
OUTLN_PKG.DROP_UNUSED
OUTLN_PKG.DROP_BY_CAT
OUTLN_PKG.UPDATE_BY_CAT
Caveats
528
529
529
531
Outline Names and Case
ALTER SESSION Issue
DROP USER does not Drop Outlines
‘CURSOR_SHARING = FORCE’ and Outlines
Outlines Use Simple Text Matching
Outlines by Default are in the SYSTEM Tablespace
OR-Expansion
Performance
The Namespace of Outlines is Global
531
533
533
533
534
535
535
536
539
Errors you Might Encounter
540
Summary
542
Chapter 12: Analytic Functions
545
An Example
546
How Analytic Functions Work
549
The Syntax
549
The Function Clause
The Partition Clause
The Order By Clause
The Windowing Clause
Range Windows
Row Windows
Specifying Windows
550
551
551
553
555
558
560
The Functions
562
viii
5254TOCcmp3.pdf 8
3/7/2005 2:06:14 PM
Table of Contents
Examples
566
The TOP-N Query
Pivot Query
Accessing Rows Around Your Current Row
566
576
582
Caveats
586
PL/SQL and Analytic functions
Analytic Functions in the Where Clause
NULLS and Sorting
Performance
586
588
588
590
Summary
591
Chapter 13: Materialized Views
593
A Brief History
594
What you’ll need to run the Examples
595
An Example
595
Uses of Materialized Views
601
How Materialized Views Work
601
Setting Up
Internal Mechanics
602
603
Query Rewrite
603
Making sure your View gets used
605
Constraints
Dimensions
605
609
DBMS_OLAP
618
Estimating Size
Dimension Validation
Recommending Materialized Views
618
620
622
Caveats
624
Materialized Views are Not Designed for OLTP Systems
Query Rewrite Integrity
624
624
Summary
625
Chapter 14: Partitioning
627
The Uses of Partitioning
627
Increased Availability
Reduced Administrative Burden
Enhanced DML and Query Performance
628
629
630
How Partitioning Works
632
Table Partitioning Schemes
Partitioning Indexes
632
637
Local Indexes
Global Indexes
638
645
Summary
655
ix
5254TOCcmp3.pdf 9
3/7/2005 2:06:15 PM
Table of Contents
Chapter 15: Autonomous Transactions
659
An Example
660
Why Use Autonomous Transactions?
662
Auditing that Can Not be Rolled Back
A Method to Avoid a Mutating Table
Performing DDL in Triggers
Writing to the Database
662
665
666
670
Really Strict Auditing
When the Environment Only Allows SELECTs
671
675
To Develop More Modular Code
How They Work
678
679
Transactional Control
Scope
679
681
Packaged Variables
Session Settings/Parameters
Database Changes
Locks
681
682
682
685
Ending an Autonomous Transaction
Savepoints
Caveats
686
687
689
No Distributed Transactions
PL/SQL Only
The Entire Transaction Rolls Back
Transaction-Level Temporary Tables
Mutating Tables
689
689
689
691
693
Errors You Might Encounter
695
Summary
696
Chapter 16: Dynamic SQL
699
Dynamic SQL versus Static SQL
700
Why Use Dynamic SQL?
702
How to Use Dynamic SQL
703
DBMS_SQL
Native Dynamic SQL
DBMS_SQL versus Native Dynamic SQL
703
709
713
Bind Variables
Number of Outputs Unknown at Compile-Time
Executing the Same Statement Dynamically Many Times
Caveats
714
718
725
734
It Breaks the Dependency Chain
The Code is More Fragile
It is Harder to Tune
Summary
734
735
735
736
x
5254TOCcmp3.pdf 10
3/7/2005 2:06:15 PM
Table of Contents
Chapter 17: interMedia
739
A Brief History
739
Uses of interMedia Text
740
Searching for Text
Managing a Variety of Documents
Indexing Text from Many Data Sources
It’s an Oracle Database, After All
Generating Themes
Searching XML Applications
741
743
743
746
747
749
How interMedia Text Works
749
interMedia Text Indexing
About ABOUT
Section Searching
753
756
756
Caveats
762
It is NOT Document Management
Index Synchronization
Indexing Information Outside the Database
Document Services
The Catalog Index
762
763
764
764
765
Errors You May Encounter
767
Index Out of Date
External Procedure Errors
767
767
The Road Ahead
768
Summary
769
Chapter 18: C-Based External Procedures
771
When Are They Used?
772
How Are They Implemented?
773
Configuring Your Server
775
Verify the extproc Program
Verify the Database Environment
Verify the Listener
777
777
779
The First Test
779
Compile extproc.c Code
Set Up the SCOTT/TIGER Account
Create the demolib LIBRARY
Installing and Running
780
780
781
782
Our First External Procedure
783
The Wrapper
The C Code
Building the extproc
Installing and Running
784
794
816
819
xi
5254TOCcmp3.pdf 11
3/7/2005 2:06:15 PM
Table of Contents
LOB to File External Procedure (LOB_IO)
821
The LOB_IO Call Specification
The LOB_IO Pro*C Code
Building the extproc
Installing and Using LOB_IO
821
823
827
829
Errors You May Encounter
833
Summary
840
Chapter 19: Java Stored Procedures
843
Why Use Java Stored Procedures?
843
How They Work
845
Passing Data
Useful Examples
849
857
Getting a Directory Listing
Running an OS Command
Getting Time Down to the Milliseconds
Possible Errors
858
860
863
863
ORA-29549 Java Session State Cleared
Permissions Errors
ORA-29531 no method X in class Y
Summary
Chapter 20: Using Object Relational Features
863
864
864
865
867
Reasons For Using These Features
868
How Object Relational Features Work
868
Adding Data Types to your System
869
Adding Data Types Wrap-Up
882
Using Types to Extend PL/SQL
882
Creating a New PL/SQL Data Type
Unique Uses for Collections
883
892
SELECT * from PLSQL_FUNCTION
Bulk Fetching into RECORD Types
Inserting Using a RECORD Type
892
895
896
Using Types to Extend PL/SQL Wrap-Up
897
Object Relational Views
897
The Types
The O-R View
Summary
898
898
910
xii
5254TOCcmp3.pdf 12
3/7/2005 2:06:15 PM
Table of Contents
Chapter 21: Fine Grained Access Control
913
An Example
914
Why Use this Feature?
914
Ease of Maintenance
Performed in the Server
915
915
Easier Application Development
Evolutionary Application Development
916
916
Avoids Shared User Accounts
Supports Shared User Accounts
Hosting an Application as an ASP
916
917
917
How it Works
918
Example 1: Implementing a Security Policy
Example 2: Using Application Contexts
919
923
Caveats
940
Referential Integrity
941
The Covert Channel
Deleting Rows
Updating Rows
941
942
943
Cursor Caching
Export/Import
945
951
Export Issues
Import Issues
952
954
Debugging
955
Errors You Might Encounter
955
Summary
959
Chapter 22: n-Tier Authentication
963
Why Use n-Tier Authentication?
964
The Mechanics of n-Tier Authentication
966
Granting the Privilege
975
Auditing Proxy Accounts
976
Caveats
977
Summary
978
Chapter 23: Invoker and Definer Rights
981
An Example
982
When to Use Invoker Rights
984
Developing Generic Utilities
Data Dictionary Applications
Generic Object Types
Implementing your own Access Control
985
988
991
991
xiii
5254TOCcmp3.pdf 13
3/7/2005 2:06:15 PM
Table of Contents
When to Use Definer Rights
994
Performance and Scalability
Security
994
995
How they Work
Definer Rights
Invoker Rights
Resolving References and Conveying Privileges
Compiling an Invoker Rights Procedure
Using Template Objects
Caveats
Invoker Rights and Shared Pool Utilization
Performance
Code must be more Robust in Handling Errors
Side Effects of Using SELECT *
Beware of the 'Hidden' Columns
Java and Invoker Rights
995
995
1000
1001
1006
1006
1010
1010
1013
1015
1017
1018
1020
Errors You Might Encounter
1025
Summary
1025
Appendix A: Necessary Supplied Packages
1027
Why Use the Supplied Packages?
1028
About The Supplied Packages
1028
DBMS_ALERT and DBMS_PIPE
1031
Why You Might Use Them
Set Up
DBMS_ALERT
Concurrent Signals by More than One Session
Repeated Calls to Signal by a Session
Many Calls to Signal by Many Sessions before a Wait Routine is Called
1032
1032
1032
1034
1036
1037
Summary
1037
DBMS_PIPE
1038
Pipe Servers versus External Routines
Online Example
Summary
DBMS_APPLICATION_INFO
1040
1041
1041
1042
Using the Client Info
1043
Using V$SESSION_LONGOPS
1045
Summary
1049
xiv
5254TOCcmp3.pdf 14
3/7/2005 2:06:15 PM
Table of Contents
DBMS_JAVA
1050
LONGNAME and SHORTNAME
1050
Setting Compiler Options
1051
SET_OUTPUT
1055
loadjava and dropjava
1055
Permission Procedures
1056
Summary
1058
DBMS_JOB
1059
Running a Job Once
1063
Ongoing Jobs
1066
Custom Scheduling
1068
Monitoring the Jobs and Finding the Errors
1070
Summary
1072
DBMS_LOB
1073
How do I Load LOBs?
1074
substr
1074
SELECT FOR UPDATE and Java
1075
Conversions
1077
From BLOB to VARCHAR2 and Back Again
Converting From LONG/LONG RAW to a LOB
Performing a Mass One-Time Conversion Illustration
Performing an 'on the fly' Conversion
1077
1081
1083
1086
How to Write a BLOB/CLOB to Disk
1088
Displaying a LOB on the Web Using PL/SQL
1089
Summary
1091
DBMS_LOCK
1092
DBMS_LOGMNR
1097
Overview
1099
Step 1: Creating the Data Dictionary
Step 2: Using Log Miner
1099
1102
Options and Usage
1107
xv
5254TOCcmp3.pdf 15
3/7/2005 2:06:15 PM
Table of Contents
Using Log Miner to Find Out When...
1110
PGA Usage
1111
Log Miner Limits
1113
Oracle Object Types
Chained or Migrated Rows
Other limits
1113
1116
1118
V$LOGMNR_CONTENTS
1119
Summary
1122
DBMS_OBFUSCATION_TOOLKIT
1123
The Wrapper
1125
Caveats
1139
Key Management
1140
The Client Application Manages and Stores Keys
Store the Keys in the Same Database
Store the Keys in the File System with the Database
Summary
DBMS_OUTPUT
1141
1141
1142
1143
1144
How DBMS_OUTPUT Works
1145
DBMS_OUTPUT and Other Environments
1149
Getting Around the Limits
1153
Using A Small Wrapper Function or Another Package
Creating DBMS_OUTPUT Functionality
Summary
DBMS_PROFILER
1153
1154
1160
1161
Caveats
1171
Summary
1171
DBMS_UTILITY
1172
COMPILE_SCHEMA
1172
ANALYZE_SCHEMA
1176
ANALYZE_SCHEMA with a Changing Schema
ANALYZE_SCHEMA does not Analyze Everything
ANALYZE_DATABASE
1178
1178
1179
xvi
5254TOCcmp3.pdf 16
3/7/2005 2:06:15 PM
Table of Contents
FORMAT_ERROR_STACK
FORMAT_CALL_STACK
GET_TIME
GET_PARAMETER_VALUE
NAME_RESOLVE
NAME_TOKENIZE
COMMA_TO_TABLE, TABLE_TO_COMMA
DB_VERSION and PORT_STRING
GET_HASH_VALUE
Summary
1179
1181
1184
1185
1186
1188
1191
1193
1193
1198
UTL_FILE
1199
The UTL_FILE_DIR init.ora parameter
Accessing Mapped Windows Drives
Handling Exceptions
Dumping a Web Page to Disk
1023 Byte Limit
Reading A Directory
Summary
1200
1201
1202
1203
1205
1205
1207
UTL_HTTP
1208
UTL_HTTP Functionality
Adding SSL to UTL_HTTP
Really Using UTL_HTTP
A Better UTL_HTTP
Summary
1209
1210
1217
1219
1228
UTL_RAW
1229
UTL_SMTP and Sending Mail
1231
UTL_SMTP – a larger example
Loading and using the JavaMail API
Summary
1231
1236
1243
xvii
5254TOCcmp3.pdf 17
3/7/2005 2:06:15 PM
Table of Contents
UTL_TCP
1244
The SocketType
1245
Summary
1256
Index
1265
xviii
5254TOCcmp3.pdf 18
3/7/2005 2:06:15 PM