Oracle Database Applications
Database Connectivity
1
Outline
•
•
•
•
•
Motivation
Architecture
7 steps
Sample program
Metadata
2
Motivation
• Most popular form of database system
is the relational database system.
• Structured Query Language (SQL) is
used among relational databases to
construct queries.
• Applications need to query databases
3
Simple Database Application
A
P
P
L
I
C
A
T
I
O
N
DBMS
DB
Oracle
4
Multi-Databases
A
P
P
L
I
C
A
T
I
O
N
DBMS 1
DB
DBMS 2
DB
DBMS 3
DB
5
Standard Access to DB
A
P
P
L
I
C
A
T
I
O
N
D
R
I
V
E
R
M
G
R
DBMS
Driver 1
DBMS 1
DB
DBMS
Driver 2
DBMS 2
DB
DBMS 3
DB
DBMS
Driver 3
6
ODBC Architecture
Application
Class1
Class2
ODBC
Driver Manager
DriverType1
DataSource1
DriverType2
DriverType3
DataSource2
DataSource3
7
Open Database Connectivity
(ODBC) Standard
• ODBC standard is an interface by which application
programs can access and process SQL databases in a
DBMS-independent manner. It contains:
– A Data Source that is the database, its associated DBMS,
operating system and network platform
– A DBMS Driver that is supplied by the DBMS vendor (Oracle)
or independent software companies
– A Driver Manager that is supplied by the vendor of the O/S
platform (e.g., Windows/UNIX/Mac) where the application is
running
– E.g Control Panel->Administrative Tools->Data Sources (ODBC)
in Windows. You can use ODBC to access MS Access or even
Excel documents using the corresponding drivers.
8
ODBC Interface
• System-independent interface to database
environment
– requires an ODBC driver to be provided for each
database system from which you want to manipulate
data.
• The ODBC driver manager bridges the
differences between
– The ODBC DBMS-independent interface and the
DBMS-dependent interface of the DBMS driver
9
An Example
Application
Oracle Driver
DriverManager
DB2 driver
SQLServer driver
10
Java Support for Database
Connectivity
• When applications written in Java want to access
data sources, they use classes and associated
methods provided by Java DBC (JDBC) API.
• JDBC is specified an “interface.”
• An interface in Java can have many
“implementations.”
– it provides a convenient way to realize many “drivers”
• JDBC can be an interface to an ODBC driver
manager.
• Also it provides a JDBC API as a standard way to
directly connect to common relational databases.
11
Application in Java
Application in
Java
Oracle Driver
DriverManager
DB2 driver
SQLServer driver
12
Application in Java
Jdbc API
Application in
Java
Oracle Driver
13
Java Support for SQL
• Java supports embedded SQL.
– Embedded SQL in Java (SQLJ is one version of it) provides a
standardized way for Java programs to interact with multiple
databases, but at a higher level than the existing JDBC
API.
– Embedded SQL allows connecting to a database by including
SQL code right in the program.
– An Embedded SQL preprocessor converts the SQL statements to Java
code at pre-compile time. The preprocessor generates code that
includes the driver functionality for direct connection to the DBMS via
JDBC.
• java.sql package and an extensive exception
hierarchy.
14
Data Source and Driver
• Data source: database created using any of
the common database applications.
• Your system should have the driver for the
database you will be using.
– E.g., MS SQL Server on a Windows system.
• There are a number of JDBC drivers available.
http://industry.java.sun.com/products/jdbc/drivers
For oracle: http://jdbc.oracle.org/
15
JDBC Components
• Driver Manager: Loads database drivers, and
manages the connection between application & driver.
• Driver: Translates API calls to operations for a
specific data source.
• Connection: A session between an application and a
driver.
• Statement: A SQL statement to perform a query or
an update operation.
• Metadata: Information about the returned data,
driver and the database.
• Result Set : Logical set of columns and rows returned
by executing a statement.
16
JDBC Classes
• Java supports DB facilities by providing
classes and interfaces for its components
• DriverManager class
• Connection interface (abstract class)
• Statement interface (to be instantiated with
values from the actual SQL statement)
• ResultSet interface
17
java.sql
• JDBC is implemented via classes in the
java.sql package
– Supports SQL-2 entry level
• Define objects for:
– Remote connection to DB
– Executing query
• 8 interfaces to define objects:
– Statement, CallableStatement, PreparedStatement,
DatabaseMetaData, ResultSetMetaData, ResultSet,
Connection, Driver
18
Seven Steps
•
•
•
•
•
•
•
Load the driver
Define the Connection URL
Establish the Connection
Create a Statement object
Execute a query
Process the result
Close the connection
DriverManager
Driver
Connection
Statement
ResultSet
19
Loading the Driver
• Registering the driver directly
automatically:
Class.forName(“org.oracle.Driver");
• Calling Class.forName, which automatically
– creates an instance of the driver
– registers the driver with the DriverManager
20
Identifying Data Sources
• Gives the required information for
making the connection to the
database
• Specified using the URL format.
<scheme>: <sub_scheme>:<scheme-specific-part>
• Example:
jdbc:oracle://foo.itu.edu/mydatabase
jdbc:oracle://localhost/testdb
21
Another Option
• Create an instance of the driver and
register it with the Driver Manager:
Driver driver = (Driver)
Class.forName("org.oracle.Driver").newInstance();
DriverManager.registerDriver(driver);
22
Connection
• A Connection represents a session with
a specific database
• Within the context of a Connection,
SQL statements are executed and
results are returned
23
Connections
• There can be multiple connections to
a database
• A connection provides “metadata”,
i.e., information about the database,
tables, and fields
• Connection object has methods to
deal with transactions
24
Creating a Connection
• Use getConnection on the Driver
Connection getConnection
(String url,
String user,
String password)
• Connects to given JDBC URL with given
user name and password
• Throws java.sql.SQLException
• returns a Connection object
25
Creating a Connection
Connection connection =
DriverManager.getConnection("jdbc:or
acle://localhost/moviedb", "testuser",
"mypassword");
26
Statements
• Statement createStatement()
– returns a new Statement object
– Used for general queries
• PreparedStatement prepareStatement(String sql)
– returns a new PreparedStatement object
– For a statement called multiple times with different values
(precompiled to reduce parsing time)
• CallableStatement prepareCall(String sql)
– returns a new CallableStatement object
– For stored procedures
27
Statements
• A Statement object is used for
executing a static SQL statement and
obtaining the results produced by it
• executeQuery is used for statements
that return an output result
• executeUpdate is used for statements
that need not return an output
28
Executing Queries and Updates
• ResultSet executeQuery(String)
– Execute a SQL statement that returns a
single ResultSet
• int executeUpdate(String)
– Execute a SQL INSERT, UPDATE or
DELETE statement
– Used for CREATE TABLE, DROP TABLE and
ALTER TABLE
– Returns the number of rows changed
29
Timeout
• Use setQueryTimeOut to set a timeout
for the driver to wait for a statement
to be completed
• If the operation is not completed in the
given time, an SQLException is thrown
• What is it good for?
30
Cursor
• What is the result of a query?
• How can a database send the result of a
query through communication lines?
• The answer: using a cursor
31
Result Set
• A ResultSet provides access to a table of data
generated by executing a Statement
• Only one ResultSet per Statement can be
open at once
• The table rows are retrieved in sequence
• A ResultSet maintains a cursor pointing to its
current row of data
• The 'next' method moves the cursor to the
next row
– you can’t rewind
32
Working with ResultSet
• boolean next()
– activates the next row
– the first call to next() activates the first row
– returns false if there are no more rows
• void close()
– disposes of the ResultSet
– automatically called by most Statement methods
33
Getting Values from Rows
• Type getType(int columnIndex)
– returns the given field as the given type
– E.g., int getInt(5); string getString(3);
– fields indexed starting at 1 (not 0)
• Type getType(String columnName)
– same, but uses name of field
– less efficient
• int findColumn(String columnName)
– looks up column index given column name
34
isNull
• In SQL, NULL means the field is empty
• Not the same as 0 or “”
• In JDBC, you must explicitly ask if a
field is null by calling
ResultSet.isNull(column)
35
Mapping Java Types to SQL Types
SQL type
Java Type
CHAR, VARCHAR, LONGVARCHAR
String
NUMERIC, DECIMAL
java.math.BigDecimal
BIT
boolean
TINYINT
byte
SMALLINT
short
INTEGER
int
BIGINT
long
REAL
float
FLOAT, DOUBLE
double
BINARY, VARBINARY, LONGVARBINARY
byte[]
DATE
java.sql.Date
TIME
java.sql.Time
TIMESTAMP
java.sql.Timestamp
36
Database Time
• Times in SQL are nonstandard
• Java defines three classes to help
• java.sql.Date
– year, month, day
• java.sql.Time
– hours, minutes, seconds
• java.sql.Timestamp
– year, month, day, hours, minutes, seconds,
nanoseconds
– usually use this one
37
Optimized Statements
• Prepared Statements
– SQL calls that you make again and again
– allows driver to optimize (compile) queries
– created with Connection.prepareStatement()
• Stored Procedures
– written in DB-specific language
– stored inside database
– accessed with Connection.prepareCall()
38
Prepared Statement Example
PreparedStatement updateSales;
String updateString = "update COFFEES " +
"set SALES = ? where COF_NAME like ?";
updateSales = con.prepareStatement(updateString);
int [] salesForWeek = {175, 150, 60, 155, 90};
String [] coffees = {"Colombian", "French_Roast",
"Espresso","Colombian_Decaf","French_Roast_Decaf"};
int len = coffees.length;
for(int i = 0; i < len; i++) {
updateSales.setInt(1, salesForWeek[i]);
updateSales.setString(2, coffees[i]);
updateSales.executeUpdate();
}
39
JDBC Class Diagram
40
Metadata
• Connection:
– DatabaseMetaData getMetaData()
• ResultSet:
– ResultSetMetaData getMetaData()
41
ResultSetMetaData
•
•
•
•
•
What's the number of columns in the ResultSet?
What's a column's name?
What's a column's SQL type?
What's the column's normal max width in chars?
What's the suggested column title for use in
printouts and displays?
• What's a column's number of decimal digits?
• Does a column's case matter?
• and so on...
42
DatabaseMetaData
•
•
•
•
What tables are available?
What's our user name as known to the database?
Is the database in read-only mode?
If table correlation names are supported
(association of a column with the table it comes
from, when multiple columns of the same name
appear in the same query - multi-table queries) ,
are they restricted to be different from the
names of the tables?
• and so on…
43
Useful Methods of Metadata
•
•
•
•
•
getColumnCount
getColumnDisplaySize
getColumnName
getColumnType
isNullabale
Imagine the case where you want to print
the result
44
Transaction Management
• A transaction: a sequence of SQL statements
• Transactions are not explicitly opened and
closed
• Instead, the connection has a state called
AutoCommit mode
• if AutoCommit is true, then every statement is
automatically committed
• default case: true
45
AutoCommit
Connection.setAutoCommit(boolean)
• if AutoCommit is false, then every
statement is added to an ongoing
transaction
• you must explicitly commit or rollback
the transaction using
Connection.commit() and
Connection.rollback()
46
Connection Manager
• For a large threaded database server, create a
Connection Manager object
• It is responsible for maintaining a certain number of
open connections to the database
• When your applications need a connection, they ask for
one from the CM’s pool
• Why? Because opening and closing connections takes a
long time
• Warning: the CM should always setAutoCommit(false)
when a connection is returned
47
More info
• Go through the tutorial at:
http://developer.java.sun.com/developer/onlineTr
aining/Database/JDBC20Intro/
• http://java.sun.com/products/jdbc/learning.html
48
Optimizing JDBC
49
Goal
–Illustrate techniques for optimizing JDBC APIbased calls from the Java platform
–Design better JDBC implementations
–Recognize potential performance bottlenecks
50
Outline
1) Why optimize?
2) Basic API techniques.
3) Design Strategies.
4) Advanced Driver Tuning methods.
51
Why Optimize?
• On average, a web request performs
4 database queries.
• Experience has shown that database
calls are typical performance
bottleneck.
• Bad JDBC can overwhelm the
database.
52
JDBC API
Most Versatile
• SQL: “SELECT * FROM TABLE”
• java.sql.PreparedStatement
• java.sql.CallableStatement
• Cache data on client.
Most Optimized
53
JDBC API
SQL Statements
• Most flexible
• Least reliable
• Must be recompiled in database for
each use
54
JDBC API
PreparedStatement
• Represents a precompiled SQL statement
• Can be used to efficiently execute
statement multiple times
• Somewhat flexible – can create new ones
as needed
55
JDBC API
PreparedStatement pstmt =
con.prepareStatement("UPDATE EMPLOYEES
SET SALARY = ? WHERE ID = ?");
pstmt.setBigDecimal(1, 153833.00);
pstmt.setInt(2, 110592);
pstmt.execute();
56
JDBC API
java.sql.CallableStatement
• Used to execute SQL stored procedures.
• Same syntax as PreparedStatement.
• Least flexible.
• Most optimized DB call.
57
JDBC API
Cache
• Keep data within client to reduce
the number of round-trips to the
database.
• Lesson: The less JDBC the better.
58
Basic Design Techniques
Use Database Connection Pool
• Don’t use DriverManager.getConnection()
often. JDBC connections can take 0.5 to 2
seconds to create.
• Create Pool of Connections and reuse them.
• Necessity for any production system.
59
Basic Design Techniques
Use multi-threading with Connection
Pooling to address network latency:
• Threads can issue queries over separate
database connections.
• This improves performance to a point.
60
Basic Design Techniques
Single-batch Transactions
Collect set of operations and submit
transaction in one statement:
BEGIN TRANSACTION
UPDATE TABLE1...
INSERT INTO TABLE2…
DELETE TABLE3
COMMIT
61
Basic Design Techniques
Single-batch Transactions
• DB obtains necessary locks on rows and
tables, uses and releases them in one step
• Depending on transaction type, separate
statements and commits can result in more
DB calls and hold DB locks longer
62
Basic Design Techniques
Don’t have transaction span user input
• Application sends BEGIN TRAN and
SQL, locking rows or tables for update
• Application waits for user to press key
before committing transaction
63
Basic Design Techniques
Smart Queries
•
•
•
•
Make queries as specific as possible
Put more logic into SQL statements
DB are designed to use SQL efficiently
Proper use of SQL can avoid
performance problems
64
Basic Design Techniques
Smart Query Ex: get employees in ENG dept
Instead of:
SELECT * FROM employees;
SELECT * FROM dept;
(and joining on Java application side)
Use database join:
SELECT employees.* FROM employees E,
dept D WHERE E.DEPTNO = D.DEPTNO
AND D.DEPTTYPE = ‘ENG’;
65
Basic Design Techniques
Smart Queries
• Minimize ResultSet before crossing
network
• Many performance problems come from
moving raw data around needlessly
66
Basic Design Techniques
•
•
•
•
Smart Query Guidelines
Use DB for filtering
Use Java for business logic
DB does filtering very well
DB business logic is poor
– At least very inconsistent between
database vendors.
67
Basic Design Techniques
Keep operational data set small as possible
• Move non-current data to other
tables and do joins for rarer
historical queries
• Otherwise, index and cluster so
frequently used data is logically and
physically localized
68
Advanced Driver Tuning
• Special options for each JDBC driver
• No common standard
• Improve performance by reducing round
trips to the database.
• Example: Oracle driver performance
extensions
69
Advanced Driver Tuning
Oracle Performance Extensions
1) Row Prefetch
2) Batch Updates
70
Advanced Driver Tuning
1. Row Prefetch
• Use client-side buffers
• Replace round trips by local manipulation of
rows returned by query
• Use OracleStatement.setRowPrefetch()
71
Advanced Driver Tuning
2. Batch Updates
• Reverse Prefetch
• Does for data going to DB what prefetch
does for data coming from it
• OraclePreparedStatement.setExecuteBatch
72
Advanced Driver Tuning
2. Batch Updates
• Standard JDBC makes a trip to DB for
each PreparedStatement.executeUpdate
• Batching: When number of queued
requests reaches batch size, sends them
to DB
73
Summary
Optimization Stages
1) Leverage the strengths of the DB
2) Use the full range of java.sql API
3) Design for Performance – Connection
Pools, Multi-Threading, etc.
4) Implement driver performance extensions
74