Robin Moffatt, WM Morrisons plc
http://rnm1978.wordpress.com

Oracle BI specialist at Morrisons plc

Big IT development programme at its early
stages implementing OBIEE, OBIA, ORDM, all
on Oracle 11g & HP-UX

A Performance Tuning Methodology

OBIEE techie stuff

Learn from my mistakes!

Response times
 Report
 ETL batch
 OLTP transaction

System impact
 Resource usage
 Scalability

Check that your system performs
 Are the users going to be happy?

Baseline
 How fast is fast?
▪ How slow is slow?

Validate system design
 Do it right, first time

Capacity planning

It’s never too late
 “You’ll never catch all your problems in pre-
production testing. That’s why you need a reliable
and efficient method for solving the problems that
leak through your pre-production testing
processes.”
— Cary Millsap - Thinking Clearly About Performance

Because it makes you better at your job
 “At the very least, your performance test plan will
make you a more competent diagnostician (and
clearer thinker) when it comes time to fix the
performance problems that will inevitably occur
during production operation.”
— Cary Millsap - Thinking Clearly About Performance

Quantifying response times
 System impact



User expectations
Problem diagnosis
Design validation
Define
Do it right
Timebox!
Evaluate design /
config options
Redefine
test
Measure
Don’t “fudge it”
Analyse
Do more testing
Review
Do more testing
Implement
Define
Measure
Analyse
Review
Implement

Define – what are you going to test
•
•
•
•
•

Aim of the test
Scope
Assumptions
Specifics
Data, environment, etc
•E.g. :
•Check that the system performs
•Baseline performance
•Prove system capacity
•Validate system design
Build – how are you going to test it
 OBIEE specific
Define
Measure
Analyse
Review
Implement
More
Fewer
components
components
= more
= easier
complex
to manage
= more=variables
more precise
= larger
= more
margin
efficient
of error
Define
Measure
Analyse
Review
Implement
Report /
Dashboard
Rendered
report
Presentation
Services
Data set
Logical SQL
BI Server
Physical SQL
statement(s)
Data set(s)
Excludes App/Web server & presentation
services plug-in
Database
Define
Measure
Analyse
Review
Implement
User &
Stopwatch
Load Testing tool
(eg. LoadRunner,
OATS)
Presentation
Services
nqcmd
LSQL
BI Server
Physical
SQL
SQL Client
Physical
SQL
Database
Define
Measure
Analyse
Review
Implement
nqcmd
LSQL
BI Server
Physical
SQL
Database
Define
Measure
Analyse
Review
Implement
Usage Tracking
or NQQuery.log
Test
script
Logical
SQL
Logical
SQL
Logical
SQL
nqcmd
BI
Server
Data
Define
Measure
Analyse
Review
Test
script
Test
script
nqcmd
Test
script
nqcmd
nqcmd
Test
script
Logical
SQL
Master test script
Implement
nqcmd
BI
Server
Data
Define
Measure
Analyse
Review
Implement

Simulates user interaction – HTTP traffic

Powerful, but can be difficult to set up
 Ajax complicates things

Do you really need to use it?

Tools
 Fiddler2
 FireBug

Reference:
 My Oracle Support – Doc ID 496417.1
 http://rnm1978.wordpress.com/category/loadrunner
Define
Measure
Analyse
Review
Implement



Be very clear what the aim of your test is
You probably need to define multiple tests
Different points on the OBIEE stack to
interface
 Pick the most appropriate one

Write everything down!
Define
Measure
Analyse
Review
Implement
Define
Measure
Analyse
Review
Implement
Enterprise
Manager
(Oracle)
Apache log
Web Server
OAS log
App Server
Analytics
log
Presentation
Services plug-in
PerfMon
(Windows)
Server metrics
e.g. : IO, CPU, Memory
sawserver.log
Presentation
Services
PerfMon
(windows
only)
NQServer.log
Usage
Tracking
NQQuery.log
Oracle OS
Watcher
(unix)
Enterprise
Manager
ASH, AWR,
SQL Monitor
BI Server
systems
management
Database
jConsole etc
Enterprise
Manager
BI Management Pack
Presentation
services
Define
Measure
Analyse
Review
Implement
Define
Measure
Analyse
Review
Implement

Lots of different ways to measure

Build measurement into your test plan
 Automate where possible
▪ Easier
▪ Less error
Define
Measure
Analyse
Review
Implement
Define
Measure
Analyse
Review
Implement
Define
Measure
Analyse
Review
Implement
Define
Measure
Analyse
Review
Implement
1
Average
(mean)
1
3.4
Response time
91
32
2
102
23
13
29
3
10
50th percentile
(Median)
2
90th percentile
9.1
Define
Measure
Analyse
Review
Implement
Dashboard
Requests
Logical SQL
• ORA_HASH(QUERY_TEXT)
Physical SQL
• SQL IDs
Execution plan • Execution plan hash id
Define
Measure
Analyse
Review
Implement
S_NQ_ACCT
START_TS
ROW_COUNT
TOTAL_TIME_SEC
NUM_DB_QUERY
QUERY_TEXT
QUERY_SRC_CD
SAW_SRC_PATH
SAW_DASHBOARD
OBIEE_REPLAY_STATEMENTS
qt_ora_hash
query_text
saw_path
dashboard
OBIEE_REPLAY_STATS
testid
testenv
qt_ora_hash
start_ts
response_time
row_count
db_query_cnt
Define
Do it right
Measure
Evaluate design /
config options
Analyse
Timebox!
Review
Implement
Don’t “fudge it”
Define
Measure
Continue
testing
Redefine
test
Analyse
Review
Implement
Implement
Define
Measure
Analyse
Review
Implement
Define
Measure
Analyse
Review
Implement

You won’t get your testing right first time
 There’s no shame in that
 Don’t cook the books
▪ Better to redefine your test than invalidate its results

Stick to the methodology
 Don’t move the goalposts
 Very tempting to pick off the “low-hanging fruit”
▪ If you do, make sure you don’t get indigestion…

Timebox

Test your implementation!
Define
Do it right
Don’t “fudge it”
Measure
Evaluate design /
config options
Redefine
test
Analyse
Do more testing
Review
Do more testing
Implement
robin@moffatt.me
·
http://rnm1978.wordpress.com
·
http://twitter.com/rnm1978