A Day in the Life of an Application
Performance Engineer
Keith Lyon - Shunra Software
keith.lyon@shunra.com
Introduction
• Who am I?
• What’s my background?
• How did I get introduced to Shunra?
Performance Testing Ecosystem Overview
A day in the life…
•
•
•
•
•
My application is so slow…
We take breaks between button clicks…
What do you mean, did we “performance test”?
Non-functional requirements?
Network \ application profiling?
Why do we performance test?
• Determine speed or effectiveness of a
computer, network, software program or device.
• Quantitative tests done in a lab, such as:
– measuring the response time or the;
– number of MIPS (millions of instructions per
second) at which a system functions.
• Qualitative attributes such as:
– reliability, scalability and interoperability may
also be evaluated.
What are “non-functional” requirements?
• A requirement that specifies criteria that can be
used to judge the operation of a system, rather than
specific behaviors. For example:
–
–
–
–
Transactions per second;
CPU Utilization;
Memory Utilization;
Maximum Throughput.
Describe the “Performance” of a system
• Response Time: Typically defined as the time
taken to complete a single business transaction.
It is usually expressed in seconds.
• Throughput: Can be defined as the total number
of business transactions completed by the
application in unit time (per second or per hour).
• Resource Utilization: Different resources factored
in, such as processor, disk (i/o controller),
memory etc.
Performance Test Plan: What To Test
• 80/20 Rule
• Classify transactions
• Critical business processes
Performance Test Environment
• Capacity
• Scalability
• Node Deployment
Performance Test: Data Preparation
• Initial Data Setup
• Ongoing Data Retention
• Test Data
Value In Reporting
CPU Utilization vs. No. of users
This graph tells us whether the CPU utilization
increases with increasing users. It also tells us
about bottlenecks as well as what is the maximum
number of users that can be supported.
Throughput vs. number of users
This graph tells us if throughput increases with
increasing number of users. This graph should
have a similar shape as CPU Utilization graph.
Performance Test: Best Practices
• Reusable Automated Core Scripts;
• Database backups after crucial runs;
• Designing scenarios to achieve right mix of
transactions;
• Do not tune, if you don't have to.
Real Life Example: The Project…
•
•
•
•
Custom Developed Application;
Hosted by Third Party;
Rolling Out to 250 International Sites;
Time to Performance Test = 1 Month.
LAN - Baseline
1 baseline
user
North America Large
Network Model
10 Mbps
40 users
North America Medium
1.5 Mbps
NA WAN 1
15 users
NA WAN 2
256 Kbps
North America Small
5 users
NA WAN 3
EMEA - Large
2 Mbps
10 users
EMEA WAN 1
45 Mbps
Data Center
768 Kbps
EMEA – Medium
EMEA WAN 2
5 users
128 Kbps
EMEA - Small
EMEA WAN 3
3 users
2 Mbps APAC
APAC - Large
APAC WAN 1
10 users
768 Kbps APAC
APAC WAN 2
APAC – Medium
5 users
128 Kbps APAC
APAC - Small
3 users
APAC WAN 3
Shunra Suite of Products and Hardware
Before
After
Time-consuming
coordination across
remote sites
Global testing conducted
locally
Fixed, permanent testing
network
Flexible testing from any
location; “what-if testing” of
latency, loss, bandwidth
Maintaining remote load
generators
Emulating WAN in lab with
shared infrastructure
Testing only during
network maintenance
windows
Testing is independent of
production network
Shunra Professional Services
Shunra VE Desktop for HP Loadrunner
TRY and FLY
Enter to Win Free Round-trip Airfare to HPSU Barcelona
Test Drive Shunra for HP™ - 30 Day Trial
Contact James Newton @ 267-519-5104 or james.newton@shunra.com
Winner announced 10/22. Must register by 9/22.
WWW.VIVIT-WORLDWIDE.ORG