Introduction to Mobile
Performance Testing
Wilson Mar, Angel Tran, Shlomi Salma
HOL330 in Sands 305, June 13, 2013
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Why performance test mobile apps?
Mobile apps are now:
• how customers locate stores and information
• how brands get exposure many times a day
• the “face” of the organization
2
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Slow app = Slow product?
3
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Baseline vs. impact of network latency, etc.
4
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
from HP Network Virtualization by Shunra
[P01]
Possible points of risk in landscape
mDev:
Data
Cachin
g
Platforms
& Versions
Network:
Latency
& Loss
Network:
Network:
Dev:
Load
Resourc
Bal.
Encrypt,
e
Algorith
Compres
Usage
m
s.
3rd Party
2G,3G,4G
Cellular Cloud Services
Firewalls
VPNs
Device Features
Screen
Sizes
ISPs
SysAdmin: Back-end:
Pooling &
Server
Threadin
Config.
g
Web
Servers
DB
Servers
Load
Balancers
Carriers
Internet
Browsers
Mobile
Web
Services Components
LANs
CDNs & CoLo.
5
App
Servers
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
DBA:
DB
Indexes
& Logs
Storage
HP LoadRunner components and workflow
End User
Client(s)
Application Server(s)
Internet/WAN
Database
4 Virtualize
5 Virtualize Network
Services
1 Author
Scripts
6 Generate
7 Measure Impact
Load
8 Analyze, Compare,
Report
HP Live
Network
Test
Scenario
2 Collaborate
3
6
Plan and Execute
Tests
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Scan for more information
http://www.hp.com/go/...
LoadRunner
Network
Virtualization
Service
Virtualization
Mobile
Testing
HP Codescan app
7
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Mobile technology terms
Windows
PC
“feature
phones”
internet
explorer
minima
l
browse
UR
r
L
Chrome
Safari
browser
m.ft.com
mobile
site
ft.com
HTML5
responsive
design
URL
ft.com
HTML4
8
Android
iPhone,
iPad
Play.
Apple
googl
App
e
Store
.com
Hybrid
app
URL
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Native
app
ft.com/ws
web
services
Mobile LoadRunner Scripting
Demos
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Android Screen Monitor
http://code.google.com/p/android-screen-monitor/
Download to folder …\sdk\platform-tools\ASM.jar
Within Windows Explorer, double click on ASM.jar
10
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Create a New Script, Mobile Category
HP LoadRunner VuGen 11.5 SP1+
11
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Multiple Approaches Available
Browser or hybrid app:
Ajax? > TruClient Ajax Mobile scripting protocol using User Agent spoofing in
Firefox
Simple > Mobile HTTP/HTML scripting protocol
Native app: > Mobile HTTP/HTML scripting protocol
Android:
No device > Record Emulator using Android Virtual Device (AVD)
Have rooted device > Mobile Recorder generate lrpcap to Analyze
iOS & others:
> LoadRunner Proxy on same subnet/Wi-Fi hub (the newest technique)
> Record & Analyze WinPcap capture on the mobile app server
12
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Recording Method TruClient Ajax
App web
server
TruClient
Ajax
13
User Agent
in Firefox >
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HP LoadRunner VuGen 11.51+
TruClient recording
HP LoadRunner VuGen 11.51+
14
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Devices with User Agent strings
Count: 27
HP LoadRunner VuGen11.52
15
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
User Agent string example
Sample User Agent for Samsung Galaxy S3
Mozilla/5.0 (Linux; U;
Android 4.0; xx-xx; GTI9300 Build/IMM76D)
AppleWebKit/534.30
(KHTML, like Gecko)
Version/4.0 Mobile
Safari/534.30
More at http://www.useragentstring.com/
HP LoadRunner VuGen11.52
16
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Android Versions
4.1 (16)
4.2 (17)
4.0 (15)
2.3 (9,10)
17
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Mobile Device Manager Tool in VuGen
18
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
VuGen Scripting Process
1.
2.
3.
4.
5.
6.
7.
8.
9.
Configure recording settings
Consider run-time settings
Object identification using script
levels
Specify Transaction Names
Specify Completion End Events
Logic
Insert flow control
Run-time logic
Vary data based on parameters
Using C script functions in a library
from HP Virtual User Generator User Guide 11.52 page 73
19
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Random Think-time in TruClient
Arguments:
Math.floor( 11 * Math.random()+5
);
http://www.perftesting.co.uk/ajax-truclient-tips-and-tricks-part-ii/2012/05/02/
20
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
TruClient Toolbox drag and drop
Arguments:
var randWait =
Math.floor( 11 * Math.random()+5
);
window.alert( “randWait=” + randWait );
http://www.perftesting.co.uk/ajax-truclient-tips-and-tricks-part-ii/2012/05/02/
21
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HTTP/HTML Mobile Recording
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Mobile HTTP vs. HTML
mode=“HTTP”
mode=“HTML”
For use with web services calls.
Does not parse for resource requests.
Parse HTML to automatically
identify additional resources
to download
• CSS
• JavaScript
HP LoadRunner VuGen
23
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Record Mobile HTTP/HTML
HP LoadRunner VuGen11.52
24
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HTTP/HTML Recording Methods Wizard
HTTP/HTML
Record &
Analyze
Traffic
Analyze
Traffic
Record
Emulator
Proxy
Recordin
TruClient
g
HP LoadRunner VuGen11.52
25
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Proxy Recording
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HTTP/HTML Recording Methods Wizard
HTTP/HTML
Proxy
Recordin
TruClient
g
HP LoadRunner VuGen11.52
27
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Proxy Recording
HTTP/HTML
Android
AP
K
Proxy
Recordin
g
LoadRunner
Proxy
VuGen
Proxy IP:
Port 8080
iOS IPA
App web
server
Port 80/443
HP LoadRunner VuGen11.5x
28
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Proxy Recording
HTTP/HTML
Proxy
Recordin
g
HP LoadRunner VuGen11.52
30
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Proxy Recording Setup on iOS
1.
2.
3. Scroll down
4.
255.255.255.0
192.168.1.2
Eleanor
192.168.1.1
Eleanor
192.168.1.3
8080
Password:
ncjm-q3ey-t1dd
Source: http://www.amsys.co.uk/2012/blog/how-to-setup-proxy-servers-in-ios/
31
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Proxy Recording Setup on Android Galaxy S3
(4.1.2)
1.
2. Long press
3.
Eleanor
ncjm-q3ey-t1dd
32
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
4.
Record Emulator
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HTTP/HTML Recording Methods Wizard
HTTP/HTML
Record
Emulator
HP LoadRunner VuGen11.52
34
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Record Emulator Android Recording Method
HTTP/HTML
Android
AP
K
Copy
Install
Backup
Play.
Google
.com
File
manager
Android… \sdk\tools
Record
Emulator
Async.
AP
K
device
@device ADT
emulator
file
App web
server
HP LoadRunner VuGen11.5x
35
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Record Emulator Requirements
HTTP/HTML
Record
Emulator
1.
2.
3.
4.
USB cable to device
USB driver for device
Android app APK file for Android app under test
Android file download manager app to extract APK file from device
5.
6.
7.
8.
JDK (Java Development Kit)
ADT (Android Development Toolkit) plug-in for Eclipse
AVD (Android Virtual Device) within Eclipse for device under test
JAVA_HOME and PATH Windows environment variables
9. HP LoadRunner 11.50 and SP2 patch to upgrade to 11.52
36
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Emulator
37
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Record & Analyze Traffic
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HTTP/HTML Recording Methods Wizard
HTTP/HTML
Record &
Analyze
Traffic
HP LoadRunner VuGen11.52
39
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Record & Analyze Traffic
HTTP/HTML
Record &
Analyze
Traffic
Android
.cap,
.pcap
network
sniffer
AP
K
Ad hoc
network
iOS IPA
App web
server
HP LoadRunner VuGen11.5x
40
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Analyze Traffic
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HTTP/HTML Recording Methods Wizard
HTTP/HTML
Analyze
Traffic
HP LoadRunner VuGen11.52
42
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Analyze Traffic
HTTP/HTML
AP
K
Analyze
Traffic
Android
Android 4.1+
(rooted)
Mobile
Recorder
.lrpcap
App web
server
HP LoadRunner VuGen11.52
43
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Android Mobile Recorder
1. Start Recording
44
2.
Perform
user
steps.
3. Stop and send email
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Android Mobile Recorder Settings
45
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Recording Methods Recap
HTTP/HTML
Record &
Analyze
Traffic
.cap,
.pcap
network
sniffer
AP
K
Analyze
Traffic
Record
Emulator
46
Play.
Google
.com
.lrpcap
AP
Ad hoc
K
network
device
@device ADT
emulator
file
Proxy
Recordin
g
TruClient
Ajax
Android
Android 4.1+
(rooted)
Mobile
Recorder
LoadRunner
Proxy
iOS IPA
VuGen
Proxy IP:
Port 8080
App web
server
Port 80/443
User Agent
in Firefox >
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HP LoadRunner VuGen11.52
Mobile scenarios
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
How can we help?
Analytics
Development
Executive Scorecard
Agile Manager
App. Lifecycle Intelligence
Performance
Center
Test results
Execute automated build
(Jenkins)
Security Code
Fortify Source Code Analysis
Scan
ALM Analysis
Tear down
Continuous Delivery Automation
(CDA)
environment
Provision & deploy
Continuous
Delivery Automation (CDA)
environment
Application Security
WebInspect
Assessment
Monitoring & Diagnostics
Business Service Management
Run
test cases
ALM Lab Management
UFT / BPT
LoadRunner
Service Virtualization
Network Virtualization
HP integrated testing suite
48
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Interconnected component services
Shopping process
Search
49
Buy
Pay
Ship
Track
Product
Customer profiles
Payment
Shipping
Track package
Composite
Managed
cloud
Traditiona
l
Public
cloud
Private
cloud
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Concurrency and rendezvous
50
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Illustrative data
[P10]
Key take-aways
• For mobile browser apps (jQuery Mobile, Sencha, PhoneGap, etc.),
little scripting is needed using User Agent spoofing via Firefox TruClient
Mobile
• JavaScript executes against a DOM for each virtual user
• Traffic captured between native apps and server are converted to VuGen
scripts
• Android native app traffic on rooted devices can be recorded on
device using
LoadRunner Mobile Recorder
• iPhone and iPad (iOS) -- all non-rooted apps – can be recorded using
LoadRunner Proxy
• Monitor and Diagnose using HP Business Service Management
• Consider HP Network Virtualization and Services Virtualization
51
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
For more information
Attend these sessions
Mobile Performance
Hands-on Lab HOL3330
Sands Room 305
• Tuesday 1p – 3p
• Thursday 3p-5p
Visit these demos
Applications: mobile performance
testing RT3112
• Wednesday June 12, 1030 am
Lido 3101A
• Thursday June 13, 930 am
Lido 3101B
After the event
• Contact your sales rep
• Visit:
http://www.hp.com/go/mobiletestin
g
http://www.hp.com/go/loadrunner
Your feedback is important to us. Please take a few minutes to complete the session survey.
52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Thank you
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.