Using GUI Ripping for Automated
Testing of Android Apps
Domenico Amalfitano
Salvatore De Carmine
Anna Rita Fasolino
Porfirio Tramontana
Dipartimento di Informatica e Sistemistica
University of Naples Federico II, Italy
Atif Memon
Department of Computer Science
University of Maryland
Motivation

Android is a very popular operative system and framework for
mobile devices




Android apps are often developed according to an evolutionary life
cycle, with very frequent releases via Google Play


Its diffusion is increasing: recently it overtook ioS and Symbian in terms of
diffusion
Over than 500,000 apps are now on the Google Play market
The most part of the apps are realized by single programmers or little teams and
lacks in quality
Bugs causing failures and, in particular, crashes are very common
There is a strong need for testing automation processes
and tools

In this talk, our techniques and tool supporting a process of Android
GUI Testing Automation will be presented
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
2
Android apps

An Android application is composed of several types of Java
components instantiated at run-time:


An Activity component:




Activities, Services, Intents, Broadcast Receivers and Content Providers.
is responsible for presenting a visual user interface;
provides a screen with which the users can interact in order to do
something.
An Android application usually includes one or more Activity
classes that extend the base Activity class provided by the
Android development framework.
Android Applications are Event-Driven Systems

Three types of events can be triggered:
 user events (such as Click, MouseOver, etc.);
 events related to the lifecycle of the Activity classes
 onStart, on Pause, onResume, onStop, onDestroy, …
 events that are triggered by other input sources
 such as GPS receiver, phone, network, etc.
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
3
Existing approaches to Android Testing Automation

JUnit Testing

Android Testing Framework

Supports the coding of executable test classes



Random Testing


Classes supporting the instrumentation and monitoring of Android apps are provided in the
Android framework itself
Libraries such as Robotium and Monkeyrunner extends the framework
Monkey (included in the Android framework) supports the randomly
automatic exploration of Android apps
Model Based Testing

TEMA Tools (Takala, Katara, Harty, 2011) represent a first tentative of
approach for the generation of test code from a design model of
Android app
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
4
Android GUI Ripping

We propose a technique and a tool for:

Automatic exploration of the GUI of an
Android application via a systematic Ripping
approach




Similar to approaches adopted for Web crawling
Crash testing
GUI model reverse engineering
JUnit test case generation
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
5
Android GUI Ripping Algorithm

Customizable





Selection of types of events to be fired
Adoption of a GUI traversing strategy (e.g. depth first, breadth first, etc.)
Termination criteria used to stop the GUI traversal
Setting of specific preconditions
Efficient

Provides heuristic solutions to the well known problem of GUI interface
states and events explosion
 GUI exploration criteria based on a Interface Equivalence Criterion
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
6
≡
Rotate
1.
2.
3.
4.
5.
6.
7.
≡
Click
Click Refresh
New Post
Task List Initialization;
while (Task List Is Not Empty) {
Click Pages
Press Menu
9.
List;
5.
…
6.
7.
8.
9.
Task List Initialization;
while (Task List Is Not
Empty) {
Extract a Task From The
Task List;
Execute the Task;
Abstract the Current GUI
Abstract State;
Update the GUI Tree
Model;
if (GUI Exploration
Criterion) then {
Define New Tasks ;
Add New Tasks To
The Task List;
10.
Click Add
Account
11.
}
}
Click About
…
}
10.
2.
4.
ClickTasks
Save
Define New
;
Add New Tasks To The Task
8.
1.
3.
Extract a Task From The Task List;
Execute the Task;
Abstract the Current GUI Abstract
State;
Update the GUI Tree Model;
if (GUI Exploration Criterion) then
{
11.
Ripping
Algorithm
}
Crash
Click
Edit
…
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
…
7
Crash Testing Process


Preconconditions are set
by generating a Snapshot
Image of the device for
the emulator
In the task execution step,
the GUI Ripping algorithm
is executed
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
8
A Demonstrational example

Android for Wordpress

A Wordpress client for
Android



Measured metrics

334 classes, ~10,000 LOCs
More than 250,000 downloads
A set of three preconditions





No Login
Single Blog
Multiple Blogs
Comparison between
Ripping and Random
Testing with Monkey
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012

Effectiveness
 Number of crashes
found
 Number of different
bugs causing crashes
 LOCs coverage
Efficiency
 Number of test cases
 Number of events
 Ripping time
9
Results
Preconditions
No Login
Single Blog
Multiple Blog
Random
(Single Blog)
#Crashes
0
6
8
3
#Bugs
0
3
4
1
LOC Cov %
2.65
39.32
37.83
25.27
Time (h)
0.2
4.88
4.58
4.46
#TC
18
166
166
-
#Events
36
517
502
14400
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
10
Bugs and Crashes

The 14 crashes
found by the Ripping
process belongs to
four different bugs


Bugs have been
submitted to the bug
tracking system and
accepted/fixed by the
programmer
Monkey was able to
find only one of
these bugs
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
Id
Crash Description
Java
Exception
Ticket and
Changeset
B1 The app crashes trying StringIndex https://android.tra
to opening the default OutOfBoun c.wordpress.org/tic
post “Hello World”
dException
ket/206
https://android.tra
c.wordpress.org/ch
angeset/398
B2 The app crashes when BadTokenE https://android.tra
the Stats activity is xception
c.wordpress.org/tic
rapidly
opened
and
ket/208
closed (via the Back
https://android.tra
key).
c.wordpress.org/ch
angeset/420
B3 The app crashes when NullPointerE https://android.tra
the Stats activity is open xception
c.wordpress.org/tic
and the Refresh button
ket/212
is clicked while the
https://android.tra
progress bar widget is
c.wordpress.org/ch
still loading.
angeset/423
B4 The app crashes when NullPointerE https://android.tra
the user opens a post xception
c.wordpress.org/tic
and tries to share it
ket/218
within his blog. The
https://android.tra
crash occurs when there
c.wordpress.org/ch
angeset/446
is a single blog in the
app.
11
Conclusions and Future Works

Our technique supports the execution of
completely automatic generation and execution
of test cases for Android applications.
In our preliminary demonstration example:





Real bugs and crashes have been detected;
Good level of code coverage has been reached;
Testing time is compatible with smoke testing
processes;
Reusable test cases has been automatically generated;
Our technique demonstrates its effectiveness with
respect to standard Random testing tools such as
Monkey
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
12
Future Works


This presentation covers the first part of a larger
project on testing automation of Android
applications
Some future works

Publication of the tool


As a standalone tool
As a service


Further experiments with more applications and
ripping parameters


A preliminary demo is available at
http://wpage.unina.it/ptramont/GUIRipperWiki.htm
Delay Time, ripping strategy (depth first/breadth first, activity
equivalence criteria, maximum depth), input generation
strategies, …
Sensor event testing
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
13
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
14
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
15
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
16
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
17
Output
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
18
Android GUI Conceptual Model

An Interface is a sort
of instance of a
user-defined class
extending Activity


Similar interfaces are
grouped together
Events can be


User events, related to the interaction of the user with the GUI
System events, related to external sources or to the lifecycle of the
activities
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
19
An example of FSM
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
20
Other experimental results
App
Depth
Limit
Eq. Crit.
Delay
Time
Time
(h)
#TC
#Evt
#Crash
#Bug
Class
Cov %
Metho
d Cov
%
LOC
Cov %
1
AardDict
3
N
Short
13.33
501
562
0
0
70%
66%
59%
2
1.4.1
N
Y
Short
1.8
67
77
1
1
69%
71%
65%
3
3
N
Long
13
485
547
0
0
67%
66%
59%
4
N
Y
Long
2.2
83
93
1
1
69%
71%
66%
Short
2.2
94
116
0
0
48%
37%
5
TomDroi
d
3
N
6
0.5.0
N
Y
Short
1.25
54
64
1
1
49%
43%
39%
7
3
N
Long
7.5
343
401
0
0
48%
37%
34%
8
N
Y
Long
1.6
76
86
1
1
49%
43%
39%
Book
Catalogu
e
2
N
33%
24%
3.8.1
N
11
12
9
36%
Short
1.75
69
77
0
0
Y
Short
62
716
818
5
2
77%
65%
53%
2
N
Long
1.9
69
77
0
0
33%
24%
21%
N
Y
Long
61
701
800
5
2
2
N
Short
1.36
76
149
N
Y
Short
4.42
155
477
9
5
15
2
N
Long
2.31
76
149
5
16
N
Y
Long
4.58
166
502
8
10
13
14
Wordpre
ss
r394
Porfirio Tramontana – ASE 2012 – Essen – September 6, 2012
21%
76%
65%
53%
44%
34%
54%
42%
3
44%
34%
31%
4
54%
43%
38%
5
3
30%
37%
21