Ben Gurion University
Software Engineering
Multi-criteria
infrastructure
for
location-based
applications
Testing Document
Ronen Abraham
Ido Cohen
Yuval Efrati
Tomer Sole'
July 2011
1
Ben Gurion University
Software Engineering
Table of Contents:
1. Introduction
3
2. Testing functional requirements
2.1.
Basic operations testing
2.2.
Framework extensions testing
2.3.
Framework monitoring and learning testing
2.4.
Server requirements testing
4
7
8
10
3. Testing non-functional requirements
11
4. Stress tests
14
5. User interface for testing
14
6. Testing build, integration & deployment
14
7. Test data
16
8. Administration
17
2
Ben Gurion University
Software Engineering
1. Introduction
"Localization Platform" is a framework for developing mobile location-based
applications. The framework enables the application developer to:
- Store the present and past locations of all the application users
- Improve the performance of the following parameters
 battery life
 network data plan
 processing time
 memory space
This framework contains:
 An Android client API.
 A storage server.
 "Registered services" API.
As seen in the following figure (for further architecture details, see ARD):
In this document, we will describe our testing strategy for the project. Every test that
we planned in the ADD, we ran and wrote its result. We will start by covering the
testing of the functional requirements, or "unit tests", for each element in the
architecture. Then, we will be reminded with our non-functional requirements and
show the different ways in which we tested them. Finally, we will cover various
testing techniques used in the scope of this project, such as stress testing, GUI client
for testing, deployment and build testing.
3
Ben Gurion University
Software Engineering
2. Testing functional requirements
2.1. Basic operations testing:
Functionality
Add user location
Description
Localization client /
registered service send to
the storage server a
location update, about a
user, a specific time, and
location coordinate.




Get last user locations
Tests
Test that Localization
client can add a valid
location.
Test that a client can’t
add a non existing
location.
Test that a nonregistered client can’t
add new location



Verify that added
locations are stored in
the DB
Test that the
Localization client can
query for single user
last location
Test that the
Localization client can
query for a group of
users last location
Test filtering by date on
a query


Test filtering by max
results query


Test that a nonregistered client can’t
get user locations

Localization client /

registered service query the
storage server about
locations of a group of
users. Optional filtering

parameters: date range,
max results to return.

4

Result
A valid location added
successfully
The server returned an
error message about
wrong location.
None-registered client
didn't success to add
new location before
registered.
All locations were
added successfully to
the database.
Last single user's
location returned
successfully

Last locations for a
group of users returned
successfully

Filtering by date – all
locations that were
matched to a given
date returned
successfully
Locations that were
more than "max" were
not returned
A non-registered client
got an error message
told him that he must
register in order to get
his locations
Ben Gurion University
Software Engineering
Basic operations testing (con't)
Functionality
Start localization
client
Description
Localization client begins
getting user's locations
every period of time, and
sending updates to the
server.
Tests
Test that starting the
client does add location
periodically



Test that the location
sent are saved in the
right order in the DB

Result
All the locations that
were sent periodically
were added successfully
to the DB.
The sent locations were,
in fact, saved in the right
order in the storage
server database.
No new locations were
added to the DB after
stopping the service.
Stop localization
client
Localization client stops the
process of requirement

Test that after stopping,
no new locations are
added for the client

Register service
A service adds itself to
receive event notifications
from the storage server
("push").
This also gives the service
permission to ask the server
for past locations and add
new user locations.

Test that a service can
register to receive
notifications from the
server.
Test that a service can’t
register itself more than
one time

New service was
successfully registered
to the server.

Test that a registered
service does get
notifications when
relevant events happen
(relevant locations are
added)

Service that were
already registered to the
server got an error
message in the second
time it tried to register
The registered service
successfully got a
notification when new
location was added.


5
Ben Gurion University
Software Engineering
Basic operations testing (con't)
Functionality
Unregister service
Description
Removes a service from the
list of known clients from
the storage server.


Register user
Unregister user
Events for the
registered service
Localization client sends a
unique identifier of the user
("strong identity") to the
storage server. The storage
server returns an ID of the
user, and that's how the
user is recognized from now
on. If the user already exists,
the server returns its
existing ID.

Localization client lets the
server know that a user has
unregistered from the
platform. The server simply
mentions the unregistration
date and time.

Previous Functional
requirements can trigger
events for any registered
service that registered to
accept them. Whenever one
of those functions happens,
the storage server triggers
the following event in the
registered services that
requested to be notified on
that event.





Tests
Test that a service can
un-register itself from
the server.

Test that an
unregistered service
doesn’t get notifications
for any event
Test that a user can
register from a client
and get an ID from the
server
Test that if an existing
user tries to register
again he will get his
stored ID (same one as
the first time)
Test the two different
registering users get
different IDs

Result
Service that was
registered successfully
removed itself by
unregistering.
Unregistered server
didn't get notifications
for any event

After a user registered
he got a unique ID from
the server

An existing user that
registered, got the same
ID he already had from
the first registration.

The ID that was returned
from the server was
unique for each user.
Test that a user can
unregister.
Test that the user is
removed from the DB

Test that the server
reply an unregistering
user with the current
time
Test that a registered
service gets events for
every update – add
location, get last
location, register a user
and unregister a user.

User unregistered
successfully and his
unique ID he got on
registration was
removed from the DB.
The server returned the
user the current
unregister time.
6

A registered service got
events notifications
when a change was
occurred, such as: add
location, get last
location, register a user
and unregister a user.
Ben Gurion University
Software Engineering
2.2. Framework extensions testing:
Functionality
Location
prediction
Predict next
location
Description
Three algorithms for
recommending the next
movement of a user in the
system will be implemented
by a "registered service".
That service will register for
the event of "add user
location", so that every time
a new location is added, the
recommendation service is
notified.
A function of the
recommendation service.
Returns the next
recommended location of
the given user based on the
algorithm inside it.


A demonstrator for the
localization client and the
registered service will be
Implemented.

Result
The right location was
returned for the user
requesting a prediction.
Test that Prediction for
a registered return a
location.
Test that the locations
returned are valid
coordinates
Test the a nonregistered user trying to
use prediction get an
error message

The prediction function
returned a location

The returned location
coordinates were valid


Test that Users can
register and unregister
using the demo client


Test that Users can add
and get location using
the demo client.


Test that it is possible to
start and stop periodic
location sending.


Test that registered
service gets all 4 types
of updates

An unregistered user
who tried to use
predication got an error
message that he needs
to be registered first.
Clicking on the register
and unregister buttons
cause to registration
and unregistration
respectively.
By clicking on "Start
tracking location"/"Send
location" the user adds
locations to the DB and
by clicking on "Predict
Location" the user gets
the predicted location.
By clicking on "Start
tracking location" and
"Stop tracking location"
the user start and stop
the location sending.
When user clicks on
those 4 option, the
service gets matching
event


Mock application
Tests
Test for a given list of
users and locations, the
right location will be
picked for a user as his
"prediction" according
to the algorithms
specifications.
7
Ben Gurion University
Software Engineering
2.3. Framework monitoring and learning testing:
Functionality
Configuration
management
Security
Description
The framework should
provide the user the ability
to control the configuration
of the data usage
frequency:
 Duration between the
location updates a
localization client sends.
 If using the location
prediction service,
frequency of calling that
service.
And additional parameters,
such as:
 Address of the storage
server
The framework should not
expose the entire database
to one user, only the
information that relevant
for him. The localization
client will get information
only relevant to the user it
represents.

Tests
Test that the default
configurations work
smoothly


Change the frequency
of sending locations
using the demo client UI


Change the update
frequency using the
demo client UI


Test the ability to
change the location
storage server


Test that after changing
the location server
location are still saved


Test that the user can’t
get prediction for other
users


Test that a user can’t
add location for another
user


Test that a user cant
register without its
unique ID

8
Result
Clicking on "Start
tracking location" with
default configuration
worked smoothly
Changing frequency of
sending locations from
settings menu worked
successfully.
Changing the update
frequency from settings
menu worked
successfully.
The user can change the
location storage server
by clicking on "Server IP
address" under settings
menu.
After changing the
location server, location
were sent to the new
server's IP address
User who has asked for
prediction, got
prediction according to
his unique ID.
Location that was sent
by a user, sent with
user's ID, in that way
only locations of this
specific users were
updated.
User that tries to
register without a
unique ID, got a new
unique ID, otherwise he
used his unique ID he
already had.
Ben Gurion University
Software Engineering
Framework monitoring and learning testing (con't)
Functionality
Logger
Deployment
Description
The system shall have a
logging mechanism, to view
all the user commands
The localization client will be
a code library, to be used as
reference inside the
location-based application
that will be developed by
the platform's user.

Tests
Test that it’s possible to
change log level from
info to debug and error

Result
The server side has a
configuration file that
sets the log level

Test that all added
location are accounted
for in the log.

Every added location is
recorded in the log file.

Test that all user logins
and logouts are
accounted for in the log.

All users' login\logout
operations are recorded
in the log file.

Test that the localization
client can be imported
and added to a new
project
Test run the client with
the imported project

The location client .jar
file was imported
successfully to a new
project.
The client was run with
the imported jar
successfully.

9

Ben Gurion University
Software Engineering
2.4. Server requirements testing:
Functionality
Server Database
Description
The storage server shall hold
a database with all the
users, the users' locations,
registered services and the
events the services are
registered to.
Server sessions
monitoring
The storage server shall
manage a log file that will
indicate who contacted the
server, when the connection
occurred and all the data
transfer.




Server
Deployment
An installation guide on how
to set up a new storage
server will be provided

Tests
Test that the information 
about the users, the
users' locations,
registered services and
events the services are
registered to is kept
through the running of
the storage server.
Test that all connections 
to the server are logged
Test that failed
connections attempt are
logged
Test that all location
data transfers are
logged
Verify that Server is
installed and working
after following the steps
described in the user
manual
10



Result
A fitting SQL schema
was specified, and the
items were saved
through the runtime of
the storage server.
Any connection to the
server is recorded in a
log file.
Any failed connection to
the server is recorded in
a log file.
Any location's data
sent/added is recorded
to the log file.
The server is installed
and worked successfully
after following the steps
described in the user
manual.
Ben Gurion University
Software Engineering
3. Testing non-functional requirements
Functionality
Speed
Capacity
Throughput
Constraints
The connection with the
storage server should take
less than 5 seconds.
 The location update from the
client to the server should
take less than 5 seconds.
 The location prediction
should take less than 10
seconds.
(In order to test the speed
constraint We will use the
emulator array)
 The number of users that can
access the storage server
simultaneously will not
exceed 999(TCP/IP
capabilities).
 The number of active users
can the storage server can
handle will not exceed 999



Reliability


The storage server should
handle as many as 150
requests simultaneously (The
server is available to max. 999
users simultaneously,
although some of them might
wait until a thread becomes
available).
The predication server should
handle as many as 150
requests simultaneously
In 100% of the cases when an
unregistered client makes a
request, the operation fails.
Each registration that is
supposed to succeed shall
take place.
Each request of a register
client should be succeeded, in
case the server is online. Each
request for location update
should be succeeded, in case

Tests
Connecting to the
server


Adding a new location

Adding a new location
took between 2 to 4
seconds

Getting a prediction

Getting a prediction
took between 2 to 6
seconds.

Test that it is possible
to add 999 users to the
system

999 users were added
successfully to the DB.

We will use the
emulator array to check
that it is possible to
connect with up to 90
client simultaneously
We will use the
emulator array to send
150 prediction requests
simultaneously.

We built a script that
can connect N client
simultaneously.


We will verify that the
150 request got a
location prediction as
an answer

By clicking on "Predict
Location" button, every
user added it's
prediction request on
the Prediction server
Eventually, every user
got his predicted
location.

Legal and illegal
registration requests
will be sent to test
durability of the
system.
Simulation of a
Registered and
unregistered users will
send prediction
request, all registered
should get a prediction



11

Result
The connection with
the storage server took
between 2 to 4 seconds
Our Mock Application
sends Legal parameters
automatically. In any
other cases, both
"Mobile ID" and
"Registration Time"
fields are supporting
input constancy.
After sending A
Prediction Request, a
registered user got a
Ben Gurion University
Software Engineering
Functionality
Reliability
(con't)
Safety &
Security



Constraints
the server is online.
Each request for location
prediction should be
succeeded, in case the server
is online.
When a user sends a location
update the storage server will
update only this specific
user's data
When a user sends a location
prediction request, he will get
the correct information only
about him
Tests
and all unregistered
should get an error
message


Portability &
Reliability





The system works on the
Android mobile operating
The server GUI web interface
will be 100% compatible with
HTML standards and
therefore be available to
every standards-compatible
web browser. The following
web browsers are expected
to be fully supported: Google
Chrome, and Firefox.
The system should support
only in an English text
In case of a network failure
any connection between
client and server will be
disconnected.
When the server crashes,
whether it is its software
crashing or its hardware
crashing, it is supposed to go
back up restoring above 90%
of the data it previously
acquired.




12
Result
prediction and an
unregistered user got
an error message.
We will use the
emulator array to
simulate a state where
many users are adding
different locations, and
check that for each
user it own unique
location was added.
We will create a state
where each user has a
different unique
prediction, and then
check that each user
gets its own specific
unique prediction.
All of the client
treatment will be
performed on Android
OS. (Emulators and real
phones)
Test that it is possible
to install the
application on the last
version of the Android
OS

After adding 150
different locations, an
unique location has
been added to every
stimulated user.

After requesting 150
Prediction Requests,
every stimulated user
got its own specific
unique prediction.

Our client has been
successfully installed on
the latest version of
android OS to this date
(Galaxy s 2 with OS
level 10 for
smartphones). In
addition, the minimum
supporting level is OS
level 3.
We will perform the
server GUI functional
test on the three web
browsers mentioned
above to make sure
that the application is
not failing on them.
Test that it is possible
to restore the system
from a daily backup
dump - To verify
support of restoring in
case of the server
crushing.

The server GUI is
working on the three
web browsers: Internet
Explorer, Google
Chrome and Mozilla
Firefox. No one of
those browsers has
cause a crash in any
part of the GUI.
Ben Gurion University
Software Engineering
Functionality
Usability



Availability



Constraints
The API documentation will
be available as static
standalone web pages
("Javadoc").
The mock application will
provide a way to check the
following functions:
 Registration.
 Send current location.
 Predict next location
The client will be able to
configure the sending current
location frequency and the
prediction frequency
The database server and
prediction server shall run at
least 95% of the time, and
listen to requests from
clients.
On the client side, the
application can be shut down
and the client decides when
to turn it on.
This will have to be tested
over a long period of running
time. Mainly in the beta
testing stage.

Tests
Test that the "javadoc"
is accessible via the
internet

Result
The "javadoc" has been
uploaded to the project
website, and it's fully
available via the
internet

Functional testing of
the mock application
will cover the usability
test of the application

The mock application
has built in a way that
it's demonstrating
every functionality of
the API platform.
Therefore, the usability
is corresponding.

Test that the android
application is starting
smoothly

No kind of any delay
has been found during
the android application
starting process.

Test that the android
application shutting
down gracefully on the
users command

The android application
has been shut down
successfully ether
during a GAC process or
without.
13
Ben Gurion University
Software Engineering
4. Stress tests
We tested up to 90 emulator clients, concurrently. We built a script that creates N
emulators and connects them simultaneously. We successfully simulated tests about
connecting to the server, adding large amount of different locations, sending
prediction requests, getting the predicted locations.
A higher capacity test will be performed at the beta testing stage.
5. User interface for testing
Since the server-side functions are considered "SOA" (service oriented architecture),
by definition, SOA testing is having an entry point for every remote function that sits
in a remote server service.
A very useful testing tool is the provided "mock application". The manual mode in
that mock application has a button for each server-side function (both storage server
and prediction server). This gave us a nice little testing tool for the server-side
methods, isolating each operation.
6. Testing build, integration
& deployment
 Our complex architecture is automatically built by pre-defined ant scripts. The
basic build of our servers contains the following operations (though we can take
some of them off from time to time as needed): clearing our database, compiling
the source code into .jar packages, wrapping them into Tomcat-friendly files
(.war), and copying them to the Tomcat folder, so that when we launch the
Tomcat server, the updated code will be loaded.
14
Ben Gurion University
Software Engineering
 A special feature of our build process is automatically creating the database data
classes. Those classes are created in the ant script process. It reads the SQL
schemas, generates Hibernate-friendly classes that represent those tables. Those
generated data classes also use JAXB (XML) serialization, and therefore they can
also be passed between the client and the server.
 For testing, we run our "deploy.all" ant command for the storage server and we
run our "deploy.all.prediction" ant command for the prediction server.
Afterwards, we start up the server (using tomcat), and then the android client can
be run.
 Version control: all our code is saved under SVN, in a server located in BenGurion University. This gives us the ability to compare past versions and share the
application between many different users.
15
Ben Gurion University
Software Engineering
7. Test data
One of our obstacles was handling the client locations data, as it was very random.
We can't control the location of their device. We did have other ways to know the
data we are expecting before receiving it in the server side:
 In our database schema, we have an extra "test data input table", which
contains more than two million rows of about 89 users and their locations.
Our "Testing Mode" uses this input table. This mode gives us the ability,
instead of sending location from the device's GPS, to just "play" that input
table rows and automatically import those "fake" records into our main
database. That way, the locations are predictable and we know in advance
what result in the server we should expect.
 For location prediction, we have 5 different pre-defined SQL scripts that
initialize users and clusters with pre-defined values, so that the server can run
test prediction for a given user ID and know exactly for what to expect.
16
Ben Gurion University
Software Engineering
8. Administration
 We created a small HTML page with some functions to test quickly by a web
browser:
This html tester tests:
 Registering an external service.
 Playing the input test data.
 Adding a new location for a user.
 Predicting a location for a given user.

The HTML infrastructure is flexible. We separated the Javascript logics into three
files, each represents a different layer:
 The GUI layer for interacting with the HTML elements, getting and setting
them values.
 The REST service client for sending and receiving arguments by calling REST
methods
 The logics layer – a set of functions to operate when pressing the HTML
page buttons. Those functions generally connect between the GUI layer and
the REST wrapper layer.
This flexibility and low coupling can allow us to contain more test functions in the
future.
17