Tablet Apps Testing - Capitalizing on Mobile
QA Challenges
Kiran Marri – Delivery Manager
Janaki Sirisha Satthiraju – Technical Test Lead
Sundaresasubramanian Gomathi Vallabhan – Senior Project Manager
Infosys Limited (NASDAQ: INFY)
Abstract
We are moving away from the business of ears to the business of eyes and the trend is relatively
clear: Cell Phones, PDA’s, Smartphones and now? - The 'Tablets'. In order to reap the benefits of
innovation, many companies have started creating and releasing thousands of Tablet applications into
the market.
Books were mankind’s' best pals - then came computers, laptops, smartphones and then tablets more than just a friend but playing the role of a philosopher, guide, entertainer, banker, co-gossiper,
fellow-gamer depending on the applications and websites you've subscribed to
2
Advent of tablets in industry today
Financial
Manufacturing
Mobile banking
Enterprise solutions
Mobile payments
Secure transactions
Floor Operations
Real time data
Real Time control
Retail
Online shipping
Point of sale data
Inventory data access
Merchandise tracking
Enterprise Mobility
Health Care
Bring your own device
Mobile device management
Point of sale solutions
Wellness management
Integrated disease-Management solutions
Energy & Utilities
Sales & Distribution
Work Management
Outage Management
Field Management
Remote Meter Reading
Mobile access to sales
Mobile inventory
Online Customer data
Point of sales
3
Tablets are here to stay…
Industry analyst firm “Yankee Group” has predicted that tablet sales is expected to reach 250 million
units worldwide by 2015. Given below is a quick graph to show Apple iPad sales over the years1
Majority of workforce today is planning to go to tablet form of devices in next 3 years for accessing
content & information, 24x7 connectivity, and improve productivity on the go…
4
Popular tablets in the market…
Tablet
Apple The New iPAD
Key Feature(s)
Quad-core graphics processor, full HD 1080P video recording,
hotspot for shared internet connections, iBook application
Operating System
iOS 5.1
Apple iPAD 2
Thinner & lighter iPAD, usability and increased battery-life, front and
rear facing cameras for video calling
iOS 4.3 and above
Samsung Galaxy Tab
7.7”, 8.9”, 10.1” models, dual core processors, Social hub, Readers’
hub, media and music hub
Android 3.X
Motorola Xoom
Unlockable boot loader, dock in support for charging & video play,
HDMI out and sensors
Amazon Tablet
Cloud accelerated browser(Amazon silk), host of amazon online
services including amazon app store, low cost
Blackberry Playbook
Dual core processor, lightest tablet (0.9 LBs), access mail and
calendar info from phone through ‘bridge’
5
Android Honey comb & Ice
cream sandwich
Customized Android 2.3
Blackberry Tablet OS (QNX)
Why this Paper?
This paper addresses the practical challenges faced by Mobile QA teams during
Tablet Apps testing, proven solutions, learning’s and Best practices to launch
‘top quality’ apps. Going beyond this concept, readers can benefit from
understanding various real time scenarios supported by appropriate
examples/case studies out of which, couple is outlined below:
 Apps developed for Smartphones but used on Tablets, and vice-versa
 Usability issues on apps designed for Tablets vs. Smartphones
6
Smartphones Vs Tablets
Feature
Smartphone
Tablet
Inference
Screen size
2” to 7”
7” – 10.X”
Though web application coding is essentially the same between
smartphones and tablets, the style-sheets have to be different for
different form factors
Orientation
Mostly portrait (exception
– high end smartphones)
Portrait and landscape –
Dual
Application needs to be developed in such a way that both the
levels of orientation are supported
Information
Mostly single view
Split view
If you select an element from a list, the information about that
element is designed to be displayed on the same screen in a
tablet – but in a smartphone, it can go to next screen.
Pop-over
Not available (except high
end phones)
Available
An interstitial screen that becomes visible when user taps on an
onscreen area
Application icon
57*57 pixels in case of
touch screen phones
72*72 pixels
Minimum size of the application icon in smartphone(s) and tablet
to be visible.
Access
Touch, Stylus, button
controlled
Mostly touch
All tablets are touch screen based
Enterprise Mobility
Blackberry
Bring your own device –
mostly iPADs
7
Tablets are slowly increasing their share of pie in Enterprise
mobility due to BYOD (bring your own device policy)
Tablet Ecosystem & Types of testing
Ecosystem
Description
Type of testing
Physical device




Compatibility testing
Performance and stress testing
Interrupt testing
Ergonomics testing
Application
Web Application, Native or Hybrid
Application







Functional testing
User Experience testing
App performance testing
Security Testing
Localization Testing
Performance Testing
Interoperability Testing
Network
GPRS or 2G/3G/4G or Wifi


Network variability
Network performance
Server
Application/Database server, User
Agent adapter


Server performance testing
Middleware/User Agent testing
Device
8
Tablet QA Challenges
Ecosystem
Type of testing
User Experience
 The biggest challenge is that the developers do is to design for the screens for tablets and mobile
 UI issues are almost 40% of the defects in majority of the projects followed by performance issues
Automation
 Though there are plenty of options in the market, most of the tools need either source code change
of the application or user agent installation / jail break.
 The % of functional automation is low (25-30%)
Compatibility
 This includes numerous browser/platform combinations and challenge would be the coverage effort.
 The QA team must work on device diversity scheme for the tablets and browsers
Security Testing
 Validating the tablet in sleep mode, jailbreak mode (if any).
 Accessibility of personal information and BYOD scenarios
 UDID check
9
Case Study 1 : News-paper effect
Figure: Web page developed for Tablet (Left) Vs page developed for smartphone (Right)
 Screen design has been optimized for large screen (iPAD) Vs small screen (Blackberry)
 Written and image content has appropriately been condensed according to screen area
 Title has been left justified in the smaller screen to accommodate the photo
10
Case Study 2 : Orientation
Tablets are designed for dual orientation ie., both horizontal and vertical (Portrait/Landscape). Given below
are some of the validation points while checking dual orientation:

If the application is by default designed for landscape mode of display, irrespective of the orientation of
the tablet while opening, the application should always open in the landscape mode. This leads user to
adjust the orientation automatically.

Virtual keyboard appearing in the screen should also orient itself to portrait or landscape mode based
on tablet’s orientation.

Keyboard should adjust its size to occupy portrait or landscape orientation and occupy the entire width
of the tablet.

All tabs and menus appearing on the application screen should occupy the width of the tablet
regardless of orientation. Stretched or truncated menu will give rise to bad user experience.
11
Case Study 3 : Split-View & Pop-over
Split View
Pop-over 
Split view and popover are features introduced in the tablet to minimize the number of screens traversed, thus keeping the
response time minimal. Some of the typical validation points to cover these features are:




Ensure that in case of split screen, the list appears on the left hand side and the information about a particular item in
the list appears on the right hand side.
If the user traverses from one item to another, ensure that the information also transitions accordingly .
In case of a pop-over, ensure that the pop-over changes according to selection of menu keys on the top of the screen.
If the pop-over involves character inputs, then the virtual keyboard should appear automatically on selecting a text
input field and should disappear in a non-input field.
12
Case Study 4 : Enterprise Mobility & Bring your own devices
In order to regularize the cost of infrastructure and encourage employees to use their personal devices for
Enterprise application access, Bring Your Own Device policy has been implemented in most of the
organizations. Most of the employees use “Tablet” (read: iPAD) as part of BYOD.

Validation of IT policy enforcement on these devices and ensure their correct functionality. For
example, employees cannot access P2P file sharing applications while connecting to enterprise network
through their devices.

Validation of remote wipe – If the device is lost (or) compromised, IT team effects a remote-wipe
feature to wipe out data in the target device.

Performance monitoring and response time verification for enterprise applications while connected
through BYOD. Since the request goes through multiple hops (may be through virtual desktop interface
or to a web server), the performance shouldn’t be too slow to hamper the operational activities while
performed through own devices.

Data synchronization check & reconciliation of data between Desktop and tablet.
13
Summary of practical usability challenges in a tablet
 User expects data but encounters blank screen
 Traversing several screens for task completion
 Bad Response time due to incorrect UI design
 Total time spent on a flow is unusually high
 Improper usage of personal information
 Application requires lot of gestures/user inputs
 Clarity of display - font, picture size/orientation
 Lack of error message or incorrect message
 Accuracy issues – not fitting screen, errors etc.,
 Inconsistent application behavior
 Lack of clean/graceful exit during exigencies
 Application ‘forgets’ local configuration settings
 Transaction/action cannot be reversed/recalled
 Lack of appropriate help messages
 Short cut to home not provided in each screen
 Application reacts badly to interrupts
 Navigation steps very difficult to memorize
 Complex app leading to low user confidence
 Text/image combination not rendered properly
 User finds it difficult to adjust to change of device
14
Conclusion – Lessons learned & best practices from tablet testing

A clear test strategy should be in place for testing the tablet applications across various operating
systems (Galaxy Tab Vs iPAD Vs Playbook), browsers (Safari, IE, Mozilla) etc., Web apps are expected
to perform the same across platforms but the subtle variations specific to operating system concept
should be captured as part of test strategy – for example Android Version 3.2 and above has a feature
to set the download limit for a particular application screen.

Measurement of response time – Tablet users expect the response time to be equal to or less than that
of the response time while accessing a laptop or desktop application. The expected response time is
roughly in the range of 2 seconds 1

Delineating Application performance from Network performance is very critical. Tablet application
should be subjected to variable network conditions (delay, jitter, packet loss, data loss, bandwidth
constraint etc.,) and ensure that the application performs a graceful exit at the worst case scenario
while performing a transaction. Also, the application response when the network switches from 3G to
Wi-Fi or vice versa should be tested.
15
Conclusion – continued…

Each operating system will have its own set of UI guidelines and best practices to be followed for
developing the application. This should be used as an input while framing user experience test
scenarios.

Automation tool should be selected based on the client’s requirement and pain points. Few tools in the
market would require the device to be rooted (in case of Apple iPAD) and few other tools would require
access to the source code for performing image based automation. Legality aspects should be duly
verified before proceeding with automation.
16
Q&A
17
References

Engaging the Tablet User: What They Expect From Web Sites – A Tablet user survey by Compuware

www.w3c.com

Apple & Android developer forum

Infosys Project Experience.

www.wikipedia.org

www.google.com
18