An Approach to Automated Software Regression Testing
Presented by
Adnet, Inc
Feb 2015

Introduction
•
Automates manual repeatable test processes using formal testing procedure
•
Controls test execution, comparison of actual outcomes to predicted outcome with test preconditions
• Saves time and deduces cost by automating repeated manual testing
Why Test Automation
•
Repeated manual test prone to human errors (overlooking)
•
Learning cost of new test teams or new members
• Regression test cycles are increasing with distributed development
• Wider acceptance of Agile model (needs frequent and more tests)
•
Manual test data setup with multiple data-sets increases complexity
2

•
Scope and Approach (each software, not enterprise)
•
What to Automate (often repeatable tests- regression tests)
•
Smoke Tests (undocumented exploratory tests)
•
Individual often repeated test cases
• More useful for key business function where accuracy required
• Easy accommodates variation in test datasets
•
What not to automate
•
Stress and Load Test
•
Need different tools
•
Different Strategy
3

•
Reduce Cost
• Reduce time in testing
• Reduce cost from automation
• ROI: Gain time in new function test
• Reduce investment by using Freeware Tools rather than Vendor based
(Options)
•
Automation strategy
•
Incrementally add more of the new functions for the next cycle of regression testing
•
First automate smoke tests and then individual test cases
•
Baseline scripts and reuse each time.
•
Develop a library for commonly used routines like login for reuse.
4
4

5

•
Record-Playback
•
Record once playback each time.
•
Data-driven testing
•
One script runs with multiple sets of data.
•
Modularity-driven testing
•
Common libraries for common functions like..
•
Login()
•
Check_VA_Enroll()
•
Check VA Eligibility()
•
Exit()
• Keyword-driven testing
•
Keywords can make up a script..
•
Initialization():Launch browser and login
•
Test1() :Test for a Requirement
•
Test2() :Test for GUI like labels, links
•
Exit() :Exit from the browser
6

Generic Functions library
HP ALM Test Resources Module
Object
Repository
Configuration
Files
Business Process
Library
Documentation
Test Data Files
Any supporting docs / user guides for the project
HP ALM Test Plan /Lab Module
Folders by
Release
Scripts are organized into folder named after releases
Automated
Tests (QTP)
Test Execution initiated from Lab
Test Machine / HEC App
Database client / drivers
Database drivers required in order query the database
Application runs in a browser
Run Scripts
Test Set
Test Set
Results
TestRunZ
TestRun
TestRun
TestRun
Results get uploaded back to Test Lab
Detailed logs
Results, logs and screenshots are stored locally for easy access
Screenshots
7
7

A snapshot of test result file in excel format.
A detailed log file will also be created .
Screenshots stored locally as well as in ALM.
For Illustration Only
8

600,00
522,33 522,33 522,33
522,33
500,00
400,00
300,00
270,00
200,00
270,00
100,00
0,00
70,5
CYCLE 1
5,50
CYCLE 2
7,50
CYCLE 3
7,50
CYCLE 4
7,5
CYCLE 5
7,5
CYCLE 6
MANUAL HOURS PER CYCLE
TOTAL AUTOMATION HOURS PER CYCLE
9

3000,00
2500,00
2000,00
1500,00
1000,00
500,00
0,00
CYCLE 1 CYCLE 2 CYCLE 3 CYCLE 4 CYCLE 5 CYCLE 6
CUMULATIVE MANUAL EFFORT
CUMULATIVE AUTOMATED HRS
CUMULATIVE SAVINGS
10

•
Automated results generated in Excel or HTML- not dependence on the test software and format easy to analyze
•
Manual tester released for new and on key features.
•
Automating regression test allow frequent small changes requests.
•
Automation framework provides flexibility to change existing
SQL queries (for test data pull).
•
Easy to customize test data case for changes in test scenario
•
Huge savings in manual effort
•
Screenshots captured traced sources of errors and easy to debug software
11

•
Higher initial cost
•
Higher skill set needed
•
More initial investment in infrastructure set-up
•
Availability of test data could be limited
•
Inconsistent application behavior
•
Application response times
12

•
Type of Tools
•
Commercial Tools:
•
Support and maintenance advantage
•
High cost
•
Freeware:
•
No support and depending on support groups
•
More labor intensive-higher implementation cost
•
Adnet-HEC In-house Expertise in
•
QTP
•
Rational Functional Tester
•
Test complete
•
Selenium
13

Shri Sinha
President
Adnet, Inc.
Shrisinha@Adnetcmm.com
703-598-4485
14