Module Based Regression Test Selection
Strategy for Web applications
Author: Annamariale Chandran
Honeywell Technology Solutions
For International Conference on Software Engineering
19th Dec 2009
Software Life cycle
Requirements
Acceptance Testing
Specification
Document
Final
System
Software
Requirements
Update
Traceability
Design
High Level Design
Low Level Design
Design
Document
Implementation
Source
Code
System testing
System Test cases
Unit/Integration testing
Unit Test cases
Maintenance
Testing Vs Regression Testing
Testing aided by
Requirement spec &
design document
How far test cases and execution
are performed in terms of spec
Applied testing techniques
In terms of Quality
Regression aided by
Collation of available artifacts
and proper impact
Analysis of new changes
Depends also on what extent
earlier documents are available
Analysis on Impacts
Regression techniques used to
derive test cases
Regression Testing
Used to validate software after some modifications
are incorporated in the application
Aims at validating not only the changes, but also the
impacted functionalities due to those changes
Process of validating modified software to provide
confidence that changed parts of software behave as
intended and un-hanged parts of the software have
not been adversely affected by the modification
Applicable Regression Testing Areas
Types of Project Development
Project development from
Scratch
Contracted software
Adopting some common
Framework
Using new trends like SOA,
Cloud computing
Full Life cycle ->
Successive build
Iteration releases
Maintenance activities
Enhancements
Any software Migration
General Hurdles
Non-availability of updated spec and test artifacts in a
single repository
Allocation of dedicated test resource
Use of specification Vs test case traceability is time
consuming when subject under test grows
In practice, use of dedicated effort for deriving impacts is
rarely possible as agile environment demands things to
move faster
Testers will be allocated for more than one project, more
than one task at the same time
Factors to be considered
Quality of derived Impacts
Productivity factor in achieving the quality
Should reduce test design time
Should reduce application knowledge
transfer time
Should provide inputs on impact to presystem phases to improve DRE
As-Is techniques & Drawbacks
Code Based
Re-test All
Specification based with
activity diagram
Risk based
Dependency with dev
team
Adhoc walkthrough
Scalability problem when
size grows
Increase in effort
Downturn in updating new
spec items
Depends on individual
skill level
Misses in finding Impacts
Ineffective review for the
derived Impacts
In Nutshell
Current Difficulties
In reality changes are validated on
independent test document
In most cases original test cases (either
steps, expected output)
will be changed
Skill level of resources will not be same
in understanding the test cases to know
high level functionality
To adopt some sort of regression
guidelines every time to extract impacts
Expectations
Instant understanding of high
level functionalities of derived
test cases
The module in which it falls
Understanding existing
scenarios from the extracted test
suites
The priority & severity of
those test cases
To provide impacts
automatically (needless to use
Guideline every time)
Try for new approach??
Basis -> Adhoc thought process while deriving impact
Module Based Regression Testing
What Theory says?
Regression analysis aims at measuring the relation of independent
variable to its dependent variable's so as to predict the future value
of dependent variable
Derived concept for Web based application
to perform Regression Testing
Future value of dependent
Variable (any change) is
Identified in terms of its
related dependent
variables
Dependent and independent variables
are considered as modules in the existing
application. To derive the regression curve
between the modules, factors that lead to
Interlink the modules is considered.
Regression Logic
Module relation with primary dependent variables
y(m) = f(x(p), β)
-----------------(1)
Where,
y(m) -> modules derived in the application under test
x(p)->primary dependent variables identified for the derived modules
β -> intelligent guessing function that ensures the correctness of mapping the primary dependent variables
to the identified modules
Final parametric relation between modules
Yd(M) = yid(m)f(α)
------------------- (2)
Where
Yd(M) ->dependent module that will get impacted another module in the application
yid(m) ->independent module referred at that instant
α
-> derived function from equation (1) to differentiate Yd(M) and yid(m)
Sample Application
Employee Request Management System
Request type1
Login
User applying for
specific request
Start
Request type2
Approval
Process
Request type3
Request type.. n
Keep track
Of records
End
Step1a: Identify Modules
Step1b: Map High level Functionalities
Objective: Derive test scenarios for the specified modules
‘Request Form’ high-level functionality
User details should be auto-populated from LDAP
for the logged in user
Mail Id field should alone be allowed for editing
Manager ID should be same as that of user’s portfolio
Data requirement validation (Refer to xxx SCM
location)
Mail should be sent to the primary approver
when request is submitted and status change
to PA (Pending primary approval)
Step2: Identify primary dependent variable1
Extract dependent variable1-Fields/Sections
(that varies based on the domain nature)
Step2: Identify primary dependent variable2
Extract dependent variable2-Common functionality
Step3: Module Linkage for dependent
variables
Step4: Final regression plot
In Practice…..
Frequency of
When Module is changed/removed
When identified top level functionality changes
When any fields or sections changed
No Need of
Updating/ linking any user defined test cases
Updating any low level changes in requirements
Ex: Display format change of user name
Any addition of filter fields in reports
Results
MBRT-Effectiveness
Deriving primary dependent variables
Review on derived impacts -> more similar to
Specification/design reviews
Knowledge on frequency of updates
Use of both functional and structural analysis for impacts
Identification
In case of support projects, use of agile practices
in an efficient way to perform reverse engineering
Derived Benefits
Test coverage is high within the limited time frame
Does not demand the resource to have complete
understanding of the system
Analyze the related impacts & come up the priority
wise modules and scenarios to be executed
Knowledge transfer time is less
Provides input to the code-level impacts (Reverse
engineering)
Estimation to validate the changes becomes more
accurate
Future Amendments
To roll out an automated selection of
impacted modules and scenarios
To apply regression logic for deriving
impacted code, by providing the code
related information to the requirement
database as one more primary
dependent variable