Software Engineering
Research paper presentation
Ali Ahmad
•Formal Approaches to Software Testing
•Hierarchal GUI Test Case Generation
Using Automated Planning
Formal Approaches to Software Testing
P. Dasiewicz
IEEE Canadian Conference on
Electrical and Computer Engineering
2002
Introduction

Software Testing: checking and validating the
correctness of software




Formal methods



Time consuming
Difficult
Error prune
Specifying and verifying software systems using
mathematical and logical approaches
Target: proof of correctness
Some verification techniques


Model Checking
State space exploration
Introduction

Current methods are not feasible



Too much manual effort
Models that are very complex for analysis
It is needed to overcome the following



(cont’d)
The state explosion problem
Cost of creating models
Applicability: Design level


Less complexity
May reduce maintenance costs
Introduction

UML




(cont’d)
Simple, general purpose, visual modeling
Specify, visualize, construct and document
UML-statechart  basis for dynamic
behavioral description
Objectives:


Overview model checking to the validation
and test case generation (using UML
statecharts and interaction diagrams)
Applicability of formal methods to component
integration and interaction testing
Integration/Component Level Testing

Offutt




Changes in component states due to change
events
UMLTEST, integrated with Rational Rose 
Highly effective test cases can be generated
for system level specification
Test prefix: ensure that the system is @ a
certain pre state before performing the test
Issues:
• Interaction is not considered
• Some states may never been entered

Incorporated UML collaboration diagrams
Integration/Component Level Testing
(cont’d)

Yoon



Test case generation based on UML sequence
and collaboration diagrams
A node that represents both integration target
and message flow
Testing technique
•
•
•
•
•

Extract sequence diagram
If concurrency, extract collaboration diagram
Divide into ASF (Atomic System Function)
Extract nodes and message flows
Obtain test cases by applying test criteria (e.g. all-edge)
Limitations
• May require a large number of sequence diagrams
• Test case coverage (only normal or abnormal flow)
Integration/Component Level Testing
(cont’d)

Kim



Divide statecharts into extended finite state
machines
Control flows are represented as paths
Coverage criteria
• Path coverage
• State coverage
• Transition coverage

Test cases
• breadth or depth first searches
• can be generated using the data flow on the UML
statechart
• Determine whether class implements correct control and
data flow
System/Integration Testing

Hartmann





Testing the interfaces among several
components
UML statecharts  Mealy Finite State
Machines with restricted point to point
synchronous communication model
Incremental composition and reduction
algorithm
Test cases: category partition method
Limitations
• Interaction are modeled as synchronous communications
• Event exchanges contain no parameters
• No support for nested machines
Source Code Based Testing



Major problem: State space explosion
problem
Reducing the state space entails
eliminating irrelevant code segments
Hatcliff



Program slicing techniques
Used by Bandera for translating Java code to Jimple, e.g.
SPIN, SMV  Generates a finite state model for the reduced
code
Java Path Finder



Integrates model checking, program analysis and Testing
Goal: Apply formal methods at source level
Initially, check for safety properties like deadlocks (LockTree)
Translation of Statecharts

Promela: input modeling language for
SPIN


C-like, extended with non-deterministic, and
loop guarded constructs
Latella presents a translation from UML
statecharts into Promela, where statecharts
are first converted to hierarchal automata. A
proof of correctness is also given
Component Interaction Testing


Major issue: Are the components
developed separately work properly
together
Formal methods of component interaction



Define testing requirements
Automatically generate the test cases using model checking
Focus of author’s research:




Component interaction of software systems
Detect subtle interaction errors without duplicating the work
@ unit level
Underlying model: Labeled transition system
ObjectState: modeling language to show the feasibility of
creating formal models, it relies on hierarchal FSM.
Hierarchal GUI Test Case Generation
Using Automated Planning
Memon A., Pollack M., Lou Soffa M.
IEEE Transactions on Software
Engineering 2000
Introduction

GUI testing is difficult



The interaction space is enormous
Determining the coverage of test cases
Regression testing is a major challenge

Automation is necessary for generating
GUI test cases

PATHS: Planning Assisted Tester for
grapHical user interface Systems


Input: Possible goals for GUI user
Generates sequences of events to satisfy the user
goals, these become test cases
Introduction
(cont’d)

PATHS performs automated analysis of the
hierarchal structure of the GUI to create
hierarchal operators that will be used during plan
generation

Contribution to Research





Make use of an AI technique (Automated Planning)
Exploits GUI structural features
Makes Regression testing easier
The test suite is portable
Allow the reuse of operator definitions that commonly
appear across GUIs
Overview



GUI consists of components like labels, buttons,
menus, and pop-up lists
Example: Microsoft WordPad
GUI has 2 types of windows


GUI windows
Object windows,
doesn’t contain any
window components
Overview

(cont’d)
PATHS- Plan Generation

Inputs
• Initial state
• Goal state
• A set of operators applied to a set of objects
• Preconditions and effects


Solution to a Planning Problem: sequence of
instantiated operators which result in the goal state
when executed in the initial state
2 Phases
• Setup
• Create a hierarchal model of the GUI
• Plan Generation
• Specify scenarios (initial and goal states)
• PATHS generate a test suit for the scenarios
Overview
(cont’d)

Role of Test Designer and PATHS during test generation

GUI Events and Planning Operators
Overview

(cont’d)
Operator Event Mapping
Abstract: Invokes a window
that monopolizes the user
interaction

Example operator: Edit_Cut
Overview

High level plan

Expanded plan
(cont’d)
Plan Generation
GUI test case generation can be modeled by
hierarchal plans that doesn’t require conflict resolution
Planning GUI Test Cases

Developing a representation for the GUI and its operators

Operator Derivation Process
• Traverse the GUI and press on its components, the label is read
off the component label

GUI events
•
•
•
•

Menu open (File, Edit)
Unrestricted focus (Basic shapes in PowerPoint)
Restricted focus (Edit-Preferences in PowerPoint)
System-interaction (Cut and paste)
Planning Operators
• System-Interaction
• Abstract


Modeling Initial and Goal States and Generating Test Cases
using algorithm
Generating multiple plans in PATHS


Creating multiple linearizations of the partial plans
Repeating the planning process, thus generating a different test case
Planning GUI Test Cases
(Cont’d)
Planning GUI Test Cases
(Cont’d)
Planning GUI Test Cases
(Cont’d)
Planning GUI Test Cases
(Cont’d)
Some Experimental Results

Multiple Tasks

Hierarchal vs. Single Layer
Questions