Model-based Testing
Effective Test Case Design
And Test Automation
Twin Cities Quality Assurance
Association (TCQAA)
September 12, 2013
Yaxiong Lin
TestOptimal, LLC
Agenda
What is Model-Based Testing (MBT)
 MBT in Action (demo)
 Benefits of MBT
 MBT Process
 MBT Use Cases

Testptimal LLC
Agenda
What is Model-Based Testing (MBT)
 MBT in Action (demo)
 Benefits of MBT
 MBT Process
 MBT Use Cases

Testptimal LLC
Business Impacts of Software Defects
• U.S. alone: $60 billion per year (NIST 2002)
Source: Crosstalk,
the Journal of Defense Software Engineering
Testptimal LLC
Where Defects are Found
Source Unknown
Testptimal LLC
Costs of Fixing Software Defects
The Real Cost of Software Defects, LKP Consulting Group
http://www.lkpgroup.com/Cost%20of%20Software%20Defects.pdf
Testptimal LLC
Software Testing Challenges
Requirement/design defect impact: 64%
 Non-developer testing catches 69% of
defects
 Costs 100x to fix defect in production
 Gain higher confidence

 better test coverage
 requirement traceability
Testptimal LLC
What is Model-Based Testing?
“Model-Based Testing is the Automation
of Test Design of Black-box Tests”
Practical Model-base Testing”, Mark Utting and Bruno Legeard
Testptimal LLC
What is Model-Based Testing?
“Model-Based Testing is the Automation
of Test Design of Black-box Tests”
Practical Model-base Testing”, Mark Utting and Bruno Legeard
Model-based testing is application of Model based
design for designing and optionally also executing
artifacts to perform software testing. Models can be
used to represent the desired behavior of a System
Under Test (SUT), or to represent testing strategies
and a test environment.
wikipedia
Testptimal LLC
Why Model?
Visual representation of complex systems
 Ask What-If to understand cause-effect
 Important role in the technology
advancement and software development
 Software testing is a complex task

Testptimal LLC
MBT Modeling Notations

Finite State Machine (FSM)
 State Diagram (UML)
Extended Finite State Machine (EFSM)
 Message Sequence Chart (MSC)

 Sequence Diagram (UML)

Control Flow Graph (CFG)
 Activity Diagram (UML)
Event Flow Diagram
 UML Testing Profile – UTP
 …

Testptimal LLC
Testing Vending Machine
Drink for $1
 Accept Quarter and HalfDollar coins
 Cancel any time for refund

Testptimal LLC
Vending Machine Test Cases
Quarter + Quarter + Quarter + Quarter => Vend
 Quarter + Quarter + HalfDollar => Vend
 Quarter + HalfDollar + Quarter => Vend
 Quarter + HalfDollar + HalfDollar => Vend
 HalfDollar + Quarter + Quarter => Vend
 HalfDollar + HalfDollar => Vend
 HalfDollar + Quarter + HalfDollar => Vend
 Quarter + Cancel => Refund
 …

Testptimal LLC
Example Model
Testptimal LLC
Generated Test Sequence
Testptimal LLC
Agenda
What is Model-Based Testing (MBT)
 MBT in Action (demo)
 Benefits of MBT
 MBT Process
 MBT Use Cases

Testptimal LLC
Types of MBT

Offline Testing

Online Testing
Testptimal LLC
Demo:
Test Case Generation for Manual Testing
or Offline Testing
Testptimal LLC
Demo:
Login Test Automation – Online Testing
Testptimal LLC
Demo:
WebStore Test Automation – Online
Testing
Testptimal LLC
Testing Web App
Testptimal LLC
WebStore MBT Model
© 2012 Testptimal LLC
Generated Test Sequence
Testptimal LLC
Agenda
What is Model-Based Testing (MBT)
 MBT in Action (demo)
 Benefits of MBT
 MBT Process
 MBT Use Cases

Testptimal LLC
Recent MBT User Survey
by Robert Binder, System Verification Associates,2012
MBT reduced escaped bugs by 59%.
 MBT reduced testing costs by 17%.
 MBT reduced testing duration by 25%.

Testptimal LLC
Benefits of MBT
Catch requirement / design defects
 Catch defects earlier
 Reduce defects in fields
 Improve efficiency (Agility)

 handling changes
 better communication on defects

Gain higher confidence
 better test coverage
 requirement traceability
Testptimal LLC
Agenda
What is Model-Based Testing (MBT)
 MBT in Action (demo)
 Benefits of MBT
 MBT Process
 MBT Use Cases

Testptimal LLC
MBT Iterative Development Process
Requirement
s
User Stories
Modeling
Analysis
MBT
Models
Execution
Test
Generation
Automation
Testptimal LLC
MBT Process Input/Output
Input
MBT Step
Output
Requirements
User Stories
Modeling
Behavior Models
Test Objectives
Test Generation
Test Strategies
Test Coverage
AUT/Spec
Automation
Automation Scripts
Environment
Configuration
Execution
Test Coverage
Defects, Perform. Stat
Execution Result
Analysis
Model Modification
Testptimal LLC
Model-Based Testing Process
http://journal.thedacs.com/issue/52/145
Department of Defense (DoD) Information Analysis Center (IAC)
Testptimal LLC
Agenda
What is Model-Based Testing (MBT)
 MBT in Action (demo)
 Benefits of MBT
 MBT Process
 MBT Use Cases (more demo)

Testptimal LLC
Applications of Model-Based Testing
Conformance Testing
 Functional Testing
 System Integration Testing
 Security Vulnerability Testing

Testptimal LLC
MBT Use Cases (demo)
Finding Temporal Defects
 Realistic Load Simulation

 Performance
 Load/Stress Testing
Session /Scenario Based Testing
 Risk Based Testing
 Quick bug fix verification

Testptimal LLC
Questions?


MBT Papers:
http://testoptimal.com/MbtReferences.html
Email: yaxiong.lin@testoptimal.com
Testptimal LLC