Multi-Paradigm Models
as Source for Automatic
Test Construction
Victor Kuliamin
ISP RAS, Moscow
Why Multiple Models?
Requirement
s
Functionality
?
Efficiency
Usability
2,000
1,800
1,600
1,400
f1()
1,200
f2()
1,000
accept()
0,800
read()
0,600
modifyAll()
0,400
0,200
0,000
1
2
3
5
10
15
20
Testing
Reliability
Modeling Techniques

Operational
Can be executed by virtual machine

Contract
Pre- and postconditions, data integrity
constraints

History-based
Constraints on possible traces

(C)(E)FSM, LTS, PN, CSP, ASM
SDL, LOTOS, Lustre, VDM,
Murphi, Simulink
Z, B, ADL, JML, Eiffel, VDM, RSL
Larch-C++
TL, MSC
Algebraic
Equivalence between different execution
histories
Larch, ML, OBJ
Tasks of Testing
Software
under Test
Construct
Single Test
Input
Organize
Bundle of
Test Inputs
Test Results
Evaluate
Correctness
Evaluate
Testing Quality
Gather
Responses
Transform Test
Inputs and
Responses
Modeling Techniques Comparison
Behavior Evaluation
Closeness to Requirements
High-level Coverage
Low-level Coverage
Scalability
Concurrency
Operational
 Contract
 History-based
 Algebraic

Test Sequence Construction
Single Input Construction
Comparison Results
There is no the best technique
 No one technique is good for everything
 May be a mix of different approaches can
fit more needs?

UniTesK Technology
Model-based testing technology
Developed in 2000 – 2002 in
ISP RAS
UniTesK Solutions

Contract specifications of behavior

FSM and LTS testing models
Contract Specifications
Functional
Requirements
Preconditions and postconditions
of interface operations and
asynchronous reactions
Data integrity constraints



Contract Specifications
Close to requirements
Suitable for oracle generation
Provide low-level coverage
criteria
FSM and LTS Testing Models
Contract Specifications
!
Coverage Requirements
Define states and admissible input
actions
More abstract than original
specifications

?
!
?
?
!
!
?

?

Guarantee some low-level
coverage
Suitable for test sequence
construction
Provide high-level coverage
criteria
Relation between Models
parameters
operation domain
2
3
coverage goals
1
states
Whole Picture I
Coverage Model
Model of Behavior
Testing Model
Software under Test
Test Oracle
Test Sequence
Construction
Whole Picture II
Operation
Operation
Event
Data
Software under Test
Operation
pre
post
Operation
pre
post
Event
pre
post
Data model
invariants
Model of Behavior
Coverage Model
Scenario method
Scenario method
State
Calculation
Testing Model
Tool Demo
Set of Integers – Scenario I
1
2
3
0
7
1
3
5
2
States of behavior model
5
States of FSM model
Mapping Abstract Call to Specific
parameters
2
3
1
states
current state
Set of Integers – Scenario II
1
2
0
7
1
3
5
2
States of FSM model = States of behavior model
Failure
{ -2147483648, 2147483647 }
Add ( -715827883 )
/ false
References
1.
2.
3.
4.
5.
6.
V. Kuliamin, A. Petrenko, N. Pakoulin, I. Bourdonov, and A. Kossatchev. Integration
of Functional and Timed Testing of Real-time and Concurrent Systems. Proc. of
PSI 2003. LNCS, Springer-Verlag, 2003.
V. Kuliamin, A. Petrenko, I. Bourdonov, and A. Kossatchev. UniTesK Test Suite
Architecture. Proc. of FME 2002. LNCS 2391, pp. 77-88, Springer-Verlag, 2002.
A. K. Petrenko, I. B. Bourdonov, A. S. Kossatchev, V. V. Kuliamin. Experiences in
using testing tools and technology in real-life applications. Proceedings of
SETT’01, India, Pune, 2001
I. B. Bourdonov, A. S. Kossatchev, V. V. Kuliamin. Using Finite State Machines in
Program Testing. "Programmirovanije", 2000, No. 2 (in Russian). Programming
and Computer Software, Vol. 26, No. 2, 2000, pp. 61-73 (English version)
I. Bourdonov, A. Kossatchev, A. Petrenko, and D. Galter. KVEST: Automated
Generation of Test Suites from Formal Specifications. Proceedings of World
Congress of Formal Methods, Toulouse, France, LNCS, No. 1708, 1999, pp. 608621
http://www.ispras.ru/groups/rv/rv.html
Contact
Victor V. Kuliamin
E-mail: kuliamin@ispras.ru
109004, B. Kommunisticheskaya, 25
Moscow, Russia
Web: http://www.ispras.ru/groups/rv/rv.html
Phone: 007-095-9125317
Fax: 007-095-9121524
Thank you!