Softeam’s testing
process: a story
Cyril Ballagny, SOFTEAM | ModelioSoft
The 13th CREST Open Workshop
London, England
13th of May 2011
www.modeliosoft.com
Introduction to Softeam
 A French SME (about 230 people)
 More than 20 years of expertise in OO development and
Modeling
 Consulting, Training, Engineering activities
 Software Publishing: Modelio and Objecteering
 Strong R&D efforts since its creation
– Innovator since 1989 (Modeling, MDA, UML)
– Heavily involved in R&D cooperative projects (FITTEST, MADES,
ENOSYS…)
– Participates in the OMG standardization efforts since 1996 (UML
profiles, SOAML, SysML…)
www.modeliosoft.com
2
Introduction to Modelio
 A CASE tool
 For UML modeling
 For MDA application
 For Code generation (Java,
C#, C++…)
 For documentation
generation (HTML, word…)
 For team work and project
sharing
 And much more (BPMN,
SysML, SOAML…)!
www.modeliosoft.com
3
Modelio development process




Agile development with bootstrapping and continuous integration
20 people involved
1 million LOC (C++/Java)
A new release every 6 months
www.modeliosoft.com
4
Modelio testing process (1)
 Code quality management with Sonar: about 500 coding
rules
 Continuous integration with Hudson: report broken build
(inconsistent API, missing source in the repository…)
 Bug tracking with Mantis: everyone from the team can
report bugs
 Specification of test plans with Test Link (natural
language): about 2000 procedures
 Bootstrapping: Modelio is validated by developers
www.modeliosoft.com
5
Modelio testing process (2)
 A test case is either
 A Mantis bug (bug 2123, bug 9478…)
 A menu item (create, delete, mask…)
 A diagram artifact (state, region, activity…)
 Most test cases are UI-based and derived from general
scenarios (e.g. apply command “xxx” to element in
{a,b,c…} )
 Test case execution is externalized to Eastern Europe
www.modeliosoft.com
6
Our issues
 Specification often changes
 Writing operational test specification is too time consuming
compared with natural language specification
 Test code must be tested too
 Mix of languages (C++/Java/Jython)
 No tools for testing HMI combining Java and C++
www.modeliosoft.com
7
Our expectations (1)
 Reduce over-testing
 Detect overlapping sections of test cases
 Recombine sections to constitute smaller test cases with greater
coverage
 Apply regression tests only on parts of the system that have
changed
 Reduce under-testing
 Detect system parts that are not covered by the test cases
 Detect sequences of interactions allowed by the GUI that have
not been tested
www.modeliosoft.com
8
Our expectations (2)
 Reduce Time-To-Market
 Generate test cases instead of specifying them
 Prioritize:
 Identify the most used features
 Identify the most frequent bugs
www.modeliosoft.com
9
To sum up
 We would like a scalable and pragmatic approach
whose ROI has been demonstrated
www.modeliosoft.com
10
Thank you for your attention!
Andrey Sadovykh
SOFTEAM | ModelioSoft
andrey.sadovykh@softeam.fr
SOFTEAM R&D Web Site:
http://rd.softeam.com
ModelioSoft Web Site:
http://www.modeliosoft.com
www.modeliosoft.com
11
M o d e l i n g s o l u t i o n s.