Information Software Systems
Model Driven Engineering in modern Software
Development Process
Vlad Volkov
Program Manager
Information Software Systems
18 May 2007
ISS
Modern SW Development Practices: Processes and Technologies
(c) 2007 Information Software Systems
18 May 2007
Information Software Systems
Contents






Who we are: short overview of model driven engineering
projects at ISS
Problem and solution
Why verification?
Verification strategy
Subsequent tests generation
What is next?: conclusion
ISS
Modern SW Development Practices: Processes and Technologies
(c) 2007 Information Software Systems
18 May 2007
2
Information Software Systems
Who we are: short overview of model driven
engineering projects at ISS



1999-2000: first Motorola project on automated
requirements capturing conducted by research group from
Glushkov Institute of Cybernetics headed b y Alexander
Letichevsky and Julia Kapitonova
2000: ISS begins his operations, Motorola locates a project
with a goal to create high-quality software product aimed at
automated support of the initial phase of software project - deriving of consistent and complete specifications from
initial requirements
2001: In parallel, ISS is requested by Motorola to define
and institutionalize SWD practices of SEI CMM Level 3
ISS
Modern SW Development Practices: Processes and Technologies
(c) 2007 Information Software Systems
18 May 2007
3
Information Software Systems
Who we are: short overview of model driven
engineering projects at ISS (cont.)




2002:first product release of VRS (Verification of
Requirements Specifications) tools for checking formal
specifications
2003: first successful piloting of VRS tools in industrial
projects of Motorola projects
2004: Developed possibilities for automated tests
generation with respect to initial MSC specifications
2004-2006: Participation in Motorola projects on automated
code C/C++ generation from models written in SDL and
UML.
ISS
Modern SW Development Practices: Processes and Technologies
(c) 2007 Information Software Systems
18 May 2007
4
Information Software Systems
Software Development Catastrophes*








CONFIRM travel information system: $160 million
Denver Airport baggage handling: $200 million
London Stock Exchange’s Taurus: £400 million
London Ambulance Service dispatching: £9 million
Pentium FDIV bug: $500 million (found with formal
analysis!)
average schedule slips by 50%
25% of all large systems are cancelled
3/4 of all large systems are operating failures
*(from Software Engineering Computer Science Tripos Part IA, Part II (General) Lent Term,
1997,Lawrence C Paulson Computer Laboratory University of Cambridge Copyright c 1997 by
Lawrence C. Paulson)
ISS
Modern SW Development Practices: Processes and Technologies
(c) 2007 Information Software Systems
18 May 2007
5
Information Software Systems
The problem and the solution

Inevitable uncertainties and conflicts in the behavioral
requirements for large and very large projects cause
enormous delays in releasing high quality products and
unwanted increase of Cost-Of-Quality

Manual analysis of behavioral/functional requirements in
large scale projects becomes a stopper

Formalization of behavioral requirements in MSC/SDL/UML
and their formal verification for consistency and
completeness may help to improve their quality at the
initial phases of a software project
ISS
Modern SW Development Practices: Processes and Technologies
(c) 2007 Information Software Systems
18 May 2007
6
Information Software Systems
Where are the costs incurred?
In requirements!
Faulty requirements have
significant impact **
Cost of fixing requirements defects
increases exponentially with time *
[B.Boehm]
Cost to fix
error
requirements
design
modeling and coding
25%
50%
25%
Requirements/
Design
ImpleTesting
mentation
* Barry W.Boehm, "Software Engineering
Economics", Prentice-Hall, Inc., 1981,
ISS
** Industry post release defects analysis,
2004


Requirements Management
Requirements Verification
Modern SW Development Practices: Processes and Technologies
(c) 2007 Information Software Systems
18 May 2007
7
Information Software Systems
40%-20%-40%



Well known estimate of efforts distribution
through software development phases (Design –
Coding – Testing) :
40%-20%-40%
- Maximum effect of efforts reduction shall be
achieved on Design and Testing phases. Main
investments into code/test automation and
generation should be made on these phases.
Now Test automation is a widely accepted
practice in industry. Verification of Requirements
& Specifications only started to be used.
ISS
Modern SW Development Practices: Processes and Technologies
(c) 2007 Information Software Systems
18 May 2007
8
Information Software Systems
Business gains with VRS



1. Increase of product quality
due to more defects found
2. Decrease of Cost-OfQuality due to earlier finding
of defects
1. Shorter time to market
due to cycle time reduction
Defects
Gain in σ-quality
New
Old
Gain in
COQ &
CTR
Time
Release date
Conditions of efforts reduction:
- Usage of formal engineering languages on the Design stage
- Total efforts reduction in SW projects is achieved by reduction of
design and testing manual efforts
ISS
Modern SW Development Practices: Processes and Technologies
(c) 2007 Information Software Systems
18 May 2007
9
Information Software Systems
Why verification?
Program testing can be used to show the
presence of bugs, but never to show their
absence!” (E. W. Dijkstra, 1979)
ISS
Modern SW Development Practices: Processes and Technologies
(c) 2007 Information Software Systems
18 May 2007
10
Information Software Systems
Verification strategy


Formalization and verification is hidden from the user
Input captured in engineering notations: MSC, SDL, UML




Specification of system behavior
Dynamic properties of the system
Library of predefined properties
“It should never be the case that a
phone connects to a phone that is
not ringing because of this call.”
Verifier operates fully automatically
Requirement
Capture
Informal
MSC
Reqmts.
SDL
(DOORS)
UML
Properties
UK
Specs
in
formal
logic
A&~B&(c=>D)
F0=>c\/~D
ISS
Verdict
Modern SW Development Practices: Processes and Technologies
(c) 2007 Information Software Systems
USA
RMTR
air_in
taxi_in
taxi_out
air_out
Scenarios violating
properties or traces
for residual testing
18 May 2007
11
Information Software Systems
Classification of defects
documentation


Defects found in
system state space


Transition inconsistency – nondeterministic behavior – equivalent preconditions, but different actions afterward

Safety – violation of a specified safety
(liveness) property – the timer is stopped
only if it was started; or “the lift door shall
be always closed while the lift is moving”

Timing violation – mismatch of specified
event ordering with respect to their
timing provided in absolute or relative
units
basic protocols
annotations
timing
ISS
Discrepancy – document problem – wrong
reference, absence of information, etc.;
typos and obvious slips are not counted!
Unreachability – the system will never be
in that state – analog of “dead code”
Deadlock – from this system state no
further transition is possible, usually
some incompleteness in requirements
Modern SW Development Practices: Processes and Technologies
(c) 2007 Information Software Systems
18 May 2007
12
Information Software Systems
Piloting VRS
Project
Telecom
municati
on 1
Telemati
cs 1
Telecom
municati
on 2
Telecom
municati
on 3
Telemati
cs 2
Telemati
cs 3
Telemati
cs 4
Telcom
municati
on 4
ISS
Defect
s
found
Generated
traces with
counterexamples
Effort
in staffweeks
Estimat
ed
COQ/
COPQ
reducti
on
50%
11
0
5.5
45%
70
100%
10
3
5.6
No data
730
192
100%
18
7
20
50%
~1500
56
50%
8
5
5.5
40%
323
219
60%
38
8
3
45%
116
42
3
1
0.7
40%
~1500
3005
129
7
22,5
55%
~2000
2311
223
17
21,3
60%
Reqs &
related
docs in
pages
Number of
MSCs in
formalized
specs
400
127
200
Coverage
of
original
reqs
100%
100%
100%
Modern SW Development Practices: Processes and Technologies
(c) 2007 Information Software Systems
18 May 2007
13
Information Software Systems
Benefits of testing based on verified
specification







Decrease of test efforts by up to 50%
5X decrease in manual efforts for verifying the product quality
Organize smooth testing life-cycle based on work with graphical
specifications only
Increase test suites reuse ratio (more than 50-60%) and support test
suites reuse
Increase confidence in absence of hard-to-find errors
Increase test coverage while reducing testing time. Generated test
suite will be minimal in size and maximal in coverage.
Generating from correct formal specifications 100% stand-alone tests
and test environment for C, C++, Java and various scripting languages
and platforms.
ISS
Modern SW Development Practices: Processes and Technologies
(c) 2007 Information Software Systems
18 May 2007
14
Information Software Systems
Where VRS could be applied?


The tool is ready for application in large-scale projects
(thousands of functional requirements) from Telematics and
Telecommunication domains.
Additional adjustment is needed for Automotive and
detection of security issues in Inet protocols.

ISS
Modern SW Development Practices: Processes and Technologies
(c) 2007 Information Software Systems
18 May 2007
15