Requirements for the Success of Formal Methods
in Practical Software Engineering
Trial Lecture
Demissie B. Aredo
March 3, 2005
¾
¾
Introduction to Formal Methods
♦
Definitions and historical background
♦
Main concepts of Formal Methods
♦
Benefits of using Formal Methods: Why Formal?
Formal Methods in Practice
♦
State-of-the-Art
♦
FMs are not in the mainstream SW engineering
♦
Limitations of FMs in industrial settings
♦
Introducing FMs into SW Engineering Practice
♦
Success Criteria for Formal Methods
¾
Conclusion
¾
References
Demissie B. Aredo
March 3, 2005, Oslo
2
Introduction to Formal Methods
¾
What is Formal Methods?
♦
FMs are mathematically-based languages, techniques and tools
for specification, development and verification of software and
hardware systems.
Source: Free On-Line Dictionary of Computing
♦
FMs are varieties of scientific and engineering techniques for
rigorous reasoning about the correctness of computer
programs, chiefly based on the use of mathematical logic.
Source: Wikipedia, the free Encyclopedia of Software Engineering
Demissie B. Aredo
March 3, 2005, Oslo
3
Historical Background
¾
1830s: Ada Augusta Lovelace wrote proof of the programs she
wrote for Charles Babbage’s engines. (in 1977 the Ada
language was named after her by US DoD)
¾
1940s and 1950s: women like Adele Goldstine, Grace Hopper
etc. were recruited to the industry because of their
mathematical abilities.
"A ship in a port is safe, but that is not what ships are for. Sail out to sea
and do new things." --Grace Hopper [1].
¾
Late 1960s: Hoare and Floyd proposed axiomatic techniques
for analyzing specification of sequential programs
¾
1970s: Dijkstra, Pnueli proposed techniques to express
properties of non-deterministic and concurrent programs
¾
Interest for formal spec-in-the-large has been growing
Demissie B. Aredo
March 3, 2005, Oslo
4
Historical Timeline
¾
1947-48: Von Neumann and Goldstine propose six-step
programming process, starting with conceptualization of the
problem mathematically
¾
1959: Backus-Naur Form (BNF) for languages
¾
1960: Hoare's Algol 60 compiler
¾
1962: PetriNets
¾
1969: Hoare's 'axiomatic basis’
¾
1973: IBM's VDM (Vienna Development Method), Cliff Jones
¾
1978: Hoare's CSP (communicating Sequential Processes)
Demissie B. Aredo
March 3, 2005, Oslo
5
Historical Timeline
¾
1980: Formal semantics for Ada
¾
1983: SPARK for Ada (safety-critical systems)
¾
1985: RAISE Esprit
¾
1989: Model-checking techniques and tools (SPIN)
¾
1993: PVS?, Maude?
¾
1996: Formal definition of B; ISO standard for VDM
¾
2002: ISO standard for Z
Taken from: [2] Richard Sharpe, Formal methods start to add up once
again, Computing, January 8, 2004.
Demissie B. Aredo
March 3, 2005, Oslo
6
Main Concepts of FMs
¾
FMs is based on mathematical techniques
¾
SL with precisely defined syntax and semantics
¾
Improve reliability of critical systems, practitioner’s understanding of
system requirements and design
¾
Supports rigorous analysis of systems
♦
checking consistency and completeness of specifications
♦
verification (prove that an implementation conforms with its
specification)
¾
Allows development of semantically-based CASE tools
¾
Allow detection of ambiguities and inconsistencies in early phases, when
correction is less costly
¾
Used to model and analyze behavior of SW and HW interactions
Demissie B. Aredo
March 3, 2005, Oslo
7
Main Concepts of FMs
¾
Formal Method = Specification language + formal reasoning
¾
Specification language [Lamsweerde 2000]
¾
♦
Syntactic domain
♦
Semantic domain
♦
Inference rules
A specification language provides a notation, a universe of objects,
precise rules defining which objects satisfy each specification [Wing 90]:
♦
A spec language L can be represented as (SynL, SemL, Sat)
»
SynL syntactic domain of L
Sat ⊆ SynL × SemL
SemL semantic domain
» Sat is a satisfies relation specified by a precise rules to determine which entities
satisfy which specification
»
♦
FM = SL + well-defined inference rules
Demissie B. Aredo
March 3, 2005, Oslo
8
Main Concepts of FMs
¾
¾
¾
Verification techniques
♦
Model-checking, Testing, Simulation (finite state space)
♦
Theorem-proving (arbitrary state space)
Refinement
♦
Process of adding implementation details (e.g. choosing
specific algorithm) to achieve an acceptable efficiency
♦
Refinement steps are transformations that are guaranteed to
preserve correctness of the program
Composition/Decomposition
♦
Structural specifications by means of component relations
♦
Behavioral specifications of components
♦
Rules for compositionality
Demissie B. Aredo
March 3, 2005, Oslo
9
Main Concepts of FM: Tools
¾
Automated tool support
♦
¾
Reasoning about real-world size problem involves a large amount
of software artifacts - too large and complex to be handled
manually. Hence, automation of rigorous analysis is necessary.
»
Compositionality
»
Prototyping
»
Theorem prover
Model checking
♦
Approaches to handle state-space explosion in real-world systems
»
Abstraction
»
Selective and light-weight application of FMs
»
Hierarchical and systematic application of FMs
»
Automation of analysis process
Demissie B. Aredo
March 3, 2005, Oslo
10
Formal Specification
¾
A formal specification is description of a system and its desired
properties
♦
Functional properties
♦
Non-functional aspects
»
»
¾
response time and storage efficiency aspects
It serves as a means of communication between stakeholders
♦
¾
Safety and security aspects
Customers & specifiers, specifiers & verifiers, and specifiers &
implementers
Writing correct spec can be as difficult as writing correct code
♦
Easier at higher level of abstraction
♦
Should include useful implementation related information
Demissie B. Aredo
March 3, 2005, Oslo
11
Specification paradigms
¾
Temporal logic, e.g. Temporal logic of Actions (TLA)
♦
¾
¾
Properties are specified as temporal logic assertions about
objects; referring to current and future states
State-based specs, e.g. Z, VDM, B
♦
States are characterized as snapshots
♦
Properties are specified as
»
Invariants constraining objects at any snapshot
»
Pre- and post-conditions on operations
Transition-based specs, e.g. State charts, PROMELA, Maude
♦
Properties are specified by a set of transition functions in the
state machine transition
Demissie B. Aredo
March 3, 2005, Oslo
12
Spec Paradigms ...
¾
Functional specs
♦
Algebraic, e.g. OBJ, LARCH
♦
First-order logic
♦
Higher-order functions, e.g. PVS, HOL
♦
Rewriting logic, e.g. Maude
»
Specifications in Maude are executable
»
Built-in mechanisms for specification of distributed
synchronous/asynchronous systems
Demissie B. Aredo
March 3, 2005, Oslo
13
Specification Issues
¾
Unambiguous
♦
More than one interpretation should not possible
»
¾
¾
An order for a product must be received at least two weeks before
expected delivery. If it is delayed inform the customer.
Consistency
♦
Contradictory requirements should not be introduced
♦
At least one implementation must be possible
♦
Easily provable? (not all FMs support this)
Completeness
♦
Are relevant issues are addressed? What to do in case of failure?
♦
Allow implementation and design freedom, e.g. choosing data
structure and algorithm
♦
Not important in verification of a hardware component
Demissie B. Aredo
March 3, 2005, Oslo
14
Formal Verification & Validation
”Verifiable program ⇒ good program” -- Ole Johan Dahl
¾
¾
Verification is the process of showing, by reasoning, that a
system implementation fulfills the desired properties, as
described in a specification;
♦
Verification = ”building the system right”
♦
Assumption: a specification and an implementation are given
Two well-established approaches to verification
♦
Model checking (finite state-space)
»
»
♦
Can be fully automated and applied to partial specifications
State explosion problem
Theorem proving (infinite state-space)
»
»
Can handle infinite state-space
Requires human interactions, e.g. proof by induction
Demissie B. Aredo
March 3, 2005, Oslo
15
Validation
¾
Validation = ”building the right system”
♦
To check that an implementation satisfies a specification
♦
Full scale verification is very difficult
»
¾
¾
Simplification: focus on selected properties using partial specification,
e.g. type safety, functional equivalence of systems
Testing approach
♦
Allows checking of limited number of cases
♦
Selecting appropriate test cases is a major issue
Assertion approach, e. g. model checking
♦
Allows prove of statements of the general form
For all parameters V, it is the case that property P holds.
Demissie B. Aredo
March 3, 2005, Oslo
16
Why Formal?
¾
¾
More complex systems are emerging
♦
Testing cost explosion (not linear in system size)
♦
Likelihood of subtle errors is much greater
♦
Concurrency (shared variables, communication)
♦
Distributed Systems
Increased dependability concerns
♦
¾
A need for increased functionality and quality
♦
¾
E-banking, workflow, stock market, railway switches etc.
security, mobility, new business process, etc.
A need for increased productivity
♦
Semantically-based automated tools
Demissie B. Aredo
March 3, 2005, Oslo
17
Why Formal?
¾
Some exhibits
♦
The ”Intel pentium bug” (1994) cost ~$475 mill.
»
Formally verifying the correctness of the chip's mathematical routines
might have avoided the Pentium's problems – Rob Kling [17]
♦
Ariane 5 was destroyed 40 seconds after take off due to a crash
in one of the software components controlling the racket
(1996), cost ~$500 mill.
♦
The Challenger disaster (1986)
Demissie B. Aredo
March 3, 2005, Oslo
18
Formal Methods in Practice
¾
State-of-the-Art
¾
Why FMs is not in the mainstream SW engineering?
¾
Limitations of FMs in industrial settings
¾
Success Criteria for Formal Methods
¾
Introducing FMs into SW development process
Key problem:
Despite their potential benefits, FMs did not gain acceptance in the
industrial sector. What requirements should we put on FMs, and
what has to be improved in order to make the software industry
believe in them?
Demissie B. Aredo
March 3, 2005, Oslo
19
State-of-the-Art
¾
SW industry
♦
Time pressure and shorter deadlines
♦
Increased complexity
♦
Less resource on design and more time on Testing
»
♦
Many SW companies have in-house FM groups
»
¾
Up to 50% consumed by testing
Light-weight approaches
Academia
♦
Concerned with development of new techniques and prototype tools
♦
Has little capacity to provide industrial-strength tools
♦
Technology transfer is difficult
»
Novel research results are more rewarded than development
»
May not have access to real-life applications
Demissie B. Aredo
March 3, 2005, Oslo
20
Why FMs is not in the mainstream SW engineering?
¾
No support for programming languages used in Practice
¾
No support to design formal requirements specification
¾
There are not enough people with the mathematical expertise
to work with FMs
♦
E.g. verification may require knowledge in higher-order-logic
¾
Verification is not integrated into standard CASE tools and
development process
¾
The invention of concurrent systems requires scaling up from
modeling a single task to modeling many concurrent tasks
¾
Developers are more concerned about performance than
correctness
Demissie B. Aredo
March 3, 2005, Oslo
21
Why FMs is not in the mainstream SW engineering?
¾
¾
The rise of object-oriented programming
♦
There are only few FMs that support the main concepts of OO
such as inheritance
♦
No free reuse (redefinition) of code is supported in FMs
FMs is used heavily in the development of military systems
♦
¾
Developers don’t talk to the public about it, hence info does not
get to the SW developers
FMs offer support to prove system correct
♦
This is true only when each module is small, otherwise a wellstructured modular code is required
♦
Proving system correct has not been a successful business
endeavor, and remained illusive
Demissie B. Aredo
March 3, 2005, Oslo
22
Why FMs is not in the mainstream SW engineering?
¾
¾
¾
Economic issues
♦
Increases costs for formal design
♦
Increases income from SW patches, e.g. Microsoft products
Zimmerman et al, 2000
♦
Lack of training
♦
Tools are difficult to learn and use
♦
FMs are not scalable
Neumann, 1996
♦
FM techniques are not accessible (90’s)
♦
Tools are not user-friendly
♦
FM didn’t keep the speed of commercial needs to get SW products to
the market
♦
Research-oriented rather than applied to real developments
Demissie B. Aredo
March 3, 2005, Oslo
23
Limitations in the Industrial Settings
¾
Despite its benefits, FMs has not been in the mainstream SW
development. Why?
¾
It is perceived that FMs are difficult to learn and apply
”Why does this magnificent applied science which saves work and
makes life easier bring us so little happiness? The simple answer
runs: because we have not yet learned to make sensible use of it” -Albert Einstein, taken from [14].
¾
¾
Developers express serious doubts about
♦
scalability
♦
cost-effectiveness
Lack of a methodology to design and manage large proofs
♦
Large proofs take a long time to design and debug
♦
Difficult to maintain and reuse (domain-specific proofs)
Demissie B. Aredo
March 3, 2005, Oslo
24
Limitations in the Industrial Settings
¾
Not all FMs scale up to handle system complexity
¾
Programmers lack training in mathematics and logic
♦
Generally not true as required level of training need not be
advanced, e.g. Z is primarily based on set theory taught at the
high school level
♦
Limitations are methods-dependent
»
Z is primarily based on set theory taught at the high school level
»
CSP is extremely difficult and requires specialized training
»
Model-based formal spec are not difficult to learn and use
»
Axiomatic and algebraic methods are more challenging
Demissie B. Aredo
March 3, 2005, Oslo
25
Seven myths associated with FMs
¾
FMs guarantee perfect SW and eliminates the need for testing
¾
It’s all about proving program correct
¾
Only critical systems benefit from using FMs
¾
Its application requires highly trained mathematician
¾
Using FMs increase SW development costs
¾
FMs are incomprehensible/acceptable to practitioners
¾
FMs are not used on real large scale projects
[5] A. Hall, Seven Myths of Formal Methods, IEEE Software, 7(5):11-19,
September 1990.
Demissie B. Aredo
March 3, 2005, Oslo
26
Seven more myths
¾
¾
¾
FMs delay the development process
♦
Lack of experience in estimating how long a development takes
♦
”Software never comes in on time or on budget, and it always
breaks down”, taken from [13].
FMs are not supported by tools
♦
ZTC: a type-checker for Z, CADiZ a suite of tools for Z
♦
Model-checker and refinement-checker for CSP
♦
IFAD’s VDM-SL Toolbox, B-Toolkit
FMs replace traditional engineering design methods
♦
The research on the integration of structured review and FMs
leads to a method that fully supports the SW life cycle;
admitting the use of FMs at spec and design phases, supporting
refinement to executable codes, and proof of properties
Demissie B. Aredo
March 3, 2005, Oslo
27
Seven more myths
¾
FMs only apply to SW development
♦
¾
The FM9001 microprocessor
FMs are not required
♦
Safety-critical and security-critical systems
¾
FMs are not supported
¾
FMs people always use FMs
”System development should be as formal as possible, but not
more formal” [6].
[6] J. Bowen and M. Hinchey, Seven more Myths of Formal Methods, IEEE
Software, 12(4): 34-41, July 1995.
Demissie B. Aredo
March 3, 2005, Oslo
28
Introducing FMs into SW Engineering
¾
Bridge the gap between high-level design and low-level code
¾
A specification should be
¾
♦
Abstract enough to omit implementation details
♦
Concrete enough to reason with it and relate to code
Integration with traditional development process
♦
Wrong assumption: FMs supersede traditional methods
♦
[Hall90: Seven Myths of FMs ...]
♦
»
FMs can guarantee perfect software and eliminate the need for testing
»
FMs are all about proving programs correct
FMs should be designed to improve traditional methods rather
than to replace them
»
Building on existing expertise, improving cost/benefit ratio of FMs
Demissie B. Aredo
March 3, 2005, Oslo
29
FMs in SW ...
¾
Clear selection criteria to help developers decide which
method or tool is most appropriate for a project
¾
Current work in the context of UML
¾
♦
Semantics of UML diagrams
♦
Enhancing consistency checking
♦
Integration of formal and informal methods (UML and Z)
Automated tool support for rigorous analysis
♦
¾
Type-checker, Model-checker, Theorem-prover
Light-weight Approaches
♦
More focused and selective application of FMs
♦
critical and complex components of the system are formally
analyzed
Demissie B. Aredo
March 3, 2005, Oslo
30
Method Integration
¾
A way to introduce FMs into practical SW engineering
¾
One method is NOT sufficient
♦
No single FM is likely to be suitable for description and
analysis of all aspects of complex systems
»
The idea that a single method should cover two different projects is
highly suspect: the difference between projects is more important than
their similarities (Tom DeMarco: Controlling Software Projects,
1982)
»
For any realistic problem, you will need more than one method. You
will have to see the problem as a complex of several overlapping
problems. Your problem is not likely to be a method X or method Y
problem. (Michael Jackson: Software Requirements, 1995)
»
Does anyone still believe that each new methodology is somehow
universally better than its predecessors? (Robert Glass: IEEE
Software, July 2000)
Demissie B. Aredo
March 3, 2005, Oslo
31
¾
Formal Method Integration
♦
at least one of the involved methods is formal
♦
semi-formal notations are integrated with a suitable formalism
making them more precise and amenable to rigorous analysis
»
»
»
more workable approach
users work with graphical models they develop
Techniques:
– Translation of semi-formal notations into formal ones
– assign formal semantics to informal notations
¾
Known Limitations of Method Integration
♦
process issues are usually afterthoughts
♦
Inter-language consistency checking
♦
limited form of integration (only consistency check)
Demissie B. Aredo
March 3, 2005, Oslo
32
Success Criteria for FMs
¾
Minimize effort to learn and use FMs
♦
Use specification language that is easy to learn and use
»
»
Support for language syntax and semantics familiar to practitioners
Explicitly defined semantics
¾
Support for prototyping and executable specifications
♦ Leads to effective testing and simulation
¾
Support for better scalability and compositionality
¾
Automated translation to the language of analysis
¾
Integration into existing development process
♦
¾
Applicability to legacy systems
Availability of guidelines that describes when and how to apply it most
effectively
♦
Instructions that adequately link FMs with the software development
processes should be made available
Demissie B. Aredo
March 3, 2005, Oslo
33
Success Criteria for FMs
¾
Support cost-effective development of reliable software
¾
Availability of a strong suite of analysis tools
♦
If language semantics are sufficiently restricted, formal
analysis can be significantly automated
»
¾
”It starts up again when some new generation of tool support comes along to
make it a bit easier. The B-tool helped. Modern model checkers are helping
again." – Susan Stepney, [2].
Formal analysis should be automated as much as possible
♦
Pushbutton, e.g. consistency checking
♦ providing integrated tools for formal specification and
verification of programs
♦
¾
Tools should be made as user-friendly as possible
Formal analysis should provide good feedback
Demissie B. Aredo
March 3, 2005, Oslo
34
Conclusion
-------
¾
The pressure to produce high-quality SW is increasing
¾
Formal specification techniques are most applicable in the development
of critical systems, protocol standards, and HW design proved useful
¾
Formal specification forces an analysis of the system requirements at an
early stage. Correcting errors at this stage is cheaper than modifying a
delivered system
¾
The problem at hand is how the technology transfer process from FMs
research to practical SW development can be facilitated by reducing the
impediments:
♦
Improve the learning curve
♦
Manage large systems using modular approach
♦
Provide analysis results in terms understood by practitioners
♦
Improve the communication with non-experts
Demissie B. Aredo
March 3, 2005, Oslo
35
Conclusion
-------
¾
Using FMs in SW engineering does not solve everything
¾
Issues that need further investigation
¾
♦
Real links between industry and academia should be established
♦
Demos of success stories of use of FMs are required to
disseminate benefits of FMs
♦
Compositionality
Formal system specification complements informal
specification techniques
♦
Method integration is among the potential areas of research that
contribute to the wide-spread use of FMs
Demissie B. Aredo
March 3, 2005, Oslo
36
----
Conclusion
----
¾
Executability, compositionality of specification is vital for
the use of FMs in practical SW engineering
¾
FMs have influenced the development of programming
languages such as PASCAL, JAVA (VM)
¾
There are several controversies about FMs
♦
Arguments of advocators of FMs versus that of the opponents
Demissie B. Aredo
March 3, 2005, Oslo
37
References
1.
E. M. Clarke, J. M. Wing, et al, Formal Methods: State-of-the-Art and Future
Directions, ACM Computing Surveys, 28(4), December 1996.
2.
Richard Sharpe, Formal methods start to add up once again, Computing,
January 2004. http://osiris.sunderland.ac.uk/~cs0moa/formal_methods.pdf
3.
M. Barjaktarovic and WetStone Tech., The Sate-of-the-Art in Formal Methods,
Research Report, January 1998.
4.
V. George and R. Vaughn, “Application of Lightweight Formal Methods in
Requirement Engineering”, CROSSTALK, The Journal of Defense Software
Engineering, 16(1), pp. 30, January 2003.
5.
A. Hall, Seven Myths of Formal Methods, IEEE Software, 7(5):11-19,
September 1990.
6.
J. Bowen and M. Hinchey, Seven more Myths of Formal Methods, IEEE
Software, 12(4): 34-41, July 1995.
7.
L. Baresi et al, Introducing Formal Specification Methods in Industrial
Practice, in the Proc. of the 19th International Conference on Software
Engineering, May 17-23, 1997, Boston, Massachusetts, USA
Demissie B. Aredo
March 3, 2005, Oslo
38
References contd.
8.
K. E. Petrie, Women in the History of Computing, August 2001.
http://www.dcs.qmul.ac.uk/~uhmm/women/history.html#HOPPER
9.
N.G. Leveson, Guest Editor's Introduction: Formal Methods in Software
Engineering, IEEE Transactions on Software Engineering, 16(9), pp. 929-931
September 1990.
10.
C. L. Heitmeyer, On the Need for Practical Formal Methods, FTRTFT’98,
LNCS 1486, pp. 18-26, September 1998.
11.
M. Zimmerman et al, Making Formal Methods Practical, Digital Aviation
Systems Conference, October 2000.
12.
J. Knight et al, Why are Formal Methods not Used More Widely?, Fourth NASA
Formal Methods Workshop, Hampton, VA, September 1997.
13.
A. Z. Spector, D. K. Gifford, Bridge Design and Construction. Communications
of the ACM, 29(4):267-283, April 1986.
14.
J. Bowen, Ten Commandments of Formal Methods, IEEE Computer, 28(4): 5663, April 1995.
15.
Jeannette M. Wing: A Specifier’s Introduction to Formal Methods, IEEE
Computer, 23(9): 8-24, September 1990.
Demissie B. Aredo
March 3, 2005, Oslo
39
References contd.
16.
A. Lamsweerde, Formal Specification: a Roadmap, in the Proc. of ICSE
'00, pp.147-159, Limerick, Ireland, June 04 - 11, 2000, ACM Press.
17.
Rob Kling: Computerization and Controversy: Value Conflicts and Social
Choices (2nd Ed.), San Diego, Academic Press, 1995.
18.
M. Hinchey and J. Bowen, Industrial-Strength Formal Methods in
Practice, Springer-Verlag, 1999.
Demissie B. Aredo
March 3, 2005, Oslo
40