SOUND METHODS and EFFECTIVE TOOLS
for ENGINEERING MODELING and ANALYSIS
_________________
by David Coppit, College of William and Mary,
and Kevin J. Sullivan, University of Virginia
Proceedings of the 25th International Conference on Software Engineering
Portland, Oregon - May 3-10, 2003
Presentation by Bryan E. Bloss - University of Central Florida, Nov. 2003
1. INTRODUCTION
Designing Software Tools for Formal Design
________________________________________________________________________________________
Engineering modeling & analysis methods
are based on modeling languages for
describing systems, with semantics for
mapping expressions (models) to
estimates of system properties (results).
To be safe and effective, a modeling method requires
a language with a validated semantics; feature-rich,
easy-to-use, dependable tools; and low engineering costs.
Hampered by shortcomings in software engineering &
languages, today we lack adequate means to develop such
methods.
Two Sub-Problems Addressed in this Paper:
- Finding a cost-effective way to ensure
semantic soundness of a complex method
- Using Package-Oriented Programming (POP) to
produce easy-to-use, functionally-rich tools from
available software packages (such as MS Office)
Results: A package-based tool “Galileo” is evaluated
favorably by NASA engineers, and development of
“Nova”, a similar tool based on a formal semantics,
proves the cost-effectiveness of a combined approach
2. FORMAL SEMANTICS & DEPENDABILITY
Why are they important?
_____________________________________________________________________________________________________
Specification is more fundamental than implementation.
Without a formal specification:
- Validation is difficult
- No basis for a definitive user reference document
- Programmers are left to make uninformed semantic
decisions; unable to thoroughly test correct functioning.
Tools used in the design of safety critical systems should be
treated as critical engineering components.
Our inability to develop low-cost, easy-to-use tools can thus
be seen as a positive safety mechanism, but far from ideal.
Safety Example: 1996 alert from U.S. Nuclear Regulatory
Commission warned of significant errors in several tools which
had been adopted for use in nuclear reactor design & analysis
Another Example (not in paper): Crater analysis tool, used
inappropriately during flight STS-107 to analyze foam damage
3. APPROACH & BASIC RESULTS
Developing the Galileo Tool for DFT Analysis
___________________________________________________________________________________________
Observation: Most applications devote less than 10% of
their code to the core function of the system!
90% is devoted to superstructure-- support functions such as
text & graphical editing, data validation, etc.
Package-Oriented Programming (POP) is intended to save
time in creating superstructure; frees more resources for the
critical design activity: applying formal methods to define &
validate the syntax and semantics of the modeling language
The Application: Dynamic Fault Trees (DFT)
Graphcal representation of every
conceivable sequence of events
that could cause a system to fail.
Each leaf is a basic event; internal
gates define relationships leading
to system failures at upper levels.
Static trees model how event
combinations lead to failures;
Dynamic trees are order-sensitive.
(Illustrations from
CAIB Report)
3./4. CASE STUDY: RELIABILITY ENGINEERING
The Problems With Current DFT Languages
___________________________________________________________________________________________
During development of Galileo tool, a non-trivial error was found in the
underlying DFT language, DIFtree, where probability of a masked
(hidden) failure wasn’t correctly computed
Also, DIFtree’s informal specification had left ambiguities on how to
handle special cases; prior software implementations answered these
questions inconsistently in different parts of the program.
And formal validation was time-consuming, due to lack of automation in
available syntax & theorem-prover tools, and slow run-time perfomance
Worse, the theorem-prover tool required too much user expertise;
guidance often needed from the tool’s author
5. The NOVA DFT Tool
Like Galileo, uses POP components from MS Office for fault tree editing:
- Word for text editing
- Visio graphical editor (enhanced for DFT modeling constructs)
- Excel for computational results
Will allow even more emphasis on formalization & validation than in Galileo
6. END-USER EVALUATION OF GALILEO
New version of Galileo tool funded by NASA Langley Research Center, to
support new modeling & analysis constructs and be usable in practice
Featured in three workshops:
1st- Managers & engineers from several NASA divisions
2nd- Space Station engineers only
3rd- Space Shuttle engineers only
A short survey (34 questions) and an in-depth survey (77 questions) were
offered to engineers, to evaluate user perceptions of usability & features
Feedback indicated that usability was same or better than other tools!
Also confirmed that dependability is crucial; a formal specificaltion of
the modeling language was second only to a comprehensive test suite as a
means for increasing trust.
____Evaluation of this Article ____
Strengths: A well-informed overview of the authors’
experience with developing a new software toolset for
practical engineering, while emphasizing formal validation
methods. Many implications for the design of other
reliability-critical software applications.
Weaknesses: Little information on costs saved by POP
method; little detail on formal proving methods. Also, the
more advanced NOVA tool had not been user-tested at
time of publication, so final results aren’t known.
____Questions? ____