Inspection of Safety-Critical
Software
Case study: Darlington nuclear power
plant shutdown system
Prepared by: Ke Tang
The Darlington Nuclear Generating Station
The Darlington Nuclear
Generating Station
 Provides 20% of Ontario’s electricity
 Four reactor station
 Each has 2 independent shutdown systems

1. Drop neutron-absorbing rods into the core

2. Injects liquid poison into the moderator
 Decided to computerize
 Design Diversity
Why computerize
Software solutions
 more complex than
hardware, but offer more
intelligent monitoring
 Offer diagnostic aids,
periodic testing can be
performed more often
 Requires less equipment
Could improve safety, worth
a try
hardware solutions
 Simple, easily studied,
Offer a great deal
confidence
 Couldn’t perform
sophisticated checking
 Requires large amount
equipment
Original Software Development 1
 Work begin 1983, multiple versions were
development, mostly by experts of safety system
engineering and control system
 Software hires for second and third versions
without backgrounds in nuclear engineering
 Surprisingly small amount of code
- 6,000 lines of assembly code
- 7,000 lines of FORTRAN, 13,000 Pascal
 Unit testing, integrating test, random testing.
Software assessment, hazard analysis
Original Software Development 2
 Review by AEBC in 1987, License not granted
 There exited too much doubt about whether the software
implemented the requirements correctly
 Functions performed by well-understood hardware devices
were performed by much more difficult understood
software routines
 Program components can’t physically separated and
worked “in parallel”
 “ …there was no widely accepted definition of what
constituted ‘good enough’ for software used in safetycritical applications such as shutdown system”
- John Harauz
Software Inspection
 AECB hired David Parnas of Queen’s University
in mid of 1987
 The document written in nature language,
information was hard to find, no confidence that
all of the problem has been found
 Parnas proposed a formal mathematical inspection
in Jan. 1989
 Reverse engineering, code existed, but formal
specification was need
 Two dimension expressions, tables, were used.
Helped both designers and reviewers
Darlington Inspection
 Uses two documents:
– Software requirement document contains description of
actions that must be taken on a single pass through the
periodic loop (the shutdown system is designed as a
periodic loop).
– Program function descriptions describe effects of the
execution of the loop body.
 Goal:
– Both documents describe the same functions.
– Confidence they are the right functions.
Software requirement document
 Nuclear plant is complex, but only a small portion
of the information is actually relevant to
understanding shutdown system
 System requirement document separate the
complex details of outside word from the complex
details of the design of the computer system
 Precisely document, the requirement of software
must satisfy. Which is needed by programmers
and QA group
 Nature language is not adequate for the task of
precise requirement specification
Example Description and 3
reasonable interpretation
 Shut off the pumps if the water level
remains above 100 m for more than 4 s”
1 T
[  WL (t )dt]  100
4 T 4
1
[ Max[T 4,T ] (WL (t ))  Min[T 4,T ] (WL (t ))]  100
2
1 T
[  WL (t )2 dt]1 / 2  100
4 T 4
Example Description and the
analogous ambiguity
 Shut off the pumps f the water level remains
above 100 m for more than 4 s”
 Shut off the pumps if the minimum water
level over past 4 s was above 100 m”
Min[T 4,T ] (WL (t ))]  100
Rigorous Approach
 Correctness can be achieved with highly formal
methods.
 Mathematical specifications offer the promise of
concise, precise description
 Rigorous Construction – Methods for deriving
programs from precise specification.
 Verification – methods for showing that a program
satisfies its specification.
 Approach based on control theory models have
been proven practical
Documentation Of Computer System design
 System Requirement
treats the computer system as a black-box, and
gives a description of the environment in terms of
environment state variables. It describes the
relations among these state variables result from
physical, natural or other constraints. It also
specifies the additional relations that must be
established and maintained by the computer
system itself when it will operate on this
environment.
Documentation Of Computer System design
 System design document
Identifies the computers within the system,
and specifies the communications between
the computers and the environment by
describing the relations that will exist
between the values of the environmental
state variables and the values of the
computer
Documentation Of Computer System design
 The System Requirement Document and Design
Document together determine the Software
Requirements.
 Software Function Specification
Record additional design decisions and describes
the exact behavior of the software
Environmental State Variables
 State variables: characterize the environment in
which the system will have to operate.
 Classified as monitored or controlled, or both.
 Monitored variables are those the system has to
measure (pressure, temperature, pushbutton states,
etc ,..);
 Controlled variables are those the values of which
the system is intended to modify and operate upon
(switch, valve states, ...).
Functional Relations
 The contents of the documents are a representation
of the following mathematical relations
1) REQ(mt,ct)
2) IN(mt, it)
3) OUT(ot,ct)
4) NAT(mt,ct)
5) SOF(it,ot)
  mt, it ,ot ,ct [IN(mt, it)^ SOF(it,ot)^
OUT(ot,ct)^NAT(mt,ct)]REQ(mt,ct)
The use of Tabular Representations
 software programs usually have a large number of
discontinuities that can occur at arbitrary points in
the domain.These functions are discrete or
piecewise continuous.
 Tabular formats have been found to be more
practical for the description and communication of
these functions between system designers and
software engineers than conventional
mathematical formulations.
Parnas Tables
 Parnas Tables use tabular constructs to
organize mathematical expressions, where
 rows and columns separate an expression
into cases
 each table entry specifies either the result
value for some case or a condition that
partially identifies some case
Design and Documentation of Modules
 Modularization is the key to control of software
complexity
 Divided modules respect to processing -> strong
coupled modules->shared resources, hard to
understand
 “information hiding” modules hide( or abstract) a
data structure,device interface, mathematical
model of some physical system.
 Must accompanied by precisely documented
module interfaces
Program Function Documentation
 Describes the precise effect of the program
without describing the intermediated states
 Each document contains representation of
certain relations, otherwise if is incomplete.
 Additional information makes the document
faulty --> information not directly related
should go elsewhere.
Program Function Documentation
 Need precise notation for relations.
 Conditional expressions is too complex.
 Multi-dimensional notation is the author’s
choice.
 Program function tables have a practical
advantage.
Program Function Documentation
 Tables control the complexity of the
expressions
1) Tables parses the expression, made it easy
to understand for the reader;
2) Tables eliminates many repetitions of sub
expressions
3) Each table entry applies to a small part of
the functions domain, the expression in
that entry can be simplified
Program Function Tables
Quality Control
 Quality Control for safety-critical software
have three components:
– Testing of the product
– Inspection of the code.
– Evaluation of the design and the methods that
they use
 Testing and verification are not alternative
approach, but complementary
Testing and review
 Planned testing and random testing are needed
– Planned Testing ensure coverage of all “interesting”
cases
– Random testing test cases that will be encountered in
practical respected to the statistical distributions.
 Planned testing is more effective in uncovering
errors, but often miss subtle cases.
 Review: Check a section of the program to see if it
actually did what they should.
 A formal review based on the formal documents.
Inspection Process
 Designed around the following ideas:
– Inspectors need quiet time to “think.”
– Inspection results must be scrutinized carefully.
– Lengthy inspections must be done is smaller
sessions (ie. multiple days)
– Inspections must focus on small sections of the
program at one time.
– All cases must be considered.
– All parts of the programs must be inspected.
AECB Inspection Teams
 Requirements team( designer of CANDU) –
produces tabular representation of the
requirements.
 Code inspection team( consultants without
knowledge of nuclear field ) – produces programfunction tables from the code.
 Comparison team(Ontario Hydro employees) –
finds equivalence between requirements tables &
program-function tables by showing step-by-step
transformations from one to the other.
 Audit team(AECB employees and consultants) –
checks the work of the other three teams
Testing and review using tables
 Tables can facilitate both random and
selective testing. The tables identify the
discontinuities in the program functions and
clearly identify the limiting and boundary
cases which are known to be major sources
of programming faults.
 Tables also identify intervals between
discontinuities and help verify that all
intervals have been covered by a test set.
Conclusions
 It is certain that formal - documentation is a key
issue in the specification, design, and review of
software for critical control and safety functions.
 Tabular forms present great advantages over other
forms of documentation techniques. Make the
systematic inspection advanced.
 This approach(Tabular forms) is practical when
reliability and trustworthiness are extremely
important and the codes are relatively small
 Darlington project: costly, but built the Canada
Standard
References:
 D.L. Parnas, G. Asmis and J. Madey, Assessment of
Safety-Critical Software in Nuclear Power Plants,
Nuclear Safety, vol. 32, no. 2, April-June 1991, pp. 189198.
 P. J. Courtois, D. L. Parnas, Documentation for safety
Critical software, Processing of the 15th international
conference on Software Engineering, p.315-323, May 1721, 1993
 Parnas D. L., Van Schouwen, A.J., Shu Po Kwan (1990)
Evaluation of Safety Critical Software. Communications of
the ACM, 33,6,636-648.
References:
 D Parnas, Inspection of Safety-Critical Software Using
Program-Function Tables, in Report No. 288,
Communications Research Laboratory, McMaster
University, Canada, June 1994
 Jeffrey Smith, Richard Bruno, Vince Fumo Inspection of
safety-critical software using program-function tables..
 J. Dennis Lawrence, Warren L. Persons, and G. Gary
Preckshot (Lawrence Livermore National Laboratory)
Evaluating Software for Safety Systems in Nuclear Power
Plants
 Canada Nuclear