SELECT--A FORMAL SYSTEM FOR
"ESTING AND DEBUGGING PROGRAMS
BY SYMBOLIC EXECUTION*
R o b e r t S. B o y e r
Bernard Elspas
K a r l N. L e v i t t
Computer Science
Group
Stanford
Research
Institute
Menlo Park, California
94025
Keywords:
Program Testing;
Test Data Generation;
Symbolic Execution;
Program Verification;
Program
Debugging;
Solution
of Systems of Inequalities.
that serve to constrain the test data to desired
regions.
A.
Abstract
Much attention has been paid to the present high
cost of producing reliable software, and to the frequency of latent bugs even after a great deal of
effort has been expended in testing and debugging.
Among the tools that have beer proposed and studied
to alleviate this situation are a variety of tech ~
niques for formally verifying the mathematical correctness of programs.
All of these formal program
verification techniques, which have been embodied
in a number of experimental program verifiers, make
use of formal specifications of the program's intent, written in a formal assertion language.
The
verification process then consists in analyzing the
actions carried out by the program, and checking
(usually by proving mathematlcal theorems) that the
output assertions will be satisfied whenever the
input data meets the conditions specified by the
input assertions.
The details of the analysis
process, and more particularly, of the checking
process differ from system to system, but all of
the present experimental verification systems
operate
in the general
w a y we h a v e d e s c r i b e d .
SELECT is an experimental system for assisting in
the formal systematic debugging of programs.
It is
intended to be a compromise between an automated
program proving system and the current ad hoc debugging practice, and is similar to a system being
developed by King e t a l . of IBM.
SELECT systematically handles the paths of programs written in a
LISP subset that includes arrays.
For each execution path SELECT returns simplified conditions on
input variables
that cause the path to be executed,
and simplified
symbolic
values for program variables at the path output.
For conditions
which
form a system of linear
equalities
and inequalities
SELECT w i l l
return
input variable
values
that
can serve as sample test data.
The user can insert
constraint
conditions,
at any point
in the
program including
the output,
in the form of symbolically
executable
assertions.
These conditions
can induce the system to select
test data in userspecified
regions.
SELECT c a n a l s o d e t e r m i n e
if
the path is correct
with respect
to an output
assertion.
We p r e s e n t
four examples demonstrating the various
modes of system operation
and
their
effectiveness
in finding
bugs.
In some
examples,
SELECT w a s s u c c e s s f u l
in automatically
finding
useful
test data.
In others,
user interaction
was required
in the form of output
assertions.
SELECT a p p e a r s
to be a useful
tool for
rapidly
revealing
program errors,
but for the
future
there
is a need to expand its expressive
and deductive
power.
I.
Mechanical program verification and Lts
shortcomings
It is generally
conceded,
even by proponents
of
particular
verification
systems
or of as yet unimplemented
techniques,
that
they are a long way off
from being able to handle practical
programs of
substantial
size.
We s h a l l
list
some of the reasons
why this
is so, and then proceed
to discuss
briefly
why even after
the present
obstacles
have been
overcome,
formal program verification
would still
fall
short
of providing
the tools needed for practical
program debugging.
The main present
obstacles
to formally
verifying
realistic
programs
are:
Introduction
This paper discusses an experimental system,
SELECT, for aiding a user in systematically debugging programs.
It differs from conventional
debuggers in that it attempts to carry out a formal processing of the program paths under user
interaction in order to (I) generate potentially
useful test data, (2) provide simplified symbolic
values for program variables at the output of a
path and (3) prove the correctness of a path with
respect to user-supplied assertions.
The path also
attempts to give preliminary evaluations on the
potential utility of formal testing tools.
In
particular, we show that the automatic generation
of test data can be useful but certainly should not
be viewed as a total solution to the problem.
A
more promising method is to incorporate user interaction in the test generation process.
SELECT
achieves this to a degree by permitting the user
to insert assertions at points within the program
1. The necessity (in most systems) for inventing
intermediate assertions (inductive assertions,
Floyd predicates, and so forth), and the difficulty experienced by the programmer-verifier in coming
up with these assertions.
2.
The difficulty
in framing
input and output
assertions
that adequately
express
the programmer's
intent.
3. The lack of sufficiently rich assertion lanquages in which to express these input/output
assertions and intermediate assertions for programs
covering a wide variety of data types and functional
primitives.
*The work reported here was supported by the National
Science
Foundation
under Grant Number GJ-36903X,
entitled "The Feasibility of S o f t w a r e
Certification."
234
than on the programmer's
intentions.
A system of
this
sort could be useful
in constructing
programs,
by helping
to clarify
the programmer's
initially
fuzzy intentions,
as well as for ex post facto
testing.
4. T e c h n i c a l l i m i t a t i o n s in p r e s e n t - d a y theoremp r o v i n g techniques (inadequate speed and large d a t a
storage
requirements).
5.
An a p p a r e n t
need for considerable
intervention
at too detailed
a level
in
proving
process.
programmer
the theorem-
Another
rather
fundamental
aspect
of formal verification
serves
as some hindrance
to its practicality.
Mathematical
proofs
of program correctness
are contingent
on a large number of assumptions
about the language
semantics,
the actual
compiler
implementation,
the treatment
of error
conditions,
overflow,
underflow,
division
by zero,
and finite
m a c h i n e arithmetic, in the actual i m p l e m e n t a t i o n
on w h i c h the p r o g r a m is supposed to run.
Thus, even
a f t e r a m a t h e m a t i c a l proof of correctness, one cannot be certain that a p r o g r a m will run as intended
on a given machine.
T e s t i n g in the real m a c h i n e env i r o n m e n t on actual d a t a would appear to be a u s e ful c o m p l e m e n t a r y
technique
to formal verification
since
it is not contingent
on such assumptions.
Obstacles
1-3 are all concerned
with formal
assertions
and the language
in which they are expressed.
O n e may e x p e c t
that the considerable
research efforts
now under way will eventually
overcome these difficulties
in large measure.
Moreover,
several
investigations
(Elspas
[1974],
Wegbreit
[1974],
Katz-Manna
[1973]) now in progress
are aimed
at simplifying
the programmer's
task of inventing
intermediate a s s e r t i o n s b y m a k i n g this a semiautom a t i c process, w i t h the c o m p u t e r g e n e r a t i n g the
routinely d e r i v a b l e assertions.
It is also likely
that c o n t i n u e d w o r k on m e c h a n i c a l theorem p r o v i n g
(especially if it is focussed on the p a r t i c u l a r
needs of p r o g r a m verification) will make a v a i l a b l e
s u f f i c i e n t l y rapid and s t o r a g e - e f f i c i e n t proof techn i q u e s in several years time.
However, it is likely
that some d e g r e e of p r o g r a m m e r intervention will
always be d e s i r a b l e in this process of v e r i f y i n g
p r o g r a m v e r i f i c a t i o n conditions.
B.
In addition
to the obstacles
discussed
above,
formal program verification
suffers
from a more
basic deficiency
which,
it would appear,
cannot be
fully
overcome within
the framework of what it purports
to do.
When i t w o r k s a t a l l ,
a mechanical
(or semimechanieal)
program verifier
can certify
correctness
only of a program that is actually
"correct"
(with respect
to the stated
input-,
output-,
and intermediate
assertions).
If, as is
u s u a l l y the case in practice, the p r o g r a m is not
correct, the verifier system's o u t p u t is not particularly useful in h e l p i n g the u s e r to decide
whether:
I.
The a s s e r t i o n s are
gram contains a bug, or
,v
correct,
2.
The program really
does
intended,
but
the assertions
framed,
or
3.
Both the
fication,
or
program
and
,,
what
were
assertions
and the pro-
the programmer
inappropriately
require
modi-
4. Both the p r o g r a m and tlle assertions are
correct,
but the theorem prover contains a bug
(or simply was inadequate for proving the required
theorems).
To state the matter succinctly, while there is a
need for a s y s t e m for m a t h e m a t i c a l l y c e r t i f y i n g
programs
that are correct,
there
is also a need for
one
that will rapidly
reveal
defects
in programs
that are not correct.
In particular,
such a system
should be able to u n c o v e r d i s c r e p a n c i e s between what
a p r o g r a m will actually do and what the p r o g r a m m e r
intended it to do, even w h e n the p r o g r a m m e r m ~
have
only ~ f u z z y idea of what he intended it to do, e.g.,
by r e v e a l i n g to him that some unintended action will
be performed u n d e r an u n a n t i c i p a t e d input data condition.
Clearly, such a system must be able to
o p e r a t e w i t h o u t m a k i n g use of p r o g r a m m e r - s u p p l i e d
formal a s s e r t i o n s of intent.
Thus, the emphasis is
on a n a l y z i n g what the p r o g r a m a c t u a l l y does, rather
235
The purpose
and
~eneral
features
of
SELECT
Having been aware for some time of both these
present-day,
and also the ultimate
limitations
to
formal c o r r e c t n e s s p r o v i n g discussed above, a
group at Stanford R e s e a r c h Institute (of w h o m the
authors c o n s t i t u t e a part) began an investigation
around J a n u a r y 1974 as to how these limitations
might be circumvented.
O u r intent was to e x p l o r e
formal m e t h o d s - - n o t g u a r a n t e e i n g m a t h e m a t i c a l cor ~
r e c t n e s s - - t h a t are more likely to reveal bugs in a
program, w i t h o u t m a k i n g use of formal assertions,
and that might still produce more c o n f i d e n c e in the
final
product
program than contemporary
testing
techniques
can provide.
At the date of writing
of
this paper a partially
implemented
system,
SELECT,
has been constructed
which exhibits
in experimental fashion
features
of semantic
analysis
of programs, construction
of input data constraints
to
cover selected
program paths,
identification
of
(some) infeasible
program paths,
automatic
determination
of actual
input data to drive
the test
program through selected
paths,
and execution
(actual
and symbolic)
of the test program with
optional
intermediate
assertions
and output
assertions.
These features,
which will be described
in
more detail
below, have been provided
by making use
of the expression-simplifier
portion
o f t h e SRI
Program Verifier,
adding thereto
facilities
for
symbolic
execution
of programs written
in a subset of LISP, and numerical
programs for determining input data sets satisfying
algebraic
inequalities
(path tracing
conditions).
It should be
noted that our system is similar
in some ways to
an unimplemented
system described
independently
by
Howden [1974],
and to a system being constructed
by
K i n g [ 1 9 7 5 ] a t IBM.
In the following
sections
o f t h e p a p e r we p r e s e n t :
a description
of the features
o f SELECT, f o u r e x a m ples illustrating
the modes of operation,
a comparis o n o f SELECT w i t h o t h e r
formal and near-formal
testing
methods,
and an evaluation
of the system including
some possible
extensions.
II.
General D e s c r i p t i o n of the SELECT Testing and
D e b u g g i n g System
A.
Overview o f the System
Before describing the path condition generator,
i t ~ i l l be c o n v e n i e n t t o d i s c u s s t h e s y n t a x and
semantics of the programming language (input language) for programs that our system can handle.
The e x p e r i m e n t a l SRI t e s t i n g and d e b u g g i n g s y s t e m ,
SELECT*, h a s a s i t s main f e a t u r e s :
C.
1.
S y m b o l i c e x e c u t i o n o f an i n p u t ( s u b 3 e c t ) p r o gram, c o v e r i n g ( i m p l i c i t l y )
all possible execution
p a t h s f r o m START t o HALT. S y m b o l i c v a l u e s o f p r o gram v a r i a b l e s a r e u p d a t e d when a s s i g n m e n t s a r e t r a v e r s e d , and s y m b o l i c p r e d i c a t e s a r e c o n s t r u c t e d ( a n d
updated) to capture the cumulative effect of the
a s s u m p t i o n s i n v o l v e d when e i t h e r t h e T o r F e x i t
from a branch statement is taken.
Both types of expressions,
involve only the input data symbols.
I n p u t Language
The l a n g u a g e f o r p r o g r a m s t o be t e s t e d by o u r
e x p e r i m e n t a l s y s t e m was s e l e c t e d t o r e s e m b l e a s u b s e t o f LISP.
A d m i t t e d l y t h i s i s an u n r e a l i s t i c
choice from the standpoint of practicality,
but it
i s a n a t u r a l one f o r o u r p u r p o s e s , f i r s t ,
because
it facilitated
m a k i n g u s e o f p o r t i o n s o f t h e SRI
Program Verifier (which used a similar input lang u a g e ) , and s e c o n d , b e c a u s e LISP i s an e m i n e n t l y
suitable language for building experimental systems.
T h u s , by e l i m i n a t i n g t h e n e e d f o r c o m p l e x t r a n s l a t o r s f r o m some more p o p u l a r i n p u t l a n g u a g e t o LISP,
we w e r e a b l e t o f o c u s on t h e r e a l i s s u e s i n v o l v e d .
2.
The a u t o m a t i c c o n s t r u c t i o n f o r e a c h p o t e n tially executable path (or for a user-selected
subset of paths) of a boolean expression that precisely
characterizes
that region of the input data space
for which execution of that path is guaranteed.
Our l a n g u a g e i n c l u d e s
3.
The d e t e r m i n a t i o n o f s i m p l i f i e d e x p r e s s i o n s
for the values of all program variables,
in terms
of symbolic input values.
H e r e a r r a y n a m e i s an i d e n t i f i e r
(pointer),
expr 1
after evaluation determines the location in the
a r r a y t o be c h a n g e d , and t h e v a l u e o f e x p r o i s p l a c e d
into that location.
The a r r a y a r r a y n a m e m ~ s t p r e
v i o u s l y h a v e b e e n d e c l a r e d , and t h e i n d e x g i v e n by
e x p r 1 m u s t f a l l w i t h i n t h e a r r a y b o u n d s , e l s e SETA
g e n e P a t e s an e r r o r .
(ELT a r r a y n a m e l ) r e t u r n s t h e
value In position i.
5.
Execution of the test program with execution
(real or symbolic) of intermediate assertions
(Floyd
assertions)
and/or output assertions.
These a s s e r t i o n s can c h a r a c t e r i z e
the programmer's intent, or
guide the system to choose test data satisfying a
constraint
n o t r e f l e c t e d in t h e code of t h e p a t h .
In the remaining portions of this section of
t h e p a p e r we d i s c u s s t h e s e s y s t e m f e a t u r e s i n more
detail.
In particular
we d i s c u s s :
the meaning
we g i v e f o r a p r o g r a m p a t h ; t h e i n p u t l a n g u a g e
w h i c h i s a s u b s e t o f LISP, f o r t h e p r o g r a m s ; t h e
symbolic execution technique used for generating
path conditions;
the packages for solving systems
of equalities
and i n e q u a l i t i e s ;
t h e method f o r
systematically
t r a v e r s i n g t h e p r o g r a m p a t h s ; and
t h e m a n n e r by w h i c h t h e u s e r c a n i n t e r a c t w i t h
the system to guide the generation of test data.
Path Analysis
constructs:
1.
Assignment:
(SETQ x e x p r ) a s s i g n s t o t h e v a r iable (identifier)
x the value of the expression
expr.
A s s i g n m e n t s ~ o a r r a y s a r e e x p r e s s e d by t h e
syntax:
(SETA a r r a y n a m e e x p r 1 e x p r 2 ) .
4. A u t o m a t i c derivation of actual (i.e., numerical) input data that can be used as sample test
data to exercise each such path.
B.
the following
2. Arithmetic functions:
The common arithmetic
functions are expressed in prefix form by PLUS, TIMES,
DIFFERENCE, QUOTIENT, MINUS (unary ftmctlon), and ABS.
PLUS and TIMES can take any number of arguments.
3. Control statements:
Control is handled by a
free combination of GO, FOR, WHILE, UNTIL, and COND
(conditional) statement types.
The GO ("go,o") has the syntax:
(GO label), and
executes an unconditional jump to the named label
(appearing as a statement label at the top level of
a PROG) in the program.
and S e l e c t i o n
We s h a l l h e n c e f o r t h u s e t h e t e r m p a t h t o r e f e r t o
any c o n t r o l s e q u e n c e f o r t h e t e s t p r o g r a m b e g i n n i n g
w i t h t h e e n t r a n c e o r START p o i n t , and e n d i n g w i t h
t h e e x i t o r HALT p o i n t .
In particular,
multiple
executions of a loop within such a "path'* will define different
paths in our sense.
T h u s , any p r o g r a m
c o n t a i n i n g one o r more l o o p s $ c o n t a i n s a p o t e n t i a l l y
infinite
number of d i s t i n c t
paths.
It is clear that
no f i n i t e a m o u n t o f t e s t i n g ( a s o p p o s e d t o f o r m a l
verification
of correctness)
can test all such paths.
H o w e v e r , t h e p u r p o s e o f t e s t i n g i s t o r e v e a l b u g s and
to improve the user's confidence in the program, not
to guarantee mathematical correctness.
We b e l i e v e
t h a t t h i s p u r p o s e c~n a l s o be s e r v e d by p e r m i t t i n g
the user of a testing system to specify (for each
embedded l o o p o r FOR, WHILE, e t c . s t a t e m e n t ) how
many t r a v e r s a l s he w i s h e s t h e s y s t e m t o e x e c u t e .
C o n c e p t u a l l y , p a t h s e l e c t i o n and t e s t d a t a g e n e r ation are separate processes, but, for reasons of
efficiency,
we h a v e c o m b i n e d them i n t h e c u r r e n t
v e r s i o n o f SELEC~
The FOR statement has the form:
(FOR X FROM I TO J
(... llst of executable statements... ))
with WHILE and UNTIL options expressed as:
(FOR X FROM I [TO J] UNTIL predicate
(... statement-llst... ))
(FOR X FROM I [TO J~ WHILE predicate
(... statement-llst...))
The clause [TO J} is also optlonal in the above three
forms.
The name SELECT was c h o s e n t o s u g g e s t t h e s y s t e m t s
principal feature: the automatic selection of test
data to insure coverage of the program.
For those
who l i k e a c r o n y m s , t r y S y m b o l i c E x e c u t i o n ~ a n g u a g e
to Enable Comprehensive Testing.
tother than loops with predetermined numbers of
traversals, e.g.,
I = I UNTIL I0
(statements>
...
**"FOR
236
Conditional statements are expressed
(COND ! e x p r 1 s t a t e m e n t - l i s t
1)
~expr2 s t a t e m e n t - l i s t ~ )
.....
as
current (symbolic) values of the program variables,
both simple variables
and subscripted
ones, as control progresses forward along the path.
A LISP
alist
(a list of dotted pairs, e.g.,
-))
The s e m a n t i c s o f COND a r e a s f o l l o w s :
the express i o n s e x p r .£, e x p r 2 , . . a r e
successively
evaluated in
turn until one is found that yields a non-NIL value,
at which point the corresponding statement-list
is
e x e c u t e d and t h e COND i s e x i t e d w i t h o u t e v a l u a t i o n
of the subsequent clauses.
The a l t e r n a t i v e
syntax
I F e x p r 1 THEN . . . ELSE I F e x p r 2 THEN . . . E L S E . . . .
is
also accommodated.
((X .
(PROG l i s t - o f - l o c a i - v a r i a b l e s
statement 1 ... statement n)
5.
Boolean functions:
The common b o o l e a n f u n c t i o n s ( l o g i c a l c o n n e c t i v e s ) AND, OR, NOT a r e u s e d
(as prefix functions)
to combine predicates
for use
in conditionals,
WHILEs, UNTILs, and i n a s s e r t i o n s ,
t o be d i s c u s s e d l a t e r .
AND and OR c a n t a k e a n y
number of arguments.
The p r i m i t i v e p r e d i c a t e f u n c tions are the six numerical equality-inequality
functions,
EQ ( = ) , NEQ ( ~ ) , GT o r GREATERP ( ~ ) , LT
or LESS, (<), GTQ (~), and LTQ (~).
(TIKES A C 1 ) )
A r r a y a s s i g n m e n t s r e s u l t i n an u p d a t i n g o f t h e
alist,
similar to simple variable assignments, but
as with the case of branch predicates,
array assignments can result in an updating of the hypothesislist as well.
The k e y r e l a t i o n
involving arrays is
the identity
o f two s u b s c r i p t s ,
since the implication I = J D A[I] = A[J] must always hold.
This
situation
a r i s e s when a n a s s i g n m e n t o f t h e f o r m
P ~ A[J] is encountered,
a s b e l o w , a nd t h e a l i s t
c o n t a i n s some d o t t e d p a i r , s a y ( A [ I ] . Q), w i t h a
r e f e r e n c e t o a n e l e m e n t o f A a s i t s l e f t h a n d co m p o nent.
6.
Subroutines or function calls:
The SELECT
system can handle arbitrary
function calls within
t h e p r o g r a m t o be t e s t e d ,
provide~, of course that
s u c h a f u n c t i o n h a s b e e n d e f i n e d by a s u i t a b l e body
of code.
Such f u n c t i o n s may c a l l o t h e r u s e r - d e f i n e d
f u n c t i o n s ( o r may c a l l t h e m s e l v e s r e c u r s i v e l y ) .
When a f u n c t i o n c a l l e x i t s ,
it returns a single value
( a s i n L I S P ) , and may a l s o p r o d u c e s i d e e f f e c t s on
external variables.
The m e c h a n i s m w h e r e b y SELECT
handles the symbolic execution of such function calls
is entirely
that of macro-expansion.
This approach
l e a v e s s o m e t h i n g t o be d e s i r e d a s w i l l be d i s c u s s e d
in Section V below.
Path Condition
(Y . C) (Z .
provides the actual storage mechanism, in this case
v a l u e s f o r X, Y, and Z. Note t h a t a l l r i g h t - s i d e
variables
i n an a l i s t d o t t e d p a i r r e f e r t o t h e v a l u e s
at program entry.
The a l i s t
is updated whenever an
assignment statement is traversed.
A n o t h e r llst,
the hypothesis-list,
similarly keeps track of the
branch predicates
that have been traversed,
an d r e p resents the conjunction of these boolean predicates
(in terms of the values of the program variables
current,
from the alist,
at the time of symbolic
execution of the branch test.)
F o r e a c h p a t h an
input data exampl e is maintained that would cause
t h e p a t h i n q u e s t i o n t o be e x e c u t e d .
The v a l u e s
f o r i n p u t d a t a a r e d e r i v e d by c a l l i n g
the solvers
described in the next subsections.
When t h e p a t h
exit is reached the alist contains the final values
of all program variables expressed in terms of the
input data; the hypothesis list provides the desired
conjunction of predicates,
also in terms of the input
d a t a , a nd t h e i n p u t d a t a e x a m p l e i s a p o s s i b l e i n p u t
for the path.
The e x p r e s s i o n s i n t h e a l i s t
and the
hypothesis list are subjected to the simplifier
used
i n t h e SRI p r o g r a m v e r i f i e r .
4.
Scoping of variables
i s a c c o m p l i s h e d by t h e
L I S P FROG w h i c h c r e a t e s f l o w c h a r t - l i k e
programs
w i t h i n a l a r g e r p r o g r a m , w i t h l o c a l PROG v a r i a b l e s
v i s i b l e o n l y w i t h i n t h e PROG. The s y n t a x i s :
D.
(PLUS A B ) )
Code
Alist
Hypothesis-List
Generator
P~
The p u r p o s e o f t h e p a t h c o n d i t i o n g e n e r a t o r i s t h e
construction
f o r any p r o g r a m p a t h ( o r a s e t o f . s u c h
p a t h s ) o f t h e n e c e s s a r y and s u f f i c i e n t
boolean expression (involving the input variables
only) for
t h a t p a t h t o be e x e c u t e d .
Thus, it must construct
essentially
a conjunction of the test predicates
(or
negations of test predicates,
d e p e n d i n g on w h e t h e r a
TRUE o r FALSE e x i t i s t a k e n f r o m a t e s t ) e n c o u n t e r e d
along that path.
However, a t e a c h t e s t t h e c u r r e n t
symbolic values of the program variables
used in that
test are used.
A[J]
(...(A[I].Q)...)
(...hypotheses
....
)
I f we c a n d e d u c e t h a t I = J ,
then the lists
become
(...(P.Q)...)
otherwise
if
(...(P.A[J]...)
T h e r e a r e two d i s t i n c t
a p p r o a c h e s t h a t c a n be t a k e n
to this simple expression-handling
task, which are
referred to as backward substitution
and f o r w a r d s u b stition.
The m e t h o d o f f o r w a r d s u b s t i t u t i o n
turns out to
b e a more n a t u r a l a p p r o a c h h e r e , and a l s o a v o i d s
some o f t h e c o m p l i c a t i o n s p r o d u c e d by a r r a y a s s i g n ments.
In forward substitution,
we e s s e n t i a l l y
m o d e l t h e m a n u a l ( o r m e n t a l ) a c t i o n c a r r i e d o u t by
a human p r o g r a m m e r i n a n a l y z i n g w h a t h a p p e n s a l o n g
an e x e c u t i o n p a t h .
The s y s t e m k e e p s t r a c E o f t h e
237
(...hypotheses...(I=J))
I~J
(...hypotheses...(If-J))
If it happens that l=J, then the effect of this
assignment (in terms of input values) is to assocl a t e P a nd Q.
T h u s , a s shown a b o v e , t h e a l l s t
will
become u p d a t e d w i t h t h e d o t t e d p a i r ( P . Q ) , an d t h e
hypothesis-llst
w i l l r e c e i v e an a d d l t l o n a l
conjunct
I = J.
Note that, as with branch predicates,
it
m u s t be d e t e r m i n e d t h a t t h e h y p o t h e s i s l l s t i s a c t u ally satisfiable
a f t e r c o n j o i n i n g t h i s new p r e d i c a t e .
On t h e o t h e r h a n d , i f t h e h y p o t h e s i s l l s t i s a l s o
satisfiable
after conjolnlr~g the expression I ~ J,
then the allst
i s u p d a t e d by t h e d o t t e d p a i r ( P . A [ J ] ) ,
exactly as for simple variables,
and the hypotheslsllst is augmented with the conjunct I ~ J.
Note
t h a t a n a s s i g n m e n t o f t h e f o r m A[K] ~ V r e q u i r e s o n l y
an u p d a t i n g o f t h e a l l s t
by t h e t e r m ~ [ E ] . V ) .
In summary, an assignment
of the form P ~ A[J] results
in the establishment
of new virtual
paths,
via
tests
for equality
(and inequality)
of J with each
array index appearing
in A-array
references
on the
left-side
of any dotted
pair in the alist.
For each
such virtual
path the hypothesis-list
is updated
with the relevant
predicate,
J = index,
or J ~ index,
and the alist
is appropriately
updated
relative
to
the corresponding
assumption
on J.
is a system of integer linear inequalities.
Strict
equality can also be handled in an obvious manner.
This program uses an integer linear programming
algorithm to optimize an (arbitrary) objective function, with the path tracing inequalities serving as
constraints°
Since the above GOMORY algorithm is limited to
solving for integer variables (these may be positive,
negative or zero), some other means was needed to
handle programs where the program variables could
take arbitrary real values.
For this purpose, a
mixed integer linear programming algorithm (called
BENDERS) was developed by adapting an existing
algorithm due to Benders [1962]. As with the GOMORY
program, however, this second algorithm is still
limited to linear constraints.
It should be emphasized
that the backward and forward substitution
methods produce entirely
equivalent
results.
King's
current
s y s t e m EFFIGY u s e s a f o r w a r d
symbolic
execution
technique
like
t h e o n e we a r e c u r rently
using,
while Howden [1974] advocates
the backward substitution
approach.
In addition
to the
advantages
in relation
to array assignments,
the forward approach
also permits
the early
exclusion
of
"impossible
paths."
Each potential
augmentation
of
the hypothesis
list
which occurs
whenever a branch
predicate
or an array assignment
is confronted,
first
precipitates
an attempt
to determine
which predicate
outcome will result
from the test
data accumulated
so far.
Also, the system determines
if there exists
any data that would result
in the contrary
outcome.
If such data do not exist,
all path extensions
of
the contrary
outcome are excluded
from further
consideration.
In the backward substitution
method
this path pruning
process
cannot be carried
out until
the whole path has been traced.
E.
Inequalities
The limitation of the above algorithms to linear
combinations is an unacceptable, and vexing, one.
For example, they could not handle an inequality
like X*Y + 10*Z - W > 5 among its constraints, unless
one were prepared to assign to X a trial value, and
then attempt a solution (assuming the other inequalities are linear).
We therefore considered various
alternatives that would not be subject to that limltation.
The most promising of these alternatives
appears to be a conjugate gradient algorithm ("hillclimbing" program) that seeks to minimize a potential function constructed from the inequalities.
There are a number o f different ways in which the
potential function can be constructed for a given
set of inequalities, with the common feature that
the desired minimum potential is attained within the
path condition subregion.
The present version of
this algorithm appears to be quite effective computationally (though it may require some user guidance)
in coming up with sample data points.
It is not at
all limited in the kinds of functional combinations
allowed between program variables; any computable
functions are permissible.
Solver
The hypotheses
produced
by the path condition
generator
just
discussed
are not an end in themselves.
Instead,
they constitute
logical
constraints
from
which one hopes mechanically
to derive
actual
instances
of input data satisfying
these conditions.
We m a y v i e w a g i v e n s e t o f p a t h c o n d i t i o n s
as defining
a subregion
of the whole input data space
such that any data point
in this
subregion
will result
in execution
of the program along the same
specified p a t h of the p r o g r a m .
However, the logical
conditions
serve only to define
this
subregion
in an
implicit
sense.
In order to come up with actual
inp u t d a t a s a m p l e s we n e e d t o " s o l v e "
the logical
conditions.
An attempt to solve the conditions is also
important in revealing impossible paths.
If the
conditions do not possess a solution there does no$
exist input data that will cause execution of the
corresponding path.
Most frequently in numerical programming the branch
conditions of the program will be equalities and inequalities among (real or integer) variables.
Hence,
the path test conditions will also be (conjunctions
of) such equalities and inequalities.
In order to
generate sample test data for a given path, we therefore need an equality/inequality solver.
In the
SELECT system we have experimented with several different programs for doing this task.
The first program (GOMORY) [see Gomory 1963] that was written is
limited to handling systems of inequalities among
(positive or negative) integer variables, where the
functions that combine these variables are limited
to linear combinations (with constant numerical
coefficients).
For example,
3*X + 2*Y -
153.Z
> 0
X +Y+Z>0
1O*x - 5*Y < 0
238
It is possible for the user, by varying parameters
of this hill-climbing program, or by modifying the
potential function, to operate the program in a
number of different modes.
For example, if it is
desired to select data points on or near the subregion boundaries, there is a mode that will favor
the selection of such points.
It is also possible
to favor the selection of test points at or near
the intersections of several boundary surfaces.
Finally, one can also insist that the selected test
point will be well within the subregion--nominally
at its "center" as defined by a potential minimum
among several additively combined sub-potential
functions, each such sub-potential arising from one
of the inequalities of the system.
However, the
hill-climbing algorithm is not guaranteed to terminate, and requires user interaction to guide it.
F.
Automatic Traversal of Paths
If the user so desires, ~ELECT will handle all
distinct input-output paths such that no loop is
traversed more than S times, S being supplied by
the user.
We call S the looping factor.
We recall
that in tracing out a particular path the system
has generated a specific input data example that
would cause execution of the path segment.
When a
branch point is reached it is determined which of
the two branch exits the data satisfies, and then
the system will continue execution of the path
III.
extension
taking
this branch.
The solvers
are invoked to determine
if there exist
input data that
will cause execution
of the alternative
branch.
If there is a solution,
a backtrack
point
is established
for future
analysis
of the alternative
b r a n c h and its extensions.
If n o s o l u t i o n is found
then the e x t e n s i o n s of the a l t e r n a t i v e b r a n c h are
ruled out of future consideration.
A p a t h is term i n a t e d when a p r o g r a m e x i t is reached.
The continued e x e c u t i o n of a b r a n c h satisfied by the input
data e x a m p l e and the b a c k t r a c k i n g p r o c e e d until a
loop is traversed more than S times°
G.
User Supplied
Program Code
Assertions
as
an Adjunct
to
In this section
we i l l u s t r a t e
the action
of
SELECT o n f o u r s m a l l p r o g r a m s .
We h a v e c h o s e n
each of these programs to distinguish
unique
features
of the system,
and to attempt
to reveal.
some of the difficult
and controversial
issues
of
automated
testing.
The first
program,
WENSLEYSQROOT ( W e n s l e y [ 1 9 5 8 ] ) ,
is intended
to be an iteratire
algorithm
for computing
the square
root of a
number p, 0 ~ p < 1.
However, the program actually
contains
a subtle
error.
The application
o f SELECT
t o WENSLEYSQROOT i l l u s t r a t e s
(1) the ~eneration
of
real number sample test data corresponding
to particular
execution
paths and (2)=the
systematic
selection
of executable
paths.
SELECT w a s a b l e t o
generate
test data useful
for revealing
the presence
of the bug.
The second program,
MOVEARRAY, i s i n tended to cyclically
shift
a subarray
of an array
by d positions,
but a bug in the program causes
incorrect
operation
for certain
negative
values of d.
This example illustrates
(3) the generation
of integer-valued
sample test data,
(4) the array semantics
that introduce
extra
hypotheses
beyond those
associated
with branch statements,
and (5) the
generation
of symbolic
values
for outputted
variables
that,
in this example,
permit disclosure
of
the bug.
The third
program,
BUGGYFIND, i s i n spired
by Hoare's
FIND ( H o a r e [ 1 9 6 1 ) 1 9 7 1 ] ) ,
but
contains
a bug that affects
a small percentage
of
permutations
of the input array elements.
The
operation
o f SELECT o n BUGGYFIND i l l u s t r a t e s
(6)
the "ignoring"
of impossible
paths,
(7) the need
for output
assertions
if the system is to con.sistently
generate
bug-revealing
sample test data,
and (8) "path proving,"
wherein
the system attempts
to demonstrate
that a user-supplied
output
assertion is satisfied
for a path.
The feature
(8) was
effective
in revealing
t h e b u g i n BUGGYFIND.
the
Once having arrived
at one or more sample points
within
a subregion
corresponding
to a given program
path,
one may cause the test
program to be executed
with those test data as input in order to actually
test
that path.
An e y e b a l l
examination
of the output data will generally
be relied
upon to determine
if the program has computed intended
results.
It
is also possible
for a user to observe
the symbolic
output
values produced
b y SELECT f o r t h e p a t h s i n
order
to try to understand
what the program is really
computing.
In these cases the only documentation
on
the program provided
t o SELECT i s t h e p r o g r a m c o d e
itself.
As another m o d e of o p e r a t i o n it is p o s s i b l e to
insert assertions, p o s s i b l y in the form of p r o g r a m s
themselves, at various points in the p r o g r a m including the output.
These a s s e r t i o n s can serve as
(i)
e x e c u t a b l e a s s e r t i o n s that are e v a l u a t e d when
the e x e c u t i o n of the p r o g r a m (on n u m e r i c a l
values of input data) reaches the a s s e r t i o n
point, or as
(2)
c o n s t r a i n t c o n d i t i o n s that enable a u s e r to
define subregions of the input space from
which SELECT is to g e n e r a t e the test data, or
(3)
s p e c i f i c a t i o n s for the intent of the p r o g r a m
from which it is p o s s i b l e to verify the p a t h s
of the program.
Note that this does not
imply that the p r o g r a m itself is correct,
which would require that all p r o g r a m p a t h s
are verified.
Examples
We w i s h t o m a k e a c o n f e s s i o n
concerning
the
origin
o f t h e b u g s i n WENSLEYSQROOT a n d BUGGYFIND.
In both cases
the bugs are due to errors
in translation
of the published
algorithms
into LISP.
Our
original
intention
w a s t o a p p l y SELECT t o c o r r e c t
versions
of these programs,
and in both cases
it
was a "struggle"
f o r SELECT t o f i n a l l y
reveal
the
presence
of the bugs.
W i t h regard to (3) suppose that the intent of a
path of a p r o g r a m P R O G can be e x p r e s s e d as an output
assertion; this output a s s e r t i o n could itself be a
p r o g r a m TESTPROG.
SELECT will e x t e n d the alist and
h y p o t h e s i s - l i s t by the statements in TESTPHOG, and
will in turn create additional d i s t i n c t virtual p a t h s
as needed.
For all paths, not e x c e e d i n g the looping
factor, that e v e n t u a l l y lead to an output of TESTPROG,
SELECT will attempt to determine if there e x i s t input
data to P R O G such that the c o n d i t i o n s of T E S T P R O G are
not satisfied.
If such input d a t a exists, for at
least one path through TESTPHOG, then e i t h e r P R O G or
T E S T P R O G is in error.
(As a beneficial s i d e - e f f e c t
counterexample data is produced.)
It is c l e a r that
this limited form of v e r i f i c a t i o n can be c a r r i e d out
by the current system p r o v i d e d the statements of
T E S T P R O G are e x p r e s s e d in the input language a c c e p t e d
by SELECT.
239
The fourth
program,
SCHED-D, i s a p r a c t i c a l ,
perhaps typical,
data processing
program with numerous paths,
each corresponding
to a different
case.
It is this type of program,
rather
than say a
c l e v e r a l g o r i t h m like FIND, that is best suited for
a p a t h - d r i v e n testing system.
WENSLEYSQROOT
The program, w r i t t e n in our input language,
depicted on the n e x t page.
is
The p r o g r a m is supposed to c o m p u t e ~ p (where
0 < p < i) to a c c u r a c y e (where 0 < e < i),
u t i l i z i n g the f o l l o w i n g plan.
For e a c h excursion
around the loop, i.e., r e t u r n i n g to L O O P f r o m
either branch of the COND statement, the p r o g r a m
computes an e s t i m a t e of / p to an a d d i t i o n a l bit of
accuracy.
The p r o g r a m locates p to within a subinterval of length d by ~ c c e s s i v e l y h a l v i n g the
half-open interval [0,I). A f t e r k traversals of
the loop, p has been determined to w i t h i n d = 2 -k.
If d ~ e, tbe program terminates
with an estimate
x = left
endpoint
of the subinterval.
(WENSLEYSQROOT
[LAMBDA (P E)
(* C O M E T :
THIS VERSION HAS A BUG!i!!)
(PROG (D X C T T )
(SETQ D 1 )
( S E T Q X 0)
(SETQ C (TIMES 2 P))
(COND
((LESSP C 2))
(T (RETURN NIL)))
LOOP(COND
((LTQ D E)
(RETURN X)))
(SETQ D (TIMES . 5 D ) )
(SETQ TT (DIFFERI~NCE C
( P L U S (TIMES 2 X) D)))
(COND
( ( G T Q T T 0)
(SETQ X (PLUS X D))
[SETQ C (TIMES 2 (DIFFERENCE C
(PLUS (TIMES 2 X) D]
(* COMMENT: THE TWO P R E C E D I N G .
S T A T E M E N T S S H O U L D BE I N T E R C H A N G E D
TO M A K E T H E P R O G R A M CORRECT)
(GO LOOP))
(T (SETQ C (TIMES 2 C))
(GO LOOP]
The branch s t a t e m e n t (GTQ TT D) e v a l u a t e s to false
w i t h the c u r r e n t e x a m p l e input d a t a (since
p < I D 4p - 4.25 < 0).
Thus c gets updated, and
the true exit from the branch (LTQ D E) leads to
the p r o g r a m exit.
F o r this path the computed test
d a t a is p = .9995, e = .4995, and w i t h these input
values the p r o g r a m returns as an a p p r o x i m a t e square
root of p the value x = 0.5.
A user can determine
that
the intent
of the program is violated--although
barely--since
the error
is ~.9995
- 0.5 > 0.4995 = e.
Hence, the bug is "revealed."
MOVEARRAY
H e r e we a p p l y SELECT t o a p r o g r a m i n w h i c h t h e p r o grammer has forgotten
to handle a particular
case
that he intended
the program to accommodate.
The
intention
of the program is to move a subarray
A[m:n]
of an array A[I:p],
1 < m < n < p, by d positions.
It is assumed that d is bounded so that if d > 0 then
m + d ~ 1, and if d ~ 0, then n + d < p.
Of course
the original
array contents
at posit~ons
m + d,
m + d + 1, ...,
n + d are destroyed
in the process.
The programmer presents
the following
program as a
solution:
(MOVEARRAY
[LAMBDA (M N D)
(FOR I F R O M N T O M B Y -i DO (SETA A (PLUS I D)
(ELT A I ] )
In o p e r a t i n g on a p a r t i c u l a r input-output path of
this program, SELECT will p r o d u c e an alist, w h i c h
will include a n u m e r i c a l value for the output variable x, an llypothesis list, and an input data example w h i c h will bind values to the two input variables
p and e. The user can observe the values returned
for x, p, and e to see if / p - e < x < / p - - t h e
intended r e l a t i o n s h i p a m o n g the values at the output.
[For any path w i t h a l o o p i n g factor of one
S E L E C T does not d e r i v e test d a t a r e v e a l i n g the presence of the bug.] A l o o p i n g factor of two is
required to d e r i v e test d a t a that reveals the bug.
For d > 0 the program is correqt,
but for
(d < 0 ~ A ( n - m > I d l ) ,
the values
of certain
positions
of the subarray
are destroyed
before
they
"have an opportunity
to be moved.
A correct
program
would commence the iteration
with i = m + d for
negative
values of d and with i = n + d for positive
values
of d.
In any event the application
o f SELECT
to the faulty
MOVEARRAY p r o g r a m p r o v i d e s
enough information
to the user to reveal
the presence
of the
bug.
L e t us assume S E L E C T " e x e c u t e s " M O V E A R R A Y w i t h
the numerical values m = 2, n = 4, and s y m b o l i c
values for d, p and the array elements.
Thus three
e x c u r s i o n s around the F O R loop are to be taken, and
no hypotheses are generated i n v o l v i n g index checking.
The first e x c u r s i o n e s t a b l i s h e d the a l i s t entry
For e x a m p l e c o n s i d e r one of the single traversals
around the main loop.
The alist becomes:
(d . 0 . 5 ) ( t t
. 2p-0.5)(x
The hypothesis-list
. 0.5)(c
contains
the
(p < I)(NOT(i ~ e))(2p - 0 . 5
The input
algorithm
data
is
example
p
which satisfies
x = 0.5.
=
the
produced
.9995,
intent
e
=
of
. 4p-3)
(A[4 + d]
conjuncts
~ 0 ) ( 0 . 5 ~ e)
by the
.9995
the
,
since
In e x a m i n i n g a n o t h e r path S E L E C T a t t e m p t s to find
if there exist input d a t a that satisfy the first
three c o n j u n c t s of the above h y p o t h e s i s list, together with the clause (NOT(0.5 < e))--i.e., the
c o m p l e m e n t of the f o u r t h conjunct.
A solution is
p = 0.9995p e = .4995.
Continuing, a l o n g the extension of the p r o g r a m code f o l l o w i n g the false side
of the branch (LTQ 0 E), we u p d a t e the alist w i t h
(d
. 0.25)(tt
,
w h e r e the q u a n t i t y on the right side of the e q u a l i t y
is always taken to apply p r i o r to p r o g r a m execution.
C o r r e s p o n d i n g to the second e x c u r s i o n we have the
a s s i g n m e n t A[3 + d] -- A[3], w h e r e A[3] refers to the
c u r r e n t value.
D e p e n d i n g on the value of d there
is u n c e r t a i n t y w h e t h e r the c u r r e n t A [ 3 ] is the
original A [ 3 ] o r the original A[4].
S E L E C T recogn i z e s the e x i s t e n c e of two p o s s i b i l i t i e s and establishes a branch point.
C o r r e s p o n d i n g to one choice
the h y p o t h e s i s
Benders
program,
• A[4])
4+d=3
is e s t a b l i s h e d and noted by the Gomory a l g o r i t h m to
have the s o l u t i o n d" = -I.
(SELECT also e s t a b l i s h e s
the c o m p l e m e n t a r y h y p o t h e s i s 4 + d ~ 3, w h i c h in
turn will lead to a n o t h e r solution for d.)
The
allst a f t e r the second excursion thus becomes
(A[3 + d] • A [ 4 ] ) ( A [ 4 + d]
. 4p-4.25)
240
• A[4])
For the third
and final
excursion
we h a v e t h e a s s i g n ment
A[2 + d] ~ A[2];
but for d = -1 the current
value of A[2] is the original
value of A[4].
Thus
the final
alist
is
(A[2 + d]
and
the
• A[4])(A[3
sample
input
+ d]
data
• A[4])(A[4
is
+ d]
(BUGGYFIND
[LAMBDA (NN F )
(PROG (M N I J TEMP)
(SETQ M l )
(SETQ N NN)
[WHILE (LESSP M N) GO
(SETQ I M)
(SETQ J N)
[WHILE (LTQ I J)
DO (WHILE (LESSP (ELT A I)
(ELT A F))
DO (SETQ I (ADD1 I)))
(WHILE (LESSP (ELT A F)
(ELT A J))
DO (SETQ J (SUB1 J)))
• A[4]),
d = -1.
For this case it suffices
to analyze
the results
of the symbolic
execution
to detect
a bug--that
is,
it is not necessary
or desirable
to generate
sample
values
for the array elements.
The rightmost
elem e n t of the alist indicates that, for d = -i, the
final value of A[3] is to be the original value of
A[4], which is as intended.
However, the second
element of the alist indicates that the final value
of A[2] is to be the original value of A[4] which
is no~ as desired.
Likewise, the final value of
A[I] is seen to be (incorrectly) the original A[4].
(COND
((LTQ I
(SETQ
(SETA
(SETA
(SETQ
(SETQ
BUGGYFIND
J)
TEMP (ELT A I))
A I (ELT A J))
A J TEMP)
I (ADD1 I))
J (SUB1 J]
(CX~D
((LTQ F J)
(SETQ N J))
((LTQ I F))
(SETQ M I))
(T (GO L]
The two previous examples illustrated the ability
of SELECT to generate relevant sample test data.
In WENSLEYSQROOT the derived test data showed that
execution combined with observation of the numerical values of output variables for a sufficient
number of loop traversals could enable a user to
note a bug.
The bug in MOVEARRAY was revealed by
SELECT deriving a value for one variable and the
user observing the symbolic value of output variables.
We now discuss a program bug which the
above facilities are not effective in revealing,
but which seems to necessitate an analysis of the
program with a user-supplied output assertion.
L])
This bug is quite difficult to detect by conventionally testing the program with random array inputs since the original goal of FIND fails to be
satisfied by extremely few input array patterns.
Assuming (for an array of four elements) that no
pair of array element values are identical, t h e r e
are only two permutations (out of 24 possible) for
which a wrong answer will emerge.
Let us now review the operation of SELECT on the code of BUGGYFIND.
In particular, let us consider a path for
which the GOMORY algorithm could have derived test
data that reveals the bug, but failed to do so.
That is, some feasible input solutions to the
hypothesis-list are relevant to the bug, but for
many other feasible solutions BUGGYFIND gives no
indication of error.
The program BUGGYFIND is Hoare's FIND (Hoare
[1961]) but with an added insidious bug. We recall that the purpose of FIND is to permute the
elements of an array such that all elements to the
left of some inputted position f, have value not
exceeding A[f], and all elements to the right of f
have value not les~ than A[f].
Of course, any
array-sorting program would achieve this goal, but
FIND seems to be appreciably more efficient.
The
strategy in FIND is to move "small-valued" elements
to the left and "large-valued" elements to the
right. Our program with the error is depicted
below.
The program BUGGYFIND is tested with NN = 4, F = 3.
One relevant path generates the hypothesis-list
~OT(A[i]
In the program, left and right boundaries
(originally at positions 1 and NN respectively)
are gradually moved together.
To the left of
the "left" boundary, elements are known to be small
and to the right of the "right" boundary, elements
are known to be large.
The key in determining
whether an element is large or small is in its comparison with some fixed value.
In Hoare's correct
version this fixed value, R, is assigned the value
A[F] whenever a new right or left boundary is established.
Thus any element smaller than R is
known to be smaller than any element larger than R.
In BUGGYFIND instead of the comparison being made
with respect to R it is made with respect to the
current value of A[F].
But, since A[F] is subject
to exchange with another element, it is conceivable
that, after such an exchange, future comparisons
with A[F] do not guarantee the satisfaction of the
large-small relationship.
and
the
< A[4]))(A[2] < A[I])(A[3] < A[4])
(NOT(A[1] < A[3]))
,
alist
(A[i] • A[3])(A[3]
• A[4])(A[4]
• A[I])
A[2] is missing as a lefthand term since it is not
assigned a new value on the path.
The input example that SELECT generates, to satisfy the hypothesis list is
A[ l] = I, A[2] = 0, A[3] = 0, A[4] = i
The final array can be derived by executing FIND
with these values, or by referring to the alist.
In any event the final array is
A[I] = 0, A[2] = O, A[3] = i, A[4] = i
The final array values obviously give no indication
of the lurking bug.
241
Let us now consider the same input-output path of
BUGGYFIND, but with an output assertion appended to
the path.
The output assertion is reflected in the
code of the program TESTFIND,
~SCHED-D
[LAMEDA (SHORT(IN LONGT~NADJTAXINC)
(PROG (L14 L15A L16A)
(SETQ L14 (PLUS SHORT~ L O N G T ~ ) )
[COND
[ ( G T L 1 4 O)
(COND
((LTQ LONGTGN 0)
(SETQ LiSA 0)
(SETQL
(DIFFERENCE Li4 L15A))
(RETURN L))
(T (COND
((LTQLONGT(~ L 1 4 )
(SETQLi5A
(TIMES .5 LONGT(iN))
(SETQ L
(DIFFERENCE Li4 L i S A ) ~
(RETURN L))
(T(SETQ LiSA
(TIMES .5 Li4))
(SETQL
(DIFFERENCE 514 LISA))
(RETURN L]
(T COND
((GTQ SHORT(~N 0)
(SETQL16A
(TIMES .5 (ABS Li4)))
(GO OUT))
((GTQ LONGT(I~ O)
(SETQ Li6A (ABS Li4))
(GO OUT))
(T [SETQ Li6A
(ABS (PLUS SHORT(iN
(TIMES .5 LONGTGN]
(GO OUT]
OUT (SETQ L (MIN i000 ADJTAXINC Li6A))
(RETURN (MINUS L])
(TESTFIND
[LAMBDA (NN F)
(PROG NIL
(BUGGYFIND NN F)
(RETURN (AND
(FOR I FROM 1 TO F ALWAYS
(LTQ (ELT A I)(ELT A F)))
(FOR I FROM (ADD1 F) TO NN
ALWAYS (LTQ (ELT A F)
(ELT A I ] )
which tests the relationship among the final (symbolic) values produced by BUGGYFIND for this path.
If SELECT can identify input data such that TESTFIND
is not true then there is an error either in BUGGYFIND or TESTFIND.
SELECT, in effect, carries out
a macro-expanslon of TESTFIND and considers the
paths of TESTFIND in the same systematic way as it
did for BUGGYFIND.
What SELECT produces, for the
path in question in BUGGYFIND is the answer that
there exists data, such that TESTFIND returns FALSE,
and that satisfies the hypothesis-list:
(NOT(A[2] < A[4]))(NOT(A[i] < A[4]))
(A[2] < A[I])~A[3] < A[4])(NOT(A[i] < A[3]))
An example of an input array satisfying these hypotheses is
All] = 3, A[2] = 2, A[3] = 0, A[4] = I
The r e s u l t a n t
A[I]
=
output
0,
A[2]
array
=
2,
is
A[3]
=
1,
A[4]
=
3
,
Even with these simplifying assumptions there are
enough s i g n i f i e a n t l y different input cases to consider to make this an interesting exampleJ The
program enters the computation of Part IIl, Schedule
D at a point where the net short-term gain/loss and
net long-term galn/loss have already been determined
by summing their components (in Parts I and II).
These two (algebraically signed) quantities, called
SHORTG~[ and LONGT(iN in the program, together with
the adjusted taxable income (ADJTAXINC), a non-negative quantity, constitute the three input parameters
to SCHED-D.
The program computes a single (signed)
quantity L representing the desired combination of
short-term and long-term gains or losses.
Various
local variables in the program, e.g., Li4, LI5, and
Li6A, represent entries on correspondingly numbered
lines of the tax form.
which reveals the bug since, for F = 3, A[2] > A[3].
This example has illustrated the inability of
SELECT always to derive relevant test data.
However,
the inclusion of an output assertion, as a program
involving functions and predicates understandable
to SELECT, can reveal a bug that affects at least
one of the paths that SELECT inspects.
As a final
note with regard to BUGGYFIND there are several
hundred syntactic paths for an array of length 4,
but SELECT weeds out the many impossible paths and
produces d a t a for the 18 feasible paths.
Schedule D--A simple non-loop program
A simple example of a program without any control
loops is provided by the program SCHED-D (in LISP)
shown below, that calculates the net (positive or
negative) contribution to income from a combination
of the net short-term and long-term capital gains
or losses.
Mainly for tutorial reasons we have simplified the calculation by assuming that Parts IVVI of Schedule D, Federal Tax Form i040 do not
apply (as is often actually the case, unless one
uses the Alternative Tax Computation, or has a
long-term capital
loss component carryover from
mo re t h a n t h r e e y e a r s b a c k , e t c . ) .
The r e a d e r i s
referred
t o t h e IRS t a x i n s t r u c t i o n s
for explanat i o n s of these terms.
There are basically six distinct paths through
this flowchart,
i f we c o n s i d e r t h a t t h e c a l c u l a t i o n o f t h e minimum o f t h e t h r e e q u a n t i t i e s :
L16A,
$ 1 0 0 0 , and ADJTAXINC c o r r e s p o n d s t o a s i m p l e p r o cedure call of the function MIN(x,y,z).
However,
our present mechanism for such procedure calls
actually does a maeroexpansion of the procedure
code, which itself
involves four sub-paths,
so that
a t o t a l o f 3 + 4*3 = 15 p a t h s t h r o u g h t h e ( e x p a n d e d )
program results.
All the program branching occurs
on i n e q u a l i t y
tests of (positive,
zero, or negative)
r e a l n u m b e r s , and a l t h o u g h t h e r e a r e no l o o p s , t h e
number of distinct
possibilities
is large enough
so that it is difficult
or impossible for most
242
p e o p l e to keep all the c o n t i n g e n c i e s c l e a r l y in
mind w i t h o u t a f l o w c h a r t o r table of cases.
When
the e x c l u d e d p o s s i b i l i t i e s (e.g., the A l t e r n a t i v e
T a x C o m p u t a t i o n ) are taken into account, this
b e c o m e s q u i t e h o p e l e s s w i t h o u t some such aids.
A n a u t o m a t e d V e r i f i c a t i o n S y s t e m (AVS) is b e i n g
d e v e l o p e d by M i l l e r [1974, 1974a] of G e n e r a l
R e s e a r c h Corporation.
A V S insures that e a c h
line of code is tested
at least
once by given
test data,
and provides
programming
aids to the
user/programmer
in the form of path conditions,
and in establishing
which further
inputs
will
e x e c u t e code in the remaining, u n e x e r c i s e d portions.
It d o e s not, however, a t t e m p t to be as
e x h a u s t i v e as our system is in e x e r c i s i n g each
line of code for each path that m i g h t include
that line of code.
N o r does it do any s i g n i f i c a n t
p r o c e s s i n g of the path conditions.
The point to our S E L E C T s y s t e m id r e l a t i o n to
such e x a m p l e s is that it will a u t o m a t i c a l l y
g e n e r a t e sample input d a t a (SHORT(I~, LONGT(i~,
A D J T A X I N C ) for each of the p o s s i b l e p r o g r a m paths.
L e t us examine how S E L E C T handles just o n e of these
15 paths.
R e f e r r i n g to the s u b f u n e t i o n MIN and to
the text of the p r o g r a m SCHED-D, the path in quest i o n c o r r e s p o n d s to the FALSE exit from the
"(GTQ
SHORTGN 0)", at which point MIN is invoked
on the arguments, X = I000, Y = ADJTAXINC,
Z = L i 6 A = (TIMES .5 (ABS (PLUS SHORT(IN LONGTGN))).
The b r a n c h i n g within the p r o c e d u r e MIN is:
(LESSP Y X) is FALSE, and (LESSP X Z) is TRUE,
r e s u l t i n g in an o u t p u t value (MINUS L) =
(MINUS X) = -i000.
For this path, S E L E C T constructs input test data: SHORTGN = 0,
LONGT(~ = -2001, and an a d j u s t e d - t a x a b l e income
A D J T A X I N C = I000.
The r e a d e r may c o n v i n c e himself
by tracing through the p r o g r a m w i t h this data to
see that the stated path is executed and that the
o u t p u t (-i000), i.e., a net d e d u c t i o n of $i000, is
correct.
N a t u r a l l y , when the p r o g r a m is a c t u a l l y
executed w i t h this input data, this is also confirmed.
IV.
Related
The TRW A u t o m a t e d S o f t w a r e Q u a l i t y A s s u r a n c e
S y s t e m (PACE) of Brown et al. [1973] is b a s i c a l l y
a c o l l e c t i o n of t r a c i n g and m a n a g e r i a l tools w h i c h
assist in a s s e s s i n g test coverage, but do n o t generate test data.
We also call the reader's a t t e n t i o n to e x c e l l e n t
c h a r a c t e r i z a t i o n s of the w h o l e subject of p r o g r a m
testing by Hetzel [1973, 1973a] in his book (Hetzel
[1973b])
devoted
to a collection
of papers on this
subject.
V.
Conclusions
We h a v e d e s c r i b e d
an experimental
system,
SELECT,
that can aid a user in testing
and debugging
programs.
The system performs
a symbolic execution
of the paths
(input
to output)
of the program.
L o o p s are handled in a s y s t e m a t i c m a n n e r u n d e r u s e r
control; each loop can be executed as m a n y times as
a u s e r feels n e c e s s a r y to c o n v i n c e himself of its
correctness.
In p r a c t i c a l p r o g r a m s m a n e syntactic
paths may n o t be e x e c u t a b l e paths.
In
growlng
a
path, S E L E C T a t t e m p t s to d e t e r m i n e if there exist
input d a t a that will d r i v e e x e c u t i o n through the
subpath developed so far.
If n o such data exists,
the p a t h is impossible and S E L E C T ceases its cons i d e r a t i o n of that path.
Thus the n u m b e r of paths
a c t u a l l y subjected to e x e c u t i o n can be s i g n i f i c a n t l y
reduced.
work
A n u m b e r of r e s e a r c h e r s have i n d e p e n d e n t l y
arrived at and d i s c u s s e d the same basic ideas on
which our system is founded, i.e., g e n e r a t i n g logical c o n d i t i o n s for each p r o g r a m paths and s o l v i n g
these logical e q u a t i o n s to provide instances of
input d a t a which will d r i v e the p r o g r a m through that
e x e c u t i o n path.
However, w i t h one exception, o u r
system is the only one we know of w h i c h a c t u a l l y
a t t e m p t s to implement these ideas in a s y s t e m that
can also carry out a semantic analysis of a program.
The e x c e p t i o n is an e x p e r i m e n t a l system c u r r e n t l y
b e i n g worked on by J. King, [1975], at the IBM Watson R e s e a r c h Center, Y o r k t o w n Heights, N e w York,
King's system (called EFFIGY) is an o u t g r o w t h of
his e a r l i e r I1969] w o r k on formal v e r i f i c a t i o n of
correctness.
E F F I G Y performs (as does SELECT) a
s y m b o l i c execution that d e v e l o p s path c o n d i t i o n s
by forward substitution.
It also makes use of a
d e d u c t i v e system (theorem prow)r) derived from his
e a r l i e r p r o g r a m v e r i f i e r to generate test d a t a for
each path, and to prove that some paths are
,f.
.
,t
Imposslble.
It handles simple v a r i a b l e s and subscripted v a r i a b l e s (arrays).
For each e x e c u t a b l e
following:
The e x p e r i m e n t a l system d e s c r i b e d in this paper
is closely related to a system plan drawn up indep e n d e n t l y by H o w d e n [1974].
In particular, Howden
also makes use of the n o t i o n s of a n a l y z i n g a prog r a m into d i s t i n c t paths, g e n e r a t i n g logical cond i t i o n s to correspond w i t h these paths, and s o l v i n g
these sets of conditions to yield, automatically,
test d a t a with w h i c h to e x e r c i s e the program.
However, to the best of o u r knowledge, Howden has
n o t carried out any actual i m p l e m e n t a t i o n of his
scheme.
N o r does H o w d e n ' s scheme a c c o m m o d a t e
array variables.
path S E L E C T
can produce
the
e
S a m p l e input test d a t a that will cause the
path to be executed.
The u s e r can run the
p r o g r a m with this d a t a and o b s e r v e the outputted results.
e
Simplified symbolic values for v a r i a b l e s that
do not require actual n u m e r i c a l values in
order to ensure e x e c u t i o n of the paths.
These
symbolic values can be observed by the u s e r as
an aid to u n d e r s t a n d i n g what the p a t h in question is computing.
@
A proof that a u s e r - s u p p l i e d output a s s e r t i o n
is satisfied for all d a t a that d r i v e s the program through the path.
At p r e s e n t S E L E C T can d e c i d e each of the above three
issues if the h y p o t h e s e s that u n i q u e l y d e f i n e a path
correspond to a s y s t e m of linear inequalities (with
integer or real o r mixed c o n s t r a i n t s on the solution values) or to c e r t a i n n o n l i n e a r inequalities.
Array semantics and rules for a l g e b r a i c and logical
s i m p l i f i c a t i o n are also known to SELECT.
243
t h a t c a n be d e t e c t e d by a f o r w a r d - a c t i n g s y m b o l i c e x e c u t o r w h i c h c a n n o t be d e t e c t e d by c o n v e n t i o n a l c o m p i l e r s .
Among s u c h c o n d i t i o n s a r e t h e p o t e n t i a l o f o v e r f l o w a n a
underflow conditions, divisionby
zero, out-of-bounds
array references,
r e f e r e n c i n g an u n i t i a l i z e d
variable,
and p e r h a p s e v e n i n f i n i t e
looping.
I n some s i m p l e
c a s e s some o f t h e s e c o n d i t i o n s c a n e v e n be d e t e c t e d a t
c o m p i l e t i m e , b u t i n g e n e r a l t h e i r d e t e c t i o n demands
the presence of a fairly sophisticated
deductive system.
An e x c e l l e n t d i s c u s s i o n o f how t h e a n a l y s i s and
d e t e c t i o n o f s u c h e r r o r c o n d i t i o n s c a n be i n t e g r a t e d
into a program verification
system has been carried
o u t by S i t e s [ 1 9 7 4 ] i n a d o c t o r a l t h e s i s .
We p l a n t o
a u g m e n t t h e SELECT s y s t e m by i n c o r p o r a t i n g f e a t u r e s
of this type.
The p r e s e n t s y s t e m can be u s e f u l f o r m o d e r a t e s i z e p r o g r a m s i n t h a t i t s y m t e m a t i z e s many o f t h e
d e b u g g i n g t r i c k s now i n common p r a c t i c e ,
The s y s tem a l s o p r o v i d e s an i n t e r a c t i o n w i t h t h e u s e r a t
a s y m b o l i c l e v e l t h a t s h o u l d be e o m p r e h e n d a b l e t o
the user.
Although there is extensive symbolic
manipulation being carried out, it is significantly
less than that associated with mathematical program
verification
by, s a y , F l o y d ' s m e t h o d .
H e n c e , SELECT
can p r o b a b l y h a n d l e l a r g e r programs than contemporary prototype verification
s y s t e m s , b u t may p r o v i d e
less assurance regarding the correctness of the
program.
On t h e o t h e r h a n d , t e s t i n g a p r o g r a m w i t h SELECTgenerated data in the actual computer envir onm e nt
w i l l be r e a l i s t i c
with respect to features (e.g.,
r o u n d - o f f , o v e r f l o w , c o m p i l e r and o p e r a t i n g s y s t e m )
t h a t may be i n c o r r e c t l y a x i o m a t i z e d ( o r i g n o r e d ~ )
in a formal proof~
There are clearly significant
limitations
to the
p r e s e n t SELECT s y s t e m , t h e m o s t p r o m i n e n t b e i n g t h e
lack of a p r o p e r mechanism f o r h a n d l i n g program
abstractions.
As a minimum t h e r e s h o u l d be a mechanism for hierarchically
handling calls to subroutines--a
facility
not present in the current
system.
We have f o r t h e p r e s e n t a r r a n g e d m a t t e r s
i n t h e p r o g r a m a n a l y z e r so a s t o p r o v i d e t h e
effect of a macroexpansion--substituting
the
e x p a n d e d (and i n s t a n t i a t e d )
code i n t o t h e p r o g r a m
itself.
[But, this expedient (though it works)
a p p e a r s u l t . i m a t e l y t o be a s e l f - d e f e a t i n g
way t o
p r o c e e d , p a r t l y b e c a u s e i t l e a d s t o an e x p l o s i o n
i n t h e n u m b e r o f p a t h s t o be f o l l o w e d , and p a r t l y
b e c a u s e i t f o r e c l o s e s any p o s s i b i l i t y
of carrying
out the testing of a modularly constructed program
in a h i e r a r c h i cal
manner.]
Our v i e w a s t o t h e
p r o p e r way t o h a n d l e t h i s s i t u a t i o n
is to attempt
to characterize
each subroutine invoked in a program
by i n p u t and o u t p u t a s s e r t i o n s ,
much a s i n t h e F l o y d
technique for program verification.
Such s u b r o u t i n e
specifications
n e e d n o t be c o m p l e t e i n t h e f o r m a l
correctness sense, nor do we need to carry out formal
proofs for each subroutine (although, that possibility
is not ruled out).
I n s t e a d , we need o n l y t e s t e a c h
s u c h s u b r o u t i n e ( a s a s e p a r a t e and d i s t i n c t
program
m o d u l e ) u n t i l we h a v e s u f f i c i e n t
confidence that it
meets its input/output specifications.
From t h a t
p o i n t o n , w h e n e v e r t h a t module i s i n v o k e d i t c a n b e
replaced, for purposes of testing the overall program,
by t h o s e i n p u t / o u t p u t
specifications.
By t h i s means
we s e e k t o accommodate p r o g r a m m o d u l a r i t y ( a n d h i e r a r c h y ) i n a n a t u r a l and e f f i c i e n t
way i n t o t h e t e s t
data generation process.
There are still
unresolved
issues in this connection.
U l t i m a t e l y , h o w e v e r , we
feel that such a combination of testing with elements
of the formal verification
p r o c e s s i s t h e b e s t way t o
proceed.
The m a i n i s s u e s a r e :
•
Choosing a suitable
the assertions.
specification
language for
•
Expanding the manipulative powers of the system
beyond t h a t of i n e q u a l i t i e s
and a l g e b r a i c s i m p l i fication,
to permit the processing of higherlevel operations that will originate from the
subroutine specifications.
Another desirable augmentation relates to the detection of various improper conditions.
There are a
number o f i m p r o p e r c o n d i t i o n s ( i n v a l i d o p e r a t i o n s )
244
Beyond t h e s e a u g m e n t a t i o n s and e m b e l l i s h m e n t s t h e r e
is a significant
need to e x p e r i m e n t w i t h f o r m a l
testing concepts of the type discussed in this paper.
We f o r e s e e p r o g r a m v e r i f i c a t i o n ,
or better the synthesis of provably correct programs, as being the
most promising approach toward improving software
reliability.
Although these techniques should ultim a t e l y be a c c e p t e d by m o s t p r o g r a m m i n g s h o p s , t h e
t e s t i n g o f p r o g r a m s w i l l r e m a i n f o r many y e a r s a s
the primary certification
tool.
We a r e c o n v i n c e d
( a s a r e many o t h e r s who h a v e s t u d i e d t h e p r o b l e m o f
testing) that there is little
hope that the fully
a u t o m a t i c g e n e r a t i o n o f t e s t d a t a w i l l be e f f e c t i v e
where used a l o n e .
A p r o g r a m m e r ' s i n s i g h t on f o c u s i n g on c e r t a i n s e c t i o n s o f a p r o g r a m o r on c e r t a i n
data relationships
s e e m s i n v a l u a b l e i n d e c i d i n g on
relevant test data.
However, the main f a l l i b i l i t y
of programmers is in covering all cases of interest-a lack that a symbolic processing system of the type
discussed here can handle.
Acknowledgments
The a u t h o r s w i s h t o a c k n o w l e d g e t h e c o n t r i b u t i o n s o f Andrew J . K o r s a k and M i l t o n W. Green
t o t h e work r e p o r t e d h e r e , i n p a r t i c u l a r
to the
algorithms used for solving systems of linear
inequalities
and n o n l i n e a r i n e q u a l i t i e s .
we a l s o w i s h t o t h a n k P e t e r J . Wong,
R a l p h E. K e i r s t e a d , L a w r e n c e R o b i n s o n and
Jack Goldberg for numerous illuminating convers a t i o n s on t h e s u b j e c t s o f p r o g r a m m i n g m e t h o d o l o g y , p r o g r a m t e s t i n g and d e b u g g i n g , and l i n e a r
and n o n l i n e a r p r o g r a m m i n g .
References
[Hoare 6 1 ]
C. A. R. Hoare, " A l g o r i t h m 65, FIND,"
Comm. ACM, Vol. 4, No. 7, p. 321 ( 1 9 6 1 ) .
[ B e n d e r s 62]
J . F. B e n d e r s , " P a r t i t i o n i n g P r o c e d u r e s f o r
S o l v i n g M i x e d - V a r i a b l e s Programming P r o b l e m s , " N u m e r i s c h e M a t h e m a t l k 4, pp. 238252 ( 1 9 6 2 ) .
[Hoare 7 1 ]
C. A. R. Hoare, " P r o o f o f a Program: FIND,"
Comm. ACM, Vol. 14, No. 1, p. 39 ( 1 9 7 1 ) .
[Brown, e t a l . , 7 3 ]
J . R. Brown, A. J . d e S a l v l o , D. E. H e i n e , and
J . G. P u r d y , " A u t o m a t e d S o f t w a r e Q u a l l t y
A s s u r a n c e , " i n Program T e s t Methods (W. C.
H e t z e l , e d . ) , P r e n t l c e - H a 1 1 , I n c . , Englewood
C l l f f s , N. J . ; pp. 181-203 ( 1 9 7 3 ) .
[Howden 74]
W. Howden, "Methodology f o r t h e A u t o m a t i c
G e n e r a t i o n o f Program T e s t D a t a , " T e c h n l c a l
R e p o r t No. 41, Dept. o f I n f o r m a t i o n and
Computer S c i e n c e , U n i v e r s i t y o f C a l i f o r n l a ,
I r v l n e (15 F e b r u a r y 1974).
[ D e u t s c h 73]
L. P. D e u t s c h , "An I n t e r a c t i v e P r o g r a m
V e r i f i e r , " Ph.D. d i s s e r t a t i o n ,
University
of Callfornla, Berkeley, California (June
1973).
[ I g a r a s h i , e t a l . 73]
S. I g a r a s h l , R. London, and D. Luckham,
" A u t o m a t i c V e r i f i c a t i o n o f P r o g r a m s I: A
L o g i c a l B a s i s and I m p l e m e n t a t i o n , " Memo
AIM-200, S t a n f o r d A r t i f i c i a l
Intelligence.
L a b . , S t a n f o r d , C a l i f o r n i a (May 1973).
[ E l s p a s , e t a l . , 72]
B. E l s p a s , M. W. G r e e n , K. N. L e v i t t , and
R. J . W a l d i n g e r , " R e s e a r c h i n I n t e r a c t i v e
Program P r o v i n g T e c h n i q u e s , " SRI R e p o r t
8398-11, S t a n f o r d Research I n s t i t u t e ,
Menlo
Park, California (1972).
[Katz-Manna 73]
S. M. Katz and Z. Manna, "A H e u r i s t i c
Approach t o Program V e r i f i c a t i o n , "
Proc.
IFCAI-73 ( A u g u s t 1973).
[ K i n g 69]
J . C. K i n g , "A Program V e r i f i e r , " Ph.D.
dissertation,
Carnegle-Me11Qn U n i v e r s i t y ,
P i t t s b u r g h , P e n n s y l v a n l a (September 1969).
[ E l s p a a , e t a l . , 73]
B. E l s p a s , K. N. L e v i t t , and R: J . W a l d i n g e r ,
"An I n t e r a c t i v e S y s t e m f o r t h e V e r i f i c a t i o n
o f Computer P r o g r a m s , " F i n a l R e p o r t , SRI
P r o j e c t 1891, S t a n f o r d R e s e a r c h I n s t i t u t e ,
Menlo P a r k , C a l i f o r n i a ( 1 9 7 3 ) .
[ K i n g 75]
J . C. K i n g , "A New Approach t o Program T e s t I n g , " 1975 I n t e r n a t i o n a l C o n f e r e n c e on
Rellable Software.
[ E l s p a s 74]
B. E l s p a s , "The S e m i a u t o m a t i c G e n e r a t i o n o f
I n d u c t i v e A s s e r t i o n s f o r P r o v i n g Program
C o r r e c t n e s s , " I n t e r i m R e p o r t , SRI P r o j e c t
2686, S t a n f o r d R e s e a r c h I n s t i t u t e ,
Menlo P a r k ,
C a l i f o r n i a ( J u l y 1974).
[Mlller 74]
E. F. M i l l e r , J r . , e t a l . , " S t r u c t u r a l l y
Based A u t o m a t i c Program T e s t i n g , " p a p e r
p r e s e n t e d a t EASCON ' 7 4 , W a s h i n g t o n , D . C . ,
O c t o b e r 7 - 9 , 1974, a v a l l a b l e from G e n e r a l
Research Corporation, Santa Barbara,
C a l l f o r n l a , A u g u s t 12, 1974.
[Gomory 63]
R. E. Gomory, "An A l g o r l t h m f o r I n t e g e r
Solutlons to Linear Programs," in Recent
A d v a n c e s i n Z a t h e m a t l c a l P r o g r a m m i n g , R. L.
G r a v e s and P. Wolfe ( e d s . ) , McGraw-Hill,
New York ( 1 9 6 3 ) .
[Miller 74~
F. F, M i l l e r , J r . , " O v e r v i e w and S t a t u s Program V a l l d a t l o n P r o j e c t , " R e p o r t d a t e d
1 O c t o b e r 1974, G e n e r a l R e s e a r c h C o r p o r a t i o n ,
Santa Barbara, Callfornla.
[Hetzel 73]
W. C. Hetzel, "A Definitional Framework," in
Program Test Methods (Hetzel 1973b); pp. 7-10.
[Sites 74]
R. S i t e s , " C l e a n T e r m i n a t i o n o f Computer
P r o g r a m s , " Ph.D. d i s s e r t a t i o n ,
Stanford
U n i v e r s i t y , S t a n f o r d , C a l i f o r n i a ( J u n e 1974).
[Hetzel 73a]
W. C. H e t z e l , " P r i n c i p l e s o f Computer Program
T e s t i n g , " i n Program T e s t Methods ( H e t z e l
1973b); pp. 1 7 - 2 8 .
[Teitelman, e t a l . , 72]
W. T e t t e l m a n , D. G. Bobrow, A. K. H a r t l e y ,
and D. L. Murphy, BBN-LISP, Tenex R e f e r e n c e
Manual ( B o l t , B e r a n e k , and Newman, I n c . ,
C a m b r i d g e , M a s s a c h u s e t t s (1972).
[ H e t z e l 73b]
W. C. H e t z e l , Program T e s t M e t h o d s , A
c o l l e c t i o n o f p a p e r s b a s e d on t h e
P r o c e e d i n g s o f t h e Computer Program T e s t
Methods Symposium h e l d a t t h e U n i v e r s i t y o f
N o r t h C a r o l i n a , C h a p e l K i l l , 1972, and e d i t e d
by W. C. H e t z e l , P r e n t i c e - H a l l , I n c . ,
Englewood C l i f f s , N. J . ( 1 9 7 3 ) .
[ W e g b r e i t 74]
B. W e g b r e l t , "The S y n t h e s i s o f Loop P r e d i c a t e s , " Comm. ACM, Vol. 16, No. 2, pp. 102112 ( F e b u r a r y 1974).
[Wensley 58]
J . H. W e n s l e y , "A C l a s s o f N o n - A n a l y t i c
I t e r a t t v e P r o c e s s e s , " Computer J o u r n a l ,
V o l . 1, No. 4, pp. 163-167 ( 1 9 5 8 ) .
245