From: AAAI-86 Proceedings. Copyright ©1986, AAAI (www.aaai.org). All rights reserved.
Generating
Tests by Exploiting
Mark
Harper
MIT Artificial
Designed
Behavior
Shirley
Intelligence
Laboratory
545 Technology Square
Cambrzdge, MA 02139
(mhs@mit-htvax.arpa)
Abstract
One of the hardest
pattern
generation
problems
for a complex
because it requires reasoning
behavior
can be extremely
operations
that
behavior.
in digital
device.
circuit
design
This is difficult
about how to control
complex.
Knowledge
the device was designed
is test
in part
a device whose
of the specific
to perform
can help solve
this problem.
The key observation
is that a device’s designed
behavior
is often far more limited than the device’s potential behavior.
This limitation
translates
into a reduction
of the search
necessary
to achieve planning
goals. We describe an implemented
program
based
on this idea.
When
this
to generate
tests
where
illustrates
both
We represent
graph,
through
the operations
defined
tion set defined
to be complete.
When
reasoning
specializing
about
how to manipulate
complex
systems (e.g. for test generation
or diagnosis),
system is designed to behave is an important
to apply.
This
used in several
knowing how the
kind of knowledge
paper demonstrates
how this knowledge
can be
ways to produce
an effective problem solver for
one of the hardest
problems
in electronic
design:
generation
for VLSI devices.
Test generation
encounters
several traditional
including
conjunctive
planning,
and remains
test
pattern
AI concerns,
an important,
un-
solved problem:
existing programs
are notably
unsuccessful
large circuits.
Human test programmers
are more successful,
part because
ning
engineered
they use knowledge
from many
on
in
and a variety
sources
of reasoning
techniques.
Here we focus on one kind of knowledge
- designed behavior
- which they find important.
At the core of our test generation
one key observation:
tions that
ited than
of inputs
When
signed
a device’s
a device
is intended
to carry
its potential behavior.
to a disk controller
solutions
behavior,
system
designed
is a planner
behavior,
based
This
to planning
problems
can be found within
this limitation
can translate
into a reduction
Hypothesis:
efficiently
work
that
relies
every
exercised
on what
component
while
staying
deof
compo-
we call the Designed-Behavior
in the device
within
can be fully and
the device’s
designed
This report describes research done at the Artificial Intelligence Laboratory of the Massachusetts
Institute of Technology and at GenRad Inc. Support for the laboratory’s Artificial Intelligence research on digital hardware
troubleshooting
is provided in part by the Digital Equipment Corporation,
and in part by the Advanced Research Projects Agency of the Department
of Defense under Office of Naval Research contract N00014-80-C-0505.
884
National
Conference
/ ENGINEERING
method
is most
interfaces,
ating a behavior
lieve the method
of the building
e.g. the instruc-
The list of operations
value,
a testing
against
matches
to create
appropriate
and
section
for devices
characteristics
of this problem
strategy.
Section 4 introduces
examples
throughout
tails of our method.
This
whose
and
plan-
command
of distinct
operations
they
the non-trivial
effort of cre-
of microprocessor
3 describes
graphs
solutions.
circuits, we betests for many
systems
(e.g. processors,
UART’s, graphics
controllers,
and the like).
The next section gives an overview
of the
problem,
is assumed
goal, like loading
the behavior
graph.
In the domain of digital
is appropriate
when generating
blocks
which
with a behavior
behavior
interfaces
are narrow in the number
admit, since each operation
requires
the
test
generation
consequences
for representation
a simple processor
the paper, while section
The method’s performance
of several
and reasoning
which provides
5 contains the deon the processor
is shown in section 6. Finally, a simple account
of processor
design is presented
in section 7, which provides justification
for the
performance
2
achieved.
on Artificial
intelligence
The
Task
We are interested
in the task
of test
generation
for VLSI com-
ponents.
This task is part of the quality control phase of circuit
manufacturing
where we verify that the physical
circuit in fact
Test generation
takes a design
produces
the designed
behavior.
description
for a circuit and produces
a set of tests.
Taken together the tests must specify sets of inputs and predicted
outputs
such that,
puts
AAAI-86
successful
is shown
tests
fails
how data is supposed
to flow
these graphs by simulation
of
by matching
a certain
neces-
out, is often far more lim-
For example, very few sequences
correspond
to valid commands.
the search necessary
to achieve planning
goals.
In testing,
the goals involve manipulating
individual
nents.
on
i.e. the opera-
operates
with
An example
at the device’s
for a processor.
the search
successfully
generates
is valid, and (quickly)
of designed
a structure
that describes
We generate
the device.
a register
Introduction
we can reduce
it is not.
cases.
the space
The planner
1
is true,
sary to generate
tests. Our program
in situations
where this assumption
match
when
the inputs
the predicted
are applied
values
to the circuit
(i.e. the circuit
we can be confident
the circuit was manufactured
Tests are traditionally
created by partitioning
and all out-
passes
all tests)
correctly.
the design
into
components
and generating
a test for each component
by (1)
working
out how to test the component
as if it were alone, and
(2) working out how to execute that test within the context
larger circuit.
The second problem is conjunctive
planning,
of the
since
it involves
achieving
multiple,
potentially-interacting
have to work out how to manipulate
to cause
a desired
The
classical
pattern
of activity
approaches
goals:
the circuit’s
(e.g.
around
primary
a component.
[Roth66])
use
a combination
of heuristic
search and constraint
propagation
using
representationsof the circuit’s structure
and behavior.
runtimes
are excessive for anything
cuits and simple state machines,
we
inputs
gate-level
Current
other than combinational
cirand we think this is a direct
consequenceof the use of gate-level
representations.
More recent
to hiresearch
(e.g. [SinghG])
appl ies similar search techniques
erarchical
models of device structure
and behavior,
yet still does
not take advantage
of knowing the designed
behavior.
For com-
plex circuits,
the human experts
any existing test generation
than
describes
several
limiting
3
ways
remain
much more successful
program,
and the next section
- inspired
by what
the
experts
Figure
In this
section
of the Problem
we describe
two
domain and the effects
and reasoning
strategy.
they
3.1
Designed
Purposefully
characteristics
have on our choices
designed
bewhich essen-
tially
circuit
are annotated
or the
notion
contain expressions
described
in terms
problem
The point
tained within
of representation
has
been
used
in several
unspecified,
analog
circuit
we need
to
was sufficient
worked.
know what
We require
that
purpose
to aid in determining
more
the relative
algorithms
behavior
of the behavior
graph representation
the set of graphs
are many patterns
information
is, i.e. the tasks
than
sizes of solution spaces
search. The sets contain
graphs
can provide
interfaces.
generated
behavior.
Hence,
dicates that there
this;
the system
the designer
behavior
how an
opergraphs
is this: conof activity
a complete
Because
from them
list of operations
de-
this list is complete,
the
represent
all normal
circuit
failure to find a useful pattern
of activity
is no solution within the normal operations
inof
the circuit.
is
supposed
to accomplish.
Our program
then uses this information
to focus on solutions
to testing problems.
Figure 1 illustrates
different test generation
For efficiency,
with variables in them, e.g. a READ cycle is
of a symbolic address and symbolic data.
fined for the circuit’s
programs
for reasoning
about physical
systems.
In [DeKleer79],
for example, the knowledge
that a device had some purpose, even
though
traces.
and the corresponding
around
every component
in the circuit.
“All our planner
must
do” is find the patterns
of activity that are useful in testing.
We
Systems
of purpose,
simulation
are parameterized,
assume
Teleology,
states
We explicitly
represent
the space of a circuit’s
havior with the behavior
graphs mentioned
above,
ations
Domain
of the
sizes of sets of circuit
do - of
search.
Characteristics
1: Relative
that
pos-
Extreme
3.2
A second
and
Complexity
dominating
characteristic
of this
domain
is the
sible states of a circuit,
i.e. the different
ways of consistently
assigning
values to the circuit’s nodes.
If the components
com-
extremely
complex behavior of modern VLSI components.
Algorithms which manage this complexity
are essential.
One way to
prising the circuit are disconnected,
then there is no constraint
between their states, and the set of possible circuit states is the
do this is to identify
cross product
of the sets of individual
component
states.
Connecting
the components
together
makes many potential
states
inconsistent,
resulting
in a smaller set of possible states.
This is
one that
focuses
closely
involves
the rest
of the circuit,
the space that a planner
with perfect ability
straints
could search.
Since local constraint
to propagate
con-
propagation tech-
The
traditional
build our problem
ing at a different
micro-planner
useful
abstractions2.
partitioning
of the problem
on a single
into
component
suggests
that
two parts,
and another
we might
that
usefully
solver in two parts, each part potentially
workabstraction
level. And this is what we do: a
is responsible
for solving
conjunctive
goals around
niques are incomplete
in their ability to detect global conflicts,
algorithms
using these techniques
search a somewhat
larger space
individual
components,
and a macro-planner
is responsible
for
taking the output of the micro-planner
and then controlling
grosser
and backtrack
aspects of the circuit
We must describe
when
inconsistencies
are found.
Within the space of consistent
states
space of behaviors
the circuit was designed
of the circuit lies the
to perform.
This is the
space our test generator
searches.
The ratio of designed behavior
to possible behavior
depends upon the particular
circuit, but the
more specialized
and complex
the circuit,
the smaller
it tends
to
be. 1
‘In the case of the MARK-2
processor,
this ratio
is on the order of 22”“.
Naturally this number says nothing about how the spaces are searched nor
about the frequency and distribution
of solutions within the spaces. We
haven’t yet done the performance measurements
for some typical circuits
that would yield these numbers. But, as suggested by the performance
of
our method on a microprogrammed
processor, we expect our method to
be a useful addition to the spectrum of techniques helpful for solving test
generation problems.
state, namely its data registers.
the circuit structure
and behavior
two parts of our program.
system models the circuit
nents,
roughly
for these
For simplicity’s
sake, the prototype
structure
as a flat network of compo-
at the level of the block
diagrams
in databooks.
3
The system uses four levels of behavioral
description.
The
simulation
models for the individual
components
are the foundation.
We only require them to predict
outputs
from inputs,
21n fact, because
are abstractions
designer.
the circuits are designed, it is not surprising
that there
readily available; they are the abstractions
used by the
3Note that, although
individual
gates.
the model is flat, it is considerably
above the level of
AUTOMATED REASONING
I 885
EXAMPLE
INPUT
SEQUENCES
+ COMPONENT
1
VIA
BEHAVIOR
SIMULATION
.
tion after
GRAPHS
~UMMIRIIAT~VIA
REGISTER
accomplished
using Design for Testability
techniques.
The next section introduces
a simple processor
which we use
as an illustration
throughout
the rest of the paper, and the sec-
BEHAVIORS
SUMMARIZATION
COMPONENT
CHANGES
that
4
The
MARK-2
The
circuit
shown
2: Behavior
fictional
Representations
two are summaries
Activation
havior
example,
as Lisp functions.
mentioned
above.
of the behavior
Sunzmary
graphs
directly
graphs
which
links
contain
if the simulator
graphs.
component
notices
that
sometime
during an instruction,
is linked to the behavior
graph
The next
The final
instances
MAC-l
the ALU performs
registers.
Figure 2 shows
tions are generated.
3.3
Summary
When
reasoning
VLSI circuits,
an add
the cuinternal
Characteristics
The first observation
gives us a way to overcome
culties resulting from the second. These observations
implications
which together
guide us to the strategy
representing
5
The
out:
first, that they are engineered
artifacts,
designed to accomplish
a
purpose,
and second, that their behavior
can be extremely
complex.
be controlled
each part
stand
the space of designed
behavior
the diffi-
The
ways
(e.g. by observing
We use simulation
because
which
fails
but
quickly
on the
it is much
amount
worse
of time with
Our
designed
present,
method
rest
so that
it is undesirable
quickly
and
either
terested
simulate
other
methods
can
generator
to fail,
be
and unpredictable
using
could
variables
The inputs
erations
the circuit’s
behavior
for solutions
to testing goals. If a solution is
then our program
will find it, and it does so quickly
and
components.
add,
of be-
flows through
the
it could be done
version
create
of the circuit).
behavior
graphs
we need
only
simulate
Rather
than
of numbers
the instruction
for data.
of operations
are the complete
set of circuit
plus perhaps
provided
inputs
by test
as programs,
experts.
because
op-
some interestthat
We repreworks
well
with another approach
to test generation
discussed in [Shirley85].
We divide the circuit state into two components:
control
state and data state4.
Control
state is initialized
the same
way for each simulation
variables.
For example,
is an effective test generation
strategy.
shows the performance
of our program
1).
6
a database
data, thus reducing
the number
simulation
we need. For example,
suppose we’re in-
because
the search space is limited.
To the extent
that the
Designed-Behavior
Hypothesis
is true, i.e. that efficient tests for
components
exist within the designed behavior,
then our method
The example in section
on a simple processor.
a working
from the databook,
sent these simulator
searches
is to create
it can easily
to the simulator
collected
ing sequences
way.
exhaustively
busses
inputs
accessible
”
intermediate
in the ADD instruction
of the MARK-2.
the instruction
once for each combination
the processor
-
for a test
for it to run for a large
no result
are not directlv
through
of the preprocessor
once,
Naturally,
23 instructions;
and data
primary
graphs which describe how information
We do this using simulation,
although
some
applied.
nodes
in
in turn.
purpose
in other
graphs
as important
is a lgbitincluding
a
for managing
global aspects of the circuit’s
captured
in the behavior
graphs.
We discuss
solves
- and just
of this circuit
is a sequencer,
implementing
of the
Details
containing
symbolic
runs (or observations)
quickly
portion
portion
indirectly
and of searching
those graphs for solutions
to testing goals.
We recognize that test generation
is a very difficult problem:
partial ‘solutions are likely to be the norm in this field for many
years to come. Consequently,
rather than trying to build a test
generator
which solves all problems,
we are aiming for one which
of the problems
example
Preprocessor
havior
circuit.
have several
of explicitly
with behavior
version
Our method
has’three
parts:
(1) a preprocessor
for creating
the
for finding patterns
of acbehavior
graphs,
(2) a micro-planner
tivity among the behavior
graphs which solve testing goals, and
representa-
two observations
as a teaching
The internal
must
micro-
presented
processor
(3) a macro-planner
state which aren’t
of the Domain
about
behavior
horizontally
modified
80 lines of microcode
and outputs.
For
then the ALU’s add operation
generated
by simulating
that in-
how the various
5 is a simple
It is a slightly
The central
the righthand
ROM holding
to the beof them.
The Register Transfer Summary captures
effects each behavior
graph has on the circuit’s
struction.
mulative
I
and the lefthand portion is a RAM. The address
and their associated
signal lines are this circuit’s
Component
The
operations
one or more
.
of our method.
Microprocessor
in figure
processor.
[Tanenbaum84].
wide datapath;
so we can implement
them
level is the set of behavior
the details
ACTIVATIONS
programmed
Figure
describes
and is left by most
The simulator
run, while
the program
instructions
data state is initialized
with
counter is initialized
to ?PC
holding
itself is event-driven,
the expression
where
an event
(+ ?PC
is a circuit
The question
of what the designer
should do when the test
generator
fails can be answered
by stepping
back from the test
generation
problem
and considering
its place within the larger
and simplifying
symbolic expressions
and (2) recording
justifications between events. These justification
links form a flow graph
context
which
of circuit
but one part
of testability
design
and manufacturing.
of the design task.
against performance
Test
generation
Thus, if generating
tests for a given circuit requires
effort or is impossible,
then the designer can consider
the
886
circuit
to make
the
/ ENGINEERING
problem
is
It is possible to trade off ease
and reliability
considerations.
easier.
Any
redesign
too much
modifying
can
be
node changing
its value.
is one component
Its essential
features
of the behavior
are (1) propagating
graph.
The expressions
‘This division depends on the level at which the circuit is described, but for
any given level it is fairly clear and generally corresponds to the division
of controller
and datapath.
For the MARK-2, the control state is the
microprogram
counter and the micro-instruction
register, and the data
state is everything
else (e.g. the accumulator
and the program counter).
Specific
Patterns
IrJPUT-1 II/PUT-2
0
65535
65535
0
1
65535
2
65535
4
65535
8
65535
16
65535
<more patterns>
A composite event
with subevents:
AT ?Time
= :PLUS
OP
IIIPUT-1 = ?Value-1
AT ?Time
Ilr'PUT-2
= ?Value-2 AT ?Time
OUTPUT = ?Output AT ?Time
and constraints:
(and (controllable? ?Value-1)
(controllable? ?Value-2)
(observable?
?Output))
Time
-------------
----------_
98
99
100
-> 101
?AC)
102
:PLUS
cuit
the values
inputs
the data
during
registers.
to say where
another,
are functions
simulation
The variables
they
much
of nodes
the
originated,
simplified,
the
initial
in these expressions
consequently,
flow graph
node back to primary
inputs.
of these two flow graphs.
5.2
of the values
run and
The behavior
.----------_
---------???
?Addend
OUTPUT
i+ ?Addend
?AC
of
of ADDD’s
Figure 4: Portion
behavior
graph
(shown
as a table)
are marked
the values
graph
form
at each
is the union
the matcher
and irrelevant
The Micro-Planner
The micro-planner’s
II:PUT-2
on cir-
states
the expressions
linking
IrlPUT-1
??? indicates an unknown value
.
indicates an unchanged value (i.e. ditto)
-> indicates the match
?Addend = Contents(RAId,?ADDR)
Figure 3: An event pattern
describing
OP
to divide the set of behavior
subsets depending
on whether
graphs into relevant
the component
“did
anything
interesting”
during a particular
behavior.
then tries each potentially
relevant behavior graph
task is to take a test for a single component,
The matcher
in turn until it
expressed
in terms of the component’s
I/O pins, and to rewrite
it in terms of the circuit’s
pseudo-inputs
and outputs
(i.e. pri-
The matching
finds a situation
which meets all of the constraints.
process itself is straightforward
unification
augmented
to check
mary
the additional
I/O
pins
plus data
registers).
The core of this planner
is
a matcher
which takes component
tests and finds
them in the behavior
graph database.
If successful,
instances
of
this planner
produces
will do most
a sequence
of the work
needed
of inputs
to cause
The remainder
of the work
Where do the component
two sources:
for standard
devices, we ask the domain
for the circuit
which
the test to occur
within
though we have not yet tried to do this.
Component
tests are expressed
in two parts:
of prototypical
operations
that the component
(e.g. a register
test data
provide
(e.g. have
must
try to load these
language
data
are just
sequences
in figure
3 describes
The pattern
for describing
and (b) actual
numbers).
prototypical
We
operations;
of numbers.
a prototypical
of the MARK-~‘S ALU. The four subevents
between place, value and time that define
tion. The additional
match only instances
we could
method,
(a) descriptions
must be able to
be able to load data)
the register
a rich
the actual
from
combinational
devices and sequential
experts.
For other combinational
de-
vices, we use a conventional
test generator.
In principle,
also use a recursive
application
of our test generation
perform
the circuit.
is done by the macro-planner.
tests come from? We get them
constraints
further
of add operations
pattern
The matcher
constraints.
finds numerous
havior graphs
i.e. when the
the behavior
graph
(ADDD)
, which
amount
stored
in main
?Time
?Value-1
?Value-2
?Output
specialize this pattern
to
that are useful for test-
inputs.
As
The
program
portion
to apply
matching
binding
pattern
by an
of this
In all of
specific test data
an event pattern
environment.
The
in figure 3 against
75
Contents(RAId, ?Addr)
?AC
(+ Contents(RAM, ?Addr) ?AC)
address
set up so that
from section
3.2 allows
which
At this point
determine
what
inverse function
came out of the ALU, by applying
the
that we observe at the circuit output.
can then
(i.e. ?Value-2)
instruction
Moreover,
Summary
relevant
==
==
==
back-solve
to determine
properly.
Similarly,
there must be a output
of the circuit which
is an invertible
function
of the ALU's output. With that we can
Activation
The
==
some
into the two primary
test data moves through
the circuit to the ALU, it is
back to the desired value, thus exercising
the ALU
The Component
memory.
instruction
of the accumulator
add operation
describe the relations
the ALU’s add opera-
then
actually
to data
the value
Recall that our purpose
here is
to the ALU. An additional
result of
against
a behavior
graph is a variable
bindings
resulting
from matching
the
the behavior
graph in figure 4 are:
isters in order to apply
In this case, one addend
data
be-
graph appears
in figure 4, displayed
in tabular
form.
the other potential
matches
the ALU is either incrementing
the
in all of these cases, one
program
counter
or the stack pointer:
of the ALU’s inputs is a constant,
and thus is not controllable.
of two distinct
primary
inputs.
This allows us to apply test data
to the ALU by transforming
the data using the inverse functions,
the inverted
the MARK-~‘S
for the ADD-Direct-from-memory
increments
environment
feeding
where
many situations
in which the ALU performs
an addition.
But
only one of these potential
matches meets the additional
restrictions needed for it to help test the ALU. This match occurs in
ing the ALU. In order to help, the expressions
on the two input
nodes must be controllable,
i.e. they must, be invertible
functions
the inverted
transformed
places
match the subevents
of the pattern
in figure 3,
processor
is “put through
its paces,”
there are
must
the equations
in this binding
how it must set up the circuit’s
regparticular
values to the ALU’s inputs.
(i.e. ?Value-1) must be in the RAM at
can be specified
later,
and the other
addend
be in the accumulator.
the micro-planner
has now identified
the ADDD
as the one to use to test the ALU’s addition operation.
it has also determined
how the data registers must be
the ADDD
instruction
the job of the macro-planner
operations which will initialize
can do this job.
It is now
to work out a sequence of circuit
the data registers
properly.
AUTOMATED
REASONING
/ 887
5.3
The Macro-Planner
The
purpose
of the
operations
macro-planner
(in this case
the circuit’s
micro-planner
descriptions
is to generate
sequences
of
set up
leave important
data in one or more registers
the ALU’s output in the accumulator),
a sec-
ond job of the macro-planner
to one of the circuit’s
is to work out how to move
outputs
The macro-planner
where
searches
based
on the sequence
contained
through
length
in the registers.
results
it can be observed.
the space
sequences
to find one which moves data
planner is implemented
as straightforward
a metric
which
state so that the single test operation
found by the
will start with the right data. Since a test opera-
tion will sometimes
(e.g. ADDD leaves
data
sequences
of instructions)
and
pattern
designed
simulation
method)
(a well-behaved
operations
for testing
macro-planner,
set up circuit
The
with
of useful
descriptions
(i.e.
6
inputs
,I
II
5
---->
on an Example
Figure
6 shows the fault
achieves with the MARK-2
The
chose the ADDD
information
for the
program
coverage
that
processor6.
was able to generate
while
to
(PRELOAD-IIEIYIORY
?Ac ?ADDR)
(LODD ?ADDR)
(PRELOAD-MEMORY ?vAm
?ADDR)
(ADDD ?ADDR)
(STOD ?ADDR)
(OBSERVE-14~140~~(+ ?VALUE ?AC) ?ADDR)
Performance
components,
of all of this is the sequence
which subsequently
included three operations
state and two operations
to observe it.
1:
2:
3:
Test ------------> 4:
Observe outputs
-> 5:
1,
6:
Setup
it failed
our
7
Explaining
Circuit
prototype
system
method
operations
Note
We believe
Consider
starts
testing
the datapath
to realize
despite
trolled almost completely
of the program’s
success
that
the fact
the program
that
should
of the
some
ality.
of the mapping
is easily
between
able to
component
following
account
a specification
of processor
for an abstract
registers
abstract
design:
machine
and’the
machine
a designer
which
includes
instruction
set. His
in hardware
while
is con-
by 80 lines of intricate
microcode.
Part
can be attributed
to its use of abstract
describing
capabilities
of the tester
two special operations
oper .ations
and observe-memory
hardware. These are the preload-memory
which write data into and read data from the RAM.
has 1G components
and 91 nodes.
Each simulation
run is
approximately
50 clock cycles long, taking about 10 seconds of real time on
a Symbolics 3640. Generation
of prototype tests, i.e. tests with symbolic
data, for this circuit takes 1 minute, including both the time taken for
successfully creating tests for some components and failing to do so for
instruction,
from
register
these
describing
how data
to register.
dataflow
graphs
Naturally
directly
To do this, he adds
identity
transformations.
in one graph
in hardware:
as
can’t
without
boxes
gle multiplexor.
collects
to the graphs components
which perform
For example,
he might insert a reg-
to shift
allowing
some
components
via time-multiplexing.
of its operations
to be shared
In another
into two graphs
When
all the control
signals
from
By this account,
tal refinement
controller
chine).
and
them
large
portions
the fact
of individual
plus
that
is designed
different
change
means
existing
components
that many datapath
processor
the merging
and creates
a
these signals at the right
any one of the well-known
from
is designed
(stylized
implementation
The availability
of components
which
chips),
a sin-
the designer
all the graphs
is very
graph
introduce
using
fit together,
the way the datapath
merging)
in time,
another
he might
and implement
the flow graphs
later
with
situation,
finite state machine
(FSM) to provide
times. This FSM is implemented
using
methods
(e.g. with a PLA).
ment
is transformed
the designer
sharing
there would be a wasteful
duplication
of functionSo he looks for ways of merging
the graphs together.
identity
in
‘We include
GThis circuit
the circuit
varies over different
parts of the circuit,
is a direct result of the design process.
the programmer
accessible
job is to implement
this
implement
that
succeeds
the datapath
the directness
the
with
thereby
It is also important
between
of the components
that implement
between
the more “direct” the relationship
operations
and circuit operations,
the bet-
and circuit operations
and that this variation
ister
be modified.
Thus our program’s
failure points out areas
circuit that a redesign program
should focus on.
on the relationship
of the processor,
and consequently,
our method
solve testing goals involving
that ALU operation.
the expert
but really
depends
ter our method
performs.
For example,
the ADD operation
of
the ALU very directly implements
part of the ADDD instruction
since data paths are typically
much wider than control paths.
’
Our program
also quickly fails on the controller,
an area which
with effort,
the Performance
and the operations
one for each
for the highlighted
to do so for the others.
testable
to solve problems
side.
meeting myriad performance,
reliability,
and cost constraints.
The specification
can be viewed as a set of dataflow
graphs,
the program
can quickly work out how to test virtually
all of the
datapath,
which includes more than half of the physical circuit,
says is partially
on the other
method
is that it
components
using
Why is there a radical
difference
in coverage
between
the datapath
and the controller ? We conjecture
that the performance
it moves
tests
through
of instruction
and the amount
below. The micro-planner
the ALU and provided
reasoning
the components
them. In particular,
individual
component
of circuit
operation
use of simulation
backwards
of our
complex
from the behavior
graphs.
the
reasoning
would try to do. A clear advantage
can work forward
through
extremely
of our
of the end product
Also,
is critical:
sequencer
is very difficult, yet this is exactly what
planners
and classical
test generation
algorithms
descriptions
of the effects of instructions)
are contained
in the
Register Transfer Summaries,
which are generated
by our system
An example
behavior.
matching
the microcode
goal-directed
involving
around
properly.
best-first
search
The operator
of the circuit’s
behavior
(e.g. while controlling
the
The program
generates
these abstract
descrip-
register
state).
tions from the
(incrementhe way the
of a state madirectly
imple-
operations
process
doesn’t
(it just adds new identity
component
operations
tend
(e.g. ALU
normally
boxes),
to “very
directly
implement”
circuit operations.
The state machine
design process,
however,
need yield no such simple relationships.
The behavior
of the whole
different
from the behavior
register,
888 / ENGINEERING
ROM,
or MUX).
controller
(a state machine)
of any controller
component
is very
(e.g. a
CA
I
MFIRK-2
Figure
5: The MARK-2
Processor
HA ,I I,
1 1 I
MARK-2
Figure
6: Coverage
of processor
components
AUTOMATED
REASONING
/ 889
This account
also suggests
a difference
between
References
our approach
to test generation
and the classical approaches.
The usual result
of the design process, a circuit topology, is the superposition
of
the dataflow graphs, plus the identity boxes added to allow sharing, plus the controlling
state machine.
include the list of operations
the circuit
This topology doesn’t
can perform,
nor does
it make explicit
the circuit,
how data
moves through
e.g. timing
[DeKleer79]
[Roth661
attempt
nents
to generate
and topology.
tests
Testing
directly
experts
from the circuit
perform
compo-
well, because
8
are an approximation
of the flow graphs.
[Shirley851
they
8
Conclusion
We have demonstrated
how knowledge of a system’s designed behavior can help solve test generation
problems, or more generally,
problems involving manipulation
The key to our method is that
be far more limited
can translate
planning
than
of complex engineered systems.
a device’s designed behavior can
its potential
into a reduction
behavior.
of the search
This limitation
necessary
to achieve
goals.
A prototype
system has been implemented
and run on a small
number of examples.
This system has 3 components:
(1) a preprocessor to create the behavior graphs using symbolic simulation of input sequences out of the databook;
(2) a micro-planner
to match
simulation
tivation
aspects
planner’s
testing goals gotten
runs (the matcher
from domain experts against
is guided by the Component
the
Ac-
to control,global
Summaries);
and (3) a macro-planner
of the circuit’s state (the behavior graphs also supply the
operator
descriptions).
The system has generated
tests for a simple
apath.
It was successful despite the fact that
processor’s
datthe datapath
is
controlled
by 80 lines of intricate microcode.
A clear advantage
of our method is that it can, by using simulation,
reason forward
through complex components,
such as the controller,
in order
solve problems involving the components
on the other side.
9
to
Acknowledgments
The following
to the contents
Yehudah
people
have read
of this paper:
Freundlich,
drafts
Randall
Jeff Van Baalen,
or otherwise
Davis,
Walter
contributed
Gordon
Kramer,
Howard Shrobe,
Ralil ValdCs-Perez,
Brian
Peng Wu. I would especially like to thank Gordon
who is the testing
expert
whose
methods
Robinson,
Hamscher,
inspired
Glenn
Williams,
Robinson,
much
of this
work.
‘The domain experts report an interesting dichotomy. They say that, for
humans, test generation problems are either very easy or very hard. The
problems are easy when they can see clearly which manipulations
of the
device’s interfaces are useful for exercising components inside the circuit,
and hard otherwise. In our work, we have have made some effort, albeit
informally, to match the performance of our test generation system to the
expert’s intuitions about which problems are easy for humans and which
are hard. And the expert does report that the results on the MARK-2
match his expectations fairly closely.
890
/ ENGINEERING
of Technology,
Roth,
J. P., “Diagnosis
1979, AI-TR-529.
of Automata
Failures:
A
278-291.
Shirley,
M.,
“An Automatic
proach to Testing,”
IEEE
lation and Test Generation
know the same things designers know and can reconstruct
and
use the designer’s flow graphs. Finally, our program is successful
for the same reason, it reconstructs
and uses behavior
graphs,
which
stitute
Calculus and a Method,”
IBM Journal
of Research and Development, Vol. 10, July 1966, pp.
and selection relationships,
both of which were explicit in the
flow graphs before they were merged.
Classical test generation
algorithms
perform poorly, because
they
de Kleer, J., Causal and Teleological Reasoning
in Circuit Recognition, Diss., Massachusetts
In-
Programming
Ap-
Workshop on SimuSept
Environments,
1985.
[Singh85]
Singh, N., Exploiting Design Morphology to Manage Complezity, Diss., Stanford
Department
of
Electrical Engineering,
April 1985.
[Tanenbaum841
Tanenbaum,
A. S., Structured
nization, Prentice-Hall,
NJ, 1984.
Inc.,
Computer
Englewood
OrgaCliffs,