On Bridging Simulation and
Formal Verification
Eugene Goldberg
Cadence Research Labs (USA)
VMCAI-2008, San Francisco, USA
Summary
• Introduction
– motivation
– main idea
• Stable sets of points (SSPs) and proofs
– checking if a set of points contains an SSP
– simulation based proof system
– extraction of sufficient test sets from proofs
• Generation of practical tests
– test generation for design change/manufacturing
faults
– theoretical and experimental justification of our
approach
•
Conclusion
Motivation
Given a design, formal verification is to check if property 
holds at all points of the search space.
Simulation is to “informally” check if property  holds by
testing it for a (relatively small) set of points.
Simulation is still the main workhorse of verification. It
works surprisingly well taking into account that only
negligible part of the search space is sampled!!
In this paper, we study this phenomenon by the example
of the satisfiability problem. As an application, we
consider testing combinational circuits.
Simulation in Terms of SAT
Let F be a CNF formula (i.e. conjunction of
disjunctions also called clauses).
Satisfiability problem (SAT): Given F, find an
assignment p such that F(p)=1 or prove that
F(all_space)=0.
Simulation: Given a CNF formula F, find a set
of points T s.t. if F(T) = 0, then F is (most
likely) unsatisfiable.
In other words, find T such that
F(T)=0 implies F(all_space) = 0
Interpretation of Points as Tests
Let N be a circuit. Let X,Y describe the input and internal
variables of N. Let z describe the output variable of N.
Let CNF formula FN(X,Y,z) specify N.
Checking satisfiability of N reduces to
checking the satisfiability of FN  z.
z
N
Y
….
X
A point p is a complete assignment to the
variables X  Y  {z}. The corresponding
test t is the projection of p to X.
So p consists of a test t and a computation
vector (y,z). Since p falsifies clauses of FN
it is computation with a fault.
Main Idea
We formulate simulation as a proof system.
In contrast to “regular”
simulation, we use points
not only for sampling
search space but for
derivation.
Instead of proving one “big”
theorem (F implies an empty
clause), we deduce a
sequence of simpler lemmas
(F implies a clause L).
In this derivation we
employ the machinery of
Stable Sets of Points.
Each derived
added to F.
clause
is
Summary
• Introduction
– motivation
– main idea
• Stable sets of points (SSPs) and proofs
– checking if a set of points contains an SSP
– simulation based proof system
– extraction of sufficient test sets from proofs
• Generation of practical tests
– test generation for design change/manufacturing
faults
– theoretical and experimental justification of our
approach
•
Conclusion
A Stable Set of Points (SSP)
A complete assignment  a point.
Example of 1-neighborhood of point p w.r.t a
clause C (written Nbhd(p,C)) such that C(p)=0.
C = x  z, p=(x=0,y=0,z=0). Nbhd(p,C) = {p1,p2},
where p1=(x=1,y=0,z=0) and p2=(x=0,y=0,z=1).
Let F be a CNF formula and T be a set of points.
T is called an SSP if
 p  T,  clause C of F such that Nbhd(p,C)  T.
A CNF formula F is unsatisfiable iff there is a set of
points that is an SSP.
Testing if a Set of Points Contains SSP
Let F be a CNF formula and T be a set of points.
To check if T is an SSP : For every point p of T, check
if there is a clause C of F s.t. Nbhd(p,C)  T.
This procedure is linear both in T and F.
Even if T is not an SSP, a subset T* of T may be an SSP.
To check if T contains an SSP : For every point p of T,
check if there is a clause C of F s.t. Nbhd(p,C)  T.
If C does not exist, remove p from T and restart this
procedure.
This procedure is quadratic in T and linear in F.
Simulation as a Proof System
We formulate simulation as a proof system with two
derivation rules.
Rule 1: Generate a point pi and derive the value of F(pi).
Rule 2: Let T={p1,…,pk} be the current set of points.
Derive a clause C such that F implies C and add it to F.
Derivation of C is valid only if T contains an SSP of the
CNF formula F ~C (and so F  C holds).
A Sufficient Test Set
Given a CNF formula F, a set of points T is called
sufficient if there is a set of lemma clauses L1,…,Lk
(Lk=) such that all k derivations
F  L1 ,
F  L1  L2 ,
….
F  L1 …  Lk-1  Lk
are valid (i.e. they can be proved by “simulation” using T)
In other words, T is a sufficient test set if the set of valid
derivations is sufficient to prove that F is unsatisfiable by
applying only the second rule (no new points are
needed).
Extracting a Sufficient Test Set
from a Resolution Proof
Let L={L1,..,Lk} be the set of resolvents of a resolution proof R
that F is unsatisfiable. We assume that resolvents Li are
numbered in their derivation order (so Lk = ).
Then there is always a sufficient test set whose size is  2k
(proving that FL1 …  Li-1  Li needs only two points of T).
Sufficient test sets can be very small even for very large
formulas!!!!
“Proving” Resolvent by Building
a Stable Set of Two Points
Let C be the resolvent of C and C. One can prove that
C C  C by building an SSP of only two points.
Example: Let C = ~x  y  z, C = x  w. The resolvent C is
y  z  w. To prove C  C  C it suffices to show the CNF
formula C  C  ~C is unsatisfiable.
(Here ~C= ~y  ~z  ~w).
After making assignments y=0,z=0,w=0, the formula C  C  ~C
turns into ~x  x. Any two points p and p with y=0,z=0,w=0, that
are different only in the value of variable x is an SSP for
C 
C  ~C .
Summary
• Introduction
– motivation
– main idea
• Stable sets of points (SSPs) and proofs
– checking if a set of points contains an SSP
– simulation based proof system
– extraction of sufficient test sets from proofs
• Generation of practical tests
– test generation for design change/manufacturing
faults
– theoretical and experimental justification of our
approach
•
Conclusion
Applying Our Theory to
(design change/manufacturing testing)
We cannot apply our theory directly to checking if F is
unsatisfiable. (To prove that a set of points is sufficient we need
a set of lemma clauses i.e. another proof)
However, there are numerous indirect ways to apply our theory.
One application is
to detect incorrect design changes/
manufacturing faults.
Let F be a CNF formula. Let R be a proof that F is
unsatisfiable.
Find a set of points such that most likely detect satisfiable
variations of F. (These variations describe tech. faults/design
changes.) Such a test set can be extracted from a sufficient
set of points specified by R.
Tight Sufficient Tests
Let T be a sufficient test set for F. Informally, T is tight if
every point p  T falsifies as few clauses as possible.
Given a resolution proof L1,..,Lk, a sufficient test set T is
built as T1 … Tk. Here Ti is a two point SSP that proves
that C  C  Li where C  C are the parents of
resolvent Li.
When building Ti one can arbitrarily assign variables that
are not in C or C.
To build a tight sufficient test, free variables are assigned so
as to minimize the number of falsified clauses.
Big Picture
Let F be an unsatisfiable formula. To detect satisfiable
variations (“faults” in F) we generate a tight sufficient test
set T. Such a set can be extracted, for example, from a
resolution proof that F is unsatisfiable.
The sufficiency of T guarantees its “completeness”.
The tightness of T increases the probability of detecting small satisfiable variations.
Suppose, for example, that a point p of T falsifies
only one clause C of F. Then if F* is a variation of F
consisting of disappearance of C (along with some
other changes) it is likely that F*(p) = 1. Hence p
detects that F* is satisfiable.
“Theoretical” Justification of Our
Approach
Let N1 and N2 be identical
copies of circuit N. Let Rnat be
a “natural” resolution proof that
their miter is unsatisfiable.
So, in a sense, our theory
“predicts” the high quality
of the stuck-at fault model
for circuit testing.
Let T={p1,..,pm} be a tight sufficient
test set specified by Rnat. We show
that
the
set
inp(T)=
{inp(p1),…,inp(pm)}
detects
all
(testable) stuck-at faults of N.
Experimental Justification
We experimentally compared quality of random test sets
and test sets extracted from resolution proofs that two
copies of a circuit were identical.
Resolution proofs were generated by a state-of-the-art SATsolver. We used MCNC benchmark circuits.
We applied these two kinds of test sets (of the same size) to
detecting literal appearance faults. Such faults are more
“subtle” than stuck-at faults and so are harder to detect.
Experiments show that tests extracted from resolution proofs
significantly outperform random tests.
Conclusion
We use SAT to show that there is a close relation between
high quality test sets and formal proofs.
We show how high quality tests can be extracted from
proofs of unsatisfiability represented as a sequence of
lemma clauses.
Our approach answers (at least in principle) two important
questions.
a) When does one stop simulation? (When the set of
generated points “encrypts” a formal proof).
b) What is an “ideal” metric in functional verification? (A
formal proof.)