Formal Verification
Using
Infinite-State Models
Randal E. Bryant
Carnegie Mellon University
http://www.cs.cmu.edu/~bryant
Contributions by graduate students:
Miroslav Velev, Sanjit Seshia, Shuvendu Lahiri
Outline
Task


Formally verify hardware and software systems
Build on success in verifying finite models
Infinite-State Models


How do they arise
Need logic that is suitably expressive, yet remains
reasonably tractable.
Verification Techniques


Range of methods with varying capabilities and limitations
Solve problems by mapping into propositional logic
 Proof engines can use powerful Boolean methods
–2–
Example: HP/Compaq Alpha 21264
Pipeline State





Multiple caches
Instruction queues
Dynamicallyallocated registers
Memory queue
Many buffers
between stages
Verification Tasks


–3–
Does it implement
the Alpha ISA?
Do specific units
satisfy desired
properties?
Microprocessor Report, Oct. 28, 1996
Temporal Logic Model Checking
Verify Reactive Systems

Construct state machine representation of reactive system
 Nondeterminism expresses range of possible behaviors
 “Product” of component state machines


Express desired behavior as formula in temporal logic
Determine whether or not property holds
Traffic Light
Controller
Design
True
“It is never possible
to have a green
light for both N-S
and E-W.”
–4–
Model
Checker
False
+ Counterexample
Finite System Modeling Example
Global Bus
Distributed, Shared
Memory System
Interface
Cluster #2
Abstraction
Cluster #3
Abstraction
Interface
Simplifying
Abstractions
Cluster #1 Bus
Mem.
Cache
Control.
Cache
Control.



Proc.
–5–
Proc.
Arbitrary reads & writes

Single word cache
Single bit/word
Abstract other
clusters
Imprecise timing
Symbolic FSM Analysis Example

K. McMillan, E. Clarke (CMU) J. Schwalbe (Encore Computer)
Encore Gigamax Cache System



Distributed memory multiprocessor
Cache system to improve access time
Complex hardware and synchronization protocol.
Verification


Create “simplified” finite state model of system (109 states!)
Verify properties about set of reachable states
Bug Detected



–6–
Sequence of 13 bus events leading to deadlock
With random simulations, would require 2 years to generate
failing case.
In real system, would yield MTBF < 1 day.
Boolean Manipulation with OBDDs



Ordered Binary Decision Diagrams
Data structure for representing Boolean functions
Key to success in hardware verification
Example:
x1
(x1  x2)  x3
Nodes represent variable tests
x2
x3
0
–7–
1
Branches represent variable values
 Dashed for value 0
 Solid for value 1
Canonical representation
 when reduction rules applied
 Makes equivalence trivial
Representing Circuit Functions
Functions


S3
Cout
All outputs of 4-bit adder
Functions of data inputs
a3
a3
b3 b 3
b3 b 3
a2
a2
a2
b2 b 2
b2 b 2
b2 b 2
a1
a1
a1
b1 b 1
b1 b 1
b1 b 1
a0
a0
a0
S2
A
B
A
D
D
Cout
S
S1
S0
Shared Representation
Graph with multiple roots
 31 nodes for 4-bit adder
 571 nodes for 64-bit adder
 Linear growth

–8–
b0
0
b0
1
Simplified Processor Example
IF/ID
PC
Op
ID/EX
Control
EX/WB
Control
Rd
Ra
Instr
Mem
=
Adat
Reg.
File
A
L
U
Imm
Bdat
+4
Rb




–9–
=
Simplified RISC pipeline
Register-Register and Register-Immediate operations
Data hazards handled by register forwarding
Each step of operation defined by function dpipe
ISA Reference Model
Op
PC
Control
Rd
Ra
Instr
Mem
Adat
Reg.
File
Imm
Bdat
A
L
U
+4
Rb




– 10 –
Only programmer-visible state
Much simpler control logic
Assume verified against instruction set definition
Each step of operation defined by function dspec
Abstracting Data from Bits to Integers
x0
x1
x2

x
xn-1
View Data as Symbolic “Terms”

Arbitrary integers
 Verification proves correctness of design for all possible word sizes


Can store in memories & registers
Can select with multiplexors
 ITE: If-Then-Else operation
p
x
y
– 11 –
1
0
ITE(p, x, y)
T
x
y
1
0
x
F
x
y
1
0
y
Abstraction Via Uninterpreted
Functions
A
Lf
U
For any Block that Transforms or Evaluates Data:


Replace with generic, unspecified function
Only assumed property is functional consistency:
a = x  b = y  f (a, b) = f (x, y)
– 12 –
Abstraction Via Uninterpreted
Functions
IF/ID
PC
Op
ID/EX
Control
EX/WB
Control
Rd
Ra
Instr
F3
Mem
=
Adat
Reg.
File
A
FL2
U
Imm
F1
+4
Rb
=
For any Block that Transforms or Evaluates Data:


– 13 –
Replace with generic, unspecified function
Also view instruction memory as function
Abstracting Reference Model
Op
PC
Control
Rd
Ra
Instr
F3
Mem
Adat
Reg.
File
Imm
Bdat
A
FL2
U
+4
F1
Rb

– 14 –
Abstract with identical functions as in pipeline model
EUF: Equality with Uninterp. Functs

Decidable fragment of first order logic
Formulas (F )
F, F1  F2, F1  F2
T1 = T2
P (T1, …, Tk)
Terms (T )
ITE(F, T1, T2)
Fun (T1, …, Tk)
Functions (Fun)
f
Read, Write
Predicates (P)
p
– 15 –
Boolean Expressions
Boolean connectives
Equation
Predicate application
Integer Expressions
If-then-else
Function application
Integer  Integer
Uninterpreted function symbol
Memory operations
Integer  Boolean
Uninterpreted predicate symbol
Correctness of Pipeline
Qspec
dkspec
Abs
Qpipe
Qspec
Abs
dpipe
Qpipe
Abstraction Function Abs


Relates state of pipeline to program state
Result of completing partially-executed instructions
Requirement

Pipeline step dpipe matches k instruction executions dkspec
 For our pipeline k = 1
 When pipeline stalls have k =0
 Superscalar pipelines can have k > 1
– 16 –
Correspondence Checking

Burch & Dill, Computer-Aided Verification ‘94
Exploit State Structure



State held in memories and pipeline latches
Memories match those of instruction set model
Latches hold additional pipeline state
Pipeline State can be “flushed”



– 17 –
Control logic to support external interrupts
Complete in-flight instructions
Without fetching any new ones
Computing Abstraction Function
Method



Start with arbitrary pipeline state Qpipe
Symbolically simulate processor with stall asserted
Project out all but programmer-visible state
Stall = 1
Stall = 1
Stall = 1
Arbitrary
Qpipe
Effect

– 18 –
dpipe
dpipe
dpipe
Proj
Pipeline
Flushed
Processor computes its own abstraction function!
Qspec
Computational Task: Single-Issue
Processor

Stall = 1
Stall = 1
Stall = 1
k=0
dpipe
Qpipe
Stall= 0
dpipe

dpipe
dpipe
Proj
Stall= 1
Stall= 1
Stall= 1
dpipe
dpipe
dpipe
k=1
dspec
=?
=?
Proj
Compare results of two symbolic simulations
 Starting from same initial state
 Number of simulation steps ~ pipeline depth

Check that resulting user-visible states identical
 Disjunctive acceptance condition
 Extra clock cycle causes either 0 or 1 new instructions to complete
– 19 –
Computational Task: Dual-Issue
Processor

k=0
Flush
Qpipe
Proj
Stall= 0
dpipe
Flush
k=1
dspec
=?
k=2
dspec
=?
=?
Proj
 Extra clock cycle causes 0, 1, or 2 new instructions to complete
– 20 –
Term-Level Symbolic Simulation
xa
f
f
T=3
0
1
2
f
xb
Ra
A
L
U
Rb
Simulator Operation

Register states are term-level expressions
 Denoted by pointers to nodes in Directed Acyclic Graph (DAG)

Simulate each cycle of circuit by adding new nodes to DAG
 Based on circuit operations

– 21 –
Construct DAG denoting correctness condition
Decision Problem
Logic of Equality with Uninterpreted Functions (EUF)

Truth Values





Integer Values




Task

Dashed Lines
Model Control
Logical connectives
Equations
Solid lines
Model Data
Uninterpreted functions
If-Then-Else operation
e1
f
T
F
e0
x0
f
T
d0


=

T
F
=
F
Determine whether formula is universally valid
 True for all interpretations of variables and function symbols
– 22 –
Finite Model Property for EUF
e1
f
T
F
e0
x0
f
T
d0


x0
=
f (x0) f (d0)

T
F
d0
=
F
Observation


– 23 –
Any formula has limited number of distinct expressions
Only property that matters is whether or not different terms
are equal
Boolean Encoding of Integer Values
Expression
x0
Possible
Values
{0}
Bit
Encoding
0
0
d0
{0,1}
0
b10
f (x0)
{0,1,2}
b21
b20
f (d0)
{0,1,2,3}
b31
b30
For Each Expression

Either equal to or distinct from each preceding expression
Boolean Encoding


Use Boolean values to encode integers over small range
EUF formula can be translated into propositional logic
 Tautology iff original formula valid
– 24 –
Benchmark Circuits
Single Issue Pipeline: 1xDLX


Analogous to DLX model in Hennessy & Patterson
Verified in ‘94 by Burch & Dill
Dual Issue Pipeline : 2xDLX-CC

Superscalar operation with two complete pipelines
Full-Featured Pipeline: 2xDLX-*

– 25 –
Multi-cycle function units, exception handling & branch
prediction
Evaluation
Using BDD Evaluation to Prove Tautology
Circuit
BDD Vars.
1xDLX
63
2xDLX-CC
173
2xDLX-*
418
BDD Nodes
2,127
51,826
986,740
CPU Secs.
0.2
20
2,635
Using SAT Checkers to Prove Tautology
Chaff (Malik, Princeton)
 Major advances in last few years
Circuit
CNF Vars.
Clauses
2xDLX-*
4,583
41,704

– 26 –
CPU Secs.
22
An Out-of-order Processor (OOO)
incr
Program
memory
PC
result bus
valid tag val
D
E
C
O
D
E
dispatch
Register
Rename Unit
retire
ALU
execute
head
tail
Reorder
Buffer
valid
value
src1valid
src1val
src1tag
src2valid
src2val
src2tag
dest
op
result
1st
Operand
2nd
Operand
Reorder Buffer
Fields
Data Dependencies Resolved by Register Renaming

Mapping from register ID to instruction in reorder buffer that will
generate register value
Inorder Retirement Managed by Retirement Buffer

– 27 –
FIFO buffer keeping pending instructions in program order
Access Modes for Reorder Buffer
Retire
Dispatch
result bus
ALU
execute
FIFO


head
tail
Content Addressable
Insert when dispatch
Remove when retire

Directly Addressable


– 28 –
Select particular entry for
execution
Retrieve result value from
executed instruction
Broadcast result to all
entries with matching
source tag
Global

Flush all queue entries when
instruction at head causes
exception
Required Logic
Increased Expressive Power

Model queue pointers
 Increment & decrement operations
 Relative ordering

Ability to construct complex memory structures
 Not just set of fixed memory types
Don’t Go Too Far


– 29 –
Want practical decision procedures
Efficient reduction to propositional logic
EUF  CLU
Terms (T )
ITE(F, T1, T2)
If-then-else
Fun (T1, …, Tk)
Function application
succ (T) Increment
pred (T)
Decrement
Formulas (F )
– 30 –
F, F1  F2, F1  F2
T1 = T2
P(T1, …, Tk)
Boolean connectives
Equation
Predicate application
T1 < T2
Inequality
EUF  CLU (Cont.)
Functions (Fun)
f
Read, Write
Uninterpreted function symbol
Memory operations
 x1, …, xk . T
Function lambda expression
Predicates (P)
p
 x1, …, xk . F
Uninterpreted predicate symbol
Predicate lambda expression
• Arguments can only be terms
• Lambdas are just mutable arrays
– 31 –
Modeling Memories with ’s
Memory M Modeled as Function
Writing Transforms Memory

M = Write(M, wa, wd)
M
a
M
wa
=

M(a): Value at location a
a
Initially
M
M
a

– 32 –
1
0
m0


wd
Arbitrary state
Modeled by uninterpreted
function m0

 a . ITE(a = wa, wd, M(a))
Future reads of address wa
will get wd
Modeling Unbounded FIFO Buffer
Queue is Subrange of Infinite Sequence
Q.head = h
 Index of oldest element
Q.tail = t
 Index of insertion location

q(h–1)
head
q(h+1)
Q.val = q
•
•
•
 Function mapping indices to values
 q(i) valid only when h  i < t
q(t–2)
Initial State: Arbitrary Queue

Q.head = h0, Q.tail = t0
 Impose constraint that h0  t0

Q.val = q0
 Uninterpreted function
– 33 –
q(h)
q(t–1)
tail
increasing indices

Already
Popped
q(h–2)
q(t)
q(t+1)
•
•
•

•
•
•
Not Yet
Inserted
Modeling FIFO Buffer (cont.)
next[t] :=
ITE(operation = PUSH, succ(t), t)
next[q] :=
 (i).
ITE((operation = PUSH & i=t),
x, q(i))
– 34 –
t
•
•
•
q(h–2)
q(h–2)
q(h–1)
q(h–1)
q(h)
next[h]
q(h)
q(h+1)
q(h+1)
•
•
•
•
•
•
q(t–2)
q(t–2)
q(t–1)
q(t–1)
q(t)
x
q(t+1)
•
•
•
h
•
•
•
next[t]
q(t+1)
•
•
•
next[h] :=
ITE(operation = POP, succ(h), h)
op = PUSH
Input = x
Decision Procedure
CLU
Formula
Lambda
Expansion
Operation



– 35 –
Series of
transformations
leading to
propositional formula
Propositional formula
checked with BDD or
SAT tools
Bryant, Lahiri, Seshia
[CAV02]
-free
Formula
Function
&
Predicate
Elimination
Function-free
Formula
Convert to
Boolean
Formula
Boolean
Formula
Boolean
Satisfiability
Finite Model Property for CLU
x  y  succ(x) > pred(y)
x x+1
x x+1
y –1 y
x = 0, y = 3
y –1 y
x x+1
y –1 y
x x+1
y –1 y
x x+1
y –1 y
x = 2, y = 1
Observation



– 36 –
Need to encode all possible relative orderings of
expressions
Each symbolic value has maximum range of increments &
decrements
Can use Boolean encodings of small integer ranges
Verification Techniques in UCLID
Bounded Property Checking



Start in reset state
Symbolically simulate for fixed number of steps
Verify a safety property for all states reachable within the
fixed number of steps from the start state
Correspondence Checking



Run 2 different simulations starting in most general state
Prove that final states equivalent
e.g. Burch-Dill Technique
Invariant Checking



– 37 –
Start in general state s
Prove Inv(s)  Inv(next[s])
Limited support for automatic quantifier instantiation
Verification of OOO : Automation vs.
Guarantee
Method
Bounded Property
Checking
Burch-Dill
Technique
Inductive Invariant
Checking
Resources Verification Auxiliary
(# of steps) variables
Invariants
Unbounded
Bounded
None
None
Fixed
Unbounded
None
Very few
Unbounded
Unbounded
Significant
Significant,
including those for
auxiliary variables
Presence of decision procedure


Efficiency : Allows improved bounded property checking
and Burch-Dill method
Automation : Reduces manual guidance in proving
invariants
 Automatic Instantiation of quantifiers
– 38 –
Technique 1 : Bounded Property
Checking
Debugging OOO using Bounded Property Checking


All the errors were discovered during this phase
Counterexample trace of great help
Debugging Motorola ELF™



– 39 –
Superscalar out-of-order processor
Reorder Buffer, memory unit, load-store queues etc.
Applied during early design exploration phase
Bounded Property Checking Results
Model
OOO unit
Elf™
steps terms
Term
formula
size
Prop
Formula
Size
UCLID
time (s)
SVC time
(s)
10
59
2566
15290
10.8
233.18
14
87
7480
62504
76.55
> 5 hrs
20
129
19921
263413
1679.12
> 1 day
6
33
218
942
1.2
10.9
8
70
1085
4481
8.4
1851.6
10
104
2467
16453
30.6
> 1 day
12
149
4553
54288
111.0
> 1 day
SVC (Stanford) : Another decision procedure to solve CLU formulas

Can decide more expressive class
CVC (Successor of SVC) runs out of memory on larger cases
– 40 –
Burch-Dill Technique for OOO
Exponential blowup with the number of ROB entries


Limited to r = 8 entries currently
r = 8 finished after case-splitting in 2.5hrs
# Of ROB
# of
terms
Term
formula
size
Prop Formula
Size
UCLID
time (s)
2
63
398
5325
6.83
3
83
618
10248
30.23
4
103
886
18175
157.41
6
143
1534
41208
3051.79
8
183
2342
82915
>31hrs
Entries
– 41 –
Technique 3 : Invariant Checking
Deriving the inductive invariants


Require additional (auxiliary) variables to express invariants
Auxiliary variables do not affect system operation
Proving that the invariants are inductive


– 42 –
Automate proof of invariants in UCLID
Eliminates need for large (often fragile) proof script
Restricted Invariants and Proofs
Restricted classes of invariants

x1x2…xk (x1…xk)

(x1…xk) is a CLU formula without quantifiers
x1…xk are integer variables free in (x1…xk)

Proving these invariants requires quantifiers
x1x2…xk (x1…xk)  y1y2…ym (y1…ym)
x1 x2…xk y1y2…ym [(x1…xk)  (y1…ym)]
Automatic instantiation of x1…xk with concrete terms

Sound but incomplete method
Reduce the quantified formula to a CLU formula

– 43 –
Can use the decision procedure for CLU
Proving Invariants
Proved automatically




– 44 –
Quantifier instantiation was sufficient in these cases
Relieves the user of writing proof scripts to discharge the
proofs
Time spent = 54s on 1.4GHz m/c
Total effort = 2 person days
Extending the Design

base
 Executes ALU instructions only

exc
 Handles arithmetic exceptions
 Must flush reorder buffer

exc/br
 Handles branches
 Predicts branch & speculatively executes along path

exc/br/mem-simp
 Adds load & store instructions
 Store commits as instruction retires

exc/br/mem
 Stores held in buffer
 Can commit later
 Loads must scan buffer for matching addresses
– 45 –
Comparative Verification Effort
base
Total
Invariants
Manually
instantiate
UCLID
time
Person
time
– 46 –
exc
exc / br
exc / br /
exc / br /
mem-simp
mem
39
67
71
13
34
0
0
0
4
8
54 s
236 s
403 s
1594 s
2200 s
2 days
5 days
2 days
15 days
10 days
Beyond Processor Verification
Systems of Identical Processes


E.g., synchronization protocols
Arbitrary number of processes, each having same operation
Software

– 47 –
Create finite model by predicate abstraction
Systems of Identical Processes
Each Process has k State Variables

•
•
•
•
•
•
sv2
•
•
•
– 48 –
sv1
•
•
•
State of Process i
•
•
•
•
•
•

Each state variable represented as array
Indexed by process Id
svk
Modeling System of Identical
Processes
On Each Step:

Select arbitrary process index p
 As if chosen by nondeterministic scheduler

Update state for selected process
•
•
•
•
•
•
inuse
state
p
0/1
next[state] := lambda(i)
case
i = p & state(i) = IDLE:
TRYING
i = p & state(i) = TRYING & inuse : TRYING
i = p & state(i) = TRYING & !inuse: CRITICAL
default:
– 49 – esac
CRITICAL
IDLE
state(i)
TRYING
Model Checking Software
Program is Hard to Model as Finite-State Machine

Large number of large data words means lots of bits
 Although “finite”, bound is very large

Recursion requires stack
 Conceptually unbounded
Creating Finite State Abstraction

Microsoft SLAM verifier
 Focus on device drivers

Start with very abstract model of program
 Every conditional can arbitrarily be taken/not-taken

Check properties
 E.g., always close files

– 50 –
Refine when find counterexample
 More careful analysis of conditionals
Code Verification Example


Adapted by Tom Ball from PCI device driver code
Initial verification run based on simple model of control flow
do {
lock(v);
old = new;
if (test()) {
unlock(v);
new++;
}
} while (new != old);
unlock(v);
– 51 –
Properties to Check



Cannot unlock v unless
locked
Cannot lock v unless
unlocked
Must exit code with v
unlocked
Model as Boolean Program

All conditionals abstracted as Boolean variables
 Allows arbitrary branching

Finite-state approximation of program
do {
lock();
if (a) {
unlock();
Apparent bug: May call lock twice
}
} while (b);
unlock();
Apparent bug: May call unlock twice
– 52 –
Refining Abstraction


Add more detail to model to prove that errors do not occur
Use lightweight theorem prover to check
Double locking
!a  !b
do {
lock();
old = new old = new;
if (test()) {
unlock();
new++;
}
} while (new != old);
unlock();
– 53 –
do {
lock();
!a
b
if (a) {
unlock();
}
} while (b);
unlock();
Refining Abstraction (cont.)

Continue using counterexamples to generate more
constraints on allowed state transitions
Double unlocking
a  b
do {
lock();
old = new old = new;
if (test()) {
unlock();
old  new new++;
}
} while (new != old);
unlock();
– 54 –
do {
lock();
old = new;
if (a) {
unlock();
a
new++;
}
!b
} while (b);
unlock();
Software Verification Status
Shows Promise


Reason about real-life code
Fully automatic
 No user-supplied assertions or induction hypotheses
Still in Early Stages

Can only deal with limited class of programs
 Memory referencing & aliasing possibilities difficult to decipher

Look for particular classes of errors
 Property checking rather than comprehensive verification
– 55 –