Grand Challenge Problem:
Model Check Concurrent Software
Edmund M. Clarke
Department of Computer Science
Carnegie Mellon University
Outline of Talk
1.
2.
3.
4.
5.
Explain what model checking is
Some successes of model checking
What makes software different
Approaches to software model checking
Some of my projects
What is Model Checking?
“The
are Glitch Project”
Bad pun for cult movie “The Blair Witch Project” ?
Temporal Logic Model Checking
• Model checking is an automatic verification technique for finite
state concurrent systems.
• Developed independently by Clarke, Emerson, and Sistla and by
Queille and Sifakis in early 1980’s.
• Specifications are written in propositional temporal logic.
• Verification procedure is an exhaustive search of the state space
of the design.
Advantages of Model Checking
• No proofs!!!
• Fast (compared to other rigorous methods such as
theorem proving)
• Diagnostic counterexamples
• No problem with partial specifications
• Logics can easily express many concurrency properties
Main Disadvantage
State Explosion Problem:
• Too many processes
• Data Paths
Much progress has been made on this problem recently!
Model of computation
Microwave Oven Example
State-transition graph
describes system evolving
over time.
s
No concurrency int
simple
~ Start
~ Close
~ Heat
~ Error
examples.
Start
~ Close
~ Heat
Error
Start
Close
~ Heat
Error
~ Start
Close
~ Heat
~ Error
Start
Close
~ Heat
~ Error
~ Start
Close
Heat
~ Error
Start
Close
Heat
~ Error
Temporal Logic
 The oven doesn’t heat up until the door is closed.
 Not heat_up holds until door_closed

(~ heat_up) U door_closed
Basic Temporal Operators
The symbol “p” is an atomic proposition, e.g. “Device Enabled”.
•
•
•
•
Fp
Gp
Xp
pUq
- p holds sometime in the future.
- p holds globally in the future.
- p holds next time.
- p holds until q holds.
Model Checking Problem
Let M be a state-transition graph.
Let ƒ be the specification in temporal logic.
Find all states s of M such that M, s ƒ.
Efficient Algorithms: CE81, CES83
The EMC System
Model Checker
(EMC)
Preprocessor
Specification
State Transition Graph
104 to 105 states
True or Counterexamples
Breakthrough!
Ken McMillan implemented our model checking algorithm using
Binary Decision Diagrams in 1987.
Model Checker SMV
Now able to handle much larger examples!!
Combating the
State Explosion Problem
• Binary Decision Diagrams can be used to represent state
transition systems more efficiently.
• The partial order reduction can be used to reduce the number
of states that must be enumerated.
• Many techniques for alleviating state explosion:
– Abstraction.
– Compositional reasoning.
– Symmetry.
– Cone of influence reduction.
– Semantic minimization.
Model Checker Performance
• Model checkers today can routinely handle systems with
between 100 and 1000 state variables.
• Systems with 10120 reachable states have been checked.
(Compare approx. 1078 atoms in universe.)
• By using appropriate abstraction techniques, systems with an
essentially unlimited number of states can be checked.
Model Checking Systems
• There are many successful examples of the use of model checking
in hardware and protocol verification.
• The fact that industry (INTEL, IBM, MOTOROLA) is starting to use
model checking is encouraging.
• Below are some well-known model checkers.
Temporal Logic Model Checkers
• The first two model checkers were EMC (Clarke, Emerson,
Sistla) and Caesar (Queille, Sifakis).
• SMV is the first model checker to use BDDs.
• Spin uses the partial order reduction to reduce the state
explosion problem.
• Verus and Kronos check properties of real-time systems.
• HyTech is designed for reasoning about hybrid systems.
Notable ExamplesIEEE Futurebus+
• In 1992 Clarke and his students at CMU used SMV to verify the
IEEE Future+ cache coherence protocol.
• They found a number of previously undetected errors in the
design of the protocol.
• This was the first time that formal methods have been used to
find errors in an IEEE standard.
• Although the development of the protocol began in 1988, all
previous attempts to validate it were based entirely on informal
techniques.
Notable Examples-IEEE SCI
• In 1992 Dill and his students at Stanford used Murphi to verify
the cache coherence protocol of the IEEE Scalable Coherent
Interface.
• They found several errors, ranging from uninitialized variables to
subtle logical errors.
• The errors also existed in the complete protocol, although it had
been extensively discussed, simulated, and even
implemented.
Notable Examples-PowerScale
• In 1995 researchers from Bull and Verimag used LOTOS to
describe the processors, memory controller, and bus arbiter
of the PowerScale multiprocessor architecture.
• They identified four correctness requirements for proper
functioning of the arbiter.
• The properties were formalized using bisimulation relations
between finite labeled transition systems.
• Correctness was established automatically in a few minutes
using the CÆSAR/ ALDÉBARAN toolbox.
Notable Examples - HDLC
• A High-level Data Link Controller was being designed at
AT&T in Madrid in 1996.
•
Researchers at Bell Labs offered to check some properties of
the design using the FormalCheck verifier.
• Within five hours, six properties were specified and five were
verified.
• The sixth property failed, uncovering a bug that would have
reduced throughput or caused lost transmissions!
Notable Examples
PowerPC 620 Microprocessor
• Richard Raimi used Motorola’s Verdict model checker to debug
a hardware laboratory failure.
• Initial silicon of the PowerPC 620 microprocessor crashed
during boot of an operating system.
• In a matter of seconds, Verdict found a BIU deadlock causing
the failure.
Notable Examples-Analog Circuits
• In 1994 Bosscher, Polak, and Vaandrager won a best-paper
award for proving manually the correctness of a control protocol
used in Philips stereo components.
• In 1995 Ho and Wong-Toi verified an abstraction of this protocol
automatically using HyTech.
• Later in 1995 Daws and Yovine used Kronos to check all the
properties stated and hand proved by Bosscher, et al.
Notable Examples-ISDN/ISUP
• The NewCoRe Project (89-92) was the first application of formal
verification in a software project within AT&T.
• A special purpose model checker was used in the development
of the CCITT ISDN User Part Protocol.
• Five “verification engineers” analyzed 145 requirements.
• A total of 7,500 lines of SDL source code was verified.
• 112 errors were found; about 55% of the original design
requirements were logically inconsistent.
Notable Examples-Edifices
• In 1995 the Concurrency Workbench was used to analyze an
active structural control system to make buildings more
resistant to earthquakes.
• The control system sampled the forces being applied to the
structure and used hydraulic actuators to exert countervailing
forces.
• A timing error was discovered that could have caused the
controller to worsen, rather than dampen, the vibration
experienced during earthquakes.
Grand Challenge:
Model Check Software !
What makes Software Model Checking
different ?
What Makes Software Model
Checking Different ?
• Large/unbounded base types: int, float, string
• User-defined types/classes
• Pointers/aliasing + unbounded #’s of heap-allocated cells
• Procedure calls/recursion/calls through pointers/dynamic method
lookup/overloading
• Concurrency + unbounded #’s of threads
What Makes Software Model
Checking Different ?
•
•
•
•
•
•
Templates/generics/include files
Interrupts/exceptions/callbacks
Use of secondary storage: files, databases
Absent source code for: libraries, system calls, mobile code
Esoteric features: continuations, self-modifying code
Size (e.g., MS Word = 1.4 MLOC)
What Does It Mean to
Model Check Software?
1. Combine static analysis and model checking
Use static analysis to extract a finite state
synchronization skeleton K from the program.
Then check that K models f (K ² f), where f is the
specification of the program.
² Bandera (Kansas State),
² Java PathFinder (NASA Ames)
What Does It Mean to
Model Check Software?
2. Simulate program along all paths in computation
tree
² Source code + backtracking (e.g., Verisoft)
² Source code + symbolic execution + backtracking (e.g.,
MS/Intrinsa Prefix)
3. Use finite-state machine to look for patterns in
control-flow graph [Engler]
What Does It Mean to
Model Check Software?
4. Design with Finite-State Software Models
Finite state software models can act as “missing link”
between transition graphs and complex software.
² Statecharts
² Esterel
² Lustre
² OO Software Models
Example
Statechart for Brake Control
BRAKE_CONTROL
ACTUATOR
DYNAMIC
DYNAMIC_
PARK_BRAKE_
ABS_OFF
[VEHICLE_SPEED>=5 and
PARKING_BRAKE_BUTTON]
ACTUATOR_
HOME
ACTUATOR_
REGULATION
[VEHICLE_RUN_MODE and
not PARKING_BRAKE_BUTTON]
[VEHICLE_SPEED<5
[VEHICLE_SPEED<5]
[WHEELSLIP>THRESHOLD and
in(RR_ACTUATOR_REGULATION)
and VEHICLE_SPEED>=5]
or
WHEELSLIP<=THRESHOLD
or
(VEHICLE_RUN_MODE and
not PARKING_BRAKE_BUTTON and
in(ACTUATOR_REGULATION))]
[VEHICLE_RUN_MODE and
not PARKING_BRAKE_BUTTON]
BRAKE_
LATCHED
[VEHICLE_SPEED<5 and PARKING_BRAKE_BUTTON]
DYNAMIC_
PARK_BRAKE_
ABS_ON
New Projects
(Joint research with Tom Reps,
Somesh Jha, and Helmut Veith)
Objective:
Develop mechanisms and build tools for establishing properties of
concurrent software:
e.g. absence of deadlock, correctness of security features,
correct response times, etc.
Project 1
From Shape-Analysis to Model Checking
Obtaining useful models of software in the presence of
•
•
•
•
User-defined types/classes
Pointers/aliasing + unbounded #’s of heap-allocated cells
Procedure calls/recursion
Concurrency + unbounded #’s of threads
Shape Analysis: Formalizing “. . .”
y
A list:
x
y
Informal:
x
y
Formal:
x
{p[x]}
{r[x]}
{p[y],r[x]}
{r[y],r[x]}
Java Threads Are Heap-Allocated Objects
 Thread Analysis  Shape Analysis
A memory configuration:
thread1
atStart
thread2
atStart
thread4
atStart
csLock
csLock
lock1
isAcquired
heldBy
csLock
csLock
thread3
inCritical
Java Threads Are Heap-Allocated Objects
 Thread Analysis  Shape Analysis
An abstract memory configuration:
thread’
atStart
csLock
lock1
isAcquired
heldBy
csLock
thread
inCritical
Java Threads Are Heap-Allocated Objects
 Thread Analysis  Shape Analysis
Here, model checking means:
Explore the space of possible transitions
among abstract memory configurations
Project 2
Verification of Real Time Embedded Systems
Real time embedded systems:
Set of tasks, to be scheduled on 1 or more processors
Execute periodically
Share resources and data
Hard deadlines
Can be safety critical
Want to verify:
Schedulability
Deadlocks
Data flow (variables used before defined...)
Sound communication
Response times,
.....
Verification of Real Time Embedded Systems
Targeted
Compiler
Embedded System
Statecharts, MATLAB
Correctness
requirements
( temporal logic, RT logic)
Model checking
RT model checking
SMV, UPPAAL, VERUS,...
Verification
results
(counterexample)
producer(int p)
{ bool prod;
p = 0; prod = false;
periodic(0,10,10) {
wait(3);
prod = true; p = p+1;
wait(1);
prod = false;
};}
main()
{ int p,c;
process producer(p);
consumer(c);
...
WCET analysis
of basic blocks
assembly code
inspection
abstract model
no recursion, pointers,
complex data types,...
Global finite
state RT model
tasks + scheduler
+ synchronization
WCET of
complex tasks
BDD based max.
path exploration
Schedulability
Analysis
rate monotonic
analysis
Project 3
Verifying Security Protocol Interaction
• Security protocols run on top of other protocols
– the SSL record protocol runs on top of TCP
• Interactions between these protocols can cause subtle bugs
• Consequence: cannot analyze these protocols in isolation
Abstract
specification
of the
protocol stack
Source
code
Relate
back to
the code
Model checker
capable of handling
security and non-security
related properties
Attacks!
Project 4
Bounded Model-Checking
•
•
•
Problem: How to compute set of reachable states? Fixpoint
computation is too expensive.
Idea: Restrict search to states that are reachable from initial state
within fixed number n of transitions
Implemented by unwinding program
Bounded Model-Checking (cont.)
•Advantages:
+Avoids expensive fixpoint computation
+Number n can be adjusted to available computational power
+Finds all bugs if program terminates quickly enough
+No need for manual loop invariants
•Disadvantages
–Runtime grows with bound, might be too small for large examples
Bounded
Model-Checking
(cont.)
void f(...) {
...
while(cond) {
Body;
}
Rest;
}
Bounded
Model-Checking
(cont.)
void f(...) {
•
...
if(cond) {
Body;
while(cond) {
Body;
}
}
Rest;
}
while() loops are
unwound iteratively
Bounded
Model-Checking
(cont.)
void f(...) {
...
if(cond) {
Body;
if(cond) {
Body;
while(cond) {
Body;
}
}
}
Rest;
}
•
•
while() loops are
unwound iteratively
Use a combination of
decision procedures to
check spec.
Bounded
Model-Checking
(cont.)
void f(...) {
...
if(cond) {
Body;
if(cond) {
Body;
if(cond) {
Body;
while(cond) {
Body;
}
}
}
}
Rest;
}
•
•
•
while() loops are
unwound iteratively
Use a combination of
decision procedures to
check spec.
Assertion may be
inserted after last
iteration: violated if
program runs longer
than bound permits
Bounded
Model-Checking
(cont.)
void f(...) {
...
if(cond) {
Body;
if(cond) {
Body;
if(cond) {
Body;
if(cond) {
assert(FALSE);
}
}
}
}
Rest;
}
•
•
•
•
while() loops are
unwound iteratively
Use a combination of
decision procedures to
check spec.
Assertion may be
inserted after last
iteration: violated if
program runs longer
than bound permits
If program terminates
“fast enough“, one gets
a positive correctness
result!
Project 5: Alternative Approach:
Automated Theorem Proving for Code
SyMP
User Interface
SyMP
Proof Manager
EMacs
Proof System
Language Modules
Abstraction
ANSI-C
Data Types
VHDL
Predicate
PVS
Decision Procedures
CMU SMV
SAT
Presburger
Bitvector Arith.
Simplex
Project Homepage
http://www.cs.cmu.edu/~emc/cip