Chair of Software Engineering
Software Verification
Stephan van Staden
Lecture 10: Model Checking
Overview
 The big picture
 Example to be verified
 Ingredients of a model checking system
 The state explosion problem
 Applications of model checking
 SLAM
2
The big picture
 Application domain: mostly concurrent, reactive systems
 Verification of particular properties
 Verify that a system satisfies a property:
1. Model the system in the model checker’s
description language. Call this model M
2. Express the property to be verified in the
specification language of the model checker. Call
this formula ϕ
3. Run the model checker to show that M satisfies
ϕ, i.e. M ⊨ ϕ
 Automatic for finite-state models
3
Example
2 processes execute in parallel
Each undergoes transitions n → r → c → n →… where
 n denotes “not in critical section”
 r denotes “requesting to enter critical section”
 c denotes “in critical section”
Requirements:
 Safety: Only one process may execute critical
section code at any point
 Liveness: Whenever a process requests to enter its
critical section, it will eventually be allowed to do so
 Non-blocking: A process can always request to enter
its critical section
4
Example (cont)
We write a program P to fulfill these requirements. But is
it really doing its job?
We construct a model M for P where M captures the
relevant behavior of P: s
0
s1
r1n2
s2
c 1n 2
s4
c1r2
n 1n 2
s5
s3
n1r2
s6
n 1c 2
r1r2
s7
r1c2
5
Example (cont)
Based on a definition of when a model satisfies a property,
we can determine whether M satisfies P’s required
properties
Example: M satisfies the safety requirement if no state
reachable from s0 (including s0) is labeled c1c2. Thus our M
satisfies P’s safety requirement
We can formalize the idea
Note: the conclusion that P satisfies these requirements
depends on the (unverified) assumption that M is a faithful
representation of all the relevant aspects of P
6
Ingredients of a model checking system
Models, specifications, a satisfaction relation, a
satisfaction checking algorithm.
Tasks:
1. Define what models are
2. Define what specifications are
3. Define when a model satisfies a specification
4. Devise an efficient algorithm to decide satisfaction
7
1. Define what models are
Many formalisms, e.g. Kripke models, LTSs, …
A transition system M = (S, →, L) is a set of states S with
a transition relation → (a binary relation on S), such that
every s ϵ S has some s’ ϵ S with s → s’, and a labeling
function L: S → Ρ(Atoms), where Atoms is a set of atomic
descriptions.
Our example: a transition system with
 S = {s0, s1, s2, s3, s4, s5, s6, s7}
 → = {(s0,s1), (s0,s5), (s1,s2), (s1,s3), (s4, s5), …}
 L(s0) = {n1,n2}, L(s1) = {r1,n2}, L(s3) = {r1,r2}, … where
Atoms = {n1, r1, c1, n2, r2, c2}
Define a concrete syntax in which models can be described
8
2. Define what specifications are
They describe properties of models
Temporal logics (e.g. LTL, CTL) are often used to specify
properties of concurrent, reactive systems.
Syntax of LTL formulas:
ϕ ::= true | false | p | (¬ϕ) | (ϕ ˄ ϕ) | (ϕ v ϕ) | (ϕ → ϕ) |
(X ϕ)
| (G ϕ) | (F ϕ) | (ϕ U ϕ) | (ϕ W ϕ) | (ϕ R ϕ)
Example specifications:
G ¬(c1 ˄ c2), G (r1 → F c1) and G (r2 → F c2)
Define a concrete syntax for specifications
9
3. Define the satisfaction relation
For a path π = s1→s2→... in a model M & LTL formula ϕ:
π ⊨ true
neXt
π⊨
/ false
π ⊨ p iff p ϵ L(s1)
Globally
π ⊨ ¬ϕ iff π ⊨
/ ϕ
Future
π ⊨ ϕ1 ˄ ϕ2 iff π ⊨ ϕ1 and π ⊨ ϕ2
Until
π ⊨ ϕ1 v ϕ2 iff π ⊨ ϕ1 or π ⊨ ϕ2
Weak-until
π ⊨ ϕ1 → ϕ2 iff π ⊨ ϕ2 whenever π ⊨ ϕ1
π ⊨ X ϕ iff π2 ⊨ ϕ (πi = si→si+1→...)
Release
π ⊨ G ϕ iff for all i ≥ 1, πi ⊨ ϕ
π ⊨ F ϕ iff there is some i ≥ 1 such that πi ⊨ ϕ
π ⊨ ϕ1 U ϕ2 iff there is some i ≥ 1 such that πi ⊨ ϕ2 and
for all 1 ≤ j < i, πj ⊨ ϕ1
10
3. Define the satisfaction relation (cont.)
π ⊨ ϕ1 W ϕ2 iff either π ⊨ ϕ1 U ϕ2 or for all i ≥ 1, πi ⊨ ϕ1
π ⊨ ϕ1 R ϕ2 iff either there is some i ≥ 1 such that πi ⊨ ϕ1
and for all 1 ≤ j ≤ i we have πj ⊨ ϕ2, or for all k ≥ 1, πk ⊨ ϕ2
Suppose M = (S, →, L) is a model, s ϵ S and ϕ is an LTL
formula. M,s ⊨ ϕ iff for every path π of M starting at s we
have π ⊨ ϕ
Thus we are able to prove for our example that:
 M,s0 ⊨ G ¬(c1 ˄ c2) (the safety property holds).
 M,s0 ⊨
/ G (r1 → F c1) (the liveness property does not hold
– a counterexample path is s0→s1→s3→s7→s1→s3→s7→...).
 The non-blocking property is not expressible in LTL.
11
4. Provide a checking algorithm
Devise an efficient algorithm to decide M ⊨ ϕ
Typically without user interaction
Approaches: semantic, automata, tableau, …
Provide a counterexample when M ⊨
/ ϕ. The
counterexample provides a clue to what is wrong:
 The system might be incorrect
 The model might be too abstract (in need of refinement)
 The specification might not be the intended one
12
The state explosion problem
The number of states grow exponentially with the number
of system components
For our example: 3 processes execute in parallel, 4
processes, etc.
Techniques to make state explosion less severe:
 Abstraction
 Induction
 Partial order reduction
…
13
Applications
Verification of specific properties of:
 Hardware circuits
 Communication protocols
 Control software
 Embedded systems
 Device drivers (e.g. SLAM project of Microsoft)
…
Real-world problems: “1020 states and beyond”
Useful: 2007 Turing Award (Clarke, Emerson and Sifakis)
We now turn to SLAM
14
SLAM introduction
Faulty drivers often responsible for OS crashes
Verifying proper driver API usage
E.g. a lock is never released without being first acquired
Prevent
15
SLAM introduction (cont)
Input: a C program P (the device driver code) and temporal
safety properties expressed as a SLIC specification ϕ
Output (if terminates): P ⊨ ϕ or P ⊨
/ ϕ
16
SLIC property specification
A SLIC spec ϕ is a state machine
Example:
state {
enum {Unlocked = 0, Locked = 1}
state = Unlocked;
}
Rel
Acq
Unlocked
Locked
Rel
Acq
Error
KeAcquireSpinLock.return {
if (state==Locked) abort;
else state = Locked;
}
KeReleaseSpinLock.return {
if (state==Unlocked) abort;
else state = Unlocked;
}
17
Translating SLIC spec to C
state {
enum {Unlocked = 0, Locked = 1}
state = Unlocked;
}
KeAcquireSpinLock.return {
if (state==Locked) abort;
else state = Locked;
}
KeReleaseSpinLock.return {
if (state==Unlocked) abort;
else state = Unlocked;
}
enum {Unlocked = 0, Locked = 1}
state = Unlocked;
void slic_abort() {
SLIC_ERROR: ;
}
void KeAcquireSpinLock_return {
if (state==Locked) slic_abort();
else state = Locked;
}
void KeReleaseSpinLock_return {
if (state==Unlocked) slic_abort();
else state = Unlocked;
}
18
The device driver code P
do {
KeAcquireSpinLock();
nPacketsOld = nPackets;
if (request) {
request = request->Next;
KeReleaseSpinLock();
nPackets++;
}
} while (nPackets != nPacketsOld);
KeReleaseSpinLock();
19
The instrumented program P’
Compose P with C version of ϕ to obtain an instrumented P’.
If SLIC_ERROR is reachable in P’, then P ⊨/ ϕ else P ⊨ ϕ.
Example P’:
do {
KeAcquireSpinLock();
A: KeAcquireSpinLock_return();
nPacketsOld = nPackets;
if (request) {
request = request->Next;
KeReleaseSpinLock();
B:
KeReleaseSpinLock_return();
nPackets++;
}
} while (nPacketsOld != nPackets);
KeReleaseSpinLock();
C: KeReleaseSpinLock_return();
enum {Unlocked = 0, Locked = 1}
state = Unlocked;
void slic_abort() {
SLIC_ERROR: ;
}
void KeAcquireSpinLock_return {
if (state==Locked) slic_abort();
else state = Locked;
}
void KeReleaseSpinLock_return {
if (state==Unlocked) slic_abort();
else state = Unlocked;
}
20
The SLAM verification process
Iterative process to determine whether SLIC_ERROR is
reachable in P’:
c2bp
prog. P
SLIC rule
slic
prog. P’
boolean program
bebop
predicates
newton
path
21
SLAM iteration
Let E0 be the predicates present in the conditionals of ϕ.
For iteration i:
1. Apply c2bp to construct the boolean program BP(P’,Ei)
2. Apply bebop to check if there is a path pi in BP(P’,Ei) that
reaches SLIC_ERROR. If bebop determines that
SLIC_ERROR is not reachable, then ϕ is valid in P and the
algorithm terminates
3. If there such a path pi, then newton checks whether pi is
feasible in P’. Two possible outcomes:
1. “yes”: ϕ has been invalidated in P and the algorithm
terminates with counterexample pi
2. “no”: newton finds a set of predicates Fi that explain
the infeasibility of pi in P’
4. Let Ei+1 := Ei ∪ Fi and i := i + 1 and proceed to the next
iteration
22
Some observations
The original satisfaction problem has been translated into
a boolean program reachability problem.
For every statement s in P’ and every predicate e ϵ Ei, c2bp
determines the effect of s on e to build the boolean
program.
23
Iteration 0
E0 = {state = Locked, state = Unlocked}
BP(P’,E0):
do {
skip;
A: KeAcquireSpinLock_return();
skip;
if (*) {
skip;
skip;
B:
KeReleaseSpinLock_return();
skip;
}
} while (*);
skip;
C: KeReleaseSpinLock_return();
decl {state=Unlocked}, {state=Locked};
{state=Unlocked} := T;
{state=Locked} := F;
void slic_abort() {
SLIC_ERROR: skip;
}
void KeAcquireSpinLock_return {
if ({state=Locked}) slic_abort();
else { {state=Locked} := T;
{state=Unlocked} := F; }
}
void KeReleaseSpinLock_return {
if ({state=Unlocked}) slic_abort();
else { {state = Unlocked} := T;
{state = Locked} := F; }
}
24
Iteration 0 (cont)
bebop provides path p0 = A→A→SLIC_ERROR
newton determines that p0 is infeasible in P’ and returns F0
= {nPacketsOld = nPackets} as explanation – it must be
both true and false on p0:
do {
KeAcquireSpinLock();
A: KeAcquireSpinLock_return();
nPacketsOld = nPackets;
if (request) {
request = request->Next;
KeReleaseSpinLock();
B:
KeReleaseSpinLock_return();
nPackets++;
}
} while (nPacketsOld != nPackets);
KeReleaseSpinLock();
C: KeReleaseSpinLock_return();
25
Iteration 1
E1 = {state = Locked, state = Unlocked, nPacketsOld =
nPackets}
BP(P’,E1):
do {
skip;
A: KeAcquireSpinLock_return();
b := T;
if (*) {
skip;
skip;
B:
KeReleaseSpinLock_return();
b := F;
}
} while (!b);
skip;
C: KeReleaseSpinLock_return();
decl {state=Unlocked}, {state=Locked};
{state=Unlocked} := T;
{state=Locked} := F;
void slic_abort() {
SLIC_ERROR: skip;
}
void KeAcquireSpinLock_return {
if ({state=Locked}) slic_abort();
else { {state=Locked} := T;
{state=Unlocked} := F; }
}
void KeReleaseSpinLock_return {
if ({state=Unlocked}) slic_abort();
else { {state = Unlocked} := T;
{state = Locked} := F; }
}
26
Iteration 1 (cont)
bebop establishes that SLIC_ERROR is unreachable in
BP(P’,E1)
SLAM concludes that P ⊨ ϕ
27
Sources
Logic in Computer Science – Modelling and Reasoning about
Systems, 2nd edition, Michael Huth & Mark Ryan,
Cambridge University Press, 2004.
Automatically Validating Temporal Safety Properties of
Interfaces, T. Ball & S. K. Rajmani, SPIN 2001
Tutorial on "Software Model Checking with SLAM" at PLDI
2003
28