Oracle Semantics for Concurrent
Separation Logic
Aquinas Hobor
Joint work with
Andrew Appel & Francesco Zappa Nardelli
Goal
We want to add concurrency to large, realistic
systems in a provably correct, modular way
Those systems may already exist in a sequential
form; we would like to re-use existing code and
machine-checked proofs wherever possible
The key is to isolate the sequential and concurrent
reasoning from each other.
2
Leroy, 2006
CompCert Project
Sequential
Source Program
(C minor)
Sequential C minor
Operational Semantics
CompCert
Compiler
Sequential
Target Program
(Power PC)
User
Sequential
Translation Correctness
Guarantee
Leroy
Sequential PowerPC
Operational Semantics
3
Leroy, 2006
CompCert Project
Sequential
Source Program
(C minor)
What is relationship here?
Sequential C minor
Operational Semantics
CompCert
Compiler
Sequential
Target Program
(Power PC)
User
Sequential
Translation Correctness
Guarantee
Leroy
Sequential PowerPC
Operational Semantics
4
Appel & Blazy, 2007
Separation Logic for C minor
Sequential
Source Program
(C minor)
Axiomatic Semantics
(Separation Logic)
Program
Verification
Sequential C minor
Operational Semantics
Soundness
Proof
CompCert
Compiler
Sequential
Target Program
(Power PC)
User
Sequential
Translation Correctness
Guarantee
Leroy
Appel & Blazy
Sequential PowerPC
Operational Semantics
5
Changes Required for Concurrency
Concurrent
Source Program
(C minor)
Axiomatic Semantics
(Concurrent
Separation Logic)
Program
Verification
Concurrent C minor
Operational Semantics
Soundness
Proof
Concurrency-Aware
CompCert
Compiler
Concurrent
Target Program
(Power PC)
User
Concurrent
Translation Correctness
Guarantee
Future Work
Concurrent PowerPC
Operational Semantics
This paper (Hobor, Appel, & Zappa Nardelli) 6
Changes Required for Concurrency
Concurrent
Source Program
(C minor)
Axiomatic Semantics
(Concurrent
Separation Logic)
Program
Verification
Concurrent C minor
Operational Semantics
Soundness
Proof
Concurrency-Aware
CompCert
Compiler
Concurrent
Target Program
(Power PC)
User
Concurrent
Translation Correctness
Guarantee
Future Work
Concurrent PowerPC
Operational Semantics
This paper (Hobor, Appel, & Zappa Nardelli) 7
Additions to C minor language
We add five new statements:
lock e
lock a lock
unlock e
unlock a lock
makelock e R turn an address into a lock
freelock e
restore an address
fork e el
start a new thread
8
Additions to C minor language
We add five new statements:
lock e
lock a lock
unlock e
unlock a lock
makelock e R turn an address into a lock
freelock e
restore an address
fork e el
start a new thread
Resource Invariant – Describes what resources a thread
acquires on lock and relinquishes on unlock
9
Shallow Embedding
Our invariants are semantic and shallowly embedded in
both the syntax of programs and Coq: we avoid the
POPLmark Quagmire
The shallow embedding means that our invariants are
easy to use and reason about in Coq, and can be freely
shared between code (in makelock) and proof
To enable this behavior, a more complex semantic model
for invariants was required
10
Modularity Principle
Sequential Features
s1 ; s2
if e then s1 else s2
while e do s
break n
(x1, …, xn) := e (e1, …, em)
return (e1, …, en)
x := e
[e1] := e2
Skip
…
Concurrent Features
lock e
unlock e
fork e (e1, …, en)
makelock e R
freelock e
…
11
Modularity Principle
Sequential Features
s1 ; s2
if e then s1 else s2
while e do s
break n
(x1, …, xn) := e (e1, …, em)
return (e1, …, en)
x := e
[e1] := e2
Skip
Connection
…
Concurrent Features
lock e
unlock e
fork e (e1, …, en)
makelock e R
freelock e
…
Easy – Just Syntax
12
Changes Required for Concurrency
Concurrent
Source Program
(C minor)
Axiomatic Semantics
(Concurrent
Separation Logic)
Program
Verification
Concurrent C minor
Operational Semantics
Soundness
Proof
Concurrency-Aware
CompCert
Compiler
Concurrent
Target Program
(Power PC)
User
Concurrent
Translation Correctness
Guarantee
Future Work
Concurrent PowerPC
Operational Semantics
This paper (Hobor, Appel, & Zappa Nardelli)13
Modularity Principle
Sequential Reasoning
Concurrent Reasoning
Sequential Separation Logic
Soundness
Concurrent Separation
Logic Soundness
Sequential Compiler
Correctness
Concurrent Compiler
Correctness
Type Systems
Synchronization
…
…
14
Modularity Principle
Sequential Reasoning
Concurrent Reasoning
Sequential Separation Logic
Soundness
Concurrent Separation
Logic Soundness
Sequential Compiler
Correctness
Concurrent Compiler
Correctness
Type Systems
Synchronization
…
…
Insert Magic Here
15
How is it done?
Key: only consider well-synchronized programs
– Mutual exclusion via Dijkstra Semaphores
– Data-race free
Our operational semantics gets stuck on
ill-synchronized programs
16
Changes to Pure Sequential Semantics
C minor small-step sequential semantics
step relation: (1, 1) ↦ (2, 2)
 = (ρ, m)
 is a control stack
World-aware C minor sequential semantics
step relation: (1, 1) ↦ (2, 2)
 = (ρ, w, m)
w is a map from addresses to ownerships
17
Ownerships
Basic ownerships:
| NONE
| VAL
| LOCK of invariant
(data)
(invariant comes from makelock)
Get stuck if you use memory without ownership
Thread worlds are disjoint – at any time, only one thread can
access each address
Sequential instructions and proofs ignore LOCK
18
Concurrent Operational Semantics
Concurrent C minor step relation
(1, K1, m1, G1) ⇒ (2, K2, m2, G2)
 is scheduler
K is thread list (ρ1, w1, 1) :: (ρ2, w2, 2) :: …
m is memory
G is global world – owner of unlocked locks
To execute a sequential instruction, use the world-aware
sequential step relation
19
Concurrent Instructions
• The concurrent step executes concurrent
instructions directly
– Updates memory at the lock instruction
– Maintains thread list
– Transfers world between threads and global pool
• unlock: world transferred to global pool
• lock:
world transferred from global pool
20
Space
Thread A
Time
w1
Thread B
w
wR
Global World
w2
w3
Memory
unlock 𝓁
w1
w2
w
wR
w3
Memory
lock 𝓁
w1
w
wR
w2
w3
Memory
wR is the unique world that satisfies l’’s lock invariant
21
Coroutine Interleaving
Executes sequentially without interleaving
Interleave when we get to concurrent operation
Key: we have well-synchronized programs
22
Not Computable
(ρ, wlock, m) ⊩ l ⇝ P
m(l) = 0
w’  wlock = w
Ki = (ρ, w, (unlock l) • )
(ρ, wlock, m) ⊩ ⊳P
m’ = [l ↦ 1] m
G  wlock = G’
K’ = [i -> (ρ, w’, )] K
Unlock
(i :: , K, m, G) ⇒ (, K’, m’, G’)
Key: Does not prevent compilation!
23
Reasoning About Concurrency
Most of the time, concurrent programs are
executing sequential code
Proofs about sequential features are hard enough
We don’t want to add the extra complexity of
concurrency when reasoning about sequential code
Idea: why can’t we just pretend it is sequential?
24
Sequential Reasoning
x := x + 1
[y] := 2
lock l
a := [x]
25
Sequential Reasoning
(1, 1)
↦
x := x + 1
(2, 2)
↦
[y] := 2
(3, 3)
lock l
STUCK
a := [x]
26
Oracular Reasoning
(1, 1)
(o1, 1, 1)
(2, 2)
↦
↦
x := x + 1
(o1, 2, 2)
(3, 3)
↦
↦
[y] := 2
(o1, 3, 3)
↦
lock l
(o2, 4, 4)
STUCK
↦
a := [x]
(o2, 5, 5)
27
Oracular Composition
An oracle o is (, K, G)
 is the scheduler
K is the other threads
G is the global world
The oracle simulates running all of the other
threads until the scheduler returns control to
this thread
28
Soundness of Oracular Reasoning
Theorem (connection of oracle and concurrent semantics)
If a thread executes in a certain way on the
oracular machine, then it executes in the
same way when executing in the concurrent
machine
29
Changes Required for Concurrency
Concurrent
Source Program
(C minor)
Axiomatic Semantics
(Concurrent
Separation Logic)
Program
Verification
Concurrent C minor
Operational Semantics
Soundness
Proof
Concurrency-Aware
CompCert
Compiler
Concurrent
Target Program
(Power PC)
User
Concurrent
Translation Correctness
Guarantee
Future Work
Concurrent PowerPC
Operational Semantics
This paper (Hobor, Appel, & Zappa Nardelli)30
CSL 1.0 by O’Hearn, 2006
Concurrent Separation Logic 2.0
Extension of separation logic to handle concurrency
Includes all of typical the rules of separation logic
Associate with each lock an invariant R
l⇝R
≡ l is a lock with invariant R
Separation
Logic
Rules for concurrent operations
{l ⇝ R}
lock l
{(l ⇝ R) * R}
{(l ⇝ R) * R}
unlock l
{l ⇝ R}
Concurrent
Separation Logic
Programs proved in CSL are well-synchronized!
31
Space
Time
Thread A
FA
𝓁⇝ R
w1
Thread B
FB
R
w
wR
Thread C
w2
w3
Memory
lock 𝓁
FA
w1
unlock 𝓁
𝓁⇝ R
w
R
wR
FB
w2
w3
Memory
{𝓁 ⇝ R} lock 𝓁 {(𝓁 ⇝ R) ∗ R}
{FA ∗ (𝓁 ⇝ R)} lock 𝓁 {FA ∗ (𝓁 ⇝ R) ∗ R}
(lock rule)
(frame rule)
32
Difficulties in modeling
Invariants need to be able to refer to other invariants
– Example: 50 ⇝ ∃l2. ((51 ↦ l2) ∗ (l2 ⇝ (∃x.l2+1 ↦ 2*x)
100
lock
101
even
l23
lock
124
even
33
Difficulties in modeling
Invariants need to be able to refer to other invariants
– Example: 50 ⇝ ∃l2. ((51 ↦ l2) ∗ (l2 ⇝ (∃x.l2+1 ↦ 2*x)))
50
lock
51
pointer
100
lock
101
even
l23
lock
124
even
Nested Invariants are Difficult to Model
34
A modal substructural logic
When a logical proposition describes when and where
another logical proposition holds, we have a modal logic
To build a semantic model of the modal logic, we will use
the “very modal model” of Appel, Melliès, Richards, &
Vouillon
Unlike in that work, where it was sufficient to have only
modal relationships, we also need to describe spatial
properties, leading to a modal substructural logic
35
Changes Required for Concurrency
Concurrent
Source Program
(C minor)
Axiomatic Semantics
(Concurrent
Separation Logic)
Program
Verification
Concurrent C minor
Operational Semantics
Soundness
Proof
Concurrency-Aware
CompCert
Compiler
Concurrent
Target Program
(Power PC)
User
Concurrent
Translation Correctness
Guarantee
Future Work
Concurrent PowerPC
Operational Semantics
This paper (Hobor, Appel, & Zappa Nardelli)36
Changes Required for Concurrency
Concurrent
Source Program
(C minor)
Axiomatic Semantics
(Concurrent
Separation Logic)
Program
Verification
Concurrent C minor
Operational Semantics
Soundness
Proof
Concurrency-Aware
CompCert
Compiler
Concurrent
Target Program
(Power PC)
User
Concurrent
Translation Correctness
Guarantee
Future Work
Concurrent PowerPC
Operational Semantics
This paper (Hobor, Appel, & Zappa Nardelli)37
Exploit Oracle Step
Axioms of sequential separation logic
are proved by machine checked proofs
of Appel & Blazy with only minor
modification
Separation
Logic
Appel &
Blazy
Concurrent
Separation Logic
The additional axioms added by
Hobor, Appel,
Concurrent Separation Logic are able to
Zappa Nardelli
ignore the difficulties of sequential control
flow and focus on the truly concurrent behavior
38
Status of Machine Checked Proofs
“As expected, it took longer than expected”
- Hobor, Appel, Zappa Nardelli
About 58k lines of proof script at the moment
“Building such scripts is surprisingly addictive, in
a videogame kind of way…”
- Xavier Leroy
39
Status of Machine Checked Proofs
Definition of the Concurrent Machine
Done
Definition of the Oracular Machine
Done
Definition of Hoare Tuple
Done
Sequential Separation Logic Rules
Done
Concurrent S.L. Rules
90% Done (Unlock)
Oracular Soundness
90% Done (Fork)
40
Future Work
Concurrent
Source Program
(C minor)
Axiomatic Semantics
(Concurrent
Separation Logic)
Program
Verification
Concurrent C minor
Operational Semantics
Soundness
Proof
Concurrency-Aware
CompCert
Compiler
Concurrent
Target Program
(Power PC)
User
Concurrent
Translation Correctness
Guarantee
Future Work
Concurrent PowerPC
Operational Semantics
This paper (Hobor, Appel, & Zappa Nardelli)41
Future Work
Concurrent
Source Program
(C minor)
Axiomatic Semantics
(Concurrent
Separation Logic)
Program
Verification
Concurrent C minor
Operational Semantics
Soundness
Proof
Concurrency-Aware
CompCert
Compiler
Concurrent
Target Program
(Power PC)
User
Concurrent
Translation Correctness
Guarantee
Future Work
Concurrent PowerPC
Operational Semantics
This paper (Hobor, Appel, & Zappa Nardelli)42
Future Work
Concurrent
Source Program
(C minor)
Axiomatic Semantics
(Concurrent
Separation Logic)
Program
Verification
Concurrent C minor
Operational Semantics
Soundness
Proof
Concurrency-Aware
CompCert
Compiler
Concurrent
Target Program
(Power PC)
User
Concurrent
Translation Correctness
Guarantee
Future Work
Concurrent PowerPC
Operational Semantics
This paper (Hobor, Appel, & Zappa Nardelli)43
Concurrent C minor Project
Concurrent
Source Program
(C minor)
Axiomatic Semantics
(Concurrent
Separation Logic)
Program
Verification
Concurrent C minor
Operational Semantics
Soundness
Proof
Concurrency-Aware
CompCert
Compiler
Concurrent
Target Program
(Power PC)
User
Concurrent
Translation Correctness
Guarantee
Future Work
Concurrent PowerPC
Operational Semantics
This paper (Hobor, Appel, & Zappa Nardelli)44
45
46
47
Questions?
48
Modularity Troubles
Sequential Step
Convenient for sequential reasoning
Concurrent Step
Convenient for concurrent reasoning
But…
But…
Gets stuck on concurrent instructions:
impossible for concurrent reasoning!
Inconvenient for sequential reasoning:
too messy!
Dependency: Concurrent Step Built from Sequential Step
49
What we want
Oracular Step
Convenient for sequential reasoning
And
Easy to access Concurrent Step
for concurrent reasoning!
Concurrent Step
Convenient for concurrent reasoning
And
We do not have to use it for sequential
reasoning
Dependency: Oracular Step Built from Concurrent Step
50
Oracle Semantics
C minor oracle semantics
step relation: (o1, 1, 1) ↦ (o2, 2, 2)
o is an oracle
 = (ρ, w, m)
For all sequential instructions, o1 = o2 and the machine
behaves identically to the world-aware sequential step
For concurrent instructions, consult the oracle to learn result
51
Modular Design
Sequential Machine
1 ↦ 2
Concurrent Machine
S1 ⇒ S2
(1, 1) ↦ (2, 2)
(1, 1) ⇒ (2, 2)
i local (e.g. registers)
i global (e.g. memory)
i threads (1i, …, ni)
i globals (i, i)
i synchronizers (e.g.
scheduler)
52