K. Rustan M. Leino RiSE, Joint work with: Peter Müller (ETH Zurich) Jan Smans (KU Leuven) Special thanks to Mike Barnett VMCAI, Madrid, Spain, 18 January 2010 Interleaving of thread executions Unbounded number of: threads, locks, … We need some basis for doing the reasoning A way of thinking! Experimental language with focus on: Shared-memory concurrency Static verification Key features Memory access governed by a model of permissions Sharing via locks with monitor invariants Copy-free non-blocking channels Deadlock checking, dynamic lock re-ordering Other features Classes; Mutual exclusion and readers/writers locks; Fractional permissions; Two-state monitor invariants; Asynchronous method calls; Memory leak checking; Logic predicates and functions; Ghost and prophecy variables Access to a memory location requires permission Permissions are held by activation records Syntax for talking about permission to y: acc(y) method Main() { var c := new Counter; call c.Inc(); } method Inc() requires acc(y); ensures acc(y); { y := y + 1; } call == fork + join call x,y := o.M(E, F); is semantically like fork tk := o.M(E, F); join x,y := tk; … but is compiled to more efficient code class var var var XYZ { x: int; y: int; z: int; method Main() { var c := new XYZ; fork c.A(); fork c.B(); } … } method A() requires acc(x); { x := x + 1; } method B() requires acc(y) && acc(z); { y := y + z; } acc(y) write permission to y rd(y) read permission to y At any one time, at most one thread can have write permission to a location class var var var Fib { x: int; y: int; z: int; method Main() { var c := new Fib; fork c.A(); fork c.B(); } … } method A() requires rd(x) && acc(y) { y := x + 21; } method B() requires rd(x) && acc(z) { z := x + 34; } acc(y) 100% permission to y acc(y, p) p% permission to y rd(y) read permission to y Write access requires 100% Read access requires >0% = + What if two threads want write access to the same location? class Fib { var y: int; method Main() { var c := new Fib; fork c.A(); fork c.B(); } … } method A() … { y := y + 21; } method B() … { y := y + 34; } class Fib { var y: int; invariant acc(y); method Main() { var c := new Fib; share c; fork c.A(); fork c.B(); } … } method A() … { acquire this; y := y + 21; release this; } method B() … { acquire this; y := y + 34; release this; } The concepts holding a lock, and having permissions are orthogonal to one another In particular: Holding a lock does not imply any right to read or modify shared variables Their connection is: Acquiring a lock obtains some permissions Releasing a lock gives up some permissions Like other specifications, monitors can hold both permissions and conditions Example: invariant acc(y) && 0 ≤ y [Chalice encoding by Bart Jacobs] class MyClass { var x,y: int; predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } … } class MyClass { var x,y: int; predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } method New() returns (c: MyClass) ensures c.Valid; { … } method Mutate() requires c.Valid; ensures c.Valid; { … } } class MyClass { var x,y: int; predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } method New() returns (c: MyClass) ensures c.Valid; { var c := new MyClass { x := 3, y := 5 }; fold c.Valid; } method Mutate() requires c.Valid; ensures c.Valid; { unfold c.Valid; c.y := c.y + 3; fold c.Valid; } } class MyClass { var x,y: int; predicate Valid { acc(c.x) && acc(c.y) && x ≤ y } method New() returns (c: MyClass) ensures c.Valid; { var c := new MyClass { x := 3, y := 5 }; fold c.Valid; } method Mutate() requires c.Valid; ensures c.Valid; { unfold c.Valid; c.y := c.y + 3; fold c.Valid; } } channel Ch(c: Cell, z: int) where acc(c.y) && c.y ≤ z; channel Ch(c: Cell, z: int) where acc(c.y) && c.y ≤ z; class Cell { var x,y: int; method Producer(ch: Ch) { var c := new C { x := 0, y := 0 }; send ch(c, 5); } method Consumer(ch: Ch) { receive c,z := ch; … } } channel Ch(c: Cell, z: int) where acc(c.y) && c.y ≤ z; class Cell { var x,y: int; method Producer(ch: Ch) { var c := new C { x := 0, y := 0 }; send ch(c, 5); } method Consumer(ch: Ch) { receive c,z := ch; … } } A deadlock is the situation where a nonempty set (cycle) of threads each waits for a resource (e.g., lock) that is held by another thread in the set Deadlocks are prevented by making sure no such cycle can ever occur The program partially order locks The program is checked to acquire locks in strict ascending order Wait order is a dense partial order (Mu, <<) with a bottom element << is the strict version of << The wait level of an object o is stored in a mutable ghost field o.mu Accessing o.mu requires appropriate permissions, as for other fields method M() requires rd(a.mu); requires rd(b.mu); requires waitlevel << a.mu; requires a.mu << b.mu; { acquire a; acquire b; … } method N() requires rd(a.mu) requires rd(b.mu) requires waitlevel << b.mu; requires b.mu << a.mu; { acquire b; acquire a; … } With these preconditions, both methods verify The conjunction of the preconditions is false, so the methods can never be invoked at the same time Recall, the wait level of an object o is stored in the ghost field o.mu Initially, the .mu field is The .mu field is set by the share statement: share o between L and H; picks some wait level strictly between L and H, and sets o.mu to that level Provided L << H and neither denotes an extreme element, such a wait level exists, since the order is dense Changing o.mu requires acc(o.mu), as usual Given as translation to Boogie Chalice Boogie is an intermediate verification language Boogie Z3 Chalice: Boogie: o.f Heap[ o, f ] where Heap is declared to be a map from objects and field names to values To encode permissions, use another map: Mask call M() = method M() requires P; ensures Q; Exhale[[ P ]]; Inhale[[ Q ]] Defined by structural induction For expression P without permission predicates Exhale P Inhale P ≡ ≡ assert P assume P Exhale acc(o.f, p) ≡ assert p ≤ Mask[o,f]; Mask[o,f] := Mask[o,f] – p; Inhale acc(o.f, p) ≡ if (Mask[o,f] == 0) { havoc Heap[o,f]; } Mask[o,f] := Mask[o,f] + p; call M() = class Cell { var y: int; method Square() requires acc(y); ensures acc(y) && y = old(y*y); assert 100 ≤ Mask[this,y]; Mask[this,y] := Mask[this,y] – 100; oldH := Heap; if (Mask[this,y] = 0) { havoc Heap[this,y]; } Mask[this,y] := Mask[this,y] + 100; assume Heap[this,y] = oldH[this,y] * oldH [this,y]; method Square(c: Cell) requires acc(c.y); ensures acc(c.y) && c.y == old(c.y*c.y); method Square(c: Cell, ghost K: int) requires acc(c.y) && K == c.y; ensures acc(c.y) && c.y == K*K; call Square(c, c.y); Logical constant: call Square(c, *); Let the verifier figure out K! method Square(c: Cell) returns (r: int) requires acc(c.y); ensures acc(c.y) && r == c.y*c.y; method Square(c: Cell) returns (r: int) requires acc(c.y, ε); ensures acc(c.y, ε) && r == c.y*c.y; method Square(c: Cell, ghost K: perm) returns (r: int) requires acc(c.y, K); ensures acc(c.y, K) && r == c.y*c.y; method Square(c: Cell, ghost K: perm) returns (r: int) requires acc(c.y, K); ensures acc(c.y, K) && r == c.y*c.y; A better notation? rd(c.y)? Does that pick one K for each method activation? Can these 3 kinds of “parameters” (real, ghost, permission) be treated more uniformly? Which kinds should be indicated explicitly by the programmer and which should be figured out by the compiler? Permissions guide what memory locations are allowed to be accessed Activation records can hold permissions Permissions can also be stored in various “boxes” (monitors, predicates, channels) Permissions can be transferred between activation records and boxes Locks grant mutually exclusive access to monitors Chalice (and Boogie) available as open source: http://boogie.codeplex.com Tutorial and other papers available from: http://research.microsoft.com/~leino