mmnet Summer School
Tuesday 20 – Wednesday 21July, 2004,
Canterbury
Speakers:
•
•
•
•
•
•
•
20 February 2004
David Bacon, IBM TJ Watson.
Emery Berger, UMass.
Robert Berry, IBM Hursley.
Hans Boehm, HP.
Dave Detlefs, Sun Microsystems.
Rick Hudson, Intel.
Eliot Moss, UMass.
UKC, February 2004
1
Birrell’s Reference Listing
Revisited
Richard Jones
University of Kent
R.E.Jones@ukc.ac.uk
Luc Moreau
University of Southampton
Peter Dickman
University of Glasgow
Outline
1.
2.
3.
4.
Distributed reference counting – benefits & issues.
Birrell’s algorithm – example.
Weaknesses of Birrell’s description.
Our approach
•
•
Graphical notation
Formalisation & Proof
5. Extensions
•
•
Fault tolerance
Optimisation
6. Conclusion.
20 February 2004
UKC, February 2004
3
Problems of
a distributed world
Concurrency everywhere
• must avoid race conditions, etc
Communication is costly
• changing the reference count of a remote object may cost
10,000 times as much as changing the count of a local
object
Not easy to get complete knowledge of object graph
• synchronisation is expensive
Faults everywhere
• communications, processes
20 February 2004
UKC, February 2004
4
Terminology
Processes: partition computational and storage
resources.
Messages pass in point-to-point channels between
processes.
Channels have properties, such as FIFO or lossy.
A reference is local if it refers to an object allocated in
the same process; alternatively, it is remote (or global).
The owner of a reference is the process that initially
allocated the object to which the reference refers.
20 February 2004
UKC, February 2004
5
Distributed
Reference Counting/Listing
Most widely used DGC technique
• Maintain a count of remote references to each
global object
• Reference listing alternative
Benefits
• Scalable solution
• Easy to implement
But…
• Cannot reclaim garbage cycles
• Easy to implement wrong!
20 February 2004
UKC, February 2004
6
Birrell’s algorithm
Birrell, Evers, Nelson, Owicki, and Wobber. Distributed
Garbage Collection for Network Objects. DEC SRC
technical report 116, 1993.
Widely used: Modula-3 Network Objects; Java RMI.
Based on reference listing, avoids race conditions of
naïve implementations, fault tolerance.
20 February 2004
UKC, February 2004
9
Birrell’s description
Process P: owner of O
Object table
Process Q: a client of O
Object table
w (o)
w (o)
weak ref
concrete O
surrogate for O
o.dirtySet = {Q,…}
Dirty
WireRep:
Object
set:
table:
identifiers
unique
maps
IDaof
ofwirerep
processes
owner,
w(o)
plusthat
to
index
the
have
local
of surrogates.
object
instance
at theof the
Concrete
and
surrogate
objects.
owner.
object.
•Dirty-set
=  the
o can
be removed
the object
table.
•Client invokes
surrogate,
whosefrom
methods
perform
RPC to owner.
•Marshalling
•Client
has surrogate for o  concrete o in object table
20 February 2004
UKC, February 2004
10
P marshalls o to Q
P pushes o onto its stack;
sends w(o) to Q.
1. Q looks it up in its object table.
P
O
Q
Present: use the object
w(o)=NIL: surrogate being created; suspend.
2.
Absent: enter w(o)=NIL in object table;
send dirty(o) to owner(o);
3. Owner adds Q to its dirtySet(o) and dirty(o)
returns.
4. Q creates surrogate(o) and adds it to its object
table.
5. Q deletes surrogate(o) and sends clean(o) to
owner(o).
20 February 2004
UKC, February 2004
11
Dirty calls
A
B
copy
ack
dirty
{A,B}
{A}
20 February 2004
UKC, February 2004
Log
keep ref on stack
copy
dirty
ack
remove from stack
12
Weaknesses
Tightly bound to RPC
• Acknowledgement mechanism.
Implementation specific.
• Assumes method invocation pushes arguments onto stack;
• Unique surrogate per process (object-listing)
Under-specified
• Critical sections
• Race conditions
• Other scenarios
Informal proof
• Depends on hard-to-formalise aspects (e.g. stack)
20 February 2004
UKC, February 2004
14
Our contribution
Novel graphical notation.
Formalisation.
Discovered requirement for pivotal new states.
Proof.
20 February 2004
UKC, February 2004
15
New graphical notation
Intuitive.
Precise.
Uniformity of ‘direction’ of transitions.
‘Obvious’ where transitions are needed.
20 February 2004
UKC, February 2004
16
Lifecycle of references
GC unreachable
send copy
OK
RRAR
rcv Ack
ccitnil
ccit
Receive reference and
note the source
...
OK
1
2
3
dirty_ack from Owner
send copy_ack to Sender
nil
Receive reference and
note the source
nil
Obvious where transitions are needed
•E.g. Receive reference at state ccit.
•ccitnil critical for correctness.
20 February 2004
UKC, February 2004
17
Slicing
20 February 2004
UKC, February 2004
18
Fault tolerance
ccitu
Slicing
OK
• Owner is aware we
have a reference.
ccit
ccitl
nilu
OK
ccitnil
nil
nil
ccitnilu
nill
• Owner is not aware
we have a
reference.
ccitnill
20 February 2004
UKC, February 2004
19
Benefits
Intuitive – fault-tolerant version literally
encapsulates failure-free version.
Identify precisely when failures can be
detected.
Define states reached after failures detected.
Remedial actions.
20 February 2004
UKC, February 2004
20
Formalisation
Abstract machine
• Processes communicating by asynchronous
message passing.
• Atomic transitions involve 1 process at a time.
Receipt of message changes only a process’
internal state
• Trigger sending of a another message?
• Store some info in a to do table?
20 February 2004
UKC, February 2004
22
Benefits
Inputs and outputs desynchronised.
Size of critical sections explicit and minimised.
Asynchronous outputs (e.g. background
daemon processes to do tables).
Suitable for mechanical proof.
20 February 2004
UKC, February 2004
23
Formalisation
Rule name: guard  pseudo-statements.
name
guard
make_copy (p1,p2,r):
p1  p2  receive_T(p1,r)=OK  locallyReachable(p1,r)

table
{
id := new Identifier;
dirty_T(p1,r) := dirty_T(p1,r) U (p1,p2,id);
post(p1, p2, copy(r,id));
}
message
20 February 2004
UKC, February 2004
24
More formally
Tables defined as functions whose first argument is a process.
Channels are bags of messages between pairs of processes.
A configuration of the abstract machine is a tuple of all tables and
message channels.
Pseudo-statements act as configuration transformers:
• Given a configuration <…,table_T,…, k>,
• table_T(a0,…an):=V denotes <…,table_T',…,k> where
table_T'(x0,…xn) = table_T(a0,…an) if (x0,…xn) (a0,…an)
table_T'(a0,…an) = V
• post(p1,p2,m) denotes <…,table_T,…,k'> where
k'(p1,p2) = k(p1,p2)  {m}
k'(pi,pj) = k(pi,pj), (pi,pj)  (p1,p2)
20 February 2004
UKC, February 2004
25
Proof style
Safety & Liveness
Invariance-based proof
• Induction on length of transitions.
• Case analysis of transitions.
• Termination measure.
Benefits
• Systematic.
• Less error prone than temporal reasoning.
– E.g. establishing fine details such as mutual exclusivity complicated
in a formalism based on temporal reasoning.
20 February 2004
UKC, February 2004
26
Example proof
Lemma: For any processes p1, p2, for any reference r,
for any identifier id and for any configuration, the
following implication holds:
If <p1,p2,id>  dirty_T(p1,r) then receive_T (p1,r) = OK
Proof: In the initial configuration, dirty tables are empty
and the implication trivially holds.
We consider the four rules that add/remove entries
to/from dirty tables and that modify the content of
receive tables to/from OK.
• make_copy (p1, p2,r): make_copy adds an entry <p1,p2,id>, and
its guard ensures that the receive-table is in the OK state.
•…
20 February 2004
UKC, February 2004
27
Key Lemmas
Safety Lemma 1: Usable Reference For any processes p1 and p2, for any
reference r with p1=owner(r) and p1p2, and for any configuration, the
following implication holds:
If receive_T(p1,r)=OK, then p1  dirty_T(p2,r).
Safety Lemma 2: Reference in Transit For any processes p1, p2, for any
reference r, for any identifier id and for any configuration, the following
implication holds:
permanent
If copy(r,id)  k(p1,p2),
then p1  dirty_T(owner(r),r), if p1  owner(r)
temporary
or <owner(r),p2,id>  dirty_T(owner(r),r), if p1 = owner(t)
Safety Lemma 3: Unusable Reference For any process p1, for any
reference r and for any configuration, the following implication holds:
If receive_T(p1, r)=nil  receive_T(p1, r)=ccitnil,
then there exists p such that p  dirty_T(owner(r), r)
or there exist p,id such that <owner(r),p,id>  dirty_T(owner(r), r).
20 February 2004
UKC, February 2004
29
Birrell’s algorithm is Safe
A DGC algorithm is safe if the collector cannot reclaim live objects.
For Birrell's algorithm, there must be an entry in the owner's dirty
table for every live object.
Birrell's Safety Requirement For all references r, and for all
processes p1 and p2 and all identifiers id,
If receive t(p1,r)=OK  receive_T(p1,r)=nil
 receive_T(p1,r)=ccitnil  copy (r,id) k(p1,p2),
then there exists p such that p dirty_T(owner(r),r)
or there exist p,id such that <owner(r),p,id> dirty_T(owner(r),r).
The proof follows directly from the 3 safety lemmas.
20 February 2004
UKC, February 2004
30
Liveness
Liveness guarantees that if all references to an object are deleted,
the owner’s dirty table will eventually become empty.
To prove this,
• We show that whenever there’s a message in a channel, a
transition can be fired to consume it.
• We introduce a termination measure on the configurations that
shows how far the abstract machine is from completing, and show
that DGC transitions cause this measure to decrease.
• Hence all transition paths terminate.
20 February 2004
UKC, February 2004
31
Termination measures
termination_measure(c) = tab_measure
+    msg_measure(m)
+  rt_measure(receive_T(p,r))
size of
tables
messages between
pairs of processes
tab_measure = 9|dirty_call_todo_T| + 7|dirty_ack_todo_T|
+ 2|copy_ack_todo_T| + 2|clean_ack_todo_T|
+ 2|blocked_T|
and
rt_measure(OK)
=5
msg_measure(copy)
= 14
rt_measure(ccitnil) =2
msg_measure(dirty)
=8
rt_measure(ccit)
=1
msg_measure(dirtyack)
=6
rt_measure(nil)
=1
msg_measure(clean)
=3
rt_measure()
=0
msg_measure(copyack) =1
msg_measure(cleanack)= 1
states of
references
in processes
values chosen
‘arbitrarily’
20 February 2004
UKC, February 2004
32
Example: receive_dirty_ack
Dmeasure
= -1
receive_dirty_ack (p1, p2,r) :
dirtyack(r)  k (p1, p2)
{
receive(p1,p2,dirtyack(r));
//-6
copyack_todo_T(p2) := copyack_todo_T(p2)  blocked_T(p2,r); //-X
// Deserialisation code to be resumed for each entry in blocked_T(p2,r)
blocked_T(p2,r) :=  ;
receive_T(p2,r) := OK;
//-X
//+5
}
Thus, termination measure decreases by 1.
20 February 2004
UKC, February 2004
33
Optimisations
FIFO channels
• Less synchronisation needed
• Fewer messages: no clean_ack
• Fewer tables.
Sender is owner
• No need for dirty_call and copy_ack
• But need message ordering to avoid races
Receiver is owner
• Fewer dirty table entries
• Again need message ordering
20 February 2004
UKC, February 2004
34
Future work
Convince ourselves of appropriateness of Birrell’s
remedial actions.
Correctness proof of fault-free version.
Explore applicability of our techniques
• Graphical notation
• Proof-techniques
• Generality
Auto-generation of code from formalism.
20 February 2004
UKC, February 2004
35
Conclusion
Intuitive graphical notation.
Formal, implementation-independent specification
and proof of a widely used algorithm.
Discovered weaknesses in original presentation.
A widely applicable technique?
20 February 2004
UKC, February 2004
36
Questions?
FINIS
20 February 2004
UKC, February 2004
38