Set 6: Mutual Exclusion in Shared Memory
CPSC 668
DISTRIBUTED ALGORITHMS AND
SYSTEMS
CSCE 668
Fall 2011
Prof. Jennifer Welch
1
Shared Memory Model
2


Processors communicate via a set of shared
variables, instead of passing messages.
Each shared variable has a type, defining a set of
operations that can be performed atomically.
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Shared Memory Model Example
3
p0
read
p1
write
X
p2
read
write
Y
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Shared Memory Model
4

Changes to the model from the message-passing
case:
 no
inbuf and outbuf state components
 configuration includes a value for each shared variable
 only event type is a computation step by a processor
 An execution is admissible if every processor takes an
infinite number of steps
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Computation Step in Shared Memory
Model
5

When processor pi takes a step:
 pi
's state in old configuration specifies which shared
variable is to be accessed and with which operation
 operation is done: shared variable's value in the new
configuration changes according to the operation's
semantics
 pi 's state in new configuration changes according to its
old state and the result of the operation
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Observations on SM Model
6


Accesses to the shared variables are modeled as
occurring instantaneously (atomically) during a
computation step, one access per step
Definition of admissible execution implies
 asynchronous
 no
failures
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Mutual Exclusion (Mutex) Problem
7

Each processor's code is divided into four sections:
entry
remainder
critical
exit
entry: synchronize with others to ensure mutually exclusive
access to the …
 critical: use some resource; when done, enter the…
 exit: clean up; when done, enter the…
 remainder: not interested in using the resource

Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Mutual Exclusion Algorithms
8

A mutual exclusion algorithm specifies code for entry
and exit sections to ensure:
 mutual
exclusion: at most one processor is in its critical
section at any time, and
 some kind of "liveness" or "progress" condition. There
are three commonly considered ones…
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Mutex Progress Conditions
9




no deadlock: if a processor is in its entry section at
some time, then later some processor is in its critical
section
no lockout: if a processor is in its entry section at
some time, then later the same processor is in its
critical section
bounded waiting: no lockout + while a processor is in
its entry section, other processors enter the critical
section no more than a certain number of times.
These conditions are increasingly strong.
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Mutual Exclusion Algorithms
10

The code for the entry and exit sections is allowed
to assume that
 no
processor stays in its critical section forever
 shared variables used in the entry and exit sections are
not accessed during the critical and remainder sections
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Complexity Measure for Mutex
11


An important complexity measure for shared
memory mutex algorithms is amount of shared
space needed.
Space complexity is affected by:
 how
powerful is the type of the shared variables
 how strong is the progress property to be satisfied (no
deadlock vs. no lockout vs. bounded waiting)
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Mutex Results Using RMW
12

When using powerful shared variables of "readmodify-write" type
number of SM upper bound
lower bound
states
no deadlock
2
2
(test&set alg)
(obvious)
n/2 + c
(2n) (n/2)
(Burns et al.)
(Burns et al.)
bounded
n2
n
in Shared Memory CSCE 668
waiting Set 6: Mutual Exclusion
(queue
alg.) (Burns & Lynch)
no lockout
(memoryless)
Mutex Results Using Read/Write
13

When using read/write shared variables
number of
distinct vars.
no deadlock
no lockout
bounded
upper bound
3n booleans
(tournament
alg.)
2n
unbounded
Set 6: Mutual
Exclusion
in Shared Memory
lower bound
n
(Burns & Lynch)
CSCE 668
Test-and-Set Shared Variable
14

A test-and-set variable V holds two values, 0 or 1,
and supports two (atomic) operations:
 test&set(V):
temp := V
V := 1
return temp
 reset(V):
V := 0
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Mutex Algorithm Using Test&Set
15

code for entry section:
repeat
t := test&set(V)
until (t = 0)
An alternative construction is:
wait until test&set(V) = 0

code for exit section:
reset(V)
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Mutual Exclusion is Ensured
16

Suppose not. Consider first violation, when some
pi enters CS but another pj is already in CS
pj enters CS:
sees V = 0,
sets V to 1
pi enters CS:
sees V = 0,
sets V to 1
no node leaves CS so V stays 1
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
impossible!
No Deadlock
17

Claim: V = 0 iff no processor is in CS.
 Proof
is by induction on events in execution, and
relies on fact that mutual exclusion holds.

Suppose there is a time after which a processor is
in its entry section but no processor ever enters
CS.
no processor is in CS
V always equals 0, next t&s returns 0
proc enters CS, contradiction!
no processor enters CS
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
What About No Lockout?
18



One processor could always grab V (i.e., win the
test&set competition) and starve the others.
No Lockout does not hold.
Thus Bounded Waiting does not hold.
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Read-Modify-Write Shared Variable
19


The state of this kind of variable can be anything
and of any size.
Variable V supports the (atomic) operation
 rmw(V,f
), where f is any function
temp := V
V := f(V)
return temp

This variable type is so strong there is no point in
having multiple variables (from a theoretical
perspective).
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Mutex Algorithm Using RMW
20



Conceptually, the list of waiting processors is stored in
a circular queue of length n
Each waiting processor remembers in its local state its
location in the queue (instead of keeping this info in
the shared variable)
Shared RMW variable V keeps track of active part
of the queue with first and last pointers, which are
indices into the queue (between 0 and n-1)

so V has two components, first and last
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Conceptual Data Structure
21
The RMW shared object
just contains these two
"pointers"
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Mutex Algorithm Using RMW
22

Code for entry section:
// increment last to enqueue self
position := rmw(V,(V.first,V.last+1)
// wait until first equals this value
repeat
queue := rmw(V,V)
until (queue.first = position.last)

Code for exit section:
// dequeue self
rmw(V,(V.first+1,V.last))
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Correctness Sketch
23

Mutual Exclusion:
 Only
the processor at the head of the queue (V.first)
can enter the CS, and only one processor is at the head
at any time.

n-Bounded Waiting:
 FIFO
order of enqueueing, and fact that no processor
stays in CS forever, give this result.
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Space Complexity
24




The shared RMW variable V has two components in
its state, first and last.
Both are integers that take on values from 0 to n-1,
n different values.
The total number of different states of V thus is n2.
And thus the required size of V in bits is 2*log2 n .
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Spinning
25

A drawback of the RMW queue algorithm is that
processors in entry section repeatedly access the
same shared variable



called spinning
Having multiple processors spinning on the same
shared variable can be very time-inefficient in certain
multiprocessor architectures
Alter the queue algorithm so that each waiting
processor spins on a different shared variable
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
26
RMW Mutex Algorithm With
Separate Spinning

Shared RMW variables:
 Last
: corresponds to last "pointer" from
previous algorithm
 cycles
through 0 to n–1
 keeps track of index to be given to the next
processor that starts waiting
 initially 0
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
27
RMW Mutex Algorithm With
Separate Spinning

Shared RMW variables:
 Flags[0..n-1]
: array of binary variables
 these
are the variables that processors spin on
 make sure no two processors spin on the same
variable at the same time
 initially Flags[0] = 1 (proc "has lock") and
Flags[i] = 0 (proc "must wait") for i
>0
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Overview of Algorithm
28

entry section:
 get
next index from Last and store in a local variable
myPlace
 increment
Last (with wrap-around)
 spin
on Flags[myPlace] until it equals 1 (means proc
"has lock" and can enter CS)
 set Flags[myPlace] to 0 ("doesn't have lock")

exit section:
 set
Flags[myPlace+1] to 1 (i.e., give the priority to the
next proc)
 use
modular arithmetic to wrap around
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Question
29


Do the shared variables Last and Flags have to be
RMW variables?
Answer: The RMW semantics (atomically reading
and updating a variable) are needed for Last, to
make sure two processors don't get the same index
at overlapping times.
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Invariants of the Algorithm
30
1.
2.
3.
At most one element of Flags has value 1 ("has
lock")
If no element of Flags has value 1, then some
processor is in the CS.
If Flags[k] = 1, then exactly
(Last - k) mod n processors are in the entry
section, spinning on Flags[i], for i = k, (k+1) mod
n, …, (Last-1) mod n.
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Example of Invariant
31
0
Flags 0
1
2
3
4
5
6
7
0 1 0 0 0 0 0
Last 5
k = 2 and Last = 5.
So 5 - 2 = 3 procs are in entry,
spinning on Flags[2], Flags[3], Flags[4]
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Correctness
32

Those three invariants can be used to prove:
 Mutual
exclusion is satisfied
 n-Bounded Waiting is satisfied.
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
33
Lower Bound on Number of
Memory States
Theorem (4.4): Any mutex algorithm with k-bounded
waiting (and no-deadlock) uses at least n states of
shared memory.
Proof: Assume in contradiction there is an algorithm
using less than n states of shared memory.
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Lower Bound on Number of Memory
States
34

Consider this execution of the algorithm:
C
p0 p0 p0 …
C0
p0 in
CS by
ND

p1
C1
p1 in
entry
sec.
p2
C2 ……
pn-1
p2 in
entry
sec.
Cn-1
pn-1 in
entry
sec.
There exist i and j such that Ci and Cj have the
same state of shared memory.
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Lower Bound on Number of Memory
States
35
Ci
pi+1, pi+2, …, pj
p0 in CS,
p1-pi in entry,
rest in rem.
Cj
p0 in CS,
p1-pj in entry,
rest in rem.
 = sched. in which
p0-pi take steps in round robin
by ND, some ph
has entered CS
k+1 times
Set 6: Mutual Exclusion in Shared Memory

ph enters CS
k+1 times while
pi+1 is in entry
CSCE 668
Lower Bound on Number of Memory
States
36


But why does ph do the same thing when executing the
sequence of steps in  when starting from Cj as when
starting from Ci?
All the processes p0,…,pi do the same thing because:
they are in same states in the two configs
 shared memory state is same in the two configs
 only differences between Ci and Cj are (potentially) the
states of pi+1,…,pj and they don't take any steps in 

Set 6: Mutual Exclusion in Shared Memory
CSCE 668
Discussion of Lower Bound
37



The lower bound of n just shown on number of
memory states only holds for algorithms that must
provide bounded waiting in every execution.
Suppose we weaken the liveness condition to just nolockout in every execution: then the bound becomes
n/2 distinct shared memory states.
And if liveness is weakened to just no-deadlock in
every execution, then the bound is just 2.
Set 6: Mutual Exclusion in Shared Memory
CSCE 668
"Beating" the Lower Bound with
Randomization
38



An alternative way to weaken the requirement is to
give up on requiring liveness in every execution
Consider Probabilistic No-Lockout: every processor
has non-zero probability of succeeding each time it
is in its entry section.
Now there is an algorithm using O(1) states of
shared memory.
Set 6: Mutual Exclusion in Shared Memory
CSCE 668