It Ain’t the Meat, it’s the
Notion
Why Theory is Essential to Teaching
Concurrent Programming
Maurice Herlihy
Brown University
In many areas of CS
• Systems research fought all the hard
battles
– FORTRAN  complexity theory
– UDP/IP  network algorithms
• Theory researchers arrived later to
shoot the wounded
• But concurrency is different
Multicore Programming
Education
2
Vive la Différence
• Concurrent programmers are made,
not born (mostly)
• Students have little experience with
concurrency (today)
• Develop intuition by
– Idealized examples first
– Logical reasoning
Multicore Programming
Education
3
Homework Challenge
Devise Wait-Free 1Enqueuer, 1-Dequeuer
Buffer using only reads
& writes
Multicore Programming
Education
4
Another Homework Challenge
How about two
dequeuers?
Multicore Programming
Education
5
Shared Memory
Atomic reads and
writes to individual
variables
write
read
Multicore Programming
Education
6
Snapshot Challenge
Wait-Free
algorithm to read
multiple variables
atomically?
Read
both
Read
both
Multicore Programming
Education
7
Snapshot Challenge
Wait-Free
algorithm to
write multiple
variables
atomically?
write
both
write
both
Multicore Programming
Education
8
Bakery Algorithm
class Bakery implements Lock {
boolean[] flag;
6
Label[] label;
2
0
n-1
public Bakery (int n) {
flag = new boolean[n]; f f t f f t f f
label = new Label[n];
0 0 4 0 0 5 0 0
for (intfirst-comei = 0; i < n; i++) {
Classic,
firstflag[i]
served= false;
mutuallabel[i] = 0;
}
exclusion
algorithm
}
…
Critical Section
Multicore Programming
Education
9
Is Locked?
isLocked(
)
True means was locked at
some instant, false means
was free at some instant.
Multicore Programming
Education
Bakery Lock
10
I Can Has Lock?
tryLock()
Acquire lock if free &
return true, otherwise
immediately return false.
Multicore Programming
Education
Bakery Lock
11
Our Experience Teaching
Concurrency
• Brown (10 years)
• Tel-Aviv (10 years)
• Industry
Multicore Programming
Education
12
Mutual Exclusion
• Peterson, Bakery, space lower bounds
• Teaches
– Deadlock
– Livelock
– Fairness, first-come-first-served, …
Multicore Programming
Education
13
Linearizability
• How to describe a concurrent object?
– By “equivalence” to sequential
• What is a concurrent API?
• How do we reason about correctness?
– Even informally!
• Learn the rules before you break
them
Multicore Programming
Education
14
Consensus Numbers
• Certain synchronization primitives are
mathematically stronger than others
• No, you can’t build lock-free X from Y
– So do not waste your time trying
• Yes, you can build lock-free X from Z
– The rest is optimization
Multicore Programming
Education
15
Uh, Oh, Spin Locks
bad
Test&test&set
less bad
Test&set
• Inexplicable in
idealized model
• Well, kids …
– We didn’t actually
lie to you
– But the architects
gave us caches
– And you have to
outwit them …
Multicore Programming
Education
16
Then What?
• Implement simple T&S lock
– Dreadful performance
• Implement T&T&S lock
– Improved performance
Multicore Programming
Education
17
Homework Challenge
Devise Wait-Free 1Enqueuer, 1-Dequeuer
Buffer using only reads
& writes
Multicore Programming
Education
18
Homework Challenge
Multicore Programming
Education
19
Another Homework Challenge
How about two
dequeuers?
Multicore Programming
Education
20
Maybe?
“hence [any wait-free object] can be written
without a CAS/LL/SC. Instead LL/SC is used
to make it more efficient but LL/SC isn't
*required* to make it wait-free.”
Found in a blog
Multicore Programming
Education
21
Another Homework Challenge
Multicore Programming
Education
22
What If we have Test&Set?
Multicore Programming
Education
23
What If we have Test&Set?
Multicore Programming
Education
24
What If we have
Compare&Swap?
Multicore Programming
Education
25
What if We Have
Compare&Sswap?
Multicore Programming
Education
26
Snapshot Challenge
Wait-Free
algorithm to read
multiple variables
atomically?
Reads from
0 and 1
Reads from
1 and 2
Multicore Programming
Education
27
27
Snapshot Challenge
Wait-Free
algorithm to read
multiple variables
atomically?
Reads from
0 and 1
Reads from
1 and 2
Multicore Programming
Education
28
28
Multiple Assignment Challenge
Wait-Free
algorithm to
write multiple
variables
atomically?
Writes to 0
and 1
Writes to 1
and 2
Multicore Programming
Education
29
29
Multiple Assignment Challenge
Wait-Free
algorithm to
write multiple
variables
atomically?
Writes to 0
and 1
Writes to 1
and 2
Multicore Programming
Education
30
30
Is Locked?
isLocked(
)
True means was locked at
some instant, false means
was free at some instant.
Multicore Programming
Education
Bakery Lock
31
Is Locked?
isLocked(
)
True means was locked at
some instant, false means
was free at some instant.
Multicore Programming
Education
Bakery Lock
32
I Can Has Lock?
tryLock()
Acquire lock if free &
return true, otherwise
immediately return false.
Multicore Programming
Education
Bakery Lock
33
I Can Has Lock?
tryLock()
Acquire lock if free &
return true, otherwise
immediately return false.
Multicore Programming
Education
Bakery Lock
34
My Very Last Slide
• Teaching theory first
– Develops intuition
– Imposes order on chaos
• Idealized problems
– Prepare for realistic, complicated ones
• Impossibility results
– Save time and effort
Multicore Programming
Education
35