Concurrent Computational
Systems
Edward A. Lee
Professor, UC Berkeley
Ptolemy Project
CHESS: Center for Hybrid and Embedded Software Systems
University of Arizona
Distinguished Lecture Series
January 13, 2005
Tuscon, Arizona
Abstract
Prevailing software abstractions are deeply rooted in the sequential
imperative model, which at its root, is poorly matched to concurrent
systems. Threads are a widely used natural extension of these
abstractions to concurrent problems. Semaphores and mutual
exclusion locks provide synchronization primitives offering some
control over the relative timing of actions in threads. In this talk, I will
argue that nontrivial concurrent programs based on threads,
semaphores, and mutual exclusion locks are incomprehensible. I will
outline a family of alternative concurrent models of computation, such
as discrete events, process networks, synchronous languages, and
dataflow, that promise more comprehensible programs. Most of these
have been around for some time, and some of them have achieved
widespread usage, but generally not in mainstream software. Indeed,
several of them offer promising views of "computation" in a broader
sense than the Turing-Church sense. They promise a view of
computation that embraces mixed hardware-software systems and
embedded systems that engage the physical world.
Lee, Berkeley 2
Standard Software Abstraction
(20-th Century Computation)
initial state
sequence
f : State  State
Alan Turing
final state
• Time is irrelevant
• All actions are ordered
Lee, Berkeley 3
Standard Software Abstraction:
Processes or Threads
Infinite sequences of state
transformations are called
“processes” or “threads”
The operating system
(typically) provides:
• suspend/resume
• mutual exclusion
• semaphores
suspend
resume
Lee, Berkeley 4
Standard Software Abstraction:
Concurrency via Interacting Threads
Potential for
race conditions,
deadlock, and
livelock
severely
compromises
software
reliability.
These methods
date back to the
1960’s
(Dijkstra).
stalled by precedence
race condition
stalled for rendezvous
Lee, Berkeley 5
A Stake in the Ground
Nontrivial concurrent programs based on
threads, semaphores, and mutexes are
incomprehensible to humans.

No amount of process improvement is going to
change this.
• the human brain doesn’t work this way.

Formal methods may help
• scalability?
• understandability?

Better concurrency abstractions will help more
Lee, Berkeley 6
A Story: Ptolemy Project Code Review
Lee, Berkeley 7
Ptolemy Project Code Review
A Typical Story

Code review discovers that a method needs to be synchronized to
ensure that multiple threads do not reverse each other’s actions.
No problems had been detected in 4 years of using the code.
Three days after making the change, users started reporting deadlocks
caused by the new mutex.
Analysis and correction of the deadlock is hard.

But code review successfully identified the flaw.



Lee, Berkeley 8
Code Review Doesn’t Always Work
Another Typical Story
/**
CrossRefList is a list that maintains pointers to other CrossRefLists.
…
@author Geroncio Galicia, Contributor: Edward A. Lee
@version $Id: CrossRefList.java,v 1.78 2004/04/29 14:50:00 eal Exp $
@since Ptolemy II 0.2
@Pt.ProposedRating Green (eal)
@Pt.AcceptedRating Green (bart)
Code that had been in
*/
public final class CrossRefList implements Serializable {
use for four years,
…
central to Ptolemy II,
protected class CrossRef implements Serializable{
with an extensive test
…
// NOTE: It is essential that this method not be
suite, design reviewed to
// synchronized, since it is called by _farContainer(),
yellow, then code
// which is. Having it synchronized can lead to
reviewed to green in
// deadlock. Fortunately, it is an atomic action,
// so it need not be synchronized.
2000, causes a deadlock
private Object _nearContainer() {
during a demo on April
return _container;
26, 2004.
}
private synchronized Object _farContainer() {
if (_far != null) return _far._nearContainer();
else return null;
}
…
}
}
Lee, Berkeley 9
And Doubts Remain
/**
CrossRefList is a list that maintains pointers to other CrossRefLists.
…
@author Geroncio Galicia, Contributor: Edward A. Lee
@version $Id: CrossRefList.java,v 1.78 2004/04/29 14:50:00 eal Exp $
@since Ptolemy II 0.2
@Pt.ProposedRating Green (eal)
@Pt.AcceptedRating Green (bart)
Safety of this code
*/
depends on policies
public final class CrossRefList implements Serializable {
…
maintained by entirely
protected class CrossRef implements Serializable{
unconnected classes.
…
The language and
private synchronized void _dissociate() {
_unlink(); // Remove this.
synchronization
// NOTE: Deadlock risk here! If _far is waiting
mechanisms provide no
// on a lock to this CrossRef, then we will get
way to talk about these
// deadlock. However, this will only happen if
// we have two threads simultaneously modifying a
systemwide properties.
// model. At the moment (4/29/04), we have no
// mechanism for doing that without first
// acquiring write permission the workspace().
// Two threads cannot simultaneously hold that
// write access.
if (_far != null) _far._unlink(); // Remove far
}
}
Lee, Berkeley 10
Image “borrowed” from an Iomega advertisement for Y2K
software and disk drives, Scientific American, September 1999.
What it Feels Like to Use the synchronized
Keyword in Java
Lee, Berkeley 11
Diagnosis: Interacting Processes are Not
Compositional
An aggregation
of processes is
not a process (a
total order of
external
interactions).
What is it?
Many software
failures are due
to this illdefined
composition.
Lee, Berkeley 12
Distributed Version of
20-th Century Computation
Force-fitting the
sequential
abstraction onto
parallel
hardware.
remote procedure call
Lee, Berkeley 13
Combining Processes and RPC –
Split-Phase Execution, Futures,
Asynchronous Method Calls, Callbacks, …
These methods
are at least as
incomprehensible
as concurrent
threads or
processes.
“asynchronous”
procedure call
Lee, Berkeley 14
Better Concurrency Models
Better concurrency models exist.
We examine a family of such models that we call
actor-oriented models.
Lee, Berkeley 15
What is an Actor-Oriented MoC?
Traditional component interactions:
class name
What flows through
an object is
sequential control
data
methods
call
return
Actor oriented:
actor name
data (state)
parameters
ports
Input data
Output data
What flows through
an object is
streams of data
Lee, Berkeley 16
Ptolemy II: Framework for Experimenting with
Actor-Oriented Models of Computation
Basic Ptolemy II infrastructure:
Large, domain-polymorphic
component library.
Director from a library
defines component
interaction semantics
Type system
for transported
data
Visual editor supporting an abstract syntax
Lee, Berkeley 17
The Basic Abstract Syntax
connection
Actor
Actor
Relation
Link
Port
Link
Port
Attributes
ec
nn
co
n
tio
ec
nn
Link
co
tio
n
Attributes
Port
Actor
Attributes
• Actors
• Attributes on actors (parameters)
• Ports in actors
• Links between ports
• Width on links (channels)
• Hierarchy
Concrete syntaxes:
• XML
• Visual pictures
• Actor languages (Cal, StreamIT, …)
Lee, Berkeley 18
Hierarchy - Composite Components
Relation
dangling
Port
Actor
opaque Port
Port
transparent or opaque
CompositeActor
toplevel CompositeActor
Lee, Berkeley 19
Abstract Semantics
of Actor-Oriented Models of Computation
Actor-Oriented Models of
Computation that we have
implemented:
execution control
data transport
Basic Transport:
receiver.put(t)
send(0,t)
init()
fire()
get(0)
P2
P1
E1
R1
token t
IOPort
IORelation
Actor
E2
Receiver
(inside port)
• dataflow (several variants)
• process networks
• distributed process networks
• Click (push/pull)
• continuous-time
• CSP (rendezvous)
• discrete events
• distributed discrete events
• synchronous/reactive
• time-driven (several variants)
•…
Lee, Berkeley 20
Models of Computation
Implemented in Ptolemy II
















CI – Push/pull component interaction
Click – Push/pull with method invocation
CSP – concurrent threads with rendezvous
CT – continuous-time modeling
DE – discrete-event systems
DDE – distributed discrete events
DDF – Dynamic dataflow
DPN – distributed process networks
DT – discrete time (cycle driven)
FSM – finite state machines
Giotto – synchronous periodic
GR – 2-D and 3-D graphics
PN – process networks
SDF – synchronous dataflow
SR – synchronous/reactive
TM – timed multitasking
Most of
these are
actor
oriented.
Lee, Berkeley 21
Models of Computation
Implemented in Ptolemy II
















CI – Push/pull component interaction
Click – Push/pull with method invocation
CSP – concurrent threads with rendezvous
CT – continuous-time modeling
DE – discrete-event systems
DDE – distributed discrete events
DDF – Dynamic dataflow
DPN – distributed process networks
DT – discrete time (cycle driven)
FSM – finite state machines
Giotto – synchronous periodic
GR – 2-D and 3-D graphics
PN – process networks
SDF – synchronous dataflow
SR – synchronous/reactive
TM – timed multitasking
Lee, Berkeley 22
Example Model of Computation:
Discrete Events (DE)
DE Director implements
timed semantics using an
event queue
Reactive actors
Event source
Signal
Time line
Lee, Berkeley 23
Semantics Clears Up Subtleties:
E.g. Simultaneous Events
By default, an actor produces events with the same time as the input
event. But in this example, we expect (and need) for the BooleanSwitch to
“see” the output of the Bernoulli in the same “firing” where it sees the event
from the PoissonClock. Events with identical time stamps are also ordered,
and reactions to such events follow data precedence order.
Lee, Berkeley 24
Semantics Clears Up Subtleties:
E.g. Feedback
Data precedence analysis has to take into account the non-strictness of
this actor (that an output can be produced despite the lack of an input).
Lee, Berkeley 25
Semantics Clears Up Subtleties:
E.g. Zeno Systems
DE systems may have
an infinite number of
events in a finite amount
of time. Carefully
constructed semantics
gives these systems
meaning.
Lee, Berkeley 26
A Rough Sense of Discrete-Event
Semantics: Metric Space Formulation
Cantor metric:

d ( x, y)  1 / 2
where  is the earliest time where x and y differ.
The set of signals
with this metric form
a complete metric
space.
x
y

Generalizations of this metric handle
multiple events at the same time. Lee, Berkeley 27
Causality in DE
Causal:
d ( y, y)  d ( x, x)
Strictly causal:
x
y
x
y
d ( y, y)  d ( x, x)
Delta causal:
  1,
d ( y, y)   d ( x, x)
A delta-causal component is a “contraction map.”
Lee, Berkeley 28
Fixed Point Theorem
(Banach Fixed Point Theorem)
Let (S n = [T  V ]n, d ) be a complete metric space and f :
S n  S n be a delta causal function. Then f has a unique
fixed point, and for any point s  S n , the following
sequence converges to that fixed point:
s1 = s, s2 = f (s1), s3 = f (s2), …
This means no Zeno!
Current work: Other formulations (using generalized
ultrametric spaces, ordinals, and posets) give meaning to
a broader class of systems.
Lee, Berkeley 29
Semantic Challenges
Create distributed discrete-event systems.
Three families of techniques are known:



Conservative techniques (Chandy & Misra)
Optimistic techniques (Time warp, Jefferson)
Distributed consensus (Croquet, Reed)
Lee, Berkeley 30
Application of DE Modeling
Wireless Sensor Nets in VisualSense
VisualSense extends
the Ptolemy II discreteevent domain with
communication between
actors representing
sensor nodes being
mediated by a channel,
which is another actor.
The example at the left
shows a grid of nodes
that relay messages
from an initiator (center)
via a channel that
models a low (but nonzero) probability of long
range links being viable.
Lee, Berkeley 31
Models of Computation
Implemented in Ptolemy II
















CI – Push/pull component interaction
Click – Push/pull with method invocation
CSP – concurrent threads with rendezvous
CT – continuous-time modeling
DE – discrete-event systems
DDE – distributed discrete events
DDF – Dynamic dataflow
DPN – distributed process networks
DT – discrete time (cycle driven)
FSM – finite state machines
Giotto – synchronous periodic
GR – 2-D and 3-D graphics
PN – process networks
SDF – synchronous dataflow
SR – synchronous/reactive
TM – timed multitasking
Lee, Berkeley 32
Example Model of Computation
Continuous Time (CT)
Director implements a “solver” that constructs an
approximation to the continuous-time behavior.
A signal has a value at
all real-valued times.
Integrator used to define
systems governed by ordinary
differential equations.
Lee, Berkeley 33
Models of Computation
Implemented in Ptolemy II
















CI – Push/pull component interaction
Click – Push/pull with method invocation
CSP – concurrent threads with rendezvous
CT – continuous-time modeling
DE – discrete-event systems
DDE – distributed discrete events
DDF – Dynamic dataflow
DPN – distributed process networks
DT – discrete time (cycle driven)
FSM – finite state machines
Giotto – synchronous periodic
GR – 2-D and 3-D graphics
PN – process networks
SDF – synchronous dataflow
SR – synchronous/reactive
TM – timed multitasking
Lee, Berkeley 34
Heterogeneous Models
Hybrid Systems: CT + FSM
The FSM
director can be
combined with
other directors
to create
modal models.
Lee, Berkeley 35
Semantic Subtleties
In hybrid systems, signals may have more than
one value at a given time.



Discontinuities
Transient states
Traditional mathematical formulation of a
signal:
x: Reals  Realsn
has to be modified:
x: Reals  Naturals  Realsn
with ensuing consequences for the theory.
Lee, Berkeley 36
Models of Computation
Implemented in Ptolemy II
















CI – Push/pull component interaction
Click – Push/pull with method invocation
CSP – concurrent threads with rendezvous
CT – continuous-time modeling
DE – discrete-event systems
DDE – distributed discrete events
DDF – Dynamic dataflow
DPN – distributed process networks
DT – discrete time (cycle driven)
FSM – finite state machines
Giotto – synchronous periodic
GR – 2-D and 3-D graphics
PN – process networks
SDF – synchronous dataflow
SR – synchronous/reactive
TM – timed multitasking
Lee, Berkeley 37
Heterogeneous Models
Mixed Signals: DE + CT
DE signal
DE model of a digital controller
CT signal
CT model of mechanical system
Lee, Berkeley 38
Models of Computation
Implemented in Ptolemy II
















CI – Push/pull component interaction
Click – Push/pull with method invocation
CSP – concurrent threads with rendezvous
CT – continuous-time modeling
DE – discrete-event systems
DDE – distributed discrete events
DDF – Dynamic dataflow
DPN – distributed process networks
DT – discrete time (cycle driven)
FSM – finite state machines
Giotto – synchronous periodic
GR – 2-D and 3-D graphics
PN – process networks
SDF – synchronous dataflow
SR – synchronous/reactive
TM – timed multitasking
Lee, Berkeley 39
Example Untimed Model of Computation:
Process Networks (PN)
actor == thread
signal == stream
reads block
Kahn, MacQueen, 1977
writes don’t
Lee, Berkeley 40
PN Semantics




x
F (x )
y
F ( y)
A signal is a sequence of values
Define a prefix order:
x
y
means that x is a prefix of y.
Actors are monotonic functions:
x
y  F(x) F(y)
Stronger condition: Actors are continuous
functions (intuitively: they don’t wait forever to
produce outputs).
Lee, Berkeley 41
PN Semantics of Composition (Kahn, ’74)
If the components
are deterministic,
the composition is
deterministic.
x y 
F ( x)  x
x
F
y
Knaster-Tarski fixed point theorem:
• Continuous function has a unique least fixed point
• Execution procedure for finding that fixed point
• Successive approximations to the fixed point
Lee, Berkeley 42
Semantic Subtleties

Giving semantics to finite sequences in
combination with infinite sequences.

Unifying fixed-point semantics with metricspace semantics (in order to give meaning to
heterogeneous models that combine timed
with untimed systems).
Lee, Berkeley 43
Application of PN and Ptolemy II
Kepler – Scientific Workflows
Led by San Diego Supercomputer Center (SDSC), with
participation from the SEEK ITR, DOE SciDAC SDM, and
others, researchers are building on Ptolemy II a
distributed workflow system called Kepler to “provide
access to any resource, any data, any service, anytime,
anywhere” via a distributed computing platform (aka the
“Grid”) supporting “high performance computing, and
federated, integrated, mediated databases within a userfriendly workbench / problem-solving environment.”
This effort is compatible with the objectives of NSF
Cyberinfrastructure, UK’s e-Science Programme, and
DOE’s SciDAC (Scientific Discovery through Advanced
Computing).
Lee, Berkeley 44
Kepler Example:
Harvesting Web Services
Example showing a web service
wrapper (Thanks to Bertram
Ludaecher, San Diego
Supercomputer Center)
Lee, Berkeley 45
Kepler Example:
Grid Computations
Thanks to
Bertram
Ludäscher
SDSC
Lee, Berkeley 46
Kepler Example:
Real-Time Data Feeds
Thanks to Bertram
Ludäscher, SDSC
ORB
This model
integrates realtime data feeds
from a ship (the
Roger Revelle)
Lee, Berkeley 47
KEPLER/CSP: Contributors, Sponsors, Projects
(or loosely coupled Communicating Sequential Persons ;-)
Ilkay Altintas SDM, Resurgence
Kim Baldridge Resurgence, NMI
Chad Berkley SEEK
Shawn Bowers SEEK
Terence Critchlow SDM
Tobin Fricke ROADNet
Jeffrey Grethe BIRN
Christopher H. Brooks Ptolemy II
Zhengang Cheng SDM
Dan Higgins SEEK
Efrat Jaeger GEON
Matt Jones SEEK
Werner Krebs, EOL
Edward A. Lee Ptolemy II
Kai Lin GEON
Bertram Ludaescher SEEK, GEON, SDM, BIRN, ROADNet
Mark Miller EOL
Steve Mock NMI
Steve Neuendorffer Ptolemy II
Jing Tao SEEK
Mladen Vouk SDM
Xiaowen Xin SDM
Yang Zhao Ptolemy II
Bing Zhu SEEK
•••
Thanks to Bertram Ludäscher, SDSC
Ptolemy II
Lee, Berkeley 48
Models of Computation
Implemented in Ptolemy II
















CI – Push/pull component interaction
Click – Push/pull with method invocation
CSP – concurrent threads with rendezvous
CT – continuous-time modeling
DE – discrete-event systems
DDE – distributed discrete events
DDF – Dynamic dataflow
DPN – distributed process networks
DT – discrete time (cycle driven)
FSM – finite state machines
Giotto – synchronous periodic
GR – 2-D and 3-D graphics
PN – process networks
SDF – synchronous dataflow
SR – synchronous/reactive
TM – timed multitasking
Lee, Berkeley 49
Synchronous Models of Computation
Director finds a value (or absent) at each “tick” of a global clock
Feedback is resolved by
finding a fixed point.
Signal has a value or is absent
at each tick of the “clock.”
Semantic foundation based on
Kanster-Tarski fixed point
theorem on Scott topologies.
Lee, Berkeley 50
Languages Based on the
Synchronous Model of Computation






Lustre (and SCADE)
Esterel
Signal
Statecharts (and UML state machines)
Argos
…
Lee, Berkeley 51
Models of Computation
Implemented in Ptolemy II
















CI – Push/pull component interaction
Click – Push/pull with method invocation
CSP – concurrent threads with rendezvous
CT – continuous-time modeling
DE – discrete-event systems
DDE – distributed discrete events
DDF – Dynamic dataflow
DPN – distributed process networks
DT – discrete time (cycle driven)
FSM – finite state machines
Giotto – synchronous periodic
GR – 2-D and 3-D graphics
PN – process networks
SDF – synchronous dataflow
SR – synchronous/reactive
TM – timed multitasking
Lee, Berkeley 52
Dataflow Models of Computation

















Computation graphs [Karp & Miller - 1966]
Process networks [Kahn - 1974]
Static dataflow [Dennis - 1974]
Dynamic dataflow [Arvind, 1981]
K-bounded loops [Culler, 1986]
Synchronous dataflow [Lee & Messerschmitt, 1986]
Structured dataflow [Kodosky, 1986]
PGM: Processing Graph Method [Kaplan, 1987]
Synchronous languages [Lustre, Signal, 1980’s]
Well-behaved dataflow [Gao, 1992]
Boolean dataflow [Buck and Lee, 1993]
Multidimensional SDF [Lee, 1993]
Cyclo-static dataflow [Lauwereins, 1994]
Integer dataflow [Buck, 1994]
Bounded dynamic dataflow [Lee and Parks, 1995]
Heterochronous dataflow [Girault, Lee, & Lee, 1997]
…
Many tools,
software
frameworks,
and hardware
architectures
have been
built to support
one or more of
these.
Lee, Berkeley 53
Ptolemy II Software Architecture
Built for Extensibility
Ptolemy II packages
have carefully
constructed
dependencies and
interfaces
Graph
Data
Kernel
Actor
Math
Lee, Berkeley 54
Conclusion







Threads are a poor concurrent MoC
There are many better concurrent MoCs
The ones we know are the tip of the iceberg
Ptolemy II is a lab for experimenting with them
Specializing MoCs can be useful
Mixing specialized MoCs can be useful.
This is a rich research area.
Lee, Berkeley 55