Programmable Matter
On The Path to
Programmable Matter
• A programmable
material…
• …with actuation and
sensing…
• …that can change its
physical properties …
• … under software
control…
• …and in reaction to
external stimuli
Seth Copen Goldstein
seth@cs.cmu.edu
And, lots of other people!
www.cs.cmu.edu/~claytronics
SenSys, 11/07
SenSys
© 2007 Goldstein
See video at: www.cs.cmu.edu/~claytronics/
1
SenSys
Types of “Programmable Matter”
Types of “Programmable Matter”
• Claytronics
• Sensor Network
• Claytronics
• Nanotechnology
© 2007 Goldstein
3
• Synthetic Biology
• FPGAs
Stoddart
Stoddart
• FPGAs
UT-Austin/UCSF
UT-Austin/UCSF
• Synthetic Biology
SenSys
Guestrin
Guestrin
• Sensor Network
Stoy
• Modular Robots
Stoy
• Modular Robots
2
© 2007 Goldstein
• Nanotechnology
SenSys
© 2007 Goldstein
4
Scaling
Scaling
• Goal: Form dynamic high-fidelity
macroscale objects
SenSys
• Goal: Form dynamic high-fidelity
macroscale objects
• High-fidelity ⇒ sub-millimeter units
• Macroscale ⇒ millions of units
• Scaling: Down in size
Up in number
• Method:
Ensemble
Effect
5
© 2007 Goldstein
SenSys
© 2007 Goldstein
Node Requirements
Challenges
Node
• Each node must have
–
–
–
–
Computation+Memory
Communication
Energy Storage
Sensing
6
– Actuation
– Adhesion
– Display
Hardware
• Yet, scaling demands nodes be:
Software
Ensemble
Design Space
– Simple
– Small
• Ensemble Axiom:
• Challenges are all intertwined
A node should include only the functionality
necessary to achieve the desired ensemble.
– Hardware/Software trade-offs
– Ensemble Axiom
• Example: Energy
SenSys
© 2007 Goldstein
7
SenSys
© 2007 Goldstein
8
Challenges
Node
Ensemble Effect & Motion
Ensemble
Hardware
Software
• Challenges are all intertwined
– Hardware/Software trade-offs
– Ensemble Axiom
• Example: Energy
SenSys
© 2007 Goldstein
9
SenSys
Ensemble Effect & Motion
10
© 2007 Goldstein
Building the Hardware
?
• Not if, but when
Goodness
Happy
HaHappy
pp
y
B’Day
B
‘D
B’day
ay
• Many approaches
– MEMS
– Bio
–?
Time
• Many challenges, but a clear path exists
SenSys
© 2007 Goldstein
11
SenSys
© 2007 Goldstein
12
Current Approach to <1mm
• How to form 3D
from a 2D process?
A potential approach
Silicon
Silicon Dioxide
Silicon
• How to form 3D
from a 2D process?
– begin with foundry
CMOS on SOI
SenSys
13
© 2007 Goldstein
SenSys
A potential approach
A potential approach
• How to form 3D
from a 2D process?
300 microns
– begin with foundry
CMOS on SOI
– pattern a flower
that includes
structure and
circuits
– lift off silicon layer
– begin with foundry
CMOS on SOI
– pattern a flower
that includes
structure and
circuits
– lift off silicon layer
• flexible
• harness stress to
form a sphere
• flexible
• harness stress to
form a sphere
© 2007 Goldstein
14
© 2007 Goldstein
15
SenSys
© 2007 Goldstein
300 microns
Reid, AFRL
• How to form 3D
from a 2D process?
SenSys
Reid, AFRL
– begin with foundry
CMOS on SOI
– pattern a flower
that includes
structure and
circuits
16
Initial Experiments
A sanity check
Computation Capability
1 mm diameter sphere
8086 Processor with
256KB memory
Mass < 1 mg
SOI-CMOS 90 nm process
with > 2M transistors.
Electrostatic Actuators
~5 body lengths / sec
Communication Capacitors
Power Storage
Power distribution
Supercapacitor stores enough
energy to execute over 200
million instructions or move 2
million body lengths
SenSys
Transmission of “energy
packets” using capacitive
coupling fills reservoir in
< 1μs.
17
© 2007 Goldstein
Sensor Networks
Rob Reid, AFRL
SenSys
© 2007 Goldstein
18
Sensor Networks as Programmable Matter
• Both have
Sense physical phenomena
–
–
–
–
Lots of nodes
• Sensor
nodeshard problem
BothNetworks
must tackle
– May be unattended for long periods of time
of programming
the ensemble
– Separate
from each other
– Severely energy constrained
Communicating with
each other
Compute something
• Programmable Matter
Result: an ensemble effect.
i.e., The whole is greater than the sum of the parts
SenSys
© 2007 Goldstein
19
Many nodes networked together
Nodes which can compute & store state
Sense the environment
Need uncertainty tolerance
– More nodes
– Mobile nodes (actuators!)
– Nodes in contact, often in a lattice
SenSys
© 2007 Goldstein
20
Two Broad Problem Areas
Thermodynamics for Computing
• Programming the Ensemble: How does
one think about coordination of millions
of elements?
• Programming the Unit: What is the
programming model for a (single)
element?
Traditional CS
Control each
unit’s actions
Control ensembles
of active units
101
100
• Let’s focus on the ensemble
Engineering Ensemble Effects
102
103
104
105
Control
global properties
of the aggregate
Phy/Chem/Econ
106
107
…
108 …
1024
1025
Number of Entities
Goal: Understand methods for
programming the ensemble as a whole.
SenSys
© 2007 Goldstein
21
SenSys
moveAround( X, Y, Point ) :neighbor( X, Y ),
brightness( X, N ),
brightness( Y, M ),
vacant( Y, Point ),
N <= M.
•Sensor calibration
– manufacturing and
deployment
introduces bias
– sensor cannot
remove bias
independently
• Attributes:
Ensemble level thinking
Concise understandable program
Scalable
Amenable to proof
Robust to failure and environmental uncertainty
SenSys
© 2007 Goldstein
22
Automatic Sensor Calibration
as Probabilistic Inference
Emergent Engineering
–
–
–
–
–
© 2007 Goldstein
23
•Probabilistic inference
for calibration:
– exploit correlations
between nearby sensors
– use probabilistic
graphical model to
represent correlation
Guestrin, CMU
SenSys
probability of bias
given sensor measurements of all nodes
probability of true, unbiased temperature
given sensor measurements of all nodes
© 2007 Goldstein
24
…
ISAT 2007
Specification
In detail for probabilistic inference…
Specification
1
3
environment description
how do
we connect?
message passing algorithm
on junction tree (data structure)
stable distributed junction tree
(even as link qualities change)
IJKO
RTUY
QTXY
SWZ
MNO
XYW
KLO
EFJK
OTY
Topological
physical
communication pattern
Geometric
SenSys
GHJO
Overlay
distributed data structure
optimizing the flow of information
Topological
combination
Logical
integrating and
summarizing information
Overlay
4
2
environment description
Logical
summary
query
query
Geometric
© 2007 Goldstein
25
Building and Maintaining a Real Overlay
Field deployment: 97 sensor
nodes deployed in office
building
SenSys
26
© 2007 Goldstein
A General Architecture
25 C
+2 C
True
Temp
Sensor
Bias
Noisy
Sensor
automatic
sensor calibration
13 messages per node
to form junction tree
(average)
building
automation
sensor field
monitoring
Logical
Overlay
GHJO
IJKO
RTUY
QTXY
SWZ
MNO
robust to node
and link failure
EFJK
Topological
Geometric
Initialization
Phase
© 2007 Goldstein
27
KLO
OTY
XYW
Cost
measures
27 C
SenSys
well-understood
in AI & machine learning
E3 abstraction
The E3 abstraction
Specification
S
Transforming Shapes
• Shape Change in Claytronics
• Requires:
High-level description of the Objective,
including goals, tasks and constraints.
Unified view of ensemble operations
Logical
–
–
–
–
Algorithmic Implementation of Specification
L
Distributed data structure
(integrity, consistency, access)
The distributed data structures necessary
to create ensemble effects from the
local communication relationships.
Overlay
O
• One example:
Communication resources available (~ IP layer)
Topology
T
Determine which robots can move
Keep modules connected
Parallel movement
Dealing with unit failure
– In C++ ~1000 lines
Communication-centric relationship between nodes.
Physical sensor node data (e.g., location, sensors)
Geometry
G
The primary interface to the physical world
And where the nodes are in the world.
SenSys
© 2007 Goldstein
29
SenSys
Morphing Ensembles of Robots
Morphing Ensembles of Robots
S
Transform shape of ensemble from into
target shape without disconnecting any
nodes.
S
Transform shape of ensemble from into
target shape without disconnecting any
nodes.
L
Move leaves towards root of their tree.
L
Move leaves towards root of their tree.
O
T
G
SenSys
Cover surface with forest of trees.
Roots are near target.
O
Links between adjacent neighbors.
Distinguish interior nodes from exterior
ones.
T
3D location in ensemble consistent
coordinate system.1
G
© 2007 Goldstein
31
30
© 2007 Goldstein
SenSys
Cover surface with forest of trees.
Roots are near target.
target
Links between adjacent neighbors.
Distinguish interior nodes from exterior
ones.
3D location in ensemble consistent
coordinate system.1
© 2007 Goldstein
32
Ensemble Compilation
Morphing Ensembles of Robots
S
Transform shape of ensemble from into
target shape without disconnecting any
nodes.
L
Move leaves towards root of their tree.
O
• Programs are 20x shorter
Understandable &
Proof friendly
• Ensemble Compilers are key:
• Where data lives
• Minimize communication
• Keep data consistent
Cover surface with forest of trees.
Roots are near target.
target
Runtime Messages
20000
T
G
• E3 Abstraction ⇒
Efficient ensembles 15000
Links between adjacent neighbors.
Distinguish interior nodes from exterior
ones.
C++
Meld
10000
3D location in ensemble consistent
coordinate system.1
5000
0
SenSys
© 2007 Goldstein
33
SenSys
The E3 abstraction exploits…
new
compiler
techniques
embed
algorithm onto
distributed
data structure
•Concise specifications
•Embarrassingly parallel
•Examples:
High-level description of the Objective,
including goals, tasks and constraints.
Logical
L
–
–
–
–
–
–
–
Algorithmic Implementation of Specification
Overlay
O
embed
overlay onto
available
communication
resources
The distributed data structures necessary
to create ensemble effects from the
local communication relationships.
Topology
T
Communication-centric
relationship between nodes.
SenSys
The primary interface to the physical world
And where the nodes are in the world.
© 2007 Goldstein
Morph
34
Amorphous computing [Nagpal]
Graph grammars [Klavins]
Programming work [Kod.]
CA+Gradients [Stoy]
Hole motion [DeRosa]
Boyd model [Boyd]
Turing stripes
Compile into
Global behavior
Local rules
Predict
•Goal:Compile Global specification into unit rules
Predict global behavior from set of unit rules
Geometry
G
Gradient Routing Localize
Global Behavior from local rules
Specification
S
© 2007 Goldstein
35
SenSys
© 2007 Goldstein
36
Thermodynamics for Computing
Programmable Matter
• Open up an entire new application space
–
–
–
–
–
–
Traditional CS
Control each
unit’s actions
Engineering Ensemble Effects
Control ensembles
of active units
101
100
102
103
104
105
Control
global properties
of the aggregate
Phy/Chem/Econ
106
107
…
108 …
1024
1025
Number of Entities
Goal: Understand methods for
programming the ensemble as a whole.
SenSys
© 2007 Goldstein
37
SenSys
© 2007 Goldstein
39
…
Antennas
Design
Entertainment
Interaction
Rescue
Metal Man
(Programmable Antennas)
(100x protein model)
(WoW in your living room)
(telepario)
(paramedic on demand)
(fault tolerant robotics)
• Vehicle for studying CS problem of the
future:
How do you design, program, maintain,
and use a billion component system?
SenSys
© 2007 Goldstein
38