Universal laws and architecture:
Foundations for Sustainable Infrastructure
John Doyle
John G Braun Professor
Control and Dynamical Systems, EE, BE
Caltech
“Universal laws and architectures?”
•
•
•
•
•
Universal “conservation laws” (constraints)
Universal architectures (constraints that deconstrain)
Mention recent papers*
Focus on broader context not in papers
Lots of case studies for motivation
*try to get you
to read them?
A rant
• Lots from cell biology
– glycolytic oscillations for hard limits
– bacterial layering for architecture
• Networking and “clean slate” architectures
–
–
–
–
–
wireless end systems
info or content centric application layer
integrate routing, control, scheduling, coding, caching
control of cyber-physical
PC, OS, VLSI, antennas, etc (IT components)
•
•
•
•
Neuroscience
Smartgrid, cyber-phys
Wildfire ecology
Medical physiology
•
•
•
•
Earthquakes
Lots of aerospace
Physics: turbulence, stat mech (QM?)
“Toy”: Lego, clothing, buildings, …
my case
studies
• Lots from cell biology
– glycolytic oscillations for hard limits
– bacterial layering for architecture
• Networking and “clean slate” architectures
–
–
–
–
–
wireless end systems
info or content centric application layer
integrate routing, control, scheduling, coding, caching
control of cyber-physical
PC, OS, VLSI, antennas, etc (IT components)
•
•
•
•
Neuroscience
Smartgrid, cyber-phys
Wildfire ecology
Medical physiology
•
•
•
•
Earthquakes
Lots of aerospace
Physics: turbulence, stat mech (QM?)
“Toy”: Lego, clothing, buildings, …
my case
studies
Want to understand the space of
systems/architectures
Some old stuff
to warm up
fragile
robust
Want robust and
efficient systems
and architectures
efficient
wasteful
Requirements on systems and architectures
accessible
accountable
accurate
adaptable
administrable
affordable
auditable
autonomy
available
credible
process
capable
compatible
composable
configurable
correctness
customizable
debugable
degradable
determinable
demonstrable
dependable
deployable
discoverable
distributable
durable
effective
efficient
evolvable
extensible
failure
transparent
fault-tolerant
fidelity
flexible
inspectable
installable
Integrity
interchangeable
interoperable
learnable
maintainable
manageable
mobile
modifiable
modular
nomadic
operable
orthogonality
portable
precision
predictable
producible
provable
recoverable
relevant
reliable
repeatable
reproducible
resilient
responsive
reusable
robust
safety
scalable
seamless
self-sustainable
serviceable
supportable
securable
simplicity
stable
standards
compliant
survivable
sustainable
tailorable
testable
timely
traceable
ubiquitous
understandable
upgradable
usable
Simplified, minimal requirements
accessible
accountable
accurate
adaptable
administrable
affordable
auditable
autonomy
available
credible
process
capable
compatible
composable
configurable
correctness
customizable
debugable
degradable
determinable
demonstrable
dependable
deployable
discoverable
distributable
durable
effective
manageable
mobile
modifiable
modular
nomadic
operable
efficient
orthogonality
evolvable
portable
extensible
precision
failure
predictable
transparent
producible
fault-tolerant
provable
fidelity
recoverable
flexible
relevant
inspectable
reliable
installable
repeatable
Integrity
reproducible
interchangeable resilient
interoperable
responsive
learnable
reusable
maintainable
robust
safety
scalable
seamless
self-sustainable
serviceable
supportable
securable
simple
stable
standards
compliant
survivable
sustainable
tailorable
testable
timely
traceable
ubiquitous
understandable
upgradable
usable
Requirements on systems and architectures
accessible
accountable
accurate
adaptable
administrable
affordable
auditable
autonomy
available
credible
process
capable
compatible
composable
configurable
correctness
customizable
debugable
degradable
determinable
demonstrable
dependable
deployable
discoverable
distributable
durable
effective
manageable
safety
mobile
scalable
modifiable
seamless
modular
self-sustainable
nomadic
serviceable
operable
supportable
orthogonality securable
efficient
evolvable
portable
simple
extensible
precision
stable
failure
predictable
standards
transparent
producible
compliant
fault-tolerant
provable
survivable
fidelity
recoverable sustainable
flexible
relevant
tailorable
inspectable
reliable
testable
installable
repeatable
timely
Integrity
reproducible traceable
interchangeable resilient
ubiquitous
interoperable
responsive
understandable
learnable
reusable
upgradable
maintainable
usable
robust
Requirements on systems and architectures
efficient
simple
sustainable
robust
Requirements on systems and architectures
efficient
simple
sustainable
robust
Requirements on systems and architectures
fragile
sustainable
robust
simple
complex
efficient
wasteful
Requirements on systems and architectures
sustainable
fragile
complex
simple
robust
efficient
wasteful
Want to understand the space of
systems/architectures
Case studies?
fragile
Strategies?
Architectures?
robust
Want robust and
efficient systems
and architectures
efficient
wasteful
Future evolution of the “smart” grid?
fragile
Future?
Now
robust
efficient
wasteful
Current
Technology?
fragile
At best we
get one
robust
efficient
wasteful
???
fragile
Often
neither
robust
efficient
wasteful
???
Bad
architectures?
fragile
?
gap?
Bad
theory?
robust
efficient
?
wasteful
Control, OR
Bode
Nash
Von
Neumann
Comms
Kalman
Pontryagin
Shannon
Theory?
Deep, but fragmented,
incoherent, incomplete
Carnot
Boltzmann
Turing
Godel
Compute
Heisenberg
Einstein
Physics
• Turing 100th birthday in 2012
• Turing
− machine (math, CS)
− test (AI, neuroscience)
− pattern (biology)
• Arguably greatest*
− all time math/engineering combination
− WW2 hero
− “invented” software
Turing (1912-1954)
Compute
*Also world-class runner.
Key papers/results
• Theory (1936): Turing machine (TM), computability,
(un)decidability, universal machine (UTM)
• Practical design (early 1940s): code-breaking, including
the design of code-breaking machines
• Practical design (late 1940s): general purpose digital
computers and software, layered architecture
• Theory (1950): Turing test for machine intelligence
• Theory (1952): Reaction diffusion model of
morphogenesis, plus practical use of digital computers
to simulate biochemical reactions
Control
Comms
Shannon
Bode
fragile?
?
slow?
•
•
•
•
•
Each theory  one dimension
Tradeoffs across dimensions
Assume architectures a priori
Progress is encouraging, but…
Stovepipes are an obstacle…
wasteful?
Carnot
Boltzmann
Turing
Godel
Compute
Heisenberg
Einstein
Physics
Cyberphysical theories
Cyber (digital)
Physical (analog)
• Turing computation (time)
• Shannon compression
(space)
• Content centric nets (time,
space, location)
•
•
•
•
Bode (latency)
Shannon (channels)
Networked control (AndyL)
Redo StatMech and
efficiency
Lots of challenges not yet addressed
(e.g. Smartgrid, biology, neuro,..)
Layering as optimization?
Turing as
“new”
starting
point?
Software
Hardware
Digital
Analog
Essentials:
0. Model
1. Universal laws
2. Universal architecture
3. Practical implementation
Turing’s 3 step research:
0. Virtual (TM) machines
1. hard limits, (un)decidability
using standard model (TM)
2. Universal architecture
achieving hard limits (UTM)
3. Practical implementation in
digital electronics (biology?)
• …being digital should be of greater interest than
that of being electronic. That it is electronic is
certainly important because these machines owe
their high speed to this… But this is virtually all that
there is to be said on that subject.
• That the machine is digital however has more
subtle significance. … One can therefore work to
any desired degree of accuracy.
• This accuracy is not obtained by more careful
machining of parts, control of temperature
variations, and such means, but by a slight increase
in the amount of equipment in the machine
1947 Lecture to LMS
Software
Hardware
Digital
Analog
• … quite small errors in the initial conditions can
have an overwhelming effect at a later time. The
displacement of a single electron by a billionth of
a centimetre at one moment might make the
difference between a man being killed by an
avalanche a year later, or escaping.
• It is an essential property of the mechanical
systems which we have called 'discrete state
machines' that this phenomenon does not occur.
• Even when we consider the actual physical
machines instead of the idealised machines,
reasonably accurate knowledge of the state at
one moment yields reasonably accurate
knowledge any number of steps later.
1950, Computing Machinery and Intelligence,
Mind
The 'skin of an onion' analogy is also helpful. In
considering the functions of the mind or the brain we
find certain operations which we can explain in
purely mechanical terms. This we say does not
correspond to the real mind: it is a sort of skin which
we must strip off if we are to find the real mind. But
then in what remains we find a further skin to be
stripped off, and so on. Proceeding in this way do
we ever come to the 'real' mind, or do we eventually
come to the skin which has nothing in it? In the latter
case the whole mind is mechanical.
1950, Computing Machinery and Intelligence, Mind
Cyberphysical
Computers
Starting point
Software
Software
Hardware
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
Hardware
Digital
Analog
Actuators/
sensors/
amplifiers
The
“plant”
Cyberphysical
Actuators/
sensors/
amplifiers
Software
Hardware
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Computers
Computers
Virtual
net
Comms
net
Physical
net
Software
Hardware
Actuators/
sensors/
amplifiers
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
-physical
Current power grid gets plug and play
stability and robustness from passive, lossy
networks and centralized generation.
All of this will change.
The
“plant”
Physical
net
The
“plant”
CyberComputers
Computers
Software
Hardware
Digital
Analog
Virtual
net
Digital
Analog
Active
Active
Lumped
Distribute
Software
Hardware
Comms
net
Lumped
Distribute
Internet gets p&p from protocols.
Packets are easy compared with physical flows.
Cyberphysical
Actuators/
sensors/
amplifiers
Computers
Software
Hardware
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Virtual
net
Comms
net
Physical
net
Can we design
protocol-based
control of
CPS systems?
Cyberphysical
Computers
Starting point
Software
Software
Hardware
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
Hardware
Digital
Analog
Actuators/
sensors/
amplifiers
The
“plant”
Cyber focus
Starting point
Software
Hardware
Digital
Analog
Start from here with Turing’s 3 step
research:
1. (universal) hard limits, (un)decidability
using standard model (TM)
2. Universal architecture achieving hard
limits (UTM)
3. Practical implementation in digital
electronics ( and biology?)
Abstractions
Want to specialize 3 notions of abstraction:
• Layer
• Virtual
• Universal (laws, architectures, implementation)
(see Rant, not sure about best terminology)
Cyber focus
Virtual
net
Software
Hardware
Digital
Analog
Active
Comms
net
Lumped
Distribute
Steps (mimicking Turing)
• Turing (time, delay only)
• Space, storage,
compression ( Shannon)
• Layering as utility max
(time, space)
Tradeoffs
• Time (latency/delay in
comp and comm)
• Space (storage and
location)
-physical focus
Controller/
actuators/
sensors/
amplifiers
Controller/
actuators/
sensors/
amplifiers
Active
Virtual
net
Active
Lumped
Distribute
Comms
net
Lumped
Distribute
The
“plant”
Physical
net
The
“plant”
-physical focus
Controller/
actuators/
sensors/
amplifiers
Active
Lumped
The
“plant”
Start from Bode:
1. Hard limits (noise, delay)
2. Architecture (filter+control)
3. Practical implementation issues
-physical focus
Controller/
actuators/
sensors/
amplifiers
Controller/
actuators/
sensors/
amplifiers
Active
Virtual
net
Active
Lumped
Distribute
Comms
net
Lumped
Distribute
One direction (Bode-Shannon)
• Shannon channels, theory
• Bode-Shannon, control with channels
Tradeoffs
• Delays (sense/act, plant, comms, comp)
• Noise (sense/act, channels, disturbances)
• Saturation (sense/act, comms bw)
One direction
• Shannon channels, theory
• Bode-Shannon
Virtual
Active
net
Another (Andy L)
• Delays
• networks of delays
Active
Lumped
Distribute
Comms
net
Lumped
Distribute
The
“plant”
Physical
net
The
“plant”
Basic confusions in layered systems
Easy:
• What is software (if in reality/physics there is only
hardware)
Related physics problems:
• Where does dissipation come from (if reality/physics is
actually lossless)… and noise, amplification, etc
• Are uncertainty principles (e.g. p x>c.) necessarily
only quantum mechanical? (no)
• What are truly far from equilibrium systems and do they
violate the 2nd law? (related to amplification and control)
• etc etc
An actual hard one
• Where does excess drag come from in turbulent flows?
Apps
Layers here
3 distinct (but
entwined) kinds
of layering
one kind of layering
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
Actuators/
sensors/
amplifiers
from layers here
are very
different
Computer
Actuator/sensor
Physical plant
The
world
Analog
kernel
Computer
from layers here
Libs, IPC
Software
Hardware
Digital
Analog
Active
Lumped
Distribute
Computers
Layers
Libs, IPC
3 distinct (but
entwined) kinds
of layering
The
world
are very
different
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
Software
Hardware
Digital
Analog
Active
Lumped
Distribute
Computers
Computer
Computer
Actuator/sensor
Physical plant
Actuators/
sensors/
amplifiers
from layers here
one kind of layering
kernel
Software is layered
Apps
Libs, IPC
kernel
Hardware is “layered”
Analog
Active
The
world
Lumped
Distribute
layers
Hardware
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
world
Libs, IPC
are very
different
Software
Hardware
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
Actuators/
sensors/
amplifiers
Apps
kernel
from layers here
from layers here
Layers here
3 distinct (but
entwined) kinds
of layering
Digital
Analog
Active
Lumped
Distribute
Computers
Cyberphysical
Actuators/
sensors/
amplifiers
Software
Hardware
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Computers
Computers
Virtual
net
Comms
net
Physical
net
Software
Hardware
Actuators/
sensors/
amplifiers
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Just as with the Internet, there is a complex
layering of real and virtual networks with
their own dynamics, semantics, etc
Virtual
net
Application
TCP
Comms
net
Physical
net
IP
Physical
Cyber focus
Computers
Computers
Virtual
net
Software
Hardware
Digital
Analog
Virtual
net
Lumped
Distribute
Digital
Analog
Virtual
net
Active
Active
Comms
net
Software
Hardware
Comms
net
Lumped
Distribute
Comms
net
3 distinct (but
entwined)
kinds of
layering
The
world
Libs, IPC
are very
different
Software
Hardware
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
kernel
from layers here
from layers here
Layers here
Cyberphysical systems must
have this kind of layering
Apps
Digital
Analog
Active
Lumped
Distribute
kernel
Software
Hardware
Digital
Analog
Active
Lumped
Distribute
Layers here
Libs, IPC
are very
different
from layers here
Apps
Antenna
3 distinct (but
entwined) kinds
of layering
from layers here
Apps
Libs, IPC
kernel
Software
Hardware
Digital
Analog
Active
Wireless
Antenna
Lumped
Distribute
Turing as
“new”
starting
point?
Software
Hardware
Digital
Analog
Essentials:
0. Model
1. Universal laws
2. Universal architecture
3. Practical implementation
Maybe start from here with
Turing’s 3 step research:
1. hard limits, (un)decidability
using standard model (TM)
2. Universal architecture
achieving hard limits (UTM)
3. Practical implementation in
digital electronics (biology?)
Building on Turing
What I basically want to explain is
• what engineers and mathematicians take too for
granted about Turing’s ideas,
• and scientists completely miss,
• with particular focus on the relevance these ideas
have for the future of systems biology and CPS.
Starting from Turing go in many directions…
Cyberphysical
Actuators/
sensors/
amplifiers
Software
Hardware
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Computers
Computers
Virtual
net
Comms
net
Physical
net
Software
Hardware
Actuators/
sensors/
amplifiers
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Sense
Actuators/
sensors/
amplifiers
Software
Hardware
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Computers
Computers
Virtual
net
Comms
net
Physical
net
Software
Hardware
Actuators/
sensors/
amplifiers
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Decide
Actuators/
sensors/
amplifiers
Software
Hardware
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Computers
Computers
Virtual
net
Comms
net
Physical
net
Software
Hardware
Actuators/
sensors/
amplifiers
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Act
Actuators/
sensors/
amplifiers
Software
Hardware
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Computers
Computers
Virtual
net
Comms
net
Physical
net
Software
Hardware
Actuators/
sensors/
amplifiers
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Sense Decide Act
Actuators/
sensors/
amplifiers
Computers
Software
Hardware
Digital
Analog
Digital
Analog
Active
Active
This “roundtrip”
delay is huge
factor in control
robustness and
performance
T
Lumped
Distribute
Lumped
Distribute
p  1 unstable pole
d
The
“plant”

 pd
delay
Comms
Actuators/
sensors/
amplifiers
Software
Hardware
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Computers
Computers
Virtual
net
Comms
net
Physical
net
Software
Hardware
Actuators/
sensors/
amplifiers
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Decide Act Comms Sense
Actuators/
sensors/
amplifiers
Software
Hardware
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Computers
Computers
Virtual
net
Comms
net
Physical
net
Software
Hardware
Actuators/
sensors/
amplifiers
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Decide Act Comms Sense
c  a  convex
Communication
delay c
Lamperski
Act/Plant/Sense
Delay a
Decide Act
Sense
c  a  convex
Act/Plant/Sense
Delay a
Decide
Comms
c  a  convex
Communication
delay c
c  a  convex
Communication
delay c
Lamperski
Act/Plant/Sense
Delay a
c  a  convex
Actuators/
sensors/
amplifiers
Software
Hardware
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
Virtual
net
Comms
net
Physical
net
Software
Hardware
Actuators/
sensors/
amplifiers
Digital
Analog
Digital
Analog
Active
Active
Lumped
Distribute
Lumped
Distribute
The
“plant”
T

 pd
Want c and d small relative to a
p  1 unstable pole
d
c  a  convex
delay
Local
control
delay d
Comm
delay c
Act/Plant/Sense
“Global “ Delay a
The
“plant”
Plant
The
“plant”
Control
Comms
Shannon
Bode
fragile?
?
slow?
•
•
•
•
•
Each theory  one dimension
Tradeoffs across dimensions
Assume architectures a priori
Progress is encouraging, but…
Stovepipes are an obstacle…
wasteful?
Carnot
Boltzmann
Turing
Godel
Compute
Heisenberg
Einstein
Physics
laws and
architectures?
fragile
Case studies
Sharpen
hard bounds
robust
efficient
wasteful
Csete and Doyle
UG biochem, math,
control theory
Chandra, Buzi, and Doyle
Most important paper so far.
CSTR, yeast extracts
Experiments
K Nielsen, PG Sorensen, F Hynne, H-G
Busse. Sustained oscillations in glycolysis:
an experimental and theoretical study of
chaotic and complex periodic behavior
and of quenching of simple oscillations.
Biophys Chem 72:49-62 (1998).
4
“Standard” Simulation
3
v=0.03
2
1
0
0
20
40
60
80
100
120
140
160
180
200
2
v=0.1
1.5
1
0.5
0
0
20
40
60
80
100
120
140
160
180
200
1
v=0.2
0.8
0.6
0.4
0.2
0
20
40
60
80
100
120
140
160
180
200
Figure S4. Simulation of two state model (S7.1) qualitatively recapitulates
experimental observation from CSTR studies [5] and [12]. As the flow of material
in/out of the system is increased, the system enters a limit cycle and then
stabilizes again. For this simulation, we take q=a=Vm=1, k=0.2, g=1, u=0.01, h=2.5.
4
Experiments
3
2
Why?
Simulation
v=0.03
1
0
0
20
40
60
80
100
120
140
160
180
200
2
v=0.1
1.5
1
0.5
0
0
20
40
60
80
100
120
140
160
180
200
1
v=0.2
0.8
0.6
0.4
0.2
0
20
40
60
80
100
120
140
160
180
200
Figure S4. Simulation of two state model (S7.1) qualitatively recapitulates
experimental observation from CSTR studies [5] and [12]. As the flow of material
in/out of the system is increased, the system enters a limit cycle and then
stabilizes again. For this simulation, we take q=a=Vm=1, k=0.2, g=1, u=0.01, h=2.5.
Why?
Levels of explanation:
1. Possible
2. Plausible
3. Actual
4. Mechanistic
5. Necessary
Science
Engineering
Medicine
Glycolytic “circuit” and oscillations
• Most studied, persistent mystery in cell dynamics
• End of an old story (why oscillations)
– side effect of hard robustness/efficiency tradeoffs
– no purpose per se
– just needed a theorem
• Beginning of a new one
– robustness/efficiency tradeoffs
– complexity and architecture
– need more theorems and applications
Theorem!
Fragility
ln
z p
z p

z
z p


ln S  j   2
d  ln
2 

 0
z p
 z  
1
z and p functions of
enzyme complexity
and amount
simple
enzyme
complex enzyme
Savageaumics
Enzyme amount
Precursors
Catabolism
Inside every cell
almost
ATP
Enzymes
Macro-layers
Building
Blocks
Crosslayer
autocatalysis
AA transl.
Proteins
ATP
Ribosome
RNA transc. xRNA
DNA Repl.
Gene
RNAp
DNAp
Precursors
Catabolism
Energy
ATP
Enzymes
Macro-layers
Building
Blocks
Crosslayer
autocatalysis
AA transl.
Proteins
Protein
RNA transc.
xRNA
biosyn
ATP
DNA Repl.
Gene
Ribosome
RNAp
DNAp
Precursors
Catabolism
Energy
ATP
energy
ATP
ATP
Yeast
anaerobic
glycolysis
Rest
of cell
Minimal
model
Precursors
Catabolism
Energy
ATP
Autocatalytic
feedback
Reaction
1 (“PFK”)
intermediate
metabolite
Yeast
anaerobic
glycolysis
x
Reaction
2 (“PK”)
energy
ATP
ATP
Rest
of cell
Minimal
model
control feedback
Reaction
1 (“PFK”)
x
disturbance
h
control
energy
ATP
g
Reaction
2 (“PK”)
Rest
of cell
Robust =
Maintain energy
(ATP concentration)
despite demand fluctuation
Tight control creates “weak linkage”
between power supply and demand
disturbance
Fragile
energy
ATP
Rest
of cell
Robust =
Robust
Maintain energy
(ATP concentration)
despite demand fluctuation
Tight control creates “weak linkage”
between power supply and demand
enzymes catalyze
reactions
Reaction
1 (“PFK”)
energy Rest
Protein
biosyn
Reaction
2 (“PK”)
enzymes
Efficient =
low metabolic overhead
 low enzyme amount
enzymes
ATP
of cell
Fragile
?
Robust =
ATP
?
Maintain
ATP
Robust
Efficient
Efficient =
low enzyme amount
Wasteful
Theorem!
Fragility
ln
z p
z p

z
z p


ln S  j   2
d  ln
2 

 0
z p
 z  
1
z and p functions of
enzyme complexity
and amount
simple
enzyme
complex enzyme
Savageaumics
Enzyme amount
Architecture
Good architectures
allow for effective
tradeoffs
fragile
wasteful
Precursors
Catabolism
Inside every cell
almost
ATP
Enzymes
Macro-layers
Building
Blocks
Crosslayer
autocatalysis
AA transl.
Proteins
ATP
Ribosome
RNA transc. xRNA
DNA Repl.
Gene
RNAp
DNAp
How general is this picture?
fragile
simple
tech
robust
Implications for
human evolution?
Cognition?
Technology?
Basic sciences?
complex tech
efficient
wasteful
Precursors
Catabolism
Inside every cell
almost
ATP
Enzymes
Macro-layers
Building
Blocks
Crosslayer
autocatalysis
AA transl.
Proteins
ATP
Ribosome
RNA transc. xRNA
DNA Repl.
Gene
RNAp
DNAp
Precursors
Catabolism
Inside every cell
almost
ATP
Ribosomes
make
ribosomes
AA transl.
ATP
Translation: Amino acids
polymerized into proteins
Proteins
Ribosome
Precursors
Catabolism
ATP
• Translation
• Transcription
• DNA Replication
Building
Blocks
AA transl.
Proteins
ATP
Ribosome
RNA transc. xRNA
DNA Repl.
Gene
RNAp
DNAp
control response
slow (  fragile)
Tradeoffs?
fast
cheap
metabolic overhead
expensive
Catabolism
expensive
Precursors
ATP
AA
ATP
transl.
Proteins
RNA transc.
DNA
Repl.
xRNA
cheap
fast
Tradeoffs
drawn
awkwardly
Ribosome
RNAp
Gene
DNAp
slow
Catabolism
expensive
Precursors
ATP
AA transl.
ATP
Proteins
cheap
fast
Ribosome
RNA transc. xRNA
DNA Repl.
Gen
RNAp
DNAp
e
slow
Precursors
Catabolism
expensive
Tradeoffs
redrawn
ATP
AA
ATP
transl.
Proteins
RNA transc.
DNA
cheap
fast
Repl.
Ribosome
xRNA
Gene
RNAp
DNAp
slow
Precursors
Catabolism
expensive
Tradeoffs
redrawn
ATP
AA
ATP
transl.
Proteins
RNA transc.
DNA
cheap
fast
Repl.
Ribosome
xRNA
Gene
RNAp
DNAp
slow
Precursors
Catabolism
Layered view
ATP
Enzymes
Macro-layers
Building
Blocks
Crosslayer
autocatalysis
AA transl.
Proteins
ATP
Ribosome
RNA transc. xRNA
DNA Repl.
Gene
RNAp
DNAp
Precursors
Shared protocols
ATP
Enzymes
• Universal
core constraints
• “virtual machines”
NAD
AA transl.
Proteins
RNA transc. xRNA
DNA Repl.
Gene
Precursors
Catabolism
Autocatalytic
feedbacks
ATP
Enzymes
Building
Blocks
AA transl.
100s of
genes
Proteins
ATP
Ribosome
RNA transc. xRNA
<10% of most
bacterial genomes
DNA Repl.
Gene
RNAp
DNAp
Precursors
Catabolism
Control
feedbacks?
ATP
Enzymes
Building
Blocks
AA transl.
Proteins
ATP
Ribosome
RNA transc. xRNA
Control
DNA Repl.
Gene
RNAp
DNAp
Precursors
Catabolism
Control
feedbacks?
ATP
Enzymes
Building
Blocks
AA transl.
Proteins
ATP
Ribosome
RNA transc. xRNA
Control
DNA Repl.
Gene
RNAp
DNAp
Precur
Catabol
ATP
Enzymes
Building
Blocks
AA transl.
Proteins
ATP
Ribosome
RNA transc. xRNA
Control
DNA Repl.
Gene
Complexity of control is huge
and poorly studied.
RNAp
DNAp
Precursors
Catabolism
expensive
ATP
Enzymes
It is control that is
fast/slow
and
Proteins
Building AA transl.
Blocks
expensive/cheap
ATP
RNA
Control
cheap
fast
Ribosome
transc.
xRNA
RNAp
DNA
Repl.
Gene
DNAp
slow
Precursors
Catabolism
Layered view
ATP
Enzymes
Macro-layers
Building
Blocks
Crosslayer
autocatalysis
AA transl.
Proteins
ATP
Ribosome
RNA transc. xRNA
DNA Repl.
Gene
RNAp
DNAp
Precursors
Catabolism
ATP
Enzymes
Building Blocks
AA
Proteins
transl.
ATP
Ribosome
RNA
transc.
xRNA
RNAp
DNA
Repl.
Gene
DNAp
<10% of
most
Control
bacterial
>90% ofgenomes
most
bacterial genomes
~300 genes,
~minimal
genome,
requires
idealized
environment
Precursors
Catabolism
Environment
ATP
Enzymes
Action
Building Blocks
AA
Proteins
transl.
ATP
Ribosome
RNA
transc.
xRNA
RNAp
DNA
Repl.
Gene
DNAp
Control
>90% of most
bacterial genomes
What about
“natural”
environments?
Precursors
Catabolism
Environment
ATP
Enzymes
Action
Building Blocks
AA
Proteins
transl.
ATP
Ribosome
RNA
transc.
xRNA
RNAp
DNA
Repl.
Gene
DNAp
Control
>90% of most
bacterial genomes
Environment
Precursors
Catabolism
ATP
Action
Enzymes
Building Blocks
AA
Proteins
transl.
Ribosome
ATP
RNA
transc.
xRNA
RNAp
DNA
Repl.
Gene
DNAp
Control
>90% of most
bacterial genomes
Precursors
Catabolism
ATP
Crosslayer
autocatalysis
Enzymes
Macro-layers
Building
Blocks
AA transl.
Proteins
ATP
Ribosome
RNA transc. xRNA
DNA Repl.
Gene
RNAp
DNAp
Precursors
Catabolism
• Building blocks
− Scavenge
− Recycle
− Biosynthesis
• Special enzymes
• Allostery, Fast
• Expensive control
• Analog
ATP
AA transl.
• Complex machines
− Polymerization
− Complex assembly
• General enzymes
• Regulated recruitment
• Slow, efficient control
• Quantized, digital
Proteins
ATP
Ribosome
RNA transc. xRNA
DNA Repl.
Gene
RNAp
DNAp
Precursors
Catabolism
expensive
ATP
AA
transl.
Proteins
ATP
Ribosom
RNA transc.
xRNA
RNA
DNA
Repl.
cheap
fast
slow
Gene
DNA
Precursors
Catabolism
• Ecosystems
• Biofilms
• Extremophiles
• Pathogens
• Symbiosis
•…
ATP
• Homeostasis
− pH
− Osmolarity
− etc
• Cell envelope
• Movement,
attachment, etc
What we’ve neglected here
AA transl. Proteins
• DNA replication
− Highly controlled
ATP
Ribosome
− Facilitated variation
− Accelerates evolution RNA transc.
xRNA
• DNA modification (e.g.
RNAp
methylation)
DNA Repl. Gene
• Complex RNA control
DNAp
Precursors
Catabolism
Inside every cell
ATP
Enzymes
Macro-layers
Building
Blocks
Crosslayer
autocatalysis
AA transl.
Proteins
ATP
Ribosome
RNA transc. xRNA
DNA
Repl.
RNAp
Gen
e
DNAp
Lower layer autocatalysis
Macromolecules making …
Enzymes
Three lower
Proteins
transl.
AA
layers? Yes: Building
• Translation
Blocks
Ribosome
• Transcription
RNA
transc. xRNA
• Replication
RNAp
DNA
Repl.
Gen
e
DNAp
Enzymes
• Translation
AA
Building
• Transcription
Blocks
• Replication
transl.
Enzymes
Proteins
Ribosome
xRNA
Enzymes
• Translation
• Transcription
• Replication
Building
Blocks
Proteins
transl.
RNA transc.
xRNA
RNAp
Gen
e
Enzymes
Building
Blocks
• Translation
• Transcription
• Replication
Proteins
transl.
transc.
DNA Repl.
xRNA
Gen
e
DNAp
Precursors
Catabolism
ATP
Pathway views
Proteins
transl.
transc.
Repl.
mRNA
Gen
e
Precursors
Catabolism
expensive
Tradeoffs
redrawn
ATP
AA
ATP
transl.
Proteins
RNA transc.
DNA
cheap
fast
Repl.
Ribosome
xRNA
Gene
RNAp
DNAp
slow
How general is this picture?
fragile
simple
tech
robust
Implications for
human evolution?
Cognition?
Technology?
Basic sciences?
complex tech
efficient
wasteful
Layered architectures
Diverse applications
TCP
IP
MAC
Switch
MAC
MAC
Pt to Pt
Pt to Pt
Physical
Proceedings of the IEEE, Jan 2007
OR
optimization
What’s
next?
Chang, Low, Calderbank, Doyle
A rant
Too clever?
Diverse applications
TCP
IP
Diverse
Physical
Layered architectures
Deconstrained
(Applications)
Networks
TCP
IP
Constrained
Deconstrained
(Hardware)
“constraints that deconstrain” (Gerhart and Kirschner)
Networked OS
Deconstrained
(Applications)
Constrained TCP/
IP
Deconstrained
(Hardware)
• OS better starting point than
phone/comms systems
• Extreme robustness confers
surprising evolvability
• Creative engineers
• Rode hardware evolution
Facilitated wild evolution
Created
• whole new ecosystem
• completely opposite
Why?
Architecture
Layered architectures
Deconstrained
(Applications)
Essentials
Few global variables
Don’t cross layers
Constrained
OS
Control, share, virtualize,
and manage resources
Deconstrained
(Hardware)
Processing
Memory
I/O
Layered architectures
Deconstrained
(diverse)
Environments
Precursors
Catabolism
ATP
Shared
protocols
Enzymes
Building Blocks
AA
Proteins
transl.
ATP
Ribosome
RNA
transc.
xRNA
RNAp
DNA
Repl.
Gene
DNAp
Deconstrained (diverse)
Genomes
Bacterial
biosphere
Architecture
=
Constraints
that
Deconstrain
Precursors
Catabolism
Inside every cell
almost
ATP
Enzymes
Macro-layers
Building
Blocks
Crosslayer
autocatalysis
AA transl.
Proteins
ATP
Ribosome
RNA transc. xRNA
DNA Repl.
Gene
RNAp
DNAp
What makes the bacterial
biosphere so adaptable?
Environment
Deconstrained
phenotype
Core conserved
constraints facilitate
tradeoffs
Precursors
Catabolism
ATP
Enzymes
Building Blocks
Action
AA
Proteins
transl.
ATP
Ribosome
RNA
transc.
xRNA
RNAp
DNA
Repl.
Gene
DNAp
Deconstrained genome
Active control of
the genome
(facilitated
variation)
Layered architectures
Deconstrained
(Applications)
Few global variables
Don’t cross layers
Direct
access
to
Control,
share, virtualize,
Constrained
OS physical
and manage resources
memory?
Processing
Memory
Deconstrained
I/O
(Hardware)
Deconstrained
(diverse)
Environments
Few global variables
Architecture
Precursors
ATP
Catabolism
Bacterial
biosphere
Enzymes
Building Blocks
AA
Shared
protocols
=
Constraints
that
Don’t cross layers
Deconstrain
Proteins
transl.
ATP
Ribosome
RNA
transc.
xRNA
RNAp
DNA
Repl.
Gene
DNAp
Deconstrained (diverse)
Genomes
Problems with leaky layering
Modularity benefits are lost
• Global variables? @$%*&!^%@&
• Poor portability of applications
• Insecurity of physical address space
• Fragile to application crashes
• No scalability of virtual/real addressing
• Limits optimization/control by duality?
Fragilities of layering/virtualization
“Universal” fragilities that must be avoided
• Hijacking, parasitism, predation
– Universals are vulnerable
– Universals are valuable
• Cryptic, hidden
– breakdowns/failures
– unintended consequences
• Hyper-evolvable but with frozen core
Layered architectures
Deconstrained
(Applications)
Few global variables?
Don’t cross layers?
Constrained
TCP/
IP
Control, share, virtualize,
and manage resources
Deconstrained
(Hardware)
I/O
Comms
Latency?
Storage?
Processing?
IPC
App
caltech.edu?
App
DNS
IP addresses
interfaces
(not nodes)
Global
and direct
access to
physical
address!
131.215.9.49
IPC
App
App
DNS
Robust?
• Secure
• Scalable
• Verifiable
• Evolvable
• Maintainable
• Designable
•…
IP addresses
interfaces
(not nodes)
Global
and direct
access to
physical
address!
Naming and addressing need to be
• resolved within layer
• translated between layers
• not exposed outside of layer
Related “issues”
• VPNs
• NATS
• Firewalls
• Multihoming
• Mobility
• Routing table size
• Overlays
•…
Application
TCP
IP
Physical
Until late 1980s, no
congestion control, which
led to “congestion collapse”
TCP
IP
Diverse
Physical
Original design challenge?
Deconstrained
(Applications)
Constrained TCP/
IP
Deconstrained
(Hardware)
Networked OS
• Expensive mainframes
• Trusted end systems
• Homogeneous
• Sender centric
• Unreliable comms
Facilitated wild evolution
Created
• whole new ecosystem
• completely opposite
Next layered architectures
Deconstrained
(Applications)
Few global variables
Don’t cross layers
Constrained
?
Control, share, virtualize,
and manage resources
Deconstrained
(Hardware)
Comms
Memory, storage
Latency
Processing
Cyber-physical
Persistent errors
and confusion
(“network science”)
Every layer
has
different
diverse
graphs.
Architecture is least
graph topology.
Application
TCP
IP
Physical
Architecture
facilitates
arbitrary
graphs.
Evolution and architecture
Nothing in biology makes sense except in the light of
evolution
Theodosius Dobzhansky
(see also de Chardin)
Nothing in evolution makes sense except in the light of
biology
?????
How general is this picture?
Very! Constraints!
i.e. hard limits and architecture
Precursors
Catabolism
ATP
Enzymes
fragile
Building Blocks
AA
Proteins
transl.
ATP
Ribosome
RNA
simple
tech
robust
RNAp
DNA
Repl.
Gene
DNAp
complex tech
efficient
xRNA
transc.
wasteful
 huge new insights!
So what’s missing?
Explaining clearly the fundamental
role that layered architectures play
in making facilitated variation
possible.
Now have extensive, and hopefully
accessible case studies:
1. Bacterial biosphere and HGT
2. Evolution of human brain
(EvoDevo still too hard)
Note: The pieces underlying these
new views are widely accepted, but
the consequences are emphatically
not!
Why so much resistance?
A big reason:
Big (wrong) idea: All complexity is
emergent from random ensembles
with minimal tuning
Sketch how this plays out
Standard (old) theory:
The modern synthesis =
natural selection + genetic drift
+ mutation + gene flow
What’s missing here?
Almost everything
Evolution fact and theory
• The fact of evolution is well-known
– Organisms and their genes change over time
– Seen in labs, hospitals, barnyards, fossil records, etc
• Darwin gave plausible explanation
– Variation and selection are both observable
– Molecular mechanisms are not known
• Modern synthesis gave molecular mechanism
• Theory has evolved and deepened
• What is “standard” modern synthesis theory now?
Standard theory:
natural selection + genetic drift
+ mutation + gene flow
Shapiro explains well what this is
and why it’s incomplete
Gene
alleles
Selection
Greatly abridged cartoon here
Standard theory: natural selection + genetic drift +
mutation + gene flow
Assumptions:
• Genome inherited from parent
• Mutations due to replication errors
• At constant rate, rare, small, localized
• Limited to small, usually deleterious effect
• Selection as “creative” mechanism
• Examples abound
• Math straightforward and obvious, like stat phys
• No architecture or anything that smells like engineering
• No new principles beyond physics and chemistry
The extreme version
Assumptions strengthened:
• Genome inherited only from parent
• Mutations due only to replication errors
• Only at constant rate, rare, small, localized
• Limited only to small, usually deleterious effect
• Selection only “creative” mechanism
• Mutations can’t be “targeted” within the genome
• Can’t coordinate DNA change w/ useful adaptive needs
• Viruses can’t induce DNA change giving heritable resistance
Big (wrong) idea: All biological complexity is
emergent via random mutations with small and
minimal tuning via natural selection
In retrospect, why is the conventional theory so popular ?
Standard assumptions:
• Inherit only from parent
• Mutations = replication errors
• Constant rate, rare, small, localized
• Usually deleterious
• Selection = “creative” mechanism
• Mutations can’t be “targeted” or coordinated
• Viruses can’t induce heritable resistance
• Simple, fits theory of “random plus minimal tuning”
• Examples abound
• Math straightforward and obvious, like stat phys
• No architecture or anything that smells like engineering
• No new principles beyond physics and chemistry
• No real phenotype, just abstract “fitness landscapes”
• Focuses on genomes which can now be measured
• Strongly rejects creationism and any sort of “design”
• But still allows for the mystical (e.g. “order for free”)
Examples abound that fit the theory
Standard assumptions:
• Inherit only from parent
• Mutations = replication errors
• Constant rate, rare, small, localized
• Usually deleterious
• Selection = “creative” mechanism
• Mutations can’t be “targeted” or coordinated
• Viruses can’t induce heritable resistance
Bacterial clones (circa 1940):
• Exponential growth from one cell in ideal lab conditions
• Accumulates random neutral mutations
• Generating (“scale-free”) phylogenetic tree of mutants
Cancer:
• Cells evolve new robustness via largely random mutations
• Hijack robust, evolvable architectures
• Tumors also hijack system resources
• Eventually kill host and themselves
• Example of evolution as grotesquely myopic
Standard theory: natural selection + genetic drift +
mutation + gene flow
Assumptions:
• Genome inherited only from parent
• Mutations due to replication errors
• At constant rate, rare, small, localized
• Limited to small, usually deleterious effect
• Selection only “creative” mechanism
• Mutations can’t be “targeted” within the genome
• Can’t coordinate DNA change w/ useful adaptive needs
• Viruses can’t induce DNA change giving heritable resistance
Gerhart and Kirschner, Shapiro, Caporale, Lane, etc etc
show  facts proving this is wildly incomplete. (Even e.g.
Koonin agrees with this though differing in details)
Organisms wildly violate all of them. But the theory
needed is now vastly more complex.
Gerhart and Kirschner
Facilitated variation
Architecture =
Constraints that deconstrain
• Weak linkage
• Exploratory mechanisms
• Compartmentalization
• (EvoDevo)
Unpopular within biology, see
Erwin’s review in Cell 2005
EvoDevo= Evolution + development
• Hottest part of evolutionary theory
• Most complicated relationship
between geno- and pheno-type
• Many unresolved issues
• Ignore for now
• Focus primarily on microbes
• and large evolutionary transitions
• Many mechanisms for “horizontal” gene transfer
• Many mechanisms to create large, functional mutations
• At highly variable rate, can be huge, global
• Selection alone is a very limited filtering mechanism
• Mutations can be “targeted” within the genomes
• Can coordinate DNA change w/ useful adaptive needs
• Viruses can induce DNA change giving heritable resistance
• Still myopic about future, still produces the grotesque
Modern synthesis assumptions:
• Genome inherited only from parent
• Mutations due to replication errors
• At constant rate, rare, small, localized
• Limited to small, usually deleterious effect
• Selection only “creative” mechanism
• Mutations can’t be “targeted” within the genome
• Can’t coordinate DNA change w/ useful adaptive needs
• Viruses can’t induce DNA change giving heritable resistance
Now almost everyone agrees with facts of an “extended” synthesis:
• Many mechanisms for “horizontal” gene transfer
• Many mechanisms to create large, functional (or neutral) mutations
• At highly variable rate, can be huge, global
• Selection alone is a very limited filtering mechanism
• Mutations can be “targeted” within the genomes
• Can coordinate DNA change w/ useful adaptive needs
• Viruses can induce DNA change giving heritable resistance
So what are the remaining issues? Lots!
Now almost everyone agrees with facts of an “extended” synthesis:
• Many mechanisms for “horizontal” gene transfer
• Many mechanisms to create large, functional (or neutral) mutations
• At highly variable rate, can be huge, global
• Selection alone is a very limited filtering mechanism
• Mutations can be “targeted” within the genomes
• Can coordinate DNA change w/ useful adaptive needs
• Viruses can induce DNA change giving heritable resistance
So what are the remaining issues? Lots!
Conjecture: Much current controversy caused by
• ignoring layered arch. from genome to phenome
• confusion between large and thin in layers
Oversimplifying: Two main “camps”
1. Minimal tweak modern syn, minimal architecture
2. Maximal architecture
The minimal architecture camp
• Definite, solid progress
• Reconciles “stat phys” view with many “new facts”
• large but closeable gap to maximal view
The maximal architecture camp
• Challenge: distill the essence of their insights
• Architecture = constraints that deconstrain
• Evolving evolvability
• “bacterial Internet” (Caporale)
• Try to close gap from max to min camp
natural selection + genetic drift
+ mutation + gene flow
+ facilitated variation
large
functional
changes in
genomes
HGT
= horizontal
gene transfer
natural selection + genetic drift + mutation + gene flow
+ facilitated variation
Genome can have large changes
Genes
natural selection + genetic drift + mutation + gene flow
+ facilitated variation
Small gene change can have large but
functional phenotype change
Phenotype
Architecture
Genotype
natural selection + genetic drift + mutation + gene flow
+ facilitated variation
Small gene change can have large but
functional phenotype change
Only possible because of shared,
layered, network architecture
Phenotype
Architecture
Genotype
Highlighted differences
Old
1. Randomness in genetic
change
New
1. Randomness in
environment
–
–
HGT
Stress on organism
Darwin on facilitated variation
"...variations which seem to us in our ignorance
to arise spontaneously. It appears that I formerly
underrated the frequency and value of these
latter forms of variation, as leading to permanent
modifications of structure independently of
natural selection.”
Darwin, Origin of Species, 6th edition, Chapter XV,
p. 395.
Some facts about bacteria:
• ~100 core conserved genes (that
define architecture) by lineage
descent only
• Most other () genes acquired
(sometime) by HGT
• “minimal” genome includes core
conserved genes plus ~200 more
• what gives?
• “Constraints that deconstrain” = architecture
• Bacterial genomes/phylogenies are great case studies
• Only the facts and the components are (so far) explained
• Needs layered architecture theory to hang the facts on
• Initial paper can be written now
More facts about bacteria:
• In humans, # bacterial cells  10x # human cells
• In humans, # bacterial genes >100x # human genes
• In ocean, >1e30 bacterial cells, >1e31 viruses
• In ocean, viruses kill half of bacterial cells every day
• Eukaryotes cells created by fusion of prokaryote cells
• Layered architecture crucial enabler
• New discoveries of intermediates?
What’s missing?
Explaining clearly the fundamental
role that layered architectures play
in making facilitated variation
possible.
Now have extensive, and hopefully
accessible, case studies:
1. Bacterial biosphere and HGT
2. Evolution of human brain
(EvoDevo still too hard)
Unfortunately, not
intelligent design
Ouch.
• Outcomes can be
grotesque, maladaptive.
• Therefore, dangerous to
rely on evolution alone,
no matter how facilitated
Evolution theory 1.0 assumptions:
• Inherit only from parent
• Mutations = replication errors
• Constant rate, rare, small, localized
• Usually deleterious
• Selection = “creative” mechanism
Modern Synthesis:
Great start iff
taken as only
a start
• Mutations can’t be “targeted” or coordinated
• Viruses can’t induce heritable resistance
Evolution facts 1.0 (examples)
Bacterial clones (circa 1940):
• Exponential growth from one cell in ideal lab conditions
• Accumulates random neutral mutations
• Generating (“scale-free”) phylogenetic tree of mutants
Cancer:
• Cells evolve new robustness via largely random mutations
• Hijack robust, evolvable architectures
• Tumors also hijack system resources
• Eventually kill host and themselves
• Example of evolution as grotesquely myopic
Evolution theory 2.0 assumptions?
• Horizontal transfer +++
• Mutation highly variable but not random
• Fast and slow, large and small, local and global
• Facilitated variation is main creative force
• Selection is weak filter on unfit
Evolution facts 2.0?
Tons of facts that violate theory 1.0, now in books
Evolution theory 2.0?
• Constraints that deconstrain is a start
• Need layered architecture story that hasn’t been told
How general is this picture?
fragile
simple
tech
robust
Implications for
human evolution?
Cognition?
Technology?
Basic sciences?
complex tech
efficient
wasteful
Theorem!
Fragility
ln
z p
z p

z
z p


ln S  j   2
d  ln
2 

 0
z p
 z  
1
z and p functions of
enzyme complexity
and amount
simple
enzyme
complex enzyme
Savageaumics
Enzyme amount
Fragility
hard limits
• General
• Rigorous
• First principle
simple
complex
Overhead, waste
Plugging in
domain details
?
• Domain specific
• Ad hoc
• Phenomenological
Wiener
Control
Bode
Kalman
robust control
Comms
Shannon
• Fundamental multiscale physics
• Foundations, origins of
– noise
– dissipation
– amplification
– catalysis
Carnot
• General
• Rigorous
• First principle
?
Boltzmann
Heisenberg
Physics
Control
Comms
Complex
networks
Wildly “successful”
Compute
“New sciences” of
complexity and networks
edge of chaos, self-organized
criticality, scale-free,…
Stat physics
Carnot
Boltzmann
Heisenberg
Physics
Popular but
wrong
D. Alderson, NPS
177
Complex
networks
doesn’t
work
Alderson & Doyle,
Contrasting Views of
Complexity and Their
Implications for
Network-Centric
Infrastructure,
IEEE TRANS ON
SMC,
JULY 2010
Stat physics
“New sciences” of
complexity and networks
edge of chaos, self-organized
criticality, scale-free,…
Alderson &Doyle, Contrasting
Views of Complexity and Their
Implications for Network-Centric
Infrastructure,
IEEE TRANS ON SMC,
JULY 2010
Complex
networks
Control
Sandberg, Delvenne,
& Doyle, On Lossless
Approximations, the FluctuationDissipation Theorem, and
Limitations of Measurement,
IEEE TRANS ON AC,
FEBRUARY, 2011
Stat physics
Carnot
Boltzmann
Heisenberg
Physics
“The last 70 years of the 20th century will be viewed as
the dark ages of theoretical physics.” (Carver Mead)
Complex
networks
“orthophysics”
From prediction
to mechanism
to control
Sandberg, Delvenne,
& Doyle, On Lossless
Approximations, the FluctuationDissipation Theorem, and
Limitations of Measurement,
IEEE TRANS ON AC,
FEBRUARY, 2011
Stat physics,
fluids, QM
Carnot
Boltzmann
Heisenberg
Physics
J. Fluid Mech (2010)
Streamlined
Laminar Flow
Turbulence
and drag?
Turbulent Flow
Physics of Fluids (2011)
Dennice Gayme,
Beverley McKeon,
Bassam Bamieh,
Antonis Papachristodoulou,
John Doyle
Physics of Fluids (2011)
y
z
Uw
y
Flow
z
u
x
x
Coherent structures and turbulent drag
high-speed
region
upflow
downflow
3D coupling
low speed
streak
Blunted turbulent
velocity profile
Turbulent
Laminar
Uw
Uw
Turbulent
fragile
robust
Laminar
?
efficient
Laminar
Turbulent
wasteful
J. Fluid Mech (2010)
Streamlined
Laminar Flow
Turbulence
and drag?
Turbulent Flow
Physics of Fluids (2011)
Dennice Gayme,
Beverley McKeon,
Bassam Bamieh,
Antonis Papachristodoulou,
John Doyle
Physics of Fluids (2011)
y
z
Uw
y
Flow
z
u
x
x
Coherent structures and turbulent drag
high-speed
region
upflow
downflow
3D coupling
low speed
streak
Blunted turbulent
velocity profile
Turbulent
Laminar
Uw
Uw
Turbulent
fragile
robust
Laminar
?
efficient
Laminar
Turbulent
wasteful
How general is this picture?
fragile
simple
tech
robust
Implications for
human evolution?
Cognition?
Technology?
Basic sciences?
complex tech
efficient
wasteful
This paper aims to bridge progress in neuroscience involving
sophisticated quantitative analysis of behavior, including the use
of robust control, with other relevant conceptual and theoretical
frameworks from systems engineering, systems biology, and
mathematics.
Very accessible
No math
Doyle and Csete, Proc Nat Acad Sci USA, online JULY 25 2011
Evolution and architecture
Nothing in biology makes sense except in the light of
evolution
Theodosius Dobzhansky
(see also de Chardin)
Nothing in evolution makes sense except in the light of
biology
?????
natural selection + genetic drift
+ mutation + gene flow
++ architecture
Gerhart and Kirschner
Facilitated variation
Architecture =
Constraints that deconstrain
• Weak linkage
• Exploratory mechanisms
• Compartmentalization
Unfortunately, not
intelligent design
Ouch.
weak
fragile
slow
Human
evolution
strong
robust
fast
hands
feet
All very
skeleton
different.
muscle
skin
gut
long helpless childhood
Apes
How is this
progress?
Homo Erectus?
weak
fragile
This much seems pretty
consistent among experts
regarding circa 1.5-2Mya
strong
robust
hands
feet
skeleton
muscle
skin
gut
Roughly
modern
Very
fragile
So how did H. Erectus
survive and expand globally?
efficient
(slow)
inefficient
wasteful
endurance
weak
fragile
speed &
strength
Apes
strong
robust
(fast)
efficient
(slow)
inefficient
wasteful
weak
fragile
(slow)
Human
evolution
hands
feet
skeleton
muscle
skin
gut
Apes
strong
robust
(fast)
efficient
(slow)
inefficient
wasteful
weak
fragile
Architecture?
Evolvable?
Hard
tradeoffs?
Apes
strong
robust
efficient
(slow)
inefficient
wasteful
endurance
weak
fragile
+
sticks
stones
fire
teams
strong
robust
efficient
(slow)
From weak prey
to invincible
predator?
Speculation? There is only
evidence for crude stone tools.
But sticks, fire, teams might
not leave a record?
Speculation? With only
evidence for crude stone tools.
But sticks and fire might not
leave a record?
weak
fragile
+
strong
robust
sticks
stones
fire
teams
efficient
(slow)
From weak prey to
invincible predator
Before much
brain expansion?
Plausible but speculation?
Cranial capacity
Before much
brain expansion?
Gap?
Today
2Mya
hands
feet
skeleton
muscle
skin
gut
weak
fragile
+
sticks
strong stones
robust
fire
efficient
(slow)
From weak prey
to invincible
predator
Before much
brain expansion?
Key point:
Our physiology,
technology,
and brains
have coevolved
Probably true
no matter what
Huge
implications.
Cranial capacity
Today
2Mya
hands
feet
skeleton
muscle
skin
gut
weak
fragile
+
sticks
strong stones
robust
fire
efficient
(slow)
Key point needing
more discussion:
The evolutionary
challenge of big brains
is homeostasis, not
basal metabolic load.
From weak prey
to invincible
predator
Before much
brain expansion?
Huge
implications.
weak
fragile
Architecture?
+
strong
robust
sticks
stones
fire
efficient
(slow)
inefficient
wasteful
hands
feet
skeleton
muscle
skin
gut
weak
fragile
+
strong
robust
sticks
stones
fire
efficient
(slow)
From weak prey
to invincible
predator
Before much
brain expansion?
weak
fragile
Architecture?
+
strong
robust
sticks
stones
fire
efficient
(slow)
inefficient
wasteful
Human
complexity?
Consequences of
our evolutionary
history?
fragile
sticks
stones
fire
robust
efficient
wasteful
Human complexity
Robust
Fragile
 Metabolism
 Regeneration & repair
 Healing wound /infect
 Obesity, diabetes
 Cancer
 AutoImmune/Inflame
Start with physiology
Lots of triage
Benefits
Robust
 Metabolism
 Regeneration & repair
 Healing wound /infect




Efficient
Mobility
Survive uncertain food supply
Recover from moderate trauma
and infection
Mechanism?
Robust
Fragile
 Metabolism
 Regeneration & repair
 Healing wound /infect
 Fat accumulation
 Insulin resistance
 Proliferation
 Inflammation
 Obesity, diabetes
 Cancer
 AutoImmune/Inflame
 Fat accumulation
 Insulin resistance
 Proliferation
 Inflammation
What’s the difference?
Robust
Fragile
 Metabolism
 Regeneration & repair
 Healing wound /infect




Controlled
Dynamic
 Obesity, diabetes
 Cancer
 AutoImmune/Inflame
Fat accumulation
Insulin resistance
Proliferation
Inflammation
Uncontrolled
Chronic
Controlled
Dynamic
Low mean
High variability




Fat accumulation
Insulin resistance
Proliferation
Inflammation
Death
Controlled
Dynamic
Low mean
High variability




Fat accumulation
Insulin resistance
Proliferation
Inflammation
Uncontrolled
Chronic
High mean
Low variability
Restoring robustness?
Robust
 Metabolism
 Regeneration & repair
 Healing wound /infect




Fat accumulation
Insulin resistance
Proliferation
Inflammation
Fragile
 Obesity, diabetes
 Cancer
 AutoImmune/Inflame




Fat accumulation
Insulin resistance
Proliferation
Inflammation
Controlled
Dynamic
Uncontrolled
Chronic
Low mean
High variability
High mean
Low variability
weak
fragile
++Technology
strong
robust
efficient
(slow)
inefficient
wasteful
Human complexity
Robust








Metabolism
Regeneration & repair
Immune/inflammation
Microbe symbionts
Neuro-endocrine
Complex societies
Advanced technologies
Risk “management”
Yet Fragile
 Obesity, diabetes
 Cancer
 AutoImmune/Inflame
 Parasites, infection
 Addiction, psychosis,…
 Epidemics, war,…
 Disasters, global &!%$#
 Obfuscate, amplify,…
Accident or necessity?
Robust
Fragile
 Metabolism
 Obesity, diabetes
 Regeneration &repair
 Cancer
Fat accumulation
 Healing wound
/infect
 AutoImmune/Inflame
Insulin resistance
 Proliferation
 Inflammation
•
•
•
•
Fragility  Hijacking, side effects, unintended…
Of mechanisms evolved for robustness
Complexity  control, robust/fragile tradeoffs
Math: robust/fragile constraints (“conservation laws”)
Both
Accident or necessity?
Constraints
(that deconstrain)
fragile
robust
Hard
tradeoffs?
efficient
wasteful
“Seeing is dreaming?”
“Seeing is believing?”
Slow,
Broad
scope
Which blue
Meta-layers
Fast,
Limited
scope
line is
longer?
delay=death
sense
move
Spine
Reflect
Reflex
sense
move
Spine
Reflect
Reflex
sense
move
Spine
Reflect
Reflex
sense
move
Spine
Reflect
Reflex
sense
move
Spine
Layered architectures
Physiology
Organs
Cells
Spine
errors
move
Actions
sense
Goals
Actions
Reflex
Reflect
Prediction
Cortex
Meta-layers
Prediction
Goals
Actions
errors
Physiology
Actions
Organs
Visual
Cortex
Why?
Visual
Thalamus
10x
?
There are 10x
feedback neurons
Prediction
Goals
Prediction
Conscious
perception
What are the
Physiology
consequences?
Goals
Actions
Organs
Actions
errors
Why?
10x
Visual
Cortex
There are 10x
feedback neurons
Visual
Thalamus
?
Seeing is dreaming
Conscious
perception
3D
+time
Simulation
Conscious
perception
Zzzzzz…..
Same size?
Same size
Same size
Same size
Toggle between this slide and
the ones before and after
Even when you “know” they are
the same, they appear different
Same size?
Vision: evolved for complex
simulation and control, not
2d static pictures
Even when you “know” they are
the same, they appear different
Seeing is dreaming
3D
Conscious
perception
+time
Simulation
+ complex
models
(“priors”)
Conscious
perception
Zzzzzz…..
Conscious
perception
Seeing is dreaming
3D
+time
Simulation
+ complex
models
(“priors”)
Conscious
perception
Prediction
errors
Seeing is dreaming
Conscious
perception
Seeing is believing
3D
+time
Simulation
+ complex
models
(“priors”)
Conscious
perception
Prediction
errors
Seeing is dreaming
Conscious
perception
Seeing is believing
3D
+time
Simulation
+ complex
models
(“priors”)
Conscious
perception
Prediction
errors
Our most
integrated
perception
of the
world
Which blue line is longer?
Which blue line is longer?
Which blue line is longer?
Which blue line is longer?
Which blue line is longer?
Which blue line is longer?
Which blue line is longer?
With
social
pressure,
this one.
Standard social psychology experiment.
Chess experts
• can reconstruct entire
chessboard with < ~ 5s
inspection
• can recognize 1e5 distinct
patterns
• can play multiple games
blindfolded and simultaneous
• are no better on random
boards
(Simon and Gilmartin, de Groot)
www.psywww.com/intropsych/ch07_cognition/expertise_and_domain_specific_knowledge.html
• Polistes fuscatus can differentiate among normal wasp
face images more rapidly and accurately than nonface
images or manipulated faces.
• Polistes metricus is a close relative lacking facial
recognition and specialized face learning.
• Similar specializations for face learning are found in
primates and other mammals, although P. fuscatus
represents an independent evolution of specialization.
• Convergence toward face specialization in distant taxa
as well as divergence among closely related taxa with
different recognition behavior suggests that specialized
cognition is surprisingly labile and may be adaptively
shaped by species-specific selective pressures such as
face recognition.
Fig. 1 Images used for training wasps.
M J Sheehan, E A Tibbetts Science 2011;334:1272-1275
Published by AAAS
Slow,
Broad
scope
Unfortunately, we’re not
sure how this all works.
Meta-layers
Fast,
Limited
scope
Seeing is dreaming
3D
Conscious
perception
+time
Simulation
+ complex
models
(“priors”)
Conscious
perception
Zzzzzz…..
Seeing is dreaming
Conscious
perception
Seeing is believing
3D
+time
Simulation
+ complex
models
(“priors”)
Conscious
perception
Our most
integrated
Prediction
perception
of the world
errors
But ultimately, only actions matter.
Conscious
perception
Prediction
Prediction
Goals
Actions
Goals
Conscious
perception
errors
Physiology
Actions
Organs