Architecture, networks,
robustness, and complexity
Jean Carlson and John Doyle
UCSB and Caltech
Kick-off Meeting, July 28, 2008
ONR MURI: NexGenNetSci
My
interests
Multiscale
Physics
Network
Centric,
Pervasive,
Embedded,
Ubiquitous
Core
theory
challenges
Sustainability?
Systems
Biology &
Medicine
Good news:
Spectacular progress
Bad news:
• Persistent errors
and confusion
• Potentially
insurmountable
obstacles?
Challenges in the NS report:
1. Dynamics, spatial location, and information
propagation in networks.
2. Modeling and analysis of very large networks.
3. Design and synthesis of networks.
4. Increasing the level of rigor and mathematical
structure.
5. Abstracting common concepts across fields.
6. Better experiments and measurements of network
structure.
7. Robustness and security of networks.
These challenges naturally fit into
several distinct categories.
Goals
•
Abstraction (common concepts across fields)
•
Rigor (& math structure)
Issues
•
Dynamics (location, propagation)
•
Robustness (& security)
Levels of understanding
0. Verbal (& cartoons)
1. Data & statistics (Experiments & measurements)
2. Modeling & simulation
3. Analysis
4. Design & synthesis
Goals
• Abstraction
• Rigor
Issues
• Dynamics
• Robustness
Levels
0. Verbal
1. Data & stats
2. Modeling & sim
3. Analysis
4. Design & synth
Good news:
Spectacular progress
Bad news:
• Persistent errors
and confusion
• Potentially
insurmountable
obstacles?
Goals
• Abstraction
• Rigor
Issues
• Dynamics
• Robustness
Levels
0. Verbal
1. Data & stats
2. Modeling & sim
3. Analysis
4. Design & synth
• Requires engaging all subfields
• Can be done
• If properly motivated.
First focused on
these limited levels
of understanding
Goals
• Abstraction
• Rigor
Issues
• Dynamics
• Robustness
Levels
0. Verbal
1. Data & stats
2. Modeling & sim
3. Analysis
4. Design & synth
Toughest challenges
Rigorous treatment
of dynamics and
robustness
at these levels of
understanding
is both the greatest
need and the most
inadequately
addressed.
Goals
• Abstraction
• Rigor
Issues
• Dynamics
• Robustness
Levels
0. Verbal
1. Data & stats
2. Modeling & sim
3. Analysis
4. Design & synth
Theory and the Internet
Good news:
Spectacular progress
Topics:
• Traffic
• Topology
• Control and
dynamics (C&D)
• Layering/distributed
• Architecture
Theory and the Internet
Traffic Topology
Verbal
Data/stat
Mod/sim
Analysis
Synthesis
C&D
Layering Architect.
Goals
• Abstraction
• Rigor
Issues
• Dynamics
• Robustness
Levels
0. Verbal
1. Data & stats
2. Modeling & sim
3. Analysis
4. Design & synth
Early Internet “pre-theory”
• Original design had
principles but no math
“theory” per se
• Good engineers
always lead theory
(and science)
• Theory adds depth,
rigor, scalability, etc
Goals
• Abstraction
• Rigor
Issues
• Dynamics
• Robustness
Levels
0. Verbal
1. Data & stats
2. Modeling & sim
3. Analysis
4. Design & synth
Early theory
(post deployment)
• Measurement
• Statistics
• Packet-level dynamic
simulation
Traffic (1993-2000)
Traffic
Verbal
Data/stat
Mod/sim
Analysis
Synthesis
• Heavy tails (HT) in net
traffic???
• Careful measurements
• Appropriate statistics
• Connecting traffic to
application behavior
• “optimal” web layout
 HT files
 HT traffic
Unnecessary confusion
(and its resolution)
Traffic Topology
Verbal
Data/stat
Mod/sim
Analysis
Synthesis
One of
the most-read
papers ever on
the Internet!
2
10
High degree hublike core
Low degree
mesh-like core
1
identical
power-law
degrees
10
Completely different
networks can have the
same node degrees.
0
10
0
10
1
10
2
10
3
10
• Low degree core
• High performance
and robustness
• Efficient, economic
• High degree “hubs”
• Poor performance
and robustness
• Wasteful, expensive
Nothing like the
real Internet.
See PNAS, Sigcomm, TransNet papers for details.
Much
“network science”
and
“complex systems”
literature
is equally
specious
Network
technology
Network
science?
physics
Why is this “science” so bad?
The Internet hourglass
my
computer
router
router
HTTP
TCP
IP
LINK
web
server
my
computer
Diverse
router
router
web
server
HTTP
TCP
IP
LINK
Diverse
Diverse
Application
Highly
conserved
core
control
processes
TCP/AQM
IP
Layered
MAC
Physical
Diverse
Diverse
Application
TCP/AQM
Highly
conservedHidden to most
core
usersIPand
control technologies
processes
Layered
MAC
Physical
Diverse
fan-in
of diverse
inputs
universal
carriers
Bowties: flows
within layers
fan-out
of diverse
outputs
Diverse
function
Universal
Control
Diverse
components
Hourglass:
layering of
control
Constraints
that
deconstrain
Variety
of
files
packets
Applications
TCP
IP
Link
Variety
of
files
Many bowties
in Internet
Huge and recent progress
Traffic
Verbal
Data/stat
Mod/sim
Analysis
Synthesis
Topology
C&D
Layering Architect.
Layering as optimization decomposition
• Lijun Chen
(Caltech)
• Steven Low
(Caltech)
• Mung Chiang
(Princeton)
• Frank Kelly
(Cambridge)
application
transport
network
link
physical
Integrating:
• TCP/AQM/IP
• Mac layer
• Scheduling/routing
• Game theory
• Network coding
(with Tracey Ho)
max
x0
U ( x )
i
i
i
subj to Rx  c( p )
x X
Layering as optimization decomposition
• Each layer is abstracted as an optimization
problem
• Operation of a layer is a distributed solution
• Results of one problem (layer) are parameters of
others
• Operate at different timescales
Application: utility
application
transport
network
link
physical
max
x0
U ( x )
i
i
i
Phy: power
subj to Rx  c( p )
x X
IP: routing
Link: scheduling
Examples
Optimal web layer: Zhu, Yu, Doyle ’01
HTTP/TCP: Chang, Liu ’04
application
transport
network
link
physical
TCP: Kelly, Maulloo, Tan ’98, ……
TCP/IP: Wang et al ’05, ……
TCP/MAC: Chen et al ’05, ……
TCP/power control: Xiao et al ’01,
Chiang ’04, ……
Rate control/routing/scheduling: Eryilmax et al ’05,
Lin et al ’05, Neely, et al ’05, Stolyar ’05
detailed survey in Proc. of IEEE, 2007
Caltech FAST Project (2001 – 08)
theory
experiment
prototype
deployment
testbed
Control & optimization of networks
ITR ANI-0113425, STI ANI-0230967
EIA-0303620, CNS-0435520
DAAD19-02-1-0283, W911NF-04-1-0095
F49620-03-1-0119
NMS Program
Collaborators:
UCLA, StarLight
CERN, SLAC
PSC, LANL
Internet2
Cisco, Level3
Corning
Commercial Deployment: FAST in a box
Internet
FastSoft
Aria
(FAST)
Throuput: LA  Tokyo
Throuput: San Fran  MIT
20000
FTP throughput (kbps)
18000
1.8x
3.8x
6.3x
17.6x 21.8x
28.1x
32.5x
16000
14000
FAST avg:
233Mbps
12000
10000
8000
6000
4000
2000
0
50x
0.1
delays
0.5
are
with Aria
1
5
common
over
File size (MB)
10
DSL
20
speed
60
links
without Aria
Reno avg:
35Mbps
Integrating network coding
 With rate control (Chen et al ’07)
max
m
m
x r , yl
s.t.
Information flow
m
U
(
x
)
 m
m
Network coding comes into
action through this constraint
H lrm x rm  y lm

y lm  c l
Physical flow
m
 With routing and scheduling (Cui-Chen-Ho ’07, …)
 Distributed data gathering in sensor networks (Cui-
Chen-Ho ’07)
 Guide the design of inter-session network coding (CuiChen-Ho ’07)
Connections
Traffic Topology
Verbal
Data/stat
Mod/sim
Analysis
Synthesis
C&D
Layering Architect.
Generalized “coding” problems
• Optimizing d-1 dimensional
cuts in d dimensional spaces…
• To minimize average size of
files or fires, subject to resource
constraint.
• Models of greatly varying detail
all give a consistent story.
Data compression
Web
Fires
Data
6
DC
5
WWW
4
3
FF
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
Data + Model/Theory
6
DC
5
WWW
4
3
FF
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
HOT
Fires
Wildfire Simulations:
• Agreement between (1) fire frequency vs. size
data, (2) abstract HOT models (robustness
tradeoffs) and (3) detailed HFire simulations
(topography, weather, fuel)
4
HOT+
Data+
HFire
10
Cumulative
P(size)
Makes this an ideal, ongoing prototype for
development of fundamental themes:
• Dynamics and Feedback
• Robust, yet Fragile
• Multiscale/Multiresolution modeling and
HFire
simulation
• Ecological mechanisms for adaptation,
selection, resilience, and complexity
• Hazards, policy, social and economic impact
4 Science data sets
+Los Padres Forest
+ HFire Simulation
3
10
HOT
2
10
1
10
0
10
10
-4
-3
10
-2
10
-1
10
1
0
10
10
size
Router
queues
Mice
Sources
Network
Elephants
Router
queues
Mice
Delay sensitive
Sources
Network
Bandwidth
sensitive
Elephants
Unfortunate interaction
of files with congestion
control
Router
queues
Mice
Delay sensitive
Better
Control
Sources
Network
Bandwidth
sensitive
Elephants
Fortunate interaction of
files with improved
congestion control
Diverse case studies
•
•
•
•
•
•
•
Internet and extensions
Ecosystems (e.g. CA coastal wildfire)
Cell/systems biology
Biomedical and physiology
Earthquake dynamics and statistics
Disasters statistics
Infrastructure: Electric power, Transportation,
Manufacturing
• Toy example: Lego
• Multiscale physics (Friction, Fracture,
Turbulence, Granular Flows, Foundations of
Statistical mechanics)
Biomedical case studies
• Immune system dynamics and senescence
• Brain imaging and modeling
– source-localization for cognitive network
determination via joint inversion of human subject
EEG and fMRI
– “grid” cortical algorithms for navigation in rats.
•
•
•
•
Exercise/training physiology
Coagulation, inflammation, trauma
Addiction
Diabetes
So what is “network science”?
•
•
What is science?
Subdomains:
–
–
–
–
–
•
Physics
Biology
Social Sciences
Math
Technology
Enormous methodological differences (that have
grown with time)
What is “network science”?
•
Already exist distinct subdomains:
–
–
–
–
–
•
•
Network Physics (NetPhys)
Network Biology (NetBio)
Network Social Sciences (NetSoc)
Network Math (NetMath)
Network Technology (NetTech)
Even greater methodological differences
Spectacular but uneven progress
Goals
• Abstraction
• Rigor
Issues
• Dynamics
• Robustness
Levels
0. Verbal
1. Data & stats
2. Modeling & sim
3. Analysis
4. Design & synth
•
Abstractions (common
concepts across fields)
•
“Network science” has
no broadly agreed upon
definition, applications,
or methods
Challenge is to develop
abstractions that can
connect common
concepts across fields
that otherwise are very
different
•
The good and bad news
•
•
•
•
Network technology has been wildly successful…
… yielding a “networked planet” for energy, food,
information, goods and materials,…
… but contributing to daunting sustainability challenges
(… that only further network technology can solve?)
•
“Network centric technologies”
– Demo and build almost anything we can envision
– But fail because they create new problems we did not
expect.
•
But most of “network science” is utterly disconnected
from these successes and challenges
The structure of scientific explanation
•
•
Different sciences use these unevenly
and in very different ways.
“Network science” is more extreme.
•
Levels
0. Verbal
1. Data & stats
2. Modeling & sim
3. Analysis
4. Design & synth
•
Network science will
demand a
fundamental
rethinking of their
roles, particularly
proofs in analysis
and synthesis.
“Tower of Babel”
must be avoided.
The “Tower”
NetTech: global infrastructure, Internet, wireless, energy,
transportation, supply chain…, silicon, fiber, …, network
centric, ubiquitous, embedded-everywhere, nano-,…
NetBio: systems biology, genomics, *omics, evo-devo,
bioinformatics, …, metabolic, transcriptional, signal
transduction, regulatory,…, medicine, epidemiology,
emerging infections, medical disaster management,
integrative physiology, ecology, …
NetPhys: multiscale, turbulence, chaos, stat mech,…, fractals,
criticality, self-similarity, scale-free, cellular automata,
universality, edge-of-chaos, renormalization,…
NetSoc: economics, sociology, psychology, politics,
management, law, ecology… , scaling, small worlds…
NetMath: graph, information, control, dynamical systems,
complexity, formal methods, optimization,…
Biology
Evolution of the Internet
Circuits, Pathways, Molecules,
Two foci:
• Components (VLSI, optics, web apps …)
– Rapid change, diversity, and rearrangement
• Architecture (TCP/IP)
– Largely fixed and universal
Architecture of the cell
Goals
• Abstraction
• Rigor
Issues
• Dynamics
• Robustness
Levels
0. Verbal
1. Data & stats
2. Modeling & sim
3. Analysis
4. Design & synth
Essential ideas
Robust
yet
fragile
Constraints
that
deconstrain
Human complexity
Robust
Yet Fragile
 Efficient, flexible metabolism  Obesity and diabetes
 Rich microbial symbionts and  Parasites, infection
 Immune systems
 Inflammation, Auto-Im.
 Regeneration & renewal
 Cancer
 Complex societies
 Epidemics, war, …
 Advanced technologies
 Catastrophic failures
Robust






Implications/
Generalizations
Efficient, flexible metabolism
Rich microbial symbionts and
Immune systems
Regeneration & renewal
Complex societies
Advanced technologies






Yet Fragile
Obesity and diabetes
Parasites, infection
Inflammation, Auto-Im.
Cancer
Epidemics, war, …
Catastrophic failures
• Complexity driven by robust/fragile
• More than minimal functionality
• It’s easier to create robustness
• but much harder to avoid fragility
• (and easiest of all to create fragility)
• New robust/fragile conservation laws
Robust yet fragile
Biology (and advanced tech) show extremes
• Robust Yet Fragile
• Simplicity and complexity
• Unity and diversity
• Evolvable and frozen
What makes this possible and/ or inevitable?
Architecture (= constraints)
Let’s dig deeper.
fan-in
of diverse
inputs
universal
carriers
Bowties: flows
within layers
fan-out
of diverse
outputs
Diverse
function
Universal
Control
Diverse
components
Hourglass:
layering of
control
Constraints
that
deconstrain
Peter Sterling and Allostasis
food
intake
Blood
Glucose
Oxygen
Amino acids
Fatty acids
Universal metabolic system
Organs
Tissues
Cells
Molecules
Highly
variable
supply
Robust
Highly
variable
demand
food
intake
Efficient
evolving
diet
Evolvable
Organs
Tissues
Cells
Molecules
evolving
function
Highly
variable
supply
food
intake
Conserved
core
building
blocks
Glucose
Oxygen
Blood
evolving
diet
Highly
variable
demand
Organs
Tissues
Cells
Molecules
evolving
function
Universal reward systems
sports
music
dance
crafts
art
toolmaking
sex
food
Prefrontal
VTA dopamine cortex
Accumbens
Dopamine,
Ghrelin,
Leptin,…
Universal reward systems
sports
music
dance
crafts
art
toolmaking
sex
food
PFC
CG
OFC
NAcc
VTA dopamine Amyg
STR
TH PIT
HIP
SN
Robust and evolvable
Reward
Drive
Control
Memory
sports
Universal reward systems
music
dance
crafts
VTA dopamine
art
toolmaking
Constraints
sex
that
food
deconstrain
Blood
Glucose
Oxygen
Universal metabolic system
Reward
Drive
Control
Memory
Organs
Tissues
Cells
Molecules
Constraints
dopamine
Blood
Glucose
Oxygen
sports
music
dance
crafts
art
toolmaking
sex
food
Reward
Drive
Control
Memory
that
deconstrain
Organs
Tissues
Cells
Molecules
Constraints: too much or too little?
Universal reward/metabolic systems
work
family
community
nature
food
sex
toolmaking
sports
music
dance
crafts
art
dopamine
Blood
Reward
Drive
Control
Memory
Organs
Tissues
Cells
Molecules
Robust and adaptive, yet …
work
family
community
nature
sex
food
toolmaking
sports
music
dance
crafts
art
cocaine
amphetamine
dopamine
Blood
Reward
Drive
Control
Memory
Organs
Tissues
Cells
Molecules
work
family
community
nature
market/
consumer
culture
money
salt
sugar/fat
nicotine
alcohol
Reward
Drive
Control
Memory
dopamine
Vicarious
sex
toolmaking
sports
music
dance
crafts
art
industrial
agriculture
Organs
Tissues
Cells
Molecules
high
sodium
hypertension
obesity
atherosclerosis
overwork
diabetes
smoking
inflammation
money
salt
sugar/fat
nicotine
alcohol
Vicarious
alcoholism
drug
abuse
immune
suppression
coronary,
cerebrovascular,
renovascular
cancer
cirrhosis
accidents/
homicide/
suicide
sports
Universal reward systems
music
Prefrontal
dance
cortex
crafts
VTA dopamine
art
Accumbens
toolmaking
sex
food
Blood
Glucose
Oxygen
Universal metabolic system
Organs
Tissues
Cells
Molecules
hypertension
high
sodium
money
obesity
VTA
salt
sugar/fat
nicotine
alcohol
Vicarious
atherosclerosis
dopamine
Yet Fragile
overwork
diabetes
smoking
inflammation
alcoholism
Glucose
Oxygen
drug
abuse
immune
suppression
coronary,
cerebrovascular,
renovascular
cancer
cirrhosis
accidents/
homicide/
suicide
Goals
• Abstraction
• Rigor
Issues
• Dynamics
• Robustness
Levels
0. Verbal
1. Data & stats
2. Modeling & sim
3. Analysis
4. Design & synth
Essential ideas
Robust
yet
fragile
Constraints
that
deconstrain
RYF Immune System (Carlson, Stromberg, …)
• A quantitative understanding of the immune system’s
threat/non-threat discrimination
• Understanding failures of the discrimination system via
exploitation of weaknesses by diseases, and
misrecognizing non-threats as threats, e.g. allergy,
autoimmune.
• Model of Immune System Resource Allocation:
lymphocytes as resources for hazard aversion.
• Overspecialization with age leads to robustness to
common infectious diseases through memory cells, but
fragility through depletion of naïve cells used to fight new
and rare infectious diseases.
• Antigen in green or gold is a pathogen
and attacked.
• Blue is self-antigens. If green encircles
them autoimmune disease is possible.
• Red is pathogen.
• Some are close to self-antigens and
shielded from the immune system by
tolerance mechanisms.
Aging and overspecialization:
Blue plot (top) shows trend
towards lower infection severity
while rare events become larger.