Architecture, networks, robustness, and complexity Jean Carlson and John Doyle UCSB and Caltech Kick-off Meeting, July 28, 2008 ONR MURI: NexGenNetSci My interests Multiscale Physics Network Centric, Pervasive, Embedded, Ubiquitous Core theory challenges Sustainability? Systems Biology & Medicine Good news: Spectacular progress Bad news: • Persistent errors and confusion • Potentially insurmountable obstacles? Challenges in the NS report: 1. Dynamics, spatial location, and information propagation in networks. 2. Modeling and analysis of very large networks. 3. Design and synthesis of networks. 4. Increasing the level of rigor and mathematical structure. 5. Abstracting common concepts across fields. 6. Better experiments and measurements of network structure. 7. Robustness and security of networks. These challenges naturally fit into several distinct categories. Goals • Abstraction (common concepts across fields) • Rigor (& math structure) Issues • Dynamics (location, propagation) • Robustness (& security) Levels of understanding 0. Verbal (& cartoons) 1. Data & statistics (Experiments & measurements) 2. Modeling & simulation 3. Analysis 4. Design & synthesis Goals • Abstraction • Rigor Issues • Dynamics • Robustness Levels 0. Verbal 1. Data & stats 2. Modeling & sim 3. Analysis 4. Design & synth Good news: Spectacular progress Bad news: • Persistent errors and confusion • Potentially insurmountable obstacles? Goals • Abstraction • Rigor Issues • Dynamics • Robustness Levels 0. Verbal 1. Data & stats 2. Modeling & sim 3. Analysis 4. Design & synth • Requires engaging all subfields • Can be done • If properly motivated. First focused on these limited levels of understanding Goals • Abstraction • Rigor Issues • Dynamics • Robustness Levels 0. Verbal 1. Data & stats 2. Modeling & sim 3. Analysis 4. Design & synth Toughest challenges Rigorous treatment of dynamics and robustness at these levels of understanding is both the greatest need and the most inadequately addressed. Goals • Abstraction • Rigor Issues • Dynamics • Robustness Levels 0. Verbal 1. Data & stats 2. Modeling & sim 3. Analysis 4. Design & synth Theory and the Internet Good news: Spectacular progress Topics: • Traffic • Topology • Control and dynamics (C&D) • Layering/distributed • Architecture Theory and the Internet Traffic Topology Verbal Data/stat Mod/sim Analysis Synthesis C&D Layering Architect. Goals • Abstraction • Rigor Issues • Dynamics • Robustness Levels 0. Verbal 1. Data & stats 2. Modeling & sim 3. Analysis 4. Design & synth Early Internet “pre-theory” • Original design had principles but no math “theory” per se • Good engineers always lead theory (and science) • Theory adds depth, rigor, scalability, etc Goals • Abstraction • Rigor Issues • Dynamics • Robustness Levels 0. Verbal 1. Data & stats 2. Modeling & sim 3. Analysis 4. Design & synth Early theory (post deployment) • Measurement • Statistics • Packet-level dynamic simulation Traffic (1993-2000) Traffic Verbal Data/stat Mod/sim Analysis Synthesis • Heavy tails (HT) in net traffic??? • Careful measurements • Appropriate statistics • Connecting traffic to application behavior • “optimal” web layout HT files HT traffic Unnecessary confusion (and its resolution) Traffic Topology Verbal Data/stat Mod/sim Analysis Synthesis One of the most-read papers ever on the Internet! 2 10 High degree hublike core Low degree mesh-like core 1 identical power-law degrees 10 Completely different networks can have the same node degrees. 0 10 0 10 1 10 2 10 3 10 • Low degree core • High performance and robustness • Efficient, economic • High degree “hubs” • Poor performance and robustness • Wasteful, expensive Nothing like the real Internet. See PNAS, Sigcomm, TransNet papers for details. Much “network science” and “complex systems” literature is equally specious Network technology Network science? physics Why is this “science” so bad? The Internet hourglass my computer router router HTTP TCP IP LINK web server my computer Diverse router router web server HTTP TCP IP LINK Diverse Diverse Application Highly conserved core control processes TCP/AQM IP Layered MAC Physical Diverse Diverse Application TCP/AQM Highly conservedHidden to most core usersIPand control technologies processes Layered MAC Physical Diverse fan-in of diverse inputs universal carriers Bowties: flows within layers fan-out of diverse outputs Diverse function Universal Control Diverse components Hourglass: layering of control Constraints that deconstrain Variety of files packets Applications TCP IP Link Variety of files Many bowties in Internet Huge and recent progress Traffic Verbal Data/stat Mod/sim Analysis Synthesis Topology C&D Layering Architect. Layering as optimization decomposition • Lijun Chen (Caltech) • Steven Low (Caltech) • Mung Chiang (Princeton) • Frank Kelly (Cambridge) application transport network link physical Integrating: • TCP/AQM/IP • Mac layer • Scheduling/routing • Game theory • Network coding (with Tracey Ho) max x0 U ( x ) i i i subj to Rx c( p ) x X Layering as optimization decomposition • Each layer is abstracted as an optimization problem • Operation of a layer is a distributed solution • Results of one problem (layer) are parameters of others • Operate at different timescales Application: utility application transport network link physical max x0 U ( x ) i i i Phy: power subj to Rx c( p ) x X IP: routing Link: scheduling Examples Optimal web layer: Zhu, Yu, Doyle ’01 HTTP/TCP: Chang, Liu ’04 application transport network link physical TCP: Kelly, Maulloo, Tan ’98, …… TCP/IP: Wang et al ’05, …… TCP/MAC: Chen et al ’05, …… TCP/power control: Xiao et al ’01, Chiang ’04, …… Rate control/routing/scheduling: Eryilmax et al ’05, Lin et al ’05, Neely, et al ’05, Stolyar ’05 detailed survey in Proc. of IEEE, 2007 Caltech FAST Project (2001 – 08) theory experiment prototype deployment testbed Control & optimization of networks ITR ANI-0113425, STI ANI-0230967 EIA-0303620, CNS-0435520 DAAD19-02-1-0283, W911NF-04-1-0095 F49620-03-1-0119 NMS Program Collaborators: UCLA, StarLight CERN, SLAC PSC, LANL Internet2 Cisco, Level3 Corning Commercial Deployment: FAST in a box Internet FastSoft Aria (FAST) Throuput: LA Tokyo Throuput: San Fran MIT 20000 FTP throughput (kbps) 18000 1.8x 3.8x 6.3x 17.6x 21.8x 28.1x 32.5x 16000 14000 FAST avg: 233Mbps 12000 10000 8000 6000 4000 2000 0 50x 0.1 delays 0.5 are with Aria 1 5 common over File size (MB) 10 DSL 20 speed 60 links without Aria Reno avg: 35Mbps Integrating network coding With rate control (Chen et al ’07) max m m x r , yl s.t. Information flow m U ( x ) m m Network coding comes into action through this constraint H lrm x rm y lm y lm c l Physical flow m With routing and scheduling (Cui-Chen-Ho ’07, …) Distributed data gathering in sensor networks (Cui- Chen-Ho ’07) Guide the design of inter-session network coding (CuiChen-Ho ’07) Connections Traffic Topology Verbal Data/stat Mod/sim Analysis Synthesis C&D Layering Architect. Generalized “coding” problems • Optimizing d-1 dimensional cuts in d dimensional spaces… • To minimize average size of files or fires, subject to resource constraint. • Models of greatly varying detail all give a consistent story. Data compression Web Fires Data 6 DC 5 WWW 4 3 FF 2 1 0 -1 -6 -5 -4 -3 -2 -1 0 1 2 Data + Model/Theory 6 DC 5 WWW 4 3 FF 2 1 0 -1 -6 -5 -4 -3 -2 -1 0 1 2 HOT Fires Wildfire Simulations: • Agreement between (1) fire frequency vs. size data, (2) abstract HOT models (robustness tradeoffs) and (3) detailed HFire simulations (topography, weather, fuel) 4 HOT+ Data+ HFire 10 Cumulative P(size) Makes this an ideal, ongoing prototype for development of fundamental themes: • Dynamics and Feedback • Robust, yet Fragile • Multiscale/Multiresolution modeling and HFire simulation • Ecological mechanisms for adaptation, selection, resilience, and complexity • Hazards, policy, social and economic impact 4 Science data sets +Los Padres Forest + HFire Simulation 3 10 HOT 2 10 1 10 0 10 10 -4 -3 10 -2 10 -1 10 1 0 10 10 size Router queues Mice Sources Network Elephants Router queues Mice Delay sensitive Sources Network Bandwidth sensitive Elephants Unfortunate interaction of files with congestion control Router queues Mice Delay sensitive Better Control Sources Network Bandwidth sensitive Elephants Fortunate interaction of files with improved congestion control Diverse case studies • • • • • • • Internet and extensions Ecosystems (e.g. CA coastal wildfire) Cell/systems biology Biomedical and physiology Earthquake dynamics and statistics Disasters statistics Infrastructure: Electric power, Transportation, Manufacturing • Toy example: Lego • Multiscale physics (Friction, Fracture, Turbulence, Granular Flows, Foundations of Statistical mechanics) Biomedical case studies • Immune system dynamics and senescence • Brain imaging and modeling – source-localization for cognitive network determination via joint inversion of human subject EEG and fMRI – “grid” cortical algorithms for navigation in rats. • • • • Exercise/training physiology Coagulation, inflammation, trauma Addiction Diabetes So what is “network science”? • • What is science? Subdomains: – – – – – • Physics Biology Social Sciences Math Technology Enormous methodological differences (that have grown with time) What is “network science”? • Already exist distinct subdomains: – – – – – • • Network Physics (NetPhys) Network Biology (NetBio) Network Social Sciences (NetSoc) Network Math (NetMath) Network Technology (NetTech) Even greater methodological differences Spectacular but uneven progress Goals • Abstraction • Rigor Issues • Dynamics • Robustness Levels 0. Verbal 1. Data & stats 2. Modeling & sim 3. Analysis 4. Design & synth • Abstractions (common concepts across fields) • “Network science” has no broadly agreed upon definition, applications, or methods Challenge is to develop abstractions that can connect common concepts across fields that otherwise are very different • The good and bad news • • • • Network technology has been wildly successful… … yielding a “networked planet” for energy, food, information, goods and materials,… … but contributing to daunting sustainability challenges (… that only further network technology can solve?) • “Network centric technologies” – Demo and build almost anything we can envision – But fail because they create new problems we did not expect. • But most of “network science” is utterly disconnected from these successes and challenges The structure of scientific explanation • • Different sciences use these unevenly and in very different ways. “Network science” is more extreme. • Levels 0. Verbal 1. Data & stats 2. Modeling & sim 3. Analysis 4. Design & synth • Network science will demand a fundamental rethinking of their roles, particularly proofs in analysis and synthesis. “Tower of Babel” must be avoided. The “Tower” NetTech: global infrastructure, Internet, wireless, energy, transportation, supply chain…, silicon, fiber, …, network centric, ubiquitous, embedded-everywhere, nano-,… NetBio: systems biology, genomics, *omics, evo-devo, bioinformatics, …, metabolic, transcriptional, signal transduction, regulatory,…, medicine, epidemiology, emerging infections, medical disaster management, integrative physiology, ecology, … NetPhys: multiscale, turbulence, chaos, stat mech,…, fractals, criticality, self-similarity, scale-free, cellular automata, universality, edge-of-chaos, renormalization,… NetSoc: economics, sociology, psychology, politics, management, law, ecology… , scaling, small worlds… NetMath: graph, information, control, dynamical systems, complexity, formal methods, optimization,… Biology Evolution of the Internet Circuits, Pathways, Molecules, Two foci: • Components (VLSI, optics, web apps …) – Rapid change, diversity, and rearrangement • Architecture (TCP/IP) – Largely fixed and universal Architecture of the cell Goals • Abstraction • Rigor Issues • Dynamics • Robustness Levels 0. Verbal 1. Data & stats 2. Modeling & sim 3. Analysis 4. Design & synth Essential ideas Robust yet fragile Constraints that deconstrain Human complexity Robust Yet Fragile Efficient, flexible metabolism Obesity and diabetes Rich microbial symbionts and Parasites, infection Immune systems Inflammation, Auto-Im. Regeneration & renewal Cancer Complex societies Epidemics, war, … Advanced technologies Catastrophic failures Robust Implications/ Generalizations Efficient, flexible metabolism Rich microbial symbionts and Immune systems Regeneration & renewal Complex societies Advanced technologies Yet Fragile Obesity and diabetes Parasites, infection Inflammation, Auto-Im. Cancer Epidemics, war, … Catastrophic failures • Complexity driven by robust/fragile • More than minimal functionality • It’s easier to create robustness • but much harder to avoid fragility • (and easiest of all to create fragility) • New robust/fragile conservation laws Robust yet fragile Biology (and advanced tech) show extremes • Robust Yet Fragile • Simplicity and complexity • Unity and diversity • Evolvable and frozen What makes this possible and/ or inevitable? Architecture (= constraints) Let’s dig deeper. fan-in of diverse inputs universal carriers Bowties: flows within layers fan-out of diverse outputs Diverse function Universal Control Diverse components Hourglass: layering of control Constraints that deconstrain Peter Sterling and Allostasis food intake Blood Glucose Oxygen Amino acids Fatty acids Universal metabolic system Organs Tissues Cells Molecules Highly variable supply Robust Highly variable demand food intake Efficient evolving diet Evolvable Organs Tissues Cells Molecules evolving function Highly variable supply food intake Conserved core building blocks Glucose Oxygen Blood evolving diet Highly variable demand Organs Tissues Cells Molecules evolving function Universal reward systems sports music dance crafts art toolmaking sex food Prefrontal VTA dopamine cortex Accumbens Dopamine, Ghrelin, Leptin,… Universal reward systems sports music dance crafts art toolmaking sex food PFC CG OFC NAcc VTA dopamine Amyg STR TH PIT HIP SN Robust and evolvable Reward Drive Control Memory sports Universal reward systems music dance crafts VTA dopamine art toolmaking Constraints sex that food deconstrain Blood Glucose Oxygen Universal metabolic system Reward Drive Control Memory Organs Tissues Cells Molecules Constraints dopamine Blood Glucose Oxygen sports music dance crafts art toolmaking sex food Reward Drive Control Memory that deconstrain Organs Tissues Cells Molecules Constraints: too much or too little? Universal reward/metabolic systems work family community nature food sex toolmaking sports music dance crafts art dopamine Blood Reward Drive Control Memory Organs Tissues Cells Molecules Robust and adaptive, yet … work family community nature sex food toolmaking sports music dance crafts art cocaine amphetamine dopamine Blood Reward Drive Control Memory Organs Tissues Cells Molecules work family community nature market/ consumer culture money salt sugar/fat nicotine alcohol Reward Drive Control Memory dopamine Vicarious sex toolmaking sports music dance crafts art industrial agriculture Organs Tissues Cells Molecules high sodium hypertension obesity atherosclerosis overwork diabetes smoking inflammation money salt sugar/fat nicotine alcohol Vicarious alcoholism drug abuse immune suppression coronary, cerebrovascular, renovascular cancer cirrhosis accidents/ homicide/ suicide sports Universal reward systems music Prefrontal dance cortex crafts VTA dopamine art Accumbens toolmaking sex food Blood Glucose Oxygen Universal metabolic system Organs Tissues Cells Molecules hypertension high sodium money obesity VTA salt sugar/fat nicotine alcohol Vicarious atherosclerosis dopamine Yet Fragile overwork diabetes smoking inflammation alcoholism Glucose Oxygen drug abuse immune suppression coronary, cerebrovascular, renovascular cancer cirrhosis accidents/ homicide/ suicide Goals • Abstraction • Rigor Issues • Dynamics • Robustness Levels 0. Verbal 1. Data & stats 2. Modeling & sim 3. Analysis 4. Design & synth Essential ideas Robust yet fragile Constraints that deconstrain RYF Immune System (Carlson, Stromberg, …) • A quantitative understanding of the immune system’s threat/non-threat discrimination • Understanding failures of the discrimination system via exploitation of weaknesses by diseases, and misrecognizing non-threats as threats, e.g. allergy, autoimmune. • Model of Immune System Resource Allocation: lymphocytes as resources for hazard aversion. • Overspecialization with age leads to robustness to common infectious diseases through memory cells, but fragility through depletion of naïve cells used to fight new and rare infectious diseases. • Antigen in green or gold is a pathogen and attacked. • Blue is self-antigens. If green encircles them autoimmune disease is possible. • Red is pathogen. • Some are close to self-antigens and shielded from the immune system by tolerance mechanisms. Aging and overspecialization: Blue plot (top) shows trend towards lower infection severity while rare events become larger.