University of Southern California Center for Systems and Software Engineering From Dependable Architectures To Dependable Systems Nenad Medvidovic Center for Systems and Software Engineering Computer Science Department Viterbi School of Engineering University of Southern California Los Angeles, U.S.A. neno@usc.edu University of Southern California Center for Systems and Software Engineering Goals of the Talk • • • • Identify some challenges Suggest some solutions Motivate future research Invite dissenting opinions University of Southern California Center for Systems and Software Engineering Goals of the Talk The problem is this big and sometimes ill-defined University of Southern California Center for Systems and Software Engineering Goals of the Talk I will talk to you about this much of it University of Southern California Center for Systems and Software Engineering Goals of the Talk I will suggest a solution for this much of it University of Southern California Center for Systems and Software Engineering Goals of the Talk But, hopefully, we will have this much fun! University of Southern California Center for Systems and Software Engineering What Is Dependability? • Degree of user confidence that the system will operate as expected • Key dimensions – – – – Availability Reliability Security Safety • But also – – – – Repairability Maintainability Survivability Fault tolerance University of Southern California Center for Systems and Software Engineering What Is Architecture? • A high-level model of a system – The system’s “blueprint” • Represents system organization – – – – Data Computation Interaction Structure • Embodies system properties – Communication integrity, performance, throughput, liveness, . . . Can/does it embody dependability? (how) Can those properties be transferred to the system itself? University of Southern California Center for Systems and Software Engineering A “Traditional” Architectural Model University of Southern California Center for Systems and Software Engineering A “Traditional” Architectural Model What/where is the dependability? University of Southern California Center for Systems and Software Engineering A “Standard” Architectural Model FileResource DirectoryResource (from resourc... (from resourc... SampleResourceIndexer ResourceFilter (from index... (from resources) HtmlStyle HttpTokenList (from ht... (from ht... ProtocolFrame HttpBag ArrayDictionary (from ht... (from ut... HtmlHead (from pro... PICS SSIStream DebugThread (from s... (from sock... (from ht... IPMatcher (from auth) ProxyRequestObserver (from a... (from pi... CGIHeaderHolder (from fram... HttpMimeType (from frames) HTTPPrincipal Segment (from ssi) VariantState (from NegotiatedFra... SocketOutputBuffer SSIFrame AuthUserPrincipal (from sock... (from s... (from a... MimeParser (from ht... (from mi... ResourceReference MimeType HttpManager FramedResource ExternalContainer (from ht... (from resourc... (from resourc... (from resources) (from comman... ServerHandlerManager HtmlGenerator HttpEntityTag PropertySet (from ht... (from ht... (from conf... (from daem... ResourceStoreManager IPMatcherNode (from sto... MimeClientFactory (from ht... IndexersCatalog (from index... (from au... CCPPRequest (from cc... ThreadCache HttpCredential AdminContext (from admin) ResourceIndexer MuxSession (from index... (from m... HttpExtList (from ht... AttributeDescription (from serializati... httpd (from servl... ResourceContext SocketClientFactoryStats (from resourc... (from sock... (from http) RequestTimeout (from ht... EventManager MuxClientFactory (from adm... (from adm... (from time... (from m... ResourceDescription (from serializati... ProfileRef (from cc... JigsawHttpSessionContext (from serializati... PlainRemoteResource ResourceException (from resourc... (from ut... (from ht... AdminWriter (from webd... (from http) CommandRegistry (from mime) Serializer DAVMimeClientFactory Client Shuffler (from ht... SampleMuxHandler Stresser (from m... (from tes... University of Southern California Center for Systems and Software Engineering A “Standard” Architectural Model HTTPD : WebBrowser : SocketClient Resources : httpd : DirectoryResource : ContainerResource : FramedResource processRequest(Request) perform(RequestInterface) : LookupResult <<create>> lookup(LookupState, LookupResult) [lookup(LookupState, LookupResult)] [lookup(LookupState, LookupResult)] setTarget(resourceReference) internalLookup() [moreComponents]: lookup(LookupState, LookupResult) Figure 10: Sequence diagram for the request processing use case. University of Southern California Center for Systems and Software Engineering A “Standard” Architectural Model HTTPD : WebBrowser : SocketClient Resources : httpd : DirectoryResource : ContainerResource : FramedResource processRequest(Request) perform(RequestInterface) : LookupResult <<create>> lookup(LookupState, LookupResult) [lookup(LookupState, LookupResult)] [lookup(LookupState, LookupResult)] setTarget(resourceReference) internalLookup() [moreComponents]: lookup(LookupState, LookupResult) What/where is for the Sequence diagram thedependability? request processing use case. Figure 10: University of Southern California Center for Systems and Software Engineering But, We Can Model Anything • Meta-H, ROOM, UniCon, etc. can help ensure real-time properties in models • Markov chains can help ensure reliability in models • Multi-versioning connectors can help ensure fault tolerance • Code/data mobility and replication formalisms can help ensure availability • ... University of Southern California Center for Systems and Software Engineering But, We Can Model Anything • Meta-H, ROOM, UniCon, etc. can help ensure real-time properties in models • Markov chains can help ensure reliability in models • Multi-versioning connectors can help ensure fault tolerance • Code/data mobility and replication formalisms can help ensure availability • ... So then the problem is solved, right? University of Southern California Center for Systems and Software Engineering Why the Problem Isn’t Solved University of Southern California Center for Systems and Software Engineering Why the Problem Isn’t Solved Architecture (ADL) University of Southern California Center for Systems and Software Engineering Why the Problem Isn’t Solved Architecture (ADL) Design (UML) University of Southern California Center for Systems and Software Engineering Why the Problem Isn’t Solved Architecture (ADL) Design (UML) Implementation C++ (middleware, bus technolgies) University of Southern California Center for Systems and Software Engineering Why the Problem Isn’t Solved Domain Entities Problem Space Components Connectors Architecture Space Class Name Attributes Class Name Class Name Attributes Attributes Operation Operation Class Name Attributes Operation Operation Class Name Attributes Classes COTS Components Operation Design Space University of Southern California Center for Systems and Software Engineering Why the Problem Isn’t Solved Domain Entities W Problem Space Components Connectors Architecture Space Class Name Attributes Class Name Class Name Attributes Attributes Operation Operation Class Name Attributes Operation Operation Class Name Attributes Classes COTS Components Operation Design Space University of Southern California Center for Systems and Software Engineering Why the Problem Isn’t Solved W W W1 2 W3 Domain Entities Problem Space Components Connectors Architecture Space Class Name Attributes Class Name Class Name Attributes Attributes Operation Operation Class Name Attributes Operation Operation Class Name Attributes Classes COTS Components Operation Design Space University of Southern California Center for Systems and Software Engineering Why the Problem Isn’t Solved W W W1 2 W3 W W11 21 W31W13 W12 W22Q Domain Entities Problem Space Components Connectors Architecture Space Class Name Attributes Class Name Class Name Attributes Attributes Operation Operation Class Name Attributes Operation Operation Class Name Attributes Classes COTS Components Operation Design Space University of Southern California Center for Systems and Software Engineering From Models to Systems Strategy Analyzer Agent SAKBUI Deployment Advisor HQ UI Strategy AnalysisKB Comman der UI Weather Repository Simulation Agent Clock Map Resource Manager Weather Analyzer Soldier UI Resource Monitor Soldier UI Comman der UI University of Southern California Center for Systems and Software Engineering From Models to Systems Strategy Analyzer Agent SAKBUI Deployment Advisor HQ UI Strategy AnalysisKB Comman der UI Weather Repository Host 1 Host 2 Simulation Agent Clock Map Resource Manager Weather Analyzer Resource Monitor Soldier UI Host 3 Soldier UI Host 4 Comman der UI Host 5 University of Southern California Center for Systems and Software Engineering From Models to Systems University of Southern California Center for Systems and Software Engineering The remainder of this talk will focus on two key questions: University of Southern California Center for Systems and Software Engineering 1. How do we get from University of Southern California Center for Systems and Software Engineering 1. How do we get from Strategy Analyzer Agent SAKBUI Deployment Advisor HQ UI Strategy AnalysisKB Comman der UI Weather Repository Simulation Agent Clock Map Resource Manager Weather Analyzer Resource Monitor Soldier UI Comman der UI Soldier UI and Host 1 Host 3 Host 2 Host 4 Host 5 University of Southern California Center for Systems and Software Engineering 1. How do we get from Strategy Analyzer Agent Strategy Analyzer Agent SAKBUI HQ UI Comman der UI Comman der UI Weather Weather Repository to Simulation Agent Clock Map Simulation Agent Clock Weather Analyzer Comman der UI Soldier UI and Host 1 Map Resource Monitor Soldier UI Host 3 Host 2 Host 4 Host 2 Resource Manager Resource Monitor Soldier UI Repository Host 1 Resource Manager Host 3 Deployment Advisor Strategy AnalysisKB HQ UI Strategy AnalysisKB Weather Analyzer SAKBUI Deployment Advisor Host 5 Soldier UI Host 4 Comman der UI Host 5 University of Southern California Center for Systems and Software Engineering 1. How do we get from Strategy Analyzer Agent Strategy Analyzer Agent SAKBUI HQ UI Comman der UI Comman der UI Weather Weather Repository to Simulation Agent Clock Map Simulation Agent Clock Weather Analyzer Comman der UI Soldier UI and Host 1 Host 2 Map Resource Manager Resource Monitor Soldier UI Repository Host 1 Resource Manager Host 3 Deployment Advisor Strategy AnalysisKB HQ UI Strategy AnalysisKB Weather Analyzer SAKBUI Deployment Advisor Resource Monitor Soldier UI Host 3 Soldier UI Host 4 Comman der UI Host 5 to Host 2 Host 4 Host 5 ? University of Southern California Center for Systems and Software Engineering 2. How do we know University of Southern California Center for Systems and Software Engineering 2. How do we know University of Southern California Center for Systems and Software Engineering 2. How do we know is “better” than ? University of Southern California Center for Systems and Software Engineering Outline • From architectures to systems • Ensuring dependability Problem definition Proposed solution • Concluding remarks University of Southern California Center for Systems and Software Engineering Outline From architectures to systems • Ensuring dependability Problem definition Proposed solution • Concluding remarks University of Southern California Center for Systems and Software Engineering How Do I Dependably Implement an Architecture? • Architectures provide high-level concepts – Components, connectors, ports, events, configurations • Programming languages provide low-level constructs – Variables, arrays, pointers, procedures, objects • Bridging the two often is an art-form – Middleware can help “split the difference” • Existing middleware technologies – Support some architectural concepts (e.g., components, events) – but not others (e.g., connectors, configurations) – Impose particular architectural styles University of Southern California Center for Systems and Software Engineering How Do I Dependably Implement an Architecture? • Architectures provide high-level concepts – Components, connectors, ports, events, configurations • Programming languages provide low-level constructs – Variables, arrays, pointers, procedures, objects • Bridging the two often is an art-form – Middleware can help “split the difference” • Existing middleware technologies – Support some architectural concepts (e.g., components, events) – but not others (e.g., connectors, configurations) – Impose particular architectural styles What is needed is “architectural middleware” University of Southern California Center for Systems and Software Engineering Architectural Middleware • Natively support architectural concepts as middleware constructs • Include system design support – Typically via an accompanying ADL and analysis tools – May support explicit architectural styles • Support round-trip development – From architecture to implementation and back • Support automated transformation of architectural models to implementations – i.e., dependable implementation • Examples – – – – ArchJava Aura c2.fw Prism-MW University of Southern California Center for Systems and Software Engineering Dependable Implementation Architecture (ADL) Design (UML) Implementation C++ (middleware, bus technolgies) University of Southern California Center for Systems and Software Engineering Dependable Implementation Architecture (ADL) Implementation C++ (middleware, bus technolgies) University of Southern California Center for Systems and Software Engineering Dependable Implementation Architecture (ADL) Implementation C++ (middleware, bus technolgies) University of Southern California Center for Systems and Software Engineering Example: Prism-MW Serializable Fifo Scheduler Abstract Scheduler Round Robin Dispatcher Abstract Dispatcher Brick Scaffold Event IPort #mutualPort Architecture Port IComponent IConnector Component IArchitecture Connector Extensible Component University of Southern California Center for Systems and Software Engineering Example: Prism-MW Serializable Fifo Scheduler Abstract Scheduler Round Robin Dispatcher Abstract Dispatcher Brick Scaffold Event IPort #mutualPort Architecture Port IComponent IConnector Component IArchitecture Connector Extensible Component University of Southern California Center for Systems and Software Engineering Example: Prism-MW Serializable Fifo Scheduler Abstract Scheduler Round Robin Dispatcher Abstract Dispatcher Brick Scaffold Event IPort #mutualPort Architecture Port IComponent IConnector Component IArchitecture Connector Extensible Component University of Southern California Center for Systems and Software Engineering Example: Prism-MW Serializable Fifo Scheduler Abstract Scheduler Round Robin Dispatcher Abstract Dispatcher Brick Scaffold Event IPort #mutualPort Architecture Port IComponent IConnector Component IArchitecture Connector Extensible Component University of Southern California Center for Systems and Software Engineering Example: Prism-MW Serializable Fifo Scheduler Abstract Scheduler Round Robin Dispatcher Abstract Dispatcher Brick Scaffold Event IPort #mutualPort Architecture Port IComponent IConnector Component IArchitecture Connector Extensible Component University of Southern California Center for Systems and Software Engineering Example: Prism-MW Serializable Fifo Scheduler Abstract Scheduler Round Robin Dispatcher Abstract Dispatcher Brick Scaffold Event IPort #mutualPort Architecture Port IComponent IConnector Component IArchitecture Connector Extensible Component University of Southern California Center for Systems and Software Engineering Using Prism-MW University of Southern California Center for Systems and Software Engineering Using Prism-MW class DemoArch { static public void main(String argv[]) { Architecture arch = new Architecture ("DEMO"); Architecture - DEMO University of Southern California Center for Systems and Software Engineering Using Prism-MW class DemoArch { static public void main(String argv[]) { Architecture arch = new Architecture ("DEMO"); // create components ComponentA a = new ComponentA ("A"); ComponentB b = new ComponentB ("B"); ComponentD d = new ComponentD ("D"); Architecture - DEMO Component A Component B Component D University of Southern California Center for Systems and Software Engineering Using Prism-MW class DemoArch { static public void main(String argv[]) { Architecture arch = new Architecture ("DEMO"); // create components ComponentA a = new ComponentA ("A"); ComponentB b = new ComponentB ("B"); ComponentD d = new ComponentD ("D"); // create connectors Connector conn = new Connector("C"); Architecture - DEMO Component A Component B Component D Connector C C University of Southern California Center for Systems and Software Engineering Using Prism-MW Component A Component B Connector C C Component D Architecture - DEMO class DemoArch { static public void main(String argv[]) { Architecture arch = new Architecture ("DEMO"); // create components ComponentA a = new ComponentA ("A"); ComponentB b = new ComponentB ("B"); ComponentD d = new ComponentD ("D"); // create connectors Connector conn = new Connector("C"); // add components and connectors arch.addComponent(a); arch.addComponent(b); arch.addComponent(d); arch.addConnector(conn); University of Southern California Center for Systems and Software Engineering Using Prism-MW Component A Component B Connector C C Component D class DemoArch { static public void main(String argv[]) { Architecture arch = new Architecture ("DEMO"); // create components ComponentA a = new ComponentA ("A"); ComponentB b = new ComponentB ("B"); ComponentD d = new ComponentD ("D"); // create connectors Connector conn = new Connector("C"); // add components and connectors arch.addComponent(a); arch.addComponent(b); arch.addComponent(d); arch.addConnector(conn); Architecture - DEMO } } // establish the interconnections arch.weld(a, conn); arch.weld(b, conn); arch.weld(conn, d) University of Southern California Center for Systems and Software Engineering Deploying a Prism-MW Architecture University of Southern California Center for Systems and Software Engineering Deploying a Prism-MW Architecture add (DataRepository : source HQ) : HQ; weld (TopDistributionConnector, C1_AvailableTroops) : Commander1; University of Southern California Center for Systems and Software Engineering Outline From architectures to systems Ensuring dependability Problem definition Proposed solution • Concluding remarks University of Southern California Center for Systems and Software Engineering Outline From architectures to systems Ensuring dependability Problem definition Proposed solution • Concluding remarks University of Southern California Center for Systems and Software Engineering Let’s Pick Availability • The degree to which a system is operational and accessible when required for use [IEEE] • Deployment architecture influences availability – Components on the same host can communicate regardless of the network’s status – Components on different hosts are insulated from each other’s failures • Quantifying availability – Ratio of the number of successfully completed interactions in the system to the total number of attempted interactions University of Southern California Center for Systems and Software Engineering Maximizing Availability a priori • We may not know many relevant system parameters – – – – – – – – – Dependability of each component Frequency of component interactions Volume of component interactions Dependability of component interactions CPU usage on each host Dependability of each host Effective bandwidth of each network connection Dependability of each network connection ... University of Southern California Center for Systems and Software Engineering Maximizing Availability a priori • We may not know many relevant system parameters – – – – – – – – – Dependability of each component Frequency of component interactions Volume of component interactions Dependability of component interactions CPU usage on each host Dependability of each host Effective bandwidth of each network connection Dependability of each network connection ... Current deployment architecture may not work well University of Southern California Center for Systems and Software Engineering Maximizing Availability a priori • We may not know many relevant system parameters – – – – – – – – – Dependability of each component Frequency of component interactions Volume of component interactions Dependability of component interactions CPU usage on each host Dependability of each host Effective bandwidth of each network connection Dependability of each network connection ... Current deployment architecture may not work well n Number of possible deployment architectures is k University of Southern California Center for Systems and Software Engineering Simplified Problem Definition Find a function f : C H such that the system’s overall availability freq(c , c n A n i i 1 j 1 n j ) rel ( f (ci ), f (c j )) n freq(c , c i 1 j 1 i j ) is maximized, and the following conditions are satisfied: k [1, n] l [1, n] (colloc (ck , cl ) 1) ( f (ck ) f (cl )) (colloc (ck , cl ) 1) ( f (ck ) f (cl )) University of Southern California Center for Systems and Software Engineering Outline From architectures to systems Ensuring dependability Problem definition Proposed solution • Concluding remarks University of Southern California Center for Systems and Software Engineering Overview of the Approach • Objective – Identify the problem • Log and examine system events Actively monitor the system during runtime – Develop a solution • Decide which data to cache • Decide which components to replicate • Introduce multiple execution modes Calculate an improved system deployment – Apply the solution to eliminate the problem • Cache or hoard data • Replicate data or code Redeploy (parts of) the system University of Southern California Center for Systems and Software Engineering Improving Availability via Redeployment Availability A2 A4 A1 A3 Time TM TE TR TO T T’M T’E T’R T’ T’O University of Southern California Center for Systems and Software Engineering First Identify the Problem Availability A2 A1 Time TM TE TR TO T University of Southern California Center for Systems and Software Engineering First Identify the Problem Availability A2 A1 Time TM TE TR TO T University of Southern California Center for Systems and Software Engineering Monitoring in Prism-MW Serializable Fifo Scheduler Abstract Scheduler Round Robin Dispatcher Abstract Dispatcher EvtFrequency Disconnection Rate Brick Scaffold IPort #mutualPort Architecture Abstract Monitor Event Port IComponent IConnector Component IArchitecture Connector Extensible Component University of Southern California Center for Systems and Software Engineering Monitoring in Action 4 15 12 34 16 6 2 7 21 5 31 19 0 17 23 29 32 18 1 10 8 3 9 33 27 13 11 26 24 14 22 25 28 20 30 University of Southern California Center for Systems and Software Engineering Then Develop a Solution Availability A2 A1 Time TM TE TR TO T University of Southern California Center for Systems and Software Engineering Then Develop a Solution Availability A2 A1 Time TM TE TR TO T University of Southern California Center for Systems and Software Engineering Estimating in Action Suite of algorithms: Exact – exponential complexity Stochastic – quadratic complexity Adaptive greedy – cubic complexity Decentralized – quadratic complexity 1000000 Time taken (in ms) 10000 100 1 1 0.8 0.6 Achieved availability 0.4 0.2 z 0 10 comps 4 hosts 100 comps 10 hosts 200 comps 1000 comps 100 comps 20 hosts 100 hosts 40 hosts 30 comps 7 hosts 300 comps 70 hosts University of Southern California Center for Systems and Software Engineering Automatic Algorithm Selection Availability Exact AE Greedy AG Stochastic AS AC Time TES TEG TR TEE T0S TR T TR T0G T0E University of Southern California Center for Systems and Software Engineering Automatic Algorithm Selection Availability Exact AE Greedy AG Stochastic AS AC Time TES TEG TR TEE T0S TR T TR T0G T0E University of Southern California Center for Systems and Software Engineering Then Apply the Solution Availability A2 A1 Time TM TE TR TO T University of Southern California Center for Systems and Software Engineering Then Apply the Solution Availability A2 A1 Time TM TE TR TO T University of Southern California Center for Systems and Software Engineering Redeployment in Prism-MW Serializable Fifo Scheduler Abstract Scheduler Round Robin Dispatcher Abstract Dispatcher Brick Scaffold Event IPort #mutualPort Architecture Port IComponent IConnector Component IArchitecture Connector Admin Abstract Admin Deployer Extensible Component Abstract Redeployment Algo Adaptive Greedy Stochastic Exact University of Southern California Center for Systems and Software Engineering Redeployment in Action 4 15 12 34 16 6 2 18 7 21 5 31 Deployer Admin Admin 19 0 Admin 23 29 32 1 10 8 3 9 33 27 13 11 26 17 24 14 22 Admin 25 28 20 30 University of Southern California Center for Systems and Software Engineering Going from Legend: Strategy Analyzer Agent SAKBUI User interface component (fixed to a host) Deployment Advisor Processing or data component HQ UI Strategy AnalysisKB Ho st 2 Comman der UI Weather Repository Host 1 Host 2 Hardware host Network link Interaction path between components Simulation Agent Clock Map Resource Manager Weather Analyzer Resource Monitor Soldier UI Host 3 Soldier UI Host 4 Comman der UI Host 5 University of Southern California Center for Systems and Software Engineering to University of Southern California Center for Systems and Software Engineering Outline From architectures to systems Ensuring dependability Problem definition Proposed solution Concluding remarks University of Southern California Center for Systems and Software Engineering Concluding Remarks Still so much to do Enrich/complete the models Focus on additional aspects of dependability Address dependability trade-offs Address feature interactions Address emergent properties Determine which concerns are (not) architectural University of Southern California Center for Systems and Software Engineering Promise or Illusion? If we can define it, we should be able to Analyze for it Build it Measure it Act on it So, why haven’t we done it yet? University of Southern California Center for Systems and Software Engineering The Playing Field Dependability Dependability Survivability Safety Security Reliability Availability Interaction Interaction Component Frequency Volume Size Available Memory Available CPU Network Reliability Network Bandwidth H/WHardware Properties Properties Component Component CPU Usage Failure Rate S/W Software Properties Properties University of Southern California Center for Systems and Software Engineering Exploring the Problem Space University of Southern California Center for Systems and Software Engineering Exploring the Problem Space University of Southern California Center for Systems and Software Engineering Exploring the Problem Space University of Southern California Center for Systems and Software Engineering Exploring the Problem Space University of Southern California Center for Systems and Software Engineering Questions?