Patterns - Distributed Object Computing

Pattern-Oriented Software Architectures Patterns & Frameworks for Concurrent & Networked Software Dr. Douglas C. Schmidt d.schmidt@vanderbilt.edu www.dre.vanderbilt.edu/~schmidt/ Professor of EECS Vanderbilt University Nashville, Tennessee Tutorial Motivation Observations • Building robust, efficient, & extensible concurrent & networked applications is hard • e.g., we must address many complex topics that are less problematic for non-concurrent, standalone applications Stand-alone Architecture Networked Architecture • Fortunately, there are reusable solutions to many common challenges, e.g.: • Connection mgmt & event demuxing • Service initialization Key challenge: How can we • Error handling & fault tolerance reuse these solutions • Flow & congestion control effectively in a changing, • Distribution heterogeneous world? • Concurrency, scheduling, & synchronization • Persistence 2 Tutorial Outline Cover OO techniques & language features that enhance software quality OO techniques & language features: • Patterns (25+), which embody reusable software architectures & designs • Frameworks & components, which embody reusable software middleware & application implementations • OO language features, e.g., classes, dynamic binding & inheritance, parameterized types Tutorial Organization 1. Technology trends & background 2. Concurrent & network challenges & solution approaches 3. Case studies 4. Wrap-up 3 Modalities e.g., MRI, CT, CR, Ultrasound, etc. Technology Trends (1/4) Information technology is being commoditized • i.e., hardware & software are getting cheaper, faster, & (generally) better at a fairly predictable rate These advances stem largely from standard hardware & software APIs & protocols, e.g.: • Intel x86 & Power PC chipsets • TCP/IP, GSM, Link16 • POSIX, Windows, & VMs • Middleware & component models 4 • Quality of service (QoS) aspects Technology Trends (2/4) Growing acceptance of a network-centric component paradigm • i.e., distributed applications with a range of QoS needs are constructed by integrating components & frameworks via various communication mechanisms Avionics Mission Computing Process Automation Quality Control Hot Rolling Mills Electronic Medical Imaging Software Defined Radio 5 Modalities e.g., MRI, CT, CR, Ultrasound, etc. Technology Trends (3/4) Component middleware is maturing & becoming pervasive … … … … Container Container Middleware Bus Replication 6 Security A/V Streaming Persistence Scheduling Notification Load Balancing •Components encapsulate application “business” logic •Components interact via ports •Provided interfaces, e.g.,facets •Required connection points, e.g., receptacles •Event sinks & sources •Attributes •Containers provide execution environment for components with common operating requirements •Components/containers can also • Communicate via a middleware bus & • Reuse common middleware services Technology Trends (4/4) Model driven development is integrating generative software technologies with QoS-enabled component middleware •e.g., standard technologies are DRE Applications emerging that can: 1. Model Middleware 2. Analyze Services 3. Synthesize & optimize 4. Provision & deploy Middleware multiple layers of QoS-enabled middleware & applications Operating Sys •These technologies are guided & Protocols by patterns & implemented by component frameworks Distributed Hardware & •Partial specialization is Networks system essential for inter-/intra-layer optimization <CONFIGURATION_PASS> <HOME> <…> <COMPONENT> <ID> <…></ID> <EVENT_SUPPLIER> <…events this component supplies…> </EVENT_SUPPLIER> </COMPONENT> </HOME> </CONFIGURATION_PASS> 7 Goal is not to replace programmers per se – it is to provide higherlevel domain-specific languages for middleware developers & users The Evolution of Middleware Applications Domain-Specific Services Common Middleware Services Distribution Middleware Host Infrastructure Middleware Operating Systems & Protocols Hardware 8 There are multiple COTS middleware layers & research/business opportunities Historically, mission-critical apps were built directly atop hardware & OS • Tedious, error-prone, & costly over lifecycles There are layers of middleware, just like there are layers of networking protocols Standards-based COTS middleware helps: •Control end-to-end resources & QoS •Leverage hardware & software technology advances •Evolve to new environments & requirements •Provide a wide array of reusable, offthe-shelf developer-oriented services Operating System & Protocols •Operating systems & protocols provide mechanisms to manage endsystem resources, e.g., •CPU scheduling & dispatching •Virtual memory management •Secondary storage, persistence, & file systems •Local & remote interprocess communication (IPC) •OS examples •UNIX/Linux, Windows, VxWorks, QNX, etc. •Protocol examples •TCP, UDP, IP, SCTP, RTP, etc. INTERNETWORKING ARCH RTP TFTP FTP MIDDLEWARE ARCH Middleware Applications HTTP TELNET DNS UDP Middleware Services TCP Middleware IP Solaris Fibre Channel Ethernet 9 ATM 20th Century FDDI Win2K VxWorks Linux LynxOS 21st Century Host Infrastructure Middleware •Host infrastructure middleware encapsulates & enhances native OS mechanisms to create reusable network programming components Common Middleware Services • These components abstract away many tedious & error-prone aspects of low-level OS APIs Distribution Middleware •Examples •Java Virtual Machine (JVM), Common Language Runtime (CLR), ADAPTIVE Communication Environment (ACE) Asynchronous Event Handling Physical Memory Access Memory Management 10 Domain-Specific Services Host Infrastructure Middleware Asynchronous Transfer of Control Synchronization Scheduling www.rtj.org www.cs.wustl.edu/~schmidt/ACE.html Distribution Middleware •Distribution middleware defines higher-level distributed programming models whose reusable APIs & components automate & extend native OS capabilities •Examples • OMG Real-time CORBA & DDS, Sun RMI, Microsoft DCOM, W3C SOAP Domain-Specific Services Common Middleware Services Distribution Middleware Host Infrastructure Middleware End-to-End Priority Propagation Client OBJ REF Object (Servant) in args operation() out args + return Scheduling Service IDL SKEL IDL STUBS Explicit Binding ORB CORE Standard Synchronizers Thread Pools Object Adapter Portable Priorities GIOP Protocol Properties realtime.omg.org/ 11 en.wikipedia.org/wiki/Data_Distribution_Service Distribution middleware avoids hard-coding client & server application dependencies on object location, language, OS, protocols, & hardware Common Middleware Services •Common middleware services augment distribution middleware by defining higher-level domain-independent services that focus on programming “business logic” •Examples •CORBA Component Model & Object Services, Sun’s J2EE, Microsoft’s .NET, W3C Web Services Domain-Specific Services Common Middleware Services Distribution Middleware Host Infrastructure Middleware •Common middleware services support many recurring distributed system capabilities, e.g., • Transactional behavior • Authentication & authorization, • Database connection pooling & concurrency control • Active replication • Dynamic resource management 12 Domain-Specific Middleware • Domain-specific middleware services are tailored to the requirements of particular domains, such as telecom, ecommerce, health care, process automation, or aerospace •Examples Siemens MED Syngo • Common software platform for distributed electronic medical systems • Used by all ~13 Siemens MED business units worldwide Boeing Bold Stroke • Common software platform for Boeing avionics mission computing systems Modalities e.g., MRI, CT, CR, Ultrasound, etc. 13 Domain-Specific Services Common Middleware Services Distribution Middleware Host Infrastructure Middleware Consequences of COTS & IT Commoditization •More emphasis on integration rather than programming •Increased technology convergence & standardization •Mass market economies of scale for technology & personnel •More disruptive technologies & global competition •Lower priced--but often lower quality-hardware & software components •The decline of internally funded R&D •Potential for complexity cap in nextgeneration complex systems Not all trends bode well for long-term competitiveness of traditional R&D leaders 14 Ultimately, competitiveness depends on success of long-term R&D on complex distributed realtime & embedded (DRE) systems Why We are Succeeding Now Recent synergistic advances in fundamental technologies & processes: Standards-based QoS-enabled Middleware: Pluggable service & micro-protocol components & reusable “semi-complete” application frameworks Why middleware-centric reuse works 1.Hardware advances •e.g., faster CPUs & networks 2.Software/system architecture advances •e.g., inter-layer optimizations & meta-programming mechanisms Patterns & Pattern Languages: 3.Economic necessity Generate software architectures •e.g., global competition for by capturing recurring structures & customers & engineers dynamics & by resolving design forces Revolutionary changes in software process & methods: Open-source, refactoring, agile methods, advanced V&V techniques, model-driven development 15 Example: Applying COTS in Real-time Avionics Goals • Apply COTS & open systems to missioncritical real-time avionics Key System Characteristics • Deterministic & statistical deadlines • ~20 Hz • Low latency & jitter • ~250 usecs • Periodic & aperiodic processing • Complex dependencies • Continuous platform upgrades Key Results • Test flown at China Lake NAWS by Boeing OSAT II ‘98, funded by OS-JTF • www.cs.wustl.edu/~schmidt/TAO-boeing.html • Also used on SOFIA project by Raytheon • sofia.arc.nasa.gov • First use of RT CORBA in mission computing • Drove Real-time CORBA standardization 16 Example: Applying COTS to Time-Critical Targets Goals • Detect, identify, track, & destroy time-critical targets Joint JointForces Forces Global Info Global InfoGrid Grid Challenge Challenges are is to make this also relevant to possible! TBMD & NMD Key System Characteristics • Real-time mission-critical sensor-to-shooter needs Adapted from “The Future of AWACS”, • Highlybydynamic QoS LtCol Joe Chapa requirements & environmental Key Solution Characteristics & scalable • Adaptive & reflective conditions Time-critical targets require immediate•Efficient response because: •Affordable & flexible •High confidence • Multi-service & •They asset pose a clear & present danger to friendly forces & •COTS-based •Safety criticaltargets of coordination •Are highly lucrative, fleeting opportunity 17 Example: Applying COTS to Large-scale Routers IOM IOM IOM BSE BSE BSE IOM IOM IOM IOM IOM IOM BSE BSE BSE Goal • Switch ATM cells + IP packets at terabit rates IOM Key System Characteristics IOM IOM •Very high-speed WDM IOM BSE BSE BSE IOM links IOM IOM •102/103 line cards •Stringent requirements www.arl.wustl.edu for availability Key Software Solution Characteristics •Multi-layer load •High confidence & scalable computing architecture balancing, e.g.: • Networked embedded processors •Layer 3+4 • Distribution middleware •Layer 5 • FT & load sharing • Distributed & layered resource management •Affordable, flexible, & COTS IOM IOM Example: Applying COTS to Software Defined Radios www.omg.org/docs/swradio Non-CORBA Modem Components RF Applications Core Framework (CF) Commercial Off-the-Shelf (COTS) OE Non-CORBA Security Components Non-CORBA I/O Components Physical API Modem Components Modem Adapter MAC API Link, Network Components Security Adapter LLC/Network API Core Framework IDL CORBA ORB & Services (Middleware) Security Components Security API Security Adapter Link, Network Components I/O Adapter LLC/Network API I/O Components I/O API (“Logical Software Bus” via CORBA) CF Services & Applications Operating System Network Stacks & Serial Interface Services Board Support Package (Bus Layer) Black Hardware Bus CF Services & Applications CORBA ORB & Services (Middleware) Operating System Network Stacks & Serial Interface Services Board Support Package (Bus Layer) Red Hardware Bus Key Software Solution Characteristics • Transitioned to BAE systems for the Joint Tactical Radio Systems • Programmable radio with waveform-specific components • Uses CORBA component middleware based on ACE+TAO Example: Applying COTS to Hot Rolling Mills Goals • Control the processing of molten steel moving through a hot rolling mill in real-time System Characteristics • Hard real-time process automation requirements • i.e., 250 ms real-time cycles • System acquires values representing plant’s current state, tracks material flow, calculates new settings for the rolls & devices, & submits new settings back to plant Key Software Solution Characteristics • Affordable, flexible, & COTS • Product-line architecture • Design guided by patterns & frameworks 20 www.siroll.de • Windows NT/2000 • Real-time CORBA (ACE+TAO) Example: Applying COTS to Real-time Image Processing www.krones.com Goals • Examine glass bottles for defects in real-time System Characteristics • Process 20 bottles per sec • i.e., ~50 msec per bottle • Networked configuration • ~10 cameras Key Software Solution Characteristics • Affordable, flexible, & COTS • Embedded Linux (Lem) • Compact PCI bus + Celeron processors 21 • Remote booted by DHCP/TFTP • Real-time CORBA (ACE+TAO) Key Opportunities & Challenges in Concurrent Applications Motivations •Leverage hardware/software advances •Simplify program structure •Increase performance •Improve response-time Accidental Complexities •Low-level APIs •Poor debugging tools Inherent Complexities •Scheduling •Synchronization •Deadlocks 22 Key Opportunities & Challenges in Networked & Distributed Applications Motivations • Collaboration • Performance • Reliability & availability • Scalability & portability • Extensibility • Cost effectiveness Accidental Complexities •Algorithmic decomposition •Continuous re-invention & rediscovery of core concepts & components 23 Inherent Complexities •Latency •Reliability •Load balancing •Causal ordering •Security & information assurance Overview of Patterns •Present solutions to common software problems arising within a certain context •Help resolve key software design forces •Capture recurring structures & dynamics among software participants to facilitate reuse of successful designs •Generally codify expert knowledge of design strategies, constraints & “best practices” AbstractService service Client Proxy service Service 1 1 service The Proxy Pattern 24 •Flexibility •Extensibility •Dependability •Predictability •Scalability •Efficiency Overview of Pattern Languages Motivation • Individual patterns & pattern catalogs are insufficient • Software modeling methods & tools largely just illustrate what/how – not why – systems are designed 25 Benefits of Pattern Languages • Define vocabulary for talking about software development problems • Provide a process for the orderly resolution of these problems, e.g.: • What are key problems to be resolved & in what order • What alternatives exist for resolving a given problem • How should mutual dependencies between the problems be handled • How to resolve each individual problem most effectively in its context • Help to generate & reuse software architectures Taxonomy of Patterns & Idioms Type Description Examples Idioms Restricted to a particular language, system, or tool Scoped locking Design patterns Capture the static & dynamic roles & relationships in solutions that occur repeatedly Active Object, Bridge, Proxy, Wrapper Façade, & Visitor Architectural patterns Express a fundamental structural organization for software systems that provide a set of predefined subsystems, specify their relationships, & include the rules & guidelines for organizing the relationships between them Half-Sync/HalfAsync, Layers, Proactor, PublisherSubscriber, & Reactor Optimization principle patterns Document rules for avoiding common design & implementation mistakes that degrade performance Optimize for common case, pass information between layers 26 Example: Boeing Bold Stroke Product-line Architecture Nav Sensors Vehicle Mgmt Data Links Mission Computer Radar Expendable Management Bold Stroke Architecture Expendable Mission Computing Services Middleware Infrastructure Operating System Networking Interfaces Hardware (CPU, Memory, I/O) • Avionics mission computing product-line architecture for Boeing military aircraft, e.g., F-18 E/F, 15E, Harrier, UCAV • DRE system with 100+ developers, 3,000+ 27 software components, 3-5 million lines of C++ code • Based on COTS hardware, networks, operating systems, & middleware • Used as Open Experimentation Platform (OEP) for DARPA IXO PCES, MoBIES, SEC, MICA programs Example: Boeing Bold Stroke Product-line Architecture Mission Computing Services Middleware Infrastructure Operating System Networking Interfaces Hardware (CPU, Memory, I/O) 28 COTS & Standards-based Middleware Infrastructure, OS, Network, & Hardware Platform • Real-time CORBA middleware services • VxWorks operating system • VME, 1553, & Link16 • PowerPC Example: Boeing Bold Stroke Product-line Architecture Reusable Object-Oriented Application Domainspecific Middleware Framework • Configurable to variable infrastructure configurations • Supports systematic reuse of mission computing functionality Mission Computing Services Middleware Infrastructure Operating System Networking Interfaces Hardware (CPU, Memory, I/O) 29 Example: Boeing Bold Stroke Product-line Architecture Product Line Component Model • Configurable for product-specific functionality & execution environment • Single component development policies • Standard component packaging mechanisms Mission Computing Services Middleware Infrastructure Operating System Networking Interfaces Hardware (CPU, Memory, I/O) 30 Example: Boeing Bold Stroke Product-line Architecture Mission Computing Services Middleware Infrastructure Operating System Networking Interfaces Hardware (CPU, Memory, I/O) Component Integration Model • Configurable for product-specific component assembly & deployment environments • Model-based component integration policies 31 Operator Real World Model Avionics Interfaces Infrastructure Services Legacy Avionics Architectures Key System Characteristics •Hard & soft real-time deadlines •~20-40 Hz •Low latency & jitter between boards •~100 usecs •Periodic & aperiodic processing •Complex dependencies •Continuous platform upgrades Avionics Mission Computing Functions •Weapons targeting systems (WTS) •Airframe & navigation (Nav) •Sensor control (GPS, IFF, FLIR) •Heads-up display (HUD) •Auto-pilot (AP) 4: Mission functions perform avionics operations 3: Sensor proxies process data & pass to missions functions 2: I/O via interrupts Board 1 1553 VME Board 2 32 1: Sensors generate data Legacy Avionics Architectures Key System Characteristics •Hard & soft real-time deadlines •~20-40 Hz •Low latency & jitter between boards •~100 usecs •Periodic & aperiodic processing •Complex dependencies •Continuous platform upgrades Limitations with Legacy Avionics Architectures •Stovepiped •Proprietary •Expensive •Vulnerable •Tightly coupled •Hard to schedule •Brittle & non-adaptive 33 Nav Air Frame WTS AP FLIR GPS IFF Cyclic Exec 4: Mission functions perform avionics operations 3: Sensor proxies process data & pass to missions functions 2: I/O via interrupts Board 1 1553 VME Board 2 1: Sensors generate data Decoupling Avionics Components Context Problems Solution • I/O driven DRE • Tightly coupled • Apply the Publisher- application components • Complex • Hard to schedule dependencies • Expensive to evolve • Real-time constraints Subscriber architectural pattern to distribute periodic, I/O-driven data from a single point of source to a collection of consumers Structure Publisher produce Event Channel attachPublisher detachPublisher attachSubscriber detachSubscriber pushEvent creates * Event Dynamics Subscriber consume : Event Channel receives : Subscriber attachSubscriber produce pushEvent event : Event pushEvent event Filter filterEvent 34 : Publisher detachSubscriber consume Applying the Publisher-Subscriber Pattern to Bold Stroke Bold Stroke uses the PublisherSubscriber pattern to decouple sensor processing from mission computing operations • Anonymous publisher & subscriber relationships • Group communication • Asynchrony Considerations for implementing the Publisher-Subscriber pattern for mission computing applications include: • Event notification model • Push control vs. pull data interactions • Scheduling & synchronization strategies • e.g., priority-based dispatching & preemption • Event dependency management • e.g.,filtering & correlation mechanisms 35 Subscribers HUD WTS Air Frame Nav 4: Event Channel pushes events to subscribers(s) push(event) Event Channel push(event) GPS IFF 5: Subscribers perform avionics operations FLIR Publishers 3: Sensor publishers push events to event channel 2: I/O via interrupts Board 1 1553 VME Board 2 1: Sensors generate data Ensuring Platform-neutral & Networktransparent Communication Context Problems Solution • Mission computing requires remote IPC • Applications need capabilities to: • Support remote communication • Provide location transparency • Handle faults • Stringent DRE • Manage end-to-end QoS requirements • Encapsulate low-level system details internal partitioning Layers Wrapper Facade Object Adapter server launching broker features request dispatching broker access broker concurrency component access request issuing Client Proxy Leader/ Followers component creation broker configuration Container Strategy connection management Component Configurator 36 request dispatching event demultiplexing Facade Acceptor/ Connector Broker OS abstraction Activator Reactor • Apply the Broker architectural pattern to provide platform-neutral communication between mission computing boards Requestor Abstract Factory Ensuring Platform-neutral & Networktransparent Communication Context • Mission computing requires remote IPC Problems • Applications need capabilities to: • Support remote communication • Provide location transparency • Handle faults • Stringent DRE • Manage end-to-end QoS requirements • Encapsulate low-level system details Structure 37 Solution • Apply the Broker architectural pattern to provide platform-neutral communication between mission computing boards Ensuring Platform-neutral & Networktransparent Communication Context Problems Solution • Mission computing requires remote IPC • Applications need capabilities to: • Support remote communication • Provide location transparency • Handle faults • Stringent DRE • Manage end-to-end QoS requirements • Encapsulate low-level system details : Client : Client Proxy operation (params) : Broker connect marshal receive_reply 38 : Object Adapter : Object register_object send_request start_up assigned port unmarshal dispatch operation (params) Dynamics result • Apply the Broker architectural pattern to provide platform-neutral communication between mission computing boards unmarshal result marshal Applying the Broker Pattern to Bold Stroke Bold Stroke uses the Broker pattern to shield distributed applications from environment heterogeneity, e.g., Subscribers HUD WTS Nav Air Frame push(event) •Programming languages Event Channel •Operating systems push(event) •Networking protocols •Hardware GPS IFF FLIR Publishers Broker A key consideration for implementing the Broker pattern for mission computing applications is QoS support •e.g., latency, jitter, priority preservation, dependability, security, etc. 39 6: Subscribers perform avionics operations 5: Event Channel pushes events to subscribers(s) 4: Sensor publishers push events to event channel 3: Broker handles I/O via upcalls 2: I/O via interrupts Board 1 1553 VME Board 2 1: Sensors generate data Benefits of Patterns Subscribers Nav WTS HUD push(event) Air Frame • Improves development team communication Event Channel push(event) GPS • Enables reuse of software architectures & designs • Convey “best practices” intuitively IFF FLIR Publishers Broker • Transcends language-centric biases/myopia • Abstracts away from many unimportant details Mission Computing Services Middleware Infrastructure Operating System Networking Interfaces Hardware (CPU, Memory, I/O) 40 www.cs.wustl.edu/ ~schmidt/patterns.html Limitations of Patterns Subscribers Nav WTS HUD push(event) Air Frame Event Channel push(event) GPS IFF FLIR Publishers Broker • Require significant tedious & error-prone human effort to handcraft pattern implementations • Can be deceptively simple • Leaves some important details unresolved Mission Computing Services Middleware Infrastructure Operating System Networking Interfaces Hardware (CPU, Memory, I/O) 41 www.cs.wustl.edu/ ~schmidt/patterns.html Software Design Abstractions for Concurrent & Networked Applications Problem •Distributed app & middleware functionality is subject to change since it’s often reused in unforeseen contexts, e.g., •Accessed from different clients •Run on different platforms •Configured into different runtime contexts Solution •Don‘t structure distributed applications & middleware as a monolithic spagetti •Instead, decompose them into modular classes, frameworks, & components 42 MIDDLEWARE Overview of Frameworks Framework Characteristics •Frameworks exhibit •Frameworks provide •Frameworks are “inversion of control” at integrated domain-specific “semi-complete” runtime via callbacks structures & functionality applications Application-specific functionality Mission Computing Scientific Visualization E-commerce GUI Networking 43 Database Comparing Class Libraries, Frameworks, & Components Component Architecture Class Library Architecture APPLICATIONSPECIFIC FUNCTIONALITY LOCAL INVOCATIONS Math Naming ADTs Events Files Strings GUI EVENT LOOP GLUE CODE Locks Logging IPC Middleware Bus A class is a unit of abstraction & implementation in an OO programming language A component is an encapsulation unit with one or more interfaces that provide clients with access to its services Framework Architecture ADTs Strings INVOKES Files Reactor NETWORKING APPLICATIONSPECIFIC FUNCTIONALITY Locking CALLBACKS GUI Locks DATABASE A framework is an integrated set of classes that collaborate to produce a reusable 44 architecture for a family of applications Class Libraries Frameworks Components Micro-level Meso-level Macro-level Stand-alone language entities “Semicomplete” applications Stand-alone composition entities Domainindependent Domainspecific Domain-specific or Domain-independent Borrow caller’s thread Inversion of control Borrow caller’s thread Using Frameworks Effectively Observations •Frameworks are powerful, but hard to develop & use effectively by application developers •It’s often better to use & customize COTS frameworks than to develop inhouse frameworks •Components are easier for application developers to use, but aren’t as powerful or flexible as frameworks Successful projects are therefore often organized using the “funnel” model 45 Overview of the ACE Frameworks NYSE Local Area Network NASDAQ Applicationspecific functionality Acceptor Connector Stream Service Configurator Task Reactor Proactor www.cs.wustl.edu/~schmidt/ACE.html 46 Features •Open-source •6+ integrated frameworks •250,000+ lines of C++ •60+ person-years of effort •Ported to Windows, UNIX, & real-time operating systems •e.g., VxWorks, pSoS, LynxOS, Chorus, QNX •Large user community The Layered Architecture of ACE www.cs.wustl.edu/~schmidt/ACE.html • Large open-source user community • www.cs.wustl.edu/~schmidt/ACEusers.html 47 Features • Open-source • 250,000+ lines of C++ • 40+ person-years of effort • Ported to Win32, UNIX, & RTOSs • e.g., VxWorks, pSoS, LynxOS, Chorus, QNX • Commercial support by Riverace • www.riverace.com/ Key Capabilities Provided by ACE Service Access & Control Concurrency 48 Event Handling Synchronization The POSA2 Pattern Language Pattern Benefits • Preserve crucial design information used by applications & middleware frameworks & components • Facilitate reuse of proven software designs & architectures • Guide design choices for application developers 49 POSA2 Pattern Abstracts Service Access & Configuration Patterns Event Handling Patterns The Wrapper Facade design pattern encapsulates the functions & data provided by existing non-object-oriented APIs within more concise, robust, portable, maintainable, & cohesive object-oriented class interfaces. The Reactor architectural pattern allows eventdriven applications to demultiplex & dispatch service requests that are delivered to an application from one or more clients. The Component Configurator design pattern allows an application to link & unlink its component implementations at run-time without having to modify, recompile, or statically relink the application. Component Configurator further supports the reconfiguration of components into different application processes without having to shut down & re-start running processes. The Interceptor architectural pattern allows services to be added transparently to a framework & triggered automatically when certain events occur. The Extension Interface design pattern allows multiple interfaces to be exported by a component, to prevent bloating of interfaces & breaking of client code when developers extend or modify the functionality of the component. 50 The Proactor architectural pattern allows event-driven applications to efficiently demultiplex & dispatch service requests triggered by the completion of asynchronous operations, to achieve the performance benefits of concurrency without incurring certain of its liabilities. The Asynchronous Completion Token design pattern allows an application to demultiplex & process efficiently the responses of asynchronous operations it invokes on services. The Acceptor-Connector design pattern decouples the connection & initialization of cooperating peer services in a networked system from the processing performed by the peer services after they are connected & initialized. POSA2 Pattern Abstracts (cont’d) Synchronization Patterns Concurrency Patterns The Scoped Locking C++ idiom ensures that a lock is acquired when control enters a scope & released automatically when control leaves the scope, regardless of the return path from the scope. The Active Object design pattern decouples method execution from method invocation to enhance concurrency & simplify synchronized access to objects that reside in their own threads of control. The Monitor Object design pattern synchronizes concurrent method execution to ensure that only one method at a time runs within an object. It also allows an object’s methods to cooperatively schedule their execution sequences. The Strategized Locking design pattern parameterizes synchronization mechanisms that protect a component’s The Half-Sync/Half-Async architectural pattern decouples critical sections from concurrent asynchronous & synchronous service processing in access. concurrent systems, to simplify programming without The Thread-Safe Interface design unduly reducing performance. The pattern introduces two pattern minimizes locking overhead & intercommunicating layers, one for asynchronous & one for ensures that intra-component method synchronous service processing. calls do not incur ‘self-deadlock’ by The Leader/Followers architectural pattern provides an trying to reacquire a lock that is held by efficient concurrency model where multiple threads take the component already. turns sharing a set of event sources in order to detect, The Double-Checked Locking demultiplex, dispatch, & process service requests that Optimization design pattern reduces occur on the event sources. contention & synchronization overhead The Thread-Specific Storage design pattern allows multiple whenever critical sections of code must threads to use one ‘logically global’ access point to retrieve acquire locks in a thread-safe manner an object that is local to a thread, without incurring locking just once during program execution. overhead on each object access. 51 Implementing the Broker Pattern for Bold Stroke Avionics Client Propagation & Server Declared Priority Models Static Scheduling Service Standard Synchonizers Request Buffering Explicit Binding Thread Pools Portable Priorities Protocol Properties www.omg.org 52 • CORBA is a distribution middleware standard • Real-time CORBA adds QoS to classic CORBA to control: 1. Processor Resources 2. Communication Resources 3. Memory Resources •These capabilities address some (but by no means all) important DRE application development & QoSenforcement challenges Example of Applying Patterns & Frameworks to Middleware: Real-time CORBA & The ACE ORB (TAO) www.cs.wustl.edu/~schmidt/TAO.html End-to-end Priority Propagation Scheduling Service Protocol Properties Thread Pools Standard Synchronizers Explicit Binding TAO Features • Open-source • 500,000+ lines of C++ • ACE/patterns-based • 50+ person-years of effort • Ported to UNIX, Win32, MVS, & many RT & embedded OSs • e.g., VxWorks, LynxOS, Chorus, QNX Portable Priorities • Large open-source user community • www.cs.wustl.edu/~schmidt/TAOusers.html 53 • Commercially supported • www.theaceorb.com, www.remedy.nl, www.prismtechnologies.com Key Patterns Used in TAO • Wrapper facades enhance portability • Proxies & adapters simplify client & server applications, respectively • Component Configurator dynamically configures Factories • Factories produce Strategies • Strategies implement interchangeable policies • Concurrency strategies use Reactor & Leader/Followers • Acceptor-Connector decouples connection management from request processing • Managers optimize request demultiplexing www.cs.wustl.edu/~schmidt/PDF/ORB-patterns.pdf 54 Enhancing ORB Flexibility w/the Strategy Pattern Context Problem Solution • Multi-domain • Flexible ORBs must support multiple resuable event & request demuxing, scheduling, middleware (de)marshaling, connection mgmt, framework request transfer, & concurrency policies Hook for marshaling strategy Hook for the event demuxing strategy Hook for the connection management strategy • Apply the Strategy pattern to factory out commonality amongst variable ORB algorithms & policies Hook for the request demuxing strategy Hook for the concurrency strategy Hook for the underlying transport strategy 55 Consolidating Strategies with the Abstract Factory Pattern Context Problem Solution • A heavily strategized framework or application • Aggressive use of Strategy pattern creates a configuration nightmare • Apply the Abstract Factory pattern to consolidate multiple ORB strategies into semantically compatible configurations • Managing many individual strategies is hard • It’s hard to ensure that groups of semantically compatible strategies are configured Concrete factories create groups of strategies 56 Dynamically Configuring Factories w/the Component Configurator Pattern Context Problem Solution • Resource • Prematurely commiting to a particular ORB • Apply the Component constrained configuration is inflexible & inefficient Configurator pattern & highly to assemble the • Certain decisions can’t be made until dynamic desired ORB factories runtime environments (& thus strategies) • Forcing users to pay for components dynamically they don’t use is undesirable • ORB strategies are decoupled from when the strategy implementations are configured into an ORB • This pattern can reduce the memory footprint of an ORB 57 ACE Frameworks Used in TAO • Reactor drives the ORB event loop • Implements the Reactor & Leader/Followers patterns • Acceptor-Connector decouples passive/active connection roles from GIOP request processing • Implements the AcceptorConnector & Strategy patterns • Service Configurator dynamically configures ORB strategies • Implements the Component Configurator & Abstract Factory patterns 58 www.cs.wustl.edu/~schmidt/PDF/ICSE-03.pdf Summary of Pattern, Framework, & Middleware Synergies These technologies codify expertise of skilled researchers & developers • Frameworks codify expertise in the form of reusable algorithms, component implementations, & extensible architectures Application-specific functionality Acceptor Connecto r • Patterns codify expertise in the form of reusable architecture design themes & styles, which can be reused event when algorithms, components implementations, or frameworks cannot • Middleware codifies expertise in the form of standard interfaces & components that provide applications with a simpler façade to access the powerful (& complex) capabilities of frameworks Stream Component Configurator Task Reactor Proactor There are now powerful feedback loops advancing these technologies 59 Example: Electronic Medical Imaging Systems Goal •Route, manage, & manipulate electronic medical images robustly, efficiently, & securely thoughout a distributed health care environment Modalities e.g., MRI, CT, CR, Ultrasound, etc. 60 Example: Electronic Medical Imaging Systems System Characteristics •Large volume of “blob” data •e.g.,10 to 40 Mbps •“Lossy” compression isn’t viable due to liability concerns •Diverse QoS requirements, e.g., •Sync & async communication •Event- & method-based invocation •Streaming communication •Prioritization of requests & streams •Distributed resource management 61 Modalities e.g., MRI, CT, CR, Ultrasound, etc. Example: Electronic Medical Imaging Systems Key Software Solution Characteristics (e.g., www.syngo.com) •Affordable, flexible, & COTS •General-purpose & embedded • Product-line architecture OS platforms • Design guided by patterns & frameworks •Middleware technology agnostic Modalities e.g., MRI, CT, CR, Ultrasound, etc. 62 Image Acquisition Scenario Diagnostic & Clinical Workstations Naming Service 4. Find Factory Radiology Client 63 7.New 3. Bind Factory 13. Activate 14. Delegate 5. Find Image Factory Proxy 8. New Xfer Proxy Image Database 10. Invoke get_image() call 2. Enter info 9. Return Ref 12. Check Authorization Image Xfer Component Executor Container 11. Query Config. Key Tasks 1.Image location & routing 2.Image delivery Image Xfer Interface Configuration Database Security Service 1. Deploy Configuration 6. Intercept & delegate Factory/Finder Applying Patterns to Resolve Key Distributed System Design Challenges Naming Service Proxy Layers Extension Interface Factory Proxy PublisherSubscriber Messaging Radiology Client Container Xfer Proxy Broker Configuration Database Security Service Image Xfer Interface Image Xfer Component Servant Component Configurator Active Object Image Database Activator Configuration Interceptor Factory/Finder Factory/Finder Patterns help resolve the following common design challenges: •Separating concerns between tiers •Improving type-safety & performance •Enabling client extensibility •Ensuring platform-neutral & networktransparent OO communication •Supporting async communication •Supporting OO async communication 64 •Decoupling suppliers & consumers •Locating & creating components scalably •Extending components transparently •Minimizing resource utilization •Enhancing server (re)configurability Separating Concerns Between Tiers Context • Distributed systems are now common due to the advent of • The global Internet • Ubiquitous mobile & embedded devices Solution Problem • It’s hard to build distributed systems due to the complexity associated with many capabilities at many levels of abstraction Presentation Tier • e.g., thin client displays •Apply the Layers pattern (P1) to create a multi-tier architecture that separates concerns between groups of tasks occurring at distinct Middle Tier layers in the distributed system • e.g., common Client Client Application comp comp business logic Services in the middle tier participate in various types of tasks, e.g., • Workflow of integrated “business” processes Database Tier • Connect to databases & other • e.g., persistent backend systems for data storage data 65 & access Server DB Server DB Server Applying the Layers Pattern to Image Acquisition Presentation Tier •e.g., radiology clients Middle Tier •e.g., image routing, security, & image transfer logic Database Tier •e.g., persistent image data 66 Diagnostic Workstations Clinical Workstations Image comp comp • Their clients are containers that provide all the resources Servers Image Database Diagnostic & clinical workstations are presentation tier entities that: •Typically represent sophisticated GUI elements •Share the same address space with their clients Patient Database •Exchange messages with the middle tier components Image servers are middle tier entities that: •Provide server-side functionality •e.g., they are responsible for scalable concurrency & networking •Can run in their own address space •Are integrated into containers that hide low-level OS platform details Pros & Cons of the Layers Pattern This pattern has four benefits: •Reuse of layers • If an individual layer embodies a welldefined abstraction & has a well-defined & documented interface, the layer can be reused in multiple contexts •Support for standardization • Clearly-defined & commonly-accepted levels of abstraction enable the development of standardized tasks & interfaces •Dependencies are localized • Standardized interfaces between layers usually confine the effect of code changes to the layer that is changed •Exchangeability • Individual layer implementations can be replaced by semantically-equivalent implementations without undue effort 67 This pattern also has liabilities: •Cascades of changing behavior • If layer interfaces & semantics aren’t abstracted properly then changes can ripple when behavior of a layer is modified •Higher overhead • A layered architecture can be less efficient than a monolithic architecture •Unnecessary work • If some services performed by lower layers perform excessive or duplicate work not actually required by the higher layer, performance can suffer •Difficulty of establishing the correct granularity of layers • It’s important to avoid too many & too few layers Overview of Distributed Object Computing Communication Mechanisms Context Problem In multi-tier systems both the tiers & the components within the tiers must be connected via communication mechanisms •A single communication mechanism does not fit all uses! Solution • DOC middleware provides multiple types of communication mechanisms: • Collocated client/server (i.e., native function call) • Synchronous & asynchronous RPC/IPC • Group communication & events • Data streaming We now explore various distribution infrastructure (P4) patterns that applications can apply to leverage these communication mechanisms 68 Core Distribution Infrastructure Patterns Pattern Communication Communication Component Style Relationship Dependencies Broker Remote Method One-to-One Invocation Component Interfaces Messaging Message Many-to-One Communication Endpoints & Message Formats Publisher/ Subscriber Events One-to-Many Event Formats • Broker makes invocations on remote component objects look & act as much as possible like invocations on component objects in the same address space as their clients • Messaging & Publisher-Subscriber are most appropriate for integration scenarios where multiple, independently developed & self-contained services or applications must collaborate & form a coherent software system 69 Improving Type-safety & Performance (1/2) Context • The configuration of components in distributed systems is often subject to change as requirements evolve Problems • Low-level message passing (e.g., using sockets) is errorprone & fraught with accidental complexity • Remote components should look like local components from an application perspective • i.e., ideally clients & servers should be oblivious to communication mechanisms & locations 70 Improving Type-safety & Performance (2/2) AbstractService Client Solution Apply the Proxy design pattern (P1, GoF) to provide an OO surrogate through which clients can access remote objects : Client : Proxy : Service service pre-processing: e.g.,marshaling service service Proxy service 1 1 Service service • A Service implements the object, which is not accessible directly • A Proxy represents the Service & ensures the correct access to it • Proxy offers same interface as Service post-processing: e.g., unmarshaling 71 • Clients use the Proxy to access the Service Applying the Proxy Pattern to Image Acquisition We can apply the Proxy pattern to provide a strongly-typed interface to initiate & coordinate the downloading of images from an image database Image Xfer Client get_image() Proxy get_image() Xfer Proxy Radiology Client Invoke get_image() call 1 1 Sync Image Xfer get_image() Image Xfer Service Image Database Proxies that are generated automatically by middleware can be optimized to be much more efficient than manual message passing •e.g., improved memory management, data copying, & compiled marshaling/demarshaling 72 Pros & Cons of the Proxy Pattern This pattern provides three benefits: •Decoupling clients from the location of server components • Clients are not affected by migration of servers or changes in the networking infrastructure since location information & addressing functionality is managed by a proxy •Separation of housekeeping & functionality • A proxy relieves clients from burdens that do not inherently belong to the task the client performs This pattern has three liabilities: •Potential overkill via sophisticated strategies • If proxies include overly sophisticated functionality, such as caching or replica management, they many introduce overhead that defeats their intended purpose •Higher overhead due to indirection • Proxies introduce an additional layer of indirection that can be excessive if the proxy •Potential for time & space implementation is inefficient optimizations • Proxy implementations can be loaded “on- • Overly restrictive type system demand” & can also be used to cache • Proxies enforce typechecking on values to avoid redundant remote calls client side • Proxies can also be optimized to improve both type-safety & performance 73 Enabling Client Extensibility (1/2) Context • Object models define how Class X operation1() operation2() operation3() operationn() components import & export functionality • e.g., UML class diagrams specify well-defined OO interfaces Problem • Many object models assign a single interface to each component • This design makes it hard to evolve components without either 1. Breaking existing client interfaces 2. Bloating client interfaces 74 : Client operation() Object : Class X Enabling Client Extensibility (2/2) Solution • Apply the Extension Interface design pattern (P2), which allows multiple interfaces to be exported by a component, to prevent breaking of client code & bloating of interfaces when developers extend or modify component functionality queryInterface() : Client operation() Ask for a reference to an interface Client callService Root queryInterface Call an operation on an interface * CreateInstance <<extends>> * Factory createComponent 1 * new Component createComponent 1+ Server initialize 75 uninititialize Implemented by Extension Interface i queryInterface service_i Extension Interface Pattern Dynamics : Client : Factory Start_client createInstance(Ext.Intf. 1) : Component new : Extension 1 : Extension 2 create Ref. To Extension1 service_1 queryInterface(Extension Interface 2) create Ref. To Extension2 service_2 service2 Note how each extension interface can serve as a “factory” to return object reference to other extension interfaces supported by a component 76 A common use of the Extension Interface pattern is to support component versioning • Off-loads versioning protocol to client… Pros & Cons of the Extension Interface Pattern This pattern has five benefits: •Separation of concerns • Interfaces are strictly decoupled from implementations & can be strongly type checked •Exchangeability of components • Component implementations can evolve independently from clients that access them •Extensibility through interfaces • Clients only access components via their interfaces, which reduces coupling to representation & implementation details •Prevention of interface bloating • Interfaces need not contain all possible methods, just the ones associated with a particular capability •No subclassing required • Delegation—rather than inheritance—is used to customize components 77 This pattern also has liabilities: •Higher overhead due to indirection • Clients must incur the overhead of several round-trips to obtain the appropriate object reference from a server component •Complexity & cost for development & deployment • This pattern off-loads the responsibility for determining the appropriate interface from the component designer to the client applications Ensuring Platform-neutral & Networktransparent OO Communication (1/2) Context •The Proxy & Extension Interface patterns aren’t sufficient since they don’t address how •Remote components are located •Connections are established •Messages are exchanged across a network •etc. Problem •A middleware architecture needs to: •Support remote method invocation •Provide location transparency •Detect & recover from faults •Allow the addition, exchange, or remove of services dynamically •Hide system details from developers 78 AbstractService Client service Proxy service 1 1 Service service Ensuring Platform-neutral & Networktransparent OO Communication (2/2) Solution •Apply the Broker architectural pattern (P1) to provide OO platform-neutral communication between networked client & server components internal partitioning Layers Broker request Object Adapter dispatching OS abstraction Wrapper Facade Requestor request issuing Invoker 79 component discovery component access component access Client Proxy Business Delegate component creation error notification Component Lookup request encapsulation Container Facade request reception Message Remoting Error broker access request dispatching Broker configuration publishsubscribe communication PublisherSubscriber Factory Method Broker Pattern Dynamics : Client : Client Proxy : Server : Server Proxy : Broker register_service method (proxy) start_up assigned port locate_server server port marshal send_request unmarshal dispatch method (impl.) result marshal receive_reply unmarshal result Broker middleware generates the necessary client & server proxies from higher level interface definitions 80 Interface Specif. Proxy Generator Proxy Code Applying the Broker Pattern to Image Acquisition Container Client Client Proxy Image Xfer Component Broker Interface Server Proxy • Common Broker pattern Common implementations Services •CORBA •COM+ •Java RMI •Med-specific Comm (MSC) • Brokers define interfaces… Object Adapter • … not implementations • Brokers simplify development of distributed applications by Communication Infrastructure automating •Object location OS & OS & •Connection management Protocols Protocols •Memory management •Parameter (de)marshaling •Brokers help shield distributed applications •Event & request demuxing from environment heterogeneity •Error handling •e.g., programming languages, operating •Object/server activation systems, networking protocols, hardware, •Concurrency etc. 81 Pros & Cons of the Broker Pattern This pattern has five benefits: • Portability enhancements This pattern also has liabilities: •Higher overhead • A broker hides OS & network system details from clients & servers by using indirection & abstraction layers, such as APIs, proxies, adapters, & bridges • Interoperability with other brokers • Applications using brokers may be slower than applications written manually •Potentially less reliable • Different brokers may interoperate if they understand a common protocol for exchanging • Compared with non-distributed messages software applications, • Reusability of services distributed broker systems may • When building new applications, brokers incur lower fault tolerance enable application functionality to reuse •Testing & debugging may be existing services • Location transparency • A broker is responsible for locating servers, so clients need not know where servers are located •Changeability & extensibility of components 82 • If server implementations change without affecting interfaces clients should not be affected harder • Testing & debugging of distributed systems is tedious because of all the components involved Supporting Async Communication (1/2) Context • Some clients want to send requests, continue their work, & receive the results at some later point in time Problem •Broker implementations based on conventional RPC semantics often just support blocking operations •i.e., clients must wait until twoway invocations return •Unfortunately, this design can reduce scalability & complicate certain use-cases 83 Supporting Async Communication (1/2) Solution •Apply the Messaging (P4/EIP) pattern to allow asynchronous communication between clients & servers Messaging data encapsulation data format transformation Message Translator Message Message Channel Message Endpoint 84 data transfer component connection data routing Message Router error notification Remoting Error Introduce intermediary queue(s) between clients & servers: • A queue is used to store messages • A queue can cooperate with other queues to route messages • Messages are sent from sender to receiver • A client sends a message, which is queued & then forwarded to a message processor on a server that receives & executes them • A Message API is provided for clients & servers to send/receive messages Messaging Pattern Dynamics : Message API : Client create send : Local Queue : Remote Queue : Message API : Message Message Message store : Message Processor Message forward receive Other processing create receive Message Message exec Reply Reply recv 85 Reply recv Reply forward store send Applying the Messaging Pattern to Image Acquisition We can apply the Messaging pattern to Radiology •Queue up image request messages remotely Client without blocking the diagnostic/clinical workstation clients •Execute the requests at a later point & return the results to the client <<send>> Message API Local Queue store forward remove Message <<route>> Remote Queue Radiology Client Image Xfer Service store forward remove <<exec>> Image Database Image Server Message Processor <<recv>> Message API This design also enables other, more advanced capabilities, e.g., •Multi-hop store & forward persistence •QoS-driven routing, where requests can be delivered to the “best” image database depending on context 86 Pros & Cons of the Messaging Pattern This pattern provides three benefits: This pattern also has some •Enhances concurrency by transparently liabilities: leveraging available parallelism •Message execution order can • Messages can be executed remotely on differ from message servers while clients perform other invocation order processing •Simplifies synchronized access to a shared object that resides in its own thread of control • Since messages are processed serially by a message processor target objects often need not be concerned with synchronization mechanisms •Message execution order can differ from message invocation order • This allows reprioritizing of messages to enhance quality of service • Messages can be “batched” & sent wholesale to enhance throughout 87 • As a result, clients must be careful not to rely on ordering dependencies •Type-safety becomes server’s responsibility • Clients & servers are responsible for formatting & passing messages •Complicated debugging • As with all distributed systems, debugging & testing is complex Supporting OO Async Communication (1/2) Problem Context • Some clients want to invoke remote operations, continue their work, & retrieve the results at a later point in time •Using the explicit message-passing API of the Messaging pattern can reduce typesafety & performance •Similar to motivation for Proxy pattern... Request messages Reply messages Request messages Client 88 Server Supporting OO Async Communication (2/2) Solution •Apply the Active Object design pattern (P2) to decouple method invocation from method execution using a strongly-typed OO programming model Proxy method_1 method_n creates Future Activation List Scheduler enqueue dispatch creates enqueue dequeue maintains * MethodRequest guard call Servant method_1 method_n • A proxy provides an interface that allows clients to access methods of an object • A concrete method request is created for every method invoked on the proxy • A scheduler receives the method requests & dispatches them on the servant when they become runnable • An activation list maintains pending method requests • A servant implements the methods Concrete MethodRequest1 89 Concrete MethodRequest2 • A future allows clients to access the results of a method call on the proxy Active Object Pattern Dynamics : Client : Proxy : Servant : Scheduler method : Future enqueue dispatch : Method Request call method write read Clients can obtain result from futures via blocking, polling, or callbacks 90 • A client invokes a method on the proxy • The proxy returns a future to the client, & creates a method request, which it passes to the scheduler • The scheduler enqueues the method request into the activation list (not shown here) • When the method request becomes runnable, the scheduler dequeues it from the activation list (not shown here) & executes it in a different thread than the client • The method request executes the method on the servant & writes results, if any, to the future • Clients obtain the method’s results via the future Applying the Active Object Pattern to Image Acquisition •OO developers often prefer methodActivation Proxy Scheduler oriented request/response semantics List to message-oriented semantics method_1 enqueue enqueue method_n dispatch dequeue •The Active Object pattern supports this preference via strongly-typed creates creates * maintains async method APIs: • Several types of parameters can be passed: • Requests contain in/inout arguments • Results carry out/inout arguments & results • Callback object or poller object can be used to retrieve results Future MethodRequest guard call get_image() put_image() get_image() Radiology Client 91 Image Xfer Service future results Servant method_1 method_n Image Database Pros & Cons of the Active Object Pattern This pattern provides four benefits: •Enhanced type-safety & efficiency • Cf. asynchronous message passing •Enhances concurrency & simplifies synchronized complexity This pattern also has some liabilities: • Higher overhead • Depending on how an active object’s scheduler is • Concurrency is enhanced by allowing client threads implemented, context & asynchronous method executions to run switching, synchronization, & simultaneously data movement overhead may • Synchronization complexity is simplified by using a occur when scheduling & scheduler that evaluates synchronization executing active object constraints to serialized access to servants invocations •Transparent leveraging of available parallelism • Multiple active object methods can execute in parallel if supported by the OS/hardware •Method execution order can differ from method invocation order • Methods invoked asynchronous are executed according to the synchronization constraints defined by their guards & by scheduling policies • Methods can be “batched” & sent wholesale to 92 enhance throughput • Complicated debugging • It is hard to debug programs that use the Active Object pattern due to the concurrency & non-determinism of the various active object schedulers & the underlying OS thread scheduler Decoupling Suppliers & Consumers (1/2) Context •In large-scale electronic medical imaging systems, radiologists may share “work lists” of patient images to balance workloads effectively Problem •Having each client call a specific server is inefficient & non-scalable • A “polling” strategy leads to performance bottlenecks • Work lists could be spread across different servers • More than one client may be interested in work list content Radiology Client Radiology Client Radiology Client 93 Radiology Client Image Database Radiology Client Decoupling Suppliers & Consumers (2/2) Solution •Apply the Publisher-Subscriber architectural pattern (P1) to decouple image suppliers from consumers Decouple suppliers (publishers) & consumers (subscribers) of events: Publisher produce Event Channel attachPublisher detachPublisher attachSubscriber detachSubscriber pushEvent creates * Event Subscriber consume • An Event Channel stores/forwards events • Publishers create events & store them in a queue maintained by the Event Channel • Consumers register with event queues, from which they retrieve events receives Filter filter • Events are used to transmit state change info from publishers to consumers • For event transmission push-models & pull-models are possible • Filters can filter events for consumers 94 Publisher-Subscriber Pattern Dynamics : Publisher •The Publisher-Subscriber pattern helps keep the state of cooperating components synchronized •To achieve this it enables one-way propagation of changes: one publisher notifies any number of subscribers about changes to its state : Event Channel : Subscriber attachSubscriber produce : Event pushEvent event pushEvent event consume detachSubscriber Key design considerations for the Publisher-Subscriber pattern include: •Push vs. pull interaction models •Control vs. data event notification models •Multicast vs. unicast communication models •Persistence vs. transient queueing models 95 Applying the Publisher-Subscriber Pattern to Image Acquisition •Radiologists can subscribe to Event Channel Modality Radiologist an event channel to receive attachPublisher detachPublisher notifications produced when produce consume attachSubscriber modalities publish events detachSubscriber pushEvent indicating the arrival of new images & studies creates receives •This design enables a group of * distributed radiologists to Event Filter collaborate effectively in a filter networked environment Radiology Client Radiology Radiology Client Client Event Radiology Radiology Channel Client Client Image Database 96 Pros & Cons of the PublisherSubscriber Pattern This pattern has two benefits: •Decouples consumers & producers of events • All an event channel knows is that it has a list of consumers, each conforming to the simple interface of the Subscriber class • The coupling between the publishers & subscribers is therefore abstract, anonymous, & minimal •n:m communication models are supported • Unlike an ordinary sync/async request/response invocation, the notification that a publisher sends needn’t designate its receiver, which enables a broader range of communication topologies, including multicast & broadcast 97 There are also liabilities: •Must be careful with potential update cascades • Since subscribers have no knowledge of each other’s presence, applications may not recognize the ultimate cost of publishing events through an event channel • A seemingly innocuous operation on the subject may therefore cause a cascade of updates to observers & their dependent objects •Performance/complexity degradation relative to point-topoint request/response interactions Locating & Creating Components Scalably (1/2) Context • Our electronic medical imaging system contains many components distributed in a network Problem •How to create new components and/or find existing ones • Simple solutions appropriate for stand-alone applications don’t scale • “Obvious” distributed solutions also don’t scale 98 Modalities e.g., MRI, CT, CR, Ultrasound, etc. Locating & Creating Components Scalably (2/2) Solution •Apply the Factory/Finder (SCP) design pattern to separate the management of component lifecycles from their use by client applications AbstractHome AbstractComp find create operation • An Abstract Home declares an interface for operations that find and/or create abstract instances of components • Concrete Homes implements the abstract Home interface to find specific instances and/or create new ones • Abstract Comp declares interface for a specific type of component class • Concrete Comp define instances • A Primary Key is associated with a component ConcreteComp ConcreteHome operation find create Primary Key 99 Factory/Finder Pattern Dynamics : Client •The Factory/Finder pattern is supported by distributed component models •e.g., EJB, COM+, & the CCM create : Home : Component : Primary Key find (“ImageXYZ”); Primary Key lookup Component operation Node Client getName Binding getObject 100 * Directory resolve listNodes navigate newBinding newSubDir remove • Homes enable the creation & location of components, but we still need some type of generalpurpose naming/directory service to locate the homes Applying the Factory/Finder Pattern to Image Acquisition •We can apply the Factory/Finder pattern to create/locate image transfer components for images needed by radiologists •If a suitable component already exists the component home will use it, otherwise, it will create a new component 3. Find Factory 101 AbstractComp find create operation ImageXferComp ImageXferHome operation find create Primary Key Naming Service Image Xfer Interface 2. Bind Factory 6. Find Image Factory Proxy Radiology Client AbstractHome Container 5.New 4. Intercept & delegate Factory/Finder Image Database 1. Deploy Configuration Pros & Cons of the Factory/Finder Pattern This pattern has three benefits: •Separation of concerns • Finding/creating individual components is decoupled from locating the factories that find/create these components •Improved scalability • e.g., general-purpose directory mechanisms need not manage the creation & location of large amounts of finer-grained components whose lifetimes may be short •Customized capabilities • The location/creation mechanism can be specialized to support key capabilities that are unique for various types of components 102 This pattern also has some liabilities: •Overhead due to indirection • Clients must incur the overhead of several round-trips to obtain the appropriate object reference •Complexity & cost for development & deployment • There are more steps involved in obtaining object references, which can complicate client programming Extending Components Transparently Context •Component developers may not know a priori the context in which their components will execute •Thus, containers are introduced to: • Shield clients & components from the details of the underlying middleware, services, network, & OS • Manage the lifecycle of components & notify them about lifecycle events • e.g., activation, passivation, & transaction progress • Provide components with uniform access to middleware infrastructure services • e.g., transactions, security, & persistence 103 • Register & deploy components Client Client Container Server Component Server Component Transaction Security Resources ... Transaction Security Resources ... Declarative Programming ... Imperative Programming Extending Components Transparently (cont‘d) Problem Solution • Developers should be able to specify •Apply the Interceptor architectural declaratively what type of execution pattern (P2) to attach interceptors environment components need to a framework that can handle • e.g., in configuration files or databases • Containers must be able to transparently particular events by invoking associated interceptors provide the right execution environment automatically • e.g., by creating a new transaction or new servant when required Context provide Framework attach_interceptor manage_interceptors AbstractInterceptor handle_event callback attach ConcreteInterceptor handle_event 104 * Dispatcher • Framework represents the concrete framework to which we attach interceptors • Concrete Interceptors implement the event handler for the system-specific events they have subscribed for • Context contains information about the event & allows modification of system behavior after interceptor completion • The Dispatcher allows applications to register & remove interceptors with the framework & to delegate events to interceptors Interceptor Pattern Dynamics : Framework : Application •Interceptor are a “metaprogramming mechanism,” create : Interceptor : Dispatcher along with attach •Smart proxies interceptor •Pluggable protocols Place interceptor in internal interceptor •Gateways/bridges run_event_loop map •Interface repositories event •These mechanisms provide create : Context building-blocks to handle delegate (often unanticipated) variation translucently & reflectively Look for handle_event registered context •More information on metainterceptors programming mechanisms can be found at www.cs.wustl.edu/~schmidt/PDF/IEEE.pdf • Interception is commonly used to handle security & transactions 105 transparently from the perspective of a component implementation • It can also enable performance enhancement strategies • e.g., just-in-time activation, object pooling, load balancing, & caching Applying the Interceptor Pattern to Image Acquisition • A container provides generic interfaces to a component that it can use to access container functionality • e.g., transaction control, persistence, Context provide attach_interceptor manage_interceptors AbstractInterceptor handle_event security,load balancing etc. callback • A container intercepts all incoming requests from clients, e.g., • It can read the component’s requirements from a XML configuration file • It can then do some pre-processing before actually delegating the request to the component • A component provides interfaces that its container invokes automatically when particular events occur • e.g., activation or passivation 106 Image Server Framework attach Container * Dispatcher handle_event Container XML config Component Interceptors are used for many other middleware tasks, as well Pros & Cons of the Interceptor Pattern This pattern has five benefits: •Extensibility & flexibility • Interceptors allow an application to evolve without breaking existing APIs & components •Separation of concerns • Interceptors decouple the “functional” path from the “meta” path •Support for monitoring & control of frameworks • e.g., generic logging mechanisms can be used to unobtrusively track application behavior •Layer symmetry • Interceptors can perform transformations on a client-side whose inverse are performed on the server-side •Reusability 107 • Interceptors can be reused for various general-purpose behaviors This pattern also has liabilities: •Complex design issues • Determining interceptor APIs & semantics is non-trivial •Malicious or erroneous interceptors • Mis-behaving interceptors can wreak havoc on application stability •Potential interception cascades • Interceptors can result in infinite recursion Minimizing Resource Utilization (1/2) Context Problem • Image servers are simply • It may not feasible to have all image one of many services running throughout a distributed electronic medical image system 108 server implementations running all the time since this ties up end-system resources unnecessarily Minimizing Resource Utilization (2/2) Solution • Apply the Activator (PLoP12/P4) design pattern to automate scalable ondemand activation & deactivation of service execution contexts to run services accessed by many clients without consuming resources unnecessarily •When incoming requests arrive, the Activator looks up whether a target object is already active & if the object is not running it activates the Service Execution Context •The Activation Table stores associations between services & their physical location •The Client uses the Activator to get service access •A Service implements a specific type of functionality that it provides to clients 109 www.cs.wustl.edu/~schmidt/PDF/Activator.pdf Activator Pattern Dynamics Service Registration • An Activator can be used to activate & passivate a server • e.g., after each method call, after each transaction, etc. • A container/component in a server can also passivate the server itself 110 Service Activation & Deactivation Applying the Activator Pattern to Image Acquisition Client •We can use the Activator pattern to launch image transfer servers on-demand Activation Table useService() lookup() insert() delete() getService •The Activator pattern is available in various COTS technologies: •UNIX Inetd “super server” •CORBA Implementation Repository 1. some_request Client Activator createService() findService() activate() deactivate() addService() remService() ImageXferService service shutdown ImR (ringil:5000) iiop://ringil:5000/poa_name/object_name 4. LOCATION_FORWARD iiop://ringil:5500/poa_name/object_name poa_name server.exe airplane_poa plane.exe ringil:5500 ringil:4500 2. ping 3. is_running 2.1 start 5. some_request 6. some_response 111 Server (ringil:5500) Pros & Cons of the Activator Pattern This pattern has three benefits: •More effective resource utilization • Servers can be spawned “on-demand,” thereby minimizing resource utilization until clients actually require them •Uniformity • By imposing a uniform activation interface to spawn & control servers •Modularity, testability, & reusability • Application modularity & reusability is improved by decoupling server implementations from the manner in which the servers are activated This pattern also has liabilities: •Lack of determinism & ordering dependencies • This pattern makes it hard to determine or analyze the behavior of an application until its components are activated at runtime •Reduced security or reliability • An application that uses the Activator pattern may be less secure or reliable than an equivalent statically-configured application •Increased run-time overhead & infrastructure complexity • By adding levels of abstraction & indirection when activating & executing components 112 Enhancing Server (Re)Configurability (1/2) Context Problem The implementation of certain image server components depends on a variety of factors: Prematurely committing to a particular image server component configuration is inflexible & inefficient: •Certain factors are known at deployment time, such as the number of available CPUs & operating system support for asynchronous I/O •Other factors are known at runtime, such as system workload • No single image server configuration is optimal for all use cases • Certain design decisions cannot be made efficiently until run-time Cache Mgmt Conn Mgmt Demuxing 113 Image Processing Threading I/O File System Enhancing Server (Re)Configurability (2/2) Solution •Apply the Component Configurator design pattern (P2) to enhance server configurability •This pattern allows an application to link & unlink its component implementations at run-time •Thus, new & enhanced services can be added without having to modify, recompile, statically relink, or shut down & restart a running application 114 Component Component * init() components fini() Repository suspend() <<contains>> resume() info() Component Configurator Concrete Concrete Component A Component B Component Configurator Pattern Dynamics : Component Configurator : Concrete Component A : Concrete Component B : Component Repository init() 1.Component initialization & dynamic linking Concrete Comp. A insert() init() Concrete Comp. B insert() run_component() 2.Component processing run_component() fini() 3.Component termination & dynamic unlinking 115 Concrete Comp. A remove() fini() Concrete Comp. B remove() Applying the Component Configurator Pattern to Image Acquisition Image servers can use the Component Configurator pattern to dynamically optimize, control, & reconfigure the behavior of its components at installation-time or during run-time Component Component * init() components Repository fini() suspend() <<contains>> resume() info() Component Configurator •For example, an image server can apply the Component Configurator pattern to configure various Cached Virtual Filesystem strategies •e.g., least-recently used (LRU) or least-frequently used (LFU) Concrete components can be packaged into a suitable unit of configuration, such as a dynamically linked library (DLL) 116 LRU File Cache LFU File Cache Only the components that are currently in use need to be configured into an image server Reconfiguring an Image Server Image servers can also be reconfigured dynamically to support new components & new component implementations Image Server Reconfiguration State Chart IDLE TERMINATE fini() TERMINATE fini() LRU File Cache # Configure an image server. dynamic File_Cache Component * img_server.dll:make_File_Cache() "-t LRU" INITIAL CONFIGURATION 117 CONFIGURE init() RECONFIGURE init() RUNNING RESUME resume() SUSPENDED Image Server SUSPEND suspend() EXECUTE run_component() LFU File Cache # Reconfigure an image server. remove File_Cache dynamic File_Cache Component * img_server.dll:make_File_Cache() "-t LFU" AFTER RECONFIGURATION Pros & Cons of the Component Configurator Pattern This pattern offers four benefits: •Uniformity • By imposing a uniform configuration & control interface to manage components •Centralized administration This pattern also incurs liabilities: •Lack of determinism & ordering dependencies • This pattern makes it hard to determine or analyze the behavior of an application until its components are configured at run-time • By grouping one or more components into a single administrative unit that simplifies development by centralizing common •Reduced security or reliability component initialization & termination • An application that uses the activities Component Configurator pattern may be less secure or reliable than an •Modularity, testability, & reusability equivalent statically-configured • Application modularity & reusability is application improved by decoupling component implementations from the manner in which •Increased run-time overhead & the components are configured into infrastructure complexity processes • By adding levels of abstraction & •Configuration dynamism & control indirection when executing • By enabling a component to be components dynamically reconfigured without •Overly narrow common interfaces modifying, recompiling, statically relinking • The initialization or termination of a existing code & without restarting the component may be too complicated or component or other active components too tightly coupled with its context to 118 with which it is collocated be performed in a uniform manner Example: High-performance Content Delivery Servers GET /index.html HTTP/1.0 HTTP Server HTTP Client www.posa.uci.edu <H1>POSA page</H1>... HTML File Protocol Parser Cache Handlers GUI Event Dispatcher Requester Graphics Adapter Transfer Protocol Goal •Download content scalably & efficiently •e.g., images & other multi-media content types Key System Characteristics •Robust implementation e.g. , HTTP 1.0 • e.g., stop malicious clients •Extensible to other protocols & Protocols • e.g., HTTP 1.1, IIOP, DICOM OS Kernel OS Kernel TCP/IP Network & Protocols Key Solution Characteristics •Support many content delivery server design alternatives seamlessly • e.g., different concurrency & event models •Design is guided by patterns to leverage time-proven solutions 119 •Leverage advanced multiprocessor hardware & software • Implementation based on COTS framework components to reduce effort & amortize prior work • Open-source to control costs & to leverage technology advances JAWS Content Server Framework Key Sources of Variation • Concurrency models • e.g.,thread pool vs. threadper-connection • Event demultiplexing models • e.g.,sync vs. async • File caching models • e.g.,LRU vs. LFU • Content delivery protocols • e.g.,HTTP 1.0+1.1 vs. IIOP • Operating system APIs • e.g., Windows, UNIX, RTOS Event Dispatcher • Accepts client connection request events, receives HTTP GET requests, & coordinates JAWS’s event demultiplexing strategy with its concurrency strategy 120 Protocol Handler • Performs parsing & protocol processing of HTTP request events. Cached Virtual Filesystem • Improves Web server performance by reducing the overhead of file system accesses when processing HTTP GET requests Applying Patterns to Resolve Key JAWS Design Challenges Component Configurator Acceptor-Connector Double-checked Locking Optimization Thread-safe Interface Strategized Locking Scoped Locking Leader/Followers Proactor Half-Sync/ Half-Async Monitor Object Reactor Wrapper Facade Thread-specific Storage Patterns help resolve the following common design challenges: • Encapsulating low-level OS APIs • Decoupling event demuxing & connection management from protocol processing • Scaling up performance via threading • Implementing a synchronized request queue • Minimizing server threading overhead • Using asynchronous I/O effectively 121 • Efficiently demuxing asynchronous operations & completions • Enhancing server (re)configurability • Transparently parameterizing synchronization into components • Ensuring locks are released properly • Minimizing unnecessary locking • Synchronizing singletons correctly • Logging access statistics efficiently Encapsulating Low-level OS APIs (1/2) Context • A Web server must manage a variety of OS services, including processes, threads, Socket connections, virtual memory, & files Applications • OS platforms provide low-level APIs written in C to access these services Problem • The diversity of hardware & operating systems makes it hard to build portable & robust Web server software • Programming directly to low-level OS APIs is tedious, error-prone, & non-portable 122 Solaris Win2K VxWorks Linux LynxOS Encapsulating Low-level OS APIs (2/2) Solution • Apply the Wrapper Facade design pattern (P2) to avoid accessing low-level operating system APIs directly & efficiently Wrapper Facade calls data calls method1() … methodN() calls API FunctionA() calls methods Application This pattern encapsulates data & functions provided by existing non-OO APIs within more concise, robust, portable, maintainable, & cohesive OO class interfaces void method1(){ functionA(); functionB(); } : Application API FunctionB() API FunctionC() void methodN(){ functionA(); } : Wrapper Facade : APIFunctionA : APIFunctionB method() functionA() functionB() 123 Applying the Wrapper Façade Pattern in JAWS JAWS uses the wrapper facades defined by ACE to ensure its framework components can run on many OS platforms • e.g., Windows, UNIX, & many real-time operating systems For example, JAWS uses the ACE_Thread_Mutex wrapper facade in ACE to provide a portable interface to OS mutual exclusion mechanisms JAWS calls methods calls mutex calls acquire() tryacquire() release() void acquire() { mutex_lock(mutex); } The ACE_Thread_Mutex wrapper in the diagram is implemented using the Solaris thread API ACE_Thread_Mutex is also available for other threading APIs, e.g., VxWorks, LynxOS, Windows, or POSIX threads download.dre.vanderbilt.edu 124 ACE_Thread_Mutex calls mutex_lock() mutex_trylock() mutex_unlock() void release() { mutex_unlock(mutex); } Other ACE wrapper facades used in JAWS encapsulate Sockets, process & thread management, memory-mapped files, explicit dynamic linking, & time operations Pros & Cons of the Wrapper Façade Pattern This pattern provides three benefits: •Concise, cohesive, & robust higherlevel object-oriented programming interfaces • These interfaces reduce the tedium & increase the type-safety of developing applications, which descreases certain types of programming errors •Portability & maintainability • Wrapper facades can shield application developers from non-portable aspects of lower-level APIs •Modularity, reusability & configurability • This pattern creates cohesive & reusable class components that can be ‘plugged’ into other components in a wholesale fashion, using object-oriented language features like inheritance & parameterized types 125 This pattern can incur liabilities: •Loss of functionality • Whenever an abstraction is layered on top of an existing abstraction it is possible to lose functionality •Performance degradation • This pattern can degrade performance if several forwarding function calls are made per method •Programming language & compiler limitations • It may be hard to define wrapper facades for certain languages due to a lack of language support or limitations with compilers Decoupling Event Demuxing, Connection Management, & Protocol Processing (1/2) Context •Web servers can be accessed simultaneously by multiple clients •They must demux & process multiple types of indication events arriving from clients concurrently •A common way to demux events in a server is to use select() Event Dispatcher select() Client Client HTTP GET Web Server request Socket HTTP GET Handles request Client Sockets Connect request select (width, &read_handles, 0, 0, 0); Problem if (FD_ISSET (acceptor, &ready_handles)) { int h; •Developers often couple do { event-demuxing & h = accept (acceptor, 0, 0); char buf[BUFSIZ]; connection code with for (ssize_t i; (i = read (h, buf, BUFSIZ)) > 0; ) write (1, buf, i); protocol-handling code } while (h != -1); •This code cannot then be •Thus, changes to event-demuxing & connection reused directly by other code affects server protocol code directly & may protocols or by other yield subtle bugs, e.g., when porting to use TLI or middleware & applications WaitForMultipleObjects() 126 Decoupling Event Demuxing, Connection Management, & Protocol Processing (2/2) Solution Apply the Reactor architectural pattern (P2) & the Acceptor-Connector design pattern (P2) to separate the generic event-demultiplexing & connection-management code from the web server’s protocol code Reactor * handle_events() register_handler() remove_handler() dispatches * Handle * owns Event Handler handle_event () get_handle() notifies handle set <<uses>> Synchronous Event Demuxer Connector Acceptor select () Service Handler 127 The Reactor Pattern The Reactor architectural pattern allows event-driven applications to demultiplex & dispatch service requests that are delivered to an application from one or more clients Reactor * handle_events() register_handler() remove_handler() dispatches * owns Handle * Event Handler handle_event () get_handle() notifies handle set <<uses>> Synchronous Event Demuxer select () 128 Concrete Event Handler A handle_event () get_handle() Concrete Event Handler B handle_event () get_handle() Reactor Pattern Dynamics : Main Program 1. Initialize phase Con. Event Handler : Concrete Event Handler Events : Reactor : Synchronous Event Demultiplexer register_handler() get_handle() Handle 2. Event handling phase handle_events() Handles select() handle_event() service() Handles Observations •Note inversion of control •Also note how long-running event handlers can degrade the QoS since callbacks steal the reactor’s thread! 129 event The Acceptor-Connector Pattern The Acceptor-Connector design pattern decouples the connection & initialization of cooperating peer services in a networked system from the processing performed by the peer services after being connected & initialized notifies notifies Dispatcher uses uses * Transport Handle owns select() handle_events() register_handler() remove_handler() uses Transport Handle owns notifies uses * * Transport Handle <<creates>> owns * Service Handler * Connector Connector() connect() complete() handle_event () * Acceptor peer_stream_ peer_acceptor_ open() handle_event () set_handle() Acceptor() Accept() handle_event () <<activate>> <<activate>> * 130 Concrete Connector Concrete Service Handler A Concrete Service Handler B Concrete Acceptor Acceptor Dynamics : Application 1.Passive-mode endpoint initialize phase : Acceptor : Dispatcher open() Acceptor Handle1 ACCEPT_ register_handler() EVENT handle_events() accept() 2.Service handler initialize phase : Handle2 : Service Handler Handle2 Handle2 3.Service processing phase • The Acceptor ensures that passivemode transport endpoints aren’t used to read/write data accidentally •And vice versa for data transport endpoints… 131 open() Service Events Handler register_handler() handle_event() service() • There is typically one Acceptor factory per-service/per-port •Additional demuxing can be done at higher layers, a la CORBA Synchronous Connector Dynamics Motivation for Synchrony • If connection latency is negligible •e.g., connecting with a server on the same host via a ‘loopback’ device : Application 1.Sync connection initiation phase 2.Service handler initialize phase 3.Service processing phase 132 Service Handler • If multiple threads of control are available & it is efficient to use a thread-per-connection to connect each service handler synchronously : Connector Addr • If the services must be initialized in a fixed order & the client can’t perform useful work until all connections are established : Service Handler : Dispatcher get_handle() connect() Handle register_handler() open() Service Handler Handle Events handle_events() handle_event() service() Asynchronous Connector Dynamics Motivation for Asynchrony • If client is establishing connections over high latency links • If client is a single-threaded application : Application Service Handler 1.Async connection initiation phase 2.Service handler initialize phase 3.Service processing phase 133 : Connector Addr • If client is initializing many peers that can be connected in an arbitrary order : Service Handler : Dispatcher get_handle() connect() Handle Handle register_handler() CONNECT Connector EVENT handle_events() complete() open() register_handler() Service Handler Handle handle_event() service() Events Applying the Reactor & Acceptor-Connector Patterns in JAWS The Reactor architectural pattern decouples: 1.JAWS generic synchronous event demultiplexing & dispatching logic from 2.The HTTP protocol processing it performs in response to events ACE_Reactor handle_events() register_handler() remove_handler() <<uses>> * ACE_Handle owns notifies * handle set Synchronous Event Demuxer select () * dispatches HTTP Acceptor handle_event () get_handle() ACE_Event_Handler handle_event () get_handle() HTTP Handler handle_event () get_handle() The Acceptor-Connector design pattern can use a Reactor as its Dispatcher in order to help decouple: 1.The connection & initialization of peer client & server HTTP services from 2.The processing activities performed by these peer services after they are connected & initialized 134 Reactive Connection Management & Data Transfer in JAWS 135 Pros & Cons of the Reactor Pattern This pattern offers four benefits: •Separation of concerns • This pattern decouples applicationindependent demuxing & dispatching mechanisms from application-specific hook method functionality This pattern can incur liabilities: •Restricted applicability • This pattern can be applied efficiently only if the OS supports synchronous event demuxing on handle sets •Modularity, reusability, & configurability •Non-pre-emptive • This pattern separates event-driven application functionality into several components, which enables the configuration of event handler components that are loosely integrated via a reactor •Portability • By decoupling the reactor’s interface from the lower-level OS synchronous event demuxing functions used in its implementation, the Reactor pattern improves portability •Coarse-grained concurrency control • This pattern serializes the invocation of event handlers at the level of event demuxing & dispatching within an application process or thread 136 • In a single-threaded application, concrete event handlers that borrow the thread of their reactor can run to completion & prevent the reactor from dispatching other event handlers •Complexity of debugging & testing • It is hard to debug applications structured using this pattern due to its inverted flow of control, which oscillates between the framework infrastructure & the method callbacks on application-specific event handlers Pros & Cons of Acceptor-Connector Pattern This pattern provides three benefits: •Reusability, portability, & extensibility • This pattern decouples mechanisms for connecting & initializing service handlers from the service processing performed after service handlers are connected & initialized •Robustness This pattern also has liabilities: •Additional indirection • The Acceptor-Connector pattern can incur additional indirection compared to using the underlying network programming interfaces directly • This pattern strongly decouples the service •Additional complexity handler from the acceptor, which ensures that a • The Acceptor-Connector pattern passive-mode transport endpoint can’t be used may add unnecessary complexity to read or write data accidentally for simple client applications that •Efficiency connect with only one server & • This pattern can establish connections actively perform one service using a with many hosts asynchronously & efficiently single network programming over long-latency wide area networks interface • Asynchrony is important in this situation because a large networked system may have hundreds or thousands of host that must be connected 137 Overview of Concurrency & Threading •Thus far, our web server has been entirely reactive, which can be a bottleneck for scalable systems •Multi-threading is essential to develop scalable & robust networked applications, particularly servers •The next group of slides present a domain analysis of concurrency design dimensions that address the policies & mechanisms governing the proper use of processes, threads, & synchronizers 138 •We outline the following design dimensions in this discussion: •Iterative versus concurrent versus reactive servers •Processes versus threads •Process/thread spawning strategies •User versus kernel versus hybrid threading models •Time-shared versus real-time scheduling classes Iterative vs. Concurrent Servers •Iterative/reactive servers handle each client request in its entirety before servicing subsequent requests •Best suited for short-duration or 139 infrequent services •Concurrent servers handle multiple requests from clients simultaneously •Best suited for I/O-bound services or long-duration services •Also good for busy servers Multiprocessing vs. Multithreading •A process provides the context for executing program instructions •Each process manages certain resources (such as virtual memory, I/O handles, & signal handlers) & is protected from other OS processes via an MMU •IPC between processes can be complicated & inefficient 140 •A thread is a sequence of instructions in the context of a process •Each thread manages certain resources (such as runtime stack, registers, signal masks, priorities, & thread-specific data) •Threads are not protected from other threads •IPC between threads can be more efficient than IPC between processes Thread-per-Request On-demand Spawning Strategy •On-demand spawning creates a new process or thread in response to the arrival of client connection and/or data requests •Typically used to implement the thread-per-request & thread-perconnection models •The primary benefit of on-demand spawning strategies is their reduced consumption of resources •The drawbacks, however, are that these strategies can degrade performance in heavily loaded servers & determinism in real-time systems due to costs of spawning processes/threads & starting services 141 Thread Pool Eager Spawning Strategies •This strategy prespawns one or more OS processes or threads at server creation time •These``warm-started'' execution resources form a pool that improves response time by incurring service startup overhead before requests are serviced •Two general types of eager spawning strategies are shown below: •These strategies based on Half-Sync/Half-Async & Leader/Followers patterns 142 The N:1 & 1:1 Threading Models •OS scheduling ensures applications use host CPU resources suitably •Modern OS platforms provide various models for scheduling threads •A key difference between the models is the contention scope in which threads compete for system resources, particularly CPU time •The two different contention scopes are shown below: 143 • Process contention scope (aka “user threading”) where threads in the same process compete with each other (but not directly with threads in other processes) • System contention scope (aka “kernel threading”) where threads compete directly with other system-scope threads, regardless of what process they’re in The N:M Threading Model •Some operating systems (such as Solaris) offer a combination of the N:1 & 1:1 models, referred to as the ``N:M'‘ hybridthreading model •When an application spawns a thread, it can indicate in which contention scope the thread should operate •The OS threading library creates a user-space thread, but only creates a kernel thread if needed or if the application explicitly requests the system contention scope 144 •When the OS kernel blocks an LWP, all user threads scheduled onto it by the threads library also block •However, threads scheduled onto other LWPs in the process can continue to make progress Scaling Up Performance via Threading Context • HTTP runs over TCP, which uses flow control to ensure that senders do not produce data more rapidly than slow receivers or congested networks can buffer & process • Since achieving efficient end-to-end quality of service (QoS) is important to handle heavy Web traffic loads, a Web server must scale up efficiently as its number of clients increases Problem • Similarly, to improve QoS for all its connected clients, an entire Web server process must not block while waiting for connection flow control to abate so it can finish sending a file to a client • Processing all HTTP GET requests reactively within a single-threaded process does not scale up, because each server CPU time-slice spends much of its time blocked waiting for I/O operations to complete 145 The Half-Sync/Half-Async Pattern Solution •Apply the Half-Sync/HalfAsync architectural pattern (P2) to scale up server performance by processing different HTTP requests concurrently in multiple threads The Half-Sync/Half-Async architectural pattern decouples async & sync service processing in concurrent systems, to simplify programming without unduly reducing performance 146 Sync Service Layer Sync Service 1 Sync Service 2 <<read/write>> <<read/write>> Queueing Layer Async Service Layer Sync Service 3 Queue <<dequeue/enqueue>> <<read/write>> <<interrupt>> Async Service External Event Source This solution yields two benefits: 1. Threads can be mapped to separate CPUs to scale up server performance via multi-processing 2. Each thread blocks independently, which prevents a flow-controlled connection from degrading the QoS that other clients receive Half-Sync/Half-Async Pattern Dynamics : External Event Source : Async Service : Queue : Sync Service notification read() work() message message enqueue() notification read() work() message • This pattern defines two service processing layers—one async & one sync—along with a queueing layer that allows services to exchange messages between the two layers 147 • The pattern allows sync services, such as HTTP protocol processing, to run concurrently, relative both to each other & to async services, such as event demultiplexing Applying Half-Sync/Half-Async Pattern in JAWS Synchronous Service Layer Worker Thread 1 Worker Thread 2 Worker Thread 3 <<get>> Queueing Layer <<get>> <<get>> Request Queue <<put>> Asynchronous Service Layer HTTP Handlers, HTTP Acceptor <<ready to read>> ACE_Reactor • JAWS uses the HalfSync/Half-Async pattern to process HTTP GET requests synchronously from multiple clients, but concurrently in separate threads 148 • The worker thread that removes the request synchronously performs HTTP protocol processing & then transfers the file back to the client Socket Event Sources • If flow control occurs on its client connection this thread can block without degrading the QoS experienced by clients serviced by other worker threads in the pool Pros & Cons of Half-Sync/Half-Async Pattern This pattern has three benefits: •Simplification & performance • The programming of higher-level synchronous processing services are simplified without degrading the performance of lower-level system services •Separation of concerns • Synchronization policies in each layer are decoupled so that each layer need not use the same concurrency control strategies •Centralization of inter-layer communication • Inter-layer communication is centralized at a single access point, because all interaction is mediated by the queueing layer 149 This pattern also incurs liabilities: •A boundary-crossing penalty may be incurred • This overhead arises from context switching, synchronization, & data copying overhead when data is transferred between the sync & async service layers via the queueing layer •Higher-level application services may not benefit from the efficiency of async I/O • Depending on the design of operating system or application framework interfaces, it may not be possible for higher-level services to use low-level async I/O devices effectively •Complexity of debugging & testing • Applications written with this pattern can be hard to debug due its concurrent execution Implementing a Synchronized Request Queue Context • The Half-Sync/Half-Async pattern contains a queue • The JAWS Reactor thread is a ‘producer’ that inserts HTTP GET requests into the queue • Worker pool threads are ‘consumers’ that remove & process queued requests Worker Thread 1 Worker Thread 2 Worker Thread 3 <<get>> <<get>> Request Queue <<get>> <<put>> HTTP Handlers, HTTP Acceptor ACE_Reactor Problem • A naive implementation of a request queue will incur race conditions or ‘busy waiting’ when multiple threads insert & remove requests • e.g., multiple concurrent producer & consumer threads can corrupt the queue’s internal state if it is not synchronized properly • Similarly, these threads will ‘busy wait’ when the queue is empty or full, which wastes CPU cycles unnecessarily 150 The Monitor Object Pattern Solution • Apply the Monitor Object design pattern (P2) to synchronize the queue efficiently & conveniently • This pattern synchronizes concurrent method execution to ensure that only one method at a time runs within an object • It also allows an object’s methods to cooperatively schedule their execution sequences Monitor Object Client 2..* sync_method1() sync_methodN() uses uses * Monitor Condition wait() notify() notify_all() Monitor Lock acquire() release() • It’s instructive to compare Monitor Object pattern solutions with Active Object pattern solutions •The key tradeoff is efficiency vs. flexibility 151 Monitor Object Pattern Dynamics : Client Thread1 : Client Thread2 : Monitor Object sync_method1() 1. Synchronized method invocation & serialization 2. Synchronized method thread suspension 3. Monitor condition notification 4. Synchronized method thread resumption : Monitor Lock acquire() dowork() wait() the OS thread scheduler automatically suspends the client thread sync_method2() the OS thread scheduler automatically resumes the client thread & the synchronized method acquire() the OS thread scheduler atomically releases the monitor lock dowork() notify() release() dowork() release() 152 : Monitor Condition the OS thread scheduler atomically reacquires the monitor lock Applying Monitor Object Pattern in JAWS The JAWS synchronized request queue implements the queue’s not-empty & not-full monitor conditions via a pair of ACE wrapper facades for POSIX-style condition variables HTTP Handler Request Queue <<put>> <<get>> put() get() Worker Thread uses uses 2 ACE_Thread_Condition ACE_Thread_Mutex wait() signal() broadcast() acquire() release() •When a worker thread attempts to dequeue an HTTP GET request from an empty queue, the request queue’s get() method atomically releases the monitor lock & the worker thread suspends itself on the not-empty monitor condition •The thread remains suspended until the queue is no longer empty, which happens when an HTTP_Handler running in the Reactor thread inserts a request into the queue 153 Pros & Cons of Monitor Object Pattern This pattern provides two benefits: This pattern can also incur liabilities: •The use of a single monitor lock can •Simplification of concurrency limit scalability due to increased control • The Monitor Object pattern presents contention when multiple threads serialize on a monitor object a concise programming model for sharing an object among •Complicated extensibility semantics cooperating threads where object synchronization corresponds to method invocations •Simplification of scheduling method execution • Synchronized methods use their monitor conditions to determine the circumstances under which they should suspend or resume their execution & that of collaborating monitor objects • These result from the coupling between a monitor object’s functionality & its synchronization mechanisms •It is also hard to inherit from a monitor object transparently, due to the inheritance anomaly problem •Nested monitor lockout • This problem is similar to the preceding liability & can occur when a monitor object is nested within another monitor object www.cs.wustl.edu/~schmidt/C++2java.html 154 Minimizing Server Threading Overhead Context •Socket implementations in certain multi-threaded operating systems provide a concurrent accept() optimization to accept client connection requests & improve the performance of Web servers that implement the HTTP 1.0 protocol as follows: accept() •The OS allows a pool of threads in a Web server to call accept() on the same passive-mode socket handle •When a connection request arrives, the operating system’s transport layer creates a new accept() accept() connected transport endpoint, encapsulates this new endpoint with a data-mode socket handle & passes the handle as the return value from accept() accept() accept() •The OS then schedules one of the threads in the pool to receive this data-mode handle, passive-mode which it uses to communicate with its socket handle 155 connected client Drawbacks with Half-Sync/Half-Async Problem •Although Half-Sync/Half-Async threading model is more scalable than the purely reactive model, it is not necessarily the most efficient design •e.g., passing a request between the Reactor thread & a worker thread incurs: •Dynamic memory (de)allocation, •Synchronization operations, •A context switch, & •CPU cache updates Worker Thread 1 •This overhead makes JAWS’ latency unnecessarily high, particularly on operating systems that support the concurrent accept() optimization 156 Worker Thread 2 Worker Thread 3 <<get>> <<get>> Request Queue <<get>> <<put>> HTTP Handlers, HTTP Acceptor ACE_Reactor Solution •Apply the Leader/Followers architectural pattern (P2) to minimize server threading overhead The Leader/Followers Pattern demultiplexes The Leader/Followers architectural pattern (P2) provides an efficient concurrency model where multiple threads take turns sharing event sources to detect, demux, dispatch, & process service requests that occur on the event sources Thread Pool synchronizer join() promote_new_leader() * Event Handler Handle uses * This pattern eliminates the need for—& the overhead of—a separate Reactor thread & synchronized request queue used in the Half-Sync/Half-Async pattern Handle Set handle_events() deactivate_handle() reactivate_handle() select() handle_event () get_handle() Iterative Handles Concrete Event Handler A Handles Concurrent Handles Handle Sets Concurrent Handle Sets Iterative Handle Sets 157 handle_event () get_handle() UDP Sockets + TCP Sockets + WaitForMultipleObjects() WaitForMultpleObjects() UDP Sockets + select()/poll() TCP Sockets + select()/poll() Concrete Event Handler B handle_event () get_handle() Leader/Followers Pattern Dynamics Thread 1 1.Leader thread demuxing Thread 2 : Thread Pool : Handle Set : Concrete Event Handler join() handle_events() join() event handle_event() 2.Follower thread promotion 3.Event handler demuxing & event processing 4.Rejoining the thread pool 158 thread 2 sleeps until it becomes the leader thread 2 waits for a new event, thread 1 processes current event join() thread 1 sleeps until it becomes the leader deactivate_ handle() promote_ new_leader() handle_ events() reactivate_ handle() event handle_event() deactivate_ handle() Applying Leader/Followers Pattern in JAWS Two options: Although Leader/Followers thread 1.If platform supports accept() pool design is highly efficient the optimization then the Leader/Followers Half-Sync/Half-Async design may be pattern can be implemented by the OS more appropriate for certain types of 2.Otherwise, this pattern can be servers, e.g.: implemented as a reusable framework • The Half-Sync/Half-Async design can reorder & demultiplexes Thread Pool prioritize client requests synchronizer more flexibly, because it has join() a synchronized request promote_new_leader() queue implemented using * ACE_Event_Handler the Monitor Object pattern uses * ACE_Handle handle_event () • It may be more scalable, get_handle() ACE_TP_Reactor because it queues requests handle_events() deacitivate_handle() in Web server virtual reactivate_handle() memory, rather than the OS select() HTTP HTTP Acceptor Handler kernel handle_event () get_handle() 159 handle_event () get_handle() Pros & Cons of Leader/Followers Pattern This pattern provides two benefits: •Performance enhancements • This can improve performance as follows: • It enhances CPU cache affinity & eliminates the need for dynamic memory allocation & data buffer sharing between threads • It minimizes locking overhead by not exchanging data between threads, thereby reducing thread synchronization • It can minimize priority inversion because no extra queueing is introduced in the server • It doesn’t require a context switch to handle each event, reducing dispatching latency •Programming simplicity • The Leader/Follower pattern simplifies the programming of concurrency models where multiple threads can receive requests, process responses, & demultiplex connections using a shared handle set 160 This pattern also incur liabilities: •Implementation complexity • The advanced variants of the Leader/ Followers pattern are hard to implement •Lack of flexibility • In the Leader/ Followers model it is hard to discard or reorder events because there is no explicit queue •Network I/O bottlenecks • The Leader/Followers pattern serializes processing by allowing only a single thread at a time to wait on the handle set, which could become a bottleneck because only one thread at a time can demultiplex I/O events Using Asynchronous I/O Effectively Context GetQueued CompletionStatus() • Synchronous multi-threading may not be the most scalable way to implement a Web server GetQueued on OS platforms that support async I/O more CompletionStatus() GetQueued efficiently than synchronous multi-threading CompletionStatus() • For example, highly-efficient Web servers can be implemented on Windows NT by invoking async Win32 operations that perform the following activities: I/O Completion • Processing indication events, such as TCP Port CONNECT & HTTP GET requests, via AcceptEx() & ReadFile(), respectively • Transmitting requested files to clients AcceptEx() asynchronously via WriteFile() or AcceptEx() TransmitFile() AcceptEx() •When these async operations complete, WinNT 1.Delivers the associated completion events passive-mode containing their results to the Web server socket handle 2.Processes these events & performs the appropriate actions before returning to its event loop 161 The Proactor Pattern Problem • Developing software that achieves the potential efficiency & scalability of async I/O is hard due to the separation in time & space of async operation invocations & their subsequent completion events <<uses>> Initiator <<uses>> Solution • Apply the Proactor architectural pattern (P2) to make efficient use of async I/O This pattern allows event-driven applications to efficiently demultiplex & dispatch service requests triggered by the completion of async operations, thereby achieving the performance benefits of concurrency <<uses>> without incurring <<invokes>> its many liabilities is associated with Asynchronous Operation Processor execute_async_op() <<enqueues>> Asynchronous Operation <<executes>> get_completion_event() 162 <<dequeues>> Completion Handler * async_op() Asynchronous Event Demuxer Completion Event Queue Handle handle_event() <<demultiplexes & dispatches>> Proactor handle_events() Concrete Completion Handler Proactor Pattern Dynamics : Initiator 1. Initiate operation 2. Process operation 3. Run event loop 4. Generate & queue completion event 5. Dequeue completion event & perform completion processing 163 : Asynchronous Operation Processor Completion Handler Completion Ev. Queue exec_async_ operation () : Asynchronous Operation : Completion : Proactor Event Queue Completion Handler async_operation() handle_events() event Result Result event Result Result handle_ event() Note similarities & differences with the Reactor pattern, e.g.: •Both process events via callbacks •However, it’s generally easier to multi-thread a proactor service() Applying the Proactor Pattern in JAWS The Proactor pattern structures the JAWS concurrent server to receive & process requests from multiple clients asynchronously <<uses>> JAWS HTTP components are split into two parts: 1. Operations that execute asynchronously • e.g., to accept connections & receive client HTTP GET requests 2. The corresponding completion handlers that process the async operation results • e.g., to transmit a file back to a client after an async connection operation completes Web Server <<uses>> <<invokes>> <<uses>> Windows NT Operating System execute_async_op() <<enqueues>> Asynchronous Operation AcceptEx() ReadFile() WriteFile() <<executes>> Asynchronous Event Demuxer I/O Completion Port GetQueuedCompletionStatus() 164 <<dequeues>> is associated with ACE_Handle ACE_Handler * handle_accept() handle_write_stream() <<demultiplexes & dispatches>> ACE_Proactor handle_events() HTTP Acceptor HTTP Handler Proactive Connection Management & Data Transfer in JAWS 165 Pros & Cons of Proactor Pattern This pattern offers five benefits: •Separation of concerns • Decouples application-independent async mechanisms from application-specific functionality •Portability This pattern incurs some liabilities: •Restricted applicability • This pattern can be applied most efficiently if the OS supports asynchronous operations natively • Improves application portability by allowing its •Complexity of programming, interfaces to be reused independently of the OS debugging, & testing event demuxing calls • It is hard to program applications •Decoupling of threading from & higher-level system services using asynchrony mechanisms, concurrency due to the separation in time & • The async operation processor executes longspace between operation duration operations on behalf of initiators so invocation & completion applications can spawn fewer threads •Performance • Avoids context switching costs by activating only those logical threads of control that have events to process •Simplification of application synchronization 166 • If concrete completion handlers spawn no threads, application logic can be written with little or no concern for synchronization issues •Scheduling, controlling, & canceling asynchronously running operations • Initiators may be unable to control the scheduling order in which asynchronous operations are executed by an asynchronous operation processor Efficiently Demuxing Asynchronous Operations & Completions Context •In a proactive Web server async I/O operations will yield I/O completion event responses that must be processed efficiently Problem •As little overhead as possible should be incurred to determine how the completion handler will demux & process completion events after async operations finish executing •When a response arrives, the application should spend as little time as possible demultiplexing the completion event to the handler that will process the async operation’s response Solution •Apply the Asynchronous Completion Token design pattern (P2) to demux & process the responses of asynchronous operations efficiently •Together with each async operation that a client initiator invokes on a service, transmit information that identifies how the initiator should process the service’s response 167 •Return this information to the initiator when the operation finishes, so that it can be used to demux the response efficiently, allowing the initiator to process it accordingly Asynchronous Completion Token Pattern Structure & Participants Dynamic Interactions handle_event() 168 Applying the Asynchronous Completion Token Pattern in JAWS Detailed Interactions (HTTP_Acceptor is both initiator & completion handler) 169 Pros & Cons of Asynchronous Completion Token Pattern This pattern has four benefits: •Simplified initiator data structures • Initiators need not maintain complex data structures to associate service responses with completion handlers •Efficient state acquisition This pattern has some liabilities: •Memory leaks • Memory leaks can result if initiators use ACTs as pointers to dynamically allocated memory & services fail to return the ACTs, for example if the service crashes • ACTs are time efficient because they need not require complex parsing of •Authentication data returned with the service response • When an ACT is returned to an initiator •Space efficiency on completion of an asynchronous • ACTs can consume minimal data space event, the initiator may need to yet can still provide applications with authenticate the ACT before using it sufficient information to associate large amounts of state to process asynchronous operation completion actions •Flexibility • User-defined ACTs are not forced to inherit from an interface to use the service’s ACTs 170 •Application re-mapping • If ACTs are used as direct pointers to memory, errors can occur if part of the application is re-mapped in virtual memory Enhancing Server (Re)Configurability (1/2) Context Problem The implementation of certain web server components depends on a variety of factors: Prematurely committing to a particular web server component configuration is inflexible & inefficient: •Certain factors are static, such as the number of available CPUs & operating system support for asynchronous I/O • No single web server configuration is optimal for all use cases •Other factors are dynamic, such as system workload • Certain design decisions cannot be made efficiently until run-time Cache Mgmt Conn Mgmt Demuxing 171 HTTP Parsing Threading I/O File System Enhancing Server (Re)Configurability (2/2) Solution •Apply the Component Configurator design pattern (P2) to enhance server configurability •This pattern allows an application to link & unlink its component implementations at run-time •Thus, new & enhanced services can be added without having to modify, recompile, statically relink, or shut down & restart a running application 172 Component Component * init() components fini() Repository suspend() <<contains>> resume() info() Component Configurator Concrete Concrete Component A Component B Component Configurator Pattern Dynamics : Component Configurator : Concrete Component A : Concrete Component B : Component Repository init() Concrete Comp. A 1.Component dynamic linking & initialization insert() init() Concrete Comp. B insert() run_component() 2.Component processing run_component() fini() 3.Component termination & dynamic unlinking 173 Concrete Comp. A remove() fini() Concrete Comp. B remove() Applying the Component Configurator Pattern to Content Servers Image servers can use the Component Configurator pattern to dynamically optimize, control, & reconfigure the behavior of its components at installation-time or during run-time Component Component * init() components Repository fini() suspend() <<contains>> resume() info() Component Configurator •For example, a content server can apply the Component Configurator pattern to configure various Cached Virtual Filesystem strategies •e.g., least-recently used (LRU) or least-frequently used (LFU) Concrete components can be packaged into a suitable unit of configuration, such as a dynamically linked library (DLL) 174 LRU File Cache LFU File Cache Only the components that are currently in use need to be configured into a content server Reconfiguring JAWS Image servers can also be reconfigured dynamically to support new components & new component implementations Web Server Reconfiguration State Chart IDLE TERMINATE fini() TERMINATE fini() LRU File Cache # Configure a image server. dynamic File_Cache Component * web_server.dll:make_File_Cache() "-t LRU" INITIAL CONFIGURATION 175 CONFIGURE init() RECONFIGURE init() RUNNING RESUME resume() SUSPENDED Web Server SUSPEND suspend() EXECUTE run_component() LFU File Cache # Reconfigure a image server. Remove File_Cache dynamic File_Cache Component * web_server.dll:make_File_Cache() "-t LFU" AFTER RECONFIGURATION Pros & Cons of Component Configurator Pattern This pattern offers four benefits: •Uniformity • By imposing a uniform configuration & control interface to manage components •Centralized administration This pattern also incurs liabilities: •Lack of determinism & ordering dependencies • This pattern makes it hard to determine or analyze the behavior of an application until its components are configured at run-time • By grouping one or more components into a single administrative unit that simplifies development by centralizing common •Reduced security or reliability component initialization & termination • An application that uses the activities Component Configurator pattern may be less secure or reliable than an •Modularity, testability, & reusability equivalent statically-configured • Application modularity & reusability is application improved by decoupling component implementations from the manner in which •Increased run-time overhead & the components are configured into infrastructure complexity processes • By adding levels of abstraction & •Configuration dynamism & control indirection when executing • By enabling a component to be components dynamically reconfigured without •Overly narrow common interfaces modifying, recompiling, statically relinking • The initialization or termination of a existing code & without restarting the component may be too complicated or component or other active components too tightly coupled with its context to 176 with which it is collocated be performed in a uniform manner Transparently Parameterizing Synchronization into Components Context Problem •It should be possible to customize JAWS •The various concurrency patterns described earlier impact component synchronization mechanisms according to the requirements of particular component synchronization application use cases & configurations strategies in various ways •Hard-coding synchronization strategies •e.g.,ranging from no locks to into component implementations is readers/writer locks •In general, components must run inflexible •Maintaining multiple versions of efficiently in a variety of components manually is not scalable concurrency models Solution •Apply the Strategized Locking design pattern (P2) to parameterize JAWS component synchronization strategies by making them ‘pluggable’ types •Each type objectifies a particular synchronization strategy, such as a mutex, readers/writer lock, semaphore, or ‘null’ lock 177 •Instances of these pluggable types can be defined as objects contained within a component, which then uses these objects to synchronize its method implementations efficiently Applying Polymorphic Strategized Locking in JAWS Polymorphic Strategized Locking class Lock { public: // Acquire & release the lock. virtual void acquire () = 0; virtual void acquire_read () = 0; virtual void release () = 0; // ... }; class Thread_Mutex : public Lock { // ... }; class File_Cache { public: // Constructor. File_Cache (Lock &l): lock_ (&l) { } // A method. const void *lookup (const string &path) const { lock_->acquire_read (); // Implement the <lookup> method. lock_->release (); } // ... private: // The polymorphic strategized locking object. mutable Lock *lock_; // Other data members & methods go here... }; 178 Applying Parameterized Strategized Locking in JAWS Parameterized Strategized Locking • Single-threaded file cache. typedef File_Cache<ACE_Null_Mutex> Content_Cache; • Multi-threaded file cache using a thread mutex. typedef File_Cache<ACE_Thread_Mutex> Content_Cache; • Multi-threaded file cache using a readers/writer lock. typedef File_Cache<ACE_RW_Mutex> Content_Cache; template <class LOCK> class File_Cache { public: // A method. const void *lookup (const string &path) const { lock_.acquire_read (); // Implement the <lookup> method. lock_.release (); } Note that the various locks need not inherit from a common base class or use virtual methods! // ... private: // The polymorphic strategized locking object. mutable LOCK lock_; // Other data members & methods go here... }; 179 Pros & Cons of the Strategized Locking Pattern This pattern provides three benefits: •Enhanced flexibility & customization • It is straightforward to configure & customize a component for certain concurrency models because the synchronization aspects of components are strategized •Decreased maintenance effort for components • It is straightforward to add enhancements & bug fixes to a component because there is only one implementation, rather than a separate implementation for each concurrency model •Improved reuse • Components implemented using this pattern are more reusable, because their locking strategies can be configured orthogonally to their behavior 180 This pattern also incurs liabilities: •Obtrusive locking • If templates are used to parameterize locking aspects this will expose the locking strategies to application code •Over-engineering • Externalizing a locking mechanism by placing it in a component’s interface may actually provide too much flexibility in certain situations • e.g., inexperienced developers may try to parameterize a component with the wrong type of lock, resulting in improper compile- or run-time behavior Ensuring Locks are Released Properly Context Problem •Concurrent applications, such as JAWS, contain shared resources that are manipulated by multiple threads concurrently •Code that shouldn’t execute concurrently must be protected by a lock that’s acquired/released when control enters/leaves a critical section •If programmers acquire & release locks explicitly, it’s hard to ensure locks are released in all code paths •e.g., control can leave scope in C++ due to return, break, continue, goto, or unhandled exception Solution •In C++, apply the Scoped Locking idiom (P2) to define a guard class whose constructor automatically acquires a lock when control enters a scope & whose destructor automatically releases the lock when control leaves the scope 181 // A method. const void *lookup (const string &path) const { lock_.acquire_read (); // The <lookup> method // implementation may return // prematurely… lock_.release (); } Applying the Scoped Locking Idiom in JAWS template <class LOCK> class ACE_Read_Guard { Generic ACE_Read_Guard Wrapper Facade public: // Store a pointer to the lock & acquire the lock. ACE_Read_Guard (LOCK &lock) : lock_ (&lock) { lock_->acquire_read (); } // Release the lock when the guard goes out of scope, ~ACE_Read_Guard () { lock_->release (); } ... private: // Pointer to the lock we’re managing. mutable LOCK *lock_; }; template <class LOCK> class File_Cache { Applying the ACE_Guard public: Instances of the guard // A method. class can be allocated const void *lookup (const string &path) const { on the run-time stack to // Use Scoped Locking idiom to acquire acquire & release locks // & release the <lock_> automatically. ACE_Read_Guard<LOCK> guard (*lock); in method or block // Implement the <lookup> method. scopes that define // lock_ released automatically… critical sections } 182 Pros & Cons of the Scoped Locking Idiom This idiom has one benefit: •Increased robustness This idiom also has liabilities: •Potential for deadlock when used recursively • This idiom increases the • If a method that uses the Scoped Locking idiom robustness of concurrent calls itself recursively, ‘self-deadlock’ will occur if applications by eliminating the lock is not a ‘recursive’ mutex common programming errors •Limitations with language-specific related to synchronization & semantics multi-threading • The Scoped Locking idiom is based on a C++ • By applying the Scoped language feature & therefore will not be integrated Locking idiom, locks are with operating system-specific system calls acquired & released • Thus, locks may not be released automatically automatically when control when threads or processes abort or exit inside a enters & leaves critical guarded critical section sections defined by C++ • Likewise, they will not be released properly if method & block scopes the standard C longjmp() function is called because this function does not call the destructors of C++ objects as the run-time stack unwinds 183 Minimizing Unnecessary Locking (1/2) Context •Components in multithreaded applications that contain intra-component method calls •Components that have applied the Strategized Locking pattern Problem •Thread-safe components should be designed to avoid unnecessary locking •Thread-safe components should be designed to avoid “self-deadlock” 184 template <class LOCK> class File_Cache { public: const void *lookup (const string &path) const { ACE_Read_Guard<LOCK> guard (lock_); const void *file_pointer = check_cache (path); if (file_pointer == 0) { insert (path); file_pointer = check_cache (path); } return file_pointer; } void insert (const string &path) { ACE_Read_Guard<LOCK> guard (lock_); // ... insert <path> into cache... } private: mutable LOCK lock_; const void *check_cache (const string &) const; }; Since File_Cache is a template we don’t know the type of lock used to parameterize it. Minimizing Unnecessary Locking (2/2) Solution •Apply the Thread-safe Interface design pattern (P2) to minimize locking overhead & ensure that intra-component method calls do not incur ‘selfdeadlock’ by trying to reacquire a lock that is held by the component already This pattern structures all components that process intra-component method invocations according two design conventions: 185 •Interface methods check •All interface methods, such as C++ public methods, should only acquire/release component lock(s), thereby performing synchronization checks at the ‘border’ of the component. •Call implementation methods to do work •Don’t call interface •Implementation methods trust •Implementation methods, such as C++ private & protected methods, should only perform work when called by interface methods. •Only call other implementation methods Applying the Thread-safe Interface Pattern in JAWS template <class LOCK> class File_Cache { public: // Return a pointer to the memory-mapped file associated with // <path> name, adding it to the cache if it doesn’t exist. const void *lookup (const string &path) const { // Use Scoped Locking to acquire/release lock automatically. ACE_Read_Guard<LOCK> guard (lock_); return lookup_i (path); Note fewer constraints } on the type of LOCK… private: mutable LOCK lock_; // The strategized locking object. // This implementation method doesn’t acquire or release // <lock_> & does its work without calling interface methods. const void *lookup_i (const string &path) const { const void *file_pointer = check_cache_i (path); if (file_pointer == 0) { // If <path> isn’t in cache, insert it & look it up again. insert_i (path); file_pointer = check_cache_i (path); // The calls to implementation methods <insert_i> & // <check_cache_i> assume that the lock is held & do work. } return file_pointer; 186 Pros & Cons of the Thread-safe Interface Pattern This pattern has three benefits: This pattern has some liabilities: •Additional indirection & extra methods •Increased robustness • This pattern ensures that selfdeadlock does not occur due to intra-component method calls •Enhanced performance • This pattern ensures that locks are not acquired or released unnecessarily •Simplification of software • Separating the locking & functionality concerns can help to simplify both aspects • Each interface method requires at least one implementation method, which increases the footprint of the component & may also add an extra level of method-call indirection for each invocation •Potential for misuse • OO languages, such as C++ & Java, support class-level rather than object-level access control • As a result, an object can bypass the public interface to call a private method on another object of the same class, thus bypassing that object’s lock •Potential overhead • This pattern prevents multiple components from sharing the same lock & prevents locking at a finer granularity than the component, which can increase lock contention 187 Synchronizing Singletons Correctly Context •JAWS uses various singletons to implement components where only one instance is required •e.g., the ACE Reactor, the request queue, etc. Problem •Singletons can be problematic in multi-threaded programs … or too much Either too little locking… class Singleton { class Singleton { public: public: static Singleton *instance () static Singleton *instance () { { Guard<Thread_Mutex> g (lock_); if (instance_ == 0) { if (instance_ == 0) { // Enter critical section. // Enter critical section. instance_ = new Singleton; instance_ = new Singleton; // Leave critical section. // Leave critical section. } } return instance_; return instance_; } } void method_1 (); private: // Other methods omitted. static Singleton *instance_; private: // Initialized to 0 by linker. static Singleton *instance_; // Initialized to 0 by linker. static Thread_Mutex lock_; }; }; 188 The Double-checked Locking Optimization Pattern Solution •Apply the Double-Checked Locking Optimization design pattern (P2) to reduce contention & synchronization overhead whenever critical sections of code must acquire locks in a thread-safe manner just once during program execution // Perform first-check to class Singleton { public: // evaluate ‘hint’. static Singleton *instance () if (first_time_in is TRUE) { { // First check acquire the mutex if (instance_ == 0) { Guard<Thread_Mutex> g(lock_); // Perform double-check to // Double check. // avoid race condition. if (instance_ == 0) if (first_time_in is TRUE) instance_ = new Singleton; { } return instance_; execute the critical section } set first_time_in to FALSE private: } static Singleton *instance_; release the mutex static Thread_Mutex lock_; }; } 189 Applying the Double-Checked Locking Optimization Pattern in ACE template <class TYPE> ACE defines a class ACE_Singleton { “singleton adapter” public: template to automate static TYPE *instance () { // First check the double-checked #if defined (NON_SC_MEMORY) locking optimization mb (); // Update the cache. #endif /* NON_SC_MEMORY */ if (instance_ == 0) { // Scoped Locking acquires & release lock. ACE_Guard<ACE_Thread_Mutex> guard (lock_); // Double check instance_. if (instance_ == 0) { instance_ = new TYPE; #if defined (NON_SC_MEMORY) mb (); // Update the cache. #endif /* NON_SC_MEMORY */ } Thus, creating a “thread} safe” singleton is easy return instance_; } typedef ACE_Singleton <Request_Queue> private: Request_Queue_Singleton; static TYPE *instance_; static ACE_Thread_Mutex lock_; }; 190 Pros & Cons of the Double-Checked Locking Optimization Pattern This pattern has two benefits: •Minimized locking overhead • By performing two first-time-in flag checks, this pattern minimizes overhead for the common case • After the flag is set the first check ensures that subsequent accesses require no further locking •Prevents race conditions • The second check of the firsttime-in flag ensures that the critical section is executed just once 191 This pattern has some liabilities: •Non-atomic pointer or integral assignment semantics • If an instance_ pointer is used as the flag in a singleton implementation, all bits of the singleton instance_ pointer must be read & written atomically in a single operation • If the write to memory after the call to new is not atomic, other threads may try to read an invalid pointer •Multi-processor cache coherency • Certain multi-processor platforms, such as the COMPAQ Alpha & Intel Itanium, perform aggressive memory caching optimizations in which read & write operations can execute ‘out of order’ across multiple CPU caches, such that the CPU cache lines will not be flushed properly if shared data is accessed without locks held Logging Access Statistics Efficiently Context •Web servers often need to log certain information •e.g., count number of times web pages are accessed update_count() update_count() Problem •Having a central logging object in a multi-threaded server process can become a bottleneck • e.g., due to synchronization required to serialize access by multiple threads 192 update_count() Web page hit count logger Logging Access Statistics Efficiently Solution •Apply the Thread-Specific Storage design pattern (P2) to allow multiple threads to use one ‘logically global’ access point to retrieve an object that is local to a thread, without incurring locking overhead on each object access Application Thread errno is a good example of threadspecific storage 193 m <<uses>> Thread-Specific Object Proxy key method1() … methodN() Key Factory create_key() n m Thread-Specific Object Set calls get(key) set(key, object) maintains n x m Thread-Specific Object method1() … methodN() Thread-Specific Storage Pattern Dynamics The application thread identifier, thread-specific object set, & proxy cooperate to obtain the Thread-Specific correct thread-specific object Object Set manages thread 1 thread m key 1 Thread-Specific Object Proxy Thread-Specific Object [k,t] accesses key n : Application Thread : Thread-Specific Object Proxy method() : Key Factory : Thread-Specific Object Set create_key() key : Thread-Specific Object TSObject 194 key set() Applying the Thread-Specific Storage Pattern to JAWS template <class TYPE> n m Thread-Specific Application m <<uses>> ACE_TSS Class ACE_TSS { calls Object Set Thread key public: get(key) operator->() TYPE *operator->() const { set(key, object) TYPE *tss_data = 0; if (!once_) { maintains n x m ACE_Guard<ACE_Thread_Mutex> g (keylock_); Key Factory create_key() if (!once_) { Error_Logger ACE_OS::thr_keycreate last_error() (&key_, &cleanup_hook); log() once_ = true; … class Error_Logger { } public: } int last_error (); ACE_OS::thr_getspecific void log (const char *format, (key_, (void **) &tss_data); ...); if (tss_data == 0) { }; tss_data = new TYPE; ACE_OS::thr_setspecific (key_, (void *) tss_data); ACE_TSS <Error_Logger> } my_logger; return tss_data; // ... } if (recv (……) == -1 && my_logger->last_error () != private: EWOULDBLOCK) mutable pthread_key_t key_; my_logger->log mutable bool once_; (“recv failed, errno = %d”, mutable ACE_Thread_Mutex keylock_; my_logger->last_error ()); static void cleanup_hook (void *ptr); }; }; 195 Pros & Cons of the Thread-Specific Storage Pattern This pattern also has liabilities: •It encourages use of thread• It’s possible to implement this pattern specific global objects This pattern has four benefits: •Efficiency so that no locking is needed to access thread-specific data •Ease of use • When encapsulated with wrapper facades & proxies, thread-specific storage is easy for application developers to use •Reusability • By combining this pattern with the Wrapper Façade pattern it’s possible to shield developers from nonportable OS platform characteristics •Portability • It’s possible to implement portable thread-specific storage mechanisms on most multi-threaded operating systems 196 • Many applications do not require multiple threads to access threadspecific data via a common access point • In this case, data should be stored so that only the thread owning the data can access it •It obscures the structure of the system • The use of thread-specific storage potentially makes an application harder to understand, by obscuring the relationships between its components •It restricts implementation options • Not all languages support parameterized types or smart pointers, which are useful for simplifying the access to thread-specific data Additional Information •Patterns & frameworks for concurrent & networked objects •www.cs.wustl.edu/~schmidt/POSA/ •ACE & TAO open-source middleware •www.cs.wustl.edu/~schmidt/ACE.html •www.cs.wustl.edu/~schmidt/TAO.html •ACE research papers •www.cs.wustl.edu/~schmidt/ACE-papers.html •Extended ACE & TAO tutorials •UCLA extension, Feb, 2008 •www.cs.wustl.edu/~schmidt/UCLA.html •ACE books 197 •www.cs.wustl.edu/~schmidt/ACE/ Example: Applying Patterns to Real-time CORBA www.cs.wustl.edu/~schmidt/POSA Patterns are used throughout The ACE ORB (TAO) Real-time CORBA implementation to codify expert knowledge & to generate the ORB’s software architecture by capturing recurring structures & dynamics & resolving common design forces 198 R&D Context for ACE+TAO+CIAO Our R&D focus: Advancing disruptive technologies to commoditize distributed real-time & embedded (DRE) systems Standards-based QoSenabled Middleware 199 Model-based Software Development & Domain-specific Languages Patterns & Pattern Languages Open-source Standardsbased COTS TAO–The ACE ORB OBJ REF in args operation() out args + return Component (Servant) Services Client IDL SKEL Container DII IDL STUBS ORB INTERFACE ORB CORE Object Adapter GIOP/IIOP/ESIOPS • Objective: Advance technology to simplify development of distributed, real-time, & embedded (DRE) systems • Approach: Use standard OO techologies & patterns 200 • More than 500 Ksloc (C++) • Open-source • Based on ACE wrapper facades & frameworks • Available on Unix, Win32, MVS, QNX, VxWorks, LynxOS, VMS, etc. • Thousands of users around the world •Commercially supported by •OCI (www.theaceorb.com) •PrismTech (www.prismtechnologies.com) •Remedy (www.remedy.nl) •etc. The Evolution of TAO •TAO can be downloaded from • deuce.doc.wustl.edu/Download.html 201 TAO ORB • Largely compliant with CORBA 3.0 • No DCOM bridge ;-) • Pattern-oriented software architecture • www.posa.uci.edu • Key capabilities • QoS-enabled • Highly configurable • Pluggable protocols • IIOP/UIOP • DIOP • Shared memory • SSL • MIOP • SCIOP The Evolution of TAO RT-CORBA • Portable priorities • Protocol properties • Standard synchronizers • Explicit binding mechanisms • Thread pools RT-CORBA 1.0 TAO 1.5 (Mar ’06) • Current “official” release of TAO • Heavily tested & optimized • Baseline for next OCI & PrismTech supported releases • www.dre.vanderbilt.edu/ scoreboard ZEN • RT-CORBA/RT-Java • Alpha now available www.zen.uci.edu 202 The Evolution of TAO DYNAMIC/STATIC SCHEDULING A/V STREAMING RT-CORBA 1.0 203 Static Scheduling (1.0) • Rate monotonic analysis Dynamic Scheduling (1.2) • Earliest deadline first • Minimum laxity first • Maximal urgency first Hybrid Dynamic/Static • Demo in WSOA • Kokyu integrated in Summer 2003 A/V Streaming Service • QoS mapping • QoS monitoring • QoS adaptation ACE QoS API (AQoSA) • GQoS/RAPI & DiffServ • IntServ integrated with A/V Streaming & QuO • DiffServ integrated with ORB The Evolution of TAO DYNAMIC/STATIC SCHEDULING FT-CORBA & LOAD BALANCING A/V STREAMING SECURITY RT-CORBA 1.0 204 FT-CORBA (DOORS) • Entity redundancy • Multiple models • Cold passive • Warm passive • IOGR • HA/FT integrated by Winter 2004 Load Balancing • Static & dynamic • Integrated in TAO 1.3 • De-centralized LB • OMG LB specification SSL Support • Integrity • Confidentiality • Authentication (limited) Security Service (CSIv2) • Authentication • Access control • Non-repudiation • Audit • Beta by Winter 2004 The Evolution of TAO DYNAMIC/STATIC SCHEDULING FT-CORBA & LOAD BALANCING A/V STREAMING SECURITY NOTIFICATIONS TRANSACTIONS Notification Service •Structured events •Event filtering •QoS properties • Priority • Expiry times • Order policy •Compatible w/Events RT-CORBA 1.0 Real-time Notification Service •Summer 2003 Object Transaction Service • Encapsulates RDBMs • www.xots.org 205 The Evolution of TAO DYNAMIC/STATIC SCHEDULING FT-CORBA & LOAD BALANCING A/V STREAMING SECURITY RT-CORBA 1.0 206 NOTIFICATIONS TRANSACTIONS CORBA Component Model (CIAO) • Extension Interfaces • Component navigation • Standardized lifecycles • QoS-enabled containers • Reflective collocation • Implements the OMG Deployment & Configuration specification • First major release (1.0) by Winter 2005 The Road Ahead (1/3) • Limit to how much application functionality can be factored into reusable COTS middleware, which impedes product-line architectures CORBA Apps CORBA Services CORBA J2EE .NET Apps Middleware J2EE Services Services .NET Services DRE Applications Apps Middleware J2EE .NET • Middleware itself has become extremely complicated to use & provision statically & dynamically Load Balancer FT CORBA RT/DP CORBA + DRTSJ Connections & priority bands RTOS + RT Java CPU & memory IntServ + Diffserv Operating Sys & Protocols Hardware & Networks 207 Workload & Replicas Network latency & bandwidth • Component-based DRE systems are very complicated to deploy & configure • There are now multiple middleware technologies to choose from The Road Ahead (2/3) • Develop, validate, & standardize model-driven development (MDD) software technologies that: DRE Applications Middleware Services <CONFIGURATION_PASS> <HOME> <…> <COMPONENT> <ID> <…></ID> <EVENT_SUPPLIER> <…events this component supplies…> </EVENT_SUPPLIER> </COMPONENT> </HOME> </CONFIGURATION_PASS> Middleware Operating Sys & Protocols Hardware & Networks 208 1. Model 2. Analyze 3. Synthesize & 4. Provision multiple layers of middleware & application components that require simultaneous control of multiple quality of service properties end-to-end • Partial specialization is essential for inter-/intra-layer optimization & advanced product-line architectures Goal is not to replace programmers per se – it is to provide higher-level domain-specific languages for middleware/application developers & users The Road Ahead (3/3) Our MDD toolsuite is called CoSMIC (“Component Synthesis using Model Integrated Computing”) www.dre.vanderbilt.edu/cosmic 209 Concluding Remarks •Researchers & developers of distributed applications face common challenges R&D Synergies R&D User Needs Standard COTS R&D •e.g., connection management, service initialization, error handling, flow & congestion control, event demuxing, distribution, concurrency control, fault tolerance synchronization, scheduling, & persistence •Patterns, frameworks, & components help to resolve these challenges •These techniques can yield efficient, scalable, predictable, & flexible middleware & applications 210

Patterns - Distributed Object Computing

Related documents

Products

Support

Patterns - Distributed Object Computing

Related documents

Add this document to collection(s)

Add this document to saved

Suggest us how to improve StudyLib