Engineering Large Evolving Systems Ben Potter Department of Informatics and Sensors, Cranfield University Contents • The case study – Network Enabled Capability • Challenges for engineering • Discussion What is NEC? • The attainment of a Network Enabled Capability in UK Defence • Sometimes also referred to as Capability Enhanced by Networking – not as snappy but perhaps more descriptive! • UK MOD Joint Service Publication JSP 777 says: – ‘The achievement of military effect will, in future, be significantly enhanced through the networking of existing and future military capabilities, under the banner of NEC’ What is NEC ? Working definition (from MOD website): "Linking sensors, decision makers and weapon systems so that information can be translated into synchronised and overwhelming military effect at optimum tempo" What is NEC ? “NEC is about the coherent integration of sensors, decision-makers and weapon systems along with support capabilities” A Navy view? An Army view? An aside • 1976 Networkshop… Some questions: • Given (say) a 12 year horizon for NEC: 1. Who can describe the military threats to be met? 2. Who can imagine the technologies to be used? 3. How can ‘traditional’ systems engineering help, given that: – 1 above suggests we can’t write the requirements – 2 above suggests we can’t describe a solution which will not be obsolete – First – a definition: Engineered Large Yet Evolving Systems – ELYES. NEC is an ELYES. – This leads us to appeal to the origins of systems engineering for help – or, at least – guidance What is a ‘System?’ System as a set of interacting parts “A system is an open set of complementary, interacting parts with properties, capabilities, and behaviours emerging both from the parts and from their interactions.” Hitchins, Advanced Systems Thinking, Engineering and Management, 2003 System as a way of conceptualising “… the concept of ‘system’ is used not to refer to things in the world but to a particular way of organising our thoughts about the world. [..] we consider the notion of ‘system’ as an organising concept …” Flood and Jackson, Creative Problem Solving – Total Systems Intervention, 2004 Different ‘organising concepts’ Engineer Family Person Chancellor of Exchequer Cyclist The System Boundary “… the scientist has to have a way of thinking about the environment of a system that is richer and more subtle than a mere looking at for boundaries. He does this by noting that, when we say that something lies ‘outside’ the system, we mean that system can do relatively little about its characteristics or its behavior. Environment, in effect, makes up the things and people that are ‘fixed’ or ‘given’, from the system’s point of view. …” Churchman, The Systems Approach, 1968 A system’s emergent properties “Emergence is the phenomenon of properties, capabilities and behaviours evident in the whole system that are not exclusively ascribable to any of its parts.” Hitchins, Advanced Systems Thinking, Engineering and Management, 2003 Summary: a system is … • • • • A organising concept – a way of structuring a concept or problem or a thing from a particular perspective. A system is made up of a set of interacting parts The system has properties and behaviour that emerge from the interaction of the properties and behaviours of the individual parts. The system has a boundary defined in terms of ‘things that affect it but that it cannot affect.’ – but the boundary can be broad and permeable. How do we analyse ‘Systems’? How we analyse systems “… This is the basic principle of ‘classical’ science, which can be circumscribed in different ways: resolution into isolable casual trains, seeking for ‘atomic’ units in the various fields of science, etc. The progress of science has shown that these principles of classical science – first encountered by Galileo and Descartes – are highly successful in a wide range of phenomena. Application of the analytical procedure depends on two conditions. The first is that interactions between ‘parts’ be non-existent or weak enough to be neglected for certain research purposes. Only under this condition, can the parts be ‘worked out’, actually, logically and mathematically, and then be ‘put together’. The second condition is that the relations describing the behaviour of the parts be linear; only then is the condition of summativity given …” Bertalanffy, General Systems Theory, 1969 How we analyse systems “… This is the basic principle of ‘classical’ science, which can be circumscribed in different ways: resolution into isolable casual trains, seeking for ‘atomic’ units in the various fields of science, etc. The progress of science has shown that these principles of classical science – first encountered by Galileo and Descartes – are highly successful in a wide range of phenomena. Application of the analytical procedure depends on two conditions. The first is that interactions between ‘parts’ be non-existent or weak enough to be neglected for certain research purposes. Only under this condition, can the parts be ‘worked out’, actually, logically and mathematically, and then be ‘put together’. The second condition is that the relations describing the behaviour of the parts be linear; only then is the condition of summativity given …” Bertalanffy, General Systems Theory, 1969 How we analyse systems “… This is the basic principle of ‘classical’ science, which can be circumscribed in different ways: resolution into isolable casual trains, seeking for ‘atomic’ units in the various fields of science, etc. The progress of science has shown that these principles of classical science – first encountered by Galileo and Descartes – are highly successful in a wide range of phenomena. Application of the analytical procedure depends on two conditions. The first is that interactions between ‘parts’ be non-existent or weak enough to be neglected for certain research purposes. Only under this condition, can the parts be ‘worked out’, actually, logically and mathematically, and then be ‘put together’. The second condition is that the relations describing the behaviour of the parts be linear; only then is the condition of summativity given …” Bertalanffy, General Systems Theory, 1969 How we analyse systems “… This is the basic principle of ‘classical’ science, which can be circumscribed in different ways: resolution into isolable casual trains, seeking for ‘atomic’ units in the various fields of science, etc. The progress of science has shown that these principles of classical science – first encountered by Galileo and Descartes – are highly successful in a wide range of phenomena. Application of the analytical procedure depends on two conditions. The first is that interactions between ‘parts’ be non-existent or weak enough to be neglected for certain research purposes. Only under this condition, can the parts be ‘worked out’, actually, logically and mathematically, and then be ‘put together’. The second condition is that the relations describing the behaviour of the parts be linear; only then is the condition of summativity given …” Bertalanffy, General Systems Theory, 1969 How we analyse systems “… the method of classical science was most appropriate for phenomena that either can be resolved into isolated causal chains, or are the statistical outcome of an ‘infinite’ number of chance processes, as is true of statistical mechanics, the second principle of thermodynamics and all the laws deriving from it. The classical modes of thinking, however, fail in the case of interaction of a large but limited number of elements of processes. Here those problems arise which are circumscribed by such notions as wholeness, organisation and the like, and which demand new ways of thinking.” Bertalanffy, General Systems Theory, 1969 How we analyse systems “… the method of classical science was most appropriate for phenomena that either can be resolved into isolated causal chains, or are the statistical outcome of an ‘infinite’ number of chance processes, as is true of statistical mechanics, the second principle of thermodynamics and all the laws deriving from it. The classical modes of thinking, however, fail in the case of interaction of a large but limited number of elements of processes. Here those problems arise which are circumscribed by such notions as wholeness, organisation and the like, and which demand new ways of thinking.” Bertalanffy, General Systems Theory, 1969 How we analyse systems “… the method of classical science was most appropriate for phenomena that either can be resolved into isolated causal chains, or are the statistical outcome of an ‘infinite’ number of chance processes, as is true of statistical mechanics, the second principle of thermodynamics and all the laws deriving from it. The classical modes of thinking, however, fail in the case of interaction of a large but limited number of elements of processes. Here those problems arise which are circumscribed by such notions as wholeness, organisation and the like, and which demand new ways of thinking.” Bertalanffy, General Systems Theory, 1969 ‘System space’ Statistics and dynamics relating to very large numbers of well defined nodes and couplings. Increasing number of ‘nodes’ Nodes that have ‘adaptive’, non-trivial couplings. Decoupled or very well defined couplings between relatively few nodes. Increasing intricacy of the ‘couplings’ Coping strategies Increasing number of ‘nodes’ Tendency to treat ‘complex’ systems within ‘comfort zones’ Tendency to try to inappropriately extend ‘comfort zone’ techniques Increasing intricacy of the ‘couplings’ Example Systems Gas in a container Weather Climate Increasing Flock of birds number of ‘nodes’ Termite hill Space shuttle Aircraft Car Organisations Teams Increasing intricacy of the ‘couplings’ Detailed example systems Emergent behaviour: ride comfort, acceleration… Emergent behaviour: pressure… … and is very determinant Car Driver Engine Drive Train Fuel system … and can be determined through statistical and probabilistic analysis Gas in a container Gas atoms and molecules Detailed example ELYES Emergent behaviour: effects… … and cannot be determined to any degree of certainty Military Capability Command Inform Operate Protect Prepare Project Sustain Example Systems Gas in a container Weather Climate Increasing Flock of birds number of ‘nodes’ Termite hill Organisations Space shuttle Aircraft Organisations ELYES System Car Teams Increasing intricacy of the ‘couplings’ Architecting an ELYES • • • ELYES Architecture is concerned with optimisation at the global level, sometimes necessarily at the expense of local considerations In pursuit of this goal, the ELYES Architect may be tempted to over-simplify and • Adopt a one-size-fits-all approach • See more homogeneity in the ELYES than there really is • Assume linear, predictable systems • Focus on static, structural elements, while ignoring dynamic interactions While this response is understandable, is it reasonable? The problem • Changing environments require systems/ELYESs to change: • Either to reflect the changes in the environment or anticipate them. • The unpredictable nature of the environment includes: • Dynamic entities (threats and contributors) within the environment. • New entities that are constantly evolving and old ones disappearing. • Relationships between the entities that are constantly changing. • The overall behaviour exhibited by inter-related entities are emergent. • The characteristics of the system/ELYES have to be at least as complex as its environment. Thesis / Premise • Traditionally we design systems by identifying the ‘static’ customer need, using a top-down, reductionist approach. This single mode of analysis struggles to cope with: • Dynamic interactions within the system • Non-functional requirements (e.g. agility and flexibility) • Complex emergent properties and behaviours • We argue that for ELYESs to operate it requires appropriate developmental and operational environments in which traditional systems can be developed and used. Systems and ELYESs Systems ELYES System is reproducible No two instantiations are alike System is realised to meet a single, pre-conceived need No single, or set of needs can be pre-conceived for the ELYES. The ELYES has to continually evolve to meet the changing need. System has well-defined boundaries and behaviour. The ELYES has ambiguous boundaries and unpredictable behaviour. Development always ends for each instance of system realisation ELYES development never ends – the ELYES evolves External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. The ELYES is self-integrating and re-integrating in order to meet the changing need. Unwanted possibilities and unknowns are removed before use of the system New possibilities are constantly assessed for utility and feasibility in the evolution of the ELYES. Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed The ELYES depends on both internal cooperation and internal competition to stimulate its evolution. Very clear distinction between development/design, production and use. Development, production and use all run in parallel. Systems: Implications for development Systems Implications for Development System is reproducible Once the design is fixed the production can be ‘mechanized’. System is realized to meet a single, pre-conceived need The design can be fixed, the ‘need’ does not change once it has been agreed. There is an agreed ‘product’. System has well-defined boundaries and behaviour. Development and production can be decomposed into sub-products with well defined behaviours, which can then be composed to form the whole. Development always ends for each instance of system realization No further development once the product is handed over to the user. External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. The user cannot change the product. Unwanted possibilities and unknowns are removed before use of the system The behaviour of the delivered product is bounded and known. Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed The product is not delivered until the user knows how to use it. Very clear distinction between development/design, production and use. Design, production and use can be treated as totally separate phases with mechanised processes and well defined boundaries and products. ELYESs: Implications for development ELYESs Implications for Development No two instantiations are alike The ‘design’ cannot be fixed and the production cannot be ‘mechanized’. No single, or set of needs can be pre-conceived for the System. The system has to continually evolve to meet the changing need. The need for the ELYES is continually changing. The ELYES can only evolve through use. System has ambiguous boundaries and unpredictable behaviour. System development never ends – system evolves System is self-integrating and re-integrating in order to meet the changing need. New possibilities are constantly assessed for utility and feasibility in the evolution of the system System depends on both internal cooperation and internal competition to stimulate its evolution. Development, production and use all run in parallel. Not all sub-units can be identified and some may not be controllable. Achieving the desired behaviour is about creating the appropriate environment and influencing sub-units. An appropriate environment has to be developed that lets the system evolve through use. The user has freedom to re-integrate as required. The ELYES must be designed to be ‘modular’ with configuration ‘rules’. The ELYES is never completed and the role of development and production is to provide the user with the means for evolution. The ELYES is heterogeneous and relies on conflicts and collaborations to evolve. These must not be ‘designed out’. The development, production and use lifecycle must be adaptable and not prescriptive. System acquisition Specification Ability to do stuff Target Requirements Time Programme ELYES Acquisition ? Specification (Tolerance) Ability to do stuff ? ? Principles ? Time Programme Implications for engineering an ELYES It’s about setting the right environment NOT designing top down. Designing the ELYES Operational Need ELYES Architecture Development Environment Cap 1 Kit Kit ELYES Design Cap 2 Cap 3 Kit Operational Environment Cap n Kit Evolving the ELYES ELYES Architecture ‘Big Picture’ Partition Rules Operational Need Development Environment Orchestration Rules Cap 1 Kit Cap n Kit Orchestration Rules Operational Environment Kit Implications for engineering an ELYES • Capability development is enabled by providing the appropriate development environment. • Number of and definition of capability building blocks – Cannot be predetermined – Will evolve independently – Will have overlapping functionality with one another. – Redundancy, i.e. variety, in the system is desirable • Operational use of the capability is defined by orchestration rules which will encourage appropriate ‘usage’ philosophy. • The orchestration rules are defined during development and will evolve. • Evolution during operational use must drive development. Discussion Specification Example: A Bridge Ability to do stuff Target Time Systems ELYES System is reproducible No two instantiations are alike System is realized to meet a single, pre-conceived need No single, or set of needs can be pre-conceived for the ELYES. The ELYES has to continually evolve to meet the changing need. System has well-defined boundaries and behaviour. The ELYES has ambiguous boundaries and unpredictable behaviour. Development always ends for each instance of system realization ELYES development never ends – the ELYESr evolves External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. The ELYES is self-integrating and re-integrating in order to meet the changing need. Unwanted possibilities and unknowns are removed before use of the system New possibilities are constantly assessed for utility and feasibility in the evolution of the ELYES Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed The ELYES depends on both internal cooperation and internal competition to stimulate its evolution. Very clear distinction between development/design, production and use. Development, production and use all run in parallel. Specification Example: Software Ability to do stuff Target Time Systems ELYES System is reproducible No two instantiations are alike System is realized to meet a single, pre-conceived need No single, or set of needs can be pre-conceived for the ELYES. The ELYES has to continually evolve to meet the changing need. System has well-defined boundaries and behaviour. The ELYES has ambiguous boundaries and unpredictable behaviour. Development always ends for each instance of system realization ELYES development never ends – the ELYES evolves External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. The ELYES is self-integrating and re-integrating in order to meet the changing need. Unwanted possibilities and unknowns are removed before use of the system New possibilities are constantly assessed for utility and feasibility in the evolution of the ELYES. Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed The ELYES depends on both internal cooperation and internal competition to stimulate its evolution. Very clear distinction between development/design, production and use. Development, production and use all run in parallel. Specification Example: Space Race Ability to do stuff Target Sub-Target Sub-Target Sub-Target Time Systems ELYES System is reproducible No two instantiations are alike System is realized to meet a single, pre-conceived need No single, or set of needs can be pre-conceived for the ELYES. The ELYES has to continually evolve to meet the changing need. System has well-defined boundaries and behaviour. The ELYES has ambiguous boundaries and unpredictable behaviour. Development always ends for each instance of system realization ELYES development never ends – the ELYES evolves External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. The ELYES is self-integrating and re-integrating in order to meet the changing need. Unwanted possibilities and unknowns are removed before use of the system New possibilities are constantly assessed for utility and feasibility in the evolution of the ELYES Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed The ELYES depends on both internal cooperation and internal competition to stimulate its evolution. Very clear distinction between development/design, production and use. Development, production and use all run in parallel. Specification (Tolerance) Example: My kitchen Ability to do stuff Time Systems ELYES System is reproducible No two instantiations are alike System is realized to meet a single, pre-conceived need No single, or set of needs can be pre-conceived for the ELYES. The ELYES has to continually evolve to meet the changing need. System has well-defined boundaries and behaviour. The ELYES has ambiguous boundaries and unpredictable behaviour. Development always ends for each instance of system realization ELYES development never ends – the ELYES evolves External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. The ELYES is self-integrating and re-integrating in order to meet the changing need. Unwanted possibilities and unknowns are removed before use of the system New possibilities are constantly assessed for utility and feasibility in the evolution of the ELYES Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed The ELYES depends on both internal cooperation and internal competition to stimulate its evolution. Very clear distinction between development/design, production and use. Development, production and use all run in parallel. Specification (Range of uses) Example: NEC Ability to do stuff Time Systems ELYES System is reproducible No two instantiat6ions are alike System is realized to meet a single, pre-conceived need No single, or set of needs can be pre-conceived for the ELYES. The ELYES has to continually evolve to meet the changing need. System has well-defined boundaries and behaviour. The ELYES has ambiguous boundaries and unpredictable behaviour. Development always ends for each instance of system realization ELYES development never ends – the ELYES evolves External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. The ELYES is self-integrating and re-integrating in order to meet the changing need. Unwanted possibilities and unknowns are removed before use of the system New possibilities are constantly assessed for utility and feasibility in the evolution of the ELYES Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed The ELYES depends on both internal cooperation and internal competition to stimulate its evolution. Very clear distinction between development/design, production and use. Development, production and use all run in parallel. Summary: Lifecycles Software Bridge Specification Specification Ability to do stuff Ability to do stuff Target Target Time Enterprise System is reproducible No two instantiations are alike System is realized to meet a single, pre-conceived need No single, or set of needs can be pre-conceived for the Enterprise. The Enterprise has to continually evolve to meet the changing need. Enterprise Systems System is reproducible No two instantiations are alike System is realized to meet a single, pre-conceived need No single, or set of needs can be pre-conceived for the Enterprise. The Enterprise has to continually evolve to meet the changing need. System has well-defined boundaries and behaviour. The Enterprise has ambiguous boundaries and unpredictable behaviour. Development always ends for each instance of system realization Enterprise development never ends – the Enterpriser evolves External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. The Enterprise is self-integrating and re-integrating in order to meet the changing need. Unwanted possibilities and unknowns are removed before use of the system Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed Very clear distinction between development/design, production and use. Development, production and use all run in parallel. System has well-defined boundaries and behaviour. The Enterprise has ambiguous boundaries and unpredictable behaviour. Development always ends for each instance of system realization Enterprise development never ends – the Enterprise evolves External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. The Enterprise is self-integrating and re-integrating in order to meet the changing need. Unwanted possibilities and unknowns are removed before use of the system New possibilities are constantly assessed for utility and feasibility in the evolution of the Enterprise. New possibilities are constantly assessed for utility and feasibility in the evolution of the Enterprise Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed The Enterprise depends on both internal cooperation and internal competition to stimulate its evolution. The Enterprise depends on both internal cooperation and internal competition to stimulate its evolution. Very clear distinction between development/design, production and use. Development, production and use all run in parallel. Ability to do stuff Space Race Specification Systems Time Target Sub-Target Sub-Target Sub-Target Time Systems Ability to do stuff Specification (Range of uses) NEC Kitchen System is reproducible No two instantiations are alike System is realized to meet a single, pre-conceived need No single, or set of needs can be pre-conceived for the Enterprise. The Enterprise has to continually evolve to meet the changing need. System has well-defined boundaries and behaviour. The Enterprise has ambiguous boundaries and unpredictable behaviour. Development always ends for each instance of system realization Enterprise development never ends – the Enterprise evolves External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. The Enterprise is self-integrating and re-integrating in order to meet the changing need. Unwanted possibilities and unknowns are removed before use of the system New possibilities are constantly assessed for utility and feasibility in the evolution of the Enterprise Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed The Enterprise depends on both internal cooperation and internal competition to stimulate its evolution. Very clear distinction between development/design, production and use. Development, production and use all run in parallel. Time Systems Enterprise No two instantiat6ions are alike System is realized to meet a single, pre-conceived need No single, or set of needs can be pre-conceived for the Enterprise. The Enterprise has to continually evolve to meet the changing need. System has well-defined boundaries and behaviour. The Enterprise has ambiguous boundaries and unpredictable behaviour. Development always ends for each instance of system realization Enterprise development never ends – the Enterprise evolves External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. The Enterprise is self-integrating and re-integrating in order to meet the changing need. Unwanted possibilities and unknowns are removed before use of the system New possibilities are constantly assessed for utility and feasibility in the evolution of the Enterprise Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed The Enterprise depends on both internal cooperation and internal competition to stimulate its evolution. Very clear distinction between development/design, production and use. Development, production and use all run in parallel. Enterprise System is reproducible No two instantiations are alike System is realized to meet a single, pre-conceived need No single, or set of needs can be pre-conceived for the Enterprise. The Enterprise has to continually evolve to meet the changing need. Time Systems Specification (Tolerance) Ability to do stuff Enterprise System is reproducible System has well-defined boundaries and behaviour. The Enterprise has ambiguous boundaries and unpredictable behaviour. Development always ends for each instance of system realization Enterprise development never ends – the Enterprise evolves External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. The Enterprise is self-integrating and re-integrating in order to meet the changing need. Unwanted possibilities and unknowns are removed before use of the system New possibilities are constantly assessed for utility and feasibility in the evolution of the Enterprise Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed The Enterprise depends on both internal cooperation and internal competition to stimulate its evolution. Very clear distinction between development/design, production and use. Development, production and use all run in parallel.