System-of-Systems Architecting:
Critical Success Factors
Dr. Azad M. Madni
Chief Executive Officer
USC-CSSE Convocation:
Executive Workshop Presentation
October 23-26, 2006
3250 Ocean Park Blvd., Suite 100, Santa Monica, CA 90405
310-581-5440 Fax: 310-581-5430 www.IntelSysTech.com
Copyright © 2006 Intelligent Systems Technology, Inc.
Outline
•
•
•
•
•
•
•
•
First Hand Experiences
System-of-Systems (SoS) Definition
SoS Architecture
SoS Architecting Complexities
Promising Approaches
Implementation Options
Challenges Ahead
Critical Success Factors
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/2
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
First-Hand Experiences
• Principal Investigator on DARPA, Army, Navy, Air Force, DoE, NIST
Research Initiatives
– Ontology-enabled Agile Visualization of Multi-domain SoS data (sponsor: MDA)
– Development of Toolset for DoDAF – compliant SoS Architecting (sponsor: U.S.
Army RDECOM)
– Process-enabled Formation and Management of Agile Virtual Enterprises
(sponsor: DARPA DSO)
– Virtual Prototyping of Combat Information Systems (sponsor: NSWCDD)
– Integrated Product-Process Development (sponsor: NSWCDD, DARPA ISTO)
– Development of Simulation-enabled Activity-Based Costing System (sponsor:
AFRL)
– Development of ANSI-EIA-632 systems engineering process library (sponsor:
ONR, NSWC, DARPA ISTO)
– Executable Models in Computer-aided Concurrent Engineering (sponsor: DARPA
DSO DICE)
– Simulation-based Design and Analysis of Agile Manufacturing Systems (sponsor:
DoE TEAM Program)
– Semantics of Process Description (sponsor: NIST PSL Program)
– Design Process Management for complex products (sponsor: DARPA ESTO)
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/3
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
SoS Definitions Vary
• Arrangement of interdependent systems connected to
provide a capability greater than the sum of the
member systems.
– GAO, January 2006
• A federation of systems, whose context, mission, and
operational logic are required for handling the
allocation and coordination of shared, integrated
resources
– Dr. Ray Paul, OASD/NII
• Several others
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/4
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
My Definition of SoS
(Architecting Perspective)
• A large, complex, ensemble …
… of interdependent systems
… developed and introduced over different time-frames
… by multiple independent authorities
… to provide multiple, interdependent capabilities
… in support of multiple missions
•An SoS capability typically exceeds the sum of the
capabilities of the member systems
(Madni, 2006)
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/5
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
DoN Operational Forces
Systems-of-Systems support these forces.
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/6
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
System-to-Capabilities Mapping
System
System
2 System
System
3 System
System
4 System
System
5 System
System
6
System
1 IA1 1 System
2 IA
3 IA
4 IA
5 IA
6 IA
System
System
2
System
3
System
4
System
5
System
6
1
System
2
System
3 (from
System
4 (from
System
5 (from
System
6
IASystem
(from
IA
(from
IA
(from
IA
(from
IA
(from
IA
(from
(from
(from
(from
NRNRNRIASystem
(from 1 IASystem
(from 2 IASystem
(from 3 IASystem
(from 4 IASystem
(from 5 IASystem
(from 6
IASystem
(from 1 NRIASystem
(from 2 NRIASystem
(from 3 NRIASystem
(from 4 NRIASystem
(from 5 NRIASystem
(from 6
NRNR-KPP)
IA (fromNR-KPP)
IA (fromNR-KPP)
IA (from KPP)
IA (from KPP)
IA (from KPP)
IA (from
NRNRNRNRNRNRIA (from KPP)NRIA (from KPP)NRIA (from KPP)NRIA (from KPP)NRIA (from KPP)NRIA (from
KPP)NRKPP)NRKPP)NRKPP)NRKPP)NRKPP)NRKPP)NRKPP)NRKPP)NRKPP)NRKPP)NRKPP)NRKPP)NRKPP)
KPP)
KPP)
KPP)
KPP)
KPP)
KPP)
KPP)
KPP)
KPP)
KPP)
KPP)
System-Level
Principal Supported MCP
(determines Lead FA CHENG)
Secondary supported MCP
(determines Supporting FA CHENGs)
Sub-Functional ISR/BA MCP C&N/I MCP TAMD MCP
Areas Strike MCP C2/DS MCP Surface Warfare
MCP
Functional
Areas
FORCEnet IA
Sea Strike IA
Logistics MCP
Force
Deployment
MCP
EMW Intel
EMW C2
Legal Portfolio
Comms, Networking, and Services Infrastructure
Sea Shield IA
Sea Basing IA
EMW IA
Sea Enterprise
Enterprise
NP21 Integrated Architecture
IA Integrated Architecture
Performance Parameter
MCP Military Capabilities Package
NR-KPP Net-Ready Key
One system can serve in different SoSs to satisfy requirements of
different military capability packages.
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/7
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
System Versus SoS
Comparison Criteria
System
System-of-System (SoS)
Focus
creation of the system
creation of a capability
Lifetime
finite; can be extended
indefinite; continually extended through
replacement/addition of member systems
Governance
single, dominant oversight
multiple, overlapping stakeholders
Boundaries
well-defined
continually changing as new systems become
part of SoS, while others leave
Membership
fixed; parent-offspring
dynamic; based on SoS capability requirements
Information Flows
well-understood, mostly
internal to the system
changing; based on changes in information
sharing requirements
(Madni, 2006)
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/8
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
System Versus SoS
Comparison Criteria
System
(cont’d)
System-of-System (SoS)
Development
COTS or custom-developed
systems procured from/developed by 3rd party
under control of system authority organizations; some COTS
Complexity
designed out
mostly ignored; imposed by dynamic mix of
systems, emergent behavior
Autonomy
ceded by the components/parts
to the system
retained by member systems which contribute
to SoS
Unintended
Consequences
negligible; foreseen/tested and
designed out
high likelihood; changing boundaries;
emergence
Biggest Challenge
satisfaction of all functional and
performance requirements
subject to cost, schedule and
“ility” constraints
interoperability at the programmatic,
constructive and operational levels
(Madni, 2006)
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/9
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
An Apt Analogy
City
SoS
• Collective action of multiple individuals acting
locally over time
• City emerges and changes over time through
loosely coordinated and regulated action of
multiple individuals
• Exhibits coherence without central control
-- mechanisms that regulate local action
• Mechanisms include city regulations,
communications, distribution, emergency services
• Built gradually in parts by people, companies,
communities to serve their own purpose
• Grows and thrives based on cultural and economic
necessities
• City is always in a state of perpetual changes
• Collective action of multiple entities acting locally
over time
• SoS behavior emerges and changes over time
through loosely coordinated and regulated action
of multiple systems
• Exhibits coherence without central control
-- mechanisms to regulate local action
• Mechanisms include acquisition policies,
communications, distributed governances
• Built and introduced gradually by organizations to
satisfy mission capability packages
• Grows and is sustained based on mission
requirements
• SoS has dynamically changing boundaries
(construction, repairs, mods, demolitions, operations)
(systems enter, leave, are modified, while operating)
Adapted from SEI ULS report, 2006
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/10
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
SoS Realities
• Different missions require different Mission Capability Packages
(MCP)
• Different capability packages require different aggregations of
systems, with the potential for some overlap
• Interoperability is key to realizing the desired capability
– syntactic interoperability assures ability to exchange data
– semantic interoperability assures ability to use and understand data
• Mission Capability Package can vary with locale and type of
threat
– urban environment versus jungle environment versus desert
environment
– type of threat (e.g., NBC, IED,…)
• Operational documents not in step with current understanding
• Acquisition process not in step with SoS interoperability
requirements
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/11
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
SoS Architecture
• A framework for structuring nodes that define the SoS
– regardless of their source (i.e., COTS, NDI, custom, legacy)
• Captures scope of the nodes, and how they are intended to
interact within the SoS
• A vehicle for:
–
–
–
–
incorporating existing/planned nodes into the SoS
performing tradeoffs and making decisions
communicating and discussing issues
documenting SoS state/status
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/12
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
SoS Architecting
Complexities
• SoSs are dynamic entities
– systems are added, modified, or removed as mission requirements
change
– such changes are handled a priori or “on-the-fly”
• The integration infrastructure needs to change with
– technological advances
– changes in Quality-of-Service (QoS) requirements
• These considerations imply that a SoS architecture needs to
enable the evolution of the SoS and be evolvable itself
• Evolvability requires up front engineering because it is costly
and difficult to introduce later
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/13
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
SoS Architecting
Challenges
• Overcoming development friction
– complex, independent-overlapping governance
• Getting all stakeholders to develop shared interests
– multiple, independent concurrent developments, overlapping
governances
• Maintaining coherence at SoS level
– independently planned programs pulling in different directions
• Achieving robustness despite inability to perform critical
tradeoffs
– complexity renders certain tradeoffs incalculable
• Maintaining interoperability (syntactic, semantic)
– dynamically changing, uncertain information sharing
requirements
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/14
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
SoS Architecting Myopia
• Focus solely on technical issues at the expense of operational
mission goals
– evident in the rush to migrate to service-oriented architectures (SoA)
• Ignore critical issues such as:
– SoS architecture coverage relative to SoS mission requirements
– likely changes to mission and SoS ability to handle these changes
– mission-imposed QoS requirements and implications for SoS
architecture
– most likely and most challenging operational scenarios and the
ability of the SoS architectures to support them
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/15
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
SoS Architecture
Tradeoffs (examples)
• Mission Coverage vs. Lifecycle Costs
– how well does the SoS architecture satisfy mission requirements and at
what cost (lifecycle, recurring)
• Security vs. Performance
– how does the SoS address security when dealing with heterogeneous,
remote nodes
– do these measures come at the expense of reduced performance
• Functionality vs. QoS
– granularity of node interface
– risk of extraneous data or multiple trips to acquire data
• Standard vs. Proprietary Communication Mechanisms
– do standard mechanisms adequately satisfy QoS requirements
– do proprietary mechanisms preclude new nodes from joining the SoS
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/16
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
Promise of Executable
Representations
• SoS architecture and design will necessarily evolve
iteratively to respond to critical capability requirements
• Executable representations of the architecture
produced/refined in each iteration can potentially:
– clarify understanding and reduce development risks
– provide key early insights into the behavior of the target SoS
– explore sensitivities of SoS attributes to changes in mission
requirements (“Mission Capability Packages”)
– validate operational scenarios using “what-if” perturbations
– verify technical realizability of the architecture
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/17
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
SoS Architecture
Implementation Options
• The three major implementation options are:
– Service-oriented Architecture
– Grid Architecture
– Component-based Architecture
• Each has its advantages and disadvantages
• Technological maturity to implement each tends to vary
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/18
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
Challenges Ahead
• No guidance on how to architect a SoS
– especially when goal(s) need to be created “on the fly”
• Creating a SoS on the fly involves dynamic discovery, composition, and
invocation
– therefore, the architecture may not be known until run-time…risky!
• Open questions
– how to determine architecture of such a SoS? is it only determined at run time
or is there sufficient static information to determine the architecture a priori?
– what is the potential impact of not knowing the architecture on the quality
attributes and performance of the SoS? is it possible to evaluate the desired
quality attributes of such a system?
– is it possible to guarantee some level of service for a SoS under development?
– how can a SoS that is put together at run-time distinguish between legitimate
and illegitimate service? … can this knowledge be used to guarantee an
acceptable QoS level?
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/19
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
Critical Success Factors
• Do as much up-front engineering as possible
– evolvability is costly and difficult to infuse later
• Avoid “SoS architecture myopia”
– attend to mission capability requirements
– map them onto SoS architecture nodes to assess coverage, and
to identify and fill gaps
• Focus on mostly likely and most challenging operational
scenarios in defining architectural requirements
• Define fundamental vocabulary, semantics, and models to
build and share best practices
• Employ iterative process in SoS architecting to increase
understanding and reduce risks
• Experiment with “guided” emergence by creating conditions
and policies that result in desired SoS capabilities
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/20
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.
Critical Success Factors
(cont’d)
• Employ executable representations of the architecture at the
appropriate levels of abstraction in each iteration to:
–
–
–
–
–
clarify understanding and reduce risks in each iteration
acquire early initial insights into the behavior of the target SoS
explore sensitivities of critical SoS attributes to perturbations
validate operational scenarios
verify technical realizability of the architecture
• Simulate, analyze and resolve human-system integration issues
arising from:
– spikes in cognitive load
– frequent context-switching on the part of humans
• Aid/Train humans to effectively function in scenarios where:
– changes in collaborators and collaboration perspectives occur
frequently
– context-switching occurs periodically
– cognitive load spikes suddenly
Copyright © 2006 Intelligent Systems Technology, Inc.
Madni/21
Information in this document is the property of Intelligent Systems Technology, Inc. Disclosure is made in confidence.
Unless otherwise permitted, use or further disclosure of the depicted information by persons outside Intelligent Systems Technology, Inc. is prohibited.