Systems of Systems:
Characteristics and Challenges
Barry Boehm, boehm@usc.edu
USC Center for Systems & Software Engineering
http://csse.usc.edu
October 25, 2006
Overview
 Definitions, Examples & Motivation
 SoS Characteristics and Challenges
 Conclusions
2
What is a “System-of-Systems”?
 Very large systems developed by creating a framework or
architecture to integrate component systems
 SoS component systems independently developed and managed
– New or existing systems
 Some outsourced
 Some externally evolving
– Have their own purpose
– Can dynamically come and go from SoS
 SoS exhibits emergent behavior not otherwise achievable by
component systems
 SoS activities often planned and coordinated by a Lead System
Integrator (LSI)
 Typical domains
– Business: Enterprise-wide and cross-enterprise integration to support
core business enterprise operations across functional and geographical
areas
– Military: Dynamic communications infrastructure to support operations
in a constantly changing, sometimes adversarial, environment
3
What Does an SOS Look Like:
Future Combat Systems
4
The Need for Net-Centric Systems of Systems (NCSOS)
 Lack of integration among stove-piped systems causes
– Unacceptable delays in service
– Uncoordinated and conflicting plans
– Ineffective or dangerous decisions
– Inability to cope with fast-moving events
 Increasing NCSOS benefits
– See first; understand first; act first
– Network-centric operations coordination
– Transformation of business/mission potential
– Interoperability via Integrated Enterprise Architectures
6
Overview
 Definitions, Examples & Motivation
 SoS Characteristics and Challenges
 Conclusions
7
SoS Characteristics and Challenges - I
 Complexity: Size and number of organizations,
interfaces, suppliers, coordination groups
– Scalability of processes, methods, and tools
 Dynamism: Number of changes/month, average time to
process change
– Rapid change impact analysis, change synthesis,
negotiation, development, validation, implementation
 Emergence: requirements not pre-specifiable
 Build-to-spec processes, contracts infeasible
– C2ISR a better metaphor for SoS acquisition than
purchasing-agent
– Command, control, intelligence, surveillance,
reconnaissance
8
Complexity of SoS Solution Spaces
 Size: 10-100 MLOC
 Number of external interfaces: 30-300
 Number of “Coopetitive” suppliers: 20-200
– Even more separate work locations
 Depth of supplier hierarchy: 6-12 levels
 Number of coordination groups: 20-200
– Reviews, changes, risks, requirements, architecture, standards,
procedures, technologies, -ilities, integration, test, deployment,
personnel, infrastructure, COTS,…
– Key personnel spend 60 hours/week in meetings
 Unprecedentedness
 Emergence
 Rapid change
Necessarily software-intensive…
9
Average Change Processing Time: 2 SoSs
 Average workdays to process
changes
160
140
120
100
80
60
40
20
0
Within
Groups
Across Contract
Groups
Mods
10
Acquisition C2ISR Via Spiral OODA Loops
Observe new/updated
Example:
objectives, constraints, alternatives
ARPANet/Internet
Spiral
• Usage monitoring
• Competition, technology,
marketplace ISR
Orient with respect to
stakeholders priorities, feasibility,
risks
• Risk/Opportunity analysis
• Business case/mission analysis
• Prototypes, models, simulations
Operate as current system
Accept new system
Act on plans, specifications
Decide on next-cycle
capabilities, architecture
upgrades, plans
• Keep development stabilized
• Change impact analysis,
preparation for next cycle (miniOODA loop)
• Stable specifications, COTS
upgrades
• Development, integration, V&V,
risk management plans
• Feasibility rationale
Life Cycle Architecture Milestone for Cycle
11
SoS Characteristics and Challenges - II
 COTS/Reuse/Legacy diversity
– Many sources of incompatibility, changes
– COTS: average 8-10mo/release; unsupported after 3 releases
 Multiple missions and stakeholders to support
– Increment and change content requires negotiation
 Independently evolving systems
– Often with “coopetitive” suppliers, interoperators
 More time needed for systems definition
– Before and after source selection
 More time needed for teambuilding, partner coordination, supplier
management, change management , integration and test
12
How much Architecting is Enough: COCOMO II Analysis
100
Percent of Time Added to Overall Schedule
90
10000
KSLOC
80
Percent of Project Schedule Devoted to
Initial Architecture and Risk Resolution
70
Added Schedule Devoted to Rework
(COCOMO II RESL factor)
Total % Added Schedule
60
Sweet Spot
50
40
100 KSLOC
30
Sweet Spot Drivers:
20
Rapid Change: leftward
10 KSLOC
High Assurance: rightward
10
0
0
10
20
30
40
50
60
Percent of Time Added for Architecture and Risk Resolution
13
COSOSIMO: I&T Sub-Model
Size Drivers
• # SoS interface protocols
• # SoS scenarios
• # unique component systems
Cost Drivers
•
•
•
•
•
•
•
•
•
Requirements understanding
Architecture maturity
Level of service requirements
SoS team capability
Maturity of LSI processes
Tool support
Cost/schedule compatibility
SoS risk resolution
Component system maturity and
stability
• Component system readiness
SoS
Integration
and Testing
LSI I&T
Effort
14
Conclusions
 Individual SoS attributes are highly challenging
– Complexity, dynamism, emergence, uncontrollables,
stakeholder diversity
– Their combinations are even more challenging
 Acquisition management and negotiation skills are at
least as important as systems engineering skills
– C2ISR a better metaphor for SoS acquisition than
purchasing-agent
 More time needed for systems definition
– Before and after source selection
15
Backup Charts
Complexity of Solution Spaces
- Breadth, Depth, and Length
Platform N
•
Width
•
•
Platform 1
Infra
C4ISR
1.0
2008
2.0
3.0
4.0
5.0
2010
2012
2014
2016
Command and Control
Situation Assessment
Info Fusion
Sensor Data Management
Sensor Data Integration
Sensors
Sensor Components
:
Depth
…
Length
Legend:
DOTMLPF
C4ISR
Doctrine, Organization,
Training, Materiel, Leadership,
Personnel, Facilities
Command, Control,
Communications, Computers,
Intelligence, Surveillance,
and Reconnaissance
17
Top-10 Risks: Software-Intensive Systems of Systems - CrossTalk,
May 2004
1. Acquisition management and staffing
2. Requirements/architecture feasibility
3. Achievable software schedules
4. Supplier integration
5. Adaptation to rapid change
6. Quality factor achievability and tradeoffs
7. Product integration and electronic upgrade
8. Software COTS and reuse feasibility
9. External interoperability
10. Technology readiness
18
Effect of Unvalidated Requirements
-15 Month Architecture Rework Delay
$100M
Arch. A:
Custom
many cache processors
$50M
Arch. B:
Modified
Client-Server
Available budget
Original Spec
1
After Prototyping
2
3
4
5
Response Time (sec)
19
Effect of Unvalidated Software Schedules
 Original goal: 18,000 KSLOC in 7 years
– Initial COCOMO II, SEER runs showed infeasibility
– Estimated development schedule in months for
closely coupled SW with size measured in
equivalent KSLOC (thousands of source lines of
code):
– Months =~ 5 * 3√KSLOC
- KSLOC
300
1000
3000
10,000
- Months
33
50
72
108
•Solution approach: architect for decoupled parallel development;
Schedule As Independent Variable (SAIV) process
20
The SAIV* Process Model
1. Shared vision and expectations management
2. Feature prioritization
3. Schedule range estimation and core-capability determination
 Top-priority features achievable within fixed schedule with 90%
confidence
4. Architecting for ease of adding or dropping borderline-priority
features
 And for accommodating past-IOC directions of growth
5. Incremental development
 Core capability as increment 1
6. Change and progress monitoring and control
 Add or drop borderline-priority features to meet schedule
*Schedule As Independent Variable; Feature set as dependent variable. Also works for cost,
schedule/cost/quality as independent variable.
21
Top-10 Risks: Software-Intensive Systems of Systems CrossTalk, May 2004
1. Acquisition management and staffing
2. Requirements/architecture feasibility
3. Achievable software schedules
4. Supplier integration
5. Adaptation to rapid change
6. Quality factor achievability and tradeoffs
7. Product integration and electronic upgrade
8. Software COTS and reuse feasibility
9. External interoperability
10. Technology readiness
22
COTS Upgrade
Synchronization and Obsolescence






Many subcontractors means an increasing number of evolving COTS
interfaces
Aggressively-bid subcontracts can deliver obsolete COTS
– New COTS released every 8-9 months (GSAW)
– COTS unsupported after 3 releases (GSAW)
– An actual delivery: 120 COTS; 46% unsupported
Emphasize COTS interoperability in source selection process
Develop contract/subcontract provisions/incentives to ensure
– Consistency and interoperability across contractor and subcontractordelivered COTS software products
– Such products are recent-release versions
Develop a management tracking scheme for all COTS software products in
all NCSOS software elements
Develop a strategy for synchronizing COTS upgrades
23
Emerging Scalable Spiral Process
- For 21st century systems engineering and acquisition
 The C4ISR Metaphor for NCSOS Acquisition
– Role of OODA loops
– Example: Internet Spiral
– Example: FCS Win Win Spiral Model
 Risk-Driven Scalable Spiral Model
– Increment view
– Life-cycle view
– Role of anchor point milestones
 Acquisition management implications
 People management implications
24
Risk-Driven Scalable Spiral Model:
Increment View
Rapid
Change
Short
Development
Increments
Foreseeable
Change (Plan)
Increment N Baseline
Short, Stabilized
Development
of Increment N
Increment N Transition/O&M
Stable Development
Increments
High
Assurance
25
Risk-Driven Scalable Spiral Model:
Increment View
Unforseeable Change
(Adapt)
Rapid
Change
Short
Development
Increments
Agile
Future Increment Baselines
Rebaselining for
Future Increments
Deferrals
Foreseeable
Change (Plan)
Increment N Baseline
Stable Development
Increments
Current V&V
High
Assurance
Resources
Short, Stabilized
Development
of Increment N
Artifacts
Increment N Transition/O&M
Concerns
V&V
of Increment N
Future V&V
Resources
Continuous V&V
26
Risk-Driven Scalable Spiral Model:
Life Cycle View
System LCA
System
Inception
System, DI1 LCA
System
Elaboration
DI2 B/L LCA
Changes
Agile DI2 (OO&D)
Rebaselining
Plan-Driven DI1 Construction
(A)
DI1 V&V
LCA: Life Cycle Architecture
IOC: Initial Operational Capability
OO&D: Observe, Orient and Decide
V&V: Verification and Validation
DI:
Development Increment
B/L:
Baselined
DI2 LCA
Plan-Driven DI2 Construction
(A)
DI2 V&V
27
Risk-Driven Scalable Spiral Model:
Life Cycle View
System LCA System, DI1 LCA
System
Inception
DI2 B/L LCA
DI3 B/L LCA
DI4 B/L LCA
Changes
System
Elaboration
Agile DI2 (OO&D)
Rebaselining
Plan-Driven DI1
Construction (A)
DI1 V&V
Changes
Update
Update
DI1 IOC
DI1
Trans’n
DI1
Usage
DI2 LCA
Agile DI3 (OO&D)
Rebaselining
Plan-Driven DI2
Construction (A)
DI2 V&V
Changes
Update
DI2 IOC
DI2
Trans’n
DI2
Usage
DI3 LCA
Agile DI4 (OO&D)
Rebaselining
LCA: Life Cycle Architecture
IOC: Initial Operational Capability
OO&D: Observe, Orient and Decide
V&V: Verification and Validation
DI:
Development Increment
B/L:
Baselined
Plan-Driven DI3
Construction (A)
DI3 V&V
DI3 IOC
DI3
Trans’n
DI3
Usage . . .
DI4 LCA
...
28
LCO (MS A) and LCA (MS B) Anchor Points
Pass/Fail Criteria
 A system built to the given architecture will
– Support the operational concept
– Satisfy the requirements
– Be faithful to the prototype(s)
– Be buildable within the budgets and schedules in the
plan
– Show a viable business case
– Establish key stakeholders’ commitment to proceed
LCO: True for at least one architecture
LCA: True for the specific life cycle architecture;
All major risks resolved or covered by a risk management plan
29
Spiral Feasibility Rationale Deliverable
 LCO, LCA reviews not just UML/PowerPoint charts
 Need to show evidence of product and process feasibility
 Evidence provided by prototypes, production code, benchmarks,
models, simulations, analysis
– Sizing and cost/schedule model results for process feasibility
 Evidence provided in advance to LCO/LCA review team
– Key stakeholders, specialty experts
 Lack of evidence risks destabilizing the process
– Needs coverage by viable risk mitigation plan
 Key new progress metric
– Feasibility evidence progress vs. plans
30
Acronym Definitions
B/L
C4ISR
Baselined
Command, Control, Communications,
Computers, Intelligence, Surveillance, and
Reconnaissance
CCD
Core Capability Drive-Through
COTS
Commercial Off the Shelf
CRACK
Collaborative, Representative, Authorized,
Committed, Knowledgeable
DI
Development Increment
DODAF
Department of Defense Architectural
Framework
DOTMLPF Doctrine, Organization, Training, Materiel,
Leadership, Personnel, Facilities
ERP
Enterprise Resource Planning
FEAF
Federal Enterprise Architectural
Framework
GSAW
Ground System Architectures Workshop
IESG
Internet Engineering Steering Group
IETF
Internet Engineering Task Force
IKIWISI
I’ll Know It When I See It
IOC
Initial Operational Capability
IPT
Integrated Product Team
IRR
Inception Readiness Review
LCA
LCO
LSI
MLOC
MS
NCSOS
OO&D
OODA
PM
PRR
SAIV
SE
SoS
SOW
SW
Sys Engr
V&V
WMI
Life Cycle Architecture
Life Cycle Objectives
Lead System Integrator
Million Lines of Code
Milestone
Net-Centric System of Systems
Observe, Orient, and Decide
Observe, Orient, Decide, Act
Person-Month/Program Manager
Product Release Review
Schedule As Independent Variable
System Engineering
System of Systems
Statement of Work
Software
System Engineering
Verification and Validation
War-fighter machine interface
31