Critical Success Factors in
Systems of Systems
Engineering
Arthur Pyster
Senior VP and Director of Systems Engineering and
Integration,
SAIC
Pat Gardner
Chief Technology Officer,
Tactical Systems & Solutions Business Unit
October 25, 2006
1
Objectives

Clarify distinction between a system and a system of
systems (SOS)

Discuss lessons learned from hands-on systems of
systems engineering (SOSE) practice

Identify and explain success factors

Warn against some common remedies that do not work
© SAIC, 2006
2
Many Definitions of SOS
“…a set or arrangement of interdependent systems that are
related or connected to provide a given capability. The loss of any
part of the system will significantly degrade the
performance or capabilities of the whole. The development of
a SOS solution will involve trade space between the systems
as well as within an individual system performance. An
example of a SOS would be a combat aircraft. While the aircraft may
be developed as a single system, it could incorporate subsystems
developed for other aircraft. For example, the radar from an existing
aircraft may be incorporated into the one being developed rather than
developing a new radar. The SOS in this case would be the airframe,
engines, radar, avionics, etc. that make up the entire combat aircraft
capability.” [Joint Capability and Integration Development System, 2005]
© SAIC, 2006
3
Many Definitions of SOS (continued)
“… an integrated force package of interoperable systems acting as a single
system to achieve a mission capability. Typical characteristics include a high
degree of collaboration and coordination, flexible addition or removal of
component systems, and a net-centric architecture…” [Naval “Systems of
Systems” Systems Engineering Guidebook, 2005]
“… a set or arrangement of systems that results when independent and
useful systems are integrated into a larger system that delivers unique
capabilities…” [Systems of Systems Engineering Guide (October 6, 2006 draft)]
“Systems of systems exist when there is a presence of a majority of the
following five characteristics: operational and managerial independence,
geographical distribution, emergent behavior, and evolutionary
development.” [Sage and Cuppan 2001]
© SAIC, 2006
4
U.S. National Airspace System as an SOS








Air space management
and safety achieved by
all component systems
working together
Individual systems
provide useful services
such as navigation to a
pilot
Individual systems are
acquired independently
with different contractors
Individual systems come
and go routinely
Individual systems are
operated by FAA,
airports, airlines, NOAA,
…
Highly network centric
Standard protocols and
interfaces abound
Geographically
dispersed



Overarching CONOPS and architecture provide for
evolutionary development
Despite extensive modeling, system complexity leads to
emergent behavior
Extensive coordination is central to achieving high levels of
efficiency and safety
© SAIC, 2006
5
Examples




The Boeing 777 is not an SOS. It is a large complex product line
system.
The U.S. National Airspace System, which includes fleets of Boeing
777s, is an SOS.
A single UAV is not an SOS. It is a member of a product line of complex
vehicles.
FCS, which includes UAVs, is an SOS.

Neither the NAS, nor any civilian aviation system, was designed
anticipating UAVs, but they are now being added. Individual system
types come and go. Individual system instances come and go.

System boundaries are often flexible and fuzzy – not what we like in
systems engineering

Emergent behavior – 9/11 changed the CONOPS for the NAS and how
we handle flight security. Example evolutionary implication: primary
radars were on the way out. No longer.
© SAIC, 2006
6
Systems of Systems Engineering
Army
USAF
FAA
SOS Systems Engineering designs and integrates programs in which:

The SOS has Concepts of Operation that are a superset of the concepts
of the individual member systems
 The behavior of the SOS is characterized by
NII/DISA
Navy
USAF
JFCOM
FAA




NASA
UK
Army
Acquisition
FAA

All
A high degree of functional collaboration among the member systems
A focus on information dominance that cannot be achieved by a single system
acting alone (e.g., through the use of enterprise service-oriented architectures)
Member systems may enter or leave (and return) while SOS operations persist
SOS acquisition has objectives that cannot be achieved by individual
procurements



A family of related systems with interlocking requirements and missions
Commonality and modularity objectives for design
SOS managed risk and design-to-cost for a group of major systems
The first-tier SOS subcontractors are prime contractors for major systems


Integrated End Items may have complete uses outside the System of Systems
End Items have unique requirements that do not apply to the System of
Systems as a whole
© SAIC, 2006
7
SAIC FCS SOSE Experience




(SAIC Prime) SAIC developed and exercised the SOSE methodology
for FCS

SOSE “Double Vee” process to engineer the system

SOS architecture for network centric warfare
(LSI with Boeing) SAIC engineers lead the Architecture Development
team

Full Enterprise and HW/SW layered architecture

NII and J6 vetted and approved methodology
(LSI) SAIC engineers provide SE&I management leadership

Managing Prime Item engineering and development

Leading vehicle/platform integration
(LSI) SAIC engineers lead the Integrated Simulation and Test team

Distributed HW/SW-in-the-loop simulation and System Integration
Laboratory environment

Direct linkages: analysis – simulation – SIL – full scale test
© SAIC, 2006
8
SOSE Process Overview
This is a particular process
example developed for an SOS
pursuit in Europe. It applies well
to many classes of problems in
SOS engineering.
Prime Contractors
Many different “SOSE process” descriptions are possible. All
are adaptations of the SE “Vee” model.
© SAIC, 2006
9
Team Organization is Crucial


Coordinating multiple SE teams through documentation, reporting, and
ICD’s does not work effectively
One systems engineering team

Integrate a distributed team using Top Down SE Management, integrated
tasking, and clear accountability
 Use common networked tools and processes that ensure communication of
information and rapid issue resolution
 Team must deal with all levels of SE issues from the End Item through
subsystems (layered architecture)

Integrate and train the SE Team

Train to modern process standards – legacy won’t work for modern,
software-intensive SOS that evolve over time


Must use up-to-date software architectures and tools
System Engineering Management is the integrating factor

Synchronize the team
 Integrate the team vertically and horizontally
© SAIC, 2006
10
Communications

Canons of systems engineering

Make sure everyone knows what the problem is
 Make sure everyone is solving the same problem
 When something changes, make sure everyone knows about it

In a complex, layered SOS, “design to spec and wait to verify” is
insufficient


Critical changes and design trades will be made after specifications
have been cut and contractors selected
Proactive communications are required on a frequent basis



Within the integration team
Between the team and the stakeholders
Among the execution team (integrators and suppliers)

Use formal methods; e.g., reviews, and technical interchange meetings
 Use innovative methods
Managing communications is the job of the System of
Systems Lead Integrator
© SAIC, 2006
11
Standards Compliance is an Asset

Proprietary, outdated, noncompliant processes ultimately compromise
team integration


Challenge: organizations with different (or nonexistent) processes
System engineering standards handle modern, complex problems
ISO-15288 – Product life cycle synchronization
 ANSI/IEC-632 – System Engineering and the Enterprise
 IEEE-1471 – Architecture development using multiple frameworks (eg.
Tailored for multimission SOS, HWCI, CSCI, information systems)




Standards lead to predictable outcomes that reduce risk
Compliant processes integrate team operations


Incorporate DODAF, FEAF, Zachman, and SW frameworks
Common expectations of outcomes and validation standards
Compliance requires vertically integrated SE management, diligence,
and verification

Core responsibility of the SOS integrator
© SAIC, 2006
12
System of Systems Life Cycle
The System of Systems life cycle is different from (and encapsulates) the life
cycles of the development End Items
Continuous Verification and Validation
Control the SOS development
with a structured set of control
gates recorded in SOS IMP/IMS
Manage the System of Systems
deployment by synchronizing the life
cycle events of the constituent systems
© SAIC, 2006
13
SOSE Management Tasks
The System of Systems
Integrator specifies and
controls the life cycle
support systems
Requirements and
Specifications
Plans and
Constraints
Prime Contractors for System
Developments
Designs and Progress Data
Schedules and Events
© SAIC, 2006
14
Role of Simulation in SOS Engineering
Modeling &
Simulation
Increasing detail
System
Integration
Laboratory
Continuous Verification and Validation (Simulation
Based Acquisition)
SOS
Requirements
SOS
Logical
Design
SOS Block
(Integrate)
SOS
Functional
Design
System of Systems
Design
SOS Block
(Validate)
End Item
(Integrate)
SOS
Design
Synthesis
Prime Item
(Integrate)
Integrated
system
Integrated End
Items
System of Systems
Integration
Integrated Prime
Items
System
Development
System of Systems design and integration are complex
activities that can be effectively supported by
continuous V&V using Modeling and Simulation tools.
© SAIC, 2006
15
SOS Have Layered Architectures


Requirements Management, Configuration Management, and
Change Management quickly lose synchronization without a
vertically integrating top-down mechanism
System of Systems have a layered architecture that is the organizing
principle for system engineering management

System breakdown structure for platforms
 Architectures and flowdown functional analysis
 Requirements derivation and flowdown
 Mission and End Item integration

Manage configuration complexity by Architecture Level


Explicit links to functions, components and interfaces on own level
(modeled in the architecture) – horizontal integration
Explicit links “up” to functions, components and interfaces on the next
level – derivation and realization are explicit – vertical integration
Effective SOS management control depends critically on
organizing the complex elements of the program.
© SAIC, 2006
16
SOS Layered Architecture Structure
Stakeholders
SOSE management control using Architecture
Levels applies to core system engineering tasks:
Vertical Integration (each End Item)
Work Breakdown Structure
Product Breakdown Structure
HorizontalCommonality
Integration
and modularity design
Specification development
Architecture development
Horizontal Integration
Requirements management
Configuration management
HorizontalTechnology
Integration
assessment and insertion
System of
Systems
End Items
Common
Systems
Mission
Systems,
Subsystems
and
Components
Control the SOS structure using Architecture Levels
© SAIC, 2006
17
Checklist for SOSE Success







Do detailed planning and analysis based on knowledge of the
structure of the problem space (and adherence to the plan)
Continually validate the SOS CONOPS – it is guaranteed to be
wrong, keep making it better
Continually model, simulate, and apply other techniques at the SOS
level to understand complex behaviors, emergent behaviors,
requirements tradeoffs, and the impact of change
Concentrate on horizontal integration and synchronization – 80% of
the time design changes and decisions don’t matter, but the other
20% will kill you
Execute in a disciplined manner that identifies, measures, and
prevents problems
Design your organization to promote communications and integrated
execution – use collaboration tools to help
Train and execute to recognized best practices and standards
© SAIC, 2006
18
Finally

Systems of system engineering (SOSE) is not fundamentally different from the
systems engineering of large complex systems

The most complex (single) systems have emergent behavior, require extensive and
continuous modeling and simulation to be understood, have evolving requirements and
implementation, many subsystems with elaborate communications, …

Nevertheless, the scale of the largest SOS is daunting and challenges our processes,
techniques, and tools
© SAIC, 2006
19