SoS Process, Acquisition, Management Critical
Success Factors Workshop
Jo Ann Lane, jolane@usc.edu
Richard Turner, turner@software.org
USC Center for Systems & Software Engineering
http://csse.usc.edu
October 26, 2006
Workshop Guidelines
 Product: briefing, preferably with notes
 Topics should include:
– Most critical success factors in each area
– Current best practices for addressing them
– Areas for further research
 Rated 0-10 on value and difficulty of
research
2
SoS Process Critical Success Factors: Technical
 Technical
– Complexity management – including adequate and
appropriate recording for ver/val methods
– Scope as a variable and negotiable
 Identify “good enough” early
– Design for graceful degradation (architecture...move)
– Plan and budget for evolution
– Engineering processes that focus on
capabilities/outcomes rather than requirements
(research opportunity)
– Avoidance of “False precision” - honesty about
variance
– Ensure that decisions support SoS, not just component
system
– Effective/efficient baseline management of SoS
3
SoS Process Critical Success Factors: Technical
(continued)
 Technical
– Limit negative impact of changes to baselines out of
locus of control (architecture)
 Standards, guidelines, conventions – prevent lack
of compatability
– Budget and Plan for Continuous integration and
degrees of continuous V&V
– Proactive future interface management (goes along
with requirements management)
– C3ISR approach
 Recon at boundaries
 Awareness of environmental changes
 Technical ISR of competition
4
SoS Process Critical Success Factors: Management
 Management
– Plan for evolution
– Deliver capability quickly (plan for)
– Continuous course correction
 Steering verses plan-and-track leadership style
– Managing velocity and acceleration of technical work
 Rather than position (tracks instead of dots)
– Identify “good enough” early
– Avoidance of “False precision” - honesty about variance
– Continuously “Right size” governance model
– Effective/efficient baseline management of SoS
– How to identify impact of proposed changes
 Understanding business process re-engineering
 Initial and revised/future
5
SoS Process Critical Success Factors: Management
(continued)
 Management
– Ability to plan for unintended consequences
 How to handle contingency
 Add margin/empty spiral development cycles
 Accept that not everything can be pre-planned and not fill
schedule with known-knowns
– Business ISR of competition
– Metrics and measurement processes that support SoS objectives
 Avoid too much detail or sub-optimization
– Progress measured by operational capabilities
– Managing compound risks
 Compound risk management and mitigation
 A “Risk of Risks”
6
SoS Process Critical Success Factors:
People/Organizational
 People
– SoS Stakeholder processes that
 Actively Balance Stakeholders
 Maintain Stakeholder Involvement and Consensus
 Identify, establish, and involve community
 Identify the right stakeholders for each decision
– How to do without CRACK representatives from all
systems and stakeholders (use surrogate ones?)
 Processes that still operate with lacking or
inadequate levels of representation
– Communicating and coordinating up and down
supplier chain (robust and speedy)
– Incentive structures
7
SoS Process Critical Success Factors: Acquisition and
Point of View
 Acquisition
– Continuously “Right size” governance model
– Scope as a variable and negotiable
– Plan for evolution
– Appropriate contracting vehicles
 Point of View
– Need to differentiate issues/process needs between a
system of systems and addressing a collection of
component systems
– In SoS context, software properties may need to be
recognized as more important than the hardware
components
– Acknowledge that process is a learning process (goes
with scoping, governance models)
8
Best Practices
 Technical
– Architecture reviews of trade space results
– WinWin to prioritize requirements
– Spiral incremental commitment model to plan and
budget for evolution [Two types of
evolution(forseeable evolution, unanticipated
changes)]
– GQM or PSM
– Baseline management automation
– Continuous Integration
9
Best Practices (continued)



Management
– Incremental Planning
– Independent expert reviews
– Add margin/empty spiral development cycles
– C3ISR approach
 Recon at boundaries
 Awareness of environmental changes
 ISR of competition
 Test Driven Development / Scenario Based Testing
– Risk identification templates
People
– IPPD
– Use surrogate representatives
– Intranet portal technology
– Collaboration tools
– Incentive templates
Acquisition
– Re-evaluation at the end of each “increment” (include in retrospective)
10
SoS Process Critical Success Factors:
Areas for further research
A. Engineering processes that focus on capabilities/outcomes rather
than requirements
B. How to fix acquisition process – current process not used right
C. Risk identification templates that can succeed in a SoS context
D. How to do without CRACK representatives from all systems and
stakeholders
E. Meta analysis of acquisition literature
F. Methods for managing steering based approach
G. Incentive structures
H. Rapid change management techniques (acquisition and technical
POV)
I. Tools for impact analysis
J. Fault tree /FEMA analysis to determine probability of success for
complex programs
K. Modelling and simulation for SoS programs
L. Cost and schedule estimation tools
M. Identification and evaluation in complex SoS of best practices in this
environment
Rate 0-10 on value and difficulty of research…
11
Research Priorities
Opp A
Research Opportunities
Opp B
Value of Research
Opp C
Opp D
10
Opp E
Opp F
Opp G
5
Opp H
Opp I
Opp J
Opp K
Opp L
0
0
5
10
Difficulty of Research
Opp M
12
Backup Charts/Additional Information
Success-Related Factors from Presentations
 Boehm: Challenges of SoS
– Complexity
– Dynamism
– Emergence
– Build-to-spec processes, contracts infeasible
 Dalrymple: Success Critical Factors for SoS
– Baseline Management (processes)
– Organizational Efficiency
– Concurrency Management (processes)
– Feasibility Assessment
 Latimer:
– Need case studies to support theory and process dev.
Success-Related Factors from Presentations (continued)
 Nielsen: SoS Issues
– Constituents (a.k.a. stakeholders)
– Purpose
– Control
– Requirements
– Ownership
– Boundaries
– Visibility
 Pyster: Critical Success Factors
– Team organization (integrated single team)
– Communications
– Standards Compliance
– Simulation and Layered Architectures
Success-Related Factors from Presentations (continued)
 Royce: Critical Success Factors (four patterns of success)
– Scope management
– Process management
– Progress management
– Quality management
 Turner: Taglines
– “Pay me now or pay me later”
– “What’s my line?”
– “Who’s on first?”
– “Truth or Consequences?”
– “What, me worry?”
Definitions
 SoS Process
– Risk management
 Identify risks process is different?
 Mitigation
– Needs to deal with emergent stuff
– SOS process needs to look like process we preach;
ability to change is important
– Don't know requirements of emergent systems
 Acquisition
 Management
17