University of Southern California
Center for Systems and Software Engineering
Incremental Commitment Model (ICM)
Guidebook Status, Plans, Key Issues
Barry Boehm and Jo Ann Lane, USC-CSSE
ICM Guidebook Workshop 3
October 29, 2008
Most charts include explanatory notes
University of Southern California
Center for Systems and Software Engineering
Outline
• Incremental Commitment Model (ICM) Overview
– ICM distinguishing features and implications
– Key ICM process views
• Proposed ICM Guidebook Organization
– Executive Summary: Key Features, Implications, Case Study
– Core Body: Phases and Activities; Case Study Illustrations
– Appendices: Detailed Practices; Common Special Cases
• Guidebook Status and Plans
• Key Issues
– Policies (5000.2, CP); Standards (IEEE-EIA, ISO)
• Conclusions; References; Acronyms; Backup charts
15 July 2008
©USC-CSSE
2
University of Southern California
Center for Systems and Software Engineering
ICM Distinguishing Features
• Not a one-size-fits-all model
– Risk-driven activity sequences; common special cases
• Incremental vs. total resource commitment
– System definition: compatible with competitive prototyping
– System development: timeboxed increments; concurrent
rebaselining; prioritized backlog
• Concurrent engineering
– Of objectives and solutions; product and process; development
and evolution; hardware, software, and human factors
– Synchronized and stabilized at major milestones
• Evidence, risk, and commitment-based milestones
– Feasibility evidence a first-class deliverable
15 July 2008
©USC-CSSE
3
University of Southern California
Center for Systems and Software Engineering
2008 SOW: DoD ICM Guidebook
• Develop a draft Guidebook for next-generation DoD
IS&SE based on the ICM
– In collaboration with DoD and industry participants
• Collaborate with selected DoD and industry parties in
experimental tailoring of draft Guidebook elements
– Would like to explore collaboration opportunities here
• Hold a series of Guidebook workshops
– Mar 19-20 (USC), July 15-17 (DC area), Oct 29-30 (USC)
• Coordinate Guidebook with other IS&SE initiatives
– Systems of systems engineering, systemic analysis,
acquisition, education, assessment, research and technology
Research is finding better ways… and concepts are being
evaluated and refined through OSD-sponsored workshops…
15 July 2008
©USC-CSSE
4
University of Southern California
Center for Systems and Software Engineering
DoD ICM Guidebook Concept
• Easily digestible top-level ICM guide
– Essentials of model and examples
• ICM guides for key new practices and areas
– Evidence-based anchor point commitment milestones
• Developing Feasibility Evidence Descriptions
– Using the ICM for systems of systems
– Using the ICM for competitive prototyping
– Using the ICM process decision table
• Detailed phase and activity guide
– Draft being adapted from 200-page predecessor
• Potential follow-ons
– Electronic Process Guide (some work underway with IBM)
– Courseware (lectures, learning modules)
15 July 2008
©USC-CSSE
5
University of Southern California
Center for Systems and Software Engineering
The Incremental Commitment Life Cycle Process: Overview
Stage I: Definition
Stage II: Development and Operations
Anchor Point
Milestones
Synchronize, stabilize concurrency via FEDs
Risk patterns
determine life
cycle process
15 July 2008
©USC-CSSE
6
University of Southern California
Center for Systems and Software Engineering
ICM HSI Levels of Activity for Complex Systems
15 July 2008
©USC-CSSE
7
University of Southern California
Center for Systems and Software Engineering
Anchor Point Feasibility Evidence Description
• Evidence provided by developer and validated by
independent experts that:
If the system is built to the specified architecture, it will
– Satisfy the requirements: capability, interfaces, level of service,
and evolution
– Support the operational concept
– Be buildable within the budgets and schedules in the plan
– Generate a viable return on investment
– Generate satisfactory outcomes for all of the success-critical
stakeholders
• All major risks resolved or covered by risk management
plans
• Serves as basis for stakeholders’ commitment to proceed
Can be used to strengthen current schedule- or event-based reviews
15 July 2008
©USC-CSSE
8
University of Southern California
Center for Systems and Software Engineering
ICM integrates strengths of current process models
But not their weaknesses
• V-Model: Emphasis on early verification and validation
– But not ease of sequential, single-increment interpretation
• Spiral Model: Risk-driven activity prioritization
– But not lack of well-defined in-process milestones
• RUP and MBASE: Concurrent engineering stabilized by
anchor point milestones
– But not software orientation
• Lean Development: Emphasis on value-adding activities
– But not repeatable manufacturing orientation
• Agile Methods: Adaptability to unexpected change
– But not software orientation, lack of scalability
15 July 2008
©USC-CSSE
9
University of Southern California
Center for Systems and Software Engineering
Process Model Principles
1.
Principles trump diagrams
Commitment and accountability
- Needed for high assurance systems
2.
Success-critical stakeholder satisficing
- Needed for multi-owner systems of systems
3.
Incremental growth of system definition and
stakeholder commitment
- Needed for emergent requirements, rapid change
4, 5. Concurrent, iterative system definition and
development cycles
- Needed for rapid change, rapid OODA loops
6.
Risk-based activity levels and anchor point
commitment milestones
Used
by 60-80% of CrossTalk
Top-5 projects, 2002-2005
15 July 2008
©USC-CSSE
10
University of Southern California
Center for Systems and Software Engineering
Incremental Commitment in
Gambling
• Total Commitment: Roulette
– Put your chips on a number
• E.g., a value of a key performance parameter
– Wait and see if you win or lose
• Incremental Commitment: Poker, Blackjack
– Put some chips in
– See your cards, some of others’ cards
– Decide whether, how much to commit to proceed
15 July 2008
©USC-CSSE
11
University of Southern California
Center for Systems and Software Engineering
Scalable remotely controlled
operations
15 July 2008
©USC-CSSE
12
University of Southern California
Center for Systems and Software Engineering
Total vs. Incremental Commitment – 4:1 RPV
• Total Commitment
–
–
–
–
–
Agent technology demo and PR: Can do 4:1 for $1B
Winning bidder: $800M; PDR in 120 days; 4:1 capability in 40 months
PDR: many outstanding risks, undefined interfaces
$800M, 40 months: “halfway” through integration and test
1:1 IOC after $3B, 80 months
• Incremental Commitment [number of competing teams]
– $25M, 6 mo. to VCR [4]: may beat 1:2 with agent technology, but not
4:1
– $75M, 8 mo. to ACR [3]: agent technology may do 1:1; some risks
– $225M, 10 mo. to DCR [2]: validated architecture, high-risk elements
– $675M, 18 mo. to IOC [1]: viable 1:1 capability
– 1:1 IOC after $1B, 42 months
15 July 2008
©USC-CSSE
13
University of Southern California
Center for Systems and Software Engineering
The Incremental Commitment Life Cycle Process: Overview
Stage I: Definition
Stage II: Development and Operations
Anchor Point
Milestones
Concurrently engr.
OpCon, rqts, arch,
plans, prototypes
14
July 2008
Concurrently engr.
Incr.N (ops), N+1
(devel), N+2 (arch)
©USC-CSSE
University of Southern California
Center for Systems and Software Engineering
Different Risk Patterns Yield Different Processes
15 July 2008
©USC-CSSE
15
University of Southern California
Center for Systems and Software Engineering
Common Risk-Driven Special Cases of the ICM
Special Case
Example
Size,
Complexity
Change Rate
%
/Month
Criticality
NDI Support
Org, Personnel
Capability
Complete
Key Stage I Activities : Incremental
Definition
Key Stage II Activities: Incremental
Development, Operations
Acquire NDI
Use NDI
Time per Build; per
Increment
1
Use NDI
Small Accounting
2
Agile
E-services
Low
1 – 30
Low-Med
Good;
in place
Agile-ready
Med-high
Skip Valuation , Architecting
phases
Scrum plus agile methods of
choice
<= 1 day;
2-6 weeks
3
Architected Agile
Business data
processing
Med
1 – 10
Med-High
Good;
most in place
Agile-ready
Med-high
Combine Valuation, Architecting
phases. Complete NDI
preparation
Architecture-based Scrum of
Scrums
2-4 weeks;
2-6 months
4
Formal Methods
Security kernel or
safety-critical LSI
chip
Low
0.3 – 1
Extra
high
None
Strong formal
methods
experience
Precise formal specification
Formally-based programming
language; formal verification
1-5 days;
1-4 weeks
5
HW component
with embedded
SW
Multi-sensor
control device
Low
0.3 – 1
Med-Very
High
Good;
In place
Experienced; medhigh
Concurrent HW/SW engineering.
CDR-level ICM DCR
IOC Development, LRIP, FRP.
Concurrent Version N+1
engineering
SW: 1-5 days;
Market-driven
6
Indivisible IOC
Complete vehicle
platform
Med –
High
0.3 – 1
High-Very
High
Some in place
Experienced; medhigh
Determine minimum-IOC likely,
conservative cost. Add deferrable
SW features as risk reserve
Drop deferrable features to
meet conservative cost.
Strong award fee for features
not dropped
SW: 2-6 weeks;
Platform: 6-18
months
7
NDI- Intensive
Supply Chain
Management
Med –
High
0.3 – 3
Med- Very
High
NDI-driven
architecture
NDI-experienced;
Med-high
Thorough NDI-suite life cycle
cost-benefit analysis, selection,
concurrent requirements/
architecture definition
Pro-active NDI evolution
influencing, NDI upgrade
synchronization
SW: 1-4 weeks;
System: 6-18
months
8
Hybrid agile /
plan-driven
system
C4ISR
Med –
Very High
Mixed parts:
1 – 10
Mixed
parts;
Med-Very
High
Mixed parts
Mixed parts
Full ICM; encapsulated agile in
high change, low-medium
criticality parts (Often HMI,
external interfaces)
Full ICM ,three-team
incremental development,
concurrent V&V, nextincrement rebaselining
1-2 months;
9-18 months
9
Multi-owner
system of
systems
Net-centric
military operations
Very High
Mixed parts:
1 – 10
Very High
Many NDIs;
some in place
Related
experience, medhigh
Full ICM; extensive multi-owner
team building, negotiation
Full ICM; large ongoing
system/software engineering
effort
2-4 months; 18-24
months
10
Family of
systems
Medical Device
Product Line
Med –
Very High
1–3
Med –
Very High
Some in place
Related
experience, med –
high
Full ICM; Full stakeholder
participation in product line
scoping. Strong business case
Full ICM. Extra resources for
first system, version control,
multi-stakeholder support
1-2 months; 9-18
months
C4ISR: Command, Control, Computing, Communications, Intelligence, Surveillance, Reconnaissance. CDR: Critical Design Review.
DCR: Development Commitment Review. FRP: Full-Rate Production. HMI: Human-Machine Interface. HW: Hard ware.
IOC: Initial Operational Capability. LRIP: Low-Rate Initial Production. NDI: Non-Development Item. SW: Software
15 July 2008
©USC-CSSE
16
University of Southern California
Center for Systems and Software Engineering
Outline
• Incremental Commitment Model (ICM) Overview
– ICM distinguishing features and implications
– Key ICM process views
• Proposed ICM Guidebook Organization
– Executive Summary: Key Features, Implications, Case Study
– Core Body: Phases and Activities; Case Study Illustrations
– Appendices: Detailed Practices; Common Special Cases
• Guidebook Status and Plans
• Key Issues
– Policies (5000.2, CP); Standards (IEEE-EIA, ISO)
• Conclusions; References; Acronyms; Backup charts
15 July 2008
©USC-CSSE
17
University of Southern California
Center for Systems and Software Engineering
Executive Summary Content
• Motivation and context
– Current DoD needs and future challenges
– How ICM addresses needs and challenges
• Key ICM distinguishing features and principles
• Relationships to DoD-relevant policies and standards
– 5000 series, competitive prototyping, ISO 15288/12207, others
• Structure and content of chapters
– Phase guidelines follow Firesmith metamodel
• Goals and Objectives; Entry Conditions; Inputs; Steps;
Exit Conditions; Work Products; Pitfalls
15 July 2008
©USC-CSSE
18
University of Southern California
Center for Systems and Software Engineering
Core Body Content
• Getting Started: The Exploration Phase
– Case study: remotely piloted vehicle control
– Common special cases
• Stage I: Valuation and Foundations Phases
– Illustrated by case study
– Artifact guidelines: Ops Concept, Requirements,
Architecture, Life Cycle Plan, Feasibility Evidence
• Stage II: Incremental Development and Operations
– Concurrent use of Increment N, development of Increment
N+1, rebaselining of Increment N+2
– Illustrated by case study
– Artifact guidelines: Plans for test phases, quality
management, implementation, life cycle support
15 July 2008
©USC-CSSE
19
Exploration
University of Southern California
Center for Systems and Software Engineering
Exploration Phase Content
Exploration
Goals & Objectives Preconditions
 Identify SCSHs
 Identify feasible
solution region
 Identify decision
table CommonCase or
alternative
process
 Evidence of
feasibility, risk
addressal
 SCSH
commitment to
way forward
15 July 2008
 Committed
protagonist
 Adequate
resources to
achieve goals
and
objectives:
funds, time,
staff, and info
 Phase plan
and rationale
 Fit to
enterprise
strategy
Inputs
 Funds, time,
staff, info
 Top-level phase
plan and
rationale
Steps (concurrent)
 Understand needs:
current and future
gaps
 Explore
opportunities
 Identify SCSHs and
their objectives
and priorities
 Analysis of
Alternatives
 Identify CommonCase process, if
there is one
 Develop and
validate shared
vision elements
 Obtain decision
commitments for
Valuation phase
©USC-CSSE
Postconditions
 Validated shared
vision
 Committed to by
SCSHs
 Common-Case
process or
special-case plan
 Feasibility
Evidence or riskmanagement
plans
 Competitive
prototyping
strategy
Work Products
 Top-level shared
vision package:
Benefits chain &
SCSHs; System
boundary, OpCon;
key solution
elements; and toplevel FED (including
business case)
 Common-Case
process or specialcase plan with FED
 SCSH Commitment
MOAs
 Identification of
next-level SCSHs,
IPT structure
Pitfalls
 Neglecting SCSHs
 Shortfalls in FEDs,
or SCSH MOAs
without riskmanagement
plans
 Poor project fit to
enterprise
strategy
 Overconstrained
and incompatible
quality factors
 Shortfalls in CP
strategy
20
University of Southern California
Center for Systems and Software Engineering
Top-Level VCR/CD Package
Operations/ Life Cycle Concept
- Top-level system boundary and environment elements
- Benefits chain or equivalent
- Links initiatives to desired benefits and identifies associated SCSs
- Including production and life cycle support SCSs
- Representative operational and support scenarios
Prototypes (focused on top development and operational risks)
Objectives, Constraints, and Priorities
- Initial Capabilities Document
Leading Solution Alternatives
- Top-level physical, logical, capability and behavioral views
Life Cycle Plan Key Elements
- Top-level phases, capability increments, roles, responsibilities, required
resources
Feasibility Evidence Description
- Evidence of ability to meet objectives within budget and schedule
constraints
- Evidence of ability to provide desired benefits to stakeholders
- Mission effectiveness evidence
15 July 2008
©USC-CSSE
21
University of Southern California
Center for Systems and Software Engineering
ICM Guidebook Content: Artifacts
• Stage I Artifacts
–
–
–
–
Operational Concept Definition
Requirements Definition
Architecture Definition
Life Cycle Plans
• Systems Engineering, Quality Management, …
– Feasibility Evidence Description
• Stage II Artifacts
–
–
–
–
15 July 2008
Detailed Increment Development Plans
Test and Assurance Plans
Implementation Plans
Life Cycle Support Plans
©USC-CSSE
22
University of Southern California
Center for Systems and Software Engineering
University of Southern California
Center for Systems and Software Engineering
University of Southern California
Center for Systems and Software Engineering
University of Southern California
Center for Systems and Software Engineering
ICM Guidebook Content: Appendices
• Using the ICM for Competitive Prototyping
• Incremental Commitment Milestones and
Feasibility Evidence Preparation
• Stakeholder Satisficing Techniques and Tools
• Risk Assessment and Management
• ICM-Oriented Estimation and Management Metrics
• Using the ICM for systems of systems
• Using the ICM process decision table
• Applying Selected ICM Practices to Ongoing
Projects
• Other special needs as appropriate
15 July 2008
©USC-CSSE
26
University of Southern California
Center for Systems and Software Engineering
ICM Transition Paths
• Existing programs may benefit from some ICM principles and
practices, but not others
• Problem programs may find some ICM practices helpful in
recovering viability
• Primary opportunities for incremental adoption of ICM
principles and practices
– Supplementing traditional requirements and design reviews with
development and review of feasibility evidence
– Stabilized incremental development and concurrent architecture
rebaselining
– Using schedule as independent variable and prioritizing features
to be delivered
– Continuous verification and validation
– Using the process decision table
15 July 2008
©USC-CSSE
27
University of Southern California
Center for Systems and Software Engineering
Outline
• Incremental Commitment Model (ICM) Overview
– ICM distinguishing features and implications
– Key ICM process views
• Proposed ICM Guidebook Organization
– Executive Summary: Key Features, Implications, Case Study
– Core Body: Phases and Activities; Case Study Illustrations
– Appendices: Detailed Practices; Common Special Cases
• Guidebook Status and Plans
• Key Issues
– Policies (5000.2, CP); Standards (IEEE-EIA, ISO)
• Conclusions; References; Acronyms; Backup charts
15 July 2008
©USC-CSSE
28
University of Southern California
Center for Systems and Software Engineering
Guidebook Status and Plans
• Concepts worked out for most elements
– A few outstanding issues; see below
• Critical mass of Electronic Process Guide in use
– On 16 real-client MS-student team projects
• Critical mass of content available as PowerPoint
charts with notes
– See Workshop Materials on this Workshop’s web site
– http://csse.usc.edu/csse/event/2008/cocomoicm08/pages/ma
terial.html
– Plan to convert Executive Summary into text by end of 2008
• Looking for pilot users and authoring collaborators
15 July 2008
©USC-CSSE
29
University of Southern California
Center for Systems and Software Engineering
Some Outstanding ICM-for-DoD Issues
• Final versions of 5000 series and related policies
• Degree of compatibility with ISO standards
• Sequencing of DoD milestones and source
selection
• Incremental Development vs. Operations and
Maintenance
• Support of competitive prototyping
• Impacts on funding, EVMS, estimation,
measurement
15 July 2008
©USC-CSSE
30
University of Southern California
Center for Systems and Software Engineering
Conclusions
• ICM proving robust, versatile on 16 test projects
– Architected Agile; Hybrid Agile/Plan-Driven; NDI-Intensive
• ICM approaches being applied on some major
programs
– Future Combat Systems, Missile Defense Agency
• Several issues remain to resolve
• Critical mass of guidance available for review and
experimentation
• Looking for pilot users and authoring collaborators
15 July 2008
©USC-CSSE
31
University of Southern California
Center for Systems and Software Engineering
References - I
Beck, K., Extreme Programming Explained, Addison Wesley, 1999.
Boehm, B., “Some Future Trends and Implications for Systems and Software Engineering Processes”,
Systems Engineering 9(1), pp. 1-19, 2006.
Boehm, B., Brown, W., Basili, V., and Turner, R., “Spiral Acquisition of Software-Intensive Systems of
Systems, CrossTalk, Vol. 17, No. 5, pp. 4-9, 2004.
Boehm, B. and Lane J., "21st Century Processes for Acquiring 21st Century Software-Intensive Systems of
Systems." CrossTalk: Vol. 19, No. 5, pp.4-9, 2006.
Boehm, B., and Lane, J., “Using the ICM to Integrate System Acquisition, Systems Engineering, and
Software Engineering,” CrossTalk, October 2007, pp. 4-9.
Boehm, B., and Lane, J., “A Process Decision Table for Integrated Systems and Software Engineering,”
Proceedings, CSER 2008, April 2008.
Boehm, B., “Future Challenges and Rewards for Software Engineers,” DoD Software Tech News, October
2007, pp. 6-12.
Boehm, B. et al., Software Cost Estimation with COCOMO II, Prentice Hall, 2000.
Boehm, B., Software Engineering Economics, Prentice Hall, 2000.
Carlock, P. and Fenton, R., "System of Systems (SoS) Enterprise Systems for Information-Intensive
Organizations," Systems Engineering, Vol. 4, No. 4, pp. 242-26, 2001.
Carlock, P., and J. Lane, “System of Systems Enterprise Systems Engineering, the Enterprise Architecture
Management Framework, and System of Systems Cost Estimation”, 21st International Forum on
COCOMO and Systems/Software Cost Modeling, 2006.
Checkland, P., Systems Thinking, Systems Practice, Wiley, 1980 (2nd ed., 1999).
Department of Defense (DoD), Defense Acquisition Guidebook, version 1.6, http://akss.dau.mil/dag/, 2006.
Department of Defense (DoD), Instruction 5000.2, Operation of the Defense Acquisition System, May 2003.
Department of Defense (DoD), Systems Engineering Plan Preparation Guide, USD(AT&L), 2004.
Electronic Industries Alliance (1999); EIA Standard 632: Processes for Engineering a System
Galorath, D., and Evans, M., Software Sizing, Estimation, and Risk Management, Auerbach, 2006.
Hall, E.T., Beyond Culture, Anchor Books/Doubleday, 1976.
15 July 2008
©USC-CSSE
32
University of Southern California
Center for Systems and Software Engineering
References -II
Highsmith, J., Adaptive Software Development, Dorset House, 2000.
International Standards Organization, Information Technology Software Life Cycle Processes, ISO/IEC
12207, 1995
ISO, Systems Engineering – System Life Cycle Processes, ISO/IEC 15288, 2002.
Jensen, R. “An Improved Macrolevel Software Development Resource Estimation Model,” Proceedings,
ISPA 5, April 1983, pp. 88-92.
Krygiel, A., Behind the Wizard’s Curtain; CCRP Publication Series, July, 1999, p. 33
Lane, J. and Boehm, B., "System of Systems Cost Estimation: Analysis of Lead System Integrator
Engineering Activities", Information Resources Management Journal, Vol. 20, No. 2, pp. 23-32, 2007.
Lane, J. and Valerdi, R., “Synthesizing SoS Concepts for Use in Cost Estimation”, Proceedings of IEEE
Systems, Man, and Cybernetics Conference, 2005.
Lientz, B., and Swanson, E.B., Software Maintenance Management, Addison Wesley, 1980.
Madachy, R., Boehm, B., Lane, J., "Assessing Hybrid Incremental Processes for SISOS Development", USC
CSSE Technical Report USC-CSSE-2006-623, 2006.
Maier, M., “Architecting Principles for Systems-of-Systems”; Systems Engineering, Vol. 1, No. 4 (pp 267284).
Maier, M., “System and Software Architecture Reconciliation,” Systems Engineering 9 (2), 2006, pp. 146-159.
Northrop, L., et al., Ultra-Large-Scale Systems: The Software Challenge of the Future, Software Engineering
Institute, 2006.
Pew, R. W., and Mavor, A. S., Human-System Integration in the System Development Process: A New Look,
National Academy Press, 2007.
Putnam, L., “A General Empirical Solution to the Macro Software Sizing and Estimating Problem,” IEEE
Trans SW Engr., July 1978, pp. 345-361.
Rechtin, E. Systems Architecting, Prentice Hall, 1991.
Schroeder, T., “Integrating Systems and Software Engineering: Observations in Practice,” OSD/USC
Integrating Systems and Software Engineering Workshop,
http://csse.usc.edu/events/2007/CIIForum/pages/program.html, October 2007.
15 July 2008
©USC-CSSE
33
University of Southern California
Center for Systems and Software Engineering
List of Acronyms
B/L
C4ISR
CD
CDR
COTS
DCR
DI
DoD
ECR
EVMS
FCR
FDN
FED
FMEA
FRP
GAO
GUI
15 July 2008
Baselined
Command, Control, Computing, Communications, Intelligence, Surveillance,
Reconnaissance
Concept Development
Critical Design Review
Commercial Off-the-Shelf
Development Commitment Review
Development Increment
Department of Defense
Exploration Commitment Review
Earned Value Management System
Foundations Commitment Review
Foundations, as in FDN Package
Feasibility Evidence Description
Failure Modes and Effects Analysis
Full-Rate Production
Government Accountability Office
Graphical User Interface
©USC-CSSE
34
University of Southern California
Center for Systems and Software Engineering
List of Acronyms (continued)
HMI
HSI
HW
ICM
IOC
IRR
IS&SE
LCA
LCO
LRIP
MBASE
NDI
NRC
OC
OCR
OO&D
OODA
O&M
15 July 2008
Human-Machine Interface
Human-System Interface
Hardware
Incremental Commitment Model
Initial Operational Capability
Inception Readiness Review
Integrating Systems and Software Engineering
Life Cycle Architecture
Life Cycle Objectives
Low-Rate Initial Production
Model-Based Architecting and Software Engineering
Non-Developmental Item
National Research Council
Operational Capability
Operations Commitment Review
Observe, Orient and Decide
Observe, Orient, Decide, Act
Operations and Maintenance
©USC-CSSE
35
University of Southern California
Center for Systems and Software Engineering
List of Acronyms (continued)
PDR
PM
PR
PRR
RUP
SoS
SoSE
SSE
SW
SwE
SysE
Sys Engr
S&SE
USD (AT&L)
VCR
V&V
WBS
WMI
15 July 2008
Preliminary Design Review
Program Manager
Public Relations
Product Release Review
Rational Unified Process
System of Systems
System of Systems Engineering
Systems and Software Engineering
Software
Software Engineering
Systems Engineering
Systems Engineer
Systems and Software Engineering
Under Secretary of Defense for Acquisition, Technology, and Logistics
Validation Commitment Review
Verification and Validation
Work Breakdown Structure
Warfighter-Machine Interface
©USC-CSSE
36
University of Southern California
Center for Systems and Software Engineering
Backup Charts
15 July 2008
©USC-CSSE
37
University of Southern California
Center for Systems and Software Engineering
ICM Approach to Systems of Systems
•
•
•
SoSs provide examples of how systems and software engineering can be
better integrated when evolving existing systems to meet new needs
Net-centricity and collaboration-intensiveness of SoSs have created more
emphasis on integrating hardware, software, and human factors engineering
Focus is on
– Flexibility and adaptability
– Use of creative approaches, experimentation, and tradeoffs
– Consideration of non-optimal approaches that are satisfactory to key stakeholders
•
Key focus for SoS engineering is to guide the evolution of constituent
systems within SoS to
• Perform cross-cutting capabilities
• While supporting existing single-system stakeholders
•
SoS process adaptations have much in common with the ICM
– Supports evolution of constituent systems using the appropriate special case for
each system
– Guides the evolution of the SoS through long range increments that fit with the
incremental evolution of the constituent systems
– ICM fits well with the OSD SoS SE Guidebook description of SoSE core elements
15 July 2008
©USC-CSSE
38
University of Southern California
Center for Systems and Software Engineering
Example: SoSE Coordination Challenge
FCR1
Valuation
SoS-Level Exploration
Candidate Supplier/
Strategic Partner n
DCR1
Architecting
OCR1
OCR2
Operation
Develop

LCO-type
Proposal &
Feasibility
Info
●
●
Source
Selection
Rebaseline/
Adjustment FCR1
●
Candidate Supplier/
Strategic Partner 1
OCRx2
OCRx1
System x   
Develop
OCRx3
Operation
Operation
OCRx5
OCRx4
Operation
Operation
DCRC
OCRC1

●
●
●
System C   
FCRC
Exploration
Valuation
Architecting
FCRB
System B   
Exploration
Valuation
15 July 2008
Exploration
Valuation
DCRB
Architecting
FCRA
System A   
Develop
Operation
OCRB1
Develop
©USC-CSSE

OCRB2
Operation

OCRA1
DCRA
Architecting
OCRC2
Develop
Operation
39

University of Southern California
Center for Systems and Software Engineering
Young Memo: Prototyping and Competition
• Discover issues before costly SDD phase
– Producing detailed designs in SDD
– Not solving myriad technical issues
• Services and Agencies to produce competitive
prototypes through Milestone B
– Reduce technical risk, validate designs and cost estimates,
evaluate manufacturing processes, refine requirements
• Will reduce time to fielding
– And enhance govt.-industry teambuilding, SysE skills,
attractiveness to next generation of technologists
• Applies to all programs requiring USD(AT&L) approval
– Should be extended to appropriate programs below ACAT I
15 July 2008
©USC-CSSE
40
University of Southern California
Center for Systems and Software Engineering
Implementing the Young Memo
• Need way of assuring continuity of funding, but with
off-ramps
– Prototyping and Competition Plan
– Incremental off-ramp milestones
• Need criteria for assessing feasibility to proceed
– Avoid overfocus on prototyping
• Feasible, scalable technology and management infrastructure
• Adequate budget, schedule, critical skills
• Key stakeholders committed to proceed
– Shortfalls in evidence are risks, need risk management plans
• ICM provides desired processes and criteria
– Anchor Point milestones and Feasibility Evidence deliverable
– Incremental phase entry and exit criteria
15 July 2008
©USC-CSSE
41
University of Southern California
Center for Systems and Software Engineering
Milestone B Focus on Technology Maturity
Misses Many OSD/AT&L Systemic Root Causes
1 Technical process (35 instances)
6 Lack of appropriate staff (23)
- V&V, integration, modeling&sim.
2 Management process (31)
7 Ineffective organization (22)
3 Acquisition practices (26)
8 Ineffective communication (21)
4 Requirements process (25)
9 Program realism (21)
5 Competing priorities (23)
10 Contract structure (20)
•Some of these are root causes of technology immaturity
•Can address these via evidence-based Milestone B exit criteria
•Technology Development Strategy
•Capability Development Document
•Evidence of affordability, KPP satisfaction, program achievability
15 July 2008
©USC-CSSE
42
University of Southern California
Center for Systems and Software Engineering
Acquiring Organization’s ICM-Based CP Plan
• Addresses issues discussed above
– Risk-driven prototyping rounds, concurrent definition and
development, continuity of support, stakeholder involvement,
off-ramps
• Organized around key management questions
– Objectives (why?): concept feasibility, best system solution
– Milestones and Schedules (what? when?): Number and timing of
competitive rounds; entry and exit criteria, including off-ramps
– Responsibilities (who? where?): Success-critical stakeholder
roles and responsibilities for activities and artifacts
– Approach (how?): Management approach or evaluation
guidelines, technical approach or evaluation methods, facilities,
tools, and concurrent engineering
– Resources (how much?): Necessary resources for acquirers,
competitors, evaluators, other stakeholders across full range of
prototyping and evaluation rounds
– Assumptions (whereas?): Conditions for exercise of off-ramps,
rebaselining of priorities and criteria
• Provides a stable framework for pursuing CP
15 July 2008
©USC-CSSE
43
University of Southern California
Center for Systems and Software Engineering
Conclusions: IS&SE with ICM
• Current DoD SysE guidance seriously inhibits SwE best practices
–
–
–
–
–
Largely sequential definition, design, development, integration, and test
Slows agility, ability to turn inside adversaries’ OODA loop
Functional “part-of” vs. layered “served by” product hierarchy
Static vs. dynamic interfaces: messages vs. protocols
One-size-fits-all process guidance inhibits balancing agility and assurance
• Incremental Commitment Model (ICM) enables better IS&SE
–
–
–
–
–
–
–
Integrates hardware, software, human factors aspects of SysE
Concurrent exploration of needs, opportunities, solutions
Successfully addresses issues above in commercial practice
Only partially proven in DoD practice (examples: CrossTalk Top-5 projects)
Key practices applied to help major programs (example: Future Combat Systems)
Being adopted by major programs (example: Missile Defense Agency)
Transition paths available for partial adoption
15 July 2008
©USC-CSSE
44
University of Southern California
Center for Systems and Software Engineering
ICM Assumptions and Critical Success Factors
• The project can establish milestones at which it can
synchronize and stabilize its elements
– May exclude some classes of systems of systems
• Ad-hoc, collaborative, some acknowledged
– But ICM principles can be applied to tailor variants
• The project can invest enough up front to generate
and evaluate feasibility evidence and address risks
• The project can acknowledge and manage its risks
– Risk-taking essential for many classes of projects
• Unprecedentedness, emergence, rapid change, immature
technology
• DARPA: 100% success rate is not doing your job
• Commercial: 40% failure rate acceptable for new ventures
15 July 2008
©USC-CSSE
45
University of Southern California
Center for Systems and Software Engineering
ICM Assumptions and Critical Success Factors - II
• The program office has enough expertise to evaluate
feasibility evidence and assess risks
– If not in-house, need to contract for it
• Feasibility evidence is a first-class deliverable
– Needs plans, EVMS monitoring of progress vs. plans
• Appropriate contracting mechanisms and incentive
structures are available for executing ICM approach
– Concurrent stabilized development, change rebaselining, V&V
– Collaborative systems of systems
– Competitive prototyping
• Representative examples are needed to clarify concepts
– Ideally, success stories with excursions to show pitfalls
15 July 2008
©USC-CSSE
46
University of Southern California
Center for Systems and Software Engineering
Implications for funding, contracting, career paths
• Incremental vs. total funding
– Often with evidence-based competitive downselect
• No one-size-fits all contracting
– Separate instruments for build-to-spec, agile rebaselining, V&V
teams
• With funding and award fees for collaboration, risk management
• Compatible regulations, specifications, and standards
• Compatible acquisition corps education and training
– Generally, schedule/cost/quality as independent variable
• Prioritized feature set as dependent variable
• Multiple career paths
– For people good at build-to-spec, agile rebaselining, V&V
– For people good at all three
• Future program managers and chief engineers
July 2008
©USC-CSSE
47