University of Southern California
Center for Systems and Software Engineering
ICM Introduction Extracted from
“Final Report:
Integrating Systems and Software
Engineering (IS&SE) Study”
Barry Boehm and Jo Ann Lane
University of Southern California
Center for Systems and Software Engineering
Arthur Pyster
Stevens Institute of Technology
December 31, 2007
University of Southern California
Center for Systems and Software Engineering
Outline
• DoD systems and software engineering trends and challenges
• State of systems and software engineering (S&SE) integration
– Sources of gaps and challenges
– Evaluation of current processes and standards
• Incremental Commitment Model (ICM) evaluation
– ICM origin, principles, and organization
– ICM evaluation
• Conclusions and recommendations
• References and acronyms
12/31/2007
©USC-CSSE
2
University of Southern California
Center for Systems and Software Engineering
ICM Practices and Assessment
• From the spiral model to the ICM
– Principles and example
• Risk-driven incremental definition: ICM Stage I
– Buying information to reduce risk
• Risk-driven incremental development: ICM Stage II
– Achieving both rapid change and high assurance
• Multiple views of the ICM
– Viewpoints and examples
• ICM Assessment
12/31/2007
©USC-CSSE
3
University of Southern California
Center for Systems and Software Engineering
From the Spiral Model to the ICM
• Need for intermediate milestones
– Anchor Point Milestones (1996)
• Avoid stakeholder success model clashes
– WinWin Spiral Model (1998)
• Avoid model misinterpretations
– Essentials and variants (2000-2005)
• Clarify usage in DoD Instruction 5000.2
– Initial phased version (2005)
• Explain system of systems spiral usage to GAO
– Underlying spiral principles (2006)
• Provide framework for human-systems integration
– National Research Council report (2007)
12/31/2007
©USC-CSSE
4
University of Southern California
Center for Systems and Software Engineering
Process Model Principles
1.
2.
3.
Commitment and accountability
Success-critical stakeholder satisficing
Incremental growth of system definition and
stakeholder commitment
4, 5. Concurrent, iterative system definition and
development cycles
Cycles can be viewed as sequential concurrentlyperformed phases or spiral growth of system
definition
6.
12/31/2007
Risk-based activity levels and anchor point
commitment milestones
©USC-CSSE
5
University of Southern California
Center for Systems and Software Engineering
Incremental Commitment in
Gambling
• Total Commitment: Roulette
– Put your chips on a number
• E.g., a value of a key performance parameter
– Wait and see if you win or lose
• Incremental Commitment: Poker, Blackjack
– Put some chips in
– See your cards, some of others’ cards
– Decide whether, how much to commit to proceed
12/31/2007
©USC-CSSE
6
University of Southern California
Center for Systems and Software Engineering
Scalable remotely controlled
operations
12/31/2007
©USC-CSSE
7
University of Southern California
Center for Systems and Software Engineering
Total vs. Incremental Commitment – 4:1 RPV
• Total Commitment
–
–
–
–
–
Agent technology demo and PR: Can do 4:1 for $1B
Winning bidder: $800M; PDR in 120 days; 4:1 capability in 40 months
PDR: many outstanding risks, undefined interfaces
$800M, 40 months: “halfway” through integration and test
1:1 IOC after $3B, 80 months
• Incremental Commitment [number of competing teams]
– $25M, 6 mo. to VCR [4]: may beat 1:2 with agent technology, but not
4:1
– $75M, 8 mo. to ACR [3]: agent technology may do 1:1; some risks
– $225M, 10 mo. to DCR [2]: validated architecture, high-risk elements
– $675M, 18 mo. to IOC [1]: viable 1:1 capability
– 1:1 IOC after $1B, 42 months
12/31/2007
©USC-CSSE
8
University of Southern California
Center for Systems and Software Engineering
The Cone of Uncertainty:
Usual result of total commitment
4x
2x
1.5x
1.25x
Relative
Cost Range
x
90% confidence limits:
- Pessimistic
Better to buy information to
reduce risk
0.8x
0.67x
0.5x
- Optimistic
0.25x
^
Inadequate PDR
Feasibility
Plans
and
Rqts.
Detail
Design
Spec.
Product
Design
Spec.
Rqts.
Spec.
Concept of
Operation
Product
Design
Detail
Design
Accepted
Software
Devel. and
Test
Phases and Milestones
12/31/2007
©USC-CSSE
9
University of Southern California
Center for Systems and Software Engineering
ICM Principles and Approach
• From the spiral model to the ICM
– Principles and example
• Risk-driven incremental definition: ICM Stage I
– Buying information to reduce risk
• Risk-driven incremental development: ICM Stage II
– Achieving both rapid change and high assurance
• Multiple views of the ICM
– Viewpoints and examples
12/31/2007
©USC-CSSE
10
University of Southern California
Center for Systems and Software Engineering
The Incremental Commitment Life Cycle Process: Overview
Stage I: Definition
Stage II: Development and Operations
Anchor Point
Milestones
Synchronize, stabilize concurrency via FRs
Risk patterns
determine life
cycle process
12/31/2007
©USC-CSSE
11
University of Southern California
Center for Systems and Software Engineering
Anchor Point Feasibility Rationales
• Evidence provided by developer and validated by
independent experts that:
If the system is built to the specified architecture, it will
– Satisfy the requirements: capability, interfaces, level of service,
and evolution
– Support the operational concept
– Be buildable within the budgets and schedules in the plan
– Generate a viable return on investment
– Generate satisfactory outcomes for all of the success-critical
stakeholders
• All major risks resolved or covered by risk management
plans
• Serves as basis for stakeholders’ commitment to proceed
12/31/2007
©USC-CSSE
12
University of Southern California
Center for Systems and Software Engineering
There is Another Cone of Uncertainty:
Shorter increments are better
4x
Uncertainties in competition,
technology, organizations,
mission priorities
2x
1.5x
1.25x
Relative
Cost Range
x
0.8x
0.67x
0.5x
0.25x
Concept of
Operation
Feasibility
Plans
and
Rqts.
Detail
Design
Spec.
Product
Design
Spec.
Rqts.
Spec.
Product
Design
Detail
Design
Accepted
Software
Devel. and
Test
Phases and Milestones
12/31/2007
©USC-CSSE
13
University of Southern California
Center for Systems and Software Engineering
The Incremental Commitment Life Cycle Process: Overview
Stage I: Definition
Stage II: Development and Operations
Anchor Point
Milestones
Concurrently engr.
OpCon, rqts, arch,
plans, prototypes
12/31/2007
©USC-CSSE
Concurrently engr.
Incr.N (ops), N+1
(devel), N+2 (arch)
14
University of Southern California
Center for Systems and Software Engineering
ICM Stage II: Increment View
Rapid
Change
Short
Development
Increments
Foreseeable
Change (Plan)
Increment N Baseline
Short, Stabilized
Development
of Increment N
Increment N Transition/O&M
Stable Development
Increments
High
Assurance
12/31/2007
©USC-CSSE
15
University of Southern California
Center for Systems and Software Engineering
ICM Stage II: Increment View
A radical idea?
Unforseeable Change
(Adapt)
Rapid
Change
Short
Development
Increments
Agile
Future Increment Baselines
Rebaselining for
Future Increments
Deferrals
Foreseeable
Change (Plan)
Increment N Baseline
Stable Development
Increments
Current V&V
High
Assurance
Resources
Short, Stabilized
Development
of Increment N
Artifacts
Increment N Transition/O&M
Concerns
V&V
of Increment N
Future V&V
Resources
Continuous V&V
No; a commercial best practice and part of DoDI 5000.2
12/31/2007
©USC-CSSE
16
University of Southern California
Center for Systems and Software Engineering
ICM Principles and Approach
• From the spiral model to the ICM
– Principles and example
• Risk-driven incremental definition: ICM Stage I
– Buying information to reduce risk
• Risk-driven incremental development: ICM Stage II
– Achieving both rapid change and high assurance
• Multiple views of the ICM
– Viewpoints and examples
12/31/2007
©USC-CSSE
17
University of Southern California
Center for Systems and Software Engineering
MBASE-RUP/ICM Anchor Points Enable Concurrent Engineering
V
C
R
12/31/2007
A
C
R
D
C
R
©USC-CSSE
C
C
D
I
O
C
O
C
R
18
University of Southern California
Center for Systems and Software Engineering
ICM HSI Levels of Activity for Complex Systems
12/31/2007
©USC-CSSE
19
University of Southern California
Center for Systems and Software Engineering
Different Risk Patterns Yield Different Processes
12/31/2007
©USC-CSSE
20
University of Southern California
Center for Systems and Software Engineering
Common Risk-Driven Special Cases of the Incremental Commitment Model (ICM)
Special Case
Example
Size,
Complexit
y
Change
Rate %
/Month
Criticality
NDI Support
1. Use NDI
Small Accounting
2. Agile
E-services
Low
1 – 30
Low-Med
Good;
in place
3. Scrum of
Scrums
Business data
processing
Med
1 – 10
Med-High
4. SW embedded
HW component
Multisensor
control device
Low
0.3 – 1
5. Indivisible IOC
Complete vehicle
platform
Med –
High
6. NDI- Intensive
Supply Chain
Management
7. Hybrid agile /
plan-driven
system
Org, Personnel
Capability
Key Stage I Activities : Incremental
Definition
Key Stage II Activities: Incremental
Development, Operations
Acquire NDI
Use NDI
Agile-ready
Med-high
Skip Valuation , Architecting phases
Scrum plus agile methods of choice
<= 1 day;
2-6 weeks
Good;
most in place
Agile-ready
Med-high
Combine Valuation, Architecting
phases. Complete NDI preparation
Architecture-based Scrum of
Scrums
2-4 weeks;
2-6 months
Med-Very
High
Good;
In place
Experienced;
med-high
Concurrent HW/SW engineering.
CDR-level ICM DCR
IOC Development, LRIP, FRP.
Concurrent Version N+1
engineering
SW: 1-5 days;
Market-driven
0.3 – 1
HighVery
High
Some in place
Experienced;
med-high
Determine minimum-IOC likely,
conservative cost. Add deferrable
SW features as risk reserve
Drop deferrable features to meet
conservative cost. Strong award fee
for features not dropped
SW: 2-6 weeks;
Platform: 6-18
months
Med –
High
0.3 – 3
MedVery
High
NDI-driven
architecture
NDI-experienced;
Med-high
Thorough NDI-suite life cycle costbenefit analysis, selection,
concurrent requirements/
architecture definition
Pro-active NDI evolution influencing,
NDI upgrade synchronization
SW: 1-4 weeks;
System: 6-18
months
C4ISR
Med –
Very
High
Mixed
parts:
1 – 10
Mixed
parts;
Med-Very
High
Mixed parts
Mixed parts
Full ICM; encapsulated agile in high
change, low-medium criticality parts
(Often HMI, external interfaces)
Full ICM ,three-team incremental
development, concurrent V&V, nextincrement rebaselining
1-2 months;
9-18 months
8. Multi-owner
system of
systems
Net-centric
military
operations
Very
High
Mixed
parts:
1 – 10
Very
High
Many NDIs;
some in place
Related
experience, medhigh
Full ICM; extensive multi-owner team
building, negotiation
Full ICM; large ongoing
system/software engineering effort
2-4 months; 1824 months
9. Family of
systems
Medical Device
Product Line
Med –
Very
High
1–3
Med –
Very
High
Some in place
Related
experience, med
– high
Full ICM; Full stakeholder
participation in product line scoping.
Strong business case
Full ICM. Extra resources for first
system, version control, multistakeholder support
1-2 months; 918 months
Complete
Time per Build;
per Increment
C4ISR: Command, Control, Computing, Communications, Intelligence, Surveillance, Reconnaissance. CDR: Critical Design Review.
DCR: Development Commitment Review. FRP: Full-Rate Production. HMI: Human-Machine Interface. HW: Hard ware.
IOC: Initial Operational Capability. LRIP: Low-Rate Initial Production. NDI: Non-Development Item. SW: Software
12/31/2007
©USC-CSSE
21
University of Southern California
Center for Systems and Software Engineering
References - I
Beck, K., Extreme Programming Explained, Addison Wesley, 1999.
Boehm, B., “Some Future Trends and Implications for Systems and Software Engineering Processes”,
Systems Engineering 9(1), pp. 1-19, 2006.
Boehm, B., Brown, W., Basili, V., and Turner, R., “Spiral Acquisition of Software-Intensive Systems of
Systems, CrossTalk, Vol. 17, No. 5, pp. 4-9, 2004.
Boehm, B. and Lane J., "21st Century Processes for Acquiring 21st Century Software-Intensive Systems of
Systems." CrossTalk: Vol. 19, No. 5, pp.4-9, 2006.
Boehm, B., and Lane, J., “Using the ICM to Integrate System Acquisition, Systems Engineering, and
Software Engineering,” CrossTalk, October 2007, pp. 4-9.
Boehm, B., “Future Challenges and Rewards for Softwarre Engineers,” DoD Software Tech News, October
2007, pp. 6-12.
Boehm, B. et al., Software Cost Estimation with COCOMO II, Prentice Hall, 2000.
Boehm, B., Software Engineering Economics, Prentice Hall, 2000.
Carlock, P. and Fenton, R., "System of Systems (SoS) Enterprise Systems for Information-Intensive
Organizations," Systems Engineering, Vol. 4, No. 4, pp. 242-26, 2001.
Carlock, P., and J. Lane, “System of Systems Enterprise Systems Engineering, the Enterprise Architecture
Management Framework, and System of Systems Cost Estimation”, 21st International Forum on
COCOMO and Systems/Software Cost Modeling, 2006.
Checkland, P., Systems Thinking, Systems Practice, Wiley, 1980 (2nd ed., 1999).
Department of Defense (DoD), Defense Acquisition Guidebook, version 1.6, http://akss.dau.mil/dag/, 2006.
Hall, E.T., Beyond Culture, Anchor Books/Doubleday, 1976.
Highsmith, J., Adaptive Software Development, Dorset House, 2000.
Jensen, R. “An Improved Macrolevel Software Development Resource Estimation Model,” Proceedings,
ISPA 5, April 1983, pp. 88-92.
12/31/2007
©USC-CSSE
22
University of Southern California
Center for Systems and Software Engineering
References -II
Krygiel, A. (1999); Behind the Wizard’s Curtain; CCRP Publication Series, July, 1999, p. 33
Lane, J. and Boehm, B., "System of Systems Cost Estimation: Analysis of Lead System Integrator
Engineering Activities", Information Resources Management Journal, Vol. 20, No. 2, pp. 23-32, 2007.
Lane, J. and Valerdi, R., “Synthesizing SoS Concepts for Use in Cost Estimation”, Proceedings of IEEE
Systems, Man, and Cybernetics Conference, 2005.
Lientz, B., and Swanson, E.B., Software Maintenance Management, Addison Wesley, 1980.
Madachy, R., Boehm, B., Lane, J., "Assessing Hybrid Incremental Processes for SISOS Development", USC
CSSE Technical Report USC-CSSE-2006-623, 2006.
Maier, M., “System and Software Architecture Reconciliation,” Systems Engineering 9 (2), 2006, pp. 146-159.
Northrop, L., et al., Ultra-Large-Scale Systems: The Software Challenge of the Future, Software
Engineering Institute, 2006.
Pew, R. W., and Mavor, A. S., Human-System Integration in the System Development Process: A New Look,
National Academy Press, 2007.
Putnam, L., “A General Empirical Solution to the Macro Software Sizing and Estimating Problem,” IEEE
Trans SW Engr., July 1978, pp. 345-361.
Rechtin, E. Systems Architecting, Prentice Hall, 1991.
Schroeder, T., “Integrating Systems and Software Engineering: Observations in Practice,” OSD/USC
Integrating Systems and Software Engineering Workshop,
http://csse.usc.edu/events/2007/CIIForum/pages/program.html, October 2007.
12/31/2007
©USC-CSSE
23
University of Southern California
Center for Systems and Software Engineering
SoSE-Related References
Carlock, P.G., and R.E. Fenton, "System of Systems (SoS) Enterprise Systems for Information-Intensive Organizations,"
Systems Engineering, Vol. 4, No. 4, pp. 242-261, 2001
Department of Defense (DoD), Defense Acquisition Guidebook, version 1.6, http://akss.dau.mil/dag/, 2006.
Department of Defense (DoD), System of Systems Engineering Guide: Considerations for Systems Engineering in a System
of Systems Environment, draft version 0.9, 2006.
DiMario, Mike (2006); “System of Systems Characteristics and Interoperability in Joint Command Control”, Proceedings of the
2nd Annual System of Systems Engineering Conference
Electronic Industries Alliance (1999); EIA Standard 632: Processes for Engineering a System
Finley, James (2006); “Keynote Address”, Proceedings of the 2nd Annual System of Systems Engineering Conference
Garber, Vitalij (2006); “Keynote Presentation”, Proceedings of the 2nd Annual System of Systems Engineering Conference
INCOSE (2006); Systems Engineering Handbook, Version 3, INCOSE-TP-2003-002-03
Kuras, M. L., White, B. E., Engineering Enterprises Using Complex-System Engineering, INCOSE Symposium 2005.
Krygiel, A. (1999); Behind the Wizard’s Curtain; CCRP Publication Series, July, 1999, p. 33
Maier, M. (1998); “Architecting Principles for Systems-of-Systems”; Systems Engineering, Vol. 1, No. 4 (pp 267-284)
Meilich, Abe (2006); “System of Systems Engineering (SoSE) and Architecture Challenges in a Net Centric Environment”,
Proceedings of the 2nd Annual System of Systems Engineering Conference
Pair, Major General Carlos (2006); “Keynote Presentation”, Proceedings of the 2nd Annual System of Systems Engineering
Conference
Proceedings of AFOSR SoSE Workshop, Sponsored by Purdue University, 17-18 May 2006
Proceedings of the 2nd Annual System of Systems Engineering Conference, Sponsored by System of Systems Engineering
Center of Excellence (SoSECE), http://www.sosece.org/, 2006.
Proceedings of Society for Design and Process Science 9th World Conference on Integrated Design and Process Technology,
San Diego, CA, 25-30 June 2006
Siel, Carl (2006); “Keynote Presentation”, Proceedings of the 2nd Annual System of Systems Engineering Conference
United States Air Force Scientific Advisory Board (2005); Report on System-of-Systems Engineering for Air Force Capability
Development; Public Release SAB-TR-05-04
12/31/2007
©USC-CSSE
24
University of Southern California
Center for Systems and Software Engineering
List of Acronyms
ACR
B/L
CCD
COTS
DCR
DI
DOTMLPF
ECR
FMEA
GUI
HSI
ICM
IOC
IRR
12/31/2007
Architecting Commitment Review
Baselined
Core Capability Drive-Through
Commercial Off-the-Shelf
Development Commitment Review
Development Increment
Doctrine, Organization, Training, Materiel, Leadership, Personnel, Facilities
Exploration Commitment Review
Failure Modes and Effects Analysis
Graphical User Interface
Human-System Interface
Incremental Commitment Model
Initial Operational Capability
Inception Readiness Review
©USC-CSSE
25
University of Southern California
Center for Systems and Software Engineering
List of Acronyms (continued)
LCA
LCO
OC
OCR
OO&D
OODA
O&M
PRR
RACRS
SoS
SoSE
TSE
VCR
V&V
12/31/2007
Life Cycle Architecture
Life Cycle Objectives
Operational Capability
Operations Commitment Review
Observe, Orient and Decide
Observe, Orient, Decide, Act
Operations and Maintenance
Product Release Review
Regional Area Crisis Response System
System of Systems
System of Systems Engineering
Traditional Systems Engineering
Validation Commitment Review
Verification and Validation
©USC-CSSE
26