University of Southern California
Center for Systems and Software Engineering
Affordability-Based Engineering
Barry Boehm, USC-CSSE
http://csse.usc.edu
SSCM/COCOMO Forum
October 16, 2012
University of Southern California
Center for Systems and Software Engineering
Outline
• Affordability definitions, concepts, issues, strategies
–
–
–
–
–
Addressing both costs and benefits
Using life cycle present value
Coping with uncertainty: incrementally; pro-actively
Coping with multi-stakeholder value diversity
Addressing tradeoffs with other -ilities
• An orthogonal framework for improving affordability costs
– Cost modeling and other insights
• Conclusions
October 16, 2012
Copyright © USC-CSSE
2
University of Southern California
Center for Systems and Software Engineering
Affordability Definitions
• INCOSE: The balance of system performance, cost, and schedule
constraints over the system life cycle, while satisfying mission
needs in concert with strategic and organizational needs.
• MORS: Cost-effective capability (USD/ATL).
• NDIA: The practice of assuring program success through the
balancing of system performance (KPPs), cost, and schedule
constraints, while satisfying mission needs in concert with the
long-range investment and force structure plans of the DoD.
• Webster: Keeping within your financial means.
October 16, 2012
Copyright © USC-CSSE
3
University of Southern California
Center for Systems and Software Engineering
Affordability Concepts
Coping with uncertainty: incrementally; pro-actively
Coping with multi-stakeholder value diversity
Life
Cycle
Benefits
Improvement
Achievable
Pareto
Boundary
Primary
Option
Space
Current
Situation
Increasing
Cost-Benefit
Improvement
Isoquants
Life Cycle Cost Improvement
October 16, 2012
Copyright © USC-CSSE
4
University of Southern California
Center for Systems and Software Engineering
Multi-Stakeholder Value Diversity
Bank of America Master Net
5
October 16, 2012
Copyright © USC-
University of Southern California
Center for Systems and Software Engineering
Outline
• Affordability definitions, concepts, issues, strategies
–
–
–
–
–
Addressing both costs and benefits
Using life cycle present value
Coping with uncertainty: incrementally; pro-actively
Coping with multi-stakeholder value diversity
Addressing tradeoffs with other -ilities
• An orthogonal framework for improving affordability costs
– Cost modeling and other insights
• Conclusions
October 16, 2012
Copyright © USC-CSSE
6
University of Southern California
Affordability and Tradespace Framework
Center for Systems and Software Engineering
Get the Best from People
Make Tasks More Efficient
Affordability
Improvements
and Tradeoffs
Staffing, Incentivizing, Teambuilding
Facilities, Support Services
Kaizen (continuous improvement)
Tools and Automation
Work and Oversight Streamlining
Collaboration Technology
Lean and Agile Methods
Eliminate Tasks
Task Automation
Model-Based Product Generation
Eliminate Scrap, Rework
Simplify Products (KISS)
Early Risk and Defect Elimination
Evidence-Based Decision Gates
Modularity Around Sources of Change
Incremental, Evolutionary Development
Value-Based, Agile Process Maturity
Risk-Based Prototyping
Value-Based Capability Prioritization
Satisficing vs. Optimizing Performance
Reuse Components
Domain Engineering and Architecture
Composable Components,Services, COTS
Legacy System Repurposing
Reduce Operations, Support Costs
Value- and Architecture-Based
Tradeoffs and Balancing
7
Copyright
October©16,
USC-CSSE
2012
Automate Operations Elements
Design for Maintainability, Evolvability
Streamline Supply Chain
Anticipate, Prepare for Change
University of Southern California
Center for Systems and Software Engineering
Costing Insights: COCOMO II Productivity Ranges
Scale Factor Ranges: 10, 100, 1000 KSLOC
Development Flexibility (FLEX)
Staffing
Team Cohesion (TEAM)
Develop for Reuse (RUSE)
Teambuilding
Precedentedness (PREC)
Architecture and Risk Resolution (RESL)
Continuous
Improvement
Platform Experience (PEXP)
Data Base Size (DATA)
Required Development Schedule (SCED)
Language and Tools Experience (LTEX)
Process Maturity (PMAT)
Storage Constraint (STOR)
Use of Software Tools (TOOL)
Platform Volatility (PVOL)
Applications Experience (AEXP)
Multi-Site Development (SITE)
Documentation Match to Life Cycle Needs (DOCU)
Required Software Reliability (RELY)
Personnel Continuity (PCON)
Time Constraint (TIME)
Programmer Capability (PCAP)
Analyst Capability (ACAP)
Product Complexity (CPLX)
1
1.2
1.4
1.6
1.8
Productivity Range
October 16, 2012
Copyright ©8 USC-CSSE
2
2.2
2.4
University of Southern California
Center for Systems and Software Engineering
COSYSMO Sys Engr Cost Drivers
Teambuilding
Continuous
Improvement
Staffing
October 16, 2012
Copyright © USC-CSSE
9
University of Southern California
Tradespace and Affordability Framework
Center for Systems and Software Engineering
Get the Best from People
Make Tasks More Efficient
Affordability
Improvements
and Tradeoffs
Staffing, Incentivizing, Teambuilding
Facilities, Support Services
Kaizen (continuous improvement)
Tools and Automation
Work and Oversight Streamlining
Collaboration Technology
Lean and Agile Methods
Eliminate Tasks
Task Automation
Model-Based Product Generation
Eliminate Scrap, Rework
Simplify Products (KISS)
Early Risk and Defect Elimination
Evidence-Based Decision Gates
Modularity Around Sources of Change
Incremental, Evolutionary Development
Value-Based, Agile Process Maturity
Risk-Based Prototyping
Value-Based Capability Prioritization
Satisficing vs. Optimizing Performance
Reuse Components
Domain Engineering and Architecture
Composable Components,Services, COTS
Legacy System Repurposing
Reduce Operations, Support Costs
Automate Operations Elements
Design for Maintainability, Evolvability
Streamline Supply Chain
Anticipate, Prepare for Change
Value- and Architecture-Based
Tradeoffs and Balancing
10
Copyright
October©16,
USC-CSSE
2012
University of Southern California
Center for Systems and Software Engineering
COCOMO II. 2000 Productivity Ranges
Scale Factor Ranges: 10, 100, 1000 KSLOC
Development Flexibility (FLEX)
Tools
Team Cohesion (TEAM)
Develop for Reuse (RUSE)
Collaboration
Technology
Precedentedness (PREC)
Architecture and Risk Resolution (RESL)
Work Streamlining
Platform Experience (PEXP)
Data Base Size (DATA)
Required Development Schedule (SCED)
Language and Tools Experience (LTEX)
Process Maturity (PMAT)
Storage Constraint (STOR)
Use of Software Tools (TOOL)
Platform Volatility (PVOL)
Applications Experience (AEXP)
Multi-Site Development (SITE)
Documentation Match to Life Cycle Needs (DOCU)
Required Software Reliability (RELY)
Personnel Continuity (PCON)
Time Constraint (TIME)
Programmer Capability (PCAP)
Analyst Capability (ACAP)
Product Complexity (CPLX)
1
1.2
1.4
1.6
1.8
Productivity Range
October 16, 2012
Copyright ©
11USC-CSSE
2
2.2
2.4
University of Southern California
Tradespace and Affordability Framework
Center for Systems and Software Engineering
Get the Best from People
Make Tasks More Efficient
Staffing, Incentivizing, Teambuilding
Facilities, Support Services
Kaizen (continuous improvement)
Tools and Automation
Work and Oversight Streamlining
Collaboration Technology
Affordability
Improvements
and Tradeoffs
Lean and Agile Methods
Eliminate Tasks
Task Automation
Model-Based Product Generation
Eliminate Scrap, Rework
Simplify Products (KISS)
Early Risk and Defect Elimination
Evidence-Based Decision Gates
Modularity Around Sources of Change
Incremental, Evolutionary Development
Value-Based, Agile Process Maturity
Risk-Based Prototyping
Value-Based Capability Prioritization
Satisficing vs. Optimizing Performance
Reuse Components
Domain Engineering and Architecture
Composable Components,Services, COTS
Legacy System Repurposing
Reduce Operations, Support Costs
Automate Operations Elements
Design for Maintainability, Evolvability
Streamline Supply Chain
Anticipate, Prepare for Change
Value- and Architecture-Based
Tradeoffs and Balancing
12
Copyright
October©16,
USC-CSSE
2012
University of Southern California
Center for Systems and Software Engineering
Agile and Plan-Driven Home Grounds:
Five Critical Decision Factors
• Size, Criticality, Dynamism, Personnel, Culture
Personnel
(% Level 1B) (% Level 2&3)
40
15
30
20
20
25
10
30
0
Many
Lives Single
Essential
Life
Discretionary
Funds
Comfort
Funds
35
Criticality
(Loss due to impact of defects)
Dynamism
(% Requirements-change/month)
1.0
0.3
3.0
10
30
3
90
10
70
30
50
100
30
300
Size
(# of personnel)
October 16, 2012
10
Culture
(% thriving on chaos vs. order)
Copyright © USC-CSSE
13
University of Southern California
Center for Systems and Software Engineering
Architected Agile Approach
•
Uses Scrum of Scrums approach
– Up to 10 Scrum teams of 10 people each
– Has worked for distributed international teams
– Going to three levels generally infeasible
• General approach shown below
– Often tailored to special circumstances
October 16, 2012
Copyright ©
14USC-CSSE
University of Southern California
Center for Systems and Software Engineering
Trends in Software Expansion (Bernstein, 1997)
MBSE:2010
1000
638
475
113
Expansion
100
Factor
The ratio
of machine
lines of
code to a
source line
of code
75
30
37.5
142
81
47
15
10
3
Order of Magnitude Increase Every Twenty Years
1
1960
1965
1970
1975
1980
1985
1990
1995
2000
Machine
Instructions
Macro
Assembler
High Level
Language
Database
Manager
On-line
Prototyping
Subsecond
Time
Sharing
Object
Oriented
Programming
Large Scale
Reuse
Regression
Testing
15
October 16, 2012
4GL
Small
Scale
Reuse
Copyright © USC-CSSE
University of Southern California
Tradespace and Affordability Framework
Center for Systems and Software Engineering
Get the Best from People
Make Tasks More Efficient
Staffing, Incentivizing, Teambuilding
Facilities, Support Services
Kaizen (continuous improvement)
Tools and Automation
Work and Oversight Streamlining
Collaboration Technology
Affordability
Improvements
and Tradeoffs
Lean and Agile Methods
Eliminate Tasks
Task Automation
Model-Based Product Generation
Eliminate Scrap, Rework
Simplify Products (KISS)
Early Risk and Defect Elimination
Evidence-Based Decision Gates
Modularity Around Sources of Change
Incremental, Evolutionary Development
Value-Based, Agile Process Maturity
Risk-Based Prototyping
Value-Based Capability Prioritization
Satisficing vs. Optimizing Performance
Reuse Components
Domain Engineering and Architecture
Composable Components,Services, COTS
Legacy System Repurposing
Reduce Operations, Support Costs
Automate Operations Elements
Design for Maintainability, Evolvability
Streamline Supply Chain
Anticipate, Prepare for Change
Value- and Architecture-Based
Tradeoffs and Balancing
16
Copyright
October©16,
USC-CSSE
2012
University of Southern California
Center for Systems and Software Engineering
COCOMO II. 2000 Productivity Ranges
Scale Factor Ranges: 10, 100, 1000 KSLOC
Development Flexibility (FLEX)
Team Cohesion (TEAM)
Develop for Reuse (RUSE)
Precedentedness (PREC)
Architecture and Risk Resolution (RESL)
Platform Experience (PEXP)
Data Base Size (DATA)
Required Development Schedule (SCED)
Language and Tools Experience (LTEX)
Process Maturity (PMAT)
Storage Constraint (STOR)
Use of Software Tools (TOOL)
Platform Volatility (PVOL)
Applications Experience (AEXP)
Multi-Site Development (SITE)
Documentation Match to Life Cycle Needs (DOCU)
Required Software Reliability (RELY)
Personnel Continuity (PCON)
Time Constraint (TIME)
Programmer Capability (PCAP)
Analyst Capability (ACAP)
Product Complexity (CPLX)
1
1.2
1.4
1.6
1.8
Productivity Range
October 16, 2012
Copyright ©
17USC-CSSE
2
2.2
2.4
University of Southern California
Tradespace and Affordability Framework
Center for Systems and Software Engineering
Get the Best from People
Make Tasks More Efficient
Staffing, Incentivizing, Teambuilding
Facilities, Support Services
Kaizen (continuous improvement)
Tools and Automation
Work and Oversight Streamlining
Collaboration Technology
Affordability
Improvements
and Tradeoffs
Lean and Agile Methods
Eliminate Tasks
Task Automation
Model-Based Product Generation
Eliminate Scrap, Rework
Simplify Products (KISS)
Early Risk and Defect Elimination
Evidence-Based Decision Gates
Modularity Around Sources of Change
Incremental, Evolutionary Development
Value-Based, Agile Process Maturity
Risk-Based Prototyping
Value-Based Capability Prioritization
Satisficing vs. Optimizing Performance
Reuse Components
Domain Engineering and Architecture
Composable Components,Services, COTS
Legacy System Repurposing
Reduce Operations, Support Costs
Automate Operations Elements
Design for Maintainability, Evolvability
Streamline Supply Chain
Anticipate, Prepare for Change
Value- and Architecture-Based
Tradeoffs and Balancing
18
Copyright
October©16,
USC-CSSE
2012
University of Southern California
Center for Systems and Software Engineering
Value-Based Testing: Empirical Data and ROI
— LiGuo Huang, ISESE 2005
100
(a)
% of
Value
for
Correct
Customer
Billing
Bullock data
– Pareto distribution
80
60
Automated test
generation (ATG) tool
- all tests have equal value
40
20
5
10
15
Customer Type
(b)
Return On Investment (ROI)
2
1.5
1
0.5
0
0
10
20
30
40
50
60
70
80
-0.5
-1
-1.5
% Tests Run
Value-Neutral ATG Testing
October 16, 2012
Value-Based Pareto Testing
Copyright ©
19USC-CSSE
90
100
University of Southern California
Center for Systems and Software Engineering
Sequential Requirements-First Risks
It’s not a requirement if you can’t afford it
$100M
Required
Architecture:
Custom; many
cache processors
$50M
Original
Architecture:
Commercial
DBMS
Original Budget
Original Spec
1
After Prototyping
2
3
4
5
Response Time (sec)
20
October 16, 2012
Copyright © USC-CSSE
University of Southern California
Center for Systems and Software Engineering
Value-Neutral Defect Fixing Is Even Worse
Pareto 80-20 Business Value
100
Automated test
generation tool
- all tests have equal value
80
% of
Value
for
Correct
Customer
Billing
60
40
Value-neutral defect fixing:
Quickly reduce # of defects
20
5
10
15
Customer Type
October 16, 2012
Copyright © USC-CSSE
21
University of Southern California
Tradespace and Affordability Framework
Center for Systems and Software Engineering
Get the Best from People
Make Tasks More Efficient
Staffing, Incentivizing, Teambuilding
Facilities, Support Services
Kaizen (continuous improvement)
Tools and Automation
Work and Oversight Streamlining
Collaboration Technology
Affordability
Improvements
and Tradeoffs
Lean and Agile Methods
Eliminate Tasks
Task Automation
Model-Based Product Generation
Eliminate Scrap, Rework
Simplify Products (KISS)
Early Risk and Defect Elimination
Evidence-Based Decision Gates
Modularity Around Sources of Change
Incremental, Evolutionary Development
Value-Based, Agile Process Maturity
Risk-Based Prototyping
Value-Based Capability Prioritization
Satisficing vs. Optimizing Performance
Reuse Components
Domain Engineering and Architecture
Composable Components,Services, COTS
Legacy System Repurposing
Reduce Operations, Support Costs
Automate Operations Elements
Design for Maintainability, Evolvability
Streamline Supply Chain
Anticipate, Prepare for Change
Value- and Architecture-Based
Tradeoffs and Balancing
22
Copyright
October©16,
USC-CSSE
2012
University of Southern California
Center for Systems and Software Engineering
Reuse at HP’s Queensferry
Telecommunication Division
70
Time
to
Market
60
(months)
40
Non-reuse Project
Reuse project
50
30
20
10
0
86
87
88
89
90
91
92
Year
23
October 16, 2012
Copyright © USC-CSSE
University of Southern California
The HMMWV
Center for Systems and Software Engineering
300,000 Produced, 22 Fielded Versions
Initial draft requirements in 1979, Initial delivery in 1984
Additional armor and
cupola raise the CG
and increase
rollovers
Bolt on armor
required upgraded
suspension, engine,
and steering
Mattracks or
wheels
Upper deck space is
always at a premium
Upgrades:
•Increased cab space
•Increased payload
capacity
• Strengthened frame
Imbalance in cupola
required motorized
drive
Base cab & flatbed with
mission modules
9
Suspension and
steering for CG shift
University of Southern California
Center for Systems and Software Engineering
Product Line Engineering and Management
October 16, 2012
Copyright © USC-CSSE
25
University of Southern California
Tradespace and Affordability Framework
Center for Systems and Software Engineering
Get the Best from People
Make Tasks More Efficient
Staffing, Incentivizing, Teambuilding
Facilities, Support Services
Kaizen (continuous improvement)
Tools and Automation
Work and Oversight Streamlining
Collaboration Technology
Affordability
Improvements
and Tradeoffs
Lean and Agile Methods
Eliminate Tasks
Task Automation
Model-Based Product Generation
Eliminate Scrap, Rework
Simplify Products (KISS)
Early Risk and Defect Elimination
Evidence-Based Decision Gates
Modularity Around Sources of Change
Incremental, Evolutionary Development
Value-Based, Agile Process Maturity
Risk-Based Prototyping
Value-Based Capability Prioritization
Satisficing vs. Optimizing Performance
Reuse Components
Reduce Operations, Support Costs
Value- and Architecture-Based
Tradeoffs and Balancing
26
Copyright
October©16,
USC-CSSE
2012
Domain Engineering and Architecture
Composable Components,Services, COTS
Legacy System Repurposing
Automate Operations Elements
Design for Maintainability, Evolvability
Streamline Supply Chain
Anticipate, Prepare for Change
University of Southern California
Center for Systems and Software Engineering
Post-Acquisition Costs Dominate (%O&M)
• Hardware [Redman 2008]
–
–
–
–
12% -- Missiles (average)
60% -- Ships (average)
78% -- Aircraft (F-16)
84% -- Ground vehicles (Bradley)
• Software [Koskinen 2010]
– 75-90% -- Business, Command-Control
– 50-80% -- Complex platforms as above
– 10-30% -- Simple embedded software
• Apply lack-of-flexibility factor to O&M
component
October 16, 2012
Copyright © USC-CSSE
27
University of Southern California
Center for Systems and Software Engineering
Overfocus on Acquisition Cost
C4ISR Contracts: Nominal-case requirements; 90 days to PDR
28
October 16, 2012
Copyright © USC-CSSE
University of Southern California
Center for Systems and Software Engineering
C4ISR Project C: Architecting for Change
USAF/ESC-TRW CCPDS-R Project*
When investments made in architecture, average time for change order
becomes relatively stable over time…
* Walker Royce, Software Project Management: A Unified Framework. Addison-Wesley, 1998.
29
October 16, 2012
Copyright © USC-CSSE
University of Southern California
Center for Systems and Software Engineering
Relative* Total Ownership Cost (TOC)
250.00%
~5% architecture
investment
200.00%
~5% architecture
investment
150.00%
~25% architecture
investment
100.00%
50.00%
0.00%
Cycle 1
Cycle 2
Project A
Cycle 3
Project B
Cycle 4
Cycle 5
Project C
* Cumulative architecting and rework effort relative to initial development effort
30
October 16, 2012
Copyright © USC-CSSE
University of Southern California
Tradespace and Affordability Framework
Center for Systems and Software Engineering
Get the Best from People
Make Tasks More Efficient
Staffing, Incentivizing, Teambuilding
Facilities, Support Services
Kaizen (continuous improvement)
Tools and Automation
Work and Oversight Streamlining
Collaboration Technology
Affordability
Improvements
and Tradeoffs
Lean and Agile Methods
Eliminate Tasks
Task Automation
Model-Based Product Generation
Eliminate Scrap, Rework
Simplify Products (KISS)
Early Risk and Defect Elimination
Evidence-Based Decision Gates
Modularity Around Sources of Change
Incremental, Evolutionary Development
Value-Based, Agile Process Maturity
Risk-Based Prototyping
Value-Based Capability Prioritization
Satisficing vs. Optimizing Performance
Reuse Components
Domain Engineering and Architecture
Composable Components,Services, COTS
Legacy System Repurposing
Reduce Operations, Support Costs
Automate Operations Elements
Design for Maintainability, Evolvability
Streamline Supply Chain
Anticipate, Prepare for Change
Value- and Architecture-Based
Tradeoffs and Balancing
31
Copyright
October©16,
USC-CSSE
2012
University of Southern California
Center for Systems and Software Engineering
Tradeoffs Among Cost, Schedule, and
Reliability, and Functionality: COCOMO II
9
(RELY, MTBF (hours))
8
(VL, 1)
Cost ($M)
7
(L, 10)
6
5
(N, 300)
4
(H, 10K)
3
(VH, 300K)
•For 100-KSLOC set of features
•Can “pick all three” with 77-KSLOC set of
features
2
1
-- Cost/Schedule/RELY:
“pick any two” points
0
0
10
20
30
40
50
Development Time (Months)
32
October 16, 2012
Copyright © USC-CSSE
University of Southern California
Center for Systems and Software Engineering
A Value-Priority Tradeoff Equalizer
October 16, 2012
Copyright © USC-CSSE
33
University of Southern California
Center for Systems and Software Engineering
Ilities in Tradespace Exploration: MIT
Changeability
Enabling Construct: Tradespace Networks
More changeable
(ie including flexible,
adaptable, scalable
and modifiable)
Colored by
outdegree
For this plot, Ĉ=C∞
Survivability
Enabling Construct: Epochs and Eras
Value Robustness
Set of Metrics
© 2012 Massachusetts Institute of
Technology
seari.mit.edu
34
University of Southern California
Center for Systems and Software Engineering
Architecture-Based Attribute Trades:
Flexibility
Arch.
Synergies
Conflicts
Flexibility
Example
(RT-18a)
Strategy
High module cohesion;
Low module coupling
Interoperability
Reliability
High Performance via
Tight coupling
Service-oriented architecture
Composability, Usability,
Testability
High Performance via
Tight coupling
Autonomous adaptive systems
Affordability via task automation;
Response time
Excess autonomy reduces
human Controllability
Modularization around sources
of change
Interoperability, Usability,
Reliability, Availability
Extra time on critical path of
Rapid Fielding
Multi-layered architecture
Reliability, Availability
Lower Performance due to layer
traversal overhead
Many built-in options, entry
points
Functionality, Accessibility
Reduced Usability via options
proliferation; harder to Secure
User programmability
Usability, Mission Effectiveness
Full programmability causes
Reliability, Safety, Security risks
Spare/expandable capacity
Performance, Reliability
Added cost
Product line architecture,
reusable components
Cost, Schedule, Reliability
Some loss of performance vs.
optimized stovepipes
October 16, 2012
Copyright © USC-CSSE
35
University of Southern California
Center for Systems and Software Engineering
Value/Risk-Based Tradespace Analysis
- Early Startup: Risk due to low dependability
- Commercial: Risk due to low dependability
- High Finance: Risk due to low dependability
- Risk due to market share erosion
Combined Risk Exposure
1
Market Share
Erosion
0.8
Early Startup
0.6
RE =
P(L) * S(L)
Sweet
Spot
0.4
Commercial
High Finance
0.2
0
VL
L
N
H
VH
RELY
COCOMO II:
0
12
22
34
54
Added % test time
COQUALMO:
1.0
.475
.24
.125
0.06
P(L)
Early Startup:
.33
.19
.11
.06
.03
S(L)
Commercial:
1.0
.56
.32
.18
.10
S(L)
High Finance:
3.0
1.68
.96
.54
.30
S(L)
Market Risk:
.008
.027
.30
1.0
36
.09
07/08/09
REm
©USC-CSSE
University of Southern California
Center for Systems and Software Engineering
Conclusions
• Affordability increasingly competition-critical
– Need to balance cost, schedule, performance, functionality
• Orthogonal framework helps tailor improvements
–
–
–
–
–
–
–
–
Getting the best from people
Making tasks more efficient
Eliminating tasks
Eliminating scrap and rework
Simplifying products
Reusing assets
Reducing operations and support costs
Value- and architecture-based tradeoffs and balancing
• No one-size-fits-all solution
October 16, 2012
Copyright © USC-CSSE
37
University of Southern California
Center for Systems and Software Engineering
Backup Charts
October 16, 2012
Copyright © USC-CSSE
38
University of Southern California
Center for Systems and Software Engineering
Magnitude of Overrun Problem: DoD
October 16, 2012
Copyright © USC-CSSE
39
University of Southern California
Center for Systems and Software Engineering
Magnitude of Overrun Problem:
Standish Surveys of Commercial Projects
Year
2000
2002
2004
2006
2008
Within budget and schedule
28
34
29
35
32
Prematurely cancelled
23
15
18
19
24
Budget or schedule overrun
49
51
53
46
44
October 16, 2012
Copyright © USC-CSSE
40
University of Southern California
Center for Systems and Software Engineering
Some Frequent Overrun Causes
• Conspiracy of Optimism
• Effects of First Budget Shortfall
– System Engineering
• Decoupling of Technical and Cost Analysis
– Overfocus on Performance, Security, Functionality
• Overfocus on Acquisition Cost
• Assumption of Stability
• Total vs. Incremental Commitment
October 16, 2012
Copyright © USC-CSSE
41
University of Southern California
Center for Systems and Software Engineering
The Conspiracy of Optimism and
The Cone of Uncertainty
October 16, 2012
Copyright © USC-CSSE
42
University of Southern California
Center for Systems and Software Engineering
Effects of First Budget Shortfall:
Added Cost of Weak System Engineering
Calibration of COCOMO II Architecture and Risk Resolution
(RESL) factor to 161 project data points
October 16, 2012
Copyright © USC-CSSE
43
University of Southern California
Center for Systems and Software Engineering
Assumption of Stability vs. Rapid Change
– Need evolutionary/incremental vs. one-shot development
4x
Uncertainties in competition,
technology, organizations, mission
priorities
2x
1.5x
1.25x
Relative
Cost Range
x
0.8x
0.67x
0.5x
0.25x
Concept of
Operation
Feasibility
Plans
and
Rqts.
Detail
Design
Spec.
Product
Design
Spec.
Rqts.
Spec.
Product
Design
Detail
Design
Accepted
Software
Devel. and
Test
Phases and Milestones
October 16, 2012
Copyright © USC-CSSE
44