COSYSMO Extension as a Proxy
Systems Cost Estimation
April 30, 2014
Reggie Cole
Lockheed Martin Senior Fellow
reggie.cole@lmco.com
Garry Roedler
Lockheed Martin Fellow
garry.j.roedler@lmco.com
1
Agenda
COSYSMO as a Proxy for System Cost Estimation
Deep Dive on the Proxy/Bias Function
Key Use Cases
Workshop Results and Recommendations
2
3
COSYSMO as a Proxy Estimator for System Cost
• Need for a Top-Down System Cost Model
– Better Buying Power 2.0 Mandate
• Perform ongoing should-cost analysis
– Better Buying Power 2.0 Implication
• Perform design-to-cost analysis and design for affordability
– There is Currently a Gap in Tools
• Primarily in early-stage analysis – when directions can still be
changed without significant repercussions
• Extending COSYSMO for System Costing
– Evaluate the Viability of Extending COSYSMO
• COSYSMO seems to have the right parameters
4
Cost Modeling Needs Change Over Time
in Terms of Speed and Accuracy
Detailed
Solution
Description
We Have a Good Selection of
Tools for Late-Stage Cost
Modeling
High-Level
Solution
Description
Increasingly
Refined
Information
About the
Solution
High-Level Solution
Assumptions
Problem-Space
Description
Cost Estimate ±
5%
Cost Estimate ±
10%
Cost Estimate ±
20%
Increasingly
Refined Cost
Estimate
Cost Estimate ±
25%
We Have Gaps in
Early-Stage Cost
Modeling
Increasing Effort and Cost-Modeling Expertise
5
SE Effort as a Proxy Measure of Overall
System Size and Complexity
• Proxy Measures
– Proxy measures are used when you cannot directly measure what you want to
measure – and when an indirect measure provides sufficient insight
– Proxy measures are often used in clinical studies since direct measurement is
often infeasible or can even alter the outcome
– It is not always possible to directly measure what you want to measure – or
directly estimate what you want to estimate
• System Engineering Effort is a Proxy Measure for System Cost
– There is strong evidence for the link between systems engineering effort and
program cost – dating back to a NASA study in the 1980s
– The optimal relationship between systems engineering effort and overall
program cost is 10% - 15%
– Industry has long used a parametric relationship between software cost and
systems engineering cost for software-intensive systems
– Systems engineering effort can be an effective proxy measure for overall
system cost
6
COSYSMO 2.0 Model Parameters Provide a Rich
Assessment of System Size and Complexity
Size Drivers
Number of System Requirements
Number of Major System Interfaces
Initial Estimate of System
Size
Reuse Factors
Managed Elements
Number of Critical Algorithms
Adopted Elements
Number of Operational Scenarios
Deleted Elements
Modified Elements
Cost Drivers
New Elements
Requirements Understanding
Architecture Understanding
Level of Service Requirements
Scaled Estimate of
System Size
Migration Complexity
Technology Risk
Level of Documentation Required
Diversity of Installed Platforms
Consolidated Cost Driver
Factor
Level of Design Recursion
Stakeholder Team Cohesion
Personnel / Team Capability
Personnel Experience / Continuity
Process Capability
Multisite Coordination
Level of Tool Support
Estimate of Systems Engineering
Effort…Also a Biased Proxy Estimator
for System Scope…And System Cost
7
Relationship Between SE Effort and Total Effort
Total Program Overrun
32 NASA Programs
200
Definition $
Definition Percent = ---------------------------------Target + Definition$
Program Overrun
180
160
NASA data supports a 10%-15% optimal
allocation of systems engineering effort as a
portion of overall program effort
Actual + Definition$
Program Overrun = ---------------------------------Target + Definition$
140
120
100
80
60
40
R2 = 0.5206
20
0
0
5
10
15
20
Definition Percent of Total Estimate
3.0
Actual/Planned Cost
2.6
INCOSE study on the value of systems
engineering also supports a 10%-15% optimal
allocation of systems engineering as a portion of
overall program effort
2.2
1.8
1.4
1.0
0%
4%
8%
12%
16%
20%
24%
0.6
28%
W. Gruhl, Lessons Learned, Cost/Schedule Assessment Guide,” Internal
Presentation, NASA Comptroller’s Office, 1992
E. Honour, “Understanding the Value of Systems Engineering,” INCOSE, 2004
SE Effort = SE Quality * SE Cost/Actual Cost
8
Putting It All Together
 Customer Requirements
 System Interfaces
 Major Algorithms
 Operational Scenarios
 Complexity Drivers (Problem/Solution)
 Requirements Understanding
 Architecture Understanding
 Level of Service Requirements
 Migration Complexity
 Technology Risk
 Documentation Needs
 Installations/Platform Diversity
 Levels of Recursion in the Design
Stakeholder Team Cohesion
 Personnel/Team Capability
 Personnel Experience/Continuity
 Process Capability
 Multisite Coordination
 Tool Support
 Reuse Factors (Solution Space)
 New
 Modified
 Deleted
 Adopted
 Managed
SE Effort is an estimator for
total system cost…but it is a
biased estimator
3.0
2.6
Actual/Planned Cost
Size Drivers (Problem Space)
Estimator Bias Function is Based on the WellEstablished Relationship Between SE Effort and
Overall Program Effort
2.2
1.8
1.4
1.0
0%
4%
8%
12%
16%
20%
24%
28%
0.6
SE Effort = SE Quality * SE Cost/Actual Cost
Estimation of Total
System Cost
9
Adjusting Our View of COSYSMO Parameters
Our view of the size drivers can be
preserved – their context doesn’t
change under COSYSMO extension
Our view of reuse requires the
most extreme adjustment – we are
not just talking about systems
engineering artifacts
Our view of the cost drivers needs
to be broadened to include all
aspects of the system, not just
systems engineering
10
Expanding Our View of Cost Drivers
Cost Drivers
Precedented systems and unprecedented
systems are fundamentally different
Requirements Understanding
Architecture Understanding
Level of Service Requirements
Migration Complexity
System modification and reuse have a
significant effect on some cost drivers
Technology Risk
Level of Documentation Required
Diversity of Installed Platforms
Level of Design Recursion
Stakeholder Team Cohesion
Personnel / Team Capability
Need to consider the entire team –
including subcontractors
Personnel Experience / Continuity
Process Capability
Multisite Coordination
Level of Tool Support
Need to consider the all processes and
tools
Expand to a view of all aspects of the system for the Cost Drivers
11
Expanding Our View of Reuse Factors
New Elements – These are elements that
need to be engineered and developed.
Just reusing systems engineering artifacts
is not sufficient.
Reuse Factors
New Elements
Modified Elements
Modified Elements – These are elements
that offer some form of reuse. Enhanced
COTS or reusable components that need
modification fall into this category.
For Requirements…
Think Functional Components
Adopted Elements – These are elements
that offer significant reuse with minimal
modification and do not require full
retesting. COTS typically falls into this
category.
For Algorithms…
Think Functional Components
Deleted Elements
For Interfaces…
Think Connection Points
Adopted Elements
Managed Elements
For Scenarios…
Think Implications to Both
Managed Elements – These are elements
that are already in the system and require
minimal regression testing. A previously
deployed element falls into this category.
Need to include the system elements, as well as the SE artifacts
12
Example
Consider the case of large C2 system. Initially developed 20 years ago, the system was unprecedented.
Twenty years later, a replacement system is needed. While the initial development was unprecedented,
the replacement system is not, which drives down the size drivers (through reuse) and cost drivers.
 Case 1 – Unprecedented System
 Case 2 – Developed Replacement System
 Case 3 – COST/GOTS-Based Replacement System
Case 2 - Replacement System (Developed)
4.00
500
2.00
0
0.00
Pessimistic
Expected
Requirements
Algorithms
Cost Driver Factor
Optimistic
0.60
800
600
0.40
400
0.20
200
0
0.00
Pessimistic
Expected
Optimistic
600
0.6
500
0.5
400
0.4
300
0.3
200
0.2
100
0.1
0
0
Pessimistic
Expected
Optimistic
System I/F
Requirements
System I/F
Requirements
System I/F
Scenarios
Algorithms
Scenarios
Algorithms
Scenarios
Cost Driver Factor
Similar Size Drivers – But Significantly
Different Cost Drivers
Cost Driver Factor
6.00
1000
0.80
1000
Size (Effective Requirements)
8.00
1500
1200
Case 3 - Replacement System (COTS/GOTS)
Cost Driver Factor
10.00
Size (Effective Requirements)
2000
Cost Driver Factor
Size (Effective Requirements)
Case 1 - Large Unprecedented System
Cost Driver Factor
Similar Cost Drivers – But
Significantly Different Size Drivers
13
Part 1 Wrap-Up
• The Approach Based on Well-Established Approaches
– COSYSMO provides the basis for estimation of systems
engineering effort – and a biased proxy estimator for overall
system cost
– There is a well-established relationship between systems
engineering effort and overall effort used to de-bias the
COSYSMO-modeled effort
• The Approach Can Improve System Cost Modeling
– It occupies an important niche – fully parametric system cost
modeling in the early stages of system definition
– It can serve as a powerful affordability analysis tool – supporting
rapid-turnaround analysis of alternatives
– But…it is not a replacement for existing models
14
15
Context of the Bias Function
Deeper Discussion of the Proxy/Bias Function is Necessary – As Well as a
Technique for Generating Cumulative Distribution of System Costs
16
The Bias Function
Cost System  EffortSE
FCal RateLabor
  StochasticAddersCost   Deter min isticAdder sCost
FConv
Where :
FCal : COSYSMO Calibratio n Factor (Determini stic)
FConv : Factor for Converting SE Effort to Total Program Effort (Stochasti c)
EffortSE : SE Effort Computed Using COSYSMO (Stochasti c)
RateLabor : Labor Rate (Stochasti c)
StochasticAddersCost : Additional Factors (e.g., travel, material, etc.) (Stochasti c)
Deter min isticAdder sCost : Additional Factors (G & A, ODC, MR, Fee, etc.) (Determini stic)
Variable
Type
Description
COSYSMO Calibration Factor
Deterministic Scalar Value
Organization-specific calibration factor
Effort Conversion Factor
Triangular Distributed Random Variable
Three-point estimate of factor to convert SE effort to total
program effort (nominally 0.08, 0.12 and 0.16)
SE Effort
Triangular Distributed Random Variable
Three-point estimate for SE effort, generated using COSYSMO
Labor Rate
Triangular Distributed Random Variable
Three-point estimate for composite labor rate
Material Costs
Triangular Distributed Random Variable
Three-point estimate for material costs
Travel Costs
Triangular Distributed Random Variable
Three-point estimate for travel costs
We are going to discuss each of these factors!
17
SE Effort (EffortSE)
COSYSMO provides a Proxy Estimate of the system cost. We
will not try to de-bias it right now….that is the next step.
Size Drivers (Problem Space)
 Customer Requirements
 System Interfaces
 Major Algorithms
 Operational Scenarios
 Complexity Drivers (Problem/Solution)
 Requirements Understanding
 Architecture Understanding
 Level of Service Requirements
 Migration Complexity
 Technology Risk
 Documentation Needs
 Installations/Platform Diversity
 Levels of Recursion in the Design
Stakeholder Team Cohesion
 Personnel/Team Capability
 Personnel Experience/Continuity
 Process Capability
 Multisite Coordination
 Tool Support
Since we want to perform Monte Carlo simulation of costs, we
would like to generate a distribution of the proxy costs.
Three different COSYMO scenarios – optimistic, expected &
pessimistic – provide the basis for a sampling distribution.
Optimistic
Expected
Pessimistic
 Reuse Factors (Solution Space)
 New
 Modified
 Deleted
 Adopted
 Managed
COSYSMO Estimate of Hours
Becomes the Parameters for Either
Triangular or Beta Pert Distribution
Triangular Distribution
Beta Pert Distribution
18
Effort Conversion Factor (FConv)
This is probably the most important factor in the bias function!
Total Program Overrun
32 NASA Programs
200
Program Overrun
Studies provide some insight into what the value should be
Definition $
Definition Percent = ---------------------------------Target + Definition$
180
160
Actual + Definition$
Program Overrun = ---------------------------------Target + Definition$
140
120
Heuristic approaches for determining SE costs for software
intensive systems are also consistent with these studies
100
80
60
40
R2 = 0.5206
20
0
0
5
10
15
20
All indications point to a range of 0.08 to 0.16 for FConv
Definition Percent of Total Estimate
And this range is consistent with all the data we’ve collected
to date…for relatively healthy programs
3.0
Actual/Planned Cost
2.6
And it provides the basis for our random variable parameters
2.2
1.8
1.4
Pessimistic
Expected
Optimistic
0.08
0.12
0.16
1.0
0%
4%
8%
12%
16%
20%
24%
28%
0.6
SE Effort = SE Quality * SE Cost/Actual Cost
Triangular Distribution
Beta Pert Distribution
19
Other Stochastic “Adder” Factors
•
Labor Costs
– Labor costs vary – especially in early stages – and needs to be treated as a random
variable
– Any number of distributions would probably be OK – Beta Pert would be a good default
– If hours are the item of interest rather than cost, this factor can be omitted
•
Material Costs
– Material costs vary – especially in early stages – and needs to be treated as a random
variable
– Any number of distributions would probably be OK – Beta Pert would be a good default
but there is at least one study that looks at using a normal distribution
– If hours are the item of interest rather than cost, this factor can be omitted
•
Travel Costs
– Travel costs vary – especially in early stages – and needs to be treated as a random
variable
– Any number of distributions would probably be OK – Beta Pert would be a good default
– If hours are the item of interest rather than cost, this factor can be omitted
•
Other
– Any number of other “stochastic adders” can be treated similarly
20
Other “Deterministic Adders”
• Some Factors Are Well Known
– To the Point They Can Be Considered Deterministic
– They are often set and apply across programs
– Examples include:
•
•
•
•
G&A Costs
Other Direct costs
Management Reserve
Fee
21
COSYSMO Calibration Factor
• Local Calibration is Important
– Keeps us from over-tuning the Effort Conversion
Factor
• This Also Serves as a Type of Organizational
Efficiency Factor
– Can vary across organizations within an enterprise
• It is a Simple Scalar Factor
– Optimally, it should be 1.0
22
Calibration for Proxy Estimation
• It is Not Necessary to Revalidate or
Recalibrate COSYSMO
– The strength of this approach is that it rests on the
COSYSMO foundation
• It is Necessary to Validate and Calibrate the
Bias Function
– Important to validate the relationship between
system costs and systems engineering costs
– Important to calibrate the COSYSMO Calibration
Function
23
Our Current Approach for Validation
• Use a Few Long-Running Programs
– They tend to have good data collection and good
process discipline – so the data is reliable
• Treat Each Major Release as a Separate Entity
– That really allows us to dig into reuse between
releases
– It is necessary to collect data on reuse – we found
that to be a little challenging
– Use some releases for validation and others for
calibration
24
Revisiting the Overall Approach
Construct the
COSYSMO
Scenarios
1
Define
Parameters for
2 Remaining
Factors in Bias
Function
Run Monte Carlo
Simulations and
Generate Cumulative
Distribution of Costs
3
25
Revisiting the Overall Approach
11. Construct
the COSYSMO
Scenarios
Optimistic
Expected
Pessimistic
Requirements
Requirements
Baseline
Interfaces
Algorithms
Architecture
Baseline
22. Define Parameters for
Remaining Factors in Bias
Function
- Effort Conversion Factor
- Stochastic Adder Factors
- Deterministic Adder Factors
3.0
Actual/Planned Cost
Scenarios
FCal RateLabor
  StochasticAddersCost   Deter min isticAdder sCost
FConv
2.6
Cost System  EffortSE
2.2
Where :
FCal : COSYSMO Calibratio n Factor (Determini stic)
33. Run Monte Carlo
Simulations and
Generate Cumulative
Distribution of Costs
Bias Function
Risky Range
Target
Cost
Target
Reserve
FConv : Factor for Converting SE Effort to Total Program Effort (Stochasti c)
1.8
EffortSE : SE Effort Computed Using COSYSMO (Stochasti c)
1.4
StochasticAddersCost : Additional Factors (e.g., travel, material, etc.) (Stochasti c)
RateLabor : Labor Rate (Stochasti c)
Deter min isticAdder sCost : Additional Factors (G & A, ODC, MR, Fee, etc.) (Determini stic)
1.0
0%
4%
8%
12%
16%
20%
24%
0.6
SE Effort = SE Quality * SE Cost/Actual Cost
28%
26
Part 2 Wrap-Up
• The Basic Bias Function
– Concerns with the basic bias function
– Suggested improvements on the basic bias
function
• The Approach for Validation and Calibration
– Concerns with the basic approach
– Suggested improvements for validation and
calibration
27
28
The Key Use Cases
• Should-Cost Analysis
– Establishing a should-cost for which cost estimates
from bidders can be evaluated
– Establishing a DTC target for performing DTC analysis
• Design-to-Cost Analysis
– Performing cost vs. capability trades as way to provide
more affordable solutions
• Analysis of Alternatives
– Evaluating alternative solution strategies
– It’s not just about the problem space – the solution
space can be evaluated too
29
Should-Cost Analysis
• Goal is to Establish a Target Cost
– Can also be a cost range
– Usually performed very early in the lifecycle
• Approach
– Given a requirements baseline, use the extended
COSYSMO approach to estimate the cost
– Use “plug” numbers for adders
• e.g., labor, material, fee, etc.
30
Part 3 – Wrap-Up
• There Are Three Very Important Use Cases
– Should-Cost Analysis
– DTC Analysis
– Analysis of Alternatives
• There Are At Least a Couple More
– Change Evaluation for Operational Systems
– Scope Creep Monitoring
– Probably Others We Haven’t Thought Of
• Discussion on Use Cases
31
32
Conclusions and Recommendations (1 of 3)
• Validity of overall approach
– Unanimous support – approach is valid and should
continue to be developed and refined for wider
application
– Feedback was all focused on ensuring all factors
had been considered and areas for refinement –
no discussion resulted in conclusion that the
approach had major issues
– Appropriately uses the concepts of COSYSMO and
tailors the perspective for systems
33
Conclusions and Recommendations (2 of 3)
• Improvement of Approach
– Best distribution to use? Cumulative, frequency, or other?
– Preference is to not change the Scale Factor
• Want to retain as close to COSYSMO as possible – ready to leverage
COSYSMO 3
– Review and refine the bias function and calibration of the bias
function
• May consider adding in other COCOMO factors, as applicable
• May need to establish rules for when to leverage a HW model using the
same approach to use as input for HW, when highly HW intensive
• May need to consider calibration for different types/classes of systems
• Bottom line – User needs to consider what tailoring/adaptation of the
bias function is needed for the system application
– Determine under what conditions it OK to eliminate a cost factor
– Additional use cases?
• Should include Impact Analysis / Change Evaluation
34
Conclusions and Recommendations (3 of 3)
• Thoughts on moving forward
– Form small, diverse working group
– Periodic meetings (USC ARR, COCOMO Forum,
INCOSE IW, …)
– Validation through pilots
• Use on completed programs
– Access the more robust data points from COSYSMO data
– Comparison of estimates
• Use on non-LM programs
– Incorporate feedback from validation and refine
– Can this support the SERC project for COSATMO?
35
BACK-UP CHARTS
36
COSYSMO Setup
Optimistic
Assume a mature supplier,
experienced in the domain,
minimal scope creep, and
cooperative stakeholders
Expected
Assume an average supplier, with
some experience in the domain,
average scope creep, and generally
cooperative stakeholders
Pessimistic
Assume a new supplier who will have
some challenges, a fair amount of
baseline volatility, and stakeholders
who need to be corralled
37
Cost Analysis Approach & Results
Optimistic
Expected
Pessimistic
Requirements
Requirements
Baseline
Interfaces
Algorithms
Architecture
Baseline
Scenarios
Use a “Plug” Number for Adders




Anticipated Distribution of Labor Rates
Anticipated Distribution of Material Costs
Anticipated Travel Costs
Anticipated Supplier Fees
Bias Function
Risky Range
Target
Cost
Target
Reserve
38
Design-to-Cost Analysis
• Goal
– The goal is, given a target cost, design a solution
to meet the target cost
• Approach
– Given a requirements baseline, identify and
prioritize capabilities
– Use the extended COSYSMO approach to estimate
the cost for each capability
– Evaluate the “cost for capability” against the
capability priority
39
Capability Decomposition
Problem:
TELECOM Operations Support System
Major Upgrade, Budget Limited to $40M
Major Capabilities
1 – Service Provisioning
2 – Service Order Orchestration
Requirements
Baseline
3 – Pre-Provisioning Support
4 – Service Order Validation
5 – Service Activation
Architecture
Baseline
6 – Network Management
7 – Dashboards for Awareness & Reporting
8 – Service Catalog Management
9 – Service-to-Subscriber Mapping
10 – Auto-Discovery
Evaluate Each Capability with
Respect to Cost and Enterprise
Utility to Determine Best-Value
Baseline
11 – Service Reconciliation
12 – Billing - Service Order Creation
13 – Billing - Trouble Ticketing
14– Service Coverage Mapping
15– Resource Management
40
Mission Utility Analysis of Capabilities
1 – Service Provisioning
Very
High
Operational Burden
High
8
7
1
2 – Service Order Orchestration
3 – Pre-Provisioning Support
6
4 – Service Order Validation
Mod
High
2
11
9
13
10
14
Med
4
5 – Service Activation
6 – Network Management
3
5
12
7 – Dashboards for Awareness & Reporting
8 – Service Catalog Management
Mod
Low
9 – Service-to-Subscriber Mapping
10 – Auto-Discovery
Low
15
Low
Mod Low
Med
11 – Service Reconciliation
Mod High
High
Very High
Operational Tempo
12 – Billing - Service Order Creation
13 – Billing - Trouble Ticketing
14– Service Coverage Mapping
Operational Tempo is a combination of the frequency with which
the capability is used operationally and its criticality to the overall
mission.
Operational Burden is a combination of the skill level required for
the capability and the time it takes to perform.
15– Resource Management
41
Cost Analysis Approach
Requirements
Requirements
Baseline
Optimistic
Expected
Pessimistic
Interfaces
Three COSYSMO Scenarios
for Each Capability
Algorithms
Architecture
Baseline
Scenarios
Use a Common Number for Adders




Anticipated Distribution of Labor Rates
Anticipated Distribution of Material Costs
Anticipated Travel Costs
Anticipated Supplier Fees
Bias Function
A Cost Curve is Produced
for Each Capability
Take the 80% Confidence
Cost as the Capability Cost
42
Simplified Cost Analysis Approach
Requirements
Requirements
Baseline
Expected
Interfaces
Algorithms
Architecture
Baseline
A Simpler Approach is Slightly Less
Robust…Bust Still Just as Valid
A Single COSYSMO
Scenario for Each
Capability
Scenarios
Use a Common Number for Adders




Anticipated Distribution of Labor Rates
Anticipated Distribution of Material Costs
Anticipated Travel Costs
Anticipated Supplier Fees
Bias Function
A Cost Curve is Produced
for Each Capability
Take the 80% Confidence
Cost as the Capability Cost
43
Analysis Results
Capabilities Sorted By Cost
Total Cost = $81.3M
Colors Map Mission Utility Priorities
 RED = High
 YELLOW = Medium
 Green = Low
Capabilities Sorted By Mission Utility
(Cost for Priority 1 Capabilities Only) $34.4M
(Capabilities at DTC Target) $39.9M
(Cost for Priority 1&2 Capabilities) $64.1M
(Cost for All Capabilities) $81.3M
44
Analysis of Alternatives
• Goal
– The goal is, given a requirements baseline, determine
the most affordable solution that meets requirements
• Approach
– Given a requirements baseline, identify possible
solution alternatives
– Use the extended COSYSMO approach to estimate the
cost for each capability
– Evaluate the cost to find the most affordable
alternative that meets all requirements
45
Case Study Overview
• 20-Year Old C2 System
– The system was unprecedented at the time
– Twenty years later, a replacement system is needed
due to obsolescence and needed changes
• Alternative Solutions
– Full Replacement System
• Develop and deploy a new replacement system
– COTS/GOTS/NDI-Based Replacement System
• Use a combination existing non-obsolete components and
COTS components
• Modify components as necessary to meet requirements
46
The Two Key Alternatives
Newly Developed System
 Ground-Up Development of System – Requirements
Refinement, Architecture, Detailed Design – Soup-to-Nuts
 But…It is No Longer an Unprecedented System – So
Requirements/Architecture Understanding, etc. Are High
Refurbished System Using Modified NDI
 Use of NDI is Maximized – Additional Components
Developed as Necessary to Meet Requirements
 Reuse Considerations Drive This Alternative
47
COSYSMO Size and Cost Drivers
 Size Drivers for the Two Alternatives Are Largely the Same
 Cost Drivers for the Two Alternatives Are Very Similar – With
a Few Notable Exceptions
Cost Drivers
Requirements Understanding
Architecture Understanding
Level of Service Requirements
Migration Complexity
Technology Risk
Level of Documentation Required
System modification
and reuse have an
effect on some cost
drivers
Diversity of Installed Platforms
Level of Design Recursion
Stakeholder Team Cohesion
Personnel / Team Capability
Personnel Experience / Continuity
Process Capability
Multisite Coordination
Level of Tool Support
48
COSYSMO Reuse Factors
Reuse Factors
New Elements – These are elements that
need to be engineered and developed.
Just reusing systems engineering artifacts
is not sufficient.
Most Elements Are New
for the Development
Alternative
Modified Elements – These are elements
that offer some form of reuse. Enhanced
COTS or reusable components that need
modification fall into this category.
Most Elements Are Modified
for the NDI Alternative
Deleted Elements – For the NDI
Alternative, Some Elements May Need to
Be Deleted/Retired
Elements That Need to Be
Retired Should Be Treated as
Deleted Elements
Adopted Elements – These are elements
that offer significant reuse with minimal
modification and do not require full
retesting. COTS typically falls into this
category.
Elements That Are “Wrapped”
Can Be Treated as Adopted for
the NDI Alternative
Managed Elements – These are elements
that are already in the system and require
minimal regression testing. A previously
deployed element falls into this category.
Elements That Stay in Place
Without Modification (or
Wrapping) Can Be Treated as
Managed
New Elements
Modified Elements
Deleted Elements
Adopted Elements
Managed Elements
49
Cost Analysis Approach
Requirements
Requirements
Baseline
Optimistic
Expected
Pessimistic
Interfaces
Three COSYSMO Scenarios
for Each Alternative
Algorithms
Architecture
Baseline
Scenarios
Use a Common Number for Adders




Anticipated Distribution of Labor Rates
Anticipated Distribution of Material Costs
Anticipated Travel Costs
Anticipated Supplier Fees
Bias Function
A Cost Curve is Produced
for Each Capability
Take the 80% Confidence
Cost as the Capability Cost
50
End of Workshop Wrap-Up
• Validity of the Overall Approach
• Improvement of the Approach
• Thoughts on Moving Forward
51
52