UNCLASSIFIED
07T-1552.01
Load Bearing Walls: Early Sizing Estimation In The
NOSTROMO Tool (U)
Brandon Gautney
Dynetics Program Software Support
brandon.gautney@dynetics.com
Dan Strickland
Dynetics Program Software Support
daniel.strickland@dynetics.com
Approved for Public Release 07-MDA-2965 (26 OCT 07)
Ashley Mathis
THAAD Project Office System Software Engineering
ashley.mathis@mda.mil
UNCLASSIFIED
DISTRIBUTION STATEMENT A – Approved for public release; distribution is unlimited.
UNCLASSIFIED
07T-1552.02
Overview (U)
• Background
• Area of Opportunity
• Heuristic Model
• Subcomponent Weight
• Complexity Weight
• NOSTROMO Concept
• NOSTROMO Additions
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
2
UNCLASSIFIED
07T-1552.03
Background (U)
• Size estimation is the cornerstone to software development cost
estimation and a mandatory input to most calibrated estimation
models
• Accurately estimating software size early in development would
provide better cost estimates
• Some organizations don’t have large databases of completed
projects to estimate by analogy
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
3
UNCLASSIFIED
07T-1552.04
Boehm Horn Curve (U)
• Size (and effort) should
converge over time
relative to development
phase
• During Concept of
Operation, estimates can
be 50% to 200% of the
actual final size
[Ref (1) - SOFTWARE COST ESTIMATION WITH COCOMO II (P 10) - BOEHM 2000]
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
4
UNCLASSIFIED
07T-1552.05
Area of Opportunity (U)
• Area between Concept
of Operations and
Requirements
Specification when early
size estimation is key
4x
2x
• Early sizing method
needs to have accuracy
within this area
1.5x
1.25x
x
0.5x
0.25x
Approved for Public Release 07-MDA-2965 (26 OCT 07)
Concept of
Operations
Requirements
Spec
Preliminary
Design Spec.
UNCLASSIFIED
Detailed
Design Spec.
Qualified
Software
5
UNCLASSIFIED
07T-1552.06
Roetzheim Heuristic Model (U)
Approximate Value for Function Points (FP):
FPs = (ValueSCOPE + ValueCLASS + ValueTYPE) 2.35
Function
Object
Object Library
Proof of Concept
Evolutionary Prototype
Internal Application
External Application
Shrink-Wrap Application
Component of System
New System
Compound System
1
2
4
5
6
8
9
10
11
12
13
Project Scope Table
Individual Use
Shareware
Academic/Engineering
Single Location -Internal
Multilocation - Internal
Contract Project - Civilian
Contract Project - Local Government
Marketed Commercially
State Government
State Government - Federally Funded
Federal Project
Military Project
Project Class Table
1
2
3
5
6
7
8
9
11
13
14
15
No Programming (4GL/Drag and Drop)
Batch
3GL Programming
Embedded - Single Board
Database Oriented
Client - Server
Mathematical
Systems
Communications
Process Control
Embedded - Multi Board
Embedded - Complete System
Social Service
1
2
4
5
6
8
9
10
11
12
13
14
15
Project Type Table
Roetzheim, William. “Estimating and Managing Project
Scope for New Development.”Cross Talk April 2005
• Project taxonomies are a method for estimating Function Points (FP)
• Function Points are a language independent approach for estimating software
development effort
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
6
UNCLASSIFIED
07T-1552.07
Heuristic Model Results
Projects Parameters
Project 1
Project 2
Project 3
Language
Project Scope
Project Class
Project Type
C++
Ada 95
C++
Component of System
Component of System
Component of System
Military Project
Military Project
Military Project
Systems
Embedded - Multi Board
Embedded - Multi Board
Results
Project 1
Project 2
Project 3
Total
Actual New KSLOC
Estimated New KSLOC
Fraction
Delta
75.00
403.65
455.75
934.40
240.76
268.66
290.59
800.00
3.21
0.67
0.64
0.86
221%
-33%
-36%
-14%
• Most large, complex programs use SLOC as system of delivery for their
sizing metrics
• “Backfiring” is a method for converting FPs to an estimated corresponding
number of Software Lines of Code (SLOC)
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
7
UNCLASSIFIED
07T-1552.08
Heuristic Model Results (cont’d)
4x
Project
Totals
Area of Opportunity
1
2x
1.5x
1.25x
x
2
3
0.5x
0.25x
Concept of
Operations
Requirements
Spec
Preliminary
Design Spec.
Detailed
Design Spec.
Qualified
Software
• Projects 2 & 3 are within acceptable boundaries of the Area of Opportunity.
• Project estimated totals are within 14% of the actual size.
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
8
UNCLASSIFIED
07T-1552.09
Subcomponents (U)
Subcomponents
Description
Weight
Few
Nominal
Many
1 - 4 Software Subcomponents
5 - 8 Software Subcomponents
9+ Software Subcomponents
0.67
1.00
1.33
Approximate Value for Function Points (FP):
FPs = (ValueSCOPE + ValueCLASS + ValueTYPE) 2.35 * (WeightSUBCOMPONENT)
Project Parameters
Project 1
Project 2
Project 3
Language
Project Scope
Project Class
Project Type
Subcomponents
C++
Ada 95
C++
Component of System
Component of System
Component of System
Military Project
Military Project
Military Project
Systems
Embedded - Multi Board
Embedded - Multi Board
Few
Many
Nominal
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
9
UNCLASSIFIED
07T-1552.10
Heuristic Model With Subcomponents Results
Actual New KSLOC
Estimated New KSLOC
Fraction
Delta
75.00
403.65
455.75
934.40
161.31
357.31
290.59
809.21
2.15
0.89
0.64
0.87
115%
-11%
-36%
-13%
Project 1
Project 2
Project 3
Total
4x
Project
Totals
Area of Opportunity
1
2x
1
1.5x
1.25x
x
2
2
3 3
0.5x
0.25x
Concept of
Operations
Requirements
Spec
Preliminary
Design Spec.
Detailed
Design Spec.
Qualified
Software
• Heuristic Model performance improves when coupled with Subcomponent Weight.
• Projects 2 & 3 are within acceptable boundaries of the Area of Opportunity.
• Project estimated totals are within 13% of the actual size.
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
10
UNCLASSIFIED
07T-1552.11
Complexity (U)
Complexity Adjustments
Problem
Complexity
1-5
Sum of Problem, Data,
and Code Complexity Scores
Code Size
Adjustment Calculators
3
4
5
6
7
8
9
10
11
12
13
14
15
0.70
0.75
0.80
0.85
0.90
0.95
1.00
1.05
1.10
1.15
1.20
1.25
1.30
1-5
Code
Complexity
1-5
Data
Complexity
Jones, Capers T. Estimating Software Cost. New York: McGraw Hill,1998.
Approximate Value for Function Points (FP):
FPs = (ValueSCOPE + ValueCLASS + ValueTYPE) 2.35 * (WeightCOMPLEXITY)
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
11
UNCLASSIFIED
07T-1552.12
Heuristic Model With Complexity Results
Project Parameters
Project 1
Project 2
Project 3
Problem Complexity
Code Complexity
Majority of simple algorithms and calculations
Some difficult and complex calculations
Some difficult and complex calculations
Well Structured (small models and simple paths)
Fair structure, but some complex paths or models
Fair structure, but some complex paths or models
Data Complexity
Sum of Complexity
Complexity Adjustment
Multiple files, switches and data interactions
Complex data elements and complex data interactions
Very complex data elements and complex data interactions
8
12
13
0.95
1.15
1.20
Results
Project 1
Project 2
Project 3
Total
Actual New KSLOC
Estimated New KSLOC
Fraction
Delta
75.00
403.65
455.75
934.40
228.72
308.95
348.71
886.38
3.05
0.77
0.77
0.95
205%
-23%
-23%
-5%
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
12
UNCLASSIFIED
07T-1552.13
Heuristic Model With Complexity Results
(cont’d)
4x
Project
Totals
Area of Opportunity
1
1
2x
1.5x
1.25x
x
2 3
2
3
0.5x
0.25x
•
•
•
Concept of
Operations
Requirements
Spec
Preliminary
Design Spec.
Detailed
Design Spec.
Qualified
Software
Heuristic Model performance is slightly improved when coupled with Complexity Weight.
Projects 2 & 3 are within acceptable boundaries of the Area of Opportunity.
Project estimated totals are within 5% of the actual size.
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
13
UNCLASSIFIED
07T-1552.14
Complete Heuristic Model
- Combined Heuristic Model with Subcomponent Weight and Complexity Weight
Project Parameters
Project 1
Project 2
Project 3
Language
Project Scope
Project Class
Project Type
C++
Ada 95
C++
Component of System
Component of System
Component of System
Military Project
Military Project
Military Project
Systems
Embedded - Multi Board
Embedded - Multi Board
Subcomponents
Complexity Adjustment
Few
Nominal
Many
0.95
1.15
1.20
Approximate Value for Function Points (FP):
FPs = (ValueSCOPE + ValueCLASS + ValueTYPE) 2.35 * (WeightSUBCOMPONENT) * (WeightCOMPLEXITY)
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
14
UNCLASSIFIED
07T-1552.15
Complete Heuristic Model Results
- Combined Heuristic Model with Subcomponent Weight and Complexity Weight
Actual New KSLOC
Estimated New KSLOC
Fraction
Delta
75.00
403.65
455.75
934.40
153.24
410.91
348.71
965.38
2.04
1.02
0.77
1.03
104%
2%
-23%
3%
Project 1
Project 2
Project 3
Total
4x
Project
Totals
Area of Opportunity
1
2x
1
1.5x
1.25x
x
2
3
2
3
0.5x
0.25x
•
•
•
Concept of
Operations
Requirements
Spec
Preliminary
Design Spec.
Detailed
Design Spec.
Qualified
Software
Heuristic Model performance improves when coupled with Subcomponent Weight and Complexity Weight.
Projects 2 & 3 are within acceptable boundaries of the Area of Opportunity.
Project estimated totals are within 3% of the actual size.
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
15
UNCLASSIFIED
07T-1552.16
Original NOSTROMO Concept (U)
Notional Obscurity STatistical Risk Observation MOdel
DATA
PAGE
NEWT
NOSTROMO
REPORTS
• NOSTROMO first takes normal COCOMO II inputs
• NOSTROMO accounts for uncertainty in the settings
• NOSTROMO uses Monte Carlo to simulate hundreds of
COCOMO II estimates using the inputs and uncertainties
• NEWT – NOSTROMO Entry Writing Tool – a Delphi
polling tool that captures Uncertainty
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
16
UNCLASSIFIED
07T-1552.17
New NOSTROMO Concept (U)
PESSIMISM
AGC
CALCULATOR
NEWT
DATA
PAGE
NOSTROMO
DEFAULT
UNCERTAINTY
EARLY SIZE
ESTIMATION
REPORTS
COMBINATION
NOSTROMO 0.3.16 offers:
• multiple methods for inputting Uncertainty
• combination of subcomponents
• AGC methodology
• Pessimistic SLOC estimation
• Early Size Estimation
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
17
UNCLASSIFIED
07T-1552.18
NOSTROMO Input Sheet – Notional Data
(U)
SI Name
Input
Software Item NamePREC
Scale Factors
Cost Drivers
Iterations
Putnam Productivity
Uncertainty
UncVal
Low Uncertainty
2
2
FLEX
RESL
TEAM
Low Uncertainty
Low Uncertainty
2
2
2
2
2
2
PMAT
Low Uncertainty
2
2
284161 Plans and Requirements 2
2
REVL
Low Uncertainty
2
2
ADAPTED CODE
Low Uncertainty
2
2
% DESIGN MOD
Low Uncertainty
2
2
% CODE MOD
Low Uncertainty
2
2
% TEST MOD
Low Uncertainty
2
2
Assessment and Adaptation
Low Uncertainty
2
2
Software Understanding
Low Uncertainty
2
2
Unfamiliarity with Software
Low Uncertainty
2
2
TIME
Low Uncertainty
2
2
STOR
Low Uncertainty
2
2
PVOL
Low Uncertainty
2
2
RELY
Low Uncertainty
2
2
DATA
Low Uncertainty
2
2
CPLX
Low Uncertainty
2
2
RUSE
Low Uncertainty
2
2
DOCU
Low Uncertainty
2
2
ACAP
Low Uncertainty
2
2
APEX
Low Uncertainty
2
2
PCAP
Low Uncertainty
2
2
PLEX
Low Uncertainty
2
2
LTEX
Low Uncertainty
2
2
PCON
Low Uncertainty
2
2
TOOL
Low Uncertainty
2
2
SITE
Low Uncertainty
2
2
SCED
Low Uncertainty
2
2
NEW CODE
Code Size
Rating
Number of Iterations
Putnam Technology Rating
Approved for Public Release 07-MDA-2965 (26 OCT 07)
Low Uncertainty
Clear Page
AGC Calculator
NEWT
Function Buttons
FALSE
Run SI
Combine Project
Pessimism
Worksheet
Default
Uncertainty
Delete Project
Early Sizing
PESSIMISM
Early Size Estimation
Pessimism Toggle
Conditional
Formatting
500
Low Uncertainty
2
UNCLASSIFIED
18
UNCLASSIFIED
NOSTROMO Early Size Estimation Worksheet –
Notional Data (U)
07T-1552.19
Early
Sizing
Language
Project Scope
Heuristic
Project Class
Model
Project Type
Ada 95
Component of System
Military Project
Communications
Weight Subcomponents
Many (9+ Subcomponents)
Problem Complexity
Weight Code Complexity
Data Complexity
Majority of simple algorithms and simple calculations
Well structured (small modules and simple paths)
Several data elements, but simple data relationships
Approximate Value for FPs:
Estimated New SLOC:
5,799
284,161
Subcomponent TRUE
Weight
1.33
Complexity Weight
TRUE
0.9
Copy to Main
Return to Main
Instructions: Enter Projected development language, heuristic model data, subcomponent data, and complexity data in drop down boxes above.
To use weighting, click appropriate box(es). Select "Add To Estimate" to use Early Sloc estimate or "Return" to return to data sheet without.
• Allows the user the option to use the Weighted Subcomponents and Weighted
Complexity with the Heuristic Model in NOSTROMO to calculate Function
Points.
• Heuristic Model estimates the number of Function Points and Backfires them
into SLOC.
• Allows the user to copy the results into NOSTROMO Input Sheet.
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
19
UNCLASSIFIED
07T-1552.20
NOSTROMO Outputs – Notional Data (U)
410 KSLOC – ESLOC vs. Effort
410 KSLOC – Estimated SLOC
Effort
6637.14
5345.51
4763.02
3927.86
3745.03
3436.15
3170.52
2925.42
2559.20
2426.91
2110.46
1880.49
1514.53
3028.34
Schedule
58.47
54.48
52.47
49.27
48.51
47.17
45.95
44.76
42.85
42.11
40.24
38.75
36.11
45.27
Size
Productivity EAF
1054990
158.95
1.60
890723
166.63
1.44
813874
170.87
1.36
699971
178.21
1.24
674359
180.07
1.21
630457
183.48
1.16
592011
186.72
1.11
555909
190.03
1.07
500702
195.65
1.00
480344
197.92
0.97
430628
204.04
0.91
393474
209.24
0.86
332209
219.35
0.77
603525
199.29
1.04
Language
Project Scope
Heuristic
Project Class
Model
Project Type
Project _Data - ESLOC vs. Effort
8000
7000
6000
Effort
Percentile
99%
95%
90%
75%
70%
60%
50%
40%
25%
20%
10%
5%
1%
Default
5000
4000
3000
2000
1000
0
0
200000 400000 600000 800000 1E+06 1E+06
ESLOC
Ada 95
Component of System
Military Project
Embedded - Multi Board
Weight
Subcomponents
Many (9+ Subcomponents)
Weight
Problem Complexity
Code Complexity
Data Complexity
Some difficult and complex calculations
Fair structure, but some complex paths or modules
Complex data elements and complex data interactions
Approximate Value for FPs:
Estimated New SLOC:
8,386
410,911
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
20
UNCLASSIFIED
07T-1552.21
Demonstration
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
21
UNCLASSIFIED
Future NOSTROMO Additions and Efforts
(U)
07T-1552.22
• Addition of Exception Conditions from Ray Madachy’s Expert COCOMO
• Transition to an application environment (MS-Access or .NET)
• Expansion of the NOSTROMO tool to address other COCOMO Family
models with uncertainties
•
•
COSYSMO – especially relevant in development of complex DoD systems
•
COQUALMO – Defect Prediction Models
COSOSIMO – System of Systems models are becoming increasingly prevalent
in DoD efforts
• Expansion of the NOSTROMO methodology and tool into Readiness Level
models
•
•
Software Readiness Levels (SWRL)
Technology Program Management Model (TPMM)
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
22
UNCLASSIFIED
07T-1552.23
Conclusion (U)
• Estimating software size accurately during the Area of Opportunity will
increase the fidelity of the estimate
• When analogous data is unavailable or unreliable, use of a heuristic
model can produce accurate metrics for size
• New NOSTROMO methodology incorporates code size estimate heuristic
model to provide better answers earlier
Early and accurate estimation of software size metrics
provides a method of true software cost estimation
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
23
UNCLASSIFIED
07T-1552.24
BACKUP
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
24
UNCLASSIFIED
07T-1552.25
Distributions of Uncertainty (U)
Level 1 - Certain
NOSTROMO assumes ceiling
and floor limits of the highest
and lowest default values for
each COCOMO II Scale Factor
and Cost Driver
Level 2 - Low Uncertainty - Normal
Level 3 - Medium Uncertainty - Triangle
Level 4 - High Uncertainty - Uniform
Setting n
Setting n+1
Setting n-1
NOSTROMO changes the distribution and sets the high and low
points based on the level of uncertainty with each factor
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
25
UNCLASSIFIED
07T-1552.26
NOSTROMO Application (U)
• Microsoft Excel worksheet with attached macros and Visual
Basic code
• Currently on version 0.3.16 (as of 10/17/07)
• Does not use outside applications for Monte Carlo simulation
(standard random number generation from Visual basic)
• Limited testing performed on a proof-of-concept application
• Suggest limiting Monte Carlo run size to 500-2000 runs
• Generates two pages of output – Data and Charts
• Data page contains all outputs from Monte Carlo simulation,
charting data, and histograms
• Charts page contains Confidence Intervals for output of
COCOMO II and Putnam models, charts, and histograms
• NOSTROMO – Data Sheet and CM Control Sheet are
Developer pages and should not be removed
Approved for Public Release 07-MDA-2965 (26 OCT 07)
UNCLASSIFIED
26