Prof. Morteza Anvari
1
Cost Analysis Requirement
MS I
MS 0
O
L
D
Concept
Exploration
MS II
MS III
Program Definition
&
Risk Reduction
Engineering &
Manufacturing
Development
(EMD)
Production,
Fielding/
Deployment
Cost Estimates
Needed
A
N
E
W
C
B
Concept
Exploration
Component
Advanced
Development
Review
Concept & Technology
Development
Pre-Systems
Acquisition
System
Integration
System
Demo
Production
Readiness & LRIP
Review
System Development&
Demonstration
Rate Production &
Deployment
Review
Support
Production & Deployment
Systems Acquisition
(Engineering and Manufacturing
Development, Demonstration, LRIP &
Production)
Sustainment
2
Cost Estimating Process
Definition Data Collect
Planning
Normalize
Estimate
Formulation
Review/
Presentation
Final
Document
DOCUMENTATION
3
Cost Analysis Model
Data/CERs
LCC Estimates
Cost Drivers
Develop Cost
Estimate
Structure
and WBS
Start
Establish
Ground
Rules and
Assumptions
Compile
Data Base/
CERs/Models
Prepare Cost
Estimates
for Each
Element
Test Total
System
Estimate
Engineering
Analogy
Parametrics
Expert Opinion
Reasonableness
Sensitivity
Analysis
Cost-Risk
Assessment
Prepare
Documentation
4
Situation
• You have just been tasked to develop a cost
estimate, that is, a professional opinion
about the cost of an item, a service or a
thing.
• Let’s discuss a process for organizing and
developing this estimate.
5
Definition and Planning
• Influences the success of the estimate
• Understanding the requirements and how
you approach the process will establish the
guidelines and procedures for the estimate.
• Ask lots of questions…They help you
understand the requirement.
6
Questions
• Why is this cost estimate needed?
• What decisions are pending on the results of this
estimate?
• Will the estimate be briefed and to whom?
• Will the results be incorporated into some
document?
• What does the recipient expect to have included or
excluded?
• What excursions or variations from the baseline
are anticipated?
7
Questions Continued
• What are the program and funding constraints
especially if the program is a Joint Program?
• What are the time constraints for this estimate?
• What is the acquisition phase of the program?
• Is the program definition mature?
• Does technology exist today to design, develop, test
and manufacture the system?
• What is the interrelationship with other systems?
• Are there previous contracts? How many? What type?
• How have the contractors performed to date?
8
Definition and Planning
Know Purpose of the Estimate
• Main purposes of estimates:
– Budget Formulation
– Comparative Studies
– Source Selection
9
Purpose of Estimate
Budget Formulation Estimates
•
•
•
•
•
•
•
Program Office Estimate (POE)
Component Cost Analysis (CCA)
Independent Cost Estimate (ICE)
What-if exercises
Rough Order of Magnitude (ROM)
Should Cost Estimates
CAIV
10
Purpose of Estimate
Comparative Studies Estimates
• Making cost & benefit comparisons between
alternatives
– Economic Analysis (EA)
– Analysis of Alternatives (AoA)
– Force Structure
– Trade-off Studies
– Source Selection
– Prioritization
11
Definition and Planning
Defining the System
• Adequate description of the technical and program
characteristics of the system
• What are the physical and performance
characteristics?
• What are the development, production, and
deployment schedules?
• How many systems are to be produced?
• How will the systems be supported: contract, inhouse, two or three levels of maintenance?
12
Defining the System
Integrated System Schedule
1993
ACTIVITY
1994
1995
1996
1997
TODAY
1998
LLTI PEO IPR
PROGRAM REVIEWS
ENGINEERING &
MANUFACTURING
DEVELOPMENT
LETHALITY
ENHANCEMENT
LRIP DECISION
MRRB
TECH SUPPORT
DEVELOPM ENT
QL
TEST & EVALUATION
UE
FT T
QUAL
IOTE
LIVE FIRE TEST
LUT
(3)(6)
LFT (A)
LFT
(26)
(B,C) (8,4)
LOG DEMO II
PVT
(25 MS L + 10 CLU)
ENHANCED
PRODUCIBILITY
PROGRAM
LOW RATE INITIAL
PRODUCTION
FY96
(28)
FY98
FY99
FIELDING
AWE
QL
FTT
UE
LUT
CA
MRRB
(9)
CA
MSLS - 698
CLUs 55
MSLS - 872
CLUs 97
MSLS - 1015
CLUs - 108
ARMY
M SLS - 1020
CLUs - 206
M SLS - 1080
CLUs - 270
M SLS - 3316
CLUs - 423
LEGEND
MSL CLU
LLTI
CA
LEAD TIME
PRODUCTION
LEAD TIME
CA
-
QUICK LOOK
FIELD TACTICAL TRAINER
USER EVALUATION
LIMITED USER TEST
CONTRACT AWARD
MATERIEL RELEASE
REVIEW BOARD
PRODUCTION
LEAD TIME
CA
MULTIYEAR I & II
(3 YR CONTRACT)
FY97
II, III, FLIGHT
I
LD
II
PROD VER TEST
FY95
2000
MILESTONE III
LITE I / WARHEAD / LOW
COST BST
FY94
1999
J A S O ND J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D
PRODUCTION
CA
USM C
141
48
194
140
741
133
LEAD TIME
PRODUCTION
CA M YII
CA
RANGER
82nd
COMPL FIELDING
BEGINS
HANDOFF FT. HOOD NTC
FUE
USA
82nd
COMPLETE
FUE
USMC
13
Defining the System
Work Breakdown Structure (WBS)
• WBS definition
– product-oriented breakdown of hardware,
software, services, data and facilities that define
the system.
• WBS breaks a total job down into manageable
pieces & portrays the way work is to be done.
• WBS displays a company’s reporting structure.
• Program managers may cite MIL-HDBK-881 “for
guidance only” in contract solicitations.
14
WBS
15
Defining the System
Cost Element Structure (CES)
1.0 Research Development Test & Evaluation
(RDT&E)
2.0 Production
3.0 Construction (CON)
4.0 Pay and Allowances
5.0 Operating and Maintenance Army (O&M)
16
Defining the System
COST ELEMENT STRUCTURE – 1.0 RDT&E
CES#
ELEMENT:
1.01
1.02
1.03
1.04
1.05
1.06
1.07
1.08
1.09
1.10
1.11
DEV. ENG.
PEP
DEV. TOOL.
PROTO MFG.
SEPM
SYS T&E
TRAINING
DATA
SUPP EQUIP.
DEV. FACILITIES
OTHER
FY00C$M
$ 39.039M
0.408
0.457
110.421
78.266
11.112
1.989
3.439
4.897
0.0
0.968
TY$M
$38.260M
0.386
0.450
107.724
76.363
10.927
1.954
3.413
4.758
0.0
0.928
17
Defining the System
COST ELEMENT STRUCTURE – 2.0 PROCUREMENT
CES#
ELEMENT:
2.01
2.02
2.03
2.04
2.05
2.06
2.07
2.08
2.09
2.10
2.11
2.12
2.13
2.14
NON REC PROD.
REC. PROD
ENG. CHG
SEPM
SYS T&E
TRAINING
DATA
SUPP. EQUIP.
OPER./SITE/ACT.
FIELDING
TRAIN. AMMO/MSLS
WAR RESV.
MODS
OTHER
FY00C$M
$ 16.110M
1,169.348
0.0
116.637
12.564
28.880
2.073
146.460
0.0
89.525
59.001
0.0
236.619
51.739
TY$M
$
16.583M
1,312.377
0.0
132.258
14.437
31.499
2.300
158.304
0.0
101.635
79.482
0.0
280.729
64.129
18
Defining the System
COST ELEMENT STRUCTURE – 3.0 CON
CES#
3.01
3.02
3.03
3.04
ELEMENT:
DEVELOP. CONSTRUCTION
PRODUCT. CONSTRUCTION
OPERATION/SITE ACTIVATION
OTHER N
FY00C$M
TY$M
19
Defining the System
COST ELEMENT STRUCTURE – 4.0
Pay & Allowances
CES#
4.01
4.02
4.03
4.04
4.05
4.06
ELEMENT:
FY00C$M
CREW
MAINTENANCE
SYSTEM SPECIFIC SUPPORT
SEPM
REPLACEMENT PERSONNEL
OTHER
TY$M
20
Defining the System
COST ELEMENT STRUCTURE – 5.0
Operating & Maintenance (O&M)
CES#
5.01
5.02
5.03
5.04
5.05
5.06
5.07
5.08
5.09
5.10
5.11
ELEMENT:
FY00C$M
FIELD MAINT., CIV LABOR
SYS. SPECIFIC BASE OPS
REPLENISHMENT DLRs
REPLEN. CONSUMMABLES
POL
END ITEM MAINTENANCE
TRANSPORTATION
SOFTWARE
SEPM
TRAINING
OTHER
TY$M
21
Defining the System
Cost Analysis Requirements Description
(CARD)
• Source of a system’s description
• Describes important features
• Is provided to other groups preparing cost
estimates
• Helps ensure all groups are costing out the same
“program.”
• Prepared by program office; approved by
DoD Component Program Executive Officer
22
Defining the System
Cost Analysis Requirements Description
(CARD) Continued
1.0 System Overview
1.1 System Characterization
1.1.1 System Description
1.1.2 System Funcitonal Relationships
1.1.3 System Configuration
1.1.4 Government Furnished Equipment/Information
1.2 System Characteristics
1.2.1 Technical/Physical Description
1.2.1.1 Subsystem Description
1.2.1.2 Functional and Performance Description
1.2.2 Software Description
1.3 System Quality Factors
1.3.1 Reliability
1.3.2 Maintainability
1.3.3 Availability
1.3.4 Portability and Transportability
1.3.5 Additional Quality Factors
1.4 Embedded Security
1.5 Predecessor/Reference System
2.0 Risk
23
Defining the System
CARD Continued
3.0 System Operational Concept
3.1 Organizational Structure
3.2 Basing and Deployment Description
3.3 Security
3.4 Logistics
4.0 Quantity Requirements
5.0 System Manpower Requirements
6.0 System Activity Rates
7.0 System Milestone Schedule
8.0 Acquisition Plan or Strategy
9.0 System Development Plan
10.0 Element Facilities Requirements
11.0 Track to Prior Card
12.0 Contractor Cost Data Reporting Plan
24
Definition and Planning
Ground Rules & Assumptions
• State the conditions which must take place
in order for the estimate to be valid
• Ground rules and assumptions must be
documented since changes in these areas
provide an audit trail for changes in the cost
estimate.
25
Data Collection and Analysis
• Collection and analysis represent a significant
amount of the overall estimating task in terms of
time. The analysis will include decisions on what
programs to include in the data set to whether to
truncate lot data on a program for which you are
calculating a learning curve.
• Document data in your analysis, and any
assumptions you make
26
Data Collection and Analysis
• The direction we take in collecting historical data will
be determined by our choice of estimating
methodologies.
• This step may also dictate a change in estimating
approach due to the availability or non-availability of
certain data.
• Data collection is not limited to cost data. We must
also collect technical and program data if we want the
total picture of the historical systems. This will help us
ensure the comparability of the systems that we are
collecting data on with the system we are estimating.
27
Data Collection and Analysis
Most Difficult Task in Cost Estimating
• Data Sources
– Data Types: Cost/Resource, Technical, Program
– Categories: Primary, Secondary
• Data Problems
– Wrong Format – Matching up – Definition
– Temporal Factors - comparability
• Normalization
• Data Location
28
Tools for
Total Ownership Cost Estimating
Life Cycle Cost Management Tools
APPN
ACEIT PRICE SEER ACDB AMCOS OSMIS SBC/ISR CO$TAT
RDT&E
X
X
X
X
X
X
PROC
X
X
X
X
X
X
MILCON
X
X
X
X
X
MILPERS
X
X
X
X
X
O&M
X
X
X
X
X
X
X
29
Definition and Planning
Select the Estimating Approach
• Techniques available
– Analogy
– Parametric
– Engineering
– Extrapolation
– Expert Opinion
Select the technique that is most applicable
to a specific WBS element
30
Definition and Planning
Estimating Methods
• Analogy
– Basic Comparison
– Factors
• Parametric
– Regression Analysis
• Engineering
– Detailed
• Expert Opinion
– Committee
– Delphi
31
Life Cycle Cost Estimates
Concept
Refinement
Technology
Development
A
System Development
& Demonstration
B
Production &
Deployment
Operations
& Support
C
Disposal
D
Cost Estimating Model
x
y
0
•
•
•
•
•
•
•
Parametric
Engineering
Actuals
x
y
0
Parametric
Engineering
Actuals
x
y
z
Parametric
Engineering
Actuals
x
y
z
Parametric
Engineering
Actuals
0
y
z
Parametric
Engineering
Actuals
Cost estimates based on confidence intervals
Parametric analysis based on similar systems and similar attributes (regression)
Engineering data based on reliability projections used for bottoms-up estimate
Actual system costs used to extrapolate future system costs
Cost estimate revised every two years after production
Weights associated with non-actuals decrease as system matures
MS B is a true hard stop for systems
32
Cost Estimating Methods
Analogy Method
• Based on direct comparison with historical
information of similar existing activities,
systems, or components.
• Compares new system with one or more
existing similar systems where there is
accurate cost and technical data.
• Analyst must show validity of comparison.
33
Cost Estimating Methods
Analogy Method
• Based on known costs of a similar program
• Adjustments for complexity, technical, physical
• Strengths
– Based on representative experience
– Less time consuming than others
– Can be used as a check on other techniques
• Weaknesses
– Small sample size
– Heavy reliance on judgment
– Sometimes difficult to identify analogy and
associated costs
34
Analogy Estimating with Factors
Cost(New) = Cost(Old) x Adjustment Factor
Element
Old Sys1 Old Sys2 Old Sys3
New Sys
Airframe
$500/lb
1.25*S1
Engine
2M/Unit 3M/Unit 5M/Unit
.8*S3
Avionics
$3K/lb
1.0*S2
Payload
6M/Unit 8M/Unit 7M/Unit
$250/lb
$2K/lb
$750/lb
$4K/lb
.65*S1
35
Cost Estimating Methods
Parametric Method
• Known as Statistical Method or Top Down
Method
• Relates cost to physical attributes or performance
characteristics
• Uses database of elements from similar systems
• Uses multiple systems
• Most beneficial in earlier stages of the system or
project life cycle
36
Cost Estimating Methods
Parametric Method
• Statistical relationships between cost and
physical or performance parameters of
past systems.
• Strengths
– Captures major portion of cost
– Quick what if type estimates
• Weaknesses
– Less detailed
– Getting accurate data
37
Cost Estimating Methods
Parametric Method
(Extrapolation)
• Use historical values to establish a trend for the
future.
• Example problem: Given the actual productivity
and labor rates in the given table. How much will it
take to complete a 3-year software development project
of 10K lines of code, if 50% is completed in the second
year and 25% is completed in first and third years?
38
Cost Estimating Methods
Engineering Method
• Known as bottom up method
• Requires extensive knowledge of system
characteristics
• Divide into segments; estimate costs for each
segment
• Combine segments plus integration cost
• Uses a combination of cost estimating methods
• Detailed knowledge of new technologies may not
be available.
39
Cost Estimating Methods
Engineering Method
• Strengths
– Detailed
• Best when long stable production process
• Weaknesses
– Requires a lot of time
– Cost
– Cannot be used until system well defined
40
Cost Estimating Methods
Expert Opinion Method
• Subjective judgment of an experienced
individual or group
• Use if time does not permit a more thorough
analysis
• Document source(s) of opinion of experts
• List attributes of the source(s)
Example: Delphi Technique
41
Cost Estimating Methods
Expert Opinion Method
• Consulting with one or more experts who
use their knowledge and experience to
arrive at an estimate
• Group techniques include
– Consensus (Committee)
– Delphi
• Strengths and weaknesses
42
Expert Opinion Method
Delphi Technique
•
•
•
•
•
Query expert opinion from group
Seek information from each expert
Summarize the results
Send report to each expert
Gather second opinion after each individual
reviews report
• Summarize results
• Iterative process continues until the experts reach
a consensus, or near-consensus.
43
Expert Opinion Method Example
Labor type
Hours
Needed
% of Total
Hourly Rate
Hrs
%*rate
Senior
Engineer
1000
$13
10.5
$1.37
Design
Engineer
3000
$11
31.6
$3.48
Tool & Die
500
$11
5.3
$.58
5000
$9
52.6
$4.73
Machinist
Totals
9500
$10.16
44
Learning Curve Theory
As the quantity of a product produced
doubles, the man-hours- per-unit
expended to produce the product
decreases at a fixed rate or constant
percentage (usually 10% to 15%).
45
Learning Curve Theory
Factors Contributing to Efficiency
• Job familiarization by both production workers and supervisory
personnel.
• Changes in product design which do not materially affect the product,
but result in increased ease and speed of production.
• Changes in tooling, machinery, and equipment which simplify or
speed up the production process.
• Improved production planning and scheduling, and improvements in
production techniques and operational methods.
• Improvements in shop organization, engineering coordination and
liaison.
• Improvements in the handling and flow of materials, and in the
materials and parts supply systems
46
Learning Curve Theory
The table is based on the assumption that the first unit required 100
person-hours to produce. The table indicates a constant rate of
reduction of 20% for each doubling of the unit number; the value of
the second and each succeeding item in the table is 80% of the value
of the preceding item.
TABLE FOR FIGURES
F-1-1 and F-1-2
Unit No.
1
2
4
8
16
32
64
Unit Person-hours
100.00
80.00
64.00
51.20
40.96
32.77
26.21
47
Learning Curve Theory
80% Unit Curve on Arithmetic Paper
48
Learning Curve Theory
1000
Unit
Cost
(LOG)
100
90%
85%
80%
10
70%
1
1
10
100
Production Unit Number
(LOG)
1000
10000
49
Learning Curve Theory
Uses
– Evaluating contract production costs.
– Assessing impact of production interruptions,
product changes and production rate change.
– Rate of improvement experienced by a particular
contractor on a prior product may be indicative of
rate of improvement expected on new product of
similar size, complexity, and construction.
– Improvement curve pattern experienced in the
production of past item can be extended to calculate
costs of future items.
50
Estimate Formulation
• We have defined our tasks, planned the estimate,
assigned cost responsibilities, and performed data
collection and analysis.
• Here we apply our estimating methodologies and
tools: develop the factors, analogies, CERs, and
learning curves.
• We will aggregate the various cost elements into
development, production, and O&S estimates,
fiscally spread the costs, and apply inflation.
51
Cost Estimating Methods
Laying Out the Estimating Approach
Aircraft System – sum of level II elements (cross-check with analogy)
Air Vehicle – sum of level III elements
Airframe – CERs
Air Vehicle Software – Expert Opinion
Propulsion – CERs
Avionics – Analogy
Armament – Catalog Price
System Eng/Program Mgt – Factor of Air Vehicle
System Test & Evaluation – Factor of Air Vehicle
Training – Factor of Air Vehicle
Data – Factor of Air Vehicle
Peculiar Support Equipment – Factor of Air Vehicle
Initial Spares – Factor of Air Vehicle
52
Review and Presentation
• We want to ensure that the estimate is
reasonable, realistic, and complete.
• Reasonableness addresses areas such as:
using appropriate and acceptable
methodologies; presenting methodologies
systematically; the use of relevant data; and
ensuring that assumptions are valid and
clearly stated.
53
Review and Presentation
(Continued)
• The realism test checks to see if the
assumptions and ground rules are consistent
with the statement of work and if the costs
are in line with historical data.
• We evaluate completeness by determining
whether the estimate includes all the work
stated in the request for proposal and
whether the costs are traceable and
reconcilable.
54
Risk and Uncertainty Analysis
Risk Analysis Approaches
• Detailed Network & Stochastic
Effort
• Discrete Technical, Schedule, and Estimating Risks
• Detailed Monte Carlo Simulation (each WBS)
• Bottom Line Monte Carlo Simulation
• Add a Risk Factor/Percentage
Detail
Risk and Uncertainty Analysis
Frequency
Probability
Distributions
WBS Cost Distribution
Total Cost
PDF
P.E.
Sum PE
+
P.E.
=
Confidence
+
MLC (Mode)
CDF
50%
20%
P.E.
+ Many More
Sum PE
MLC
Risk and Uncertainty Analysis
Probability Density Function
Forecast: Results=
Cell B28
Frequency Chart
4,978 Trials Shown
.024
117
.018
87.7
.012
58.5
.006
29.2
Mean = $2,198
.000
0
$1,250
$1,750
$2,250
$2,750
$3,250
Risk and Uncertainty Analysis
Cumulative Distribution Functions
Forecast: Results=
Cell B28
Cumulative Chart
4,977 Trials Shown
.995
4977
.747
.498
.249
.000
0
$1,250
$1,750
$2,250
$2,750
$3,250
Reasons for Risk
Technical
Programmatic
Cost
Schedule
Physical
Properties
Material
properties
Material
Availability
Skill
Requirement
Sensitivity to
Assumptions
Sensitivity to
Technical Risk
Degree of
Concurrency
Sensitivity to
Technical Risk
Software
Complexity
Environmental
Impact
Integration
Interface
Contractors
Stability
Sensitivity to
Programmatic
Risk
Sensitivity to
Schedule Risk
Sensitivity to
Programmatic
Risk
Number of
Critical Path
Requirement
Changes
Funding Profile Estimating
Errors
Operational
Environment
Political
Advocacy
Estimating
Errors
59
Cost Schedule Curve
Parallel Effort
More ECP
Less Mature Design
Typical
Life Cycle Cost
Fixed Cost
Technology Outdate
Min
Optimal
Development Schedule
60
Development Schedule
Mean Cycle Time to IOC
DoD
Pre-1992 Starts
132 months (11 years)
Post-1992 Starts
89 months (7.4 years) on-going
F-22
216 months (18 years) IOC 2004
Comanche
264 months (22 years) IOC 2006
Commercial
Boeing 777
54 months (4.5 Years)
61
Schedule Goals
Commercial Drivers
• Technology Drives Schedule
• Constraint Schedule
• Goal is ROI Maximization
Cost
Defense
Commercial
DoD Drivers
• Funding Drives Schedule
• Unconstrained Schedule
• Goal is Cost Reduction
Development Schedule
62
DoD Program Schedule Drivers
Cost
Funding Allocation
Acquisition Process
Program Execution
Cycle Time Reduction Goal
Development Schedule
63
Final Documentation
• Provide the means for other analysts to get the
same results that we have in our cost estimate.
• Providing good directions and a clear trail to
follow are essential in having an estimate that can
be replicated.
• Provide step-by-step documentation of the
methodologies, supporting data, ground rules, and
assumptions, equations, examples, etc.,
• Ability to interpret or evaluate someone else’s cost
documentation is as important as ability to prepare
good cost documentation.
64
CAIV Process
• Set realistic but aggressive cost objectives
early in acquisition program
• Manage risks
• Track progress using appropriate metrics
• Motivate Government and industry
managers to achieve program objectives
• Incorporate incentives to reduce O&S costs
for fielded systems
65
Cost As An Independent Variable
(CAIV)
• Best time to reduce cost is early in the
process.
• Involves the stakeholders in the process.
• Cost tradeoffs must be addressed early in
the acquisition process and embedded in
program requirement documents, Request
For Proposals (RFPs), contract provisions,
and source selection process.
66
Cost Analysis Domain
Inputs (Descriptions)
Requirements
Acquisition Assumptions
Design Parameters
Risk Assessment
Process (Models)
Outputs (Costs)
ASARC
CAIG, DAB
APB
PPBES
AOA
Cost Analysis Process
Databases, Tools, & Models
Types of Cost Studies
Current
Capability
Needed
Capability
Studies
Should be Known
Unknown
Class
Cost Estimating
Descriptions, Models
Costs
Analysis
CER Development
Description, Historical
Costs
Models
Synthesis
CAIV
Cost Goals, Models
Descriptions ( Design and
Performance Parameters)
Control
Performance Based and Design Based Cost Models Currently do not Exist
67
Analysis of Alternatives
• Set realistic, aggressive cost objectives early
in development
• Manage risks
• Track progress with appropriate metrics
• Motivate government/industry managers
to achieve program objectives
• Incorporate incentives to reduce O&S costs
for fielded systems.
68
Cost
Analysis
Art or
Science?
69