Air Armament Center
War-winning Capabilities…On Time, On Cost
Acquisition Cost
Estimating Update
28 Feb 08
Ken Kennedy, AAC/FMC
Kristy Golden, AFCAA/FMAE
DISTRIBUTION STATEMENT A: Approved for public release; distribution is unlimited.
1
Outline
War-winning Capabilities…On Time, On Cost
•
•
•
•
•
•
•
•
•
Estimating Trends
Leadership Concern
Congressional Reaction
Root Cause Analysis
2007/2008 Initiatives
AAC Cost Demand
AFCAA Standup
Industry Help
Summary
2
Estimating Trends
War-winning Capabilities…On Time, On Cost
• GAO-07-406SP Defense Acquisitions
Assessments of Selected Weapon Programs
– GAO assessed 62 weapon systems with a total
investment of over $950 billion, some two-thirds of the
$1.5 trillion DOD plans for weapons acquisition
– “Fully mature technologies were present in 16 % of the
systems at development start” - the point at which best
practices indicate mature levels should be present.
– Programs that began development with immature
technologies (84%) experienced a 32.3 percent cost
increase, whereas
– Those that began with mature technologies (16%)
increased just 2.6 percent.”
3
3
Leadership Concern
War-winning Capabilities…On Time, On Cost
• Sue Payton: Jane's Defence [sic] Weekly,
07/17/2007
– Reduce pressure on budget by improving methods for
calculating the overall cost of acquisition programme
[sic]
– Stop taking contractors' one-off cost estimates at face
value
– Develop own methods for calculating the true, long-term
cost of a weapon system
– Tendency to highlight the basic price tag of a weapon
system and play down the much bigger 'lifecycle cost
– Wait until a preliminary design review has been
completed, rather than trying to guess a weapon's price
before the final requirements are defined
– Push contractors to build more prototypes … important
insight into future development costs
4
4
Congressional Reaction
War-winning Capabilities…On Time, On Cost
Nunn-McCurdy Breach Thresholds
• 2006 Defense Appropriations Act (Public Law No:
109-148)
Type Breach
Initial APB
Current APB
Growth Over Baseline
Notification
Certification
30%
50%
15%
25%
• Requires Increased emphasis on
–
–
–
–
–
Accurate TRL assessments
Quantification of risk and uncertainty
Optimal decisions within trade space constraints
Better cost and schedule forecasts
Budgeting to Independent Cost Estimates
5
5
Root Cause
War-winning Capabilities…On Time, On Cost
Range of Missile Program Cost Outcomes
AOA
Risk Reduction
SDD
LRIP FRP
Med
Low
Med
Risk/Uncertainty
High
We are too optimistic about
risk/uncertainty
Cost risk levels
Extremely
Extremely
High
High
High
High
High
1
2
3
4
Med-High Med
Med
Med-High
5
6
Low
Low
Med-Low
Med-Low
7
8
9
TRL Level
Risk is retired as a program progresses and technology matures
6
2007/2008 Initiatives
War-winning Capabilities…On Time, On Cost
•
•
•
•
Better Risk ID and Modeling
Continued Sufficiency Reviews
New Source Selection Focus
More Contract Risk Sharing
7
Better Risk ID and Modeling
War-winning Capabilities…On Time, On Cost
• Integrate Probability of Consequence
Screening (P/CS) “risk cube” and
Probability of Program Success (PoPS)
assessment into cost estimating process
– Existing infrastructure tools
– Realistic and consistent approach
– Results in a “time now” WBS risk rating
• Use Monte Carlo simulation with WBS
correlation to define total program cost
– Present decision maker with better picture of
uncertainty based on knowledge today
8
Continued Sufficiency Reviews
War-winning Capabilities…On Time, On Cost
• Cost “gray beard” review
–
–
–
–
Reinforce estimating best practices
Verify data sources and methodology
Suggest alternate benchmarks
Request appropriate SME inputs
• Independent technical assessment
• Independent schedule assessment
9
New Source Selection Focus
War-winning Capabilities…On Time, On Cost
• Cost risk # 1 evaluation priority
– High, medium, or low rating based on
percent difference between proposed
contractor cost and government estimate
of offeror’s approach
• Requires clear communication
– Full data source ID and extract for element
basis of estimates (BOEs)
– Full data source adjustment justification in
element BOEs
– Full discussion of risk and uncertainty in
element BOEs
10
More Contract Risk Sharing
War-winning Capabilities…On Time, On Cost
• Solicitation and proposal
– Request WBS element uncertainty/risk and $
magnitude as part of each narrative basis of
estimate worksheet
– Request sum of element risk $ to be included
as a separate proposal line
• Evaluation
– Description of element uncertainty used to
model risk for independent estimate
• Execution
– Proposal risk line becomes earned value MR
– SOO requirement for discussion before use
11
Air Force Cost Analysis Agency
(AFCAA) Standup
Organization Headquarters
THE ASSISTANT SECRETARY
OF THE AIR FORCE
(Financial Management & Comptroller)
DEPUTY ASSISTANT
SECRETARY
(Budget)
DEPUTY ASSISTANT
SECRETARY
(Cost & Economics)
DEPUTY ASSISTANT
SECRETARY
(Plans, Systems & Analysis)
• Deputy Assistant Secretary (Cost & Economics), AFCAA
– Policy and functional oversight
– Major program milestone decisions -- Independent Cost Estimates
(ICE) for delegated programs (required by law); Component Cost
Analysis (CCA); AF CAIG service cost position (SCP)
– Headquarters/functional support (analysis, review, etc.)
Deputy Assistant Secretary
(Cost & Economics) SAF/FMC
Deputy Assistant Secretary for Cost & Economics (SAF/FMC)
Executive Director (AFCAA/FM)
Mr Hartley, SES*
Associate Deputy Assistant Secretary for Cost & Economics
Ms Woods, SES
AFCAA
Technical Director
Mr. Jordan
Principal Economic
& Financial Advisor
Mr. Hosey
Economics &
Business Mgmt
(SAF/FMCE)
Mr. Connair
ASC/AAC
OL
A/C & Weapons
Division
(AFCAA/FMA)
Mr. McVicker
Space Programs
Division
(AFCAA/FMS)
Mr. Seeman
Integration
Division
(AFCAA/FMC)
Col DuPre
AFSCP/
SMC
OL
*The Deputy Assistant Secretary holds two titles as SAF/FMC and AFCAA/FM
Info Tech
Division
(AFCAA/FMI)
Mr. Moul
ESC
OL
Force Analysis
Division
(AFCAA/FMF)
Lt Col Hurley
Resources
Division
(AFCAA/FMR)
Ms. Cann
AFCAA Eglin OL
Kristy Golden
Chief, AFCAA Eglin OL
YC-501-03
John Cargill
Technical Director
YD-1515-03
Paul Pinkowski
Vacant
Justin Neece
Beth Flores
Cost Analyst
YA-501-02
Cost Analyst
YD-1515-02
Cost Analyst
YD-1515-01
Cost Analyst (PAQ)
YD-1515-01
Operating Locations
Hanscom AFB MA
AFCAA - DC
Wright-Patterson AFB OH
Peterson AFB CO
Los Angeles AFB CA
Eglin AFB FL
Air Force Concern
There is a Cost Growth Problem.
Cost Estimating is a Significant, but not Lone, Contributing Factor.
However, it is a factor we can manage!
• Current AF-wide policy does not consistently require adequate,
objective cost analysis or review
– Limited to infrequent milestone reviews
– Inconsistent leadership value placed on cost analysis (contrast DAB
to AFIPT)
• Cost resources down 60% since 1992 -- approximately 400
positions!
– Staffs hit hardest, no career path/pipeline, compare poorly with Army,
Navy, others
• Little “functional” governance
– Inexperienced analysts “isolated” in programs with little functional
support -- fractured estimating methods, databases, models -inconsistent data/estimate quality, analyst qualifications
Acquisition Cost
Capability Initiative
Improve Air Force Cost Estimating
• Better cost estimating processes
– Policy and standards
– Re-writing AFPD 65-5 and accompanying AFIs and AFMANs
– Building standards and templates
– Performance metrics
– Collaboration
– Encourage team efforts
– Establish operating locations at Product Centers and MAJCOMs
– Transparency
– Share data / research / methods
• Repopulate personnel pipeline
• Upgrade education, training, and certification programs
– Provide Cost Estimators with the tools to succeed
Current Status
• Policy AFD 65-5
– The acquiring organization shall build program cost estimates*
•
•
•
•
•
Annually
Milestone decisions
Nunn-McCurdy Cost Breaches
Source Selections
POM initiatives, disconnects, and offsets
* Led by government employees
* SAF/FMC defined standards and
procedures
– Headquarters shall provide non-advocate cost / risk assessments to decision makers
– Collaboratively develop and execute training, data collection/methods and model
development programs
– Headquarters shall review cost, economic, or business case analyses to be presented
to SECAF, USECAF, CSAF, VCSAF
• Personnel
– Hired 20+ positions
• Operating Locations
– Completed beddown agreements at Peterson AFB, Wright-Patterson AFB, Eglin AFB,
Hanscom AFB, and LA AFB
– Working Host Tenet Support Agreement at each location
• Training and Certification
– Partnered with SCEA and DAU to improve certification program
– Developed Training OI for new analysts
AFCAA OL Mission
Shorten “Kill Chain” (Cost
Budget)
• AFCAA-OL will function as the liaison with the Center
• Work collaboratively with the program office estimators
• Non Advocate Cost Assessments (NACAs)
• Independent Cost Estimates (ICEs)
How the AFCAA-OLs will Help
• Collaborative program cost estimates
–
–
–
–
Build cost models
Cost estimating relationships
Risk analysis
Data / research / methods
• Potential roles
–
–
–
–
–
–
Team lead
Team member
Sufficiency reviewer
Consultant
Researcher
Facilitator between Program Office and DCARC (OSD CAIG)
Industry Help
War-winning Capabilities…On Time, On Cost
• Don’t over promise capability with
unrealistic resources
• Define and quantify uncertainty for
your proposed concept
• Provide true risk now versus what it
could be in a future year
• Identify the right level of MR to
neutralize problems before they
overwhelm the program
22
Summary
War-winning Capabilities…On Time, On Cost
• Cost estimating community needs to
provide more realistic resource
projections for Congress and DoD
• Additional AFCAA resources are in
place now with more FMC positions
coming to help programs
• New initiatives are in work to improve
estimate quality, quantify uncertainty,
and fund appropriate risk efforts
23