Acquisition Cost Growth –
The Stats Matter!
Quantitative Methods in Defense and National Security 2010
Ms. Anne Twist
Deputy Director, Resource Analysis
Office of the Under Secretary of Defense (AT&L)
25 May 2010
MDAP Portfolio Tornado Chart
JSF (F-35)
BMDS
SSN 774
DDG 51 Destroyer
C-17A
F-22 Raptor
V-22 (OSPREY)
F/A-18E/F
CVN 21
TRIDENT II MISSILE
Joint MRAP
Sunk Cost
-100
P-8A
PATRIOT/MEADS CAP
Chem Demil-CMA
CH-53K Program
BLACK HAWK UH-60M
AMRAAM (AIM-120)
FMTV
DDG 1000 Destroyer
JTRS GMR
E-2D AHE
LPD 17 CLASS
WIN-T INCREMENT 3
EFV (Formerly AAAV)
Stryker
C-130J Hercules
SBIRS High
BAMS
MH-60R
Global Hawk
CH-47F
LONGBOW APACHE
Advanced EHF
USMC H-1 UPGRADES
REAPER
EA-18G
PATRIOT PAC-3
AB3
AMF JTRS
Navstar GPS
Chem Demil-ACWA
JLENS
MH-60S
JASSM
C-5 RERP
MUOS
TOMAHAWK(R/UGM-109E)
LHA 6
SM-6
C-130 AMP
GMLRS
IAMD
JDAM
JPATS
JTRS HMS
ER/MP UAS
JSOW
WIN-T Increment 2
CEC
ATIRCM/CMWS
GPS-IIIA
FAB-T
JHSV
FBCB2
WIN-T Increment 1
LITTORAL COMBAT SHIP
AIM-9X
WGS
PREDATOR
Inc 1 E-IBCT
VTUAV
MIDS
Excalibur
IDECM
NMT
HIMARS
JCA
LUH, UH-72A Lakota
JTRS NED
AGM-88E AARGM
COBRA JUDY REPLACEMENT
NAS
RMS
B-2 RMP
MP-RTIP
C-5 AMP
EA-6B ICAP III
GBS
JPALS
SBSS BLOCK 10
B-2 EHF
ASIP
LAIRCM
-50
0
Remaining Cost (FY11+)
50
100
150
200
250
300
Investment Funding (TY$B)
We have a bad behaving population of MDAPs; the magnitude of the JSF
program makes portfolio-wide statistics meaningless. Measuring portfolio
cost growth and other similar statistics is Bad Math.
2
Measuring Cost Growth:
GAO Method vs. AT&L Method
The Stats Matter!
Doomsday Press Reports
April 1, 2008: The Government
Accountability Office found that 95 major
systems have exceeded their original
budgets by a total of $295 billion, bringing
their total cost to $1.6 trillion, and are
delivered almost two years late on
average. In addition, none of the systems
that the GAO looked at had met all of the
standards for best management practices
during their development stages.
Auditors said the Defense
Department showed few signs of
improvement since the GAO began
issuing its annual assessments of
selected weapons systems six years ago.
"It's not getting any better by any means,"
said Michael Sullivan, director of the
GAO's acquisition and sourcing team.
"It's taking longer and costing more."
March 31, 2009: Development costs for
the Pentagon's major weapons systems
soared last year, helping drive overruns
that are "staggering," the Government
Accountability Office said in a report
released yesterday. Overall, the cost
overruns associated with the military's
major weapons …”total near $300 billion,
and the average program delay has
stretched from 21 to 22 months,”
The figures reflect a weapons
development and procurement system
that is woefully broken… “Pentagon
planners don't do a good enough job of
analyzing those requirements to
understand whether they have the
technologies and designs to build to
them," GAO analyst Michael J. Sullivan
said.
4
Measuring Cost Growth:
GAO Method vs. AT&L Method
•
•
•
•
GAO Methodology
Single Program Example: DDG 51
Oldest 10 programs
MDAP Portfolio
“The new GAO report continues to sensationalize the assessed cost
growth of $296 billion on 96 programs. This number has been cited by
many people as a condemnation of the defense procurement process. I
have analyzed the components of this GAO number, and I would suggest
that the number is misleading, out-of-date, and largely irrelevant to the
current management of DoD programs.”
--Mr. John J. Young, former USD(AT&L), 2009
The Stats Matter!
GAO Methodology
• GAO publishes an annual report on cost growth
– Summarizes all Major Defense Acquisition Programs (MDAPs) –
highlight is their “Total Cost Growth” calculation
– The FY10 report did not give a total cost growth figure
• GAO defines cost growth as the change in total
program acquisition costs from the first original
estimate to the current estimate
• Problems with GAO’s methodology:
1) Procurement of additional quantities and/or required capability
counts as cost growth
2) Poor early performers can never recover, even if they have been
performing well for many years
3) Differing “portfolios” compared on an annual basis
4) Pre-Milestone B estimates used (i.e., before the program defined)
5) Acquisition painted with broad brush as if all programs are broken
6
GAO Methodology
• Quantity growth and age of the programs greatly
inflate cost growth
• The fine print….
– Past sins are never forgiven: Many programs have been in the
portfolio for more than 15 years and continue to carry huge cost
growth figures even if they have been performing well for years
– Quantity Changes are Leadership Decision to Shape the Force:
Procurement of additional quantities and/or required capability is
included in GAO’s cost growth calculation, often making more
desired programs appear to be broken
• These two factors consistently had impact on program
cost, and therefore we must isolate these factors to
really understand what drives cost growth
Focus on Measurement: Cost growth is an issue that must be controlled;
however, the amount of cost growth has been significantly overstated.
7
DDG 51 Example
The DDG 51 is a multi-mission guided missile destroyer designed to operate offensively
and defensively, independently, or as units of Carrier Strike Groups (CSG), Expeditionary
Strike Groups (ESG), and Missile Defense Action Groups in multi-threat environments
that include air, surface, and subsurface threats. These ships will respond to Low
Intensity Conflict/Coastal and Littoral Offshore Warfare (LIC/CALOW) scenarios as well
as open ocean conflict providing or augmenting power projection, forward presence
requirements, and escort operations at sea.
8
GAO Method: DDG 51
Program Original
Estimate to Current
DDG 51 Destroyer
Original
Current
Cost Growth (GAO
MS B Date Estimate ($B) Estimate ($B) Method) ($B)
Feb 1988
27.6
93.4
65.8
PAUC = Program Acquisition Unit Cost
• GAO Method simply subtracts the Original estimate from the Current
estimate to calculate the total program cost growth
• DDG 51 had a MS B in February, 1988. At that time, only 23 units were
planned to be produced. The Dec 2009 SAR estimate shows that DoD
now plans to procure 71 units
• The increase in unit cost for DDG 51 is relatively small; however it is
incurring a huge cost growth penalty for its increase in quantity
• The increase in quantity was a leadership decision driven by the fact that
this is a successful program – according to GAO, this is one of the worst
performers in the inventory!
9
AT&L Method: DDG 51
Program 5 Year Estimate
DEC04
DEC04
to Current
MS B Date Quantity PAUC ($B)
DDG 51 Destroyer
Feb 1988
62
1.2
Current Current
Quantity PAUC ($B)
71
1.3
Actual Cost
Unit Cost
Quantity Cost Growth (ARA
Growth ($B) Growth ($B) Method) ($B)
4.3
11.8
4.3
PAUC = Program Acquisition Unit Cost
• First, we eliminate historical bias by comparing the Current Estimate to
the 5 year prior estimate
• Second, we separate cost growth due to changes in unit cost from cost
growth strictly due to quantity increases
• The difference between GAO Method and AT&L Method is significant in
many programs – for DDG 51 it is about a $61.5B difference
• We used this process for about 100 MDAPs, and saw that when quantity
is considered, Actual Cost Growth becomes very concentrated among a
few programs
10
Oldest 10 Programs: GAO Method
All data are in BY$10 Billions
Index
1
2
3
4
5
6
7
8
9
10
Older Acquisition Programs
TRIDENT II (D-5) MISSILE
DDG 51 Destroyer
V-22 OSPREY
FMTV
AMRAAM
C-17A
F-22 Raptor
F/A-18E/F
JSOW Baseline/BLU-108
CVN-77
First Original
APB
Jul 1987
Feb 1988
Feb 1988
Oct 1988
Dec 1988
Dec 1988
Feb 1992
Jun 1992
Jun 1992
Jun 1993
Sub-total ($B)
First
Original
Estimate
Dec 09
SAR
Estimate
50.4
27.6
31.4
10.2
16.4
50.1
89.0
79.7
2.8
6.1
50.9
93.4
55.5
21.1
23.8
81.5
76.6
54.1
2.2
6.8
363.7
465.9
GAO Cost
Growth
($B) from
MS B
0.5
65.8
24.1
10.9
7.4
31.4
-12.4
-25.6
-0.6
0.7
102.2
• If we replicate the GAO Method using Dec 2009 SARs, the 10 oldest
programs (out of a total of 98) account for $102.2B of cost growth
11
Oldest 10 Programs: AT&L Method
All data are in BY$10 Billions
Index
1
2
3
4
5
6
7
8
9
10
Older Acquisition Programs
TRIDENT II (D-5) MISSILE
DDG 51 Destroyer
V-22 OSPREY
FMTV
AMRAAM
C-17A
F-22 Raptor
F/A-18E/F
JSOW Baseline/BLU-108
CVN-77
First Original
APB
Jul 1987
Feb 1988
Feb 1988
Oct 1988
Dec 1988
Dec 1988
Feb 1992
Jun 1992
Jun 1992
Jun 1993
Sub-total ($B)
First
Original
Estimate
Dec 09
SAR
Estimate
50.4
27.6
31.4
10.2
16.4
50.1
89.0
79.7
2.8
6.1
50.9
93.4
55.5
21.1
23.8
81.5
76.6
54.1
2.2
6.8
363.7
465.9
GAO Cost AT&L Cost
Growth
Growth
($B) from ($B) last 5
MS B
years
0.5
2.1
65.8
4.3
24.1
1.8
10.9
3.4
7.4
2.3
31.4
-5.8
-12.4
2.0
-25.6
-1.7
-0.6
0.0
0.7
-0.3
102.2
8.2
• From earlier slide, GAO method shows that the 10 oldest programs
account for $102.2B of cost growth
• When we account for age of the program and quantity increases, the
AT&L actual cost growth for these same 10 programs is $8.2B!
12
GAO Method: FY10 MDAP Portfolio
100
80
Portfolio Cost Growth $B
60
40
20
JSF
$281B
DDG 51
$93B
C17A
$82B
V-22
$56B
SSN 774
$81B
FMTV $21B
AMRAAM$24B
0
Total Program Cost
is represented by
the size of the circle
Apache
$14B
C-130J
$15B
CH-47F
$13B
Blackhawk
$22B
MRAP
$38B
Trident II
$51B
F-22
$77B
-20
DDG 1000
$20B
F/A-18
$54B
-40
1985
1990
1995
2000
2005
2010
2015
Milestone B Date
Using GAO Method, cost growth is concentrated in JSF and several very old
programs, many of which have been performing well for some time.
13
AT&L Method: FY10 MDAP Portfolio
100
80
Portfolio Cost Growth $B
60
JSF
$281B
Total Program Cost
is represented by
the size of the circle
40
The Stats Matter!
20
0
DDG 51
Trident II $93B
$51B
C17A
$82B
F-22 $77B
F/A-18
$54B
CVN 21
$34B
SSN 774
$81B
GMLRS
$6B
MRAP
$38B
DDG 1000
$20B
-20
-40
1985
1990
1995
2000
2005
2010
2015
Milestone B Date
Using AT&L Method, it is obvious that JSF and a small set of troubled
programs account for the bulk of cost growth.
14
Cost Growth or Capability Trade-off?
• There are numerous methods we can use to classify cost growth,
but the real issue is determining what capability can be supplied
to the warfighter in a given period of time
• Sometimes we are willing to “pay,” or incur cost growth for capability
– Immediate warfighting needs
– Necessary quantity or technology increase
– Flexibility from numerous simultaneous acquisition programs
– Efficient production schedules
• Other times cost growth is not willingly incurred
– Poor initial cost estimates
– Poor program management
– Unrealistic expectations
– Premature technology
– Redundant capability
• Reducing the capability lost to the warfighter as a result of unplanned
cost growth should be our primary focus!
15
F-22 Funding Example
The F-22 combines stealth, speed, maneuverability, integrated avionics, multirole precision attack, and improved supportability. It is a critical component of
the Global Strike concept of operations with unmatched air-to-air and air-toground capabilities. The Raptor ensures air dominance, rapidly and at great
distances, by countering and defeating threats, including those attacking
friendly forces and those attempting to deny access by our forces.
16
F-22 RDT&E Program Funding
FY85$M
4000
3500
80%
60%
40%
20%
0%
-20%
-40%
Total R&D Program Cost
Relative to Original Baseline
SAR Date
3000
2500
2000
1500
1000
500
0
83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 '00 '01 '02 '03 '04 '05 '06 '07 '08
17
F-22 RDT&E Program Funding
FY85$M
4000
MS B: Feb 1992
Original Unit Cost: $137M
Current Unit Cost: $408M
3500
80%
60%
40%
20%
0%
-20%
-40%
Total R&D Program Cost
Relative to Original Baseline
SAR Date
3000
Original Estimate
2500
2000
1500
Current Estimate
1000
500
0
83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 '00 '01 '02 '03 '04 '05 '06 '07 '08
18
Acquisition Reform Initiatives
• Weapon Systems Acquisition Reform Act
(WSARA) of 2009
• MS A and MS B Certifications
• IMROVE Acquisition Act: HR 5013
• QDR & FY11 Budget
• Acquisition Workforce Reform
• Tactical Acquisition Reform
• Strategic Acquisition Reform
19
WSARA Funding Requirements
• OSD(CAPE) responsible for developing guidance governing
cost estimation and cost analysis processes, including
confidence levels for cost estimates and conducting
independent estimates/analyses for all MDAP and MAIS for
which USD(AT&L) is the Milestone Decision Authority (MDA)
• D,CAPE and the Secretaries of Military Departments must
disclose the confidence level used in cost estimates for
MDAPs/MAISs
• If less than 80 percent, include such disclosure in any
documentation approving a cost estimate, and in the next
Selected Acquisition Report (SAR) or quarterly report
20
Funding to 80% Confidence Level
• Previously, MDAPs were funded so it was equally
likely that actual costs will be higher or lower than
the estimate
• Problems with funding to the 80% confidence level:
– What is the underlying distribution?
– Money into the program is more than we expect to spend is this a self-fulfilling prophecy?
• MDAP spending in 2011 is $77B – what would it be
if we had to fund to 80% confidence level?
21
Ongoing Studies
• Numerous studies on program stability and portfolio cost
growth have been done
• Common themes:
– Funding Stability
– PM Tenure
– Accurate cost estimates
– Requirements Creep
• AT&L is aggressively addressing cost growth:
– Formation of a Value Team, headed by the PDUSD(AT&L)
– MDAP Transparency Study to identify program costs in the financial
& acquisition databases
22
Conclusions
• When controlled for quantity and time, cost growth
becomes a much more manageable problem for the
Department
• Not all cost growth is bad. There are smart and necessary
capability increases to platforms that result from deliberate
& intelligent decision making for the benefit of the
warfighter and National Security
• Large cost growth in individual acquisition programs is the
exception, not the rule
• The Department continues to implement many policies &
procedures to help control cost growth – results indicate
they are working!
• The Stats Matter!
23