THE ARTS
This PDF document was made available
CHILD POLICY
from www.rand.org as a public service of
CIVIL JUSTICE
the RAND Corporation.
EDUCATION
ENERGY AND ENVIRONMENT
Jump down to document6
HEALTH AND HEALTH CARE
INTERNATIONAL AFFAIRS
NATIONAL SECURITY
POPULATION AND AGING
PUBLIC SAFETY
SCIENCE AND TECHNOLOGY
SUBSTANCE ABUSE
TERRORISM AND
HOMELAND SECURITY
TRANSPORTATION AND
INFRASTRUCTURE
WORKFORCE AND WORKPLACE
The RAND Corporation is a nonprofit
research organization providing
objective analysis and effective
solutions that address the challenges
facing the public and private sectors
around the world.
Support RAND
Purchase this document
Browse Books & Publications
Make a charitable contribution
For More Information
Visit RAND at www.rand.org
Explore RAND Project AIR FORCE
View document details
Limited Electronic Distribution Rights
This document and trademark(s) contained herein are protected by law
as indicated in a notice appearing later in this work. This electronic
representation of RAND intellectual property is provided for noncommercial use only. Permission is required from RAND to reproduce, or
reuse in another form, any of our research documents.
This product is part of the RAND Corporation monograph series.
RAND monographs present major research findings that address the
challenges facing the public and private sectors. All RAND monographs undergo rigorous peer review to ensure high standards for
research quality and objectivity.
Lessons Learned from the
F/A–22 and F/A–18E/F
Development
Programs
Obaid Younossi, David E. Stem,
Mark A. Lorell, Frances M. Lussier
Prepared for the United States Air Force
Approved for public release; distribution unlimited
The research described in this report was sponsored by the United States
Air Force under Contract F49642-01-C-0003. Further information may
be obtained from the Strategic Planning Division, Directorate of Plans,
Hq USAF.
Library of Congress Cataloging-in-Publication Data
Lessons learned from the F/A–22 and F/A–18 E/F development programs /
Obaid Younossi ... [et al.].
p. cm.
“MG-276.”
Includes bibliographical references.
ISBN 0-8330-3749-8 (pbk.)
1. United States. Air Force—Procurement—Evaluation. 2. United States. Navy—
Procurement—Evaluation. 3. F/A–22 (Jet fighter plane) 4. Hornet (Jet fighter plane) I.
Younossi, Obaid.
UG1123.L48 2005
358.4'183—dc22
2005001397
The RAND Corporation is a nonprofit research organization providing
objective analysis and effective solutions that address the challenges
facing the public and private sectors around the world. RAND’s
publications do not necessarily reflect the opinions of its research clients
and sponsors.
R® is a registered trademark.
© Copyright 2005 RAND Corporation
All rights reserved. No part of this book may be reproduced in any
form by any electronic or mechanical means (including photocopying,
recording, or information storage and retrieval) without permission in
writing from RAND.
Published 2005 by the RAND Corporation
1776 Main Street, P.O. Box 2138, Santa Monica, CA 90407-2138
1200 South Hayes Street, Arlington, VA 22202-5050
201 North Craig Street, Suite 202, Pittsburgh, PA 15213-1516
RAND URL: http://www.rand.org/
To order RAND documents or to obtain additional information, contact
Distribution Services: Telephone: (310) 451-7002;
Fax: (310) 451-6915; Email: order@rand.org
Preface
The process of acquiring new weapon platforms requires the U.S.
military to invest substantial time and money in development, testing, and production. Two recent fighter aircraft programs, the Air
Force’s F/A-22 and the Navy’s F/A-18E/F, illustrate the real difficulties and successes of this process. This report evaluates the historical
information of these platforms to understand how costs and schedules
had changed during the development. The study derives lessons that
the Air Force and other services can use to improve the acquisition of
such aircraft as the Joint Strike Fighter and such other hardware systems as unmanned aerial vehicles and missile programs.
This is one of a series of reports from a RAND Project AIR
FORCE project, “The Cost of Future Military Aircraft: Historical
Cost-Estimating Relationships and Cost Reduction Initiatives.” The
purpose of the project is to improve the tools used to estimate the
costs of future weapon systems. It focuses on how recent technical,
management, and government policy changes affect cost.
The project was sponsored by Lieutenant General John D. W.
Corley, Principal Deputy Assistant Secretary of the Air Force for
Acquisition, and conducted within the Resource Management Program of RAND Project AIR FORCE. The project technical point of
contact was Jay Jordan, Technical Director of the Air Force Cost
Analysis Agency (AFCAA). The observations of this study were
mainly drawn from various cost and schedule reports including
Selected Acquisition Reports, Contract Cost Data Reports, and Contractor Performance Reports. The historical data were supplemented
iii
iv
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
by additional information from the AFCAA and the Naval Air Systems Command’s (NAVAIR) Cost Department.
This report should be of interest to the military aircraft acquisition community and defense acquisition policy professionals generally.
Other RAND Project AIR FORCE reports that address military
aircraft cost-estimating issues include the following:
• In An Overview of Acquisition Reform Cost Savings Estimates,
MR-1329-AF, Mark Lorell and John C. Graser used relevant
literature and interviews to determine whether estimates of the
efficacy of acquisition reform measures are robust enough to be
of predictive value.
• In Military Airframe Acquisition Costs: The Effects of Lean
Manufacturing, MR-1325-AF, Cynthia Cook and John C. Graser examined the package of new tools and techniques known as
“lean production” to determine whether it would enable aircraft
manufacturers to produce new weapon systems at costs below
those predicted by historical cost-estimating models.
• In Military Airframe Costs: The Effects of Advanced Materials and
Manufacturing Processes, MR-1370-AF, Obaid Younossi,
Michael Kennedy, and John C. Graser examined cost-estimating
methodologies and focused on military airframe materials and
manufacturing processes. This report provides cost estimators
with factors useful in adjusting and creating estimates based on
parametric cost-estimating methods.
• In Military Jet Engine Acquisition: Technology Basics and CostEstimating Methodology, MR-1596-AF, Obaid Younossi, Mark
V. Arena, Richard M. Moore, Mark Lorell, Joanna Mason, and
John C. Graser presented a new methodology for estimating
military jet engine costs and discussed the technical parameters
that derive the engine development schedule, development cost,
and production costs and presented quantitative analysis of historical data on engine development schedule and cost.
• In Test and Evaluation Trends and Costs in Aircraft and Guided
Weapons, MG-109-AF, Bernard Fox, Michael Boito, John C.
Preface
v
Graser, and Obaid Younossi examined the effects of changes in
the test and evaluation (T&E) process used to evaluate military
aircraft and air-launched guided weapons during their development programs.
• In Software Cost Estimation and Sizing Methods, Issues and
Guidelines, MG-269-AF, Shari Lawrence Pfleeger, Felicia Wu,
and Rosalind Lewis recommended an approach to improve the
utility of the software cost estimates by exposing uncertainty and
reducing risks associated with developing the estimates.
RAND Project AIR FORCE
RAND Project AIR FORCE (PAF), a division of the RAND Corporation, is the U.S. Air Force’s federally funded research and development center for studies and analyses. PAF provides the Air Force with
independent analyses of policy alternatives affecting the development,
employment, combat readiness, and support of current and future
aerospace forces. Research is conducted in four programs: Aerospace
Force Development; Manpower, Personnel, and Training; Resource
Management; and Strategy and Doctrine.
Additional information about PAF is available on our web site at
http://www.rand.org/paf.
Contents
Preface....................................................................... iii
Figures ...................................................................... xi
Tables ...................................................................... xiii
Summary ....................................................................xv
Acknowledgments.......................................................... xxi
Abbreviations .............................................................xxiii
CHAPTER ONE
Introduction.................................................................1
Two Multirole Fighter Aircraft Programs Emerged at the End of the
Cold War ..............................................................1
These Programs Performed Differently During Their Development
Phases ..................................................................5
Purpose of This Report .................................................... 11
Organization of This Report............................................... 11
CHAPTER TWO
Acquisition Strategies and Industrial Base Issues....................... 13
The F/A-22 Program Sought to Maximize Participation by Multiple
Contractors .......................................................... 13
The F/A-22 Represented an Important Opportunity for the
Combat Aircraft Industrial Base ..................................... 14
The Packard Commission Led to a Two-Team Approach for
Demonstration and Validation ...................................... 16
The F/A-22 Program Created an Artificial Split in Workload ......... 18
vii
viii
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
The F/A-22 Did Not Have a Stable Industrial Base .................... 19
The F/A-18E/F Program Drew on Preexisting Expertise and
Contractor Relationships ............................................ 21
The Navy Adopted a Cautious Approach to F/A-18E/F
Acquisition ........................................................... 21
The F/A-18E/F Program Developed a Worksharing Agreement
Based on Contractor Specialties ..................................... 24
The Use of IPTs Further Minimized Technical and Programmatic
Challenges............................................................ 25
Conclusions ................................................................ 27
CHAPTER THREE
Potential Contributors to Cost and Schedule Growth ................. 29
Comparison of the EMD Program Costs............................... 30
An Assessment of Cost Growth of Major Subsystems ..................... 34
Airframe Cost Growth .................................................. 34
Avionics Cost Growth .................................................. 39
Propulsion System Cost Growth........................................ 45
Conclusions ................................................................ 46
CHAPTER FOUR
Use of Cost Performance Data ........................................... 47
Different Methods of Measuring Progress Affect the Management
of Program Costs and Schedule ...................................... 47
The F/A-22 Program Used Contract Performance Goals to
Measure Progress..................................................... 48
The F/A-22 Contractor Allocated Little for Management Reserve ..... 49
The F/A-18E/F Program Used EVM Data to Measure Progress ....... 50
Conclusions ................................................................ 52
CHAPTER FIVE
Conclusions and Lessons Learned ....................................... 55
Conclusions ................................................................ 55
Lessons Learned for the U.S. Air Force .................................... 55
Contents
ix
APPENDIX
DoD and Congressional Oversight ...................................... 59
Bibliography ............................................................... 69
Figures
S.1. F/A-22 Experienced Schedule Slips and Cost Growth, While the
F/A-18E/F Completed Development on Time and on Cost .... xvii
1.1. Acquisition Process for F/A-22 and F/A-18E/F Programs .........6
1.2. F/A-22 Experienced Schedule Slips and Cost Growth,
While the F/A-18E/F Completed Development on Time
and on Cost .........................................................7
1.3. F/A-22 Schedule Estimates Grew Throughout the Development
Phase ................................................................8
1.4. F/A-18E/F Schedule Estimates Remained Steady Throughout
the Development Phase .............................................8
1.5. F/A-22 Costs Rose Steadily, While F/A-18E/F Costs
Remained Stable ....................................................9
1.6. F/A-22 Schedule Slippage Is Higher Than the Historical
Average ..............................................................9
1.7. F/A-22 Cost Growth Is Higher Than the Historical Average .... 10
2.1. F/A-22 EMD Work Was Artificially Distributed Among
the Contractors ................................................... 19
2.2. F/A-18E/F EMD Work Breakout Was Less Complex Than
the F/A-22 and Had Historical Roots ............................ 25
3.1. F/A-22 and F/A-18E/F EMD Program Cost Drivers ............ 31
3.2. F/A-22 Cost Growth Trends for Major Systems ................. 32
3.3. F/A-18E/F Cost Growth Trends for Major Systems ............. 33
3.4. Comparison of the Air Vehicle Design Hours .................... 35
3.5. Airframe Material Distribution ................................... 36
3.6. F/A-22 Airframe Weight Changes over Time .................... 38
xi
xii
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
3.7.
3.8.
3.9.
3.10.
4.1.
4.2.
4.3.
4.4.
A.1.
A.2.
F/A-18E/F Weight Estimates over Time ......................... 39
F/A-22 Avionics Cost Growth by Component ................... 41
The F/A-18E/F Avionics Development Program................. 44
Development Cost Comparison of the F119 and
F414 Engines ..................................................... 45
The F/A-22 Program Used Contract Performance Goals to
Measure Progress.................................................. 49
The F/A-22 Contractor Allocated Small Amounts of
Management Reserve ............................................. 51
The F/A-18E/F Used EVM Data to Measure Progress ........... 51
The F/A-18E/F Maintained a Large Management Reserve....... 52
Timeline of Congressional Actions ............................... 65
Timeline of DoD Actions......................................... 67
Tables
1.1. Comparison of Some of the F/A-22 Performance Gains over
Legacy Aircraft ......................................................4
1.2. Comparison of Some of the F/A-18E/F Performance Gains
over Legacy Aircraft .................................................4
5.1. Summary of Lessons Learned from Each Program ............... 55
xiii
Summary
Two Multirole Fighter Aircraft Programs Emerged at the
End of the Cold War
From the late 1980s through the present, the U.S. Air Force and the
U.S. Navy have been acquiring two multirole fighter aircraft platforms. The Air Force has pursued the F/A-22, the world’s first supersonic stealth fighter, while the Navy has developed the F/A-18E/F, a
carrier-capable fighter with air-to-air, interdiction, and close air support capability. Currently, the F/A-22 is in the late stages of development, while the F/A-18E/F is in full production and has already
been deployed in Operation Enduring Freedom and Operation Iraqi
Freedom.
The design of the F/A-22 includes advancements in all the
major areas of the aircraft, including airframe, avionics, and propulsion. The airframe incorporates an advanced stealth design to lower
its radar cross section and uses large amounts of advanced materials,
such as composites and titanium. The integrated avionics suite of the
aircraft brings together information collected from several sensors on
the aircraft to be displayed to the pilot. The propulsion system features two high thrust, Pratt and Whitney, F119 jet engines to allow
the F/A-22 to supercruise above the speed of sound without using the
fuel-consuming afterburner. The airframe design, flight controls, and
thrust vectoring are also used to improve the maneuverability of the
aircraft.
The F/A-18E/F Super Hornet was designed to be an upgrade to
the existing F/A-18A/B/C/D multirole aircraft fleet. The program
xv
xvi
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
sought to increase the aircraft’s range, payload, and survivability. The
program was an outgrowth of a Secretary of Defense memorandum
from July 1987, directing the Navy to investigate advanced versions
of the F/A-18 for 2000 and beyond. The trade studies, known as
Hornet 2000, led to a Milestone IV/II review in March 1992 to
begin formal Engineering and Manufacturing Development (EMD)
of the program in July 1992. The F/A-18E/F is 4.2 feet longer than
the legacy platform, has a 25 percent larger wing area, and can carry
33 percent additional internal fuel. The airframe design was largely
new with very little commonality with the original design. It incorporated some limited radar cross-section reduction techniques, such as
new inlets and attention to door and panel edges. The avionics for
the initial release of the F/A-18E/F incorporated the suite from the
C/D model. Provisions were made for a series of avionics upgrades to
be performed subsequent to the basic air vehicle development. The
propulsion is provided by two General Electric F414 jet engines.1
These Programs Reflect the Challenges of Developing
Major Weapons Platforms
The F/A-22 program has experienced significant cost growth and
schedule delays, whereas the F/A-18E/F program completed its
development on cost and without any significant delays. As shown in
Figure S.1, the F/A-22 program had exceeded its original schedule by
more than 52 months as of the date of the last Selected Acquisition
Report (SAR) examined (December 31, 2001), while the F/A-18E/F
was virtually on time. The total cost of developing the F/A-22 grew
by $7.6 billion in Fiscal Year 1990 dollars, compared to the F/A18E/F program, which met its original cost estimates. The schedule
and cost overruns in the F/A-22 program have generated considerable
____________
1
The information is from the F/A-18 Selected Acquisition Report (SAR), F/A-18E/F Cost
Analysis Requirements Description, and the Naval Air System Command’s Hornet hyperlink, available at http://pma265.navair.navy.mil/Public%20Affairs/stores/shornet/shornet.
html, accessed February 2004.
Summary
xvii
concern from the Department of Defense and Congress, leading to
close scrutiny of the program and reductions in the number of aircraft to be produced.
The office of the Assistant Secretary of the Air Force for Acquisition asked RAND Project AIR FORCE (PAF) to investigate the reasons behind the cost growth and schedule delay of the F/A-22
program and those contributing to the cost and schedule stability of
the F/A-18E/F program during EMD. This report examines the
acquisition strategies employed by the F/A-22 and F/A-18E/F programs from their inception through the demonstration and validation
(Dem/Val) and EMD phases. The analysis is based of various cost
and schedule reports available to PAF as well as data and information
available in open sources. For instance, the SARs, Contract Cost Data
Reports (CCDRS) and Cost Performance Reports (CPRs) were the
Figure S.1
F/A-22 Experienced Schedule Slips and Cost Growth, While the F/A-18E/F
Completed Development on Time and on Cost
Schedule
Total cost of development
30
160
25
140
120
Billions of 1990 $
Time between milestones II and III
(months)
180
100
80
60
20
15
10
40
5
20
0
0
F/A-22
F/A-18E/F
*Includes government costs.
SOURCE: Selected Acquisition Reports.
RAND MG276-S.1
F/A-22
F/A-18E/F
xviii
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
main sources of data for the cost and schedule growth analysis. Other
documents, such as Cost Analysis Requirements Description
(CARD), contractor’s weight reports, General Accounting Office
(GAO) reports, and published articles and reports were also examined. The purpose of this analysis is to derive lessons for improving
future Air Force acquisitions.
Multiple Factors Contributed to Problems or Stability in
Each Program
The F/A-22 and F/A-18E/F programs pursued different approaches
to securing contractors, encountered various technical challenges
during development, and employed distinct methods to monitor contract performance data during the EMD phase. All of these factors
contributed to the separate cost and schedule outcomes seen in Figure
S.1:
• Each program used different methods to solicit contractor
proposals and to divide work among contractors during
their development phase. Concerns about the needed mix of
technical expertise and other industrial base issues led the F/A22 program to distribute the work equally among three contractor team members. This arrangement resulted in an artificial distribution of work during the EMD phase and may have
contributed to the schedule and cost problems experienced.
Other business base concerns with respect to the program
teaming structure as well as a move from Burbank to Marietta
may have contributed to the program’s instability and ultimately
to its cost growth and schedule delays. By contrast, the F/A18E/F program drew on preexisting relationships and contractor
expertise to minimize the technology risks involved in the project. The program also implemented a number of acquisition
reform strategies designed to control costs and schedule, such as
the principle of cost as an independent variable (CAIV). These
Summary
xix
measures helped keep the F/A-18E/F program on schedule and
within cost during EMD (see pp. 13–27).
• Concurrent development of new technology created greater
technical challenges for the F/A-22, while incremental
improvements reduced technical risk in the F/A-18E/F. The
F/A-22 cost growth was mainly the result of design challenges in
the airframe (arising from stealth requirements), the integrated
avionics suite, and the new propulsion system. Some of these
challenges were either assumed to be low risk or were not
accounted for in the initial program cost estimates. Also, concurrent development and integration of all aspects of the F/A-22
may have compounded the cost growth and schedule slippage.
In contrast, the F/A-18E/F requirement was met by incremental
improvements with minimal stealth requirements, a mostly
existing avionics system from its predecessor aircraft, and a
derivative engine design. This low-risk approach may have contributed to the F/A-18E/F’s stable cost and schedule (see pp.
29–46).
• The programs allocated different portions of their budgets
for management reserve. Management reserve is a budget withheld for management control purposes and is mostly used to
cover unknown problems in a development program. The F/A22 program allocated only about 2 percent of its budget to management reserve. This reserve was depleted in about the first year
of the EMD effort because of the technical challenges described
above. By contrast, the F/A-18E/F program maintained a substantial management reserve, roughly 10 percent of contract value. As the program proceeded through its development and
unforeseen problems arose, the amount of management reserves
covered these problems and was decreased accordingly (see pp.
47–53).
This report provides the Air Force and other services with lessons learned to improve the acquisition of such future and current
weapon systems as the Joint Strike Fighter and such other hardware
systems as unmanned aerial vehicles and missile programs. Our major
xx
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
lessons learned for the Air Force acquisition decisionmakers are the
following:
• Early, realistic cost and schedule estimates set the program on
the right path for the rest of the development program.
• A stable development team structure, proper team expertise,
clear lines of responsibility and authority, and a lead contractor
responsible for overall program progress are critical to program
success.
• An experienced management team and contractors with prior
business relationships help eliminate early management problems.
• Concurrent development of new technology for the airframe,
avionics, and propulsion adds significant risk.
• Reducing the cost and risk of avionics should be a key focus of
the concept development phase. Avionics is a considerable cost
driver of modern weapon systems, and new concepts should be
demonstrated along with the new airframe designs.
• Preplanned, evolutionary modernization of high-risk avionics
can reduce risk and help control costs and schedules.
• Careful monitoring of airframe weight is important. Airframe
weight instability is an early indicator of problems.
• Earned value management (EVM) data should be used to monitor and manage program costs at the level of integrated product
teams (IPTs).
Appropriate use of management reserve can help address program cost risk and can mitigate cost growth.
Acknowledgments
The authors of this study had extensive discussions with knowledgeable Air Force and Navy professionals. We owe gratitude to those
who generously provided us with data and shared their insights.
First, we would like to thank Lt. Gen. John Corley, SAF/AQ,
and Blaise Durante, SAF/AQX, for sponsoring this project. We are
also grateful to Maj. Gen. Rick Lewis for his review of the draft
report. From the Air Force Cost Analysis Agency (AFCAA), we thank
Richard Hartley, its director, and Jay Jordan, its technical director.
We also extend our appreciation to Ranae Woods, Scott Adamson,
and Patrice Jones from AFCAA for providing data, historical documents, and insights. From Naval Air Systems Command, we thank
Brenda Bizier, Chris Bowser, Joe Landfield, Al Pressman, Mark
Mutschler, and Jim Myers for sharing their data and insights with our
team. Col. Bob Lyons, USAF (Ret.), from the Air Force Materiel
Command/Acquisition Excellence office shared his insights of the
F/A-22 avionics program and for that we are grateful. We also thank
Chris Zearley of ASC/WFABA for sharing information. Mike Novak,
USD(AT&L), and Tom Coonce and Fred Janicki, OSD PA&E, also
shared their thoughts and information and for that we are grateful.
We are also indebted to William Stussie, former Deputy Assistant
Secretary of Navy for Acquisition and former F/A-18E/F program
manager, currently with Raytheon, for sharing his program knowledge. Furthermore, we thank Lane Pierrot and Jo Ann Vines from the
Congressional Budget Office for sharing their insights and experience
on the two programs.
xxi
xxii
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
Our RAND colleagues Cynthia Cook and John Schank
reviewed this document. Their comments and thorough review occasioned many changes and improved the quality and the content of
this report. For that we are grateful. Finally, we are additionally
indebted to our colleagues Bob Roll, PAF Resource Management
Program’s Director, and Jack Graser, the former head of Weapon
System Costing Project, for their leadership; Jack Welch, for sharing
his F/A-22 program insights; Robert Ellenson, for his help with
documentation; and Nate Tranquili for his research and administrative assistance.
Abbreviations
ACS
ACWP
AESA
AMC&D
ASTE
ATA
ATD
ATF
ATFE
ATFLIR
AWACS
BAC
BCWP
BCWS
BUR
BVR
CAIG
CAIV
CARD
Advanced Crew Station
Actual Cost of Work Performed
Active Electronically Scanned Array
Advanced Mission Computer and Displays
Advanced Strategic and Tactical Expendables
Advanced Tactical Aircraft
Achieved to Date
Advanced Tactical Fighter
Advanced Tactical Fighter Engine
Advanced Targeting Forward-Looking Infrared
(pod)
Airborne Warning and Control System
Budget at Completion
Budgeted Cost of Work Performed
Budgeted Cost of Work Scheduled
Bottom-Up Review
Beyond visual range
Cost Analysis Improvement Group
Cost as an Independent Variable
Cost Analysis Requirements Description
xxiii
xxiv
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
CBB
CBO
CCDR
CMWS
CNI
CPR
CTC
DCS
Dem/Val
DoD
DoDD
DOT&E
DT
DTW
DVMC
EMD
EVM
EW
FMS
FOTD
FY
GAO
IDECM
ILS
IOC
IOT&E
IPT
JET
Contract budget baseline
Congressional Budget Office
Contract Cost Data Report
Common Missile Warning System
Communication, Navigation, and Identification
Cost Performance Report
Contract Target Cost
Digital Communication System
Demonstration and validation
Department of Defense
DoD Directive
Director of Operational Test and Evaluation
Development test
Design-to-weight
Digital Video Map Computer
Engineering and manufacturing development
Earned Value Management
Electronic warfare
Foreign Military Sales
Fiber Optic Towed Decoy
Fiscal year
General Accounting Office
Integrated Defensive Electronic Countermeasures
Integrated Logistics Support
Initial operational capability
Initial operational test and evaluation
Integrated product team
Joint Estimating Team
Abbreviations
JHMCS
LO
LRE
LRIP
NATF
NAVAIR
OFP
OSD
OTB
P3I
PA&E
PAF
PDR
PGMs
PIDS
PM
QDR
RAM
RAS
RDT&E
RFI
RFP
RO
RUG I
RUG II
SAM
SAR
SE/PM
Joint Helmet-Mounted Cuing System
Low observable
Latest revised estimate
Low-rate initial production
Navy Advanced Tactical Fighter
Naval Air Systems Command
Operational Flight Program
Office of the Secretary of Defense
Over-target baseline
Preplanned Product Improvement
Program analysis and evaluation
Project AIR FORCE
Preliminary Design Review
Precision-guided munitions
Positive Identification System
Program manager
Quadrennial Defense Review
Radar-absorbing materials
Radar-absorbing structures
Research, development, test, and evaluation
Request for information
Request for proposal
Reduced observable
Radar Upgrade (from APG-65)
Radar Upgrade (from APG-73)
Surface-to-air missile
Selected Acquisition Report
Systems engineering and program management
xxv
xxvi
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
SHARP
SLOCs
SMS
SPO
ST&E
STOL
T&E
T/R
TAMMAC
USD (AT&L)
VLO
VMS
WBS
Shared Advanced Reconnaissance Pod
Source lines of code
Software management system
System Program Office
System test and evaluation
Short takeoff and landing
Test and evaluation
Transmitter and receiver
Tactical Air Moving-Map Capability
Under Secretary of Defense (Acquisition,
Technology, and Logistics)
Very low observable
Vehicle Management System
Work Breakdown Structure
CHAPTER ONE
Introduction
Two Multirole Fighter Aircraft Programs Emerged at the
End of the Cold War
From the late 1980s through the present, the U.S. Air Force and the
U.S. Navy have been engaged in acquiring two new multirole fighter
aircraft platforms. The Air Force has pursued the F/A-22, the world’s
first supersonic stealth fighter,1 while the Navy has developed the
F/A-18E/F, a carrier-capable fighter with air-to-air, interdiction, and
close air support capability. Currently, the F/A-22 is in the late stages
of development, while the F/A-18E/F is in full-rate production and
has already been deployed in Operation Enduring Freedom and
Operation Iraqi Freedom.
Although these aircraft entered the Engineering and Manufacturing Development (EMD) at approximately the same time, their
missions and performance goals differ widely. These differences are
shaped in part by changes in the threat environment following the
end of the Cold War. The F/A-22 was originally designed to counter
what was perceived to be the growing Soviet fighter threat. Specifically, the Air Force wanted a new air-to-air fighter capable of defeating improved Soviet Su-27 and MiG-29 aircraft. The Air Force
postulated that these new Soviet aircraft would have the capability to
detect and fire on enemy fighters at lower altitudes (known as “lookdown, shoot-down capability”) and that they would be fielded in
____________
1
The Lockheed F-117, the first stealth combat aircraft, is a subsonic attack aircraft used
primarily for air-to-ground operations.
1
2
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
large numbers. Supported by a Soviet version of the U.S. Airborne
Warning and Control System (AWACS), this new threat would
require a technically superior U.S. fighter to counter it. Because the
F-117 could perform the attack role, what was needed, according to
the Air Force, was a new, sophisticated and stealthy air-to-air fighter
to replace the aging McDonnell Douglas F-15 (Aronstein, Hirschberg, and Piccirillo, 1998, p. 41). This vehicle was known as the
Advanced Tactical Fighter (ATF). When the postulated Soviet threat
failed to materialize, however, questions were raised concerning the
need for large numbers of a technologically advanced fighter dedicated solely to the air-to-air mission.2 Perhaps in response to this
criticism, in summer 2002 the Air Force, arguing that it needed to be
able to counter a growing proliferation of surface-to-air missile
(SAM) threats, added suppression of enemy air defenses to the new
platform’s list of missions and redesignated the aircraft the F/A-22.
Thus, the F/A-22 has evolved into a multirole fighter.
The F/A-18E/F was not designed to counter a new threat or an
improved Soviet capability. In fact, by the late 1980s the Navy had
concluded that the threat to its battle group from enemy aircraft had
diminished sufficiently that a replacement for the F-14 was not warranted. Consequently, the E/F program was initiated essentially to
address the shortcomings of the F/A-18A/B and C/D models—
specifically, limited bring-back capability3 and less than desired
range—that limited their ability to carry out missions associated with
littoral warfare. As stated in the F/A-18E/F Cost Analysis Require____________
2
In April 1995, the Defense Science Board F-22 Task Force final report alluded to looking
at relaxing some F-22 requirements. Specifically, it recommended that “there are substantial
margins throughout the F-22 specifications and a very capable aircraft would result even if
performance fell somewhat short of meeting many or even all of these specs. It is therefore
important that the [System Program Office] and [the Office of the Secretary of Defense] not
take a rigid stance on meeting all specs but rather, as the program progresses, look at the
overall performance, cost and schedule impacts on deciding which, if any, performance areas
need further work.”
3
“Bring back” is the ability of an aircraft to land on a carrier with a certain amount of
unused ordnance and fuel. This is increasingly important with costly precision munitions
becoming more the norm for combat ordnance.
Introduction
3
ments Description (CARD),4 “the objective of the F/A-18E/F program is to develop, test, produce, and deploy an upgraded F/A-18
with increased mission range, increased aircraft carrier recovery payload, additional growth potential, and enhanced survivability.” With
the increased attention during Operation Desert Storm from the use
of precision-guided munitions (PGMs), the F/A-18E/F was designed
with a greater “bring-back” capability. Thus, from the beginning, the
F/A-18E/F was designed as a multirole fighter to perform the same
missions and counter the same threats as earlier models of the F/A-18
but with some incremental increase in capability.
Tables 1.1 and 1.2 illustrate the different performance goals as
well as other metrics of each platform compared with the aircraft they
were intended to replace.5 The performance metrics are speed, payload, range, and engine thrust. We also compare the avionics weight
as a proxy for complexity. Finally, we compare the stealth feature of
each platform. Because of the classified nature of military aircraft
stealth technology, we use a subjective method to provide the reader
with a qualitative relative comparison. These qualitative categorizations are very low observable (VLO), low observable (LO), reduced
observable (RO), and minimum treatment (MIN). VLO airframes
____________
4 The
CARD documents the ground rules and assumptions for a program at a specific point
in time, typically at a major milestone decision point. It is required to be provided by DoDI
5000.2 at milestones B and C or when an economic analysis is required. The information in
the CARD describes the physical, performance, and contract assumptions behind the program and should be used as the basis of the program office estimate. The program office or
sponsoring agency is responsible for preparing the CARD in draft format at initiation of the
independent cost activities and a final version prior to the milestone decision. Areas of
information include a system overview, risk, operational concept, quantities, manpower
requirements and activity rates, schedule, acquisition plan, development plan, facilities
requirements, track to prior CARD, and Contract Cost Data Report (CCDR) plan.
5
The information on these tables for the F-14, F/A-18A/B, F-16, and F-15A/D came from
the Institute for Defense Analysis Report, Military Tactical Aircraft Development Costs
(R-339), 1988. The values for the F/A-18E/F and the F/A-22 came from the CARDs,
unclassified program office briefings, and other open source publications. One difficulty of
this depiction is ensuring common definitions of performance across the platforms. Perfectly
congruent definitions were not possible for comparing all these platforms because of the
varied sources used and the visibility of performance characteristics.
4
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
Table 1.1
Comparison of Some of the F/A-22 Performance Gains over Legacy
Aircraft
Performance Characteristics
Speed (knots)
Payload (pounds)b
Range (nautical miles)
Avionics Weight (pounds)
Engine Thrust (pounds) c
Stealthd
F/A-22
a
1,434
17,589
415
1,891
35,000
VLO
F-15C/D
F-16C/D
1,434
26,635
648
1,250
23,840
MIN
1,184
20,094
346
1,045
23,840
MIN
a
Mach 2 class.
Payload is defined as the useful load, which equates to the weight that
can be carried in stores (weapons, pods, and fuel tanks) and the weight of
the internal fuel. It was generally calculated by subtracting the empty
weight of the aircraft from the maximum (or gross) takeoff weight of the
aircraft.
c
Thrust is typically the maximum thrust of each engine.
d
Stealth was given a subjective range judged by the authors to include
the following points: MIN, RO, LO, and VLO.
b
Table 1.2
Comparison of Some of the F/A-18E/F Performance Gains over Legacy
Aircraft
Performance Characteristics
Speed (knots)
Payload (pounds)
Range (nautical miles)
Avionics Weight (pounds)
Engine Thrust (pounds)
Stealth
F/A-18E/F
F-14
F/A-18C/D
1,059
35,436
520
1,411
20,832
RO
1,170
31,663
521
2,821
20,900
MIN
1,029
28,850
363
1,289
17,775
MIN
include not only a significant amount of Radar-Absorbing Materials
(RAM) used in the treatment of all airframe surfaces but also RadarAbsorbing Structures (RAS) and the overall shaping of the airframe.
The use of RAM and RAS progressively decrease from LO to MIN.
As shown on Table 1.1, the F/A-22 was designed to provide significant performance gains over the F-15 and F-16 fighters. Most
important was the inclusion of cutting-edge innovations in stealth
and an integrated avionics suite. The F/A-18E/F, as shown on Table
1.2, was intended to provide some incremental improvements over
the C/D model, especially in the areas of stealth, range, and payload
capacity. However, it is important to note that the F/A-18E/F sought
Introduction
5
lower performance in some areas compared with the F-14. As we discuss in Chapter Two, these lower performance goals were motivated
by concerns about the platform’s cost.
These Programs Performed Differently During Their
Development Phases
The acquisition of new combat aircraft is a lengthy process involving
a series of milestones and approvals that must be obtained from the
Department of Defense (DoD). This process is intended to ensure
that a new platform can provide its promised technical capabilities in
a timely and cost-effective way. The F/A-22 and F/A-18E/F acquisitions were modeled after the earlier version DoD instruction 5000.2.6
Figure 1.1 illustrates this process. The old instruction divided the
acquisition process into two distinct phases: the research, development, test, and evaluation (RDT&E) phase and the production
phase. In the old instruction, RDT&E was broken down into the
demonstration and validation (Dem/Val) phase and EMD phase.
The F/A-22 and F/A-18E/F programs are important examples of the
stability or instability that new aircraft platforms may experience
during the EMD phase. Figure 1.2 compares the schedule and cost
planned for each program at the beginning of its development phase
with the actual time and cost it took to complete this phase. As
shown on the left, the F/A-22 has exceeded its original schedule by
more than 52 months as of the date of the last Selected Acquisition
Report (SAR) examined (December 31, 2001), while the F/A-18E/F
was virtually on time. The total cost of developing the F/A-22 grew
by $7.6 billion in fiscal 1990 dollars compared to the F/A-18E/F
program, which met its original cost estimates.
____________
6
The current version has streamlined the acquisition process by eliminating unnecessary
reports and reviews.
6
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
Figure 1.1
Acquisition Process for F/A-22 and F/A-18E/F Programs
Research, Development, Test, and Evaluation (RDT&E)
Concept
Exploration
Milestone 0
Demonstration
and Validation
(Dem/Val)
Milestone I
Engineering and
Manufacturing
Development
(EMD)
Milestone II
Production and
Fielding/
Development
Milestone III
RAND MG276-1.1
These differences represent gradual trends evident over the
course of the development phase of each program. For example, Figures 1.3 and 1.4 display the estimated completion date of each step in
the EMD phase (the y-axis) as reported in annual SARs (the x-axis).
A flat line indicates that the program objective was met on the originally planned date, while a rising line depicts a slip in schedule. As
shown on Figure 1.3, every major milestone of the F/A-22 development program slipped. For example, initial operational test and
evaluation (IOT&E) and Milestone III completion dates slipped by
more than two years. Moreover, we would like to emphasize that the
test and evaluation (T&E) program is not complete and program
development is not over; therefore further slippage might occur. In
contrast, as shown in Figure 1.4, very few schedule slips occurred in
the F/A-18E/F program, with the exception of the initial operational
capability (IOC) date.
Similarly, Figure 1.5 depicts the estimated development costs for
each program as reported in the annual SARs. As shown at the top,
the F/A-22 RDT&E, which includes both Dem/Val and EMD,
Introduction
7
Figure 1.2
F/A-22 Experienced Schedule Slips and Cost Growth, While the F/A-18E/F
Completed Development on Time and on Cost
Schedule
Total cost of development*
30
160
25
140
120
Billions of 1990 $
Time between milestones II and III
(months)
180
100
80
60
20
15
10
40
5
20
0
0
F/A-22
F/A-18E/F
F/A-22
F/A-18E/F
*Includes government costs.
SOURCE: Selected Acquisition Reports.
RAND MG276-1.2
increased at a gradual rate. The F/A-18E/F, shown at the bottom,
showed very little fluctuation during the entire development period.
How do the F/A-22 and F/A-18E/F experiences compare with
other Air Force and Navy tactical fighter programs in recent years?
Historical experience suggests that such programs often take longer
and cost more to develop than originally planned. However, the F/A22 and F/A-18E/F represent exceptional cases. The F/A-22 cost and
schedule is substantially higher than historical combat aircraft cost
and schedule growth, and the F/A-18E/F is substantially lower than
that average. Figure 1.6 shows how well previous fighter aircraft have
met their estimated schedules for achieving first flight, first production, and IOC. A value of 1.00 indicates that the program met its
schedule goal. As shown on the right, the degree of slippage in the
F/A-22 schedule far exceeds that of previous tactical fighter programs.
8
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
Figure 1.3
F/A-22 Schedule Estimates Grew Throughout the Development Phase
F/A-22
Jan-07
IOC
Reported date of event
Jan-05
Milestone III
Jan-03
Complete DT&E
Jan-01
LRIP contract award
Complete IOT&E
Jan-99
Start IOT&E
First flight, start DT&E
Jan-97
Engine initial flight release
Jan-95
Jan-93
Jan-91
Dec
91
Airframe critical design review complete
Airframe preliminary design review complete
Milestone II
Dec
92
Dec
93
Dec
94
Dec
95
Dec
96
Dec
97
Dec
98
Dec
99
Dec
00
Dec
01
Date of SAR
RAND MG276-1.3
Figure 1.4
F/A-18E/F Schedule Estimates Remained Steady Throughout the
Development Phase
F/A-18E/F
Jan-07
Reported date of event
Jan-05
Jan-03
IOC
Jan-01
Complete DT&E
Jan-99
First flight, start DT&E
Jan-95
Jan-91
Dec
91
Complete IOT&E
LRIP contract award
Jan-97
Jan-93
Milestone III
Engine initial flight release
Airframe critical design review complete
Airframe preliminary design review complete
Milestone II/IV
Dec
92
Dec
93
Dec
94
Dec
95
Dec
96
Dec
97
Date of SAR
RAND MG276-1.4
Dec
98
Dec
99
Dec
00
Dec
01
Introduction
Cost (FY 1990 $B)
Figure 1.5
F/A-22 Costs Rose Steadily, While F/A-18E/F Costs Remained Stable
30
25
20
15
10
5
0
F/A-22 RDT&E $
RDT&E funding
(FY 1990 $B)
Dec Dec Dec Dec Jun Dec Dec Jun Dec Jun Dec Dec Sep Dec Dec
91 92 93 94 95 95 96 97 97 98 98 99 01 01 02
Date of SAR
30
25
20
15
10
5
0
F/A-18E/F RDT&E $
Jul
92
Dec Dec Dec Dec
92
93
94
95
Jun May Dec Dec Dec Sep Dec Dec
96
97
97
98
99
00
01
02
Date of SAR
SOURCE: Selected Acquisition Reports 1991–2002.
RAND MG276-1.5
Figure 1.6
F/A-22 Schedule Slippage Is Higher Than the Historical Average
2.00
FF
First prod
IOC
EMD schedule growth factor
(from MS II Est = 1.0)
1.80
1.60
1.40
1.57
1.38
1.20
1.00
1.76
0.96
1.19
1.17
1.16
1.00 1.00
1.03
1.12
1.07
1.00
1.00 1.00
1.02 1.00
0.85
0.80
0.60
0.40
0.20
0.00
F-14
F-15
F-16
F-18A
Aircraft
B-1B
NOTE: Numbers on top of bars reflect current estimate.
RAND MG276-1.6
F/A-18E/F F/A-22
On time
9
10
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
The F/A-22 program took 76 percent longer than estimated to
achieve first flight and 57 percent longer to reach first production,
and it is expected to take 19 percent longer to reach IOC. The next
set of bars to the left indicate that the F/A-18E/F took only 2 percent
longer than estimated to reach first flight, reached first production on
schedule, and took only 12 percent longer to reach IOC.
The results are similar if we compare the cost growth figures for
previous tactical fighter programs. Figure 1.7 displays cost growth (or
reduction) as measured by dividing the last reported cost in the program SAR at the end of the program by the original cost estimate at
Milestone II. A value of 1.00 represents no cost growth. As the figure
shows, the F/A-22 cost growth is second only to that of the F-14,
with the potential to continue growing until the development phase
is complete. The F/A-18E/F is lower than the historical average and is
the only program to complete its development under cost.
Figure 1.7
F/A-22 Cost Growth Is Higher Than the Historical Average
1.60
1.45
EMD schedule growth factor
(from MS II Est = 1.0)
1.40
1.21
1.20
1.33*
1.29
1.15
1.13
1.09
0.98
1.00
0.80
0.60
0.40
0.20
0.00
F-14
F-15
B-1A
F-16
F-18A
Aircraft
*Based on March 03 program office estimate.
RAND MG276-1.7
Avg. =
1.22
B-1B
F/A- F/A-22
18E/F
On cost
Introduction
11
Purpose of This Report
The schedule and cost overruns in the F/A-22 program have generated considerable concern from DoD and Congress, leading to close
scrutiny of the program and reductions in the number of aircraft to
be produced. The office of the Assistant Secretary of the Air Force for
Acquisition asked RAND Project AIR FORCE (PAF) to investigate
the reasons behind the cost growth and schedule delay of the F/A-22
program and those contributing to the cost and schedule stability of
the F/A-18E/F program during EMD. This report examines the
acquisition strategies employed by the F/A-22 and F/A-18E/F programs from their inception through the Dem/Val and EMD phases.
The analysis is based on various cost and schedule reports available to
PAF as well as data and information available in open sources. For
instance, the SARs, CCDRs, and Cost Performance Reports (CPRs)
were the main sources of data for the cost and schedule growth analysis. Other documents, such as Cost Analysis Requirements Description (CARD), contractor’s weight reports, General Accounting Office
(GAO) reports, and published articles and reports were also examined. The purpose of this analysis is to derive lessons that the Air
Force and other services can use to improve the acquisition of such
future aircraft as the Joint Strike Fighter and such other systems as
unmanned aerial vehicles and missile programs.
Organization of This Report
Chapter Two discusses the acquisition strategies and industrial base
issues that affected the performance of each program from inception
through the EMD phase. Chapter Three analyzes the specific technology issues that surfaced during each program’s EMD phase and
discusses their effect on schedules and costs. Chapter Four evaluates
the management approaches of each program to determine how
schedules and costs were controlled during EMD. Chapter Five
summarizes our conclusions and provides a list of lessons learned that
the Air Force acquisition community should consider in developing
12
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
future weapon systems. Finally, Appendix A discusses the Department of Defense oversight and congressional interests in each program, some of which may have posed further challenges.
CHAPTER TWO
Acquisition Strategies and Industrial Base Issues
This chapter discusses the acquisition strategies employed by the F/A22 and F/A-18E/F programs from their inception through their
EMD phases. We examine the methods used by each program to
solicit and evaluate contractor proposals during the Dem/Val phase
and the division of work among contractors during the EMD phase.
As we shall see, concerns about the needed mix of technical expertise
and other industrial base issues led the F/A-22 program to distribute
the work equally among the three contractor team members in the
program, which resulted in an artificial distribution of work during
the EMD phase. This strategy may have contributed to the schedule
and cost problems experienced in the program. By contrast, the F/A18E/F program drew on preexisting relationships and contractor
expertise to minimize the technology risks involved in the project. In
addition to having a team with historical ties, the program implemented a number of acquisition reform strategies designed to control
costs and schedules.
The F/A-22 Program Sought to Maximize Participation by
Multiple Contractors
We begin with an overview of the F/A-22 program’s acquisition strategy from the ATF program in the early 1980s to the Dem/Val and
EMD phases.
13
14
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
The F/A-22 Represented an Important Opportunity for the Combat
Aircraft Industrial Base
In November 1981, DoD formally launched the ATF program.1 As
noted in Chapter One, this program was intended to produce a
replacement for the McDonnell Douglas F-15, then the Air Force’s
premier air superiority fighter, with a supersonic stealth aircraft that
would use state-of-the-art advances in aerospace technologies and
capabilities.
Very soon, it appeared to the U.S. aerospace industry that the
ATF would be the only opportunity to develop an all-new, cuttingedge-technology supersonic fighter for the next decade or more. This
was because in 1983, because of budget constraints and competing
priorities, the U.S. Navy put on hold its plans for procuring a new
common fighter (labeled the VMFX) to replace both the Grumman
F-14 fleet air defense fighter and the Grumman A-6 attack aircraft.
The Navy replaced the VMFX program with a new program to
upgrade existing F-14s and A-6s and to procure a new stealthy subsonic attack aircraft, called the Advanced Tactical Aircraft (ATA). 2
Thus, after 1983, U.S. contractors could expect at most only one
major development program for an all-new supersonic air-superiority
fighter—the ATF—and one other program for a subsonic attack
aircraft—the ATA—at least over the following decade.
All nine then-active U.S. defense aerospace prime contractors
hoped to compete for the ATF full-scale development effort: General
Dynamics, McDonnell Douglas, Lockheed, Northrop, Boeing,
Grumman, Rockwell North American, Vought, and Fairchild. As
early as May 1981, the Air Force had issued a Request for Information (RFI) seeking conceptual design studies to all nine contractors.
____________
1 The
ATF program was the forerunner of the F-22 program, which later became the F/A-22
program. The Defense Resources Board first approved ATF program start-up on November
23, 1981. The most detailed unclassified history of the early period of the F-22 program
through the end of the system demonstration and validation phase is found in Aronstein,
Hirschberg, and Piccirillo (1988).
2 In
November 1984, two teams won preliminary design contracts for the ATA: McDonnell
Douglas teamed on an equal basis with General Dynamics, while Northrop led a team that
included Grumman and Vought.
Acquisition Strategies and Industrial Base Issues
15
At this early stage, competition among contractors was fierce. Every
participant knew that many of the losers would ultimately have to
withdraw as stand-alone prime contractors from the fighter-attack
aircraft market sector. Indeed, the aftermath of the ATF and ATA
competitions witnessed the beginning of a massive corporate consolidation and downsizing that transformed the very structure of the U.S.
aerospace industry.
As was the case during the early stages of the F-X (F-15) program nearly two decades earlier, considerable debate existed initially
within the Air Force and DoD regarding the most desirable mission
focus and performance characteristics for the ATF. During 1982, a
consensus began to emerge that a modified version of the F-15 or
F-16 could perform the air-to-ground role, permitting the ATF to be
optimized for air superiority. 3 By mid 1983, the ATF had clearly
been defined as an F-15 air superiority fighter replacement.
Following the emergence of this consensus, the Air Force
awarded concept development contracts in September 1983 to further refine the design concepts for the ATF. Nine months later, in
May 1984, after numerous design iterations, the seven remaining participating airframe prime contractors concluded the ATF concept
development phase by submitting their final reports.4 At this time Air
Force acquisition officials planned to select as many as four prime
contractors late in 1985 to begin a 32-month concept Dem/Val
phase.
By this time it was generally recognized that the extremely
demanding ATF performance requirements being refined by the Air
Force would pose substantial technological, design, engineering, and
____________
3
General Dynamics and McDonnell Douglas developed prototypes of their competing
modification proposals called F-16XL and F-15 Strike Eagle, respectively, which first flew in
1982. Two years later, the Air Force selected McDonnell Douglas’s entry for full-scale development as the F-15E. The F-16XL and F-15E programs had an effect on the ATF program
parallel to the decision to procure the LTV A-7 in the 1960s, a decision that permitted the
F-X (F-15) requirement to focus on the mission of air superiority. See Lorell and Levaux
(1998).
4
Early in the concept development stage, Vought and Fairchild opted out, leaving the
remaining seven contractors to compete for the contract.
16
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
manufacturing challenges for industry. For example, the Air Force
decided to seek supercruise capability (the ability to cruise at supersonic speed without afterburner) and engines with vectoring nozzles
for short takeoff and landing (STOL) capability and greater agility,
combined with stealth and F-15/F-16-class maneuverability. Planners
decided to require development of the first fully integrated fighter
avionics system, and incorporate revolutionary new technologies into
the fire control radar based on a solid-state active electronically
scanned array (AESA) technology.
In recognition of the daunting technological challenges posed by
the program, the Air Force soon funded technology development and
risk-reduction programs applicable to the ATF, such as efforts to
develop new materials applicable to stealth, antenna arrays for AESA
radars, and the F-15 STOL and Maneuver Technology Demonstrator
program (S/MTD or NF-15B). In September 1983, the government
launched the joint Advanced Tactical Fighter Engine (ATFE) program by awarding contracts to Pratt & Whitney and General Electric
to develop advanced technology engine demonstrators. During 1985,
seven contracts were awarded for predesign studies on the integrated
avionics program called Pave Pillar. As anticipated, the Air Force sent
out requests for proposals (RFPs) for a Dem/Val phase for the ATF
in October 1985.
The Packard Commission Led to a Two-Team Approach for
Demonstration and Validation
The F/A-22 program’s Dem/Val phase was affected by changes
brought by a Presidential Blue Ribbon Commission, also known as
the Packard Commission. In 1986, the commission reviewed the
Department of Defense’s management and acquisition process and
recommended a number of reform initiatives. Most important among
these reforms were the requirements for two competing contractors to
develop technology demonstration prototypes during Dem/Val,
greater emphasis on performance specifications as opposed to detailed
technical specifications, and contractor sharing in development costs.
These initiatives were intended to reduce technological risk and to
Acquisition Strategies and Industrial Base Issues
17
maintain competitive pressures to promote cost savings and greater
innovation throughout the early phases of development.
From the beginning, the ATF program was managed within the
standard regulatory and organizational structure of traditional major
Air Force acquisition programs. Nonetheless, the Packard Commission reforms led to important program innovations. The Air Force
reorganized the ATF System Program Office (SPO) around
government-industry integrated product teams (IPTs) and granted
the competing contractors considerable control over the structuring
of the prototype technology demonstration effort. IPTs are composed
of a team of individuals representing various engineering and management functions. It is a key component of the integrated product
development approach. The F/A-22 program was one of the first
major acquisition programs to implement the IPT structure before
DoD mandated its use in 1995.5 These innovations led to a restructuring and extension of the Dem/Val program schedule by two years,
with a slip of the anticipated Milestone II decision (for the full-scale
development phase) from December 1988 to December 1990.6
The seven participating prime contractors all responded to the
Dem/Val RFP with serious design proposals. Partly because of concerns over the future of the industrial base, the Air Force encouraged
the prime contractors to team together so that broad industrywide
participation could be maintained at least through the Dem/Val program. The Office of the Secretary of Defense (OSD) granted Milestone I (now A) approval and the Air Force selected Lockheed and
Northrop in October 1986 to lead competing teams during a
planned 54 month Dem/Val phase. Lockheed led a team composed
of General Dynamics and Boeing, while Northrop teamed with
McDonnell Douglas. Only one team, of course, would receive the
final award for full-scale development at the end of the competitive
____________
5 For
6
a more detailed discussion of the IPT structure, see Cook and Graser (2002).
In October 1989, the Defense Acquisition Board (DAB) slipped completion of the
Dem/Val stage another six months to June 1991. DoD has since renamed the full-scale
development phase twice for all major acquisition programs: first as EMD, then as the system development and demonstration (SDD) phase.
18
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
Dem/Val stage. GE and P&W received new contracts to continue
technology development for the newly named ATFE program.
In mid 1990, the two ATF contractor teams began flight-testing
their technology demonstrator prototypes: the Lockheed YF-22 and
the Northrop YF-23. On April 23, 1991, more than five years after
the beginning of the Dem/Val phase and following an extensive flight
test program, the Air Force selected the Lockheed/General Dynamics/
Boeing YF-22 for full-scale development as the next Air Force air
superiority fighter, and designated it the F-22. In August, the Air
Force awarded Cost Plus Award Fee EMD contracts to Lockheed and
Pratt & Whitney. Despite the technical challenges, most observers
consider the Dem/Val program to have been well managed and
highly successful.7
The F/A-22 Program Created an Artificial Split in Workload
Despite the successful completion of the Dem/Val phase, the division
of work among contractors during the EMD phase proved to be a
source of problems. One issue was the division of EMD work equally
among the three major contractors. Lockheed Martin, the prime contractor, was clearly the leader in stealth aircraft design with F-117
experience. As team members, it chose General Dynamics for its
fighter aircraft experience and Boeing for its innovative manufacturing approaches that had made it the industry leader. Both the contractors and the government justified the work split as a way to
ensure that each contractor maintained its capability to remain competitive as prime contractors for future business.
This work split may have led to an artificial distribution of the
development effort. As shown in Figure 2.1, the F/A-22 EMD work
was divided among the three contractors in such a way that the major
elements of the airframe, avionics, and support systems were given to
different team members. For instance, although the F/A-22 avionics
suite is a highly integrated system, various elements are managed and
controlled by different team members. Other F/A-22 design features
____________
7 For
example, see Myers (2002) .
Acquisition Strategies and Industrial Base Issues
19
are highly integrated as well, and combining the systems developed
by different team members has proven to be a challenge for the prime
contractor, as discussed further in Chapter Three.
The F/A-22 Did Not Have a Stable Industrial Base
Another source of problems during EMD was the F/A-22’s lack of an
existing industrial base and members of a supplier network experienced in working with one another in fabricating, assembling, and
producing the high-technology components necessary for the new
aircraft. Three problems in particular delayed the project’s schedule
and increased its costs.
First, the EMD program management and design oversight
responsibilities were moved from Burbank, California, to Marietta,
Georgia, in January 1991. Lockheed and the U.S. government publicized the move as a cost-cutting measure. The Marietta facility, mainly a production facility with transport aircraft (C-141, C-5, and CFigure 2.1
F/A-22 EMD Work Was Artificially Distributed Among the Contractors
Support
Avionics
Airframe
Lockheed Martin
Fort Worth
(formerly GD)
• Center fuselage
• Armament
• Electronic warfare
• CNI (TRW)
• Stores management
system
• Internal navigation
system
Lockheed Martin
Marietta
(Overall weapons
system integration)
•
•
•
•
•
•
•
•
•
Foward fuselage
Vertical tails
Flaps
Landing gear
Final assembly and
checkout
Avionics architecture
Controls and displays
Air data system
Apertures
• Support system
SOURCE: Lockheed Martin F/A-22 website.
RAND MG276-2.1
Boeing Seattle
• Wings
• Aft fuselage
• APU
• Avionics subsystem
integration lab
• Flying test bed
• Radar (NGC)
• Training system
20
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
130) experience, lacked an in-house design team that understood the
technology and innovation required for a state-of-the-art air superiority fighter.8 Less than 10 percent of the core team that had
worked on the ATF during Dem/Val as well as the early stages of the
EMD phase moved from Burbank to Marietta. Because the facility
had no previous experience with fabricating high-technology aircraft
or parts, it has taken longer than expected to assemble and deliver test
aircraft during EMD. Moreover, in 1999, the program experienced
problems fabricating the F/A-22’s wings from composites. This
problem forced Boeing to qualify a second supplier to speed deliveries, thereby exacerbating the cost and schedule problems. A machinists’ strike in 2002 further delayed the delivery of test aircraft. This
inability to attract engineers and managers who gain specialized experience during the early phase of development from Burbank to Marietta along with Marietta’s lack of a design team capable of meeting
the F/A-22’s engineering challenges arguably may have been the root
of many problems during development.
Second, the F/A-22 faced the problem of overcapacity in its
production facilities when it entered production. The Marietta facility was expanded in 1992 to accommodate a production line capable
of assembling 48 F/A-22s per year. Since the early 1990s, however,
peak production has been cut to 32 per year. The likelihood of foreign military sales (FMS) to increase production rates is also dim. The
F-22’s high cost makes it an unlikely candidate for a significant number of FMS, and DoD and the State Department may be reluctant to
release the advanced technology in the F/A-22 to overseas customers.
Finally, the GAO expects that lower-than-expected FMS of the
C-130J, also assembled at Lockheed Martin’s Marietta facility, will
raise overhead costs at the plant. Some of these costs would ultimately
have to be borne by the F-22 program (GAO, 2000, p. 16).
Concerns about the needed mix of technical expertise and other
industrial base issues had led the F/A-22 program management to
____________
8
According to press reports, Lockheed Martin decided to assemble the F-22 at its Marietta
plant under pressure from then Senate Armed Services Committee Chairman Sen. Sam
Nunn. See Grossman (2002).
Acquisition Strategies and Industrial Base Issues
21
distribute the work equally among the three contractor team members, which may have resulted in an artificial distribution of work
during the EMD phase and potentially contributed to the schedule
and cost problems experienced in the program.
The F/A-18E/F Program Drew on Preexisting Expertise
and Contractor Relationships
We turn now to an assessment of F/A-18E/F acquisition strategies
and industrial base issues from the program’s inception through the
EMD phase.
The Navy Adopted a Cautious Approach to F/A-18E/F Acquisition
As discussed in Chapter One, the U.S. Navy began pursuing plans to
develop a new supersonic fighter/attack aircraft in the early 1980s.
The Navy considered several options to meet this requirement. One
option was to modify the existing F/A-18C/D design. The Navy
examined at least seven major new configurations for the “Hornet
2000” conceptual design effort, including significantly more
advanced designs incorporating totally new wing designs and avionics
changes.9 Another alternative was to modify and upgrade the Grumman F-14. A third option was to procure a variant of the ATF, which
was then in the Dem/Val phase. Indeed, in September 1988, the
Navy contracted with the ATF airframe and engine contractors to
conduct design studies for a naval version of the ATF called the Navy
ATF (NATF). The Navy established an NATF SPO collocated with
the Air Force ATF SPO at Wright-Patterson AFB, Ohio.10
____________
9
General Dynamics also continued to conduct modification and design upgrade studies of
improved versions of the F-16 for the Air Force and for potential foreign customers.
10 In March 1986, the Navy formally agreed to evaluate the ATF or a variant as a possible
F-14 replacement and the Air Force agreed to evaluate the ATA as an F-111 replacement.
This agreement and the later formal but brief Navy participation in the ATF (NATF) Dem/
Val program were driven in large part by congressional pressure. See Aronstein, Hirschberg,
and Piccirillo (1998).
22
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
Ultimately, the Navy chose to modify the F/A-18C/D. By and
large, cost considerations drove this decision. In January 1988, the
Navy had selected the McDonnell Douglas/General Dynamics team
to develop the stealthy ATA, now known as the A-12. However, by
mid-1990, the A-12 program was already at least $1 billion over cost
and 18 months behind schedule. In January of 1991, then–Secretary
of Defense Dick Cheney canceled the program. The A-12 program
proved to be a major embarrassment to the Navy. With the F/A-22
likely to evolve into a high-end, high-priced air superiority fighter
optimized for Air Force requirements and smarting from the A-12
fiasco, the Navy preferred to pursue the lower-cost, lower-risk multimission design approach based on the Hornet 2000 studies.
The cancellation of the A-12 program in 1991 had a profound
effect on the management of the F/A-18E/F program. Not wishing to
meet the same fate, the E/F team of Navy managers and contractors
was adamant that they would learn from the failure of the A-12 program. As a result, the E/F program office established a system for
closely monitoring the contractor’s cost and schedule performance.
The program office sought to work closely with the contractors and
did so by setting up a routine of daily phone calls between the Navy
program manager and his counterpart from the prime contractor,
McDonnell Douglas, and weekly video or teleconferences that also
included representatives from Northrop and General Electric, the
contractors responsible for the modified center and aft portions of the
fuselage and the engines, respectively. The program manager also
established a dedicated data line so that analysts in the program office
would have the same access to cost and performance data as the contractors. Thus, although the A-12 cancellation did not affect the F/A18E/F program directly, it motivated the program manager to take
steps to tightly control cost and schedule performance.
The technology requirements for the aircraft were deliberately
crafted to control technological risk and to constrain costs. The Navy
directed McDonnell Douglas to undertake further risk-reduction
studies throughout 1991, resulting in the rejection of some of the
more radical contractor design modification proposals from the Hornet 2000 effort. In October of that year, the Navy formally requested
Acquisition Strategies and Industrial Base Issues
23
designation of the Hornet 2000 program, now known as the F/A18E/F, as a major modification effort rather than as a new program
start. Although the new F/A-18E/F design entailed major airframe
modifications, the Navy intended to incorporate existing F/A-18C/D
avionics and a derivative of the existing engine. While all new, the
airframe design for the F/A-18E/F was aerodynamically similar to the
F/A-18C/D.11 Perhaps most important, the new aircraft design was
burdened with far less demanding performance requirements than the
F/A-22 was. For example, the F/A-18E/F eschewed such technologically challenging and potentially costly F/A-22 requirements as supercruise, full stealth capability, thrust vectoring, fully integrated new
avionics, and AESA radar.12 In addition to these acquisition strategies, the F/A-18E/F program employed a key acquisition reform concept later formalized as Cost as an Independent Variable (CAIV).13
By early 1992, the senior Navy leadership had made it clear that the
E/F program would not proceed unless the cost estimates for the
development program and for the average unit flyaway costs
remained under strict ceilings dictated by likely funding realities.14
In May 1992, OSD formally designated the F/A-18E/F program
as a major modification program (Milestone IV/II approval),
____________
11
Unlike the Hornet 2000 design proposals, which retained the basic F/A-18C/D fuselage
and merely inserted plugs for greater length, the follow-on designs that evolved into the F/A18E/F design were completely new and different from the F/A-18C/D design.
12 Whether the F/A-18E/F design could most accurately be characterized as a major modification of the F/A-18C/D or as a totally new design remained controversial with Congress.
Because of its designation as a “major modification” rather than a new start, the F/A-18E/F
avoided a significant amount of programmatic documentation, oversight, and review
requirements normally associated with the early phases of major DoD acquisition programs.
Perhaps most accurately, the F/A-18E/F program could be characterized as a spiral development program or one following the acquisition philosophy of Preplanned Product Improvement (P3I). Thus, while no AESA radar or Forward-Looking Infrared (FLIR) systems were
required for the initial variant, they were included in the engineering specification as planned
upgrades for future variants.
13 The basic concept of CAIV is that it raises cost goals to the same priority level as schedule,
performance, and other key program and system goals during the design, development, and
production phases of a weapon system.
14 These were $4.9 billion for R&D and $39 million for unit flyaway cost (both in FY 1991
dollars).
24
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
enabling direct entry into EMD.15 On July 20 of that year, the Navy
awarded a sole-source contract for full-scale development to McDonnell Douglas. After award of the basic EMD contract, the Navy and
industry completed a design review of the General Electric F414
engine intended for the fighter. The F414 was an evolutionary engine
based on the F404 and used the F412 core, which was partially
developed for the A-12 program. Thus the engine selection was also
intended to reduce technological risks and costs.
The F/A-18E/F Program Developed a Worksharing Agreement Based
on Contractor Specialties
Unlike the F/A-22, the F/A-18E/F used an existing workshare breakout based on the history of contracts on the F/A-18A/B/C/D development and production. McDonnell Douglas (now a part of Boeing)
was considered the prime contractor, and Northrop (now Northrop
Grumman) was a major subcontractor on the effort (as opposed to a
partnership of equals in the F/A-22). The F/A-18E/F team of
McDonnell Douglas and Northrop had substantial experience on the
F/A-18C/D. Both contractors had experienced design teams in place
and drew heavily from existing suppliers and industrial base. As
shown in Figure 2.2, this workshare arrangement allowed the contractors to concentrate on their specialties from the predecessor program and to use their existing subcontractor industrial base. Similar
to the F/A-18C/D, Northrop Grumman remained responsible for the
aft fuselage section of the aircraft and the ultimate responsibility to
integrate the entire weapon system rested on McDonnell Douglas.
Major subsystem subcontractors, such as General Electric for the
engine and Hughes for the radar, remained the same. Other avionics
suite components remained common with the F/A-18C/D. Thus the
program leveraged the existing vendor base to a large extent.
____________
15 Although the program proceeded through a Milestone IV/II review, very little of the airframe was common between the F/A-18C/D and F/A-18E/F. The OSD (CAIG) memo
from March 1992 states, “The F/A-18E/F will be a new aircraft. We estimated EMD costs
based on a new airframe and engine, as did the Navy.”
Acquisition Strategies and Industrial Base Issues
25
Figure 2.2
F/A-18E/F EMD Work Breakout Was Less Complex Than the F/A-22 and Had
Historical Roots
Boeing–St. Louis
(formerly McDonnell Douglas)
Airframe
• Forward fuselage
• Wings
• Horizontal tails
Avionics
• Radar (Hughes)
• OFP software
Northrop Grumman–
El Segundo
Airframe
• Center/aft fuselage
• Vertical tails
Subsystems
• ECS
• APU
• Power transmission
• Hydraulics
• Fuel system
RAND MG276-2.2
The Use of IPTs Further Minimized Technical and Programmatic
Challenges
While the F/A-18E/F program was not free from serious technological and programmatic challenges, it progressed throughout the 1990s
more or less in accordance with its original schedule estimates. One
reason was that the program office adopted a variety of new acquisition reform strategies that promoted program stability and close
cooperation between the Navy program office and the contractor.
One such strategy was the use of IPTs. The F/A-18E/F program
was in effect an informal Navy and DoD pilot program for development of the IPT concept. In 1992, the Commander of Naval Air Systems Command (NAVAIR) commissioned a special study group to
develop a new strategy for major system acquisitions. The team rec-
26
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
ommended moving away from functional “stovepipes” toward a
product orientation approach, which fully integrated functional areas,
including both government and industry sides. The Navy developed
an implementation plan and selected the F/A-18E/F program as a
pilot program for “proof of concept” of this approach. Thus the F/A18E/F program management, like the F/A-22, was organized in
accordance with IPT principles three years before the Department of
Defense officially mandated IPTs (OSD, 1995). Of course by the
time the F/A-18E/F development was undertaken, the lessons from
the IPT implementation from the early implementers were learned
and the implementation approach was certainly more mature than
when it was initially implemented in the F/A-22 program. Many
observers believe effective use of the IPT approach was one of the
most important management initiatives promoting stability and effective management of technological challenges in the F/A-18E/F
effort.16
Numerous technical challenges were identified during developmental flight testing, but none led to major restructuring of the program or significant cost growth. The first F/A-18E/F test aircraft
(Aircraft Number 1) flew in November 1995, 32 days ahead of
schedule. The second test aircraft first flew one month later. The
formal developmental flight test program began early the following
year at NAS Patuxent River, Maryland. One problem that attracted
considerable public attention was the “wing drop” problem, discovered in 1996. During certain maneuvers, one wing of the aircraft
would unexpectedly stall or dip, causing the aircraft to roll. The Navy
and contractors worked together closely to develop fixes. This type of
a problem is not uncommon during the development of a new airframe. Other technical and performance areas that caused some
controversy during development included combat range and survivability. Most of these problems were either successfully resolved or
dealt with by other adjustments.17
____________
16 See
Bailey (1998).
17 For
example, see GAO (1999). Also see F/A-18E/F SAR, December 31, 1997.
Acquisition Strategies and Industrial Base Issues
27
Thus, with more limited technical objectives, tighter management controls, and continuing success in maintaining cost and
schedule performance, plus the fact that there were few other shortterm options for USN carrier aviation, the F/A-18E/F progressed
through EMD with greater ease than the F/A-22.
Conclusions
The following major lessons can be gleaned from the acquisition
approaches of these programs:
• The “equal” teaming and work share may have led to an artificial work distribution in the F/A-22 program and may have
contributed to cost and schedule problems. In contrast, the F/A18E/F contractor team was structured according to experience
on the F/A-18 A/B/C/D programs. Lines of responsibility were
clearly defined between contractors and subcontractors.
• The F/A-22 program team included subcontractors with limited
prior working relationships. In contrast, the F/A-18E/F program
drew upon preexisting expertise and relationships.
• The F/A-22 program was one of the first implementers of IPT
management structure. In contrast, the F/A-18E/F implemented
the IPT per DoD mandate and when some lessons had already
been learned.
CHAPTER THREE
Potential Contributors to Cost and Schedule
Growth
This chapter discusses the technical factors that may have contributed
to the F/A-22’s cost and schedule growth and to the relative stability
in the F/A-18E/F program. We evaluate the cost data reports and
technical documents from the F/A-22 and F/A-18E/F program provided by the Air Force and the Navy cost analysis agencies. The cost
information is from SARs1 ending with the December 31, 2001, version for each program. We also used Contract Cost Data Reports
(CCDRs)2 as well as the Cost Performance Reports (CPRs).3 We
____________
1
SARs are status reports provided to Congress required by Title 10, USC 2432 for Major
Defense Acquisition Programs (MDAPs). They provide information that covers program
background, schedule, performance characteristics, funding summaries, and top-level contract information on the program. The schedule shows major design milestones within the
program as it progresses from development into production and fielding. The funding summaries show the yearly amount of funding for the various appropriations that are used for the
program. Typically, SARs are produced at the beginning of development (Milestone B) and
annually at the end of the calendar year but may be required more frequently for significant
changes on the program.
2 CCDR
reporting is required for all major acquisition category (ACAT) level 1 programs by
DoDD 5000.4M and generally uses a product-oriented work breakdown structure (WBS) to
categorize costs. This WBS is somewhat common across different programs thus allowing for
collection of costs for systems in the same commodity. The CCDR also separates the nonrecurring and recurring efforts typically associated with a contract. The reporting requirement
is flowed down to supporting contractors who perform a significant portion of the work.
3
The CPR is a management tool that uses cost information to measure progress on a specific contract. The Earned Value Management System integrates technical, cost, and schedule information on a contract to allow the contractor and the government to obtain insight
into the program on a timely basis (reports are generally provided monthly). An overall con-
29
30
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
were able to collect the top-level costs from the monthly CPRs on the
main air vehicle contracts for the length of the EMD effort on both
programs. In addition, while we could collect all the available F/A18E/F CCDR data for EMD, we could only obtain F/A-22 EMD
CCDRs from September 1995 to September 2002. Thus our analysis
does not include the detailed cost for development from earlier years.
Comparison of the EMD Program Costs
EMD program costs are driven in part by the complexity of the major
subsystems within a fighter aircraft. A comparison of EMD program
costs for the F/A-22 and the F/A-18E/F shows that the more farreaching innovations in the F/A-22’s airframe, avionics, and propulsion contributed to the cost overruns in that program. By contrast,
we find that the more incremental advances pursued by the F/A18E/F program helped keep EMD costs stable.
As discussed in Chapter One, the F/A-22 EMD effort includes
advancements in all the major areas of the aircraft: airframe, avionics,
and propulsion. A key objective of the airframe development efforts
was to design a radar cross section that uses large amounts of
advanced materials, such as composites and titanium. The integrated
avionics suite of the aircraft brings together information collected
from several sensors on the aircraft to be displayed to the pilot. The
propulsion system features two high-thrust, Pratt & Whitney F119
jet engines to allow the F/A-22 to supercruise at speeds above the
speed of sound without using the fuel-consuming after burner. The
airframe design, flight controls, and thrust vectoring are also used to
improve the maneuverability of the aircraft.
As can be seen in Figure 3.1, the F/A-22 EMD program was
more than three times more costly that the F/A-18E/F EMD. Almost
______________________________________________________
tract is broken down into smaller, specific work tasks with specific budgets and linkages to
completing the overall contract. This assignment of budget to scheduled tasks on a contract
is called a performance measurement baseline and is used as the benchmark for measuring
schedule and cost performance.
Potential Contributors to Cost and Schedule Growth
31
Figure 3.1
F/A-22 and F/A-18E/F EMD Program Cost Drivers
In-house
Integrated Logistics Support
Systems engineering and
program management
and data (non-ILS)
ST&E
Avionics
Propulsion
Airframe
Cost
$17,121 M
$4,883 M
F-22
F/A-18E/F
Aircraft
SOURCES: F/A-18E/F MSII Program Office estimate. F/A-22 March 2003 Program Office
estimate and September 2002 CCDR used for WBS breakout.
RAND MG276-3.1
one-third of the F/A-22 program budget has been spent on the
avionics—more than any other subsystem, including the airframe.
The F/A-18E/F’s major cost element was the airframe, with far
fewer dollars spent on the avionics.4
In our attempt to explore the main contributors of the cost
growth, we examined the CCDR from 1995 through 2002 for the
F/A-22 and from 1992 through 1998 for the F/A-18E/F, summarized in Figure 3.2 and 3.3. Most contractors engaged in a DoDsponsored major development activity are required to submit
CCDRs. The CCDR is a cost report prepared by the contractor that
____________
4
We also note that about 23 more weapons were certified for the F/A-18E/F program than
for the F/A-22. In terms of percentages, the E/F spent more money on system test and
evaluation (ST&E) than did the F/A-22, albeit the F/A-22 test is not yet complete. The F/A22 program certified only three weapon systems: the AIM-120C Advanced Medium-Range
Air-to-Air Missile (AMRAAM), the AIM-9M Sidewinder missile, and the 1,000-pound
GBU-32 Joint Direct-Attack Munition (JDAM), as defined in the April 2000 CARD.
32
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
shows the actual cost for the contract at various points during the
program. They provide labor hours expended and other incurred cost
information by WBS and functional labor categories, for example
engineering, manufacturing, tooling, and quality assurance.
In Figures 3.2 and 3.3, the percentage change of the estimates to
complete the development work from the initial report available to us
are shown at semiannual points in the two EMD programs. However,
since the period of performance is roughly the same for both aircraft
and cost growth is the focus of the chart, not absolute cost, we elected
to display the data as submitted by the contractors. In both F/A-22
and F/A-18E/F cases, the data capture the majority of the development EMD phase, but the early F/A-22 data (from 1991 until 1995)
was not available in the F/A-22 case. Therefore, the F/A-22 information does not provide a complete picture of the cost growth of its
major subsystems from those early years.
The F/A-22 airframe cost growth of 42 percent is much higher
than any other subsystems, followed by the avionics at 25 percent as
Figure 3.2
F/A-22 Cost Growth Trends for Major Systems
50
Percentage cost growth
40
30
Airframe
Systems Engineering and
Program Management
and data
Avionics
20
10
Propulsion
0
ST&E
–10
–20
Support
–30
–40
–50
Sep
95
Mar
96
Sep
96
Apr
97
Sep
97
Mar
98
Sep
98
Mar
99
Date of report
RAND MG276-3.2
Mar
00
Sep
00
Mar
01
Sep
01
Sep
02
Potential Contributors to Cost and Schedule Growth
33
Figure 3.3
F/A-18E/F Cost Growth Trends for Major Systems
50
Percentage cost growth
40
30
20
10
Propulsion
Airframe
Support
0
Avionics
–10
Systems Engineering and
Program Management
and data
–20
–30
ST&E
–40
–50
Aug
92
Feb
93
Aug
93
Feb
94
Aug
94
Feb
95
Aug
95
Feb
96
Aug
96
Feb
97
Aug
97
Feb
98
Aug
98
RAND MG276-3.3
shown in Figure 3.2. However, as of the date of this report, the airframe design is almost complete, whereas significant amount of work
remains to be done on the avionics, so that cost growth will likely
increase. Other cost elements, such as propulsion development, system test, and support, indicated minimal or no growth during the
examined timeframe. As with avionics development, the test program
is far from completion. Recent information indicates that development test (DT) is scheduled for completion in December 2005 or
June 2006 and Milestone III (approval for full-rate production) is
scheduled for March 2005. Ironically, the support costs have
decreased as a percentage of total costs.
In contrast, as we can observe from Figure 3.3, the E/F airframe
cost grew by 12 percent. However, this growth was offset by the
declines in other cost categories such as System Engineering and Program Management, System Test and Evaluation (ST&E) and support. In addition, adequate management reserve may have also played
a key role in keeping costs under control. A more detailed discussion
of management reserve occurs in the next chapter. The net result was
34
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
no growth. In the next section, we attempt to compare the major aircraft elements of the F/A-22 and F/A-18E/F and examine the reason
for the cost growth of the F/A-22 program.
An Assessment of Cost Growth of Major Subsystems
We turn now to a more detailed discussion of the specific subsystems
that contributed to cost growth in the F/A-22 and stability in the
F/A-18E/F. The scope of technological advance and the needed innovation to deliver the required performance differs greatly between
these two aircraft programs and so has the magnitude of technological
challenges facing them. We first discuss the F/A-22 cost growth in
the airframe, then the avionics, and finally the propulsion systems.
For comparative purposes, we also discuss the F/A-18E/F results in
the same areas.
Airframe Cost Growth
A major reason for the cost differences shown in Figure 3.1 is that
each program had different requirements for airframe design. The
F/A-22 airframe needed a large amount of composite materials to satisfy its stealth requirements and meet its weight constraints. Engineering these materials added time and manpower to the
development phase. The F/A-18E/F had minimal stealth requirements and was able to use more traditional airframe materials, thus
reducing the time needed for development. Moreover, the F/A-22
program encountered airframe design problems that required continual adjustment, leading to an overall rise in the expected airframe
weight. The F/A-18E/F program maintained a relatively stable airframe weight, suggesting that design problems were solved with
seemingly minimal effort. We explore these issues in more detail in
the sections below.
The F/A-22 Stealth Requirement Was a Challenge. Stealth is a
major feature of the F/A-22 airframe design. New radar-absorbing
materials and structures along with internal weapon carriage capability allow for the exceptional low-observable (LO) characteristics of
Potential Contributors to Cost and Schedule Growth
35
the airframe. However, LO has been a major engineering challenge
for the airframe designers during the development. All the LO design
aspects, such as internal weapon carriage, further complicate an
already challenging design problem. In contrast, the F/A-18E/F
stealth requirement was minimal and internal carriage of the weapons
is not required.
Figure 3.4 shows the amount of engineering hours required or
forecast to design the F/A-22 Air Vehicle. 5 These hours are based on
data from CCDRs from the YF-22 Dem/Val and the F/A-22 EMD
efforts. When we compare the F/A-22 engineering hours to the F/A18E/F air vehicle engineering hours, we can see that the F/A-18E/F
hours are substantially less—that is, the F/A-18E/F EMD engineering
hours are less than half of those of the F/A-22 EMD. Even if we
include the entire engineering hours spent on the YF-17 and F/A18A/B development contracts and compare that amount to the total
Engineering hours
Figure 3.4
Comparison of the Air Vehicle Design Hours
F/A-22 EMD
F/A-18E/F
EMD
YF-22 D&V
F/A-22
F/A-18A/B
EMD
YF-17
F/A-18E/F
RAND MG276-3.4
____________
5 Air
Vehicle design hours exclude design hours associated with the avionics and propulsion
systems.
36
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
engineering hours expended on the YF-22 and F/A-22 development
contract, the F/A-18 still shows significantly fewer hours. Also, it
should be noted that the F/A-18A/B development effort was for
essentially a new airframe design with no real commonality to the
F/A-18E/F airframe design, as was previously mentioned in this
report.6
As mentioned earlier, stealth requirements as well as differences
in the material composition of the two airframes may account for
some of the differences in the engineering hours.
This difference is illustrated in Figure 3.5. The F/A-22 airframe
design includes 24 percent carbon epoxy composites and about 40
percent titanium. Composites, in addition to their low relative weight
compared to metals, allow for LO features. Titanium structures provide strength and temperature control needed for LO air superiority
Figure 3.5
Airframe Material Distribution
40
F/A-22
F/A-18E/F
35
Percentage
30
25
20
15
10
5
0
Aluminum
Steel
Titanium
Thermoset
Thermoplastic
Other
SOURCE: Younossi, Kennedy, and Graser (2001).
RAND MG276-3.5
____________
6
We excluded the F/A-18C/D effort from this analysis since most of the C/D effort was
focused on avionics and was accomplished through a series of engineering change proposals.
Potential Contributors to Cost and Schedule Growth
37
fighters. Both these materials are more complex than aluminum, the
traditional material used in airframe designs. Manufacturing of composites is more time-consuming than aluminum and steel. Titanium
is an expensive metal and it is hard to machine.7 In addition, design
information of composite structures is not as mature as that of metal
structures, so more engineering hours are normally needed for like
structures. In contrast, the F/A-18E/F uses significantly more aluminum and steel in the airframe structure than the F/A-22—
approximately 31 percent and 14 percent, respectively. Both of these
materials have traditionally been the main structural material in
airframe designs.
Weight Instability Was an Early Indicator of Problems for the
F/A-22. Another airframe design feature linked with the complexity
and stability of the airframe configuration is weight. Weight fluctuations as a result of redesign to meet requirement or to accommodate
additional performance directly affects cost and schedule. Keeping the
aircraft weight under control has historically been a challenge to
aircraft designers. The F/A-22 airframe weight has been unstable and
has grown during most of the EMD period. At the beginning of the
F/A-22 EMD program, both the Design-to-Weight (DTW) and the
parametric weight estimate were significantly lower than the current
weight of the aircraft, or Achieved-to-Date (ATD) weight. 8 The F/A22 DTW increased and grew more realistic and converged with the
ATD as the program progressed. Figure 3.6 depicts the ATD weight
from the beginning of the program until September 2002.
____________
7
For more information on these materials and other advanced materials, see Younossi et al.
(2001).
8
The DTW is a weight goal for the program to achieve. The parametric weight estimate is
generated using historical weight data from historical aircraft programs. It is allocated to an
IPT that develops a “build-to” data package (engineering, material, planning, and tooling
data). This target weight includes a decrement to the proposal weight, which allows for the
growth that normally occurs during design development. The ATD weight reflects the
weight of the aircraft by using the actual drawings of the aircraft design. The ATD weight is
the weight that reflects the current drawings. It usually represents calculated or actual weight
data but can include estimated weight. As a result, ATD weight reporting lags a design
decision by a significant time, sometimes several months.
38
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
The figure shows the contractor responsible weight9 changes
over time. After the initial drop of 21 percent, the weight has grown
by 11 percent since the program’s preliminary design review (PDR)
in April 1993. The data also indicate that considerable efforts may
have been made to bring down the F/A-22 airframe weight estimate
before the PDR because the weight dropped about 5 percent in the
six months preceding PDR. The figure also shows that, after this initial drop, the weight steadily increased ever since. It is interesting to
note that the weight dropped just before and increased right after
both the PDR and critical design review. Significant weight decrease
just before a major design review reflects unstable airframe design.
In contrast to the F/A-22, the F/A-18E/F airframe weight has
remained relatively stable, with a minor 2 percent growth during
EMD. Figure 3.7 shows the F/A-18E/F DTW over time. Even
though the F/A-18E/F program experienced notable weight stability,
the contractor weight estimates did fall before both design reviews
Figure 3.6
F/A-22 Airframe Weight Changes over Time
Change of ATD weight compared to
original Phase I DTW (percentage)
40
35
30
25
20
15
10
5
0
Feb
92
Feb
93
Feb
94
Feb
95
Feb
96
Feb
97
Feb
98
Feb
99
Feb
00
RAND MG276-3.6
____________
9 Contractor
responsible weight is the aircraft empty weight minus the engine weight.
Feb
01
Potential Contributors to Cost and Schedule Growth
39
and increased after the reviews, but in the 1 to 2 percent range.
Although the overall weight of the F/A-18E/F grew substantially less
than the F/A-22, the weight data still show the same phenomena of
weight drops prior to the design reviews (June 1993 and June 1994),
albeit at a much smaller magnitude.
A major contributor to the airframe weight instability may have
been some key airframe components of the F/A-22 airframe, such as
the wings and vertical and horizontal stabilators. The program has
also encountered design problems that required some airframe redesign. In contrast, the F/A-18E/F airframe problems were solved with
seemingly minimal effort.
Avionics Cost Growth
Another contributor to the cost growth has been the F/A-22’s avionics suite, which is far more challenging than any other fighter electronics system to date. The integrated avionics suite fuses information
collected from several sensors on the aircraft to be displayed to the
Figure 3.7
F/A-18E/F Weight Estimates over Time
Change of ATD weight compared to
original DTW (percentage)
40
35
30
25
20
15
10
5
0
–5
Jul
92
RAND MG276-3.7
Jul
93
Jul
94
Jul
95
Jul
96
Jul
97
Jul
98
Jul
99
Jul
00
Jul
01
Jul
02
40
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
pilot. In contrast, the F/A-18E/F avionics for the initial release of the
F/A-18E/F incorporates the suite from the C/D model. Provisions are
made for a series of avionics upgrades to be performed subsequent to
the basic air vehicle development during the EMD program. These
separate approaches account for a large portion of the cost differences
depicted in Figure 3.1. We examine each issue in detail below.
The F/A-22 Program Embraced a Challenging Avionics Design.
The F/A-22’s integrated avionics system uses new technology electronically scanned radar, which uses transmit and receive modules,
and Beyond Visual Range (BVR) weapons control. In addition the
avionics suite includes a state-of-the-art electronic warfare (EW) system and communication, navigation, and identification (CNI) suites.
The avionics suites on most previous fighter aircraft used federated components—that is, separate avionics boxes provide information to the pilot independent of information from various other
systems. In the F/A-22, a central core processor fuses information
from many sensors and electronics and presents an integrated picture
to the pilot. Thus, the processing capability requirement is huge. The
new approach integrates all the information from numerous subsystems, and therefore these components need a significant processing
capability—somewhere on the order of 250 million instructions per
second for data processing, and more than 100 times that—10 billion
instructions per second—for signal processing. Also, each aircraft
must be capable of extensive sensor fusion from several active and
passive organic sensors as well as harmonizing data from two or more
other F/A-22s during a mission. These extensive demands on the
computing systems in the aircraft caused system lock-up during test
flights, (GAO, 2003b). According to the F/A-22 Program Office, the
issue of avionics software stability was solved prior to the start of
IOT&E in April 2004. Finally, the radar being developed for the
F/A-22 is based on new technology with modules that both transmit
and receive. These modules have neither been used previously in the
space-constrained environs of a fighter aircraft nor have they ever
been produced in the large quantities that will be needed to support
the F/A-22 program.
Potential Contributors to Cost and Schedule Growth
41
To discover where the cost growth occurred in the avionics, we
examined CCDR information from the September 1995 and September 2002 reports. The largest growth occurred in the core processor followed by the EW and CNI modules. Surprisingly, the radar,
the traditional cost and risk driver, experienced the least amount of
growth by percentage.
Figure 3.8 shows the F/A-22 EMD avionics costs broken out by
the major parts of the avionics suite.10 The entire avionics development was funded as part of the EMD program. As can be seen in the
figure, the CNI, EW, radar, and core processor account for about 80
percent of the total avionics development cost with the core processor
being the most expensive and the one with the most cost growth.
These data were current as of September 2002.
Problems with developing, testing, and correcting deficiencies in
the ambitious avionics in the F/A-22 have caused program stretchFigure 3.8
F/A-22 Avionics Cost Growth by Component
Radar
4
13
Software management system
18
Vehicle management system
21
Electronic warfare
CNI
28
Avionics system design
31
Information security module
34
Controls and displays
35
47
Core processor
0
10
20
30
Percentage
40
50
RAND MG276-3.8
____________
10 These costs are from the F/A-22 EMD Team CCDR. The numbers indicated estimated
cost at completion and include both hardware and software costs.
42
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
outs and delays. The EMD portion of the F/A-22 program is now
more than 10 years old. The program has been in development for 20
years since the contract for concept development was awarded in September 1983. This extended development period has led to several
analyses that find some of the components of the F/A-22 obsolete or
will be obsolete by the time the aircraft actually enters the force. In
1998, then–Under Secretary of Defense for Acquisition, Technology,
and Logistics Jacques Gansler said that “the F-22 . . . is not yet into
production but, with electronic products becoming obsolete in as little as 18 months, [it] already contains outdated parts” in congressional testimony (Gansler, 1998). A later quote from a DoD analyst
stated that “the avionics for the F-22 was obsolete before the plane
even went into production” because the chips in the processors were
outdated in 1992 (Cockburn and St. Clair, 2001). Finally, in September 2002 Brig. Gen. William Jabour, arguing for introduction of
an upgraded radar starting with Lot 5 aircraft, stated that “the current
radar . . . is the best radar flying right now. It is, however, ten-yearold technology”(Colarusso, 2002). The cutting edge of technology, it
seems, moves faster than the DoD acquisition process.
Also, the current modernization program plans to implement a
series of spiral developments to improve the F/A-22’s air-to-ground
capability significantly. This modernization effort occurs concurrently
with the EMD activities. Because the avionics design is still not completed, it may further exacerbate cost and schedule problems.
Software Growth May Have Contributed to the F/A-22 Avionics
Program Cost Growth. Another contributor to the avionics cost is
software. The overall size of the air vehicle software and the increase
in the number of source lines of code (SLOC) may have been contributors to the cost growth. The F/A-22 SLOC grew by 565,000
lines of code, approximately 34 percent, between October 1993 and
April 2000. Although the F/A-22 data are relatively old, they do show
a significant growth in the number of lines of code between 1993 and
2000.11 These SLOC counts are based on each program’s CARD.
____________
11 The F/A-18E/F SLOC count comes from the Milestone II (dated 1992) and Milestone
III (dated October 1999) CARDs. The F/A-22 SLOC count comes from the F-22 Weapons
Potential Contributors to Cost and Schedule Growth
43
Similarly, the F/A-18E/F SLOC grew by 405,000, or about 40
percent. Although this growth seems sizable, most of it was attributable to upgrades outside the F/A-18E/F EMD program. The E/F’s
initial baseline capability was equivalent to the C/D’s. However,
during the E/F’s development, upgrades were made to the common
C/D and E/F Operational Flight Program (OFP) and therefore the
OFP at the conclusion of EMD included additional upgrades. Hence,
the reported costs may not reflect all of this additional effort.
The F/A-18E/F Avionics Development Reflect an Evolutionary
Process. As previously mentioned, the F/A-18E/F program was a
major modification of an existing aircraft with only modest upgrades
to the F/A-18C/D avionics as part of its EMD program. During the
EMD, the program did not encounter many significant technology
problems. More than 90 percent of the E/F’s electronics in the initial
production aircraft are common to the F/A-18C/D. Some upgrades
to the subsequent production aircraft were included as preplanned
product improvements from the beginning of the program. These
include improved cockpit instrumentation, a new and improved
forward-looking infrared (FLIR), and an electronically scanned array
for the radar. Other improvements have been added to the program
during the past several years, including a reconnaissance pod, a
helmet-mounted cuing system, and some integrated electronic defensive countermeasures. Figure 3.9 shows the F/A-18E/F avionics
development effort. None of these significant improvements in avionics over the C and D models was part of the EMD phase of the
F/A-18E/F program, and so did not raise any concerns or cause any
delays during development. The modernization to the avionics was,
and continues to be, funded separately in relatively small packages.
Because the avionics system is a federated system, these upgrades are
easily incorporated when available, without any significant effect on
the overall weapon system availability. Each avionics upgrade, being a
smaller and separate program, has the advantage of having the exist______________________________________________________
System Software Development Plan: Engineering and Manufacturing Development, dated
October 1993, and from the F/A-22 CARD dated April 2000, Appendix I.
44
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
Figure 3.9
The F/A-18E/F Avionics Development Program12
1,800
ACS, SHARP, PDS/DCS,
ALR-67(V)3, IMPLC, ALE-50
Contract cost (FY 90 $M)
1,600
DVMC
ANAV
Cumulative contract cost
AESA RUG III
HOL
1,400
AMC&D
ATFLIR
1,200
APG-73 RUG II
IDECM
TAMMAC
CMWS
JHMCS
EMD
ASTE
(Avionics portion)
$194 M
1,000
800
600
APG-73 RUG I
400
200
—
1990
1992
1994
1996
1998
2000
2002
Contract award year
NOTE: Figures reflect F/A-18 cumulative total since 1990 $1.6 B (FY 90 $)
RAND MG276-3.9
ing system in place in case problems are encountered with the
upgrade system. This approach provides a backup solution such that
the entire aircraft program can move ahead. It also has the added
benefit of removing the external scrutiny from the main program.
That is, the program manager can keep attention on the main program and incorporate the new systems as they become available without affecting the program’s critical path.
____________
12 The APG-73 RUG and E/F EMD data are from CCDRs and CPRs. Tactical Aircraft
Moving-Map Capability (TAMMAC), Joint Helmet-Mounted Cuing System (JHMCS),
Advanced Mission Computers and Displays (AMC&D), Advanced Targeting ForwardLooking Infrared (ATFLIR), Active Electronically Scanned Array (AESA), Advanced Crew
Station (ACS), Shared Advanced Reconnaissance Pod (SHARP), and Digital Video Map
Computer (DVMC) data are from CPRs and CSSRs. R-3 exhibit from February 2000 was
used for the following programs: Integrated Defensive Electronic Countermeasures
(IDECM), Common Missile Warning System (CMWS), Advanced Strategic and Tactical
Expendables (ASTE), ALR-67(V)3, Integrated Multiplatform Launch Controller (IMPLC),
and ALE-50. And the Navy Budget R-2 exhibit from February 2003 was used for HigherOrder Language (HOL), Positive Identification System/Digital Communication System
(PIDS/DCS), and Accurate Navigation System (ANAV).
Potential Contributors to Cost and Schedule Growth
45
Propulsion System Cost Growth
The third major cost area is the propulsion system development cost.
The F/A-22 propulsion system features two high thrust, Pratt &
Whitney F119 jet engines to allow the F/A-22 to supercruise at
speeds above the speed of sound without using the fuel-consuming
afterburner. This development effort was relatively costly because it
required a new core engine. In contrast, the F/A-18E/F propulsion
system is provided by two General Electric F414 jet engines that produce about 20 percent more thrust than the engines from the original
design. As a derivative design, this system was easier and less costly to
develop.
Figure 3.10 summarizes the cost of the F119 and F414 development costs. The F/A-22 propulsion system, the F119 engine, provides the aircraft with supercruise capability without the use of
afterburners. The F119 new engine core development effort is significantly more difficult than the derivative development approach of the
F414 engine used in the F/A-18E/F.
The F119 development was about two times more expensive
than the F414 development. During ground testing, the engine expe-
Propulsion development cost
Figure 3.10
Development Cost Comparison of the F119 and F414 Engines
F119
F414
YF119
F/A-22
RAND MG276-3.10
F412
Estimate
F/A-18E/F
46
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
rienced some problems with blade failure stemming from overheating
and variability in material properties (Druyun, 1999). Although these
problems have apparently been corrected, they indicate some initial
challenges in developing an engine capable of meeting the stringent
requirements in the F/A-22 program. Conversely, the F/A-18E/F has
a derivative engine that uses the core developed for the F412 engine,
which was planned to power the A-12 aircraft. The F414 engine also
benefited from previous experience on the F404, which powers the
F/A-18C/D.
The increased development cost for the F/A-22’s new core
engine is consistent with RAND’s recent study on engine costs,
which suggests that an engine with a new core design is significantly
more costly than one that includes a derivative design (Younossi et
al., 2002). The analysis of the historical engine development costs in
that study suggests that a new core development is much more costly
that a derivative approach.
Conclusions
The scope of technological advance and the innovation needed to
deliver the required performance differed greatly between the F/A-22
and F/A-18E/F aircraft programs and so has the magnitude of technological challenges facing each. As shown previously, the F/A-22
cost growth was mainly the result of design challenges in the airframe
arising out of the stealth requirements, the integrated avionics suite,
and, finally, the new propulsion system. In contrast, the F/A-18E/F
airframe requirement was met by incremental improvements with
minimal stealth requirements, using mostly the existing avionics system from its predecessor aircraft, and a derivative engine design.
Concurrent development and integration of all aspects of the F/A-22
may itself have contributed to the cost growth and schedule slippage,
whereas the incremental improvements in the airframe, use of mostly
existing avionic components, and a derivative engine may have facilitated the F/A-18E/F cost and schedule control.
CHAPTER FOUR
Use of Cost Performance Data
Contractor cost performance data are extremely valuable for the program manager to gain insights into the financial and schedule health
of a program. They provide specific metrics that the program manager uses to track the performance of work described and required
under the contract. This chapter provides insight into the use of cost
performance data collected by the F/A-22 and F/A-18 programs.
Different Methods of Measuring Progress Affect the
Management of Program Costs and Schedule
One method used by the government and contractors to manage
both the F/A-22 and F/A-18E/F programs was Earned Value Management (EVM). EVM is a tool that provides insight into technical,
cost, and schedule progress on development contracts. The EVM data
from the contractor is documented in Cost Performance Reports
(CPRs), which are then provided periodically to the government. A
good EVM system ensures that the program manager has access to
the accurate, valid, and timely cost and schedule information on a
regular basis. The data reflect time-phased budgets for specific contract tasks and indicate work progress against planned schedules and
costs. The data reported to DoD should be similar to the data used
by the contractor to manage the work under contract. A WBS is used
to allocate the contract’s Statement of Work effort to lower-level
work packages. The EVM common metrics are Budgeted Cost of
47
48
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
Work Scheduled (BCWS),1 Budgeted Cost of Work Performed
(BCWP),2 and Actual Cost of Work Performed (ACWP). 3 In the
next sections, we will show EVM data in a cumulative fashion as the
F/A-22 and F/A-18E/F contract work was accomplished from contract award date until November 2002.
The F/A-22 Program Used Contract Performance Goals to Measure
Progress
Rebaselining strategy is used when contract costs grow and program
schedules slip for various technical or contractual reasons. At times,
the government clients allow contractors to rebaseline so their cost
variance is readjusted to zero and their contract performance is measured against a new baseline.
Figure 4.1 depicts the monthly CPR data for the F/A-22 Air
Vehicle EMD contract.4 The lower portion of the figure indicates
several cost and schedule overruns during the EMD program. The
program has experienced four specific rephasings of the budget where
at each point the program schedule and scope was modified. For
instance, cost variance, which is the difference between the BCWP
and the ACWP, increased through April 1995 until it was zeroed by a
rebaseline of the program in December of 1995, when the variances
disappeared. Another similar rebaseline took place around February
1997, after the Joint Estimating Team (JET) review.5 The cost variance continued to grow until the present report to $490 million. The
____________
1 The
BCWS is the sum of all the budgets from all the actual and estimated work as well as
planning packages for future work.
2 The
BCWP is the earned value of the sum of the completed work packages and completed
portion of the open work packages.
3 The
ACWP is the actual cost for completing the work in the time period.
4 CPRs
5 The
track the cost and schedule progress of various key indicators on the program.
JET was a panel of high-level government and contractor personnel assigned to review
the program plans during the end of 1996 and into 1997 in response to the Defense Science
Board (DSB) Task Force review of the program in 1995. The JET’s findings concluded that
the EMD program plan required additional funding and time to complete and recommended restructuring the program and moving the Milestone III date.
Use of Cost Performance Data
49
Figure 4.1
The F/A-22 Program Used Contract Performance Goals to Measure
Progress
Cost
First Rephase
November 1993
• Reduced number of
aircraft and engines
• Added 12 months to
schedule
Third Rephase
March 1996
• Deferred “B” model
and deleted PPV
• Added 6 months
to schedule
Red Team Review
November 2002
• Added $876 M
• Added 4 months
to schedule
(Not included in data)
Second Rephase
December 1994
• Added 8 months
to schedule
Oct
92
Oct
93
Oct
94
Oct
95
BCWS
BCWP
ACWP
LRE
Rebaseline
February 1997
• Joint Estimate Team
recommendations
incorporated
Rebaseline
December 1995
Oct
91
Nunn-McCurdy Breach
Fourth Rephase
September 2001
February 1997
• Added 6 months
• Added 9 months
to schedule
to schedule
Oct
96
Oct
97
Oct
98
Oct
99
Oct
00
Oct
01
Oct
02
Reporting period
RAND MG276-4.1
program experienced a Nunn-McCurdy SAR breach in September
2001.6
Also, we can see that Budget at Completion (BAC), the sum of
contract’s work package budgets, grew from $9.2 billion in October
1991 to $13.4 billion. Similarly, the Latest Revised Estimate (LRE),
the contractor’s estimate of the cost to complete the contract, grew
from $8.9 billion to $13.7 billion. In November 2002, an Air
Force–led Red Team reviewed the entire program and recommended
additional funds and time to complete the development phase.
The F/A-22 Contractor Allocated Little for Management Reserve
Figure 4.2 depicts the use of management reserve in the F/A-22 program. Management reserve is a budget withheld for management
control purposes, and it is mostly used to cover future unknown
____________
6
A Nunn-McCurdy (named after Sen. Sam Nunn [D-Ga.] and Rep. Dave McCurdy [DOkla.] unit cost breach occurs when a major defense acquisition program experiences a unit
cost increase of at least 15 percent.
50
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
problems. The figure shows management reserve as a percentage of
LRE. According to the EVM Implementation Guide,7 management
reserve is to be used to enable project managers to adjust for uncertainties on a contract and should not be used as a contingency fund
to absorb the cost of contract changes. The guide also states that the
total allocated budget may exceed the contract budget baseline (CBB)
resulting in an over-target baseline (OTB) that can allow for replanning of future work to provide a more realistic amount for performance measurement than is currently indicated on the program.8 In
other words, once a program is behind on cost or schedule, a new
baseline for reporting should be established to allow assessment of
future work as it progresses. Otherwise, prior negative variances can
hide new problems. As Figure 4.2 shows, only a relatively small
amount of management reserve was allocated at the onset of the F/A22 program and was used up quite rapidly. The management reserve
was totally exhausted from May 1995 through February 1997. However, as a result of the JET recommendations the Air Force increased
the program funding and the contractor allocated another sum of
money to management reserve that was also quickly consumed.
The F/A-18E/F Program Used EVM Data to Measure Progress
In contrast to the F/A-22 program, the F/A-18E/F Air Vehicle EMD
contract CPR data, as shown in Figure 4.3, reflect a smoother
accomplishment of work performed without any noticeable rebaselines of the EVM metrics. Also, the management reserve was targeted
to be approximately 10 percent of the remaining work as measured
by the difference between the BAC and the BCWP.
Figure 4.4 shows the total management reserve as a percentage
of LRE. The figure shows that the contractor had set aside a healthy
____________
7
The Earned Value Management Implementation Guide, Defense Contracts Management
Command (DCMC), available at http://www.acq.osd.mil/pm/currentpolicy/jig/evmigl.htm.
8
When total budget allocated to work exceeds CBB, replanning future or current work and
adjusting variances may be necessary. When implementing OTB, baseline budget changes
must be documented and traceable. Establishing management reserve in OTB is acceptable.
Use of Cost Performance Data
51
Figure 4.2
The F/A-22 Contractor Allocated Small Amounts of Management Reserve
Management reserve percentage
of latest revised estimate
12
10
First rephase
November 1993
Third rephase
March 1996
Second rephase
December 1994
8
Joint Estimate Team (JET)
recommendations
• Incorporated 1997
6
Quadrennial Defense Review (QDR)
• Slowed ramp up to production
June 1997
4
2
0
Oct
91
Oct
92
Oct
93
Oct
94
Oct
95
Oct
96
Oct
97
Oct
98
Oct
99
Oct
00
Oct
01
Year
RAND MG276-4.2
Cost
Figure 4.3
The F/A-18E/F Used EVM Data to Measure Progress
BCWS
BCWP
ACWP
LRE (=CBB)
Oct
92
Oct
93
Oct
94
Oct
95
Oct
96
Oct
97
Reporting period
RAND MG276-4.3
Oct
98
Oct
99
Oct
00
Oct
02
52
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
Figure 4.4
The F/A-18E/F Maintained a Large Management Reserve
Management reserve percentage
of latest revised estimate
12
10
8
6
4
2
0
Jul
92
Jul
93
Jul
94
Jul
95
Jul
96
Jul
97
Jul
98
Jul
99
Jul
00
Year
RAND MG276-4.4
sum of management reserve and, as the program progressed during
the EMD and encountered unforeseen challenges, the management
reserve funds were used. This budget was planned early to allow the
program manager the capability to adjust for uncertainties of the program and was mostly expended as the contract reached completion.
Conclusions
The EVM method was used by the government and the contractor in
the management of both the F/A-22 and F/A-18E/F programs. The
EVM data provided insight into technical, cost, and schedule progress on the development contracts. The F/A-22 data indicate cost
and schedule trouble as early as October of 1992, whereas the F/A18E/F data show a program that was virtually on cost and schedule
throughout the entire development phase. One of the major contributors to the F/A-18E/F program’s cost and schedule stability may
have been the existence of a substantial management reserve. As the
Use of Cost Performance Data
53
program proceeded through its development and unforeseen problems arose, the amount of management reserves covered these problems and was decreased accordingly. In contrast, only a small amount
of management reserve (about 2 percent) was allocated for the F/A-22
program, which was depleted in about the first year of the EMD
effort.
CHAPTER FIVE
Conclusions and Lessons Learned
This chapter summarizes our conclusions about the factors that contributed to either growth or stability in the F/A-22 and F/A-18E/F
programs. We further aggregate these findings into a set of lessons
that the Air Force and other DoD services may apply in their acquisition of future military platforms. Certainly, many of these findings
are not unique to these programs, but have been evident in other
DoD acquisition programs as well.
Conclusions
Table 5.1 summarizes the major factors contributing to the F/A-22
program’s schedule slippage and cost growth in the EMD phase and
the major factors contributing to the stability of the F/A-18E/F program during the EMD phase.
Lessons Learned for the U.S. Air Force
The process of acquiring new weapon platforms requires the U.S.
military to invest substantial amounts of time and money in development, testing, and production. The lessons derived in this study
from an evaluation of the F/A-22 and F/A-18E/F programs provide
the Air Force and other services with ways to improve the acquisition
55
56
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
Table 5.1
Summary of Lessons Learned from Each Program
Lessons from the F/A-22
Program
Lessons from the F/A-18E/F
Program
Technology
Development
The F/A-22 program pursued
revolutionary technologies
and performance improvements over the legacy systems.
The F/A-18E/F program used
the same technologies or pursued only evolutionary technology and performance
improvements.
Cost and Schedule Estimates
The initial cost and schedule
estimates seem to have been
unrealistically low.
Cost and schedule estimates
were relatively accurate and
stable.
Contractor
Teaming
“Equal” teaming and workshare may have led to an artificial work distribution and
may have contributed to cost
and schedule problems. The
move from Burbank to Marietta led to the loss of most of
the core design and management teams from the
Dem/Val phase.
The contractor team was structured according to prior experience on the F/A-18A/B/C/D
programs. Lines of responsibility were clearly defined
with a designated prime contractor ultimately responsible
for contract performance.
Development
Concurrency
The concurrent development of
high-risk technologies in airframe, engine, and avionics
within the same contract was
a high-risk endeavor.
The Navy took an evolutionary
development approach for
the moderately risky avionics
technologies, which was
funded outside of the EMD
program. The existing F/A18C/D avionics was always
available as a backup solution.
Airframe
Weight
The airframe weight data show
significant fluctuations, which
indicate airframe design
instability.
The airframe weight had only
minor increases, reflecting a
stable design.
Management
Reserve
Very little management reserve
budget was allocated to cover
design risks.
Sufficient budget and the contractor allocated sufficient
management reserve to cover
unforeseen problems.
of such airframe platforms as the Joint Strike Fighter and such other
hardware systems as unmanned aerial vehicles and missile programs.
Here is the summary of our major lessons learned for the Air
Force acquisition decisionmakers:
Conclusions and Lessons Learned
57
• Early, realistic cost and schedule estimates set the program on
the right path for the rest of the development program. These
estimates must be adjusted over time.
• A stable development team structure, proper team expertise,
clear lines of responsibility and authority, and a lead contractor
responsible for overall program progress are critical to success.
• An experienced management team and contractors with prior
business relationships help eliminate early management problems.
• Concurrent development of new technology for the airframe,
avionics, and propulsion adds significant risk to the program,
not only from the risk of the individual component development but also from the integration of three significant, technically challenging, concurrent activities.
• Reducing the cost and risk of avionics should be a key focus of
the concept development phase. Avionics is a considerable cost
driver of modern weapon systems, and new concepts should be
demonstrated along with the new airframe designs—that is,
during early development rather than after Milestone II/B.
• Preplanned, evolutionary modernization of high-risk avionics
can reduce risk and help control costs and schedules. It is important to recognize the speed of developments in the electronics
industry, especially compared to the airframe or engine industry,
and to develop a plan to stay current during development.
• Careful monitoring of airframe weight is important. Airframe
weight instability is an early indicator of problems.
• EVM data should be used to monitor and manage program costs
at the level of IPTs.
• Appropriate use of management reserve can help address program cost risk and can mitigate cost growth.
APPENDIX
DoD and Congressional Oversight
The Department of Defense and U.S. Congress have been keenly
interested in the F/A-22 and F/A-18E/F programs. These programs
account for a significant share of the tactical air forces acquisition
budget. This appendix describes in detail the extent of congressional
oversight in both programs. It also outlines the OSD involvement.
Congressional Oversight
Congress, through its control of the purse strings, can influence the
execution of DoD programs. Four Congressional committees, the
Armed Services and Appropriations Committees in the House and
the Senate have direct jurisdiction over acquisition programs and can
approve all, part, or none of the funding requested by DoD for its
programs, or they can recommend the withholding of funds until
specific conditions are met. The recommendations of the committees,
if enacted in the authorization or the appropriations bills, carry the
force of law. Even if the requests and recommendations of the committee are not signed into law, ignoring them puts DoD at its peril.
Over the past 20 years, Congress has paid close attention to the
progress of the F/A-18E/F and F/A-22 programs. This section
describes in detail specific congressional actions that have affected the
two programs.
The F/A-22 program, from the early stages when it was known
as the ATF program until the present, has been the focus of much
59
60
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
congressional interest and concern, particularly with regard to the
program’s cost, technical challenges, and schedule.
Congress Questioned the F/A-22 Cost Estimates and Acquisition
Strategy
Congress and its agencies have questioned the affordability and feasibility of the F-22 from the very beginning of the program. The Congressional Budget Office (CBO), in its April 1985 review of Air Force
tactical budget issues, stated that cost estimates related to the ATF
could be unrealistically low (CBO, 1985, p. 56). This was in part
because the $30 million price tag quoted by Air Force officials in
“informal conversations” with CBO was not much higher than the
$15 million and $25 million flyaway unit costs of the F-16 and F-15
at that time. Three years later, GAO issued a report that highlighted
the schedule risks associated with initiating low-rate production
before completing tests of prototype aircraft equipped with a fully
integrated avionics system as well as the risk associated with having
parallel development of the airframe, engine, and avionics (GAO,
1988).
Congress itself voiced its concerns regarding the planned acquisition strategy. In 1989, the House Appropriations Committee cut all
funding for the ATF from its version of the fiscal year (FY) 1990
appropriations bill (Aronstein, Hirschberg, and Piccirillo, 1998, p.
130). This drastic action was taken because the committee felt that
the ATF program combined “both an unacceptable degree of concurrence or parallel activities between development and production with
a highly unrealistic assumption of substantial outyear funding levels”(Cooper, 1996). Although full funding for the ATF was eventually restored for FY 1990, the congressional interest kept all of the
issues at a high level of visibility as the program prepared to enter
EMD.
Congressional interest continued after the program entered
EMD in mid-1991 and intensified as the program experienced problems attaining its cost and schedule goals and the projected threat
from Soviet fighters faded. In 1992, 1993, and 1994, Congress did
not appropriate the full amount of funds requested for EMD, reduc-
DoD and Congressional Oversight
61
ing annual requests of about $2 billion by $200 million, $163 million, and $119 million in the three years, respectively. At roughly the
same time, GAO testified before a subcommittee of the Senate
Armed Services Committee regarding the lack of urgency for a
replacement for the F-15 (GAO, 1994). Although GAO’s testimony
did not convince Congress to stop funding development of the F-22,
it did reflect questions being raised by some members of Congress.
Starting in 1995, the F-22 program received additional annual
scrutiny by Congress and was often subject to congressionally
imposed restrictions. In April 1995, GAO issued a report that questioned the degree of concurrent development that existed in the
F-22’s acquisition strategy, particularly in light of the F-22’s dependence on significant technological advances (GAO, 1995). The Senate
Armed Services Committee reflected these concerns in its report
accompanying the Senate’s version of the FY 1996 authorization bill,
stating that it would have serious concerns about any program that
involved an inappropriately high level of concurrency that possesses
high risk (Senate Report, 1995, p. 159). The committee made no
finding on the level or risk of concurrency in the F-22 program. It
did, however, direct the Secretary of Defense to submit a report to
Congress that addressed its concerns on concurrency, weight, and
specific fuel consumption.1
The following year, the Senate Armed Services Committee
raised concerns, reflected in the authorization bill passed in September 1996, about the cost of the program.2 Congress directed the Secretary of Defense to charge the Cost Analysis Improvement Group
(CAIG) to conduct a new independent cost review and to submit a
report to Congress by March 30, 1997, that compared the new cost
estimate with the only previous independent cost estimate of production costs conducted by the CAIG in 1991.
____________
1 Specific
2
fuel consumption is the ratio of the fuel flow rate to the thrust.
Public Law 104-201, National Defense Authorization Act for Fiscal Year 1997, September
23, 1996, Section 217.
62
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
In January 1997, CBO released a report that examined the
Administration’s plans for tactical air forces through 2020 and the
associated costs. The report highlighted several issues concerning the
F-22, including the unlikelihood of the Air Force meeting its cost
goals and the high level of concurrency in the F-22’s schedule. In
November of the same year, the authorization bill for FY 1998 passed
by Congress required GAO to conduct annual reviews of the F-22
program (CBO, 1997). It also set a cap—equal to the estimate
reported by the Joint Estimating Team (JET)—of $18.7 billion on
total expenditures for EMD and a cap of $43.4 billion on the total
amount to be obligated or expended for F-22 production.
The next major congressional constraints were imposed in 1999
attached to funding for FY 2000. The authorization bill required the
Secretary of Defense to certify that the testing plan in the EMD phase
of the F-22 program was adequate for determining the operational
effectiveness and suitability of the F-22 and that the projected total
costs of EMD and production would not exceed the caps set by Congress the previous year, after adjustment for inflation 3 before the Air
Force could award a contract for low-rate initial production (LRIP).
The appropriations bill echoed these sentiments and added additional
restrictions. Specifically, it provided funds for additional test aircraft,
but it did not provide any funds for LRIP aircraft.4 Instead, the bill
restricted award of the LRIP contract until after the first flight of an
F-22 test aircraft equipped with Block 3.0 software; the Secretary of
Defense certified that the test plan was adequate and that the cost for
EMD would not exceed the inflation adjusted cap; and the Director
of Operational Test and Evaluation (DOT&E) reported that the test
plan was adequate to measure and predict the performance of the
F-22’s avionics, stealth, and weapon delivery systems.
Congress was less restrictive in 2000 and 2001. The authorization bill for FY 2001, passed in October 2000, loosened the EMD
____________
3
Public Law 106-65, National Defense Authorization Act for Fiscal Year 2000, October 5,
1999, Section 131.
4 Public
Law 106-79, Department of Defense Appropriations Act for Fiscal Year 2000, October
25, 1999, Section 8146.
DoD and Congressional Oversight
63
cost cap, allowing the Secretary of the Air Force to raise it by 1.5 percent if the DOT&E determined that the additional funds were
needed to support adequate testing. 5 The Appropriations Act for FY
2001 confirmed the prerequisites set in the previous year’s Appropriations Act—that before a fully funded contract to begin LRIP of ten
aircraft could be awarded, the requirements set forth in the Appropriations and Authorization Acts for FY 2000 had to be fulfilled
(GAO, 2001, p. 9). In December 2000, Congress passed separate
legislation providing the Air Force with authority to obligate up to
$353 million of the FY 2001 production appropriation if award of
the full LRIP contract for ten aircraft was delayed beyond December
31, 2000, because the program could not satisfy congressional prerequisites (GAO, 2001, p. 9). In 2001, the cap for EMD was eliminated in the defense authorization bill for FY 2002, although the
production cap and the requirement that GAO report annually on
the progress of the F/A-22 program were retained (GAO, 2002, p.5).
In 2002, congressional concerns resurfaced, however, because of
problems experienced during F/A-22 testing. While providing funds
for 23 production F/A-22s, the Senate Appropriations Committee
accepted assurances that recent problems with the aircraft’s tail section and avionics would not cause delays or additional structural
changes. If, however, future events proved otherwise, the committee
expected the Air Force to cover additional costs from within planned
funding levels (Senate Report, 2002, p. 147). The House Appropriations Committee was less sanguine in its acceptance of Air Force
assurances. In its report, the House Appropriations Committee provided funding for the 23 production aircraft requested by the Air
Force, but prohibited the Air Force from ordering more than 16 F/A22s until the Under Secretary of Defense for Acquisition, Technology, and Logistics (USD (AT&L)) certified that the costs for any retrofits discovered during developmental and operational testing would
be absorbed within the current total program cost (House Report,
2002, p. 167). It also required the USD (AT&L) to submit—also
____________
5
Public Law 106-398, National Defense Authorization for Fiscal Year 2001, October 30,
2000, Section 219.
64
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
before purchasing more than 16 F/A-22s—a cost–benefit analysis
comparing the cost advantages of increasing aircraft production in
2003 to the potential cost of retrofitting production aircraft once the
operational test and evaluation had been completed. The appropriations bill for FY 2003 that was finally passed in October 2002 provided funds for 23 F/A-22s but prohibited obligation of funds for
more than 16 production aircraft until the USD (AT&L) submitted
to the congressional defense committees a formal risk assessment of
the costs associated with increasing F/A-22 production rates before
operational testing and certified that the current production plan was
less risky and costly than a revised plan.6 DoD submitted the risk
assessment and certification in December 2002, but subsequent
events have raised additional congressional concerns.
Since passage of the FY 2003 appropriations bill in October
2002, the F/A-22 program announced additional schedule delays and
cost increases in its EMD and production programs. In response to
these events, GAO issued a report in February 2003, and, as required
by Congress, an annual assessment of the F/A-22 program in March
2003 (GAO, 2003a, 2003b). In these reports, GAO recommended
that the Secretary of Defense provide Congress with documentation
regarding the potential for growth in production costs if current cost
reduction plans did not work as planned and the likely number of
aircraft that could be purchased within the congressional production
cost cap. The March report included additional recommendations
that DoD delay increases in the production rate until after completion of operational testing and provide Congress with an update of
the risk assessment and certification submitted in December 2002. It
is too early to know if the Congress will act on any of GAO’s recommendations but given recent congressional concern, Congress will
likely place some additional constraints or requirements on the program for FY 2004. Figure A.1 summarizes these actions.
____________
6 Public
Law 107-248, October 23, 2002, Section 8119.
DoD and Congressional Oversight
65
Figure A.1
Timeline of Congressional Actions
1993 authorization bill
• Reduces funding by $190 M
• Sets ceiling for EMD: $4.0 B (1990 $)
• Requires SecDef to certify that E/G will
not exceed C/D by more than 25%
GAO report
questions costeffectiveness
1985
I
F/A-18E/F
F/A-22
II/IV
1990 II
1994 appropriations bill
cuts funding by $163 M
GBO questions
program cost
estimate
House Appropriations
Committee expresses
concern, but does
not reduce funding
1995 appropriations bill
cuts funding by $110 M
CBO raises issues
• Cost
• Concurrency
1998 authorization bill
• Requires annual GAO
review
• Sets EMD cap = $18.7 B
1997 authorization bill withholds
10% of procurement funds pending
report comparing cost and performance
of E/F and C/D
III
1995
2000
2000 authorization and
appropriations bills: OSD
must certify
• Test plan
• EMD under inflation-adjusted
cap
Appropriations bill also restricts
LRIP award
• Denies funds
• Sets conditions for award
III
2005
2003 authorization bill
• Provides funds for 23
aircraft
• Prohibits ordering more
than 16 until OSD certifies
retrofits will not add cost
2002 authorization bill
• Removes EMD cap
• Retains annual GAO report
2001 authorization bill raises EMD cost cap to fund testing
Separate legislation provides funds to cover delays stemming
congressional actions
RAND MG276-A.1
The F/A-18E/F Effectiveness Endured Congressional Scrutiny but Not
Its Cost Estimates
The F/A-18E/F faced its own share of criticism from the GAO and
some members of Congress for not providing improvements in capability commensurate with the investment needed to attain them
(GAO, 1996). Nevertheless, specific limitations of funding for the
F/A-18E/F were imposed only twice from 1990 to 2002. The defense
authorization bill for 1993, passed in October 1992, authorized $944
million for the F/A-18E/F program, $190 million less than requested.
The same bill also set several conditions that had to be met before any
of the funds could be obligated. Two of these required the Secretary
of Defense to certify that management systems were in place to
ensure that total EMD costs would not exceed $4.88 billion (in 1990
dollars) and that the cost of the E/F model would not exceed 123
percent of the flyaway cost of the C/D model unless the Navy demonstrated that the higher flyaway costs would produce greater warfighting effectiveness (House Report, 1992). In 1996, congressional
66
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
concern flared again. In June, GAO issued a report questioning the
cost-effectiveness of the F/A-18E/F program (GAO, 1996). Later that
year, Congress passed the bill authorizing defense appropriations for
FY 1997. The accompanying conference report withheld 10 percent
of the procurement funding authorized for 1997 pending a report to
Congress from the Secretary of Defense on the cost and performance
of the E/F model compared to the C/D model (House Report,
1996). In the years since 1996, Congress has placed no additional
significant restrictions on the F-18E/F program.
Both Programs Received Substantial DoD Oversight
DoD reviews major programs at several levels and in various forums
during a program’s life. Programs undergo both service and OSD
reviews of planned funding each year during the annual budget
review process. In addition, OSD reviewed the need for a status of all
major programs during the Quadrennial Defense Reviews (QDRs)
conducted in 1996 and 2000. Finally, OSD and the services can
request reviews of individual programs when problems or questions
arise.
In part because the F/A-22 program was more ambitious in its
development than most programs, it has experienced much more
intervention from DoD during its acquisition phases (see Figure A.2).
The F/A-22 was affected indirectly early on by events outside
the program. As mentioned earlier in the report, the emphasis of the
1985 Packard Commission on acquisition reform encouraged the
services to eliminate uncommercial-like processes. According to a
study by ANSER (Aronstein, Hirschberg, and Piccirillo, 1998), this
led the Air Force, in an effort to make the F/A-22 a model for Air
Force acquisition ingenuity, to assign an initial flyaway cost goal of
$35 million (in FY 1985 dollars), $10 million below the program
office’s original cost goal (Aronstein, Hirschberg, and Piccirillo,
1998). That artificially low goal would come back to haunt the Air
Force and the F/A-22 program years later.
The Bottom-Up Review (BUR) of the nation’s defense forces
conducted by then–Secretary of Defense Les Aspin in 1993 addressed
DoD and Congressional Oversight
67
Figure A.2
Timeline of DoD Actions
QDR cuts production quantities
Total: from 1,000 to 548
Annual peak: from 60 to 48
1985
I
F/A-18E/F
F/A-22
Packard Commission
emphasis on
acquisition reform
and affordability
II/IV
III
1995
1990 II
Bottom-Up Review cuts
quantities from 648 to 442
Budget cut of $65 M
2000
JET reports results in
adding one year
and $1.45 B to EMD
Funding cuts in two
consecutive years
total $200 M
III
2005
OSD comptroller
cuts total program
quantities from
338 to 276
DoD document directs
study of smaller fleet
QDR cuts total program quantities
from 442 to 338
RAND MG276-A.2
the size and composition of U.S. theater air forces. The report issued
in October 1993 recommended proceeding with development and
procurement of the F/A-22 (Aspin, 1993, p. 37). Nevertheless, the
size of the production was decreased following the review from 648 to
442 aircraft to reflect a reduction in the number of Air Force tactical
air wings.
During the mid-1990s—from 1993 to 1995—the F/A-22 program experienced annual decrements to its budgets during the annual
review cycles. Although these cuts were small compared to the program’s total annual funding—$65 million to $100 million was cut
from annual budgets of roughly $2 billion—the reductions did
require the program office to adjust the development schedule.
The QDR conducted in 1996 led to another decrease in the size
of the F/A-22 program. To be consistent with the F/A-22’s enhanced
capability compared to the F-15 that it was replacing and with a
slightly reduced Air Force force structure, then Secretary of Defense
William S. Cohen decreased the total procurement of the F/A-22
from 438 to 339 aircraft (Cohen, 1997, p. 45). He also reduced peak
annual production from 48 to 36 aircraft and slowed the initial ramp-
68
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
up to maximum production to decrease concurrency in development
of key subsystems in the program.
During the same time period, the Assistant Secretary of the Air
Force for Acquisition formed a JET composed of personnel from the
Air Force, OSD and private industry to address the potential for cost
growth that had been identified in management reviews of the program (GAO, 1997). In response to the JET’s findings, the Air Force
proposed adding one year and $1.45 billion to the F/A-22 EMD
phase to reduce risk before entering production. The USD (AT&L)
approved the proposed restructuring in February 1997.
In the FY 2004 defense guidance documents issued in May
2002, Secretary of Defense Donald Rumsfeld directed the Air Force
to study the implications of a much smaller F/A-22 fleet (Thompson,
2002, p. 1). Specifically, the document directed the Air Force to
compare the performance of a smaller fleet of F/A-22s with the thenplanned fleet of 339 aircraft, as well as alternative means for achieving
national security goals, including the F-35 Joint Strike Fighter,
unmanned aerial vehicles, naval strike systems, space systems, and
various applications of information technology. In fall 2002, the Air
Force vigorously and successfully defended its planned fleet of more
than 300 F/A-22s. However, four months later in January 2003, the
OSD Comptroller proposed and got approved a cut in the planned
total production to 276 F/A-22s because of cost overruns projected
during EMD (Butler, 2003).
Another DoD action that shaped the F/A-18E/F program was
the QDR conducted in 1996. As a result of that review, the Navy was
directed to reduce the total size of the F/A-18E/F program from
1,000 to 548 aircraft. The peak annual production was also cut from
60 to 48 aircraft, and the ramp-up to full production was delayed by
two years (Cohen, 1997, p. 45).
Bibliography
Aronstein, David C., Michael J. Hirschberg, and Albert C. Piccirillo, From
the Advanced Tactical Fighter to the F-22 Raptor: Origins of the Air Force’s
21st Century Air Dominance Fighter, Arlington, Va.: ANSER Corporation, March 24, 1998.
Aspin, Les, Report of the Bottom-Up Review, October 1993.
Bailey, Elizabeth K., The F/A-18E/F: An Integrated Product Team (IPT) Case
Study, prepared for the Office of the Director, Test, Systems Engineering, and Evaluation, April 9, 1998.
Butler, Amy, “Air Force Hopes Program ‘Efficiencies’ Will Buy Back Lost
F/A-22s,” Inside the Air Force, January 24, 2003.
Cockburn, Alexander, and Jeffrey St. Clair, “The Madness of the F-22
Fighter,” Counterpunch, May 1, 2001.
Cohen, William S., Report of the Quadrennial Defense Review, May 1997.
Colarusso, Laura M., “Air Force Officials Announce Radar Upgrade for
F/A-22 Raptor,” Inside the Air Force, September 20, 2002.
Congressional Budget Office (CBO), Tactical Combat Forces of the United
States Air Force: Issues and Alternatives, April 1985.
_____, A Look a Tomorrow’s Tactical Air Forces, January 1997.
Cook, Cynthia R., and John C. Graser, Military Airframe Acquisition Costs:
The Effects of Lean Manufacturing, Santa Monica, Calif.: RAND Corporation, MR-1325-AF, 2002.
Cooper, Bert, Jr., CRS Issue Brief 87111: F-22 Aircraft Program, Congressional Research Service, December 30, 1996.
69
70
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
Drezner, Jeffery A., and Richard A. Krop, The Use of Baselining in Acquisition Program Management, Santa Monica, Calif.: RAND Corporation,
MR-876-OSD, 1997.
Drezner, Jeffery A., Jeanne M. Jarvaise, Ronald Wayne Hess, Paul G.
Hough, and D. Norton, An Analysis of Weapon System Cost Growth,
Santa Monica, Calif.: RAND Corporation, MR-291-AF, 1993.
Druyun, Darleen A., statement by Principal Deputy Assistant Secretary of
the Air Force for Acquisition and Management to the Subcommittee on
Airland Forces of the Senate Armed Services Committee, March 17,
1999.
Earned Value Management Implementation Guide, Defense Contracts
Management Command (DCMC), available at http://www.acq.osd.mil/
pm/currentpolicy/jig/evmigl.htm.
Gansler, Jacques, prepared statement to the Acquisition and Technology
Subcommittee of the Senate Armed Services Committee, March 18,
1998.
General Accounting Office, Aircraft Development: The Advanced Tactical
Fighter’s Costs, Schedule, and Performance Goals, GAO/NSIAD-88-76,
January 1988.
_____, Tactical Aircraft: F-15 Replacement Issues, GAO/NSIAD-94-176,
May 1994.
_____, Tactical Aircraft: Concurrency in Development and Production of F-22
Aircraft Should Be Reduced, GAO/NSIAD-95-59, April 1995.
_____, Navy Aviation: F-18E/F Will Provide Marginal Operational Improvement at High Cost, GAO/NSIAD-96-98, June 1996.
_____, Tactical Aircraft: Restructuring of the Air Force F-22 Fighter Program,
GAO/NSIAD-97-156, June 1997.
_____, Defense Acquisitions: Progress of the F-22 and F/A-18E/F Engineering
and Manufacturing Development Programs, GAO/T-NSIAD-99-113,
March 17, 1999.
_____, F-22 Aircraft: Development Cost Goal Achievable if Major Problems
Are Avoided, GAO/NSIAD-0-68, March 2000.
_____, Tactical Aircraft: F-22 Development and Testing Delays Indicate Need
for Limit on Low-Rate Production, GAO-01-310, March 2001.
Bibliography
71
_____, Tactical Aircraft: F-22 Delays Indicate Initial Production Rates Should
Be Lower to Reduce Risks, GAO-02-298, March 2002.
_____, Tactical Aircraft: DoD Needs to Better Inform Congress About Implications of Continuing F/A-22 Cost Growth, GAO-03-280, February
2003a.
_____, Tactical Aircraft: DoD Should Reconsider Decision to Increase F/A-22
Production Rates While Development Risks Continue, GAO-03-431,
March 2003b.
Grossman, Elaine M., “F-22 Fighter Unlikely to Meet Some Required
Capabilities by Test Time,” Inside the Pentagon, August 22, 2002.
House Report 107-532, House Appropriations Committee, Department of
Defense Appropriations Bill, 2003, June 25, 2002.
House Report 102-966, National Defense Authorization Act for Fiscal Year
1993, Conference Report to Accompany H.R. 5006, October 1992.
House Report 104-724, National Defense Authorization Act for Fiscal Year
1992, Conference Report to Accompany H.R. 3230, July 1996.
Institute for Defense Analysis Report, Military Tactical Aircraft Development
Costs, (R-339), 1988.
Jarvaise, Jeanne, Jeffrey A. Drezner, and D. Norton, The Defense System
Cost Performance Database: Cost Growth Analysis Using Selected Acquisition Reports, Santa Monica, Calif.: Rand Corporation, MR-625-OSD,
1996.
Johnson, Robert, and John L. Birkler, Three Programs and Ten Criteria:
Evaluating and Improving Acquisition Program Management and Oversight
Processes with the Department of Defense, Santa Monica, Calif.: RAND
Corporation, MR-758-OSD, 1996.
Large, Joseph P., and John L. Birkler, An Assessment of F/A-18E/F Airframe
Cost Estimates, Santa Monica, Calif.: RAND Corporation, WD-6293-1USN, FOUO, 1992.
Lorell, Mark A., and John C. Graser, An Overview of Acquisition Reform
Cost Savings Estimates, Santa Monica, Calif.: RAND Corporation, MR1329-AF, 2001.
Lorell, Mark A., and Hugh Levaux, The Cutting Edge: A Half Century of
U.S. Fighter Aircraft R&D, Santa Monica, Calif.: RAND Corporation,
MR-939-AF, 1998.
72
Lessons Learned from the F/A-22 and F/A-18E/F Development Programs
Myers, Dominique, “Acquisition Reform—Inside the Silver Bullet: A
Comparative Analysis, JDAM Versus F-22,” Acquisition Review Quarterly, Defense Acquisition University, Fall 2002.
Office of the Secretary of Defense (OSD), Use of Integrated Product and
Process Development and Integrated Product Teams in DoD Acquisition,
May 10, 1995.
Senate Report 104-112, Report on Authorizing Appropriations for Fiscal Year
1996, July 12, 1995.
Senate Report 107-213, Department of Defense Appropriations Bill for Fiscal
Year 2003, July 18, 2002.
Thompson, Loren, “Gunning for Raptor,” Defense Week, July 22, 2002.
Younossi, Obaid, Michael Kennedy, and John C. Graser, Military Airframe
Costs: The Effects of Advanced Materials and Manufacturing Processes,
Santa Monica, Calif.: RAND Corporation, MR-1370-AF, 2001.
Younossi, Obaid, Mark V. Arena, Richard M. Moore, Mark A. Lorell,
Joanna Mason, and John C. Graser, Military Jet Engine Acquisition:
Technology Basics and Cost-Estimating Methodology, Santa Monica, Calif.:
RAND Corporation, MR-1596-AF, 2002.