ASMC PDI
2010
Nashville, TN
Workshop #57
X
(you are here)
Risk, Uncertainty, and
Trouble: Escaping the
RUT of Program
Instability
Col Brian Shimel, CDFM
Chief, Financial Analysis
Air Force Space Command
2010
Risk, Uncertainty, and
Trouble: Escaping the
RUT of Program
Instability
Col Brian Shimel, CDFM
Chief, Financial Analysis
Air Force Space Command
2010
What is Behind the Curtain
for this Lucky Audience?
•Appropriations—The Hidden Assumptions
•Purpose, Amount, and Time
•Risk, Uncertainty & a bit of statistics
•Examples in Operations, Acquisition
•Risk, Uncertainty, and the Point Estimate
•Defending the Budget
•Acquisition: Why do we have to keep reforming it?
•Ethics
•8 Step Problem Solving Process
•Decision Support
•Backups -- Sources & GAO Excerpts
Appropriations
•
•
•
•
•
O&M--3400/3500 (prd of performance-1yr)
R&D – 3600 (2 yr)
Procurement -- 3010/3020/3080 (3 yr)
MILCON & MFH (5 yr)
Accept the $$, accept a “project” schedule
– Base Year (BY) -- $B, $M, $K
– Then Year (TY) composite rate (outlay profile)
• 11% 42% 34% 9% 4%
Risk
• The Chance of an Unfavorable Event
– Cost Risk – How Much?
– Technical Risk – Will it Work?
– Schedule Risk -- When? How long?
UNCERTAINTY
What should I say about it?
UNCERTAINTY
• Unknown, unpredicted variation around an
expected value/outcome (a prediction of a
future state)
• Risk and uncertainty are situational and can
be inverse, they can be related
– Affected by expectations
UNCERTAINTY
U
UNCERTAINTY
Y
UNCERTAINTY
U
Y
Dependent
A Distribution
Independent
Dependent
A Distribution
Independent
Dependent
A Distribution
Independent
Dependent
A Distribution
Independent
Dependent
A Distribution
Independent
Normal
Mode = Median = Mean
Small Measure of Dispersion
Narrow, Normal, Certain
Age of Children in Kindergarten
Making Light Bulbs
A-12
Widely distributed, Normal, Varied
Age or Height of People in Audience
Right Skewed
Mean > Median
Right Skewed
dispersion strongly favors one side
Right
Right Skewed
Air Force Right
BASELINE
AAFES in Cheyenne Mt
•
•
•
•
•
August 2004; Mess Attendant contract
New proposal --$500K over budget
Dining Facilities in 2 locations
3 people without separate rations
A captive clientele
Narrow, Normal, Certain
Narrow, Normal, Certain
The Point Estimate
Normal but Not Certain
Normal but Not Certain
The Point Estimate?
2 Different Processes
2 Different Owners
3 Different Processes
What Do We Communicate?
Program Bid, Baseline, Estimate
Actual Space Program Cost Distribution
250
Should a leader make his/her
decision based on an 80% CL
value for this estimate of total
program costs …
200
Frequency
… or an 80% CL value for
this estimate?
150
100
50
5,500
6,000
6,500
7,000
7,500
8,000
BY02$M
8,500
9,000
9,500
10,000
The elephant in the room
Making the Elephant Disappear
This is our Story & we’re . .
HQ Budget Drill
Not Gone at the Working Level
Accepted Risk
Unexpected Risk
Little Big Problem
Taking on the Management Challenge
Estimates –v- Reality
Right Skewed—Mean is greater than
the median
Classic DoD Project
Classic DoD Project
Classic DoD Project
Weapons Development
Complex Development
Sub-Components
Improved Cost & Schedule Control
Stable and Predictable
GAO-08-619, July 2008
Knowledge-based Acquisition
GAO claims DoD overruns its
acquisition baseline by 30%
Uncertainty
(so does the RAND Corp, and so does AFCAA)
I want it now
Time
I am willing to think it through
GAO-08-619, July 2008
• Cost estimates that lack knowledge and rely
heavily on assumptions have inherently high
levels of risk and uncertainty
• DOD’s flawed funding process is largely driven
by decision makers’ willingness to accept
unrealistic cost estimates and DOD’s
commitment to more programs than it can
support.
Risk Uncertainty and Trouble
• It is a disservice to present a decision-maker
with an estimate for a new groundbreaking
weapon system (at a high confidence with
only a small increase in funding) . . . And it is
foolhardy for a decision-maker to accept that
estimate.
UNITED STATES DEPARTMENT OF DEFENSE
Fiscal Year 2011 Budget Request
Chapter 5
REFORMING HOW DoD DOES BUSINESS
Reform How DoD Buys
Acquisition reform is the process by which DoD is
changing how it buys its weapons and other
important systems and investments. DoD is
focused on acquisition reform at the strategic and
tactical level.
•
At the strategic level, we are ensuring
investments are affordable and consistent
with warfighting priorities.
•
At the tactical level, we are focused on
acquiring warfighting capability via a
disciplined, effective and knowledgebased acquisition process.
Acquisition Reform
• WSARA (2009)
• OSD
– CAPE
– BCEFM
– PARCA (Nunn-McCurdy Root Cause?)
• AF
– AIP
– AFCAA evaluation of POM inputs
– Section 852 funding
Implementation of WSARA
Performance Assessment & Root Cause Analysis (PARCA)
Role of the senior official for PARCA:
•Conduct performance assessments for MDAPs periodically or
when requested by SECDEF, USD(AT&L), Secretary of Military
Dept, or head of Defense Agency
•Conduct root cause analysis for MDAPs as required by 10 USC
2433a, or when requested by SECDEF, USD(AT&L), Secretary
of Military Dept, or head of Defense Agency
•Advise acquisition officials on MDAP performance issues:
‫ ـ‬Prior to certification under 10 USC 2433a (critical cost growth in
MDAPs);
‫ ـ‬Prior to entry into full-rate production; and
‫ ـ‬Upon consideration of decision to request authorization for multi-year
procurement contract
Policy Impact: Newly established position to perform required functions
TITLE 10—ARMED FORCES
•
•
•
•
•
Subtitle A—General Military Law
Subtitle B—Army
Subtitle C—Navy and Marine Corps
Subtitle D—Air Force
Subtitle E—Reserve Components
TITLE 10 > Subtitle A > PART IV
• PART IV—SERVICE, SUPPLY, AND
PROCUREMENT
• TITLE 10 > Subtitle A > PART IV > CHAPTER 144
• CHAPTER 144—MAJOR DEFENSE
ACQUISITION PROGRAMS
Unit Cost Reports
• § 2430. Major defense acquisition program defined
• § 2430a. Major subprograms
• § 2431. Weapons development and procurement schedules § 2432.
Selected Acquisition Reports
• § 2433. Unit cost reports
• § 2434. Independent cost estimates; operational manpower requirements
• § 2435. Baseline description
• § 2436. Major defense acquisition programs: incentive program for
contractors to purchase capital assets manufactured in United States
• § 2437. Development of major defense acquisition programs: sustainment
of system to be replaced
• [§§ 2438, 2439. Repealed.]
• § 2440. Technology and industrial base plans
Nunn-McCurdy
• § 2433(e)(2). If the program acquisition unit cost or procurement unit cost
of a major defense acquisition program or designated major subprogram
(as determined by the Secretary under subsection (d)) increases by a
percentage equal to or greater than the critical cost growth threshold for
the program or subprogram, the Secretary of Defense, after consultation
with the Joint Requirements Oversight Council regarding program
requirements, shall carry out an assessment— (A)
• and state that, (B)
– (i) such acquisition program is essential to the national security;
– (ii) there are no alternatives to such acquisition program which will provide equal or
greater military capability at less cost;
– (iii) the new estimates of the program acquisition unit cost or procurement unit cost are
reasonable; and
– (iv) the management structure for the acquisition program is adequate to manage and
control program acquisition unit cost or procurement unit cost;
HOUSE ARMED SERVICES COMMITTEE PANEL ON
DEFENSE ACQUISTION REFORM FINDINGS AND
RECOMMENDATIONS
March 23, 2010
The Panel’s approach is to expand the mandate of the Office of
Performance Assessment and Root Cause Analysis (PARCA) to serve as a
performance assessment function for the acquisition system. PARCA would
track organizations throughout the defense acquisition system in meeting
pre-negotiated goals for acquisition performance. Organizations would
see real consequences for both success and failure in meeting their goals.
The intent of the performance metrics and performance management
recommendations in this report is not to create new layers of bureaucracy.
If the performance management measures envisioned in the Panel’s
recommendations are implemented properly, program managers could be
freed from certain reporting responsibilities specific to their individual
program.
The IMPROVE Acquisition Act, 14 Apr 10
The IMPROVE Acquisition Act, H.R. 5013, would implement
recommendations of the House Armed Services Committee Panel on
Defense Acquisition Reform. The panel issued its final report in March
after a year-long comprehensive review of how Defense buys goods and
services.
“When you consider that 60 percent of the Pentagon's procurement
dollars are for services contracts alone, the legislation we are
introducing today has the potential to affect major changes at the
Department of Defense and save billions of taxpayer dollars," said Rep
Howard McKeon, R-Calif.
Among other things, the bill would require the department to revamp
the requirements process, apply performance management to the
defense acquisition system and create metrics supported by goals and
standards. The newly created Office of Performance Assessment and
Root Cause Analysis would be responsible for assessing compliance with
the metrics.
Estimates -v- Reality
?
8 Step Problem Solving Process
1. Clarify and Validate the Problem
2. Break Down the Problem and Identify Performance Gaps
(Observe)
3. Set Improvement Target
4. Determine Root Cause (Orient)
5. Develop Countermeasures (Decide)
6. See Countermeasures Through
7. Confirm Results and Process
8. Standardize Successful Processes (Act).
Establishing a Culture of Accountability
CNA Independent Assessment:
Air Force Acquisition Return to Excellence, Feb 2009
Weapons Development
Where do we draw the bullseye?
?
Winnie The Pooh
on Process Improvement
Here is Edward Bear, coming downstairs
now, bump, bump, bump, on the back of his
head, behind Christopher Robin. It is, as far
as he knows, the only way of coming
downstairs, but sometimes he feels that there
really is another way, if only he could stop
bumping for a moment and think of it.
-- Winnie The Pooh by A A Milne
Why?
• Personal -- Money, sex, power, envy, greed &
sloth, a sense of entitlement?
• Professional
– Misplaced loyalty to the mission? Group think?
Defending the Chain of Command
• “Our role is to provide Ground Truth, so
leaders can make an informed decision”
• Before we
“ . . . convince ourselves we can do it”
• Lt Gen Ted Bowlds
The Danger of Compromise
Ethical Bullseye
The Danger of Compromise
Ethical Bullseye
Initial Decision
The Danger of Compromise
NEW AIMING RING!
Initial Decision
Ethical Bullseye
The Danger of Compromise
2nd generation decision:
Subordinate’s action
Ethical Bullseye
Initial Decision
The Danger of Compromise
2nd generation decision:
Subordinate’s action
Ethical Bullseye
Initial Decision
ACTUAL
AIMING RING
How Do We Mentor Ethics?
1. Discuss Ethical Concepts
2. Be Vigilant for Warning Signs
3. Correct Misconduct
Integrity - Service - Excellence
Top 11 List - Rationalization
1. What’s the worst thing they can do to us
2. That’s the way we’ve been doing it for years
3. Don’t ask a question if you can’t stand the
answer
4. Not enough time for a legal review
5. It’s a stupid rule
6. Boss said to make it happen
7. Warfighter needs it now
8. I saw someone else do it that way
9. No harm, no foul
10.We’re thinking outside the box
11.It’s for a good cause
Rationalized Standards Jeopardize Core Values & Fiscal Integrity
85
Adapt to Decision Support
• Get involved earlier in the decision process
– Give advice BEFORE the decision is made, and
--“It must be brilliant because the Boss likes it”
• Use more math, law, policy, objective analysis
• Accept more Risk and Uncertainty
• Do more public speaking
– A little bit scarier than death to many
Estimates -v- Reality
Questions?
Sources
• Risk, Uncertainty and Trouble
• Nov/Dec 08 Defense AT&L Magazine
– http://www.dau.mil/pubscats/PubsCats/atl/new/nov-dec08.pdf
• The National Estimator, Fall 2008
• Air Force Comptroller, Summer 2008
• Looking at the Root Cause of Problems
• May/Jun 08 Defense AT&L Magazine
– http://www.dau.mil/pubscats/PubsCats/atl/new/may-june08.pdf
• Army History of Acquisition
• Adapting to Decision Support
• Armed Forces Comptroller, Spring 2009
Cheyenne Mountain revisited
• NORAD Mission Uncertainty
• AAFES in the Mountain
• Long Memories
Range of RDT&E Estimates
BY$M
Baseline
DEVELOPMENT PHASE
Statistics
Calculated with 500 iterations
Original
Estimate
CONTRACTOR
RFI ROMS
100%
Confidence Level (CDF)
90%
80%
New
ESTIMATE
ACAT I
70%
60%
50%
40%
30%
20%
10%
0%
0
200
400
600
800
1000
BY2008 $M
Baseline (cdf)
Point Estimate
Mean
50% Confidence Level
80% Confidence Level
Budget Estimate
Original
ACAT I Threshold ($BY) = $428M
91
Contractor RFI ROMs
Narrow, Normal, Certain
pdf
Cumulative Density Function
Narrow, Normal, Certain
Not so Much
GAO-08-619, July 2008
• A Knowledge-Based Funding Approach Could Improve Major Weapon
System Program Outcomes
• DOD often does not commit full funding to develop its major weapon
systems when they are initiated, despite the department’s policy to do so.
For a majority of the weapon system programs GAO reviewed, costs have
exceeded the funding levels initially planned for and reflected in the
Future Years Defense Program (FYDP)—DOD’s investment strategy. To
compensate for these shortfalls, DOD makes unplanned and inefficient
funding adjustments, like moving money from one program to another,
deferring costs into the future, or reducing procurement quantities.
GAO-08-619, July 2008
• DOD’s flawed funding process is largely driven by decision makers’
willingness to accept unrealistic cost estimates and DOD’s commitment to
more programs than it can support. DOD often underestimates
development costs—due in part to a lack of knowledge and optimistic
assumptions about requirements and critical technologies. At the same
time, DOD’s continued failure to balance its needs with available resources
promotes unhealthy competition among programs for funding. This
creates incentives for service and program officials to establish
requirements that make their particular weapon systems stand out, with
less consideration of the resources needed to develop them. Ultimately,
DOD tends to push the need for funding to the future rather than limit
program length or adjust requirements.
• Inaccurate cost estimates are often the result of
limited knowledge about requirements and
technologies. Our best practices work has shown
that conducting early disciplined analysis, such as
systems engineering, builds knowledge that enables
a developer to identify and resolve gaps between
requirements and available resources before
beginning product development.
• The successful commercial companies that GAO has
previously reviewed achieve adequate and stable funding for
product development programs by following a disciplined,
knowledge-based approach to estimating program costs;
using manageable development cycles to increase the
predictability of funding needs and the likelihood of program
success; and using portfolio management practices to make
decisions about which programs to pursue. Once programs
are approved, these companies firmly commit to fully fund
them.
• Cost estimates that lack knowledge and rely heavily on assumptions have
inherently high levels of risk and uncertainty. Conducting quantitative risk
and uncertainty analysis provides a way to assess the variability in an
estimate. Using this type of analysis, cost estimators can model such
effects as a schedule slipping or a key technology failing to materialize,
thereby identifying a range of likely costs around an estimate. Presenting
decision makers with the range of likely costs around an estimate provides
insight into the amount of cost, schedule, and technical risks they are
being asked to accept and conveys a level of confidence associated with
achieving the proposed estimate. A range of costs also provides a basis for
deciding how much funding a program needs to be successful.
• In developing cost estimates, the services and the CAIG often
do not present decision makers with the range of costs
around an estimate. Instead, they present a single, or point,
estimate as the most probable cost, which OSD expects to be
a 50 percent chance that actual program costs will be at or
below the estimated value. Several recent studies have
questioned DOD’s approach and recommended establishing
estimates with higher levels of confidence under the
assumption that the result will be more realistic estimates.
• For example, the Defense Acquisition Performance
Assessment Panel recommended adjusting program cost
estimates to reflect “high confidence”—defined as an 80
percent chance of completing development at or below the
estimated amount. Requiring a higher confidence level could
provide a better basis for determining program funding, but
only if the quality of the cost estimate is sound and the
underlying risk and uncertainty associated with the estimate
are accurately captured. If the quality of the estimate is poor
to begin with, simply applying a higher confidence level to the
estimate will not make it any more realistic
Findings #1 & #2
• Develop and implement a strategy to bring the department’s
current portfolio into balance by aligning the number of
programs and the cost and schedule of those programs with
available resources.
• Require that all new programs have manageable development
cycles, realistic cost estimates, and have planned and
programmed full funding for the entire development cycle.
Findings #3
• Require all cost estimates submitted for funding a
program at milestone decisions to be reported as a
range of likely costs and reflect the associated levels
of risk and uncertainty. At Milestone A, require
estimates that allow for a wide range of likely costs.
At Milestone B, require estimates that, based on
knowledge gained, are more precise—in line with
best practice standards.
GAO response to DoD Reply
• In written comments on a draft of this report, DOD partially
concurred with our first and second recommendations and
non-concurred with the third. DOD’s partial concurrences are
rooted in the belief that its current policies and initiatives
address our recommendations. We agree that aspects of
DOD’s current policies appear consistent with our
recommendations, and the initiatives could contribute to
better program outcomes if implemented appropriately.
However, we have found no evidence of widespread adoption
of these policies or any other process that would better
ensure adequate funding for DOD’s major weapon system
acquisition programs and increase the likelihood of achieving
successful outcomes.
More Chapter 5
Reform How DoD Buys
• Our objective is to achieve predictable cost, schedule
and performance outcomes based on mature,
demonstrated technologies and realistic cost
estimates. We are also implementing initiatives that
will increase the numbers and capabilities of the
acquisition workforce, improve funding stability,
enhance the source selection process, and improve
contract execution. Our intent is to provide the
warfighter with world class capability while being
good stewards of the taxpayer dollar.
8 Step Problem Solving Process
1.
The OODA Loop (Observe, Orient, Decide, Act) was originated in the
1950's by Col. John Boyd. It is an objective description of the decision
making process. Due to an emphasis on the infinitely repeating nature of
decision making, the OODA Loop is an excellent match for the AFSO21
principal of continuous improvement. The eight steps of the Problem
Solving Model fit within the frame of the OODA Loop. The eight steps are
(1) Clarify and Validate the Problem, (2) Break Down the Problem and
Identify Performance Gaps (Observe), (3) Set Improvement Target,
(4)Determine Root Cause (Orient), (5) Develop Countermeasures
(Decide), (6) See Countermeasures Through, (7) Confirm Results and
Process, and (8) Standardize Successful Processes (Act). Following the
eight steps will ensure that actions lead to the desired results with an
absolute minimum of wasted effort. Following the steps will also ensure
the results are aligned with the needs of the organization.
Implementation of WSARA
Root Cause Analysis
Considers the underlying cause or causes for shortcomings in
cost, schedule, and performance including the role, if any, of:
‫ ـ‬Unrealistic performance expectations;
‫ ـ‬Unrealistic baseline estimates for cost and schedule;
‫ ـ‬Immature technologies or excessive manufacturing or integration
risk;
‫ ـ‬Unanticipated design, engineering, manufacturing, or integration
issues arising during program performance;
‫ ـ‬Changes in procurement quantities;
‫ ـ‬Inadequate program funding or funding instability;
‫ ـ‬Poor performance by government or contractor personnel
responsible for program management;
‫ ـ‬or any other matters.
Weapons Systems Acquisition Reform Act
“The key to successful acquisition programs is
getting things right from the start with sound
systems engineering, cost estimating, and
developmental testing early in the program cycle.
The bill that we are introducing today will require the
Department of Defense to take the steps needed to
put major defense acquisition programs on a sound
footing from the outset. If these changes are
successfully implemented, they should help our
acquisition programs avoid future cost overruns,
schedule delays, and performance problems.”
–Senator Carl Levin, Chairman, Senate Armed Services Committee
Reducing Cycle Time
“The only way to reduce cycle time is to know
what you are going to do before you start”
-- Any parent who ever put together anything on
Christmas morning while their child waited for them to
finish before they could start playing.
Implementation of WSARA
Critical Cost Growth (1)
DTM contains policy implementing new 10 USC 2433a, Critical Cost
Growth of MDAPs, that amends 10 USC 2433, Unit Cost Reports, and
supersedes all previous USD(AT&L) policies addressing actions that must
be taken following critical cost growth of a MDAP or designated
subprogram
•PM shall notify the CAE immediately, whenever there is a reasonable cause to
believe that the current estimate of either the program acquisition unit cost
(PAUC) or average procurement unit cost (APUC) of a MDAP or designated
subprogram (in base-year dollars) has increased by 25 percent (or more) over
the PAUC or APUC objective of the currently approved APB estimate, or 50
percent (or more) over the PAUC or APUC of the original APB estimate.
•If the CAE determines that there is an increase in the current estimate of the
PAUC or APUC of at least 25 percent over the PAUC or APUC objective of the
currently approved APB, or 50 percent over the PAUC or APUC of the original
APB, the CAE shall inform the USD(AT&L) and the Head of the DoD Component.
Implementation of WSARA
Critical Cost Growth (2)
• If the Component Head subsequently determines that there is, in fact, an
increase in the current estimate of the PAUC or APUC of at least 25 percent
over the currently approved APB, or 50 percent over the PAUC or APUC of the
original APB, the Head of the DoD Component shall notify Congress, in writing,
of the determination of critical cost growth and the increase with respect to the
program or subprogram concerned.
• The notification shall be not later than 45 days after the end of the quarter, in
the case of a quarterly report; or not later than 45 days after the date of the
report, in the case of an out-of-cycle report based on critical change occurring
between quarters. In either case, notification shall include the date that the
Head of the DoD Component made the determination.
• In addition, the Component Head shall submit an SAR for either the fiscal year
quarter ending on or after the determination date, or for the fiscal year quarter
that immediately precedes the fiscal year quarter ending on or after the
determination date. This SAR shall contain the additional critical cost growthrelated information.
Implementation of WSARA
Critical Cost Growth (3)
• The USD(AT&L), after consultation with the JROC, shall determine the root
cause or causes of the critical cost growth in accordance with applicable
statutory requirements and DoD policies, procedures, and guidance based
upon the root cause analysis conducted by the senior official for PARCA; and in
consultation with the DCAPE, shall carry out an assessment of:
a. The projected cost of completing the program if current requirements are
not modified.
b. The projected cost of completing the program based on reasonable
modification of such requirements.
c. The rough order of magnitude of the costs of any reasonable alternative
system or capability.
d. The need to reduce funding for other programs due to the growth in cost
of the program.
Implementation of WSARA
Critical Cost Growth (4)
• After conducting the reassessment, the USD(AT&L) shall terminate the program
unless the USD(AT&L) submits a written certification to Congress before the
end of the 60-day period beginning on the day the SAR containing the unit cost
information is required to be submitted to Congress. The certification must
state:
a. The continuation of the program is essential to the national security.
b. There are no alternatives to the program that will provide acceptable
capability to meet the joint military requirement (as defined in section
l8l(g)((1) of Title 10, U.S.C) at less cost.
c. The new estimates of the PAVC or APUC have been determined by the
DCAPE, to be reasonable.
d. The program is a higher priority than programs whose funding must be
reduced to accommodate the growth in cost of the program.
e. The management structure for the program is adequate to manage and
control PAUC or APUC.
Implementation of WSARA
Critical Cost Growth (5)
• The written certification shall be accompanied by a report presenting the root cause
analysis and assessment and basis for each determination made in accordance with
the five certification criteria listed previously
• If the USD(AT&L) elects NOT to terminate a MDAP that has experienced critical cost
growth, the Secretary of Defense shall:
a. Restructure the program in a manner that addresses the root cause or causes of the
critical cost growth, and ensures that the program has an appropriate management
structure as set forth in the written certification;
b. Rescind the most recent milestone approval for the program or designated subprograms
and withdraw any associated certification(s) pursuant to section 2366a or 2366b.
c. Require a new milestone approval for the program or designated subprograms before
taking any contract action to enter a new contract, exercise an option under an existing
contract, or otherwise extend the scope of an existing contract under the program,
except to the extent determined necessary by the MDA, on a non-delegable basis, to
ensure that the program can be restructured as intended by the Secretary of Defense
without unnecessarily wasting resources.
d. Include in the report a description of all funding changes made as a result of the growth
in cost of the program, including reductions made in funding for other programs to
accommodate such cost growth. (The report specified here is the first SAR for the
program submitted after the President submits a budget in the calendar year following
the year in which the program was restructured.)
Implementation of WSARA
Critical Cost Growth (6)
• Additionally, for each MDAP that has exceeded the critical cost thresholds, but has
not been terminated, the senior official for PARCA shall conduct semi-annual reviews
until 1 year after the date a new milestone approval is received. The senior official for
PARCA, shall report the results of the semi-annual reviews to the USD(AT&L) and
summarize the results in the Director's next annual report.
• If a MDAP is terminated after experiencing a critical cost breach, the USD(AT&L) shall
submit to Congress a written report with the following information:
a. An explanation of the reasons for terminating the program.
b. The alternatives considered to address any problems in the program.
c. The course the Department of Defense plans to pursue to meet any continuing
joint military requirements otherwise intended to be met by the program.