Program
Management
and
Earned Value
Management
David G. Ahern
Director, Portfolio Systems Acquisition
Deputy Under Secretary of Defense (A&T)
Agenda
•
•
•
•
•
Interdependencies & Government/Industry Dynamics
Principles of Program Management
Data Quality & Integrity
EVM-Related Issues/Challenges
Improving Program Performance & Collaboration
Program Manager's Challenge:
Striking The Right Balance
PROGRAM
•
•
•
•
GOVERNMENT
•
•
•
•
REQUIREMENT
CONTRACT TYPE
TERMS AND CONDITIONS
AWARD AND ADMINISTRATION
PERFORMANCE
COST
SCHEDULE
SUPPORTABILITY
INDUSTRY
• PRODUCT
PERFORMANCE
• INVESTMENT
• FINANCING/CASH
• PROFIT
Understanding Government/Industry
Dynamics
Cost Growth
Sales & Profit
$200,000
$65,000
$25,000
Sales / Revenue
$180,000
Earnings / Profit
$60,000
$160,000
$55,000
$140,000
Est. Cost
Growth
$50,000
$45,000
$40,000
DIVERGENT
CENTERS
OF GRAVITY
$20,000
$120,000
$15,000
$100,000
$80,000
$10,000
$60,000
$40,000
$35,000
$5,000
$20,000
$30,000
Orig.
CBB
EAC
$0
$0
2001
2002
2003
2004
2005
2006
CUSTOMER
MARKET
Warfighter / Taxpayer
Stockholder
Program Performance
Financial Performance
Cost / Schedule / Technical
Sales / Profit / Cash Flow
We Must Find A Balance
4
RDT&E Cost Growth
DEVELOPMENT COST GROWTH
BY CATEGORY
AVG RDT&E COST GROWTH
FROM ACQUISITION
PROGRAM BASELINE
ESTIMATING
ASSUMPTIONS
20%
• CER’s
• LABOR RATES
• CA TO CDR
• CDR TO FIRST FLT
• FIRST FLT TO IOC
(ACROSS ALL PROGRAMS)
65%
SCHEDULE
ASSUMPTIONS
21%
TECHNICAL BASELINES
59%
• WEIGHT
• VOLUME
• COMMONALITY
• SOFTWARE
• # TEST ARTICLES
• # TEST FLIGHTS
Development Programs Experiencing Significant Cost
Growth From Acquisition Program Baselines
Principles of Program
Management
• Set the baseline – what is the task?
– Performance, Cost, Schedule, and Risk
• Integrated Product and Process Development
– Optimize the design, manufacturing, and supportability
processes
– By integrating all essential acquisition activities through the use
of multidisciplinary teams.
•
Dynamic Environment
– Expect Changes
– Communicate, Cooperate
• Measurement and Reporting
– EVM, C/SSR Reporting
DoD Program Management
TWO DISTINCT BUT RELATED AREAS OF FOCUS
Acquisition Program
Management
• Acquisition Planning
• APB
• Source Selection Plan
• Award Fee
• SAR/DAES
Integrated Project
Management (IPM)
• Project Status/CPI & SPI
• Estimate at Completion
• Staffing
• Tasks Completed
• Management Processes
• Risk Mitigation
• Technical Reviews
IPM/EVM Is Not Just CPI & SPI
IPM
Cost
Schedule
Technical
7
Integrated Project Management
Earned Value Management
PLANNING
•
•
•
•
•
•
•
•
•
•
•
Contract Requirements
Source Selections
Award Fee Criteria
Program Item Descriptions (PID)
Management System
Assessments (MSAs)
Integrated Baseline Reviews
(IBRs)
Integrated Master Schedule
(IMS)
Cost Analysis Requirements
Description (CARD)
Risk Identification
Key Performance Indicators
(KPPs)
Resource Forecasts
EXECUTION
IPM
Cost
Schedule
Technical
• Technical Reviews (PDR, CDR,
etc.)
• Management System
Assessments (MSA)
• Integrated Baseline Reviews
(IBRs)
• Award Fee Determination
• Schedule Risk Assessments
(SRAs)
• Program Management Baseline
(PMB)
• Estimates at Completion (EACs)
• Risk Mitigation
• Program Performance
Measurement
• Burn Rate Trends
• Management Reserve Usage
• Staffing Plans
• Baseline Execution Evaluations
IPM = Cost, Schedule And Technical Integration
“The Application of Basic Project Management Principles”
8
Leading Indicators –
Contractor Staffing And Task
Completion
MACRO ANALYSIS
MICRO ANALYSIS
Month 48
0 1 2
1400
1000
Full-Tim e Equivalents (FTEs)
1200
INCOMPLETE WORK
800
600
400
3 4 5 6 7 8
Year 2
Year 3
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36
Award
Feb-06
M ar-06
Apr-06
M ay-06
Jun-06
Jul-06
Aug-06
Sep-06
O ct-06
N ov-06
D ec-06
Jan-07
Feb-07
Mar-07
Apr-07
May-07
Jun-07
Jul-07
Aug-07
Sep-07
Oct-07
Nov-07
Dec-07
Jan-08
Feb-08
M ar-08
Apr-08
M ay-08
Jun-08
Jul-08
Aug-08
Sep-08
Oct-08
Nov-08
Dec-08
Jan-09
Month 47
Month 46
Month 45
Year 1
Month 44
Month 43
Month 42
Month 41
Month 40
Month 39
Month 38
Month 37
Month 36
Month 35
Month 34
Month 33
Year 4
Month 32
Month 31
Month 30
Month 29
Month 28
Month 27
Month 26
Month 25
Month 24
Month 23
Month 22
Month 21
Year 3
Month 20
Month 19
Month 18
Month 17
Month 16
Month 15
Month 14
Month 13
Month 12
Month 11
Month 10
Month 9
Year 2
Month 8
Month 7
Month 6
Month 5
Month 4
Month 3
Month 2
Award
Year 1
800
700
600
500
400
300
200
100
0
-100
-200
-300
-400
-500
-600
-700
-800
-900
-1000
-1100
-1200
UNDERSTAFFED
* equivalent of 3 man-months
per person
P lanned S taff
A ctual S taff
C ontracted S taff
N ext six m onths staffing plan
Last M onth's six m onth staffing plan
A ctual m an-m onths over or under plan
C um ulative m an-m onths over or under plan
200
0
STAFFING SHORTFALL
B a s e l in e E x e c u tio n
400
S t ill In c o m p le te
F in is h O ve r O n e M o n t h L a te
F in is h le s s t h a n O n e M o n t h L a t e
F in is h O n Tim e
F u t u re t a s k p la n n e d
F in is h E a rly
350
-200
300
-400
N u m b er o f T a sk s
250
INCOMPLETE WORK
-600
200
150
-800
* >40% of delinquent
tasks not yet started
-1000
-1200
100
50
8
8
-0
b
e
F
7
7
-0
an
J
7
0
-0
-0
v
ec
o
N
D
7
7
-0
tc
O
7
0
-0
l-
g
p
u
u
e
S
A
7
7
-0
y -0
n
a
p
u
J
J
M
7
7
7
0
r -0
-0
a
e
b
r-
A
F
M
6
6
6
7
-0
J
an
-0
ec
D
6
0
-0
v
tc
O
o
-0
-0
p
g
u
Program C Cumulative Understaffed
e
Program C Incomplete Tasks
Program B Cumulative Understaffed
S
Program B Incomplete Tasks
Program A Cumulative Understaffed
A
Program A Incomplete Tasks
N
6
0
-1400
INITIAL RESULTS INDICATE FEW CONTRACTORS HAVE THE CAPABILITY TO DEVELOP AND
MAINTAIN A HEALTHY INTEGRATED MASTER SCHEDULE.
LAUNCHED THREE ANALYSIS ACTIVITIES
–
–
–
SCHEDULE CAPABILITY
SKILLS STAFFING ENVIRONMENT
SUBCONTRACTING ENVIRONMENT
PRIMES WHO SUBCONTRACT LARGE PORTIONS OF WORK EXPERIENCE GREATER SCHEDULE
DEVELOPMENT & MAINTENANCE DIFFICULTY, PARTICULARLY IF WORK IS SUBBED USING FFP.
PRIMES ARE EXPERIENCING SKILLS SHORTAGES AND OVERAGES – MOST ARE INCAPABLE OF
ACCURATELY PREDICTING OR FULFILLING SKILLS DEMANDS FOR +1 MONTH.
SUBCONTRACTED WORK IS INCREASING; UP TO 7-FOLD INCREASES IN BUY DECISIONS OVER
PRIOR EFFORTS, UP TO 3-FOLD INCREASES IN SKILLS SUBCONTRACTING FOR DESIGN EFFORTS.
Skills Staffing And Task Completion Inextricably Linked
Program Management Tools
• Type of Contract
• Length of Contract
• Technical Maturity
• Integration
• Constant Vigilance
–Accurate, rolling baseline
–Continuous, timely, accurate, execution data
How are DoD Contracts
Performing?
Notional data for
illustrative
purposes only
Program “X”
Enterprise Data Quality
Dimensions
EV / Funding Data Quality
Funding Data Quality: Acceptable for Critical Measurements and Decision-making
EVM Data Quality: Unacceptable for Critical Measurements and Decision-making
EVM Data Integrity
EVM-Related Issues/Challenges
•
EVM requirements incorrectly placed on
contract
•
Contract incentives that counter EVM’s
objectives
•
Deficiencies in contractor EVM systems
•
Lack of timely, realistic, and executable
performance measurement baselines
OSD and DCMA have identified several EVM implementation
issues on DoD contracts
EVM Requirements
•
Issue: EVM requirements incorrectly placed on contract
–
–
–
–
–
–
•
EVM not being used as management tool; perception EVM is just a financial report
Failure to include applicable EVM requirements
Inappropriately modifying EVM requirements; incorrectly tailoring DIDs for CPR and IMS
Specifying contract requirements not consistent with EVM policy and EVMS guidelines
Use of EVM not considered in contract type selection decisions
Disallowing use of contractor’s EVMS on contracts not requiring reporting
Action: Ensure EVM requirements are identified and incorporated into
solicitations and contracts, and are then properly executed
–
–
–
–
–
–
Work closely with program manager and EVM community throughout acquisition process
Involve EVM subject matter experts in establishing and implementing contract requirements
(statement of work (SOW), request for proposals (RFP), CDRLs)
Consider EVM implications when selecting contract type
Discuss requirements in pre- and post-award conferences or similar forums
Include EVM on agenda at contracting meetings and conferences
Ensure the “right” EVM training is provided to contracting community
It’s important to get the EVM requirements right up front –
fixing problems later is more painful … and costly
Contract Incentives
•
Issue: Contract incentives that counter EVM’s objectives
– Contracting policies can work at cross purpose with EVM
– Use of award fee incentives that drive undesired behavior – incentivizing
achievement of good numbers rather than good performance
– Result is PMs are denied objective performance information on their programs
•
Action: Find better ways to incentivize contractors (better use of fees)
– Avoid use of cost and schedule indices (CPI and SPI) as only EVM-related award
fee criteria
– Work with EVM community to devise objective, yet effective, contract incentives
– Consult EVMIG for ideas on appropriate EVM-related incentives (e.g., realistic
and current cost and schedule forecasts)
Paying contractors to maintain cost and schedule indices
within predetermined boundaries encourages undesirable behavior
Contractor EVM Systems
•
Issue: Deficiencies in contractor EVM systems
–
–
–
–
–
–
–
–
–
•
Lack of knowledge by control account managers
Span of control issues
Inadequate schedules
Data integration and integrity problems
Undisciplined work authorization and change control processes
Poor variance analyses
No formal corrective action processes
Inadequate estimate at completion procedures
EVM used to report data, not to manage
Action: Contact DCMA for help with contractor EVMS problems
– Work with local DCMA EVM specialist and/or the EVM Center
– Implement appropriate remedial actions for EVMS non-compliance
Based on findings from recent DCMA EVMS compliance reviews
Baseline Planning/Execution
•
Issue: Lack of timely, realistic, and executable Performance Measurement
Baseline (PMB)
–
–
–
–
•
Conduct of IBRs impacted by delays in defining work scope
Contractual direction to defer baseline planning until definitization
Incongruities between SOW and PMB (e.g., undefinitized contract actions)
Lag time in incorporating changes into baseline
Action: Know, understand, and implement ways to improve timeliness and
quality of baseline planning and execution
– Ensure IBRs are conducted within 180 days after contract award, and after
exercise of significant options or major modifications
– Stop direction to delay baseline planning until after contract definitization
– Incorporate authorized unpriced work into PMB – detail plan near-term effort
– Work with EVM community to ensure integrity of work scope and PMB –
achievable contract schedule and proposal estimate
Establishing an executable baseline is a critical
prerequisite to the successful implementation of EVM
Improving Program Performance
lessons learned
REQUIREMENTS DEFINITION
M/S A
DAB
JROC
JROC
ICD
ICD
++
CONOPS
CONOPS
Draft
Draft
ICD
ICD
Draft
Draft CDD
CDD ++
CONOPS
CONOPS
Rev
Rev 00
1
Draft
Draft CDD
CDD ++
CONOPS
CONOPS
Rev
Rev 1..n
1..n
Final
Final CDD
CDD ++
CONOPS
CONOPS
3
2
4
5
AoA
AoA
Preferred
Preferred
System
System
Concept
Concept
System
System
Definition
Definition
(SDS)
(SDS)
System
System
Spec
Spec
Technology Development &
System Requirements Development
Maturity Assessment
& Technology Demonstration
Concept Refinement
6
SSA
TMA
TMA
Assessment
Assessment
Production
Production
Spec
Spec
RFP
RFP
System Development and Demonstration
Models &/or
Prototypes
Defense
Acq
System
ASR
ITR
Phase 0
SRR I
Phase I
AoA: Analysis of Alternatives
CDD: Capability Development Document
FSA: Functional Solution Analysis
RFP: Request For Proposal
SDS: System Design Specification
SRR II
SFR
Test
Articles
PDR
CDR
Phase II
Phase III
KEY:
TMA: Technology Maturity Assessment
DAB: Defense Acquisition Board
JROC: Joint Requirements Oversight Council
ICD: Initial Capabilities Document
SSA: Source Selection Authority
Technical
Underpinnings
Cost Risk
Assessment
Schedule Risk
Assessment
Technical Risk
Assessment
Solid Understanding of Requirements
Realistic Baseline
PARTNERING WITH INDUSTRY
INFORMATION VISIBILITY
Boeing Common WBS Elements
LRE in $,000
$60,000
LRE Variance $
$50,000
$40,000
$30,000
UNFAVORABLE
$20,000
FAVORABLE
$10,000
$0
($10,000)
($20,000)
Collaborative
Process
Overhead
Management Reserve
General & Administrative
Flight Control
Field Services
Cost of Money
Product Assurance
Configuration Mgmt
Undistributed Budget
Furnishings / Equipment
Program / Project Management
Loading / Payloads / Equipment
Services
Air Vehicle
Technology
Survivability
Information Systems
Testing & Evaluation
Crew Station / Systems
Maintenance Instructions
Logistical Support Analysis
Design to Cost / Life Cycle Cost
Wing
Other
Supportability / Support Equipment
Training
Avionics
Airframe
ECP / LLP
In Service Engineering
Data / Software - Handling & Management
Armament
Propulsion
Landing Gear
Instructions / Publications
Electrical & Electrical Systems
Fuselage
Flight Test
Repairability
Environmental
Penetration Aids
Targeting & Sensors
Communication / Navigation
Supplier / Supply Chain Mgmt
Systems Engineering / Integration
($30,000)
Cockpit Mgmt, Controls, & Display
lessons learned
AoA
AoA
Plan
Plan
Hydraulics / Pneumatics / Mechanical
EXECUTION
PLANNING
FSA
Improvement Activity
“Lean Six Sigma”
Continuous Improvement
lessons learned
Burning
Platform
Cockpit
Chart
Leading
Indicators
Transform Data into Information
lessons learned
JCIDS
PROGRAM BASELINE
M/S B
DAB
M/S A’
Summary
• We must improve program execution
• Program Management is key to program
success
–
PMs must have mutual understanding of the task
• EVMS should provide an accurate picture of
program status relative to the baseline plan
– If data quality and integrity are suspect, then
“garbage in-garbage out” applies
• We must collaborate to improve EVM system
implementation and PM’s comprehensive
understanding/use of the data