Identifying Enterprise Leverage Points in Defense Acquisition Program Performance

advertisement
Identifying Enterprise Leverage Points in Defense
Acquisition Program Performance
Robb Wirthlin, Major, USAF
PhD Candidate
January 21, 2009
The views expressed in this work are those of the author and do not reflect the official policy or position
of the United States Air Force, the Department of Defense (DoD), or the U.S. Government.
Early Research Observations
•
Anomalous behaviors of acquisition are in-fact
emergent behaviors of a much larger, complex
system
• Prior studies suggest many of the ‘pathologies’ outside
of the control of the acquisition managers
•
These studies point out a need to understand the
entire system
• To cut any piece out (like acquisition) and study it
•
exclusively would preclude finding a full set of root
causes
Therefore, I began to look “outside” of acquisition
http://lean.mit.edu
© 2009 Massachusetts Institute of Technology LAI Research Summit 01/21/09- 2
Challenge: Find an appropriate
representation for the Enterprise
•
Here’s what I did
• Borrowed ideas and concepts from Value Stream
•
•
•
diagramming
Used commonly accepted understandings of
probabilities and occurrence
Represented activity in frame of reference
understandable to target audience: military and civil
servants in AF (used DAU “Wall Chart” organizing
framework)
Scoped it to a reasonable abstraction and size
•
•
•
http://lean.mit.edu
Main unit of measurement is a “program”
Restricted to ACAT I, II, and III programs
Limited to Milestone C and earlier in Acquisition parlance
© 2009 Massachusetts Institute of Technology LAI Research Summit 01/21/09- 3
Top Level Abstraction
A Representation of the Enterprise of “Cradle to Grave”
Acquisition in the US Air Force
Swim Lane
Pre-MS
“A”
Pre-MS
“B”
(Concept
Refinement)
(Technology
Development)
Pre-MS “C”
(System
Development &
Demonstration)
Pre-Full Rate
Production
(Production &
Deployment)
Operations
and
Sustainment
User
Requirements
Money
Acquisition
Scope of Model
Contractor
Time
http://lean.mit.edu
© 2009 Massachusetts Institute of Technology LAI Research Summit 01/21/09- 4
The Enterprise Model
Pre-MS A
Pre-MS B
Pre-MS C
Requirements Swimlane
Requirements Swimlane
Requirements Swimlane
W ait fort echnical results
W ait f or tec hnical r esult s
Put n
i to moderniz at o
i n pla nning
and send to Ac quisition
MDAP
Thres hold
crossed?
t o A cquis ti ion Moder nization/Sus tain ment Ac tivity
(n
i dividual or g)
Yes
Pr epar e Concept of Oper at o
i n
Capability Roadmap "wis h list " for budget s ubmission
Thr eshold Object vi e Tradeof f s
CCD
No
Y
Draft Br iefing and materials
D ecision t o pur sue r equir ements
"Coor d" and "Concur "
(n
i dividual or ganiz at o
i ns)
(n
i dividual or g)
MAJCOM "A" Lett er s
Update and schedule calendar
Y
Pre- RSR
( in dividual or g)
Y
Fn
i aliz e RSR & Calendar it ems
RSR
( Individual or g)
Mr. Harr y Disbr ow
H Q USAF A8
MAJCOM A8
Cont n
i ue
Y
I n s cope of exist ing doc uments?
Form High Per formance Team (HPT)
HQ USAF A8 or g
(indiv d
i ual org)
D raf t
C CD
Rout e t o pr oper organization
Ext er nal input
KPP Development
Decision to purs ue requir ement s
(individual or g)
( Requir ements Policy Shop)
Draf t Br e
i fing and materials
"C oor d" and "Concur "
(individual or ganiz ations)
Y
MAJCOM "A" Lett er s
Update and schedule cale ndar
Pr e- RSR
(individual or g)
Finaliz e RS R & Calendar items
RSR
( Individual or g)
Mr. Harr y Dis br ow
H Q USAF A8
N
Ot her n
i put s
Decision t o pur sue r equir ements
( in dividual or g)
Proces s St ar t
Y
MAJCOM A8
Requirement s Pr oc ess
Input s fr om v arious sources
For exist ing progr am?
(Requir ement s Policy shop)
No
Draft Br iefing and materials
"Coord" and "Conc ur"
(n
i dividual or ganiz at o
i ns)
MAJ COM "A" Let ters
Y
Updat e and sc hedule c alendar
Pre-RSR
( individual org)
MAJCOM A8
Y
Finaliz e RSR & Calendar it ems
RSR
(I ndividual org)
Mr. Harr y Disbr ow
HQ USAF A8
Fundin g available?
Y
Conduct St udy or Analysis
(individual or g)
( in dividual or g)
Decision t o pur sue r equir ement s
Dr af t Br e
i fing and mater ials
(n
i dividual or g)
(indiv d
i ual or ganizations)
"Coord" and "C oncur"
MAJCOM "A" Lett er s
Updat e and schedule c alendar
Pre- RSR
( in dividual or g)
MAJCOM A8
Rout e to Advanc ed concepts
(Requirement s Polic y Shop)
No
N
N
N
AFROC pr eparations
AFR OC decision?
MAJCOM or g
AFR OC
For m High Perf ormance Team ( HPT)
HQ USAF A8 or g
Update br ief and mater ials
( Individual or g)
Y
Hg
i h P erf or mance Team ( MPT) wor k
Dr af t Document ( ICD)
H Q USAF A8 org
Air Staff Pr oc ess
Crit ical Comment s?
HQ AF A5R D
HQ AF A5R D
Finalize RSR & Calendar items
RSR
( Individual or g)
Mr. Harr y Disbr ow
HQ USAF A8
N
No
( in dividual or g)
Joint inter es t?
AFROC
OSD
N
N
Air Staff Pr ocess
High Perf or mance Team ( MPT) wor k
Pr epare f or Milest one Rev iew
I nf o t o Acquis ition about
CONOPS
capabilit y "wish lis ts "
Yes
Int er act ion with Acquisition
Update brief and materials
More Detailed r esult s
fr om Prot ot ypes/ Engineering Models
Acquisition Res ults
( I ndividual or g)
Y
Dr af t Document ( ICD or draf t CDD)
Cr ti ical C omment s?
AFROC prepar at o
i ns
AFROC decision?
HQ A F A5RD
MAJ COM or g
AFROC
Prepar e f or Milest one Review
Yes
CPD?
J oint Pr oces s
J8
Y
C ritical Comments ?
J8
Ar chive
N
Individual or g
N
MAJCOM org
N
J8
Dec ide to pursue
OSD
Comment Res olution
MAJ COM approval?
N
Cr itical Comment s?
Inf or mation t o Acquis ition Sys tem
for prepat or y w ork
Y
Join t interest?
N
Y
No
Inf or mation t o Acquis ition Sys tem
for prepat or y w ork
Ar chiv e
J8
Per sonnel fr om A cquis ti ion of fice
wor king on t his par ticular ac tivity
Y
Indiv idual or g
Join t Process
N
D ecide t o pur sue
I ndividual org
HQ AF A5RD
Draft Document ( CDD)
Decide t o pur sue
Y
CDD?
MAJCOM org
MA JCOM appr oval?
Y
N
No
No
N
HQ USAF A8 org
Y
Comment Resolu tion
Dr af t Doc ument ( CDD)
HQ USAF A8 org
( Individual or g)
HQ USAF A8 or g
( Individual or g)
N
J8
JROC Prepar ations
JROC decision?
MAJCOM org
JROC
Y
Details of Pr ef er red
Syst em Concept ( Preliminar y)
JROC Pr epar at ions
N
MAJ COM or g
Yes
J8
Y
AFROC decision?
MAJC OM or g
High Perf ormance Team ( MPT ) wor k
Update br ief and materials
High Per formance Team (MPT) work
Joint Proc ess
Cr ti ical Comment s?
Comment Resolu tion
JROC dec si o
i n?
Y
JROC
N
I CD?
MAJCOM org
I nf or mation t o Acquis ti ion Syst em
f or prepat ory work
J ROC Pr eparat ions
N
AFROC prepar ations
HQ AF A5RD
N
No
Mor e Acquis ti ion Res ults
No
Y
Y
MAJCOM appr ov al?
Crit ci al Comments ?
HQ AF A5RD
No
F orm High Per for manc e Team (HPT)
HQ USAF A8 org
Y
N
N
No
N
Air S taf f Pr oces s
Y
Update br ief and mater ials
I ndividual org
A rchiv e
N
No, because of work alr eady done
conduc t AoA?
Preliminar y Results fr om Acquis it ion
OSD
N
Obt ain funding f rom
Adv anced Conc ept s Branch
Decide to purs ue
Y
Joint int er est ?
No
Y
Form High Per for manc e Team (HPT)
HQ USAF A8 org
Y
N
MAJ COM or g
JROC dec si o
i n?
Y
No
N
Ar chive
N
JROC
N
Budgeting and Programming Swimlane
Budgeting and Programming Swimlane
Budgeting and Programming Swimlane
Go to "Prepar e
Courses of Act ion"
( Acquis ti ion Step)
Repeat n+2 t m
i es
Repeat n+2 times
AF lev el pr oc ess
MAJCOM PEM
Decision to
pur sue
Adv anc ed Conc ept s Budget Reques t
Nex t st ep?
MAJCOM Panel
MA JCOM Gr oup
Next St ep?
MAJCOM Council
Next St ep?
Y
Pr ogr ammin g Iss ues &
Budget ing Hear ings
PBD
MAJ COM PEM
D ecision t o
pursue
Advanced Concepts Budget Request
Y or N
Y or N
OMB & Congr ess
N
law
Yes
changes t o
BES?
MAJCOM P EM
Decis io n t o
pur sue
Adv anced Conc ept s Budget Reques t
BES
DoD POM?
Y
Next s tep?
MAJ COM Panel
Y or N
Go t o "Pr epare
Cour ses of Act o
i n"
(Ac quisit o
i n Step)
Go t o "Pr epar e
Cour ses of Ac tion"
( Acquisition St ep)
Repeat n+2 times
Yes
Yes - inf or mation only
Y
PDM?
Next St ep?
MAJCOM Group
Next St ep?
MA JCOM Council
Nex t Step?
AF lev el process
Next St ep?
OSD Pr ocess
Y
Next st ep?
MAJC OM Panel
Y es
Yes - infor mat o
i n only
Y
PDM?
MAJCOM Gr oup
Nex t Step?
MAJC OM Council
Next St ep?
AF lev el proc ess
Next St ep?
OSD Pr oces s
Congr es s
a
l w
N
Yes
Yes
PBD
Yes - inf ormation only
Y or N
Y or N
Y or N
Y or N
c hanges t o
BES?
Congres s
N
law
Yes
changes t o
BES?
BES
BES
DoD POM?
Y
PDM?
PBD
OSD
DoD POM?
Y or N
OMB
Y or N
OMB
Yes
Retr y?
R etr y?
Ret ry ?
No
Ar chive
Ar chiv e
Archive
R eleas e moneys
to Acquisition
To Budget inputs
Release money s
to Ac quisit o
i n
Release money s
to Acquisition
To Budget inputs
To Budget inputs
To A cquisit ion
Fallo ut of B udget Proces s
m
i pact n
i g planning
To Acquis ti ion
Fallout of Budget Pr ocess
m
i pact ing plannin g
To Acquisition
Fallout of Budget Pr oc ess
impac ting planning
Full Funding in FYDP
Full Funding in FYDP
KPPs f rom
Requirement s
Acquisition
Pr eparation f or Acquisition Panels
Acquisition
Acquisition Panels
Yes
MDA A ppr oval?
Milest one A
Ac quisit o
i n Panel (DAB , MD A, ADM, et c)
Af fordabilit y Assessment
Pr eparat ion for Acquisition Panels
Ac quisit o
i n Program Bas elin e
Acquisition
Acquisition Panels
Pr eparation f or Acquisition Panels
Ac quisit o
i n Panels
Yes
MDA Appr oval?
KPPs f rom
Requirement s
Milest one C
Program Of fice Cos t Est m
i ate
Cont ract or Cos t Est m
i at e
No
Af fordability Ass essment
Independent Cost Est imate
Pr epare f or Acquisition Panels
No
Acquis ti ion Pr ogr am Baseline
Y es
MDA Approval?
N
Contr ac t Management
F rom Requir ements sy st em:
Dr aft document f or basis of wor k
RFP r elease and Sourc e Select ion
From Requirement s
No
Dr af t RFP Prepar ation
Y
Prot es t?
To MAJCOM PEM
Budget input
Yes
Funds
Redir ect ed
RFP r elease and Sourc e S elect ion
Dr aft RF P Pr eparation
Scope and Awar d Technology D evelopment c ont racts
I nt er nally or ex ter nally gener ated
mult iple times per y ear
Periodic Rev iews
RFP c oor dination pr ocess
Inter nally or externally gener at ed
multiple times per year
Pr epar e Progr am budget dec si ion (PBD)
Upheld?
To MA JCOM PEM
Budget input
Yes
Delay
No
Ar e Obs &
Exp OK?
No
Yes
Upheld?
To MAJC OM PEM
B udget input
Yes
Repeat "n" years
No
Scope and Award Sys tem Development and Demonst ration contr ac ts (or AoA)
Yes
RFP coor dination pr oc ess
Repeat "n" y ear s
Yes
Are Obs &
Exp OK?
No
A cquis ti ion Planning Ac tivit e
i s
No
Pr otest ?
Contr act Management
No
RFP r eleas e and Sourc e Select o
i n
Repeat "n" years
From Planning & Budgeting
C ont ract Management
RFP coordination proc ess
Int er nally or ex ternally gener at ed
mult iple times per year
Scope and Award St udy contr ac ts ( or AoA)
Delay
Yes
Funds
Redirec ted
No
Per o
i dic Reviews
Des ign Rediness
Review?/ MDA
Appr oval
Yes
YES
No
scope growth or
tec hnical pr oblems
Seek funds?
PEM/other st aff
find moneys
Yes
No
Obt ain
Funds?
( timely)
No
Ar e Obs &
Exp OK?
No
No
Sourc e Select io n Pla ns
Yes
Funds
Redir ect ed
No
Per iodic Rev e
i ws
Yes
Yes
To Requirement s
sc ope gr owt h or
t echnic al problems
Prepar e C our ses
of Ac tion
Prefer red
Sys tem
Concept
Funding
Shor t fall
Cont ract Change
Requir ed?
s cope growth or
t echnical pr oblems
Seek f unds?
PE M/ ot her s taff
f ind moneys
Yes
No
Obtain
Funds?
( timely)
No
Seek funds ?
Yes
PEM/other st af f
fn
i d money s
No
Obtain
Funds?
(t m
i ely )
No
Yes
Prepar e Cour ses
of Act o
i n
Source Sele ct o
i nP a
l ns
Y es
Yes
Funding
Short fall
P repar e Cour ses
of Ac tion
Sys tem Per for manc e
Specif ci ation
Contr act Change
Required?
To R equir ements
Init ial Rate pr oduct o
i n
Baseline
Yes
Ac quisit o
i n Panels
Fundin g
Shor tf all
Cont rac t Change
R equir ed?
End loop
Yes
Adjus t Program and Cont ract
(c ost /s chedule est imates rev si ed) .. .
End loop
P repar ation f or Ac quisit o
i n Panels
E nd o
l op
To Requir ement s
Yes
Change Cont rac t/ Resc ope
ef for t ( cost / schedule impact )
Bes t Mat erial
Appr oac hes
Yes
Change Cont ract /Res cope
eff ort ( cost/ sc hedule impact)
Dif fer ent Distr ibut ion pr ofile
and e
l ngt h dependin g upon
path t aken
SRR?
Adjust P rogram and Cont rac t
(cost /sc hedule est m
i ates r evis ed) .. .
Change Contr ac t/ Resc ope
eff or t (c ost /s chedule impact )
Pr eliminary
Integrated
Ar chetic tur e
No
No
Diff er ent Dis tr b
i ution profile
and lengt h depending upon
path taken
To Requir ements
Engin eer ing
Reviews?
Sys tems Engineer ing
To Ev ent Happens st ep
To Sy stems E ngineering s tep
Rewor k and Dela y
Yes
Dif fer ent Dis tr ibut o
i n profile
and e
l ngt h depending upon
path t aken
Acquisition Planning Act vi ti ies
Syst em
Perf ormance
Specific ation
A djust Pr ogr am and Contr act
( cost/ sc hedule est m
i at es r evised). . .
No
To Event Happens s tep
Sys tems Engineer ing
Dat a f rom
User St udie s
Test Readeiness
Review
Yes
Yes
Make Trades ?
Pr ef er red
Sys tem
Concept
No
No
Sys tem
Ver ification
Review?
To Requirements
Acquis ition Plannin g Act vi ti ies
Sy st ems Engineering
P rot ot ypes/
Engineer n
i g
Models
Syst ems Engin eer ing
To Ev ent Happens st ep
To Requirement s
To Event Happens
To Event Happens
To Sys tems Engin eer ing
To S yst ems Engineer n
i g St ep
To Event Happens
To Sys tems Engineer ing
To Event Happens
To Syst ems Engin eer ing
Pr ot ot ypes/
Engineering
Models
Contractors
Contractors
Perf or m wor k
Cont r act St ar t- up
Event Happens
Contr ac t Start -up
Sc ope Growth/
Technic al Pr oblems ?
Rew ork and Delay
Perf or m wor k
To Ev ent Happens st ep
Rework and Delay
Rewor k and Dela y
Rework and Dela y
Perf or m wor k
Cont ract St art - up
Event Happens
Contractors
Sc ope Growth/
Technic al Pr oblems ?
YES
YES
YES and NO
NO
Fabricat ion
No
T ask
Complete?
Assembly
I ntegrated t es ting
Development al t est ing (S yst em), Liv e Fir e Test , Operat ional Ass essment
Combined Test ing (DT&E, OT& E, LFT&E)
To "Prepar e Cour ses of Action"
in Acquisition Swimlane
No
Task
Comple te?
To "Prepar e Cour ses of Act o
i n"
in Acquisition Swimlane
Ev ent Happens
Scope Growth/
Tec hnical Problems?
Yes
To Pr eferr ed Sys tem Concept
YES
Yes
Yes of NO
To SRR
No
Task
Complet e?
To "P repar e Cour ses of Ac tion"
in Acquis ti ion Swimla ne
Yes
To Prototype Engineering Models
Follow pat h at appr opr a
i te st age
Output consists of time elapsed and actual cost for single program
- and tracks comparison to proposed time required and original proposed costs
http://lean.mit.edu
Sour ce Select ion Plans
Draft RFP Pr eparat ion
Milest one B
Approval?
© 2009 Massachusetts Institute of Technology LAI Research Summit 01/21/09- 5
Close-up view
RSR – Decision Point
-Sources: Official Docs,
Interviews (Krussel,
Moen)
-Probability: 98%
Funding Available? –
Decision point
-Sources: Interviews
(Krussel, Schilling,
Duda )
-Probability: 80%
http://lean.mit.edu
Model Design: Every
decision point, every
process task, is
documented and sourced;
now validated
RSR
Mr. Harry Disbrow
HQ USAF A8
N
Funding available?
(individual org)
Y
Conduct study or analysis
– Task
-Sources: Official docs,
Interviews (Krussel, Duda)
-- Time Distribution: 45 to
180 days; binomial, p=0.40
Conduct Study or Analysis
(individual org)
N
© 2009 Massachusetts Institute of Technology LAI Research Summit 01/21/09- 6
Discussion and Policy Implications
•
A Few examples
• PDR (and new DoD 5000.2 response)
• Rework modeling (explicit vs. implicit)
• Variance at local levels vs. Enterprise variance
• Undocumented handoffs, waiting periods
•
Policy Implications
• Consider ^ overlay of staffing profiles to Enterprise
•
•
model behavior
Consider ^ error bars of program vs. point estimates
^ Oversight functions, managing risk, value-added govt
workforce activities
http://lean.mit.edu
© 2009 Massachusetts Institute of Technology LAI Research Summit 01/21/09- 7
Future Work
•
Candidate issues to pursue:
• How much time is spent “waiting” in the system?
• What is the cost of waiting?
• How can the ratio of waiting to non-waiting taks be
improved?
•
Which tasks do the data suggest have a more influential
role in taking time (or cost) in the overall process?
•
•
What is the overall impact of a process’ time to Enterprise
outcomes?
Where can an Enterprise leader suggest “investments”
into the process with the greatest impact to cycle time or
cost reduction?
http://lean.mit.edu
© 2009 Massachusetts Institute of Technology LAI Research Summit 01/21/09- 8
Summary
•
•
•
Introduction to problem and motivation
Model development
Discussion and Implications
http://lean.mit.edu
© 2009 Massachusetts Institute of Technology LAI Research Summit 01/21/09- 9
Download