Systems Engineering Revitalization
and
Sustainment
presented to:
2006 DoD Maintenance Symposium & Exhibition
25 Oct 06
Bob Skalamera
Deputy Director, Systems and Software Engineering
(Enterprise Development)
Office of the Deputy Under Secretary of Defense (A&T)
SE Revitalization:
Links to Sustainment
¾ Current SE Policy requires a SEP at MS C
¾ Defense Acquisition Guidebook – Chapter 4 on SE
• Section 4.3.5 – Operations and Support Phase
• Section 4.4 – Design Considerations
• Calls for a dedicated technical review (In-Service Review) in O&S
¾ SEP Prep Guide – technical planning focus areas include aspects
for post-MS C and with emphasis on Sustainment
¾ Integrated Defense Acquisition, Technology, & Logistics Life
Cycle Management Framework – “the Wall Chart”
¾ DAU’s Systems Planning, Research, Development, and Evaluation
(SPRDE) courseware refocused on life cycle (SYS-202 and SYS203)
¾ Program Support Reviews – ongoing, active logistics
participation
http://www.acq.osd.mil/se/
Sustainment in Design Reviews
Across the Life Cycle
Sustainment in the design and throughout program execution
Operations and Support Phase:
Links to Sustainment
Product Support/
PBL Management
Operations &
Sustainment
INPUTS
OUTPUTS
•Service Use Data
•User Feedback
•Failure Reports
•Discrepancy Reports
•SEP
•Input to CDD for next
increment
•Modifications/upgrades
to fielded systems
•SEP
In-Service
Review
Monitor and Collect
All Service
Use Data
Implement and
Field
Trades
Analyze
Analyze Data to
Determine
Root Cause
Assess Risk of
Improved System
Determine
System Risk/
Hazard Severity
Integrate & Test
Corrective Action
Develop
Corrective
Action
• Process Change – Hardware/Support
• Materiel Change
Program Support Reviews:
Representative Reliability Issues
¾ Reliability Growth Program
• Underestimating difficulty and resources to achieve and sustain reliability
growth
• Lack of proper planning, managing, and executing reliability growth
activities
• Program test design incompatible with reliability growth program aspects
• Reliability growth program not funded throughout
• Failure to consider correct use conditions/environment for reliability test
¾ Reliability Program
• In some programs, failure to design in reliability up front
• Inadequate quality and timeliness of root cause analysis and corrective
actions
¾ DT vs. OT
• OT evaluators using differing interpretations of failure or system use vs.
early buy-in and agreement on the artifacts that illuminate the
requirement (e.g., Operational Mode Summary/ Mission Profile and Failure
Definition and Scoring Criteria)
• Immaturity of scoring conference and process prior to use
The Sustainment Challenge
¾ Our joint challenge is to make Sustainment:
•
Part of the upfront planning and review process through participation
in Integrated Product Teams
•
Part of the event-based technical reviews
•
Part of design trades but not traded away during development
•
Part of the acquisition phase entrance and exit criteria
¾ Training and Guidance Resources
•
CLL 008 – Design for Supportability
•
CLE 003 – Technical Reviews
•
CLE 017 – Technical Planning
•
Guides: RAM, SEP, Risk Management, DAG Chapters 4 & 5
www.dau.mil/basedocs/trainingcourses.asp
Back-up
Approved Sustainment KPP
and Mandatory KSAs
¾ Single KPP:
•
Materiel Availability: measures percentage of the entire population capable of
performing an identified mission
Requires both system design and sustainment approach to be addressed:
Reliability, Maintainability, Service Life, Sustainment Strategy, Preventative
Maintenance, Diagnostics, Supply Chain, Distribution, Transportation
¾ Mandatory KSAs:
•
Materiel Reliability: measures confidence an operational, ready end item will
successfully complete its mission without a critical failure when tasked
•
Ownership Cost: measures what it costs to sustain a system after it is placed
in service
¾ Goals:
•
Correct number of operational end items capable of performing the mission
when needed
•
Confidence systems will perform the mission and return home safely without
failure
•
Cost balance: solutions cannot result in availability and reliability “at any cost”
Systems Engineering Plan Timing
Reflects Program
Acquisition Strategy
Capability Needs
Capabilities Sought
AS R
Pr
Mg g m
m S
Bu t A truc
s S pp tur
tra roa e
teg ch
y
Must be Consistent
SE
P
RFP
RFP
Source
Selection
W
S O es
/
O us ia
SO Cla iter
H Cr
SS
Shapes Contractor’s
Execution
Sets Contractor’s
Technical Approach
Technical Approach
Reflects Program
Technical Strategy
Update SEP to negotiated contractor’s technical approach
Design for supportability needs to be part of the
technical plan, upfront and early
SE Revitalization Overview
¾ Issued Department-wide Systems Engineering (SE) policy
¾ Issued guidance on SE, T&E, and SE Plans (SEPs)
¾ Instituted system-level assessments in support of DAB, OIPT,
DAES, and ad hoc reviews
¾ Established SE Forum to ensure senior-level focus within DoD
¾ Integrating DT&E with SE policy and assessment functions--focused
on effective, early engagement of both
¾ Instituting a renewed emphasis on modeling and simulation in
acquisition
¾ Working with Defense Acquisition University to revise curricula
(SPRDE, T&E, PQM, LOG, PM, ACQ, FM, CONT)
¾ Leveraging close working relationships with industry and academia
Reliability Trends
1985-1990
8000
met
1000
900
41% Met Requirements
800
700
59% Not Met
600
500
FOTE
400
OT II
300
IOTE
200
User Test
100
DT/OT
0
0
200
400
600
Requirement MTB_
800
1000
1200
1400
Reliability Trends
1996-2000
2500
1000
900
20% Met
Requirements
800
700
80% Not Met
600
500
FIELD
400
LUT
300
FOT
200
IOT
100
DT/OT
0
0
200
400
600
800
Requirement MTB_
1000
1200
1400
Model for Achieving RAM
¾ Understand and document the User needs and
constraints
¾ Design and redesign for RAM
¾ Produce reliable, maintainable products
¾ Monitor field performance
Operations and Support
Inputs
Outputs
Service Use Data
Data for In-Service
Review
User Feedback
Input to CDD for next
increment
Failure Reports
Modifications/upgrades to
fielded systems
Discrepancy Reports
SEP
SEP
Reliability Bright Spots
¾ Good RAM requirements
¾ Reliability Management Board, with representation from PMO, user
representative, operational tester, and prime contractor
¾ Unity of purpose regarding reliability between PMO and Prime Contractor
¾ Genuine pursuit of reliability by management of PMO, Prime and Subcontractors
¾ Using experienced reliability staff in PMO and Prime
¾ Well planned approach to RAM
¾ FMECA undertaken in design phase in a timely manner by design team to
identify and remove failure modes from design concepts
¾ Failure database for failures in DT and OT
¾ A well funded and supported reliability growth program with active
identification and removal of failure modes
¾ Use of advanced reliability growth models
¾ Dedicated reliability data collectors
¾ Provision of early operational testing
¾ Addressing false alarm rate with multifunctional team in DT
¾ DT and OT periods commensurate with RAM demonstration requirements
Program Support Reviews
Representative Reliability Issues
¾ Poor Requirements in ORD/CDD
• Arbitrary values for Reliability Availability
Maintainability (RAM) requirements
• In some programs, failure to identify mission context
or intended use profile, etc.
• Failure to identify when reliability values are required
(reliability and availability maturation points)
• Failure to model to ensure harmony between
reliability, availability, maintainability, and
supportability characteristics
• Failure to appreciate stochastic character of RAM and
hence suitably consider statistical confidence issues