Dayton Aerospace
Why Revitalize
Systems Engineering?
or
How We Got To
This Point Today
July 21, 2004
Tom Hoog
Dayton Aerospace, Inc.
Ideas & Products Enabling Your Success!
© 2004 Dayton Aerospace, Inc.
1
Dayton Aerospace
Systems Engineering Eulogy
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
SE born in the 1930s as one of RCA Broadcasting Services & Standards
Military development of Ops Research in the 1940s was instrumental in shaping its youth
First formal education attempted at MIT in 1950
SE became the darling of NASA & military systems in the 1960s; processes imposed on
prime contractors
Its many offspring were Chief Systems Engineers, Directors of Engineering, VPs of
Systems Engineering as well as many Systems Engineers
It became bloated & unwieldy in late 1960s; became known as AFSCM 375-5
Attended fat farm & reappeared as Mil-Std-499A; grew in stature & acceptance
SE became check & balance in massive engineering efforts
It suffered an identity crisis when universities taught industrial engineering as systems
engineering
In the 1980s SE attempted to re-identify itself as Mil-Std-499B
Later cashiered out of the military, it found civilian employment as an IEEE standard
It wasted away with an illness brought on by rejection of its processes & bastardizing of its
techniques & methods
Design reviews became re-design meetings
SE died a lingering death in the 1990s & was replace by IPTs, Design Teams & Group
Gropes
SE is survived by & remembered by a cult of old systems engineers who still practice their
now meaningless rituals
SE was almost 70 years old at the time of its death
No services are planned
Full text of this eulogy can be found at http://acc.dau.mil/simplify/ev.php?ID=6174_201&ID2=DO_TOPIC
Ideas & Products Enabling Your Success!
2
Dayton Aerospace
Outline
Background
• DoD Transformation
– 5000
– JCIDS
– Scope of SE
• Systems Engineering Revitalization Initiatives
– USD (AT&L) Imperatives
– SAF/AQ Initiatives
– USD (AT&L) Initiatives
• Health of SE – DAI’s View (Jul 03)
Ideas & Products Enabling Your Success!
3
Dayton Aerospace
Outline
Background
• DoD Transformation
– 5000
– JCIDS
– Scope of SE
• Systems Engineering Revitalization Initiatives
– USD (AT&L) Imperatives
– SAF/AQ Initiatives
– USD (AT&L) Initiatives
• Health of SE – DAI’s View (Jul 03)
Ideas & Products Enabling Your Success!
4
Dayton Aerospace
Background
DoD Decision Support Systems
Requirements
Generation
System
Defense
Acquisition
System
• Three parts to DoD Decision
Support Systems
• Requirements Generation
– Governed by CJCSI 3170.01
– reissued 24 June 2003
• Defense Acquisition
Financial
Management
System
– Governed by 5000 series
– Reissued 12 May 2003
• Financial Management
– Governed by multiple sources
– Being revised
Ideas & Products Enabling Your Success!
5
Dayton Aerospace
Background
Strategic Policy
Guidance
JCIDS Analysis
Joint Operations Concepts
Joint Operating Concepts
Joint Functional Concepts
Integrated Architectures
Functional Area Analysis
CPD
Functional
Needs
Analysis
DOTMLPF
Analysis
CDD
Materiel Changes
CJCSI 3170
process
Ideas for
Materiel
Approaches
Analysis
of Materiel
Approaches
Alternative N
Alternative 2
Alternative 1
DOTMLPF Changes
CJCSI 3180
Process
Post
Independent
Analysis
ICD
DOTMLPF Change
Recommendation
Functional Solution Analysis
Ideas & Products Enabling Your Success!
6
Dayton Aerospace
THE “New” 5000 MODEL
Process entry at Milestones A, B, or C
Start into Increments where appropriate
Increments are interrelated
Continues for development life of system
User Needs &
Technology Opportunities
Pre-Systems
Acquisition
Systems Acquisition
Sustainment
(Program
B Initiation)
A
Concept
Refinement
C
Technology System Development
Development & Demonstration
Production &
Deployment
Design
System
System
Readiness
Integration Review Demonstration
Concept
Decision
MATURE
B
ADAPT
DEMO
MATURE
DEV
DEMO
DEV
FOC
IOC
FRP
Decision
Review
LRIP/OT&E
Operations &
Support
C
INTEGRATE
TEST AND FIELD
B
ADAPT
C
INTEGRATE
DEMO
B
ADAPT
TEST AND FIELD
C
INTEGRATE
TEST AND FIELD
MATURE
Spiral Development - Insert technology into program when it is mature
Ideas & Products Enabling Your Success!
Adapted From DoDI 5000.2, 12 May 03
7
Dayton Aerospace
SE in Evolutionary Acquisition Environment
• Multiple concurrent increments lead to
increased need for disciplined SE process (vs.
single step process)
• Continuous requirements definition &
technology maturation tasks
– Complete definition of requirements
• Plan to mature requirements
• Traceable from top level operational requirements to design
implementation
• Inclusion of joint requirements (FoS, SoS, net–centric)
• Allocation of requirements to increments
– Mature producibility, affordability, supportability, of
technology as well as performance
• Need to cover both government & industry
aspects
Ideas & Products Enabling Your Success!
© 2004 Dayton Aerospace, Inc.
8
Dayton Aerospace
5000 Series Extracts
• DoDD 5000.1
– Para E 1. 27 Acquisition programs managed through
application of SE approach that optimizes total system
performance & minimizes total ownership cost. A modular open
systems approach shall be used
• DoDI 5000.2
– Para 3.9.2.2 Sustainment uses SE methodology
– Para E 7.2 Human factors is part of SE
– Para 7.7 Include ESH in SE
• DoD 5000.2-R (Interim Defense Acquisition Guidebook)
Para C2.3.2 Acquisition strategy includes SE
Para C2.8.2 Support strategy is part of SE
Para C2.8.5.5 Human factors is part of SE
Para C2.8.6 ESH is part of SE
Para C6.7.2.7 Critical Program Information is part of SE
Para C5.2 18 pages on SE
Ideas & Products Enabling Your Success!
9
Dayton Aerospace
DoD 5000.2-R, Para C5.2
• Defines what SE does
• Defines the SE activities
•
• COTS
• Reliability, availability & maintainability
• Human systems integration
– Requirements analysis
– Functional analysis/allocation • Environment, Safety & operational health
– Design synthesis & verification • Interoperability
• Survivability
– System analysis & control
• Mission assuredness
– Manufacturing & production
• Information assurance
– Modeling & simulation
• Anti-tamper provisions
– Quality
•Other Design considerations
– Acquisition logistics
•WBS
– Open systems design
•Performance Specifications
– Software management
•Metric system
Other Design Considerations
•Insensitive munitions
– WBS
•Value engineering
– Performance specifications
•Precise time & time interval
•Accessibility
– Metrics
•Corrosion prevention & control
– Insensitive munitions
Ideas & Products Enabling Your Success!
10
Dayton Aerospace
Scope of Systems Engineering
Overarching Policy
NSS/NMS/Joint vision
Joint Concept of
Operations
Functional
Area
Analysis
Functional Area
Functional Concept
Integrated Architecture
Feedback
D
O
T
M
L
P
F
MS B
- Materiel Process
Analysis of
Materiel
Approaches
MS C
CDD
MS A
DOTLPF
Process
JROC DAB
CPD
AoA
ICD
Technology
Development
Concept
CD
Refinement
JROC
DAB
JROC DAB
Increment 1
Demo
MS B
Acquisition community participation
in early requirements process to
assist in understanding
Demo
MS C
Increment 2
MS B
Demo
MS C
Increment 3
Acquisition
Requirements
Systems Engineering
Ideas & Products Enabling Your Success!
Adapted from DoDI 5000.2 12 May 2003
11
Dayton Aerospace
SE Definitions
• Systems Engineering (SE) A comprehensive, iterative
Technical Management (TM) process that includes
translating operational requirements into configured
systems, integrating the technical inputs of the entire
design team, managing interfaces, characterizing and
managing technical risk, transitioning technology from
the technology base into program specific efforts, and
verifying that designs meet operational needs. It is a life
cycle activity that demands a concurrent approach to
both product and process development. (11th Glossary
of DoD Acqu Terms)
• Systems Engineering - An inter-disciplinary approach
and means to enable the realization of successful
systems. (EIA 731.1)
Ideas & Products Enabling Your Success!
12
Dayton Aerospace
Some Systems Engineering Characteristics
• Encompasses government and contractor efforts
– Either may lead
– Some joint
• Iterative process at multiple levels
– System level
– Subsystem/component level
• Specific tasks depend on program phase
– Pre-Acquisition, Acquisition, Sustainment
• Specific focus of tasks/trades depends on type of
program
–
–
–
–
New, modification
Prime or support equipment; system or subsystem
Aircraft, space, electronics, armament
Complete system, subsystem, System of Systems / Family of
Systems
Ideas & Products Enabling Your Success!
© 2004 Dayton Aerospace, Inc.
13
Dayton Aerospace
System Engineering Focus Shifts over
Program Life Cycle
• Pre-Acquisition
– Operational & system performance requirements
– Life cycle affordability requirements
• Acquisition
– Design requirements & implementation
– Manufacturability, supportability, upgradeability, etc.
– System integrity,
• Sustainment
– Supportability
– System integrity
Ideas & Products Enabling Your Success!
Modification/Upgrades:
Repeat the Cycle
© 2004 Dayton Aerospace, Inc.
14
Dayton Aerospace
Requirements Analysis
Input
Customer needs/objectives/
requirements
Analyze missions & environments
System Analysis
& Control
Identify functional requirements
Define/refine performance & design
constraint requirements
Technology base
Outputs from prior phase
Trade-off studies
Program decision
requirements
Effectiveness analyses
Requirements Loop
Requirements applied through
specifications & standards
Risk management
Configuration
management
Interface Definition
Functional Analysis/Allocation
Decompose to lower level functions
Allocate performance & other limiting requirements
to all functional levels
Performance based
progress measurements
IMP
IMS
Technical reviews
Define/refine functional interfaces
Define/refine/integrate functional architecture
Design Loop
Verification
Synthesis
Transform architectures
Define alternatives
system concepts,
configuration items &
system elements
Define/refine physical
interfaces
Ideas & Products Enabling Your Success!
Adapted from Mil-Std-499B (Draft)
Select preferred product
& process solutions
Output
Phase dependent
Decision support
data
System
architecture
Specifications &
15
baselines
Dayton Aerospace
Ideas & Products Enabling Your Success!
16
Dayton Aerospace
Outline
Background
• DoD Transformation
– 5000
– JCIDS
– Scope of SE
• Systems Engineering Revitalization
Initiatives
– USD (AT&L) Imperatives
– SAF/AQ Initiatives
– USD (AT&L) Initiatives
• Health of SE – DAI’s View (Jul 03)
Ideas & Products Enabling Your Success!
17
Dayton Aerospace
USD (AT&L) Imperatives
• “Provide a context within which I can make
decisions about individual programs.”
• “Achieve credibility and effectivness in the
acquisition and logistics support processes.”
• “Help drive good systems
engineering practice back
into the way we do business.”
From Feb 04 OSD (AT&L) - NDIA SE Division Meeting
Ideas & Products Enabling Your Success!
18
Dayton Aerospace
SAF/AQ SE Revitalization
• SAF/AQ Dr. Sambur Memo, Apr 9, 2003
– Focus attention on application of SE & elevate these disciplines
commensurate with cost & schedule
– Develop SE performance incentives
– Include status of SE in future program reviews
– Identify key SE processes & practices in acquisition
documentation
– Acquisition Strategy Panel briefings must address significant SE
areas
• SAF/AQ Memo, Increment 2, Jan 7, 2004
– Intended to institutionalize key attributes of an acceptable SE
approach
– Insert the appropriate language into the following acquisition
documents
• Solicitations
• Award Fee Plan/Incentive Fee Contract
• Contracts
Ideas & Products Enabling Your Success!
19
Dayton Aerospace
USD (AT&L) Initiatives
• AT&L SE Policy Memo, Feb 20, 2004
–
–
–
–
Effective immediately; included in next DoD 5000 revision
All programs, all ACATs shall apply a robust SE approach
SEP for MDA approval
Drive good SE back into the way we do business
• AT&L SEP Guidance Memo, Mar 30, 2004
– Purpose of SEP is to lay out a plan to guide the technical
aspects of an acquisition program
– Living document, tailored to the program; roadmap to support
program management
– No prescribed format; address integration of technical aspects
to include:
• The systems engineering processes to be applied in the program
• The systems technical baseline approach
• Event-driven timing, conduct, success criteria, and expected
products of technical reviews
• The integration of systems engineering into the program’s IPTs
Ideas & Products Enabling Your Success!
20
Dayton Aerospace
Outline
Background
• DoD Transformation
– 5000
– JCIDS
– Scope of SE
• Systems Engineering Revitalization
Initiatives
– USD (AT&L) Imperatives
– SAF/AQ Initiatives
– USD (AT&L) Initiatives
• Health of SE
– Briefing to AT&L (SE OPR)
– Quick Study
Ideas & Products Enabling Your Success!
21
Dayton Aerospace
Health of SE Tasks
• DoD 5000 & JCIDS reissued in May & June 03
• Briefing in Jul 03
• DAI personnel experiences
–
–
–
–
–
Lack of execution discipline on programs
18 problem areas cited
Common Denominators
List of Issues
Candidate emphasis areas listed
• DAI asked to provide supporting data
• Quick Study Report/Briefing in Sep 03
Ideas & Products Enabling Your Success!
22
Dayton Aerospace
Common Denominators
• Program Teams are incentivized by cost and/or
schedule - not execution of disciplined SE
practices
• Products and Processes are getting out of
balance
• Inadequate depth in applied SE capability
Ideas & Products Enabling Your Success!
© 2004 Dayton Aerospace, Inc.
23
Dayton Aerospace
Issues
Lack of SE Execution Incentives
• Contracts reward cost and schedule
performance
Product and Process Imbalance
• Emphasis on speedy delivery of product can
De-emphasize “processes”
• Misapplication of spiral / evolutionary
development intent is used to justify program
shortcomings – “fix it in the next spiral”
• Lack of true multifunctional activities in IPTs
Ideas & Products Enabling Your Success!
© 2004 Dayton Aerospace, Inc.
24
Dayton Aerospace
Issues (cont)
Inadequate Depth in Applied SE Capability
• Sound SE execution requires many trade
studies
• Government and industry “de facto” rely on their
personnel to pass along “lessons learned”
• Government and their contractors are losing
“critical mass” in some areas
• Systems Engineering processes are adequately
defined BUT not followed, not understood, or
are short circuited
• Advertised CMMI ratings do not assure effective
program implementation
Ideas & Products Enabling Your Success!
© 2004 Dayton Aerospace, Inc.
25
Dayton Aerospace
Summary of Observed Problems
• Observed Problems - Approach
• Three Categories of Problems
– Requirements Management (8)
– Systems Engineering Process (7)
– Engineering Management (6)
Ideas & Products Enabling Your Success!
26
Dayton Aerospace
Observed Problems - Approach
• Based on analysis of many programs
• Each problem area included
– Observed Problem/Issue
• Statement of problem or issue
• Program examples
– Impact
• What happened because of problem
– Recommendations
• What can be done to avoid the problem
• Focus was on systems engineering - recognized that
systems engineering execution is not independent of
overall program management
• Objective was to improve overall program performance
Ideas & Products Enabling Your Success!
© 2004 Dayton Aerospace, Inc.
27
Dayton Aerospace
Observed Problems
Requirements Management
•
•
•
•
•
•
•
Incomplete Definition of Requirements
Requirements Creep
Requirements Volatility
Lack of a Specification Tree
Incomplete/Weak Verification requirements
Late requirements Forced Into Software
Interface requirements and Interface
Management
• Failure to Implement and Maintain
Requirements Traceability Tools
Ideas & Products Enabling Your Success!
© 2004 Dayton Aerospace, Inc.
28
Dayton Aerospace
Observed Problems (cont)
Requirements Management Example
• Inadequate definition of requirements
– Statement of operational requirements
– Statement of contractual performance requirements
• Problems
–
–
–
–
ORD too detailed; includes solution
SRD restates ORD
Contract spec paraphrases SRD; includes design solution
Requirements not refined at lower tiers; incomplete requirements to
subcontractors
– Incomplete requirements analysis
• Impacts
– Design & implementation cannot proceed effectively
– Incomplete design; scrap & rework; expectations not met
• Recommendations
– Enforce requirements reviews at Milestone B Review
– Establish process to assure the SRD is complete & accurate
– Reinforce requirement that IMP have well defined accomplishments
with entrance & exit criteria; SRR must be key event for “opening the
gate” to proceed with design
– Develop templates, guides checklists to aid in review processes
Ideas & Products Enabling Your Success!
© 2004 Dayton Aerospace, Inc.
29
Dayton Aerospace
Observed Problems (cont)
Systems Engineering Process
• Systems Engineering Process Not Defined on the
Program
• Systems Engineering Process Defined But Not Applied
• Lack of Flow Down of Systems Engineering Process
Requirements to Development Subcontractors
• Lack of Robust Systems Engineering Applied to Top
Level System Design
• Lack of Timely System Integration and Test (I&T)
Planning
• Inadequate Planning for Obsolescence and
Sustainment
• Design Incorporated into Performance Specifications
Ideas & Products Enabling Your Success!
© 2004 Dayton Aerospace, Inc.
30
Dayton Aerospace
Well
Documented
Industry
Standard
Very Generic
Corporate
Process
Company Approach Top
Level
Applicable to
all Products
Division
Process
Applicable to Product lines
Program
Process
Rarely
Documented
Applicable to Specific Program
Without a Tailored & Documented Systems Engineering Plan,
Program Execution Suffers
Ideas & Products Enabling Your Success!
31
Dayton Aerospace
Why a Program Specific SEP
• Directly tied to rest of program
– Consistency across program plans &
documentation (SOW, WBS, EVMS, etc.)
– Readily correlated to IMP & IMS for execution
• Establishes technical approach for
stakeholders
• Includes program unique nuances
– Established by program requirements
– Provides for explanation of unique tailoring
Ideas & Products Enabling Your Success!
© 2004 Dayton Aerospace, Inc.
32
Dayton Aerospace
Observed Problems (cont)
Engineering Management
• Inadequate Schedules and Budget to
Accomplish Development Effort
• Assumed Reuse Not Confirmed
• Lack of Active Engineering Management of
Development Subcontractor
• Integrated Master Plan (IMP) / Integrated
Master Schedule (IMS) Not Adequately
Addressed and/or Not Followed on the Program
• No Single Technical focal Point (Chief Engineer)
Ideas & Products Enabling Your Success!
© 2004 Dayton Aerospace, Inc.
33
Dayton Aerospace
Conclusions
• Execution of sound SE processes is more
important than ever
• Need forcing functions that drive
execution of disciplined SE processes
Sound SE execution needs to be key
ingredient in contract incentives
Ideas & Products Enabling Your Success!
© 2004 Dayton Aerospace, Inc.
34
Dayton Aerospace
Questions
Ideas & Products Enabling Your Success!
35