Aeronautical Systems Center
Dominant Air Power: Design For Tomorrow…Deliver Today
Environmental, Safety,
and Occupational Health
(ESOH)
Programmatic Risk Tool
Amy Mercado Vince
ASC/RAP
DSN 785-2412
amy.vince@wpafb.af.mil
Overview
Birthplace, Home and Future of Aerospace
• New Acquisition Program Guidance
• ASC’s Response to New Guidance:
ESOH Programmatic Risk Tool
• Benefits, Teaming, and Summary
• Questions/Comments
2
ESOH Programmatic Risk Tool
Background
Birthplace, Home and Future of Aerospace
Requirement: ASC should more effectively
manage ESOH-related risk, keep program
managers informed of ESOH issues, and
comply with new direction in DoDI 5000.2
Purpose:
- To inform you of the tool ASC is developing to address this
requirement
- To show you what to expect from your ESOH professionals
with respect to the systems engineering process
3
ESOH Programmatic Risk Tool
Systems Engineering Application
Birthplace, Home and Future of Aerospace
DoDI 5000.2, in concert with the application of the MILSTD-882 system safety methodology, make up the DoD's
systems engineering (SE) approach for eliminating
ESOH hazards or minimizing ESOH risks across the
entire system life cycle.
12 Sep 04 USD (AT&L) Policy Memo, Defense Acquisition
System Safety, requires PMs to:
1) Use MIL-STD-882 to integrate ESOH risk
management into SE
2) Report status of residual risk acceptance
decisions at technical and program reviews
4
ESOH Programmatic Risk Tool
Systems Engineering Application
Birthplace, Home and Future of Aerospace
Q: Why Establish an ESOH program early?
A: Reduces program risk
•Reduces costs:
– Improves system design early
– Reduces cost of implementing needed changes (i.e.,
technical manuals and training)
•Reduces schedule risk:
– Lowers potential for major redesign
– Lowers potential for additional testing
•Reduces performance constraints:
– Minimizes operational limitations
– Avoids environmental restrictions
5
DoD Acquisition Process
Birthplace, Home and Future of Aerospace
PESHE: A living management/decision making document that
guides identification and management of ESOH risks.
A PESHE is the ONLY DoD-required ESOH document!
A
Concept
Refinement
B
Technology
Development
C
System Development
& Demonstration
Concept Decision
Design Readiness
Review
IOC
FOC
Production &
Deployment
FRP Decision
Review
Operations &
Support
PESHE
Start
Initial Completion
Update Required
Starts as a planning document
Becomes an ESOH Risk Management Tool
6
Overview
Birthplace, Home and Future of Aerospace
• New Acquisition Program Guidance
• ASC’s Response to New Guidance:
ESOH Programmatic Risk Tool
• Benefits, Teaming, and Summary
• Questions/Comments
7
Past ESOH Metrics/Standards Reporting
ASC Reconnaissance Wing, CY 2000
Birthplace, Home and Future of Aerospace
Environmental
Quality Perf. Indicators (EQPIs)
Current
Note:
Note: RAP
RAP and
and Sr.
Sr. Year
Year will
will be
be combined
combined for
for the
the next
next EWG
EWG
Program Ofc. Prg. Doc. (Rev.) Prg. Doc. (Inp.) ESH Plan.
Senior Yr.
Green
Green
Green
Det 3
Green
Green
Green
RAB
Green
Green
Green
RAJ
Green
Green
Green
RAV
Yellow
Green
Green
RAP
Green
Green
Green
RAI
Green
Green
Green
Senior Yr.
Det 3
RAB
RAJ
RAV
RAP
RAI
NEPA Doc.
Green
Green
Green
Green
Green
Green
Green
ODS Elim.
Green
Green
Green
Green
Yellow
Green
Green
ESH Integ.
Green
Green
Green
Green
Yellow
Green
Green
HRA
Green
Green
Green
Green
Green
Green
Green
ESH Training
Green
Green
Green
Yellow
Yellow
Yellow
Green
ASC’s old
metric and
reporting
system
•Issues
could be
masked in
the roll up
within
topics
8
ASC’s Plan
ESOH Programmatic Risk Tool
Birthplace, Home and Future of Aerospace
Goals
• Provide program managers
an assessment of the
“health” of their ESOH
program(s)
• Provide a ready means of
addressing the ESOHrelated regulatory
requirements
• Quantify and qualify the
ESOH programmatic risks
• Provide reports to highlight
issues
• Facilitate development of
the PESHE
• Make more effective ESOH
programmatic decisions
Bottom line: Give Program Managers a tool to
effectively manage ESOH risks and resources
9
ESOH Programmatic Risk Tool
Birthplace, Home and Future of Aerospace
Three Primary Components:
Success Tree, ESOH Risk Manager, and PESHE Template
– Uses reverse failure tree analysis to assess
program’s management effectiveness
– Applies 5M model construct
– Weighted questions developed from Success Tree
Program
Assessment
10
ESOH Programmatic Success
Tool Work
Breakdown Structure
Birthplace, Home and Future of Aerospace
ESOH Program Success
(Planning & Management
Template)
Mission
Success
Man
(Team)
Management (Guidance,
Procedures Standardization,
& Enforcement
Machine
Media
(Tools)
Organic ESOH
Professionals
Contractor
ESOH Staff
Goals/
Objectives/
Targets
PESHE
Process
Acquisition
Strategy
Integration
Hazard
Analysis
Techniques
Regulators/
Stakeholders
Responsibilities/
Authorities
IPTs/Working
Group
Results/
Actions
OSS&E
ESOH Risk
Min/Mgt
Process
Funding
Management
Oversight/
Audit
System
Support
ESOH System
Engineering
Integration
NEPA
Process
Coordination
ESOH
Information,
Data, Reports
Legal/
Regulatory
Reqmts/Stds
P2/HAZMAT
Reduction
DEMIL
Integration
Life Cycle
Cost
Identification
Progress
Tracking
ESOH
Compliance
Hazard I Mgt
Procedures
Functional
Team
Support
11
Mission Program Success
Birthplace, Home and Future of Aerospace
Three Primary Components:
Success Tree, ESOH Risk Manager, and PESHE Template
–
–
–
–
Extracts answers from questionnaire to formulate risk
Uses the six ORM Steps
Identifies benefits associated with addressing risks
Program management tool for ESOH issues
12
Identifying & Assessing Risk
Birthplace, Home and Future of Aerospace
Questions provide basis to evaluate compliance with
ESOH requirements and implement ORM process
13
Mitigation Plans/Tracking Progress
Birthplace, Home and Future of Aerospace
Mitigation plans can be built to achieve
compliance for all required actions
Allows continuous review process to mitigate
risks, make decisions, and keep progress on track
14
MPS Summary Report
Example
Birthplace, Home and Future of Aerospace
15
Example
Birthplace, Home and Future of Aerospace
0%
High Risk
.
Compliance
nce
Risk Matrix Report
.10.3
10.7
20%
. 6.11
. 4.4
40%
. 4.19
60%
. 10.10 . 10.4
80%
100%
. 4.3
. 4.7
Low Risk
. 4.6
0.0
0.3
0.5
4.3
Preliminary Risk Assessment
4.4
Design Analysis
4.6
Part and Material Selection
4.7
Concept Studies and Analysis
4.19 Design for Assembly
6.11 NEPA Schedule
Risk Health Issue
10.3Medium
Occupational
10.4 Technical Risk Assessment
10.7 Air Conformity Issues
10.10 HAZMAT Issues
0.7
0.9
Impact
16
5M Model Implementation
Birthplace, Home and Future of Aerospace
MISSION
Operational
Mission
Goals/
Thresholds/
Objectives
Objectives
Results/
Analysis/
Actions
The ESOH Program Success
portion is based in part on
the 5M Model
ESOH Programmatic
Risk Tool
Man
Media
Acq
ESOH
Mission
Machine
Management
5M Model--Assess
Program & Identify Risks
The 5-M model includes Man, Machine,
Media, Management, and Mission
• Man, Machine, and Media interact to produce
a successful Mission
• Management provides the policies and
procedures to govern the interactions
between Man, Machine, and Media
• Management is often the controlling factor in
operational success or failure
17
ESOH Programmatic Risk Tool
Birthplace, Home and Future of Aerospace
Three Primary Components:
Success Tree, Risk Manager, and PESHE
Template
– Award-winning “cookbook” format that reduced PESHE
development cycle time and cost by 97%
– Achieves format and content standardization across ASC
18
Overview
Birthplace, Home and Future of Aerospace
• New Acquisition Program Guidance
• ASC’s Response to New Guidance:
ESOH Programmatic Risk Tool
• Benefits, Teaming, and Summary
• Questions/Comments
19
ESOH Programmatic Risk Tool
Benefits
Birthplace, Home and Future of Aerospace
Will promote detailed understanding of ESOH
requirements to facilitate decision making
 Supports resource allocation: funding, manpower, schedule
 Assesses ESOH program health through planning and
execution phases
 Quantifies programmatic ESOH risks
 Assesses regulatory compliance
 Provides reports to highlight issues
 Builds mitigation plans
 Provides progress tracking methodology
 Measures performance: ESOH goals/objectives/standards
20
ESOH Programmatic Risk Tool
Benefits
Birthplace, Home and Future of Aerospace
Promotes standardization across Acq Program Offices
– Greater E, S, and OH synergy
– Facilitates system life cycle updates by crossfeeding information to
PESHE
Provides examples of risk minimization and acceptance
benefits and burdens from similar weapon systems
– E.g., HAZMAT substitution success; NEPA issue
Ensures compliance with DoDI 5000.2 and MIL STD 882
– Provides a solid risk mitigation and tracking mechanism
– Records decision making process: lessons learned will result in fewer
ESOH burdens
Will enhance the Acquisition and Operational missions
21
Teaming
Birthplace, Home and Future of Aerospace
• Strong support from Pentagon: SAF/AQRE, SAF/IEE,
AF/ILEV
• Space and Missiles Center and Missile Defense Agency
joined ASC’s team
– Space and missiles requirements will be incorporated into work
breakdown structure and questions
– Alpha phase will include Space-Based Space Surveillance, Patriot,
and Airborne Laser programs plus select ASC programs
– Kickoff team meeting planned for May in DC
• Electronic Systems Center and Air Armament Center will
be invited to participate in beta testing
• Army and Navy will be invited
22
Milestones
Birthplace, Home and Future of Aerospace
• Pre-Alpha Evaluation
– Evaluate/rate the templates/questions
• Initiate teaming with other centers
• Alpha Test
– Train users & populate the tool
– Assess effectiveness
• Refine tool and user’s guide
• Beta Test
– Open to all program offices
– Populate and assess tool and users’ guide
• Finalize tool, user’s guide, and deploy (ECD Spring 06)
23
Summary
Birthplace, Home and Future of Aerospace
• Historically, PESHE used for ESOH management
resource decisions
• ASC has defined, documented results based upon
PESHE application
• ASC’s response to new DoDI 5000.2 and MIL STD
882 guidance will result in a more hard-hitting,
meaningful evaluation processes based on risk,
integral to the systems engineering process
• Alpha and beta testing with new team members will
refine program
24
Overview
Birthplace, Home and Future of Aerospace
• New Acquisition Program Guidance
• ASC’s Response to New Guidance:
Acquisition Mission Program Success
and PESHE Template
• Benefits, Teaming, and Summary
• Questions/Comments
25
Questions/Comments?
Birthplace, Home and Future of Aerospace
BACKUP
28
The ORM 6-step Model
Birthplace, Home and Future of Aerospace
6. Supervise
and Review
5. Risk Control
Implement
4. Make
Control
Decisions
1. Identify
the Hazards
2. Assess
the Risks
3. Analyze
Risk Control
Measures
AFPAM 91-215, Section A, Figure 2
29
ESOH Programmatic Risk Tool
Systems Engineering Application
Birthplace, Home and Future of Aerospace
12 May 03, DoDI 5000.2, Operation of the Defense Acquisition System:
Requires Program Managers (PMs) to provide a Programmatic Environment,
Safety, and Occupational Health Evaluation (PESHE). Required at Milestones
B & C and Full-Rate Production Decision Review.
A PESHE is the ONLY DoD-required ESOH document!
30
New PESHE Guidance
Birthplace, Home and Future of Aerospace
DoDI 5000.2 revised and released in May ’03
Provided new Program Environmental Safety Health Evaluation (PESHE) requirements and
direction.
A PESHE is the ONLY DoD-required ESOH document!
Stresses:
• Documented Risk Management
Process: ASC uses Operational
Risk Management (ORM)
PESHE format
• Risk Management
• Acquisition Strategy
Summary Report
• National Environmental Policy
Act (NEPA) scheduling
• ESOH Integration into Systems
Engineering
• Less prescriptive in format
31
PESHE History
Birthplace, Home and Future of Aerospace
The PESHE: A single, live
document summarizing the
Single Manager’s ESOH
strategy
R
DoDI 5000.2-
Original guidance in DoD 5000.2-R
– Detailed explanation of PESHE format
– Liberal guidance on risk tracking and
reporting
32
ESOH Programmatic Risk Tool
Three Primary Components
Birthplace, Home and Future of Aerospace
ESOH Program
Success Tree
Questionnaire
ESOH Risks
ESOH Risk
Manager
ESOH Risk
Management
• Apply ORM
ESOH
Grade
Card
• Reports
• Action
Plans
PESHE
• Program
Integration
• System & Health
Safety
• NEPA
• Compliance
• HAZMAT
• P2
33
ESOH Programmatic Risk Tool’s
Software Platform
Birthplace, Home and Future of Aerospace
TRIMS: Technical
Risk Identification
and Mitigation
System (US Navy)
• Process-oriented tool
–
–
–
–
–
–
–
–
–
Evaluate ESOH requirements
Identify hazards
Assess risks
Analyze risk control
measures
Make control decisions
Risk control implementation
Supervise and review
Assign responsibilities
Integrate with PESHE
• Windows application
software
34
ESOH Programmatic Risk Tool
Model Development and Tailoring
Birthplace, Home and Future of Aerospace
Categories: Process Steps
Templates: Subprocesses where risk
levels are assigned
Questions: Methodology to
accomplish ORM:
• Compliance and risk level
determination
• Responsibility assignment
• Mitigation plans and
progress tracking
35
Template/Question Development for
TRIMS
Birthplace, Home and Future of Aerospace
We are finalizing the weighting
Questions
Reference/ Source
Document
1.Has the PM, in concert with systems
engineering and associated system safety
professionals, determined what system safety
effort is necessary to meet program and
regulatory requirements?
MIL-STD 882d,
para A.4.3
2. Has the program chief engineer incorporated a
robust systems engineering program, including
configuration management to attain OSS&E
assurance?
MIL-STD 882d,
para 11.5, subpara 7
Why it is important/other
Question
Weight
(1 to 10)
Before formally documenting the
system safety approach, the program
manager, in concert with systems
engineering and associated system
safety professionals, must determine
what system safety effort is
necessary to meet program and
regulatory requirements.
AFI 92-202, para 9.2 also reads,
“The PM shall ensure that system
safety engineering is an integral part
of the systems engineering process
and receives proper management
attention.”
36
ESOH Programmatic Risk Tool
Process
Birthplace, Home and Future of Aerospace
ESOH Programmatic Risk Tool
ESOH Risk
Manager
ESOH Pgm
Plan & Mgt
Concept
Refinement
Risks
Tech Devel
Risk
Sys Dev &
Demo Risk
Prod & Deploy
Risk
ESOH Program
Success
Ops & Spt
Risk
Management
Reports
Benefits of
ESOH Risk Min
Success
Program Mgt
Assessment
ORM Process
Costs of ESOH
Risk Min Failure
Management
Action Plans
PESHE
Data &
Information
37
Summary Report Example
Birthplace, Home and Future of Aerospace
38
Sample WBS in TRIMS
Birthplace, Home and Future of Aerospace
39
ESOH Programmatic Risk Tool
Resource Benefit
Birthplace, Home and Future of Aerospace
How will the Program enhance/sustain
resources?
Identifies
ESOH
risks
Identifies
potential
solutions
Provides tools
for managing
risks
Identifies
Life Cycle
Costs
Facilitates
Risk vs. Cost
Decisions
The Program should enable management
to find cost effective solutions in less time
40
Sample Question in TRIMS
Birthplace, Home and Future of Aerospace
41
Sample Question in TRIMS
Birthplace, Home and Future of Aerospace
42
Sample Question in TRIMS
Birthplace, Home and Future of Aerospace
43
Sample Action Plan in TRIMS
Birthplace, Home and Future of Aerospace
44
Sample Question in TRIMS
Birthplace, Home and Future of Aerospace
45
Sample Action Plan in TRIMS
Birthplace, Home and Future of Aerospace
46
Sample Question
with
Why is this important? Screen in TRIMS
Birthplace, Home and Future of Aerospace
47
Mitigation Plans/Tracking Progress
Birthplace, Home and Future of Aerospace
Mitigation plans can be built to achieve
compliance for all required actions
Allows continuous review process to mitigate
risks, make decisions, and keep progress on track
48
Generated Reports
Birthplace, Home and Future of Aerospace
Summary
Report
Detailed
Report
Risk Matrix
Report
A graphical depiction of risk levels
Data from TRIMS’ questions and
mitigation plans
A conventional probability vs.
consequence risk matrix
Data in these reports can be tailored to provide managers
the information necessary to make decisions
49
Example NEPA Issue
Birthplace, Home and Future of Aerospace
• A major aircraft program planned part of its OT&E at an Army
test range -- It was believed an existing test range EIS on the
would cover the aircraft OT&E
• It was determined that a new EIS was required on the test
range to support the OT&E
– This determination was ascertained 7 months before
certification and 9 months out from start of testing
– A new EIS would have taken at least one year
– The program's OT&E plan was restructured to eliminate
the need for that test range
• This example shows how external forces can affect a weapon
system program and that continual attention to ESOH issues
minimized the adverse impacts on the program
Source: PESHE Development Guide for PMs, Dec 03, ESC
50
HAZMAT Substitution Success
Birthplace, Home and Future of Aerospace
• Aircraft Radome Chemical Depainting involved MEK, an EPAlisted chemical solvent
• Searched for alternative process
• Selected a chemical alternative, compatible with the radome
materials & capable of removing the protective coatings
– Solvent is a blend of dibasic esters [DBE] w/ low vapor pressure and
low toxicity
• Benefits
–
–
–
–
–
–
–
–
Easy to use
Goes further [less expensive to use]
Compatible with radome materials
Not an EPA-listed material -- Environmentally compliant
Lowers health risks to workforce
$0 implementation cost and Reduces operating costs by $30K
Reduces HAP emissions by 78,000 lbs
Eliminates abatement requirements [$2M]
51
Integrating with PESHE
Birthplace, Home and Future of Aerospace
Reports exported to PESHE
Summary
Report
Detailed
Report
Risk Matrix
Report
Promotes updated risk review and decision making
52