ISSTS 2014
• Technical Papers
• Tutorials
• Panel Discussions
• Workshops
UM
OSI
MP
SY
Safety
GATEWAY
TO SAFETY
U
20
14
.L
ST
Arching
O
G
INTER
NA
TI
O
N
Program
OverM SAFET Y TR
AI
STE
Y
N
S
IN
L
A
IS
,
T
MI
US
SSO
UR I · AUG
8,
4-
Things to Do
in St. Louis
Gateway Arch, Six Flags,
Saint Louis Zoo, Forest Park,
Ballpark Village/Busch
Stadium, Anheuser-Busch
Brewery, Saint Louis Science
Center, Missouri History
Museum, The Muny,
Saint Louis Art Museum,
Missouri Botanical Garden
St. Louis Union Station DoubleTree Hotel
1ST FLOOR
Foyer C
C
Grand Ballroom
F
E
D
Foyer A
B
Foyer B
B
Regency Ballroom
A
A
Caboose
Reg Office
Elevator Depot Reg Office
Stairs
C
Outdoor
Courtyard
Restrooms
Access to
Ballrooms
Terminal
Atrium
Switchman
Room
2ND FLOOR
Storage
Missouri Pacific
Illinois Central
Pegram
Wabash Cannonball
Terminal
Atrium B
Texas Special
Atrium
Knickerbock
Midway Bridge to Garden Rooms
II
I
Front Entrance Market Street
Grand
Hall
Stairs
Registration
2nd Floor
Conference Center
Speakers Breakfast
Presentations/Tutorials/Workshops
Receptions/Exhibitor Area
Opening Ceremony/
General Meeting/Luncheons
Committee & Group Meetings
To Grand Hall
Meeting Rooms
2
III
Gothic Corridor
Front
Desk
IV
Midway Suites
Jeffersonian
Station
Grille
Access to
Hotel
Prefunction
Colorado Eagle
Elevator
Stairs
Terminal Atrium A
Grand Hall
Balcony
Conductor
Room
Men
Midnight Special
Meteor
Midway
Burlington Route
Ladies
Dixie Flyer
Station
Master Room
Frisco
New York Central
Zephyr Rocket
Red Caps
Room
Presenter Preparation
Staff Office
General Information...................................................................................................................................................... 3
Greetings......................................................................................................................................................................... 5
Speakers........................................................................................................................................................................... 8
Schedule........................................................................................................................................................................ 12
Tutorials........................................................................................................................................................................ 21
Panel Discussions/Forums.......................................................................................................................................... 29
Workshops..................................................................................................................................................................... 30
Paper Presentations..................................................................................................................................................... 33
About the ISSS.............................................................................................................................................................. 48
Special Functions......................................................................................................................................................... 50
ISSTS 2014 Program
Contents
Organizing Committee
The following volunteers contributed to the success of the conference.
Conference Chair
Technical Program Chair
Tutorial Coordinator
Pam Kniess
Don Swallom
Debbie Hale
Vice Chair
Sponsor/Exhibitor Chair
Sponsor/Exhibitor
Warren Naylor
Barry Hendrix
Melissa Emery
Facilities Chair
Protocol/Speakers
CEUs
Darrell Stokes
Bill Edmonds
Dr. Rod Simmons
Social/Off-site Chair
Registration
St. Louis Site Coordinator
Matt Johnson
Cathy Carter
Carol Barnes-Schmedake
Communications
Communications/Social
Publishing
Saralyn Dwyer
Arch McKinlay
Heather French
International Chair
Webmaster
Bob Fletcher
Don Swallom
1
w w w. a p t- re s e a rc h .co m
System Safety Engineering & Analysis. Mission Assurance. Range
Safety. Test Planning. Explosives Safety. Software System Safety.
Industrial Engineering. Quality Engineering. Reliability Engineering.
Software Development & Modeling. Independent Risk Assessments.
Standards Development. And now...
C-IED Training &
Training Devices
APT personnel have extensive experience
in an environment saturated with IED
weaponry. With this knowledge and
experience, our experts are qualified to
conduct training in the latest, advanced
techniques for IED defusal, detonation,
and disposal, and to supply quality inert
reproductions of terrorist IEDs.
Providing safe solutions. Protecting your most valuable assets.
Founded in 1990, APT (Analysis, Planning,
Test) is an employee-owned, small business
located in Cummings Research Park near
Redstone Arsenal in Huntsville, Alabama.
Our corporate vision is to provide
state-of-the-art expertise and ensure the
highest levels of customer satisfaction.
A-P-T Research, Inc.
4950 Research Drive
Huntsville, Alabama 35805
Phone: 256.327.3373
Fax: 256.837.7786
www.apt-research.com
DoD photo by Senior Airman Jodi Martinez, U.S. Air Force/Released. Modified from original.
2
Registration Desk. The Registration Desk is in the
Depot Registration Office near the access to the
Grand and Regency Ballrooms.
Messages. An easel is located in the registration area
to post conference information and messages to
attendees.
Badges & Special Event Tickets. Go to Registration
Desk upon arrival to pick up badge. All personnel
must display a 32nd ISSTS badge while attending
sessions, luncheons and social events. Once a
badge is issued, it is the sole responsibility of the
registrant to ensure that it is not lost. If you are a
sponsor, the name on each badge can be changed
as often as necessary between sessions, if approved
by Registration. Exhibitor badges are nontransferable.
32nd ISSTS Daily News. The daily news will be
available at 7:30 each morning and can be picked up
at the Registration Desk.
Spousal Program. A meeting will be held Monday at
9:00 in the in the Frisco/Burlington room on the
second floor. Information and maps will be provided
and a St. Louis resident will be there to answer
questions.
ISSTS 2014 Program
General Information
Tuesday Evening Sponsor & Exhibitor Social. Please join
us for a St. Louis steakhouse dinner buffet at the
Special event tickets may be purchased by spouses
Sponsor and Exhibitor Social. Visit the sponsor and
and guests at the Registration Desk at least 24 hours
exhibitor booths. The Boeing Jazz Band will provide
before the event. Tickets for the Tuesday night
entertainment.
social event and Wednesday night off-site event
Wednesday Evening Off-site. Fly in a simulator or just
will be sold for spouses and guests. Kids tickets
explore, and enjoy dinner and a star show at the
will be available for the Wednesday night off-site
James S. McDonnell Planetarium at the Saint Louis
event. Tickets for Monday, Tuesday, Wednesday
Science Center. Bus transportation will leave the
and Thursday luncheons will also be sold at the
Registration Desk at least 24 hours before the event. hotel between 5:00pm and 5:30pm and return to the
hotel between 9:00pm and 10:00pm.
Internet. There is complimentary wireless internet
in guest rooms.
Transportation. There are many transportation
options in St. Louis. The MetroLink is behind
the hotel. MetroLink One Ride Ticket is $2.50.
MetroLink Two Hour Pass is $3.00. The Metro
Weekly Pass is $27.00. If you drive to the hotel,
be sure to use Valet Parking. The 2014 ISSTS gets
Valet Parking at half price which makes it cheaper
to Valet Park than to Self Park.
Tutorial Program and CEUs. Continuing Education
Units (CEUs) will be issued by the 2014 ISSTS for
attending the conference tutorials. To receive
CEUs for a tutorial, you must attend the entire
tutorial. You must sign in after returning from
any breaks that occur during the tutorial, and you
must be present at the end of the tutorial. The
certificates will be issued on the basis of 0.1 CEU
per instruction contract hour.
Dress Code. We would like you to feel comfortable
while you are in the sessions, so we advise
“business casual” attire. The Awards Luncheon on
Thursday is traditionally the time when you may
want to dress more formally with business dress.
All off-site events are business casual.
3
Thank you to
Boeing
for supporting
the 32nd International System Safety
Training Symposium
www.boeing.com
4
From the Society President
I want to extend a special welcome to the 2014 International
System Safety Training Symposium. I am extremely happy to
have you here in my home town of St. Louis for this 32nd annual
conference and training opportunity. I want to thank our sponsors
and exhibitors for making this conference possible. Please join us
on Tuesday night for a social event and dinner recognizing them.
In addition, our speakers and authors are a vital component of
this conference. Most importantly, I want to thank all of you for
caring enough about your professional development to attend this
conference.
ISSTS 2014 Program
Greetings
This is a week of opportunity! We all work in similar disciplines
with similar issues. Our learning together, exchanging ideas, and
developing professional networks improves our worth to our companies. Furthermore, it raises the
worth of our profession in society. The International System Safety Society (ISSS) and these conferences
are important contributors to the safety of products and services across industry. I think we need to
remember that as we recognize the value of the ISSS.
In addition to the daytime professional development offerings, there are a number of evening
opportunities for networking, socializing, and enjoying the local area. As you may know, St. Louis
was host to the 1904 World’s Fair part of which was located in the area that is now Forest Park. Our
Wednesday night off-site event will take us to the James S. McDonnell Planetarium at the St. Louis
Science Center in Forest Park. As you travel through Forest Park, please note that one of the buildings
which was built for the World’s Fair now houses the Saint Louis Art Museum. If you are a sports fan, the
St. Louis Cardinals are playing the Boston Red Sox at Busch Stadium on Thursday night.
Enjoy this week! Get to know the other attendees. Take advantage of the many opportunities to develop
your professional skills. This week is for you and I hope you find it rewarding.
Thank you for meeting us in St. Louis!
Robert A. Schmedake
President,
International System Safety Society
5
© 2012 Lockheed Martin Corporation
3, 2, 1 Safety
System safety is paramount. It impacts our products, employees, technicians, and
maintenance personnel. And safety is no accident – it is designed into everything we do.
We are proud to sponsor this year’s International System Safety Conference and
their mission to think outside the box when it comes to the best processes, methods,
and techniques. We’re committed to delivering innovative ideas and solutions that
help connect, protect, and explore our universe.
www.lockheedmartin.com/ssc
6
307-64315_TurningVisions_ISSC.indd 1
7/16/12 5:01 PM
From The Conference Chair
Welcome to St. Louis and the 32nd International System Safety
Training Symposium. The conference committee has worked
hard to make this a world class event, and we are committed to
providing an enriching experience for all. We realize that budgets
are tight and are happy that you have chosen this conference
to attend. As the system safety environment changes, we have
tried to include tutorials, workshops, panels and papers on new
methods, technologies and ideas. Tutorials will earn continuing
education units to document your professional development.
ISSTS 2014 Program
Greetings
After the General Assembly address on Monday there will be a
panel composed of several of the Society Fellows titled “Exploring
Society History as a Guide for its Future Course.” This panel
provides an informal opportunity to learn from longtime members and hopefully use that information to
shape the future of our society.
On behalf of the entire conference committee, we sincerely hope you enjoy your stay in St. Louis. There
are many things to see and do in the city and the surrounding areas. St. Louis Union Station, the site of
this year’s conference, is a National Historic Landmark. On September 1, 1894 it opened as the largest
train terminal in the U.S. The MetroLink is behind the hotel and stops at many of the historic and scenic
areas in St. Louis. On Tuesday evening we have a social event and dinner honoring our sponsors and
exhibitors. Come prepared to enjoy food and networking opportunities. The Wednesday night offsite event at the James S. McDonnell Planetarium at the Saint Louis Science Center will be an exciting
experience, including flight simulators, dinner, and a star show. If you enjoy baseball, the Boston Red Sox
are in town playing the hometown St. Louis Cardinals, and there is a game on Thursday night.
If you have a special request or need assistance at any time during the conference, please feel free to see
me or any of the conference committee for assistance.
Sincerely,
Pam Kniess
Conference Chair
International System Safety Society
7
Speakers
Carl A. Avila, Director, Advanced Programs, Boeing Phantom Works, Keynote Speaker
Carl Avila is the Director of the Advanced Weapons and Missiles
Systems organization within the Advanced Boeing Military
Aircraft organization in Phantom Works. Carl’s team is responsible
for developing products and technologies supporting next
generation tactical systems.
Carl has held various Program Management and Engineering
assignments during his 36 year career with Boeing.
Prior to this assignment, Carl was Program Manager of the
Air Launched Cruise Missile and Conventional Air Launched
Cruise Missile (ALCM/CALCM) Program, responsible for ongoing
production of CALCM missiles, as well as support of fielded
systems. Carl was the F/A-18 Affordability Manager, responsible
for developing and implementing the Program’s Affordability
plan, including Cost Reduction programs aimed at reducing the
cost of the F/A-18 E/F. Carl was Program Manager of the Joint
Direct Attack Munition (JDAM), responsible for the day-to-day
operations of this joint U.S. Air Force, Navy and Marine Corps missile program during its transition from
development into high rate production.
Joining Boeing in 1978 as a logistics engineer, he also held several engineering positions associated with
the design, development and fielding of tactical missile systems and launch platforms. He was Chief
Engineer and Deputy Program Manager on the Bradley-Linebacker program, and Chief Engineer for the
Avenger Air Defense System.
Avila holds a Bachelor of Science degree in Electrical and Computer Engineering from the University of
Massachusetts, and an Executive Masters in International Business from Saint Louis University.
8
Tom Pfitzer holds a Master’s Degree in Industrial Engineering
(System Safety Option) from Texas A&M University. He is a
graduate of the U.S. Army Intern Program in Safety Engineering.
He has over 35 years’ experience in System Safety, Range Safety,
and Risk Analysis and has held various positions in safety and risk
assessment in both Government and industry. Prior to establishing
the Safety Engineering and Analysis Center (SEAC), he founded
A-P-T Research, Inc., in 1990, a company that employs over 100
practicing safety professionals. Early in his career he was the
Safety Officer at a national range monitoring safety for over 200
launches.
ISSTS 2014 Program
Tom Pfitzer, President, A-P-T Research, Inc.,
Sponsor & Exhibitor Luncheon Speaker
Tom has supported numerous U.S. and international agencies that
are developing risk-based standards. He is currently a member
of the Society of Risk Assessment, a senior member of System
Safety Society, and on the Board of Directors of the International
Association for the Advancement of Space Safety, chairing the
Launch Safety Committee. He has authored more than 20 papers in technical journals.
Gabriele Schedl, Frequentis, International Luncheon Speaker
An Austrian safety pioneer, Gabriele Schedl has been strongly
dedicated to the development of appropriate safety education,
information and research in Austria over more than a decade.
Her notable contribution was not only the establishment and
implementation of a Safety Management System at the company
Frequentis, but also the foundation of a safety education program,
as well as development of a study program for integrated safety
(security) management at the University of Applied Sciences in
Vienna. She also gives regular lectures in IT Service Management
and Project Management in the safety-critical area at university.
Mrs. Schedl’s responsibility as Director of Safety Management
at Frequentis, where she has been working since 1999, contains
furthermore, the management and performance of extensive
safety training programs for employees and safety trainings for
international customers.
Raising Safety awareness, putting the priorities of team above
individual success, as well as stressing the impact of one’s own actions in a safety critical environment
are the main topics her work is focused on.
She holds an Engineering Degree and a Master of Science in Electrical Engineering from the University
of Technology, Vienna, has finished a post-graduate education in business computer science and has
completed several safety courses at Eurocontrol, the University of York, and the University of Southern
California. She is committed to lifelong learning.
9
Technology Meets Tradition
in the UH-60M BLACK HAWK and S-97 RAIDER™ helicopters.
Sikorsky continues its long history of providing continuous improvement
and leading-edge solutions for its current production aircraft while
actively preparing for vertical lift requirements of tomorrow.
Updated with impressive new capabilities, the UH-60M BLACK HAWK
helicopter is today’s multi-mission workhorse, while the S-97 RAIDER™
aircraft is a bold and innovative investment in the future armed aerial
scout mission.
sikorsky.com |
10
6351 6/14
Dr. Garza came to Saint Louis University after service in the
Federal Government. In August of 2009, Dr. Garza was appointed by
President Obama and confirmed by the US Senate as the Assistant
Secretary for Health Affairs and Chief Medical Officer for the U.S.
Department of Homeland Security (DHS) where he served until
April of 2013. Dr. Garza led the health and security efforts for
DHS which included the health aspects of terrorism and natural
disasters.
His office led DHS programs in CBRNE including the BioWatch
program and the National Biosurviellance Integration Center. He
has served as the DHS lead in response to the H1N1 pandemic,
and the health lead for multiple disasters including the Fukishima
Tsunami, the Deepwater Horizon oil spill, the Haiti earthquake
among others. In 2009, Dr. Garza was also appointed by President
Obama to serve on the Presidential Commission for the Study of
Bioethical Issues. He participated and helped author reports to
the President on numerous issues including the use of Synthetic Biology and the ethics of bioterrorism
medical countermeasure development and testing on children.
ISSTS 2014 Program
Alexander Garza, MD, MPH, Associate Dean and Professor in Epidemiology, Saint Louis
University College of Public Health and Social Justice, Awards Luncheon Speaker
Dr. Garza began his career in health as an EMT in 1986 and has worked in every level of providing care
and leadership in emergency medicine. He has extensive experience working as a paramedic and flight
medic and is a board certified emergency physician working at academic medical centers. Dr. Garza
has worked as the Medical Director for Emergency Medical Services for the City of Kansas City Health
Department and the State of New Mexico. He is a Lieutenant Colonel in the US Army Reserves and a
veteran of Operation Iraqi Freedom.
He has written and lectured extensively on issues involving EMS, health, security and preparedness
and has counseled leadership at the highest levels of government. Dr. Garza is considered an expert
in weapons of mass destruction, health threats to national security and strategic and operational
excellence.
Dr. Garza has received numerous awards for his military service and civilian career including the
Bronze Star and Combat Action Badge as well as the Young Investigator Award by the American Heart
Association for his research in out of hospital cardiac arrest. He and his wife Melissa enjoy spending time
with their three boys doing camping, scouting, swimming and baseball.
11
Schedule Monday, 4 August
Gothic Corridor: 6:30 - 8:00 Speakers’ Breakfast
Jeffersonian/Knickerbock: 8:00 - 17:00 Presenter Prep
Grand B
8:00 - Safety Topics 1 (Gonzalez)
8:50
“Mishap Prevention Utilizing Unlimited &
Government Purpose Data Rights”
McDougall
9:00 9:50
Grand C
Hazard Analysis 1 (Rose)
“Cases For Tailoring The MIL-STD-882E Risk Matrix
For US Air Force Space & Launch Vehicles”
Moran, Jackson
“Practical Insights for the Exchange of Leading
“MIL-STD-882E, A Near Miss About System Safety”
Practices Lessons Learned in Accident Investigation
Sadler
and Lessons Learned from Incident Investigations”
Johnson, Reinartz, Rebentisch
10:30 - “System Safety Challenges in High Energy
“Accurate Risk Assessment using Multi11:20 Laser (HEL) Weapon Systems”
Relational Hazard/Mishap Pairings”
Chizek
Eller, Zemore, Kady
11:30 - Luncheon & Opening Ceremony, Grand F
13:30
Carl A. Avila, Director, Advanced Programs, Boeing Phantom Works, Keynote Speaker
13:30 - General Assembly Address, Grand F
14:20
Society President Bob Schmedake
14:30 - Panel, Regency C
15:20
Exploring Society History as a Guide for its Future Course
16:00 - Gordon
16:50
12
Regency B
Tutorial
Why Should You Care About the “-ilities”
Southwick (3 hrs)
Health Hazard Assessment in System Safety Evaluation
Geiger, Simmons (3 hrs)
9:00 9:50
ISSTS 2014 Program
8:00 8:50
Regency A
Tutorial
10:30 11:20
11:30 13:30
13:30 14:20
14:30 15:20
16:00 16:50
WHAT A R E Y OU R STR ENGT HS
AS AN ENGINEER?
“thinking system-wide”
“providing in-depth analysis of
the latest technologies”
“solving real-life problems”
ONLINE MASTER’S DEGREES
- RELIABILTY ENGINEERING
- PROJECT MANAGEMENT
- CYBERSECURITY
LEARN MORE
www..advancedengineering.edu/issc
13
Schedule Tuesday, 5 August
Gothic Corridor: 6:30 - 8:00 Speakers’ Breakfast
Jeffersonian/Knickerbock: 8:00 - 17:00 Presenter Prep
Grand A
Grand B
8:00 - Tutorial
Aviation Safety (Liming)
8:50
Functional Hazard Analysis “Aviation Safety Risk Modeling:
Scharl, Stottlar, Kady,
Lessons Learned from Multiple
Ingram
Knowledge Elicitation Sessions”
(6 hrs)
Luxhoj, Ancel, Green, Shih, Jones, Reveley
9:00 “DO-278A Impacts on Legacy MIL-STD9:50
882E Air Traffic Management Programs”
Bartos, Oviedo
10:30 “Modeling Increased Complexity and
11:20
the Reliance on Automation: FLightdeck
Automation Problems (FLAP) Model”
Ancel, Shih
11:30 - Sponsor & Exhibitor Luncheon, Grand F
13:30
Guest Speaker Tom Pfitzer, President, A-P-T Research, Inc.
13:30 - Tutorial (cont’d)
14:20
14:30 15:20
16:00 16:50
Creating A Culture of Chronic Unease
What Does it Take to Really “Learn
from Incidents” Lessons get shared,
but sharing is not learning
Best Practices in Process Safety Culture
- Lessons from the Energy Industry
Safety Topics 2 (Atencia-Yepez )
Safety Topics 3 (Pottratz)
“Defining Layered Safety Concepts to
Guide Multi-Supplier Development of
Interoperable Safety-Critical Systems”
Shi, Ailey, Gough
Rhapsody Model Safety Tagging for
Model Development Driven (MDD)
Approach to System Architecture
and Requirements Development”
Duong
“Gaining Deeper Operational
Intelligence Using Human
Performance Learning Teams”
Edwards
“Reducing High Impact Events by
Implementing a Deep Dive Process
Early in the Event Closure Lifecycle”
Laabs, Allison, Russell, Stewart
“Reconciling Developmental Weapons
Safety Tests in MIL-STD-2105”
Tomasello, Dray, Adams
18:00 - Sponsor & Exhibitor Social located in Grand D/E/F
21:00
Entertainment will be provided by the Boeing Jazz Band
14
Grand C
Workshops (Pearlman, Scott )
“Effects of Unintended Longitudinal
Acceleration and Deceleration
Profile Magnitude and Duration on
Driver Performance Behaviors”
Vernacchia, Green, Llanares
Regency B
Tutorial
Regency C
Tutorial
Melding DOD and FAA
System Safety Methods
Jones (3 hrs)
Human Factors and Systems Safety
Sandom (3 hrs)
New Approaches to the CyberSecurity of Safety-Critical Systems
Johnson (5 hrs)
Tutorial
Tutorial (cont’d)
9:00 9:50
ISSTS 2014 Program
8:00 8:50
Regency A
Tutorial
10:30 11:20
11:30 13:30
13:30 - Workshop
14:20
Human and Organizational
Performance (HOP) Fundamentals
Edwards (4 hrs)
Weapons Systems Software Safety
Criticality and Level of Rigor (LoR)
Bower (3 hrs)
14:30 15:20
16:00 16:50
18:00 21:00
15
Schedule Wednesday, 6 August
Gothic Corridor: 6:30 - 8:00 Speakers’ Breakfast
Jeffersonian/Knickerbock: 8:00 - 17:00 Presenter Prep
Grand A
Grand B
8:00 - Tutorial
Robotics/Unmanned Systems (Kady)
8:50
Software Safety Analysis
“Models for Assessment of
Rogers, Zemore, Whitford, Unmanned Air Vehicle Hazards”
Tilghman, Funkhouser (6
Chiam
hrs)
9:00 9:50
10:30 11:20
“Safety, Autonomy, Latency, and the
Unmanned or Remotely Piloted Vehicle”
McKinlay
“System Safety Considerations for
Unmanned Ground Vehicles”
Owens
Grand C
Software Safety 1 (Axelrod)
“Interpretation of the Software Control
Categories for MIL-STD-882C”
Tan
“Decoding The Software Control Category”
Onn, Ericson, Brown
“System and Software Safety
Challenges for Widespread
Acceptance of Driverless Vehicles”
Turgeon
11:30 - International Luncheon, Grand F
13:30
Guest Speaker Gabriele Schedl, Director of Safety Management at Frequentis in Austria
13:30 - Tutorial (cont’d)
14:20
Safety Topics 4 (Rozanski)
Software Safety 2 (Schedl)
“The Evolution of System Safety at NASA”
Dezfuli, Groen, Everett
14:30 15:20
“Utilizing Error Prevention Data and
Lean Six Sigma Techniques to Verify
the Existence of Error Prone Zones”
Laabs
“Study of a Method for Early Interface
Verification with Hierarchical
Executable Software Model”
Ujiie, Katahira, Hernek, Rubio
“Predicting Software
Performance - Software 1”
Zito
16:00 16:50
“What is ‘Unnecessary’ Code
and Why Is It Unsafe?”
McKinlay
17:00 - Dinner at James S. McDonnell Planetarium at the Saint Louis Science Center
22:00
Will include flight simulators and a star show.
Buses leave between 17:00 and 17:30 and return to hotel between 21:00 and 22:00.
The gift shop will extend their hours until 18:00.
16
Regency B
Tutorial
Regency C
G-48 Meeting
Frisco/Burlington
G-48 Meeting
(cont’d)
Tutorial
“Hazardous Dependency of Critical
Hands-On System Safety
Infrastructures on Global Navigation
Basics, Focused on FHA
Satellite Systems Services”
Fritz, Schedl (3 hrs)
Atencia-Yepez, Cueto-Santamaría,
Cezón-Moro
9:00 - “Major Hazardous Events for
9:50
Unmanned Space Systems”
Durmaz
10:30 - “Simulating the Risks of Sub-Orbital
11:20 Space Flight for Air Traffic Management”
Johnson, Sarconi
ISSTS 2014 Program
8:00 8:50
Regency A
Space Safety (Thomas)
11:30 13:30
13:30 - Tutorial
14:20
“The STAMP Model, STPA Hazard
Analysis and CAST Accident Analysis”
Fletcher (2 hrs)
Fault Tree Analysis
with CAFTA
Roy (3 hrs)
14:30 15:20
16:00 16:50
17:00 22:00
17
Schedule Thursday, 7 August
Gothic Corridor: 6:30 - 8:00 Speakers’ Breakfast
Jeffersonian/Knickerbock: 8:00 - 17:00 Presenter Prep
Grand A
Grand B
8:00 - Tutorial
Hazard Analysis 2 (Durmaz)
8:50
Practical Generation “Innovation vs Safety: Hazard Analysis
of Safety Cases With Techniques to Avoid Premature Commitment
the Help of GSN
During the Early Stage Development of
Gerstinger, Schedl National Critical Infrastructures”
(3 hrs)
Johnson
9:00 “Mind-Mapping the Hazard Space of a System”
9:50
Ericson
10:30 11:20
Grand C
Risk Assessment 1 (Rinaldo)
“Distribution of Risk”
Banerjee
“Personnel Risk Assessment for Random
Reentry Considering Casualty Expectation”
Dang, Meyers, Jackson
“The Challenges of a Quantitative
Approach to Risk Assessment”
Kady, Ranasinghe, Zemore, Eller
11:30 - Awards Luncheon, Grand F
13:30
Guest Speaker Alexander Garza, MD, MPH serves as Associate Dean and
Professor in Epidemiology at the St. Louis University College of Public Health and Social Justice
13:30 - Workshop
Security & System Safety (Owens)
Risk Assessment 2 (Pierson)
14:20
Systems-of-Systems “Supporting the Exchange of Lessons
“Usability for System Safety Engineers:
(SoS) Workshop
Learned from Cyber-Security Incidents
Using Nielson’s Ten Heuristics to Identify
on Framework,
in Safety-Critical Systems”
the Increased Potential of Human Error
Collection,
Johnson
as a Contributor to Mishap Risk”
Processing and
Funkhouser, St. Laurent, Sperry
Organizing
for
14:30 “Cyber Safety of Voice Communication Systems: “Influence Diagrams: Generalizing Fault
15:20 System Safety and About Security Threats and Safety Analysis”
Trees for Informed Decision Making”
Software Safety
Riedl, Schedl
Monat, McCracken, Obaldo, Sweany
McKinlay,
16:00 - Murgatroyd (4 hrs) “Inadequate Legal, Regulatory, and Technical
16:50
Guidance for the Forensic Analysis of CyberAttacks on Safety-Critical Software”
Johnson
18
9:00 9:50
Regency B
Workshop
Regency C
Panel
Joint Munitions Safety
Testing Initiative
Hawley, Dray (1 hr)
System Safety Handbook
Working Group
Muniak (2 hrs)
G-48 Workshop – The Most Pressing
Issues Facing System Safety
West (3 hrs)
ISSTS 2014 Program
8:00 8:50
Regency A
Tutorial
Open Forum
10:30 11:20
11:30 13:30
13:30 - Open Forum (cont’d)
14:20
Panel
Developing Global System
Safety Perspectives
Naylor, Fletcher (2 hrs)
14:30 15:20
16:00 16:50
Friday, 8 August
Gothic Corridor: 6:30 - 8:00 Speakers’ Breakfast
Regency C
8:00 - Best Papers Presentations (Swallom)
8:50
Best Paper #1
9:00 - Best Paper #2
9:50
10:00 10:30
10:30 11:20
Frisco/Burlington
Lessons Learned &
ISSTS Staff Turn-Over Meeting
19
Manage your risks
Improve your safety!
At Controls and Data Services (CDS), we provide solutions to
understand and manage operational risks, helping you to run a
safer, more efficient and fully compliant business.
VISIUMKMS™ helps you effectively manage
change through a proven system for
tracking corrective and preventive actions,
conducting process hazard analyses (PHAs),
audit facilities and managing incident
investigations.
VISIUMRISK™ combines CAFTA (fault trees
analysis) with maintenance programme
data, to provide insight into operational
safety and enable decisions that improve
both safety and productivity.
Choose our proven solutions to support your business.
www.controlsdata.com
info@controlsdata.com
CAFTA is an EPRI developed product. EPRI: Electric Power Research Institute, Palo Alto, California.
CDS is a licensed re-seller of CAFTA and related EPRI software products.
20
Advert 6"x9".indd 1
09/07/2014 18:06
The symposium organizers have an information-packed tutorial program planned for the ISSTS 2014.
Attending tutorials, as well as other elements of the Technical Program at the ISSTS 2014, meets the
requirements for Continuance of Certification credit through the Board of Certified Safety Professionals
(BCSP). The International System Safety Society will issue Continuing Education Units (CEUs) for
participation in the symposium tutorials. CEUs are issued on the basis of 0.1 CEU per instructional contact
hour. You must be present for at least 90% of the tutorial to receive CEUs and a tutorial completion
certificate. Your attendance is verified via the process outlined below:
• At the start of the tutorial, you’ll clearly print your name in the attendance form exactly as you want it
to appear on the certificate.
• After returning from each break during the tutorial (morning, lunch, and/or afternoon), you’ll initial
the attendance form.
• You must be present at the end of the tutorial to receive your certificate and the CEUs. If there are misspellings on the CEU certificates, please mark the corrections, give back to the instructor or leave at the
registration desk.
ISSTS 2014 Program
Tutorials
Monday // 08-04-14 // 8:00-11:20 // Regency A // TUTORIAL 0.27 CEU
Why You Should Care About the “-ilities”
Instructor: Alan Southwick, Raytheon Company, Portsmouth, Rhode Island, United States
Abstract: Topic addresses the interrelationships developed from Quality, Quality Control, and
Quality Engineering, pursuing Specialty Engineering Roles and Relationships, including: Reliability,
Maintainability, supportability, Human Factors, Safety, and Security (Information Assurance) from an
overview perspective. The tutorial is designed to be somewhat interactive presenting examples and
questions to the audience related to the various topics discussed, thereby engaging and providing
participants with insights to the various disciplines and how they relate within “Specialty Engineering”.
Objective: To assist folks in understanding how the “-ilities” integrate multiple disciplines for successful
programs and projects.
Monday // 08-04-14 // 8:00-11:20 // Regency B // TUTORIAL 0.27 CEU
Health Hazard Assessment in System Safety Evaluation
Instructor: Mark Geiger, Naval Safety Center Liaison Office, Arlington, Virginia, United States;
Rodney Simmons, PhD, CSP, The Petroleum Institute, Abu Dhabi, United Arab Emirates
Attendees will be able to: Understand and apply the basic principles of recognition, evaluation and
control to occupational health hazards common to defense and industrial settings; understand the limits
of such evaluations and be introduced to sources of assistance
1. Predict/Recognize; Recognize potential health hazards common to defense and industrial operations
2. Evaluate Understand how to conduct or evaluate Health Hazard Assessments in System Safety Evaluation consistent with Military Standard 882 Task 207
3. Risk Estimation Understand the concept of dose response relationship between occupational exposures and potential disease outcome, on a population basis. This will help users evaluate risk on the
basis of known or estimated population exposures relative to
4. Control Understand the application of the system safety hierarchy of controls to mitigation of health
hazards through optimal measures- elimination/substitution; process controls/barriers; to least protective - procedures and protective equipment.
5. Where to get help Source of information and assistance will be identified.
Short Overview: Basic principles of health hazard evaluation will be described. Use of the process
outlined in Military Standard 882, Task 207 Health Hazard Evaluation, will be illustrated with examples
21
and case studies. Basic introductory lectures on the general topic will be followed by terse specific
introduction(s) to specific areas and exercises with working group review of problems.
Abstract: The workshop will introduce participants to the basic principles of health hazard evaluation in
the context of system safety and systems engineering development and risk mitigation for development/
design of systems and equipment. Introductory lectures will be followed by class exercises reviewing
and applying the principles of hazard recognition, risk evaluation and identification of potential control
measures.
Outline:
• Introduction to health hazard assessment
• Noise- the most common health hazard
• Chemical hazards
• Physical agent hazard- ergonomics
• Ventilation
• Exercise
Tuesday // 08-05-14 // 8:00-16:50 // Grand A // TUTORIAL 0.53 CEU
Functional Hazard Analysis
Instructors: Kevin Stottlar, Naval Surface Warfare Center Dahlgren Division, Dahlgren, Virginia, United
States; Adam Scharl, Naval Surface Warfare Center Dahlgren Division, Dahlgren, Virginia, United States;
Rani Kady, PhD, Naval Surface Warfare Center Dahlgren Division, Dahlgren, Virginia, United States;
Michael Ingram, Naval Ordnance Safety and Security Activity, Indian Head, Maryland, United States
Attendees will be able to: Generate a Functional Hazard Analysis
Abstract: The Functional Hazard Analysis (FHA) Tutorial will provide an understanding of the purpose
of a FHA and the applicability of the analysis to the system acquisition lifecycle. The Tutorial provides
background on system architecture products and utilizes example architecture products to articulate the
processes necessary to perform a FHA. The Tutorial presents a methodology for identifying and mitigating
functional hazards early in the system acquisition lifecycle.
Tuesday // 08-05-14 // 8:00-11:20 // Regency A // TUTORIAL 0.27 CEU
Melding DOD and FAA System Safety Methods
Instructor: Marge Jones, Safety Analytical Technologies, Huntsville, Alabama, United States
Attendees will be able to: Tailor the MIL-STD-882E tasks and DIDs to obtain a blended process that would
have documented results that satisfy both DoD and FAA requirements
Abstract: Some DoD projects find it desirable to utilize the top-down safety process used in certification
of commercial aircraft particularly in the area of software safety. Early identification of design safety
requirements is a proven hazard mitigation technique. The tutorial will highlight the differences and
similarities between MIL-STD-882E and associated Data Item Descriptions (DIDs) with the commercial
aircraft system safety process (AC25.1309, SAE ARP4761 and ARP4754). By understanding the differences
and similarities, Statement of Work and CDRL requirements can be tailored to define a blended system
safety process that could satisfy both.
Tuesday // 08-05-14 // 8:00-11:20 // Regency B // TUTORIAL 0.27 CEU
Human Factors and Systems Safety
Instructor: Carl Sandom, PhD, iSys Integrity Limited, Sherborne, Dorset, United Kingdom
Attendees will be able to: Have an appreciation of the human factors and ergonomics issues relating to
safe systems development.
22
Short Overview: This half-day Human Factors for Safe Systems tutorial will provide an introduction to the
Abstract: Human Factors are often cited as the cause of hazards within safety-related systems; yet system
safety cases often contain no mention of them. Conversely, system operators or users often provide
substantial mitigation between hazards and their associated accidents; yet this is also often overlooked. If
human factors risks are not considered, a system will not achieve the required level of integrity. If human
factors mitigations are not considered, the technical system components may be over engineered at
additional cost to achieve a target level of safety.
Objective: The tutorial will include coverage of the following objectives: - Addressing Human Factors
issues impacting Safety engineering - Integrating of Human Factors within the systems engineering
context - Integrating of Human Factors within other Safety assurance activities
Outline:
• Human Factors and System Safety
• Human Factors
• Definition & Scope
• Human (Cognitive) Limitations
• Accident Causation and Barriers
• Human Factors Exercise
-- Avionics Module change
-- People, Procedures and Equipment
ISSTS 2014 Program
Human Factors discipline along with an overview of the scope of human factors analyses in the context of
safety-related systems engineering.
• Analyzing Human Error
• Questions & Answer Session
Tuesday // 08-05-14 // 8:00-15:20 // Regency C // TUTORIAL 0.45 CEU
New Approaches to the Cyber-Security of Safety-Critical Systems
Instructor: Chris Johnson, University of Glasgow, Glasgow, Scotland, United Kingdom
Attendees will be able to: Identify the principle threats and vulnerabilities for the cyber-security of
safety-critical systems. Address the regulatory and certification challenges posed by attacks on safetycritical software. Select appropriate architectures that maintain safety and sustain resilience in the face
of attacks on safety-critical systems. Develop a plan for the detection, mitigation and forensic analysis of
malware in a safety-related environment.
Short Overview: This tutorial will present a number of case studies in which malware has been detected
inside the operational systems of safety-critical applications in Air Traffic Management, Healthcare,
Rail and Energy distribution. We will summarize the problems that arise in maintaining application
safety once an attack has been detected. We will also summarize the main stages in the forensic
analysis of safety-critical systems, when it may not be possible to immediately halt operation without
increasing the risk to application operators or to the general public. We will also address the challenge
of convincing regulators that it is safe to continue operation after an attack. The afternoon will focus on
new techniques that increase resilience to cyber-attacks. We will focus on novel software architectures
that balance safety and security. We will also explain why conventional approaches to threat detection
- which involve the regular exchange of threat signatures, create particular problems for safety-related
applications. Some of these approaches involve heterogeneous hardware; they have to support legacy
controllers from the 1980s as well as new generations of FPGAs and ‘smart’ devices. Others focus more
on mitigating the threats from insider attacks; which are a rising source of concern given the increasing
diversification of the supply chain in many safety-related industries.
Abstract: Over the last three years, I have been involved in the forensic analysis and recovery from
a number of cyber-attacks on safety-critical systems across the aviation, healthcare and energy
distribution industries. In most cases, the malware has stemmed from problems in the supply chain -
23
where sub-contractors have failed to implement the security policies of the companies they support.
In consequence, mass-market malware has been inadvertently introduced into operational systems in
safety-critical applications either through the use of third-party libraries or through secondary memory
- including but not limited to infected USB devices. It is difficult to imagine the problems that this creates
when, for instance, it is technically impossible to be one hundred per cent sure that an infection has been
removed (more sophisticated forms of attack deliberately compromise anti-viral products). In addition,
I have worked for a range of organizations taking the first steps to protect safety-critical infrastructures
against more sophisticated advanced persistent threats. These pose enormous technical and regulatory
challenges - it is often difficult to show that anti-viral products and threat detection systems meet the
reliability requirements for safety-critical software. The aim of this tutorial is to introduce delegates to
the technical, regulatory and organizational challenges of increasing cyber-security in safety-critical
systems. We will consider the weaknesses of existing security standards that cannot easily be applied
in safety-critical applications. I will also describe a number of different approaches to cyber-security
adopted by regulators across Europe and North America. The intention is to provide an interactive forum
where participants can share their own experiences but also to introduce a number of more advanced
software architectures that are being used to increase the resilience of safety-critical applications. A
particular focus will be on mitigating the insider threat and on meeting new legislative requirements, as
a number of governments develop obligations to report cyber-attacks on safety-related infrastructures.
Objective: To show area where existing cyber-security techniques do not work for safety-critical systems.
To identify hybrid and novel approaches that increase security without undermining safety. To identify
hybrid and novel approaches that increase safety without undermining security.
Outline: Session 1: Understanding the Threats - Case studies of previous attacks in aviation, rail,
healthcare and energy distribution and their impact on safety/certification/legal liability; - What are
the vulnerabilities (ubiquitous use of Linux, IP, VOIP, slow introduction of service architectures including
the Cloud and consequent problems securing the supply chain, integration of legacy and novel devices/
systems etc); - What are the technical threats? (Spear phishing, hacktivism, Stuxnet, Duqu etc); Session
2: What Can We Do - Relevant standards and guidelines (ISO/ENISA/NIST); - Why many of the obvious
approaches don’t work in safety-critical systems (certification and regulation of security management
systems in safety-critical industries, problems of audit). - Software architectures for safety and security. New legal and regulatory obligations (cyber-incident reporting and analysis requirements in the US and
Europe) - Threat detection and Forensic techniques, - Cyber-security as a Business Opportunity in Safetyrelated industries (Future of cyber-insurance)
Tuesday // 08-05-14 // 13:30-16:50 // Regency B // TUTORIAL 0.27 CEU
Weapons Systems Software Safety Criticality and Level of Rigor (LoR)
Instructor: Douglas Bower, MS, Naval Ordnance Safety and Security
Activity, Indian Head, Maryland, United States
Attendees will be able to: Conduct of Functional Hazard Analysis of a Weapons System and Apply the
Appropriate level of rigor of software safety analysis.
Short Overview: Present a detailed overview of Hypothetical Missile System and use that system as
an example to lead the participants through the conduct of a Functional Hazard Analysis (FHA), the
determination of the criticality of the software functions, and the application of the associated level of
Rigor (LoR) of software safety analysis and testing.
Abstract: Software intensive weapons system require that the software safety practitioner be able
to determine the criticality of the system’s safety critical software functional components and apply
the commensurate level of rigor of software safety analysis and testing. This tutorial will present a
detailed overview of the design of a hypothetical missile system (intentionally differentiated from any
real system) to serve as a foundational example. The tutorial will then present the Functional Hazard
Analysis (FHA) and walk the FHA of the missile systems. The tutorial will then present the software
safety criticality assessment process and walk the determination of the criticality of the missile systems
24
Objective: Develop and greater appreciation and understanding of the application of appropriate level of
rigor (LoR) of software safety analytical techniques.
Outline: I. Hypothetical Missile System Design Overview A. Missile Structural Design Overview B. Missile
Flight Control System C. Missile Payload Content and Activation System II. Functional Hazard Analysis
(FHA) Methodology III. Software Safety Criticality Determination (MIL-STD 882E) A. Control Categories
/ Mishap Severity B. Criticality Levels / Level of Rigor (LoR) C. Documentation of Application of LoR
IV. Application of Methodology to Missile System A. Conduct of the FHA B. Determine Software Safety
Criticality / LoR required C. Apply the LoR and Document the Results
Wednesday // 08-06-14 // 8:00-16:50 // Grand A // TUTORIAL 0.53 CEU
ISSTS 2014 Program
safety significant functions and required level of rigor. (LoR). Finally, the tutorial will walk through the
documentary evidence required to substantiate the application of the appropriate LoR.
Software Safety Analysis
Instructors: Peggy Rogers, Naval Ordnance Safety and Security Activity, Indian Head, Maryland, United
States; Michael Zemore, Naval Surface Warfare Center, Dahlgren Division, Dahlgren, Virginia, United
States; Stuart Whitford, Naval Ordnance Safety and Security Activity, Indian Head, Maryland, United
States; Carolyn Tilghman, Naval Surface Warfare Center Dahlgren Division, Dahlgren, Virginia, United
States; Rebecca Funkhouser, Naval Surface Warfare Center, Dahlgren Division, Dahlgren, VA, United States
Attendees will be able to: Understand the Software Safety Analysis process.
Abstract: This Tutorial provides the participant with a “hands on” learning experience in understanding
the overall process for performing Software (S/W) Safety Analysis. The S/W Safety Analysis and
Verification Process, S/W Criticality Matrix and Level of Rigor (LOR) tasks are introduced. Understanding
the purpose of the S/W Criticality Matrix and LOR will help the participant become familiar with specific
analyses and tests that are recommended based on the required LOR. S/W Safety Analysis results in
improving designs and reducing the likelihood that S/W will initiate a hazardous condition or mishap.
Objective: Understand the Software Safety Analysis process.
Wednesday // 08-06-14 // 8:00-11:20 // Regency B // TUTORIAL 0.27 CEU
Hands-On System Safety Basics, Focused on FHA
Instructors: Lukas Fritz, PhD, Frequentis AG, Vienna, Austria;
Gabriele Schedl, Frequentis AG, Vienna, Austria
Attendees will be able to: 1. Understand the role of the safety lifecycle within the project lifecycle 2.
Understand some major safety analysis techniques 3. Perform a Functional Hazard Assessment
Abstract: An overview of a generic safety process, best suited for small to medium sized projects, in
relation to the project lifecycle, is given. For each major project phase the respective safety process
phase, safety objectives, necessary in- and outputs are detailed. Some state of the art analysis techniques
are explained. Special emphasis is put on the Functional Hazard Assessment, where a practical guidance
for a Functional Failure Modes and Effects Analysis is presented. The content of this tutorial is based on
experience from an international working company.
Outline:
• Basic Definition
• Safety Process
• Safety Requirements
• Techniques/Methods
• Case Study
25
Wednesday // 08-06-14 // 13:30-15:30 // Regency A // TUTORIAL 0.2 CEU
The STAMP Model, STPA Hazard Analysis and CAST Accident Analyses
Instructor: Robert Fletcher, MSc, US Postgraduate School, Royal Military College, Ottawa, Ontario, Canada
Attendees will be able to: Understand the format of the STAMP Model and the STPA hazard analysis
process and perform CAST accident analysis. Learn the CAST process including; Identify the Accident
(Loss), Identify the Hazards, Identify the Proximal Events, Draw the Safety Control Structure, Analyze
each component of the Physical System and the Controllers
Short Overview: System-Theoretic Process Analysis (STAMP), Systems Theoretic Process Analysis (STPA)
and Causal Analyses Based on STAMP (CAST) area powerful hazard analysis methods designed to go
beyond traditional safety techniques-such as Fault Tree Analysis (FTA)-that overlook important causes of
accidents like flawed requirements, dysfunctional component interactions, and software errors. Although
traditional techniques have been effective at analyzing and reducing accidents caused by component
failures, modern complex systems have introduced new problems that can be much more difficult to
anticipate, analyze, and prevent. In addition, a new class of accidents, component interaction accidents,
has become increasingly prevalent in today’s complex systems and can occur even when systems operate
exactly as designed and without any component failures.
Abstract: System-Theoretic Process Analysis (STAMP), Systems Theoretic Process Analysis (STPA) and
Causal Analyses Based on STAMP (CAST have proven to be effective at addressing problems; however,
application has been ad-hoc with no rigorous procedures or model-based design tools to guide the
analysis. In addition, although no formal structure has yet been defined for STPA and CAST, the process is
based on a control-theoretic framework that could be formalized and adapted to facilitate development
of automated methods that assist in analyzing complex systems. This dissertation defines a formal
mathematical structure underlying CAST and STPA and introduces a procedure for systematically
performing an STPA and CAST analysis based on that structure. A method for using the results of the
hazard analysis to generate formal safety-critical, model-based system and software requirements is also
presented. Techniques to automate both the STPA and CAST analysis and the requirements generation
are introduced, as well as a method to detect conflicts between safety requirements and other functional
model-based requirements during early development of the system.
Outline: 1. STAMP 2. STPA 3 CAST
Wednesday // 08-06-14 // 13:30-16:50 // Frisco/Burlington // TUTORIAL 0.27 CEU
Fault Tree Analysis with CAFTA
Instructors: Jean-François Roy, Electric Power Research Institute, Palo Alto, California, United States
Abstract: This tutorial will introduce Fault Tree Analysis using CAFTA Software. Attendees will be first
reviewing fault tree methodology and terminology. Construction of a fault tree model in CAFTA will
then follow a brief review of CAFTA’s components and symbol types. In constructing the Fault Tree
model, topics covered will include projects, navigation, editing, shortcuts and how to add probabilities.
An overview of basic event probability formulas, type codes and variables will be included, as well as
printing and quantification processes.
Thursday // 08-07-14 // 8:00-11:20 // Grand A // TUTORIAL 0.27 CEU
Practical Generation of Safety Cases With the Help of GSN
26
Instructors: Andreas Gerstinger, PhD, Frequentis AG, Vienna, Austria;
Gabriele Schedl, Frequentis AG, Vienna, Austria
Attendees will be able to:
• Understanding of the concept of safety cases o Understanding of the benefits and potential pitfalls of
safety cases
• Overview of Goal Structuring Notation (GSN) o Ability to read GSN
• Ability to create simple arguments in GSN
Abstract: Detailed outline of the tutorial: Introduction (1h): The tutorial will start with a survey of
current safety standards (IEC 61508, ISO 26262, EN 50128, DO-178C,...) and analyse their views and
requirements regarding safety cases. We will then delve into the nature of safety cases, briefly touch
their historical origins, and clearly consider what can and what can’t be expected from a safety case.
Based on our practical experience we will also highlight some typical bad practices when constructing
safety cases. This helps to correctly and critically read them, and is also a helpful guideline for reviewing
other safety documentation. This part of the tutorial is largely a presentation. Goal Structuring Notation
(45min): We will now introduce the main elements of the Goal Structuring Notation (GSN), which is a
helpful tool to document safety cases. The presentation of the notation will be interleaved with brief
examples, excercises and questions, so that attendees have the chance to fully understand the meaning
and purpose of the various symbols. A structured method how to proceed when drafting safety cases will
also be presented. Hence, this part of the tutorial is much more interactive, requiring active participation
of attendees. Case Study (45min): A realistic case study will then be handed out. It is expected to be
solved as a group work (groups of 3-5 people are expected). The task of the groups will be to draft and
present a sound safety argument for a given claim that the system in the case study is acceptably safe
for a specific application in a given environment. GSN shall be used as a notation for this purpose. At the
end, the groups present their solutions, and the advantages/disadvantages of the presented solutions are
discussed. This part of the tutorial is a group work. Concluding Remarks (30min): Finally, we will bring
some concluding remarks, consisting of hints how to avoid common errors and fallacies in safety cases,
show some examples of real-world safety cases and a we will finish with a personal conclusion.
ISSTS 2014 Program
Short Overview: This tutorial will introduce you to the concept of safety cases. Safety cases are
structured arguments that support the claim that a system is safe to be used for a given application in
a given environment. Several standards require the production of such safety cases as a prerequisite
for approval. The tutorial will highlight good and bad practices when developing safety cases and will
introduce you to a notation specifically developed for the generation of safety cases, the Goal Structuring
Notation (GSN). There will be practical examples which need to be solved by the attendees, so that handson practice and experience is gained.
Objective: This tutorial will introduce you to the concept of safety cases. Safety cases are structured
arguments that support the claim that a system is safe to be used for a given application in a given
environment. Several standards require the production of such safety cases as a prerequisite for approval.
The tutorial will highlight good and bad practices when developing safety cases and will introduce you to
a notation specifically developed for the generation of safety cases, the Goal Structuring Notation (GSN).
There will be practical examples which need to be solved by the attendees, so that hands-on practice and
experience is gained.
Outline:
• Introduction
• Goal Structuring Notation
• Case Study
• Conclusion
Thursday // 08-07-14 // 8:00-8:50 // Regency A // TUTORIAL 0.1 CEU
Joint Munitions Safety Testing Initiative
Instructors: Eric Hawley, Naval Ordnance Safety and Security Activity, Indian Head, Maryland,
United States; Diane Dray, Booz Allen Hamilton, Arlington, Virginia, United States
Attendees will be able to: Understand the origin and goals of the Joint Munitions Safety Testing
Initiative (JMST). They will be able to access the published Joint Ordnance Test Procedures (JOTP) and
Allied Ammunition Safety and Suitability for Service Publications (AAS3P) and will understand how
these documents should be used in the current joint weapons acquisition environment. They will also
be familiar with the JOTP process and how JOTPs can be used for standardization by other functional
Department of Defense communities.
27
Short Overview: This tutorial will present the origins and goals of the Joint Munitions Safety Testing
Initiative. The tutorial will cover the established development and implementation process. Presenters
will go over completed publications, current development efforts, and future work, and how these
products should be implemented in DoD weapons acquisition. Opportunities to other DoD functional
communities offered by the JOTP process will also be covered.
Abstract: Department of Defense (DoD) acquisition has challenges in safety testing of munitions
that include duplicate or overlapping standards and multiple applications of the same standard by
different Services. The Under Secretary of Defense (OUSD) for Acquisition, Technology and Logistics
(AT&L) assigned an action to standardize basic engineering rationale to safety testing to achieve
data acceptability and interchangeability between the Services and US Special Operations Command
(USSOCOM). There already existed a Defense Safety Oversight Council (DSOC)-funded initiative, entitled
Joint Service Safety Testing Requirements (JSSTR), which was looking at implementation of the Joint
Requirements Oversight Council (JROCM 2005) directive that recognizes all weapon / explosive systems
as categorically joint. To meet these challenges, several interwoven initiatives to enhance support
to USSOCOM and the Joint Warfighting Environment have been established. These initiatives enable
collaboration on joint weapons safety reviews, integrating joint weapons safety requirements in Joint
Capabilities Integration and Development System (JCIDS), and developing joint service weapons safety
testing standards. Early phases of JSSTR validated the need among Stakeholders, published the AT&L
Study, and championed the study recommendations by the Joint Weapons Safety Technical Advisory
Panel Council (Navy-NOSSA Chair). Study recommendations were institutionalized by achieving joint
stakeholder consensus on a list of system independent environmental safety tests for use by Sponsors
in drafting JCIDS documents. Under the auspices of the NATO AC326 Subgroup 3, the Working Group for
the Development of Safety and Suitability for Service (S3) documents developed STANAG 4629 “Safety
for Service Assessment Testing of Non-Nuclear Munitions,” a standardization agreement paving the
way for improved munitions type-specific joint test standards. The S3 Working Group efforts were
complimentary with the JSST recommendations, and the efforts were linked in a dual path domestic
and international process, referred to as JMST. JMST has transitioned from a DSOC initiative and its
deliberations and product now fall under the purview of the Joint Weapons Safety Working Group
(JWSWG). The NATO S3 Working Group develops Allied Ammunition Safety and Suitability for Service
Publications (AAS3P) for munition commodity groups. The U.S. joint working group publishes a
commodity-specific corresponding Joint Ordnance Test Procedure (JOTP) for each AAS3P. The JOTPs
are designed to be retired when the AAS3P is ratified. A MIL-STD may also be used as an alternative
to eventually replace the JOTPs. The end result will be a more streamlined process for developmental
weapons safety testing involving fewer tests using fewer and non-overlapping standards. This, in turn,
will help to shorten the acquisition cycle for weapons. The JOTP has also been recognized and used as
an efficient standardization method by other functional groups such as the Explosive Ordnance Disposal
community and the Fuze community. It provides an opportunity for these groups to quickly publish and
begin using joint standards, when they may not exist or current standards are insufficient, and updates
to the existing standards are not imminent.
Objective: Familiarize participants with the background and goals of the Joint Munitions Safety
Testing Initiative and provide tools
and references for applying the joint
requirements developed to date and
potentially the JOTP standardization
process.
28
Monday // 08-04-14 // 14:30-16:30 // Regency C // Panel
Exploring Society History as a Guide for its Future Course
Includes historical notes and an open panel discussion
Moderator, Rex B. Gordon, CSP, P.E. Fellow Emeritus, Society Historian
A panel session, with audience participation solicited, will be convened. John Frost and Dr. Rod Simmons
will join Rex Gordon in leading this historically oriented discussion. Hopefully, this will lead to more fully
exploring the substance of these opposing viewpoints, with an eye toward the optimum future course for
the ISSS.
ISSTS 2014 Program
Panel Discussions/Forums
Last year at the conference in Boston, Rex presented the various historical factors that all came to gather
in Los Angeles which resulted in the charter meeting of our society on December 4, 1963. This year,
Rex will continue the chronicle with a brief review of the key movers and shakers that helped bring the
society off “Life Support” and into the “Players” room. There were then, and still are, many players in
the arena of fixing safety problems – each with solutions for some aspect or another of the complex, and
ever evolving accident prevention milieu. It so happens that in the 1960s minimizing the catastrophic
mishaps, then occurring in the cold war environment of inter-continental, nuclear tipped, ballistic
missiles, was a paramount issue for the U.S. Air Force. This had a significant influence on the nature and
subsequent evolution of the system safety concept, the promotion of which was the essential purpose for
forming the society. This environment was a driving force behind the development of MIL-STD-882, and
also some unintended limitations in the area of innovations to meet new challenges currently evolving in
today’s world.
Some current and former members of the society have questioned the validity of spending time
reviewing the past, when today’s technical and social environment is so removed from that of the 1960s.
They would have a valid case, if the past had no lessons to provide toward navigating the future course
of the society. However, others take the position that understanding the fundamental principles of the
system safety concept, and how they came into existence, is an essential key plotting the future course
of the society. However, others take the position that understanding the fundamental principles of the
system safety concept, and how they came into existence, is an essential key in plotting the future course
of the International Systems Safety Society.
Thursday // 08-07-14 // 8:00-11:20 // Regency C // Panel
G-48 Workshop – The Most Pressing Issues Facing System Safety
Moderator: David B. West, CSP, P.E., CHMM, Systems, Software, and Solutions Operation,
Science Applications International Corporation (SAIC), Huntsville, Alabama, United States
A panel of speakers (TBD) will give a series of presentations on the most pressing issues facing the System
Safety community today.
Thursday // 08-07-14 // 13:00-15:20 // Regency C // Panel
Developing Global System Safety Perspectives
Moderator (s): Warren Naylor, NGES Sr. System Safety Consulting Engineer (United States)
and Robert W. Fletcher, P.Eng., PMP, PCIP (Canada), ISSS International Director
There will be a brief presentation followed by a facilitation session where the chair will ask the audience
to respond to preselected questions such as:
• How can a Global Federation of System Safety Societies and Associations be developed? and
• How can an International Standard for certifying a professional International System Safety Engineer
and International System Safety Manager be developed?
29
Workshops
Tuesday // 08-05-14 // 8:00-8:50 // Grand C // WORKSHOP
Creating A Culture of Chronic Unease
Presenters: Laurence Pearlman, BBA, MA, Oliver Wyman, Chicago, Illinois, United
States; Susie Scott, BS ChE, Oliver Wyman, Chicago, Illinois, United States
Upon completion attendees will be able to: - Understand what Chronic Unease means - Recognize
leadership behaviors that support chronic unease - Understand behavioral ‘traps’ that prevent ourselves
from acting consistently with the desired culture - Evaluate how well leaders welcome bad news - Pro
Short Overview: This session will help safety leaders move the needle on culture from ‘we haven’t had an
incident, things must be going well’ to ‘we haven’t had an incident, I wonder what might be sneaking up
on me.’ - Understand what Chronic Unease means - Recognize leadership behaviors that support chronic
unease - Understand behavioral ‘traps’ that prevent ourselves from acting consistently with the desired
culture - Evaluate how well leaders welcome bad news - Provide discussion on how to ‘act on reds and
yellows and challenge greens’
Abstract: A state of “Chronic Unease” is achieved when leaders at all levels have created a culture where
they are made aware of weak signals of potential failure, and make effective and timely challenges and
interventions on risk assessments and decision making. It starts with openness, where we welcome bad
news and treat incidents as an opportunity to learn. When out on the site we ask the right questions and
pick up on signals of potential failure. It is:
• An Outcome and a feeling of never being complacent
-- from living a set of leadership behaviors
-- aimed at sharpening our focus (reducing our tolerance) to risk
• NOT a program and NOT about our frontline staff living in a state of fear
Tuesday // 08-05-14 // 9:00-9:50 // Grand C // WORKSHOP
What Does it Take to Really ‘Learn from Incidents’ Lessons get shared, but sharing is not learning
Presenters: Laurence Pearlman, BBA, MA, Oliver Wyman, Chicago, Illinois, United
States; Susie Scott, BS ChE, Oliver Wyman, Chicago, Illinois, United States
Upon completion attendees will be able to: Understand the difference between learning and sharing
incidents Understand ways to prioritize what gets shared Tips for communicating incidents What it takes
to learn from incidents Overcoming asset owner pride Making it sustainable
Abstract: Typical issues encountered by organizations while implementing LFIs - Trying to learn
from everything vs. a focused, risk based prioritization approach - Ambiguity to whom investigation
outcomes and lessons learned are applicable - Encountering organizational barriers to learning (e.g.,
culture, structure, roles, etc.) - Ineffective and/or inactive leadership drive and engagement - Focus on
compliance vs. commitment - Misalignment of what is said vs. the what actually happens in the field (the
‘Say-Do’ gap) - Overcomplicated and ineffective safety communication - Inability to identify the true root
causes - Lack of systems to properly capture and disseminate lessons - Lessons fail to result in revised
documentation of standards and procedures for proper re-use This session will focus on how to achieve
the benefits of successful learning: Reduce Risk - Lower overall organizational risk tolerance and incident
rate - Effective performance management systems and continuous improvement processes - Active audit
programs, identified safety barriers and robust safety-critical activities - Prompt investigation of issues
with focus on determining the true root cause - Immediate, underlying and latent cause (s), including
human/behavioral factors Committed Leadership & Organization - Leaders demonstrate a commitment
to safety and learning in their decisions and behaviors - Clearly defined basis from which to learn as well
as roles/accountabilities Impactful Knowledge Management - Effective safety communication - Simple,
memorable and visual communication - Effective systems for capturing, codifying, storing and retrieving
30
Tuesday // 08-05-14 // 10:30-11:20 // Grand C // WORKSHOP
Best Practices in Process Safety Culture - Lessons from the Energy Industry
Presenters: Laurence Pearlman, BBA, MA, Oliver Wyman, Chicago, Illinois, United
States; Susie Scott, BS ChE, Oliver Wyman, Chicago, Illinois, United States
Upon completion attendees will be able to: Understand best practices in the energy industry for driving
process safety culture Identify leadership as a core component for process safety culture Understand how
learning and employee roles are vital parts of establishing a process safety culture
Short Overview: This session shares industry best practices for establishing a process safety culture. It
will discuss how to align senior leaders to a common definition through metrics and how to sustain a
safety culture
ISSTS 2014 Program
lessons Lessons learned are auditable and identifiable across the organization Extensive networking,
crossing organizational boundaries to develop knowledge.
Abstract: This session is meant to provoke participants to assess their process safety culture by reviewing
a few fundamentals of culture and understanding what ‘good’ looks like. Participants will be encouraged
to use small group discussion to provoke thinking.
Objective: Understand best practices in the energy industry for driving process safety culture Identify
leadership as a core component for process safety culture Understand how learning and employee roles
are vital parts of establishing a process safety culture.
Outline of Session:
Definitions Committed leadership Capabilities Measurement Systems Continuous Improvement Change &
Communications
Tuesday // 08-05-14 // 13:30-16:50 // Regency A // WORKSHOP
Human and Organizational Performance (HOP) Fundamentals
Presenter: Robert “Bob” Edwards, GE, McMinnville, Tennessee, United States
Upon completion attendees will be able to:
• Understand HOP principles and have a working knowledge of the basic terms and concepts
• Apply HOP principles to their change their site management’s response to failure
• Conduct HOP Learning Teams for event investigations & to solve operational challenges
Short Overview: General Electric, with the coaching of Todd Conklin is undergoing a transformation. By
embracing the concepts of Human and Organizational Performance, the leadership teams at many GE
sites are realizing that there is a better way to respond to failure. They are becoming a lot less surprised
by human error and failure and they are now becoming a lot more interested in learning. The employees
at the GE sites that are undergoing this transformation are beginning to talk more openly than ever
before about their complex work environment and what it actually takes to get work done. As managers,
front line leaders and workers come together in a blame free, open environment, the real stories are
being told, and solution sets created by these teams are more thorough, effective and sustainable. This is
a journey worth taking and one worth sharing.
Abstract: This is a workshop to teach the fundamentals of General Electric’s Human and Organizational
Performance (HOP) initiative. Attendees will learn these HOP principles and see how GE and other
companies are applying these principles through Todd Conklin’s Learning Teams to gain deeper
operational intelligence. See the Learning Team process and witness real life examples from
manufacturing sites in GE’s Appliances, Lighting and Aviation divisions. Attendees will be able to
take home a very practical approach to try at their sites that will help them change their response to
failure, conversation about failure and ability to learn from failure. The workshop is very interactive
with several simulations and the knowledge gained can be applied immediately to improve response to
31
safety and quality events, near misses, and operational upsets and to help solve challenging design and
process issues. Bob Edwards has experience leading more than 200 HOP Learning Team sessions and
has a proven track record for bringing operational learning to a new level. Attendees will enjoy his high
energy presentation style and practical and inclusive approach to operational discovery and problem
solving. The skills gained from this workshop will blend perfectly with safety, lean, six sigma, concurrent
engineering and quality programs.
Activities: Two Learning Team simulations and several interactive activities.
Objective:
• Learn the basics terms and concepts
• Apply concepts to simulations
• Take back enough information, training and confidence to give Learning Teams a try at their site.
Outline of Session:
• Introductions
• GE HOP Journey
• HOP Principles
• Simulations
• Closing Comments
Thursday // 08-07-14 // 8:00-10:00 // Regency B // WORKSHOP
System Safety Handbook Working Group
Moderator: Chuck Muniak will lead the discussion. The meeting will be open to all.
Topics for Handbook Working Group Meeting
1. Summary of comments on the draft handbook
2. Discussion of the topics and depth of knowledge needed for certification
3. Discussion of proposed references that could be used as a source for certification exam questions and
other references that should be included in the handbook
Thursday // 08-07-14 // 13:30-16:50 // Grand A // WORKSHOP
Systems-of-Systems (SoS) Workshop on Framework, Collection, Processing,
and Organizing for System Safety and Software Safety
Presenters: Archibald McKinlay, MS, St Louis Regional Federal Executive Board, St Louis,
Missouri, United States; John Murgatroyd, BS, Team BCI, Dahlgren, Virginia, United States
Upon completion attendees will be able to: Layout a program-specific framework to outline SoS
attributes
Short Overview: Attendees will be taught how to begin a systems-of-systems (SoS) safety program by
assisting mission engineers and modelers in the collection, collation, processing, and organizing of SoS
attributes. These will be arranged within a program-specific framework such as one needed to (a) handoff to mission engineers and agent modelers, and (b) start analyses, or (c) build an SoS test plan. A basic
systems-of-systems model will be used. Basic understanding of what is required to trace safety from
mission and performance requirements will be highlighted.
Abstract: The workshop is intended to explain preparations and understanding of complex, adaptive,
systems-of-systems so that subsequent safety arguments, analyses, and tests are all developed from
the same context and data throughout the program in work, and to enable informed discussions across
systems and SoS boundaries. The size and complexity of systems-of-systems (SoS) requires a framework
from which to begin. These frameworks consist of building blocks but are arranged in program-specific
and in a context-specific manner. The framework from this workshop and from any field work is only the
32
ISSTS 2014 Program
set-up to enable modeling, analyses, and SoS test planning and will not be the SoS analyses themselves.
That is, the attendee will be taught how to collect those inputs for many engineers to use thereafter in
many SoS taskings. The context generally is derived from Concepts of Operations (CONOPS), Concepts
of Employment (CoE), and adjusted using experience with related systems, doctrines, current missions
and training programs. The building blocks, or swim lanes, concepts will be explained and used to build
out this workshop SoS framework. The collection of operational, system, and user data and artifacts will
be mapped out and example data arranged within the framework, and subsequently used to modify the
framework. The data within the framework will then be processed to assess completeness. A discussion
of weaknesses and strengths of the model, the collection processes, and the arrangement itself will
follow. Next steps will be discussed and how the next steps are impacted by the amount and types of data
within this framework. Preparations for modelers and mission engineers will be continuously assessed.
At the end of the workshop a lengthy question and answer period will be provided. No quiz or test will be
required.
Objective: Understand how to set up for a Systems-of-Systems program.
Paper Presentations
Monday // 08-04-14 // 8:00-11:20 // Grand B // Safety Topics 1 // Chair: Gonzalez
Mishap Prevention Utilizing Unlimited & Government Purpose Data Rights
Francis McDougall, MS System Engineering, BA Physics, Los Angeles
Air Force Base, El Segundo, California, United States
Assessing, reporting, tracking, accepting and documenting mishap risk using MIL-STD-882 methodology
is required by multiple DOD and Air Force policies. In order to ensure mishap risk tracking databases
are government and contractor accessible during the entire lifecycle of space systems, Air Force has to
maintain access to intellectual property data. The optimum method to do this is through negotiated
unlimited data rights while government funds analysis of alternatives, preliminary design, critical
design, test and evaluation, production and sustainment. In several cases unlimited data rights to
contractors’ intellectual property does not occur due to costs, the contractors’ assertion to retain
intellectual property and through acquisition of commercial off the shelf products integrated into space
systems via acquisition through open architecture. When unlimited data rights are not feasible, practical
or cost prohibitive then Program Managers should acquire government purpose data rights. This will
ensure effective transfer of mishap risk tracking databases linked to contractor intellectual property
to different contractors involved with Air Force space systems throughout acquisition and sustainment
lifecycles; thereby, ensuring consistent and effective mishap prevention programs.
Practical Insights for the Exchange of Leading Practices Lessons Learned in
Accident Investigation and Lessons Learned from Incident Investigations
Chris W. Johnson, School of Computing Science, University of Glasgow, Glasgow, United Kingdom;
Susan Reinartz and Michael Rebentisch, European Railway Agency, Valenciennes, France
Accidents are rare events across the transportation industries of Europe and North America. It is,
therefore, important that we learn as much as possible from small numbers of adverse events and
near miss incidents. This depends on the skill and expertise of accident investigators. Although a large
number of previous studies report on novel approaches to accident investigation, very few describe the
tools and techniques that investigators themselves would recommend to other investigation agencies. In
contrast, this paper will summarize the findings of a joint study between the European Railway Agency
and the University of Glasgow to encourage the exchange of lessons learned from investigations and to
encourage best practices in incident investigation across national borders.
33
System Safety Challenges in High Energy Laser (HEL) Weapon Systems
Martin S. Chizek, P.E., C.S.P., Raytheon Missile Systems, Tucson, Arizona, United States
Lasers are rapidly being implemented in modern weapon systems, either as additions to conventional
weapons platforms, or as stand-alone devices deployed on land and from aircraft and ships. As
technology has allowed for High Energy Laser (HEL) output from ever smaller packaging footprints, laser
missions have progressed from traditional range-finding and target designation to actively degrading
or destroying hostile sensors, missiles, aircraft and even surface craft. This paper explores the unique
hazards associated with operating and maintaining HELs, the commercial and military standards and
safety requirements to manage these hazards, and the residual risks that will inevitably remain after
reasonable mitigation measures have been implemented. Examples of current laser weapon systems
will be presented, and safety features, to include hardware, software and human interactions, will be
suggested to provide an effective HEL safety program. Finally, the international legal and regulatory
environment will be explored, and the user’s potential liability in deploying HELs will be discussed.
Monday // 08-04-14 // 8:00-11:20 // Grand C // Hazard Analysis 1 // Chair: Rose
Cases For Tailoring The MIL-STD-882E Risk Matrix For US Air Force Space & Launch Vehicles
Myles Moran, BS in Mechanical Engineering, Los Angeles Air Force Base, El Segundo,
California, United States; Tyrone Jackson, BS in Electrical Engineering & Certified
Reliability Engineer, ManTech, El Segundo, California United States
Tailoring the MIL-STD-882E 4x5 mishap risk matrix for specific US Air Force Space and Launch Vehicle
applications is subject to approvals or perhaps waivers from the Office(s) of Primary Responsibility
(OPRs) for those DoD and Air Force Instructions that specify the use of MIL-STD-882E. The SMC System
Safety organization (SMC/SE) has identified several Air Force policies and regulations which limit
making changes to three broad categories of MIL-STD-882E risk assessment metrics, i.e. mishap severity
category definitions; probability threshold definitions; and Risk Assessment Codes (RACs) aka Hazard
Risk Index (HRI) levels with associated risk acceptance authorities. During the past four years, there has
been an upsurge in instances of SMC program managers complying with their Air Force Instruction (AFI)
mandated responsibility to prepare a written safety risk assessment for spacecraft component-level
risks requiring Component Acquisition Executive (CAE) or Program Executive Officer (PEO) acceptance.
This upsurge involves “late-coming” component-level risks that were “perhaps” implicitly accepted
as part of system-level reliability requirements that were approved in the acquisition strategy many
years ago. These “late- coming” component-level risk assessment packages are becoming a significant
administrative burden on all management levels of the Air Force space systems acquisition process, but
Air Force policy may prevent affected program managers from implementing the easiest solution to this
problem, which would be to waive their mandated responsibility to identify component-level hazards
whenever the approved acquisition strategy includes quantified system-level reliability requirements.
The second easiest way to solve this problem would be to “tailor down” the MIL-STD-882E risk metrics
definitions to make them less stringent. However, making risk metrics definitions less stringent for
death, disability, injury, occupational illness, or environmental hazards would probably be met with
significant social backlash. So “tailoring down” MIL-STD-882E risk metrics must be done carefully else it
may affect death, personnel or environmental hazard assessments. This paper examines the advantages
and disadvantages of four different options for “tailoring down” the MIL-STD-882E risk assessment
metrics. These options correspond to tailoring the mishap severity category definitions; tailoring the
probability threshold definitions; tailoring the HRI levels; and tailoring the risk acceptance authorities.
The reader is left with choosing which option is best for their particular situation.
MIL-STD-882E, A Near-Miss About System Safety
David R. Sadler, BSME EE, Naval Surface Warfare Center, Dahlgren
Division, Dahlgren, Virginia, United States
Complex, procedurally-driven, or tightly coupled systems are inherently at risk of mishaps. Even the
best system safety programs embedded within the design efforts still produce systems that demonstrate
hazards via catastrophic mishaps, such as the Titan II missile. It is important, therefore, that we take
34
Accurate Risk Assessment using Multi-Relational Hazard/Mishap Pairings
Regina Eller, Naval Surface Warfare Center, Dahlgren Division, Dahlgren, Virginia, United States;
Michael Zemore, Naval Surface Warfare Center, Dahlgren Division, Dahlgren, Virginia, United States;
Rani Kady, Naval Surface Warfare Center, Dahlgren Division, Dahlgren, Virginia, United States
Current methods for defining safety risk force a single “worst case” assessment. Unfortunately, the
worst case assessment approach fails to capture the complexity of hazard/mishap relationships or
the reality of multiple effects from any given mishap. This failure limits a complex risk picture to be
characterized using only a portion of the relevant safety engineering and assessment data. This paper
describes focused research to suggest an innovative methodology for defining risk associated with
multiple hazard contributors and multiple effects. Research focus areas included defining complex
hazard path relationships, mathematical calculations of risk, and the development of requirements for
a relational engineering tool. The combined research and results is intended to transform MIL-STD-882E
based mishap risk assessment from a worst case into a complex multi-contributor risk definition and
accurately characterize risk using combined effects of personnel injury, equipment/property damage and
environmental damage. Ability to identify significant risk factors as those that influence multiple mishap
scenarios enhance the ability to focus on the most influential risk factors to gain substantially more
safety risk mitigation.
ISSTS 2014 Program
advantage of all involved to help identify potential hazards once a system is fielded. One untapped source
of information is the result of a process known as near-miss reporting. A near-miss is an event that
signals a system weakness that, if not remedied, could lead to a future mishap. A near-miss process is how
the industrialized world manages such near-miss events. A near-miss process works to identify hazards
prior to a mishap, as opposed to the reactive method of developing a mishap report after the event,
and then blaming someone. There are numerous national and international peer-reviewed publications
discussing the value of the near-miss process and identifying the methodology. Military Standard (MILSTD)-882E does not establish the “how” but it does identify the “what” of system safety. It is difficult to
understand why MIL-STD-882E does not specifically define and identify a near-miss process as a “what”
to be done.
Tuesday // 08-05-14 // 8:00-11:20 // Grand B // Aviation Safety // Chair: Liming
Aviation Safety Risk Modeling: Lessons Learned from Multiple Knowledge Elicitation Sessions
James Luxhoj, Ph.D., LCR, Somerset, New Jersey, United States; Ersin Ancel, Ph.D.; National
Institute of Aerospace, Hampton, Virginia, United States; Lawrence Green, Ann Shih,
Ph.D., Sharon Jones, Ph.D., NASA Langley Research Center, Hampton, Virginia, United
States; Mary Reveley, NASA Glenn Research Center, Cleveland, Ohio, United States
Aviation safety risk modeling has elements of both art and science. In a complex domain, such as the
National Airspace System (NAS), it is essential that knowledge elicitation (KE) sessions with domain
experts be performed to facilitate the making of plausible inferences about the possible impacts of future
technologies and procedures. This study discusses lessons learned throughout the multiple KE sessions
held with domain experts to construct probabilistic safety risk models for a Loss of Control Accident
Framework (LOCAF), FLightdeck Automation Problems (FLAP), and Runway Incursion (RI) mishap
scenarios. The intent of these safety risk models is to support a portfolio analysis of NASA’s Aviation
Safety Program (AvSP). These models use the flexible, probabilistic approach of Bayesian Belief Networks
(BBNs) and influence diagrams to model the complex interactions of aviation system risk factors. Each
KE session had a different set of experts with diverse expertise, such as pilot, air traffic controller,
certification, and/or human factors knowledge that was elicited to construct a composite, systems-level
risk model. There were numerous “lessons learned” from these KE sessions that deal with behavioral
aggregation, conditional probability modeling, object-oriented construction, interpretation of the safety
risk results, and model verification/validation that are presented in this paper.
35
DO-278A Impacts on Legacy MIL-STD-882E Air Traffic Management Programs
Ronald J. Bartos, PE, CSP, Raytheon Company, Sudbury, Massachusetts, United States;
Enrique Oviedo, Ph.D., Raytheon Company, Tucson, Arizona, United States
Most of the System Safety Programs for US and foreign civilian and military Air Traffic Management
(ATM) systems were developed in the past under MIL-STD-882E Standard Practice System Safety. Many
new or updated ATM systems are now being tasked with complying with DO-278A Software Integrity
Assurance Considerations for Communication, Navigation, Surveillance and Air Traffic Management
(CNS/ATM) Systems in their contracts. These new programs need to understand the cost and technical
impacts of DO-278A to not only the System Safety Program, but also to other areas of development such
as Systems Engineering, Software Engineering, and Test/Validation Engineering. The system safety
professionals need to understand how the DO-278A requirements compare to the MIL-STD-882E tasks
to take the lead on DO-278A certification for their systems. There are more and different tasks and
deliverables required under DO-278A compared to MIL-STD-882E. This paper highlights the additional
efforts required under DO-278A for a program to be certified to DO-278A.
Modeling Increased Complexity and the Reliance on Automation:
FLightdeck Automation Problems (FLAP) Model
Ersin Ancel, Ph.D., National Institute of Aerospace, Hampton, Virginia, United States;
Ann Shih, Ph.D., NASA Langley Research Center, Hampton, Virginia, United States
This paper highlights the development of a model that is focused on the safety issue of increasing
complexity and reliance on automation systems in transport category aircraft. Recent statistics show
an increase in mishaps related to manual handling and automation errors due to pilot complacency
and over-reliance on automation, loss of situational awareness, automation system failures and/or pilot
deficiencies. Consequently, the aircraft can enter a state outside the flight envelope and/or air traffic
safety margins which potentially can lead to loss-of-control (LOC), controlled-flight-into-terrain (CFIT),
or runway excursion/confusion accidents, etc. The goal of this modeling effort is to provide NASA’s
Aviation Safety Program (AvSP) with a platform capable of assessing the impacts of AvSP technologies
and products towards reducing the relative risk of automation related accidents and incidents. In order
to do so, a generic framework, capable of mapping both latent and active causal factors leading to
automation errors, is developed. Next, the framework is converted into a Bayesian Belief Network model
and populated with data gathered from Subject Matter Experts (SMEs). With the insertion of technologies
and products, the model provides individual and collective risk reduction acquired by technologies and
methodologies developed within AvSP.
Tuesday // 08-05-14 // 13:30-16:50 // Grand B // Safety Topics 2 // Chair: Atencia-Yépez
Defining Layered Safety Concepts based on Open System Architectures as Foundation
for Multi-Suppliers to Develop Interoperable Safety Critical Systems
Fenggang Shi, PhD; Douglas Ailey, P.Eng; Huw-Michael Gough; Thales
Canada Transportation Solutions; Toronto, Canada
It is quite common to choose one supplier for developing a safety critical system on a project. With
the increasing scale of systems in certain domains, this single supplier approach creates technical,
schedule, and cost risks due to limitations of a supplier’s product and/or their capability against
demanding technical, project delivery and maintenance requirements. To reduce such risks, integrating
interoperable systems from multiple suppliers is practiced based on the concept of operational
compatibility. The quality of the final integrated systems is limited by the degree of compatibility
between suppliers’ products. A high degree of compatibility is difficult to achieve because technical
differences between suppliers’ products naturally exist. Therefore, an Open System Architecture
approach is a way to reduce technical differences at the integration level, and it expects subsystems
developed by different suppliers to be interchangeable. However, safety engineering for the final
integrated system encounters various challenges, because suppliers have different solutions and domain
technology, and each supplier considers its technology and safety techniques as its proprietary property.
This paper discusses the main challenges and proposes the development of layered safety concepts
36
Rhapsody Model Safety Tagging for Model Development Driven (MDD)
Approach to System Architecture and Requirements Development
Hung Duong, BS, Naval Surface Warfare Center, Dahlgren Division, Dahlgren, Virginia, United States
Rhapsody is an integrated Model Development Driven (MDD) environment for systems, software, and
testing. Rhapsody enables the user to specify systems and software designs graphically, as well as to
simulate and validate the system as it is being built, to ultimately produce full production code from
the model. Identifying and tracing of Safety Critical Requirements (SCRs) in a MDD environment vice
traditional text requires a new approach. This paper describes the proposed approach for tracing SCRs
from the user Software Requirement Specification (SRS) to Dynamic Object Oriented Requirements
System (DOORS) module, to the Rhapsody Systems Engineering (SE) and Software (SW) Models.
ISSTS 2014 Program
as the foundation for multiple suppliers to manage these safety engineering challenges in delivering
interoperable systems on a project.
Gaining Deeper Operational Intelligence Using Human Performance Learning Teams
Robert Edwards, BSME, SGE, Roper Corp / GE Appliances, LaFayette, Georgia, United States
Understanding operational drift related to accidents, quality escapes, operational upsets and challenging
engineering design issues is critical to the success of our organizations. Getting the “story behind the
story” (Dekker) can be difficult using conventional retrospective investigation techniques. What we
need is a more holistic approach to understanding failures or the potential for failure. We are finding
that the use of Human Performance Learning Teams (Conklin) is giving us a deeper level of operational
intelligence then we have ever obtained in the past. By bringing the right people together with the right
attitude towards failure we are opening up that conversation. The resulting discovery process is giving
us greater depth of knowledge and increased employee engagement leading us to collaboratively develop
solution sets that are more affective, more thorough and more sustainable. With each successful Learning
Team effort we are building more confidence and capacity in our employees and more resilience and
reliability into our products and our organizations.
Tuesday // 08-05-14 // 13:30-16:50 // Grand C // Safety Topics 3 // Chair: Pottratz
Reducing High Impact Events by Implementing a Deep Dive
Process Early in the Event Closure Lifecycle
Shawn M. Laabs, System Safety Engineer, United Launch Alliance, Centennial, Colorado, United States;
James E. Allison, Error Prevention Lead, United Launch Alliance, Centennial, Colorado, United States;
Joseph P. Russell, System Safety Engineer, United Launch Alliance, Centennial, Colorado, United States;
James S. Stewart, System Safety Engineer, United Launch Alliance, Centennial, Colorado, United States
The Event Closure Lifecycle did not originally leverage the subject matter expertise of the Error
Prevention Point of Contact until later in the process. Assigning a preliminary Impact Index score
provides the opportunity to implement a Deep Dive Process. Defining an Impact Index threshold is
important when implementing a Deep Dive Process. Events with an Impact Index Score above the
threshold trigger the Deep Dive Process. The Impact Index threshold can be reduced periodically to
increase Deep Dive Process oversight requirements. The Deep Dive Process augments the Event Closure
Lifecycle, leveraging the subject matter expertise of the Error Prevention Point of Contact earlier in the
Event Closure Lifecycle. The Deep Dive Process resulted in an increased level of depth and improved
results for the causal analysis and corrective action determination / implementation processes. The
increased level of depth and improved results ultimately led to a reduction in system risk over time that
was obtained by reducing the number of Events with Impact Index over the threshold. The overall level
of safety within the enterprise was increased by leveraging the subject matter expertise of the Error
Prevention Point of Contact early in the Event Closure Lifecycle.
37
Reconciling Developmental Weapons Safety Tests in MIL-STD-2105
Ken Tomasello, Navy Insensitive Munitions Office, Indian Head, Maryland, United States;
Diane Dray, Associate, Booz Allen Hamilton, Arlington, Virginia, United States;
John Adams, Associate, formerly Booz Allen Hamilton, Arlington, Virginia, United States
Since MIL-STD-2105 was revised to add additional Insensitive Munitions (IM) tests as called out by
the Joint Service Requirement for Insensitive Munitions (JSRIM), and by various NATO STANAGs, it is
common practice to assess munitions with respect to hazards using IM tests in MIL-STD-2105; Hazard
Classification (HC) tests in TB 700-2 (transportation and storage purposes); and safety and suitability for
service (S3) assessment testing. The basic safety tests have recently begun promulgation in a series of
interconnected NATO documents with the weight of STANAGs. This paper explores the contemporary
situation of delineation of these tests in MIL-STD-2105D and an approach to improve the transparency
of this seminal set of standards. This paper will recommend how the safety tests can be reconciled in
MIL-STD-2105D, the roadmap for reconciling the safety tests in MIL-STD-2105D, and the series of NATO
standards containing the safety tests. The expected result is an updated MIL-STD-2105 will be a less
confusing and a more focused standard for IM testing and S3 assessment testing will be accessed in a
well-defined set of NATO standards.
Effects of Unintended Longitudinal Acceleration and Deceleration Profile
Magnitude and Duration on Driver Performance Behaviors
Mark A. Vernacchia, MS, PE, General Motors Company, Milford, Michigan, United States;
Charles A. Green, Ph.D., General Motors Company, Warren, Michigan, United States; Robert E.
Llaneras, Ph.D., Virginia Tech Transportation Institute, Blacksburg, Virginia, United States
The automotive industry continues to implement increasingly complex and efficient propulsion control
systems in an effort to provide customers with the best balance of performance and fuel economy,
while still satisfying required safety-critical systems criteria. Understanding how driver performance
is affected by events such as unintended longitudinal accelerations and unintended longitudinal
decelerations is a critical aspect in the design, development, and verification of these safety criteria.
This presentation explores the results of an unintended longitudinal acceleration and unintended
longitudinal deceleration vehicle study performed by GM and VTTI. The magnitude and duration of
erroneous longitudinal impulses were varied during the study and the resulting driver behaviors and
vehicle data were recorded. The study identified various driver behavior signatures that were equated
to “startle” and “panic” events where some drivers did not maintain appropriate control of the vehicle
once an unexpected impulse was introduced. The resulting driver behaviors and vehicle data were used
to characterize “panic signatures” associated with unintended longitudinal acceleration and unintended
longitudinal deceleration events.
Wednesday // 08-06-14 // 8:00-11:20 // Grand B // Robotics/Unmanned Systems //
Chair: Kady
Models for Assessment of Unmanned Air Vehicle Hazards
David Chiam Tou Wei, ST Aerospace, Singapore
In the course of evaluation of system hazards during the development of Unmanned Aerial Vehicles
(UAV), it is realized that the impact of some failure conditions normally considered to be of high severity
for manned aircraft needs to be assessed differently. Unlike manned aircraft, the UAV operates without
a human pilot aboard, thus the potential to harm aircraft occupants is non-existent. The focus instead
shifts to the potential to inflict harm on humans in the area of operations. Physics based models which
take into account the characteristics of the UAV were developed to evaluate the conditional probabilities.
This contributes to a more representative mishap risk assessment.
Safety, Autonomy, Latency, and the Unmanned or Remotely Piloted Vehicle
Archibald McKinlay, MSc., University of Southern California (USC), Los Angeles, California, United States
This paper will outline the differences safety must consider unique when analyzing a remotely piloted
38
System Safety Considerations for Unmanned Ground Vehicles
Tiffany Owens, Naval Surface Warfare Center, Dahlgren Division, Dahlgren, Virginia, United States
The applications and capabilities of Unmanned Ground Vehicles (UGVs) have increased in the last
several years in the private and military sectors. More and more UGVs are employed to execute tasks
that are considered dangerous to personnel such as chemical and biological warfare monitoring, wildfire
elimination, surveying hostile environments and deactivating explosive devices. UGVs also aid to reduce
personnel tasking requirements such as patrolling borders and transporting loads on the battlefield.
While the benefits and usages of UGVs are welcomed, UGVs also introduce new safety concerns to be
addressed due to the decreased role of the human operator. This paper will present application areas, and
discuss system safety concerns and analysis required to support the safe operation of UGVs.
ISSTS 2014 Program
or unmanned system safety program by drawing on parallels to the more common distributed system.
Building on the DoD’s Unmanned System Safety Guidebook and relevant analyses, the risks and issues
which are unique to these unmanned systems shall be separated and listed. An assessment of these
risks, issues, and attributes shall examine whether these are extensions of current MIL-STD-882, or
other standards, or whether new analyses, inspections, or tests should be required or whether existing
processes can absorb them to adequately state the integrated system safety risk.
Wednesday // 08-06-14 // 8:00-11:20 // Grand C // Software Safety 1 // Chair: Axelrod
Interpretation of the Software Control Categories for MIL-STD-882C
Tan Shen Chin, Singapore Technologies Kinetics, Singapore
Classifying a piece of software function in accordance to the MIL-STD-882C Software Control Categories
is not as simple as to pick and match to one of the following. The definition in the standard has its fair
shares of shades of grey which is a source of great debates among the software safety practitioners,
experts and within the project team members. Not many good literatures are available to provide a clear
understanding of the definition or a practical
approach for justifying the assignment of the
software control categories. This paper attempts
to provide a method to address the assignment
of the control categories to a software function
as closely matched to the definition as possible.
The method takes into account of the product
nature which the software was designed for, and
using sequence diagrams in the justification of
the of the software characteristic. In addition, the
software architecture, the degree of coupling and
cohesion relationship of the software functions
would also be used to determine the assigned
classification. This method had been adopted by
the author in his course of work and the approach
was able to provide a more assuring classification
for reviews.
39
Decoding The Software Control Category
Eng Ling Onn, ST Kinetics Ltd, Singapore; Clifton Ericson, BSEE, MBA, URS Corp., Fredericksburg,
Virginia, United States; Michael Brown, BSEE, MS, URS Corp, California, United States
This paper consolidates the development of software safety standards to aid understanding in the
application of Software Control Category (SwCC). Ambiguity in the Software Control Category (SwCC)
criteria defined in MIL-STD-882C and E is highlighted. Such ambiguity is one of the significant source
of uncertainty in the software module assessment and it often leads to inconsistent or incorrect
determination of the Software Criticality Index (SwCI). The authors re-collected the key basis used for
establishing the SwCC in MIL-STD-882C. Changes to the SwCC from MIL-STD-882C to MIL-STD-882E are
compared to ascertain the impact to the interpretation of SwCC. Further improvement to the SwCC
criteria is suggested to eliminate ambiguity.
System and Software Safety Challenges for Widespread Acceptance of Driverless Vehicles
Gregory Turgeon, GT Software Services, Decatur, Alabama, United States
Widespread use of driverless automobiles has the potential to save tens of thousands of lives annually in
the United States. While the technology for these vehicles exists today, there are unique system safety
and software safety challenges that must be addressed to ensure this lifesaving potential can be realized.
The primary system safety challenge is developing techniques to assess the complex system of systems
that is composed of the vehicle safety equipment interacting with other vehicles and the constantly
varying highway environment. The primary software safety challenges are to ensure rigorous software
standards are enforced and to tightly control the configuration of the software in each vehicle. These
new challenges will require development and update of safety standards, regulation and enforcement
by government agencies, acceptance by automotive manufacturers and suppliers, and training of the
professionals developing these systems. The paper describes a potential driverless vehicle transportation
system, identifies the system and software safety challenges to assure safety of that system, and lists
specific steps for system safety professionals to generate the foundation to fully realize the lifesaving
potential of this new technology.
Wednesday // 08-06-14 // 8:00-11:20 // Regency A // Space Safety // Chair: Thomas
Hazardous Dependency of Critical Infrastructures on Global Navigation Satellite Systems Services
Amaya Atencia-Yepez, Safety & Dependability Manager; Marta Cueto-Santamaría,
GNSS Project Manager; Ana Cezón-Moro, GNSS Section Head; GMV, Spain
The growth of GNSS technology is very fast and there are uncountable applications that rely on it. GNSS
is integrated in the daily habits of many millions of people. Almost in every sector around us there are
applications that utilize GNSS technology ranging from safety of life applications (like navigation aid
for maritime, road, railway or aviation transportation) to common utilities (like well-known in-car
or personal navigation). With such a high rate of dependency on GNSS signals, many national critical
infrastructures in key economic and social sectors would be severely affected in case of failures or
outages of GNSS: “Positioning, navigation and timing PNT services” are fundamental for transportation
sector and “Timing services” are also key for the efficient management and operations of national and
cross-border utilities. Understanding GNSS vulnerabilities and defining mitigations strategies (like
back-up systems) is essential for making them more resilient and for protecting crucial services that are
dependent of GNSS. The objective of this paper is to evaluate the degree of reliance of a given set of user
communities and to identify means that could provide PNT services to critical users in the absence of
GNSS without causing a major impact on the society.
Major Hazardous Events for Unmanned Space Systems
Burak Durmaz M.Sc., Eng., System Safety Specialist, Turkish
Aerospace Industries, Inc. (TAI), Ankara, Turkey
Unmanned satellites are being widely used in different application areas such as earth observation,
telecommunication and scientific researches. The increase in demand for these space systems also
increases the technological complexity and introduces different types of hazards into satellite and
40
ISSTS 2014 Program
associated ground support equipment design. The aim of this study is to briefly outline the major
hazardous events that shall be taken into account in the safety program of an unmanned satellite
program. The hazardous events are including propellant leakage or explosion, explosion or leakage
from battery, dropping of filled/unfilled spacecraft during vertical lifting or horizontal transportation,
electrical ignition, inadvertent thruster firing or leakage, uncontrolled release of potential energy
(rupture of pressurized tank and lines, etc.) and/or kinetic energy (momentum wheel, etc.) and radio
frequency susceptibility. The aim is to give the possible hazard causes, hazard control means, safety
verification methods, and the data which is going to be used as an evidence for the assigned verification
method. The evidence material may be spacecraft and associated ground support equipment design
documentation and drawing, various analysis results such as stress, thermal and electromagnetic
compatibility and qualification and acceptance test reports. Also, complementary verification activities
are listed as a reminder.
Simulating the Risks of Sub-Orbital Space Flight for Air Traffic Management
Chris W. Johnson and Marco Sarconi, School of Computing Science,
University of Glasgow, Glasgow, United Kingdom
The next decade will see an increasing number of sub-orbital space flights for both scientific reasons and
for space tourism. In the longer term, these initiatives may also lead to the development of sub-orbital
transportation – for instance, to support military fast response without the need for costly, high-risk
local deployments. As part of the longer term planning for these flights, it is important to assess the
possible risks to civil aviation and, in particular, the hazards that might arise from their interaction
with controlled air space. In this paper, we present the results from integrating live data about aircraft
flights using an Automatic Dependent Surveillance-Broadcast (ADS-B) server together with up to data
meteorological information. The users of the system describe the performance characteristics of a suborbital vehicle together with the coordinates of a potential accident. The system then calculates the
resulting debris field and presents a predictive model of the consequent impact on surrounding aircraft
at different flight levels. The closing sections of the paper identify future directions for research to assess
the safety impact of sub-orbital flights.
Wednesday // 08-06-14 // 13:30-16:50 // Grand B // Safety Topics 4 // Chair: Rozanski
The Evolution of System Safety at NASA
Homayoon Dezfuli, Ph.D., NASA Headquarters, Washington DC, United States;
Frank Groen, Ph.D., NASA Headquarters, Washington DC, United States;
Chris Everett, Information Systems Laboratories, New York, United States
The NASA system safety framework is in the process of change, motivated by the desire to promote an
objectives driven approach to system safety that explicitly focuses system safety efforts on system-level
safety performance, and serves to unify, in a purposeful manner, safety-related activities that otherwise
might be done in a way that results in gaps, redundancies, or unnecessary work. An objectives-driven
approach to system safety affords more flexibility to determine, on a system-specific basis, the means by
which adequate safety is achieved and verified. Such flexibility and efficiency is becoming increasingly
important in the face of evolving engineering modalities and acquisition models, where, for example,
NASA will increasingly rely on commercial providers for transportation services to low-earthorbit. A key
element of this objectives-driven approach is the use of the risk-informed safety case (RISC): a structured
argument, supported by a body of evidence, that provides a compelling, comprehensible and valid case
that a system is or will be adequately safe for a given application in a given environment. The RISC
addresses each of the objectives defined for the system, providing a rational basis for making informed
risk acceptance decisions at relevant decision points in the system life cycle.
Utilizing Error Prevention Data and Lean Six Sigma Techniques
to Verify the Existence of Error Prone Zones
Shawn M. Laabs, United Launch Alliance, Centennial, Colorado, United States
Historically, potential Error Prone Zone existence was identified through histogram analyses. Error Prone
41
Zones were originally thought to exist for Events occurring within certain Times of Day and Months
of Year. Initial research was unable to verify Error Prone Zone existence. In an attempt to verify Error
Prone Zone existence, ULA developed and analyzed a suite of Control Charts. The Control Chart analyses
increased the knowledge and understanding of Error Prone Zones. The analyses verified the existence
of some Error Prone Zones and did not verify the existence of others. There are a variety of benefits
that were realized from the analyses. One is Error Prone Zone verification. This allows more effective
application of resources on Error Prevention activities. This also allows more effective Lessons Learned
sharing about Error Prone Zones and their existence. Another benefit is increased ability to conduct
additional research on Error Prone Zones. Better understanding allows the exploration of possible effect
factors, which may affect the work force’s capability to perform error-free tasks. The Error Prevention
Team can now influence planning decisions and reschedule critical tasks to occur outside of an Error
Prone Zone in an attempt to reduce the risk of Event occurrence.
Wednesday // 08-06-14 // 13:30-16:50 // Grand C // Software Safety 2 // Chair: Schedl
Study of a Method for Early Interface Verification with Hierarchical Executable Software Model
Ryo Ujiie, Japan Aerospace Exploration Agency, Tsukuba, Ibaraki, Japan;
Masafumi Katahira, Japan Aerospace Exploration Agency, Tsukuba, Ibaraki, Japan;
Maria Hernek, European Space Agency, Noordwijk, Netherland; Pablo Abad Rubio,
Deutsches Forschungszentrum für Künstliche Intelligenz, Kaiserslautern, Germany
The Japan Aerospace Exploration Agency (JAXA) has collaborated on certain international projects.
In the projects, the interface among different organizations caused many problems, some of which
were found late in the development phase, that negatively impacted the projects. Therefore, various
organizations have attempted to define and verify adequate Interface Control Documents (ICDs) in order
to find problems early in the development phase. For JAXA’s software Independent Verification and
Validation (IV&V), engineers verify the interface early in the development phase to ensure successful
projects. However, past verification consisted mainly on validating static consistency between ICDs and
target software specifications; behavioral inconsistency among software causes problems late in the
development phase. JAXA and the European Space Agency (ESA) IV&V teams undertook a pilot project
with the objective of studying an effective method for verifying interface related to software behavior
early in the development phase. In this pilot project, we developed a verification method based on an
executable hierarchical model, and applied it to an actual international spacecraft project. As a result, 92
ICD issues were identified before implementing the software; however, significant time was consumed.
To improve efficiency, we suggest developing models based on the software component architecture
framework.
Predicting Software Performance - Software I
Richard R. Zito, Richard R. Zito Research LLC, Tucson, Arizona, United States
Simple and robust model section methods are presented for software. These models allow forward
projection of failure (defect) rates based on collection of a limited amount of early change control data
during the software development process. Naturally, model selection is simpler when more data is
available. But here, the focus is on “early model projection” because financial and temporal constraints
often limit the amount of resources available for more complete collection of software performance and
reliability data. If a maximum in the defect rate occurs at time t = tmax into the software development
cycle, then key model selection and refinement times occur at t = 2tmax and 3tmax.
What is “Unnecessary” Code and Why Is It Unsafe?
Archibald McKinlay, MSc., University of Southern California (USC), Los Angeles, California, United States
This paper will address one type of unused software code, that is, unnecessary code. Review will include
the arguments and claims regarding a set of definitions which define software code as “unnecessary”
and contrast that term with other terms often applied to unused code, such as not loaded, unreadable,
dead, partitioned, not instantiated, unreachable, or any other software code other than that intended
to execute within this runtime. Once defined and clarified, unnecessary code shall be examined to
42
Thursday // 08-07-14 // 8:00-9:50 // Grand B // Hazard Analysis 2 // Chair: Durmaz
Innovation vs Safety: Hazard Analysis Techniques to Avoid Premature Commitment
During the Early Stage Development of National Critical Infrastructures
Chris W. Johnson, School of Computing Science, University of Glasgow, Glasgow, United Kingdom
Preliminary hazards analysis helps identify safety concerns during the early stages of development.
However, these techniques rely on scoping studies and functional decompositions that can be hard
to sustain without premature commitment to particular software architectures. For example, small
alterations to the high-level design of a critical infrastructure can force radical changes in the underlying
hazard analysis. This creates tensions – safety managers become “the enemies of innovation” if they
oppose modifications that trigger additional hazard analyses. Equally, it can be hard for safety managers
to control project costs if alterations force continual changes in their safety assessments. These tensions
are compounded because many hazard analysis techniques have their roots in the 1960s when issues of
scale, modularity and reuse were arguably less significant than today. These arguments are illustrated
by the EATS project creating an Advanced Testing and Smart Train Positioning System for the next
generation European Train Control System. EATS integrates a range of wireless infrastructures with input
from Satellite Based Augmentation Systems to reduce reliance on trackside infrastructures. However, the
dynamic, multidisciplinary nature of the work has created a need for continuous feedback on potential
safety concerns as lab and bench studies innovate with novel software architectures and prototype
implementations. We present a number of approaches that can be used to balance the need for design
commitment to support safety assessments and the flexibility required in the early-stage development of
critical infrastructures.
ISSTS 2014 Program
determine whether sufficient analyses, tests, or demonstrations may be applied to satisfy a safety
argument that allows that unnecessary code to remain loaded with no further actions required.
Mind-Mapping the Hazard Space of a System
Clifton A. Ericson II, Fredericksburg, Virginia, United States
Everyone knows how to identify hazards, right? Just look at a system functional diagram and start listing
hazards that would result from component and functional failures. Or, look at all the system energy
sources and list the bad things that can happen if they fail or malfunction. Or, look at system control
laws and evaluate the effect of potential malfunctions. Although these approaches sound plausible and
are taken by many, the problem is that effective hazard recognition is not quite that simple. One of the
major problems encountered during hazard analysis is properly organizing the overall analysis such that
the correct and proper hazards can be identified. Sometimes systems are so large and complex that an
analyst easily goes off in the wrong direction and misses hazards or misidentifies hazards. This situation
is analogous to the old adage that one cannot see the trees for the forest. The solution is to develop a
system mishap model (SMM) that aids in visualizing the trees in the forest. The SMM helps the safety
analyst to organize a hazard analysis and to visualize the trees within the forest; it maps the overall
hazard space of a system. The SMM is not a hazard analysis, but rather a hazard analysis aid. This paper
explains the SMM and its usage, along with several examples.
Thursday // 08-07-14 // 8:00-11:20 // Grand C // Risk Assessment 1 // Chair: Rinaldo
Distribution of Risk
Aaron Banerjee, M.S. EE, Naval Surface Warfare Center, Dahlgren, Virginia, United States
Safety risk is expressed in terms of probability and severity. Currently, Military Standard (MIL-STD)-882D
provides a matrix whereby, given a probability and severity, the risk level can be determined. Currently,
when risk is quantified in terms of probability and severity, the worst credible scenario is typically used
as the single risk scenario. While that approach increases the confidence that the assessed risk represents
the worst possible outcome, it may not provide a comprehensive picture of all relevant risk (e.g., such as
the case where there is a credible catastrophic scenario with a very small likelihood of occurrence, but
also a serious scenario with a relatively high likelihood of occurrence). This paper proposes an approach
43
where a broad picture of risk may be obtained by expressing risk of a mishap as a series of scenarios and
then using a summing technique to obtain a Composite Risk Index (CRI). The approach utilizes factors
representing the big picture of risk, such as severities of possible mishap effects and population size
(affecting composite probability). The purpose of this approach is not to establish an absolute measure
for risk, but to provide a balanced expression of risk level to support sound safety risk decisions.
Personnel Risk Assessment for Random Reentry Considering Casualty Expectation
Lan Dang, B.S. ChE, USAF Space and Missile Systems Center, Los Angeles AFB, El Segundo,
California, United States; Tom Meyers, PE CSP, USAF Space and Missile Systems Center, Los
Angeles AFB, El Segundo, California, United States; Tyrone Jackson, BS Electrical Engineering
& Certified Reliability Engineer, ManTech, El Segundo California, United States
Assessing, reporting, accepting and documenting mishap risk to personnel using the MIL-STD-882E risk
assessment methodology is required by multiple DOD and Air Force policies. In contrast, a tradition
(also rooted in policy and national guidelines) in the Air Force Space enterprise is to assess and accept
risk using Casualty Expectation (Ec) models and estimates. Casualty Expectation differs from the MILSTD-882E risk assessments in that it does not provide separate probability estimates for a given severity
(e.g. probability of fatality of one or more personnel), but instead provides an estimate of the number of
casualties (including significant injuries and fatalities) for a particular operation and re-entry. This paper
introduces the relationship between MIL-STD-882E risk and Casualty Expectation, describes how MILSTD-882E risk assessment methodology might be used in conjunction with Casualty Expectation models
and estimates along with considerations of terminal effects, and explains the advantages and difficulties
of using such a procedure. The paper also proposes a process improvement to enable appropriate use of
MIL-STD-882E risk assessments in these cases.
The Challenges of a Quantitative Approach to Risk Assessment
Rani Kady, PhD, Naval Surface Warfare Center, Dahlgren Division, Dahlgren, Virginia, United
States; Arjuna Ranasinghe, PhD, Alabama A&M University, Huntsville, Alabama, United
States; Mike Zemore, MS, NSWCDD, Dahlgren, Virginia, United States; Regina Eller, BA,
Naval Surface Warfare Center, Dahlgren Division, Dahlgren, Virginia, United States
Risk assessment and documentation, as an element of the system safety process in the general
requirements of MIL-STD-882E, assesses the severity category and probability level of a potential
mishap. The likelihood of occurrence of a mishap determines the probability level for a given hazard at
a given point in time. Since quantitative assessments are perceived to be more accurate than qualitative
ones, several attempts have been made to accurately measure the appropriate probability level for
a hazard. Researchers of such attempts are challenged to use appropriate and representative data to
define frequency or rate of occurrence for a hazard. This paper describes the nature and complexity of
a mathematical representation to risk assessment. It outlines a detailed quantitative risk assessment
framework to guide the system safety community attempts. A case study will be presented to illustrate
the application of the framework and highlight the challenges to the system safety community in terms
of mathematical approach, assumptions, and variable conditions of risk assessment.
Thursday // 08-07-14 // 13:30-16:50 // Grand B // Security & System Safety // Chair: Owens
Supporting the Exchange of Lessons Learned from CyberSecurity Incidents in Safety-Critical Systems
Chris W. Johnson, School of Computing Science, University of Glasgow, Glasgow, United Kingdom
Over the last decade, a small but growing number of cyber-attacks have been detected in safetycritical systems. One reason is that ‘mass market’ malware targets Commercial Off-The Shelf (COTS)
infrastructures, including Linux and the IP stack, which are increasingly being used in critical
applications. Another reason is the rise of state-sponsored attacks; ‘sniffer’ programs disclose
infrastructure information from inside control networks that are not usually connected directly to the
Internet. There are considerable barriers to the disclosure of information about previous attacks. Many
commercial infrastructure providers do not want to admit that their systems have been exposed to ‘mass
44
Cyber Safety of Voice Communication Systems: About Security Threats and Safety Analysis
Maximilian Riedl, Frequentis AG, Vienna, Austria; Gabriele Schedl, Frequentis AG, Vienna, Austria
Security threats are a growing hazard for safety-relevant systems operating in cyberspace. The demands
to interact and exchange with other systems is ever-growing and results in an increasing number of
vulnerabilities. Apart from technical threats like sophisticated malware and the conflicting situation
of software assurance and outdated software, another challenge is the paradigm shift from a strict
separation of safety and security domain to a common and unified approach. Due to different interests
and requirements, this separation is understandable from a historic perspective, but disadvantageous
for the future cyber safety of systems. The paper will present promising results achieved by the
integration of security-risks into safety- and hazard analyses in the field of voice communication and
control systems. The joint examination of standards, methods, requirements and scientific literature of
both domains, has led to a mutual understanding and closer cooperation of safety domain experts with
security specialists. The cooperation has resulted in substantial improvement regarding security and
safety of the considered systems. Therefore the approach for harmonization of safety and security is
highly recommended.
ISSTS 2014 Program
market’ malware. There are also forensic and national security concerns associated with the disclosure of
information about more sophisticated forms of attack. The following pages identify ways of supporting
the exchange of information about previous cyber-attacks without disclosing data that might encourage
future attacks. In particular, we consider architectures for security-incident reporting systems that
encourage the exchange of lessons learned in safety-critical applications.
Inadequate Legal, Regulatory, and Technical Guidance for the Forensic
Analysis of Cyber-Attacks on Safety- Critical Software
Chris W. Johnson, School of Computing Science, University of Glasgow, Glasgow, United Kingdom
National and international organizations including NIST and ENISA have published guidance that is
intended to help organisations respond to, and recover from, cyber incidents. They provide detailed
information about contingency planning, about the processes needed to gather and analyse evidence,
about appropriate ways to disseminate the findings from forensic investigations. Legal frameworks,
including the Federal Rules of Evidence, also help companies to identify ways of preserving a chain
of evidence with the digital data gathered in the aftermath of a cyber-attack. It is essential that
companies apply these guidelines to increase their resilience to future attacks. However, they provide
the least support where they are needed the most. Existing guidelines focus on corporate office-based
systems; they cannot be applied to support companies dealing with cyber-attacks on safety-critical
infrastructures. This is an important omission. It is impossible to immediately disconnect infected
systems where they provide life-critical functions. There are conflicts between the need, for instance,
to preserve the evidence contained in volatile memory and the requirement to return safety-critical
applications to a safe state before any forensic work can begin. The following pages identify the problems
that arise when applying legal, regulatory and technical guidance to the cyber security of safety-critical
applications. The closing sections focus on techniques that can be used to support the forensic analysis of
cyber incidents and promote recovery from attacks without placing lives at risk.
Thursday // 08-07-14 // 13:30-16:50 // Grand C // Risk Assessment 2 // Chair: Pierson
Usability for System Safety Engineers: Using Nielson’s Ten Heuristics to Identify
the Increased Potential of Human Error as a Contributor to Mishap Risk
Rebecca Funkhouser, Naval Surface Warfare Center, Dahlgren Division, Dahlgren, Virginia,
United States; Rachel St. Laurent, Department of the Navy, Naval Surface Warfare Center,
Dahlgren Division, Dahlgren, Virginia, United States; Samantha Sperry, Department of the
Navy, Naval Surface Warfare Center, Dahlgren Division, Dahlgren, Virginia, United States
System safety experts have acknowledged that collaboration with HSI is important. Various documents,
courses, workshops, and journal articles address system safety and HSI: NAVSEA SW020-AH-SAF-010
“Weapon System Safety Guidelines Handbook” has an entire chapter on HSI, the Weapon System
45
46
ISSTS 2014 Program
Explosives Safety Review Board (WSESRB) Interactive Safety Environment (WISE) Training includes a
course on System Safety and HSI, previous International System Safety Conference papers and workshops
have focused on HSI, and several Journal of System Safety articles have been written about system safety
and HSI. Some of these resources define HSI and the seven domains which make up HSI. Other resources
describe Human Reliability Analysis (HRA), an HSI analysis technique which was developed to calculate
the probability of human error based on cognitive and environmental influences. It is useful for system
safety engineers to be aware of other HSI analysis techniques that are available. This paper will introduce
system safety engineers to the concept of usability and will teach them to identify when a system design
violates basic usability “rules of thumb.” If the safety engineer sees one of these usability problems,
especially in a safety critical system, the design in this area deserves additional scrutiny. This scrutiny
should include collaborating with HSI to determine how human error could increase the probability of a
mishap.
Influence Diagrams: Generalizing Fault Trees for Informed Decision Making
Jeremy Monat, PhD, James McCracken, Ariel Obaldo, Gary Sweany,
Systems Planning and Analysis, Washington, DC, United States
Informed safety decision making can require numerous fault tree analyses (FTA) and probabilistic risk
assessments (PRA) for a single hazard. This is especially true in complex projects. One influence diagram,
a generalization of FTA and PRA, can embody multiple analyses. Influence diagrams allow the analyst to
calculate mishap probabilities which depend on system parameters such as equipment or mitigations
used. While FTA are widely used to visualize mishap scenarios and estimate their probability, they are
cumbersome to adapt to varying circumstances to answer “what-if ” questions. Influence diagrams offer
a way to generalize FTA to add flexibility that makes these types of situations easier to address; e.g.
multiple mitigations may be added as parameters to a single model and then the results of all scenarios
(sets of mitigations) of interest can be calculated simultaneously. Thus, expeditious, informed safety
decisions are easier to make; for example, determining what mitigation(s) one must implement to keep
the mishap probability over the exposure interval below a certain level. We present a case study where a
fault tree was generalized using an influence diagram and show how more comprehensive information
can be obtained which supports informed program-wide decision making.
47
About the ISSS
The International System Safety Society is a non-profit organization of professionals dedicated to the
safety of systems, products and services through the effective implementation of the system safety
concept. Under this concept, appropriate technical and managerial skills are applied so that a systematic,
forward-looking hazard identification and control function becomes an integral part of a project,
program or activity at the planning phase and continues through the design, production, testing, use and
disposal phases.
The Society’s Objectives
•
•
•
•
To advance the art and science of system safety
To promote a meaningful management and technological understanding of system safety
To disseminate advances in knowledge to all interested groups and individuals
To further the development of the professionals engaged in system safety
• To improve public understanding of the system safety discipline
• To improve the communication of system safety principles to all levels of management, engineering
and other professional groups
International System Safety Society, Inc. P.O. Box 70, Unionville, VA 22567-0070
www.system-safety.org, email: systemsafety@system-safety.org
Points of Contact
Officers
Directors
Robert Schmedake
President
robert.a.schmedake@boeing.com
Gerry Einarsson
Chapter Services
einargk@rogers.com
Lynece Pfledderer
Conferences
lynece.pfledderer@lmco.com
Dr. Chuck Muniak
Education & Professional
Development
cmuniak@stevens.edu
Dr. Rod Simmons
Executive Vice President
rod.simmons@me.com
Dr. Matt Johnson
Executive Secretary
mdjohnson76@acm.org
Pam Kniess
Treasurer
pamkniess@gmail.com
Gary Braman
Immediate Past President
gbraman@sikorsky.com
48
Steve Mattern
Mentoring, R&D
smattern@bastiontechnologies.com
Robert Fletcher
International Development
rwfletcher@sympatico.ca
Melissa Emery
Member Services
memery@apt-research.com
Debbie Hale
Gov. & Intersociety
Hale0324@hotmail.com
Saralyn Dwyer
Publicity & Media
sdwyer@apt-research.com
Bay Area
Graham Murray
408 756 2674
Graham.t.murray@lmco.com
Tennessee Valley
Ken Rose
256 842 3246
ken@isss-tvc.org
Central California
Kathleen Brenna
805 606 2308
Kathleen.Brenna.1@us.af.mil
Twin Cities
Bill Blake
763 744 5086; 763 245 0165
Bill.blake@atk.com
Georgia
Odell Ferrell
770 494 4814
Odell.ferrell@lmco.com
Virtual
Doanna Weissgerber
408 289 4407
Doanna.Weissgerber@baesystems.com
Houston
Derek Robins
281 820 8828
Derek.Robins@mwcc-usa.com
Washington DC
Sean Peters
540 663 7369
Sean.peters@urs.com
New Mexico
William (Bill) Harwood
505 853 4595
William.harwood@mda.mil
Australian
Dr. Holger Becht
+61 (0)7 3102 9742
Holger.becht@rgbassurance.com.au
Northeast
Scott Beecher
860 565 7022
Scott.Beecher@PW.utc.com
Canada
Maury Hill
613 220 0533
Mauryhill@rogers.com
North Texas
Frank Rinaldo
817 762 3075
Frank.r.rinaldo@lmco.com
Singapore
Lin Mei Ten
65-63081006
tlinmei@dso.org.sg
ISSTS 2014 Program
Chapters
Saguaro
Amanda Boysun
520 794 5487
Amanda.Boysun@raytheon.com
Sierra High Desert
Jerry Banister
760 377 4690
Safety.citadel@earthlink.net
Southern California
Francis McDougall
310 653 1309
Francis.mcdougall@us.af.mil
49
Special Functions
Saturday
ISSS Executive Council Meeting
13:00 - 17:00
Frisco/Burlington
Sunday
ISSS Executive Council Meeting
8:00 - 17:00
Frisco/Burlington
Monday
Speakers’ Breakfast
6:30 - 8:00
Gothic Corridor
Spousal Information Meeting
9:00 – 10:00
Frisco/Burlington
Luncheon & Opening Ceremony
11:30 – 13:30
Grand F
Keynote Speaker: Carl A. Avila, Director, Advanced
Programs, Boeing Phantom Works
General Assembly Address
13:30 – 14:20
Grand F
Speaker: Bob Schmedake, Society President
Pioneer Panel
14:30 – 16:30
Regency C
Tuesday
Speakers’ Breakfast
6:30 - 8:00
Gothic Corridor
Sponsor & Exhibitor Luncheon
11:30 – 13:20
Grand F
Speaker: Tom Pfitzer
President, A-P-T Research, Inc.
50
Sponsor & Exhibitor Social
18:00 - 21:00
Grand D/E/F
Wednesday
Speakers’ Breakfast
6:30 - 8:00
Gothic Corridor
International Luncheon
11:30 - 13:20
Grand F
Speaker: Gabriele Schedl, Director of
Safety Management at Frequentis
Off-Site Social Event
17:00 – 22:00
Event buses leave between 17:00 & 17:30 and
will return between 21:00 & 22:00
Thursday
Speakers’ Breakfast
6:30 - 8:00
Gothic Corridor
Awards Luncheon
11:30 - 13:20
Grand F
Speaker: Alexander Garza, MD, MPH
Associate Dean at St. Louis University
Friday
Speakers’ Breakfast
6:30 - 8:00
Gothic Corridor
Best Papers Presentation
8:00 - 10:00
Regency C
ISSTS 2014 Program
Notes
51
Notes
52
Program
ISSTS2014
Thank you Sponsors & Exhibitors!
Corporate Sponsor
Advertisement Page
Booth Location
The Boeing Company. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Sponsors
A-P-T Research, Inc. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Isograph. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Lockheed Martin. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Controls and Data Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Sikorsky. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Exhibitors
Columbia Southern University. . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Electric Power Research Institute . . . . . . . . . . . . . . . . . . . . . . . . 28 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Item Software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
University of Maryland . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
4
6
5
ISSC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Grand Ballroom D
1
2
3
4
5
6
Dining Tables
Dining Tables
7
8
9
10
11
12
Grand Ballroom E
International System Safety Society • P.O. Box 70, Unionville, VA 22567-0070 USA • www.system-safety.org
Designed and published by
A-P-T Research, Inc. Publications.