SYRAS & QMAS
Applications Examples
Bob Bea
National Science Foundation
1
RESIN PROJECT
P[F] = P[FI|E] P[E]+P[FI|E]P[E]+P[FE|E]P[E]
due to intrinsic
causes with no error
due to intrinsic
causes with error
due to extrinsic
causes with error
2
Quality
Attributes
SYRAS©
Durability
Safety
QA / QC
Design
Phase
QA / QC
Construction
Phase
Compatibility
Serviceability
Lifecycle
Task Organization
Operations
Phase
Task Organization
QA / QC
Maintanence
Phase
Task Organization
Task 1
Task 2
Task 3
Task 1
Task 2
Task 3
Base Rates
Base Rates
Base Rates
Base Rates
Base Rates
Base Rates
Influencing
Factors
Influencing
Factors
Influencing
Factors
Influencing
Factors
Influencing
Factors
Influencing
Factors
3
4
5
6
Example QRA/PRA
7
Proactive - Robustness
= damage tolerance
defect in tolerant
defect tolerant
Intensity of damage or defects
8
9
Independent
&
Dependent
Errors
Detection
Analysis
QA/QC
•lack of expertise
•resource deficiencies
•excessive authority gradients
•communication malfunctions
•management breakdowns
•rejection of information
•violations
Errors
Correction
10
Probability of uncorrected extrinsic cause P(E):
Quality Assurance and Quality Control (QA/QC)
P(D)=1-P(D),


P(A)=1-P(A),
P(C)=1-P(C)

D = Detection
A = Analysis
C = Correction
P( E )  P( E ')[1  P( D) P( A) P(C )]
Base rate of Extrinsic ‘malfunction’
11
12
Probability of Error: P(E)
.
Mean Probability of Human Error
per Activity
1
new or rarely performed task
extreme stress, very little time
severe distractions & impairments
10 -1
highly complex task
considerable stress, little time
moderate distractions & impairments
10-2
complex or unfamiliar task
moderate stress, moderate time
little distractions & impairments
10 -3
difficult but familiar task
little stress, sufficient time
very little distractions or impairments
10-4
simple, frequently, skilled task
no stress, no time limits
no distractions or impairments
10 -5
13
14
15
Performance Shaping Factors
(PSF)
P(Ejkm)  P(E' jkm)   PSF Ejkm  1.0
Operators
Structure
Organizations
Interfaces
Procedures
Hardware
Environments
16
SMAS Grade
1
Minor negative influence
Moderate negative influence
Extreme negative influence
1
1E1
1E2
1E3
4
5
6
7
Below average
Poor
Very poor
does not meet
any standards
or requirement s
Moderate positive influence
Extreme positive influence
Negligible influence
3
1E- 1
Good, average
exceeds all standards
and requirements
1E- 2
Minor posit ive influence
2
Very good
1E- 3
Excellent
Outstanding
exceeds all standards
and requirements
SYRAS PSF
17
SMAS/QMAS©
•
•
•
•
Assessment instrument
Assessment protocol
Assessment team development
Follow-up
18
QMAS components
Operating
Teams
Factors
Structural
Factors
Organizational
Factors
Interfaces
Factors
Environmental
Factors
Hardware
Factors
Procedural
Factors
19
QMAS components, factors, attributes
20
QMAS factors
Operating Teams
Process Aud iti ng
Safety Culture
Ris k Perception
Emergen cy Preparednes s
Command & Controls
Training
Communications
Requisit e Variety
Equip ment
Design gu idelines and
specification s
Materials
Demand systems
Power systems
Configurations
Control systems
Operators & other
Organ izations & other
Orga nizational
Process Aud iti ng
Safety Culture
Ris k Perception
Emergen cy Preparednes s
Command & Controls
Training
Communications
Resources
Structure
Design gu idelines & specs.
Materials
Loadings
Structure configura tion
Computer programs
Research, development and
testing background
Interfaces
Procedur es & other
Envi ronmental & other
Proce dures
Operating
Maintenance
QA/QC
Contractor selection
Pre-start up review
Emergen cy response
Manage ment of change
Validations
Environmental
Externa l ( Weather)
Internal
Social External (Regu latory, Society)_
Social (Internal)
(Within organ ization and
operating team)
Equip me nt & other
Structure & other
21
22
QMAS gradings
23
Components
QMAS mean results
Interfaces
Environments
Highest mitigation priorities
Structure
Equipment
Procedures
Organizations
Operators
0
1
2
3
4
Grades
5
6
7
24
QMAS assessors
• QMAS counselor
• QMAS assessors
– Facility operators
– Facility management
• QMAS training program
– HOF background
– QMAS operations
– QMAS applications
25
QMAS assessors: THE MOST
IMPORTANT PART!
• Experienced
– Facility
– Process auditing
•
•
•
•
•
Respected
Integrity
Motivated
Observant
Thoughtful, insightful
26
QMAS
applications
27
Sleipner A Platform Failure
(U.S. $1 billion)
Contributing
•Loss of corporate memory
•Cut-back in QA/QC
Initiating
•Error in FEA
sank during
construction
Compounding
•Ineffective QA/QC design
•Ineffective QA/QC construction
28
Sleipner FEA caused failure
Interfaces
Env ironment
Structure
Equipment
Procedures
Organizations
Operators
1
2
3
4
Grades
5
6
7
29
All design phase accidents
PSF’s
Interfaces
2.0 E4 to
Environment
Structure
7.9 E5
Equipment
COV’s
Procedures
1.4 to
Organizations
2.1
Operators
1
2
3
4
5
6
7
Grades
30
31
32
33
34
Diver Incident or Fatality
(Ascent and
decompression phase)
1.5E -1
Detect ion and
Correct ion by dive
Buddy or Standby
Diver
[1-P(D)P(C)]
P(UE)= 4.1E-1
2.6E -1
Decompression
Situation
Fails to follow
decompression
schedule
Exceeds ascent
rat e
Accidental BC
inflation
1E-2 (11) (1.3)
1E-3 (11) (1.3)
1E-3 (11) (1.3)
Low on Air /
Out of Air
Situation
Misreads
Pressure gauge
1E-2 (11) (1.3)
Exceeds time
limit
1E-3 (11) (1.3)
Exceeds
planned dive
dept h
1E-2 (11) (1.3)
35
I-QMAS design assessment
Cost - $ billions
• Innovative system
• Innovative design
• Value Improvement
Program (VIP)
1.0
0.75
0
Time - months
12
36
I-QMAS assessment team
•
•
•
•
•
•
Engineers from design contractors (DC)
Engineering managers from DC
Engineers from owner / operator (O/O)
Engineering managers from O/O
Outside engineers and managers
HOF training program
37
Materials
Initial review
new
little experience
standard
good experience
Construction
(procedures, systems)
Structure
• Complicated procedures
• Low level engineering
experience
• Debates with management
• Advanced technology
• Lack of evaluation of results
of VIP on reliability
Design
(procedures, systems)
Construction
(personnel, management)
Design
(personnel, management)
Technology
Financial Resources
sufficient
insufficient
Personnel Resources
Tim e Resources
Quality Incentives
38
PRONE TO
LOW QUALITY
PRONE TO
HIGH QUALITY
Q/SMAS components, factors, attributes
39
I-QMAS review results
Interfaces
Environm ents
A
Structure
Equipment
Procedures
Organizations
Operators
0
1
2
3
4
Grades
5
6
7
40
I-QMAS recommendations
• Methods to evaluate reliability effects of
VIP alternatives (SYRAS + QMAS)
• Implement challenge process for design
procedures
• Assign experienced design engineers
• Temporarily assign construction &
operations personnel
• Develop robustness design program
41