Quantitative Methods for DecisionDecision
Making Under Uncertainty
Sankaran Mahadevan
Vanderbilt University, Nashville, TN
Email:
Website:
sankaran.mahadevan@vanderbilt.edu
www.reliability-studies.vanderbilt.edu
Vanderbilt University
reliability-studies.vanderbilt.edu
Reliability and Risk Engineering, Analysis, and Management
NSF-IGERT Graduate Program at Vanderbilt
Educational and Research Themes
y Multidisciplinary integration
y Large, complex systems
y Modeling and simulation
y Economic, legal, regulatory,
and social perspectives
Cross-cutting
g methodologies
g
y Uncertainty quantification, propagation
y Risk quantification
y Decision-making under uncertainty
Devices,
Components
Large
Systems
Model
Integration
Uncertainty
& Risk
Methods
Participants
• 42 graduate students (34 Ph.D., 8 M.S.)
• 30 professors Æ Engineering, Math, Economics, Business, Psychology,
Medicine
• Summer researchers (undergraduates, high school teachers)
Vanderbilt University
reliability-studies.vanderbilt.edu
Reliability and Risk Engineering, Analysis, and Management
Multiple
p Resources at Vanderbilt
CRESP -- Multi-university
Nuclear Waste Risk
Assessment Consortium
(D. Kosson)
ACCRE High Performance
Computing Network
Business systems
Risk Assessment and
Management
(B Cooil)
(B.
Vanderbilt University
Structural/Mechanical
Systems Reliability
(S Mahadevan)
(S.
M h d
)
Institute for Space and
Defense Electronics
(R. Schrimpf)
Multidisciplinary Doctoral
Program in Reliability/Risk
Engineering & Management
VECTOR Transportation
Research Center
(M. Abkowitz)
Institute for Software
Integrated Systems
(G. Karsai)
Vanderbilt Center for
Environmental Management
Systems (VCEMS)
(M Abkowitz)
(M.
reliability-studies.vanderbilt.edu
Reliability and Risk Engineering, Analysis, and Management
Industry
y & Government Support
pp
INDUSTRY
y Boeing
y Kellogg, Brown & Root
y Fedex
y General Motors
y Chrysler
y General Electric
y Pratt and Whitney
y Bell Helicopter
y Medtronic
y Union Pacific
y Xerox
GOVERNMENT
y U. S. DOD ((AFRL,, Army,
y, Navy,
y, AFOSR))
y U. S. DOE (EM)
y U. S. DOT (FHWA)
y NASA (LaRC,
(LaRC MSFC,
MSFC GRC,
GRC ARC,
ARC JPL)
y DOE Labs (SNL, LANL, INL, SRNL)
y Nuclear Regulatory Commission
y Federal Aviation Administration
Laboratories
• Southwest
S th
t Research
R
h IInstitute
tit t
• Transportation Technology Center
Vanderbilt University
Nature of support
S
Summer
iinternships
t
hi
Collaborative research projects
Advisory committee membership
p
Seminar speakers
Student recruitment
reliability-studies.vanderbilt.edu
Risk Analysis Issues
• System modeling
•
•
•
•
Physics-based behavior models Æ finite elements, bond graphs
Surrogate models (GP, PC, RBF, NN)
Fault trees, event trees, petri-nets
Bayes networks
• Risk analysis
• Multi-level -- Material Æ component Æ subsystem Æ system
• Risk variation over space and time
• Multi-physics, multi-scale problems
• Data Uncertainty
• Sparse data, interval data, measurement uncertainty
• Expert opinion
• Heterogeneous information
• Model Uncertainty
y
• Model form, model parameters
• Errors Æ some deterministic, some stochastic
Vanderbilt University
reliability-studies.vanderbilt.edu
Reliability
y and Risk Engineering,
g
g Analysis,
y
and Management
g
y Materials durability, fatigue, fracture
y Systems health diagnosis and prognosis
y Decision-making under uncertainty
y Model uncertainty,
y calibration, validation
Information
Uncertainty
Physical
Variability
Model
Uncertainty
Vanderbilt University
FEM and
Dynamic
Analysis
Uncertainty
Modeling
Damage
Mechanism
Analysis
Usage
monitoring
Probabilistic
Fatigue
Prognosis
Inspection/
Testingg
Model
update
Bayesian
Updating
Decision making /
Risk management
Reliability
update
p
reliability-studies.vanderbilt.edu
Rotorcraft Damage Tolerance (FAA)
Rotorcraft mast
Analyses
•
Two-diameter hollow cylinder
•
•
Elliptical
Elli
ti l surface
f
crack
k iin fill
fillett
region
•
Sub modeling technique Æ
Accuracy in stress intensity factor
Vanderbilt University
Model calibration
– C
Calibrate EIFS,
S model parameters
– Estimate model errors in different stages
of modeling
•
Model validation
•
Prediction uncertainty quantification
•
Global sensitivity analysis
•
Load monitoring and updating
reliability-studies.vanderbilt.edu
Sources of Uncertainty
• Physical variability
• Loading
• Material Properties
• Data uncertainty
• Sparseness of data used to quantify material properties
• Output measurement uncertainty (final crack size, detection probability)
• Model uncertainty/errors
•
•
•
•
•
•
Analysis assumptions Æ LEFM,
LEFM planar crack
Finite element discretization errors
Combination of multiple crack modes
Approximation due to surrogate model
Crack growth law Æ model form
Model parameters Æ crack growth model
initial flaw size
Vanderbilt University
reliability-studies.vanderbilt.edu
Dynamic Bayes Network
Cycle
y i
Cycle
y
i+1
ai+1
ai
θ
Vanderbilt University
ΔK
εexp
ΔK
C, m, n
C, m, n
εcg
εcg
ΔKth, σf
ΔKth, σf
Slide 8 of 22
aN
A
reliability-studies.vanderbilt.edu
Model Validation Metrics
Model response
Observed values
x
y
Null Hypothesis
H0: y ∈ f(x)
Alternative Hypothesis:
H1: y ∉ f(x)
Classical hypothesis testing
H0: E(y) = E(x)
Var(y) = Var(x)
H1: E(y) ≠ E(x)
Var(y) ≠ Var(x)
t test
chi-square test
Bayesian hypothesis testing
From Bayes
theorem
Æ
P (M i observation )
⎡ P (observation M i ) ⎤ ⎡ P(M ) ⎤
i
=⎢
⎥⎢
⎥
P (M j observation ) ⎢ P observation M j ⎥ ⎢⎣ P (M j )⎥⎦
⎣
⎦
(
B
Bayes
F t
Factor
B
Vanderbilt University
)
“Confidence”
Confidence
= B/B+1
reliability-studies.vanderbilt.edu
Crack size prediction UQ
Std. Dev.
Bayes
Factor
Vanderbilt University
Confidence
reliability-studies.vanderbilt.edu
Sensitivity Analysis
• Local Æ Only one uncertainty is considered and all
others are ‘frozen’
frozen at the mean values
• Global Æ Analyze sensitivity of output over the entire
domain of inputs
p
rather than at mean values
• First order effects (S) & Total effects (ST)
Si =
V X i ( E X ~i (Y | X i ))
V (Y )
Initial flaw size
Vanderbilt University
STi =
E X ~i (VX i (Y | X ~i ))
V (Y )
Crack model error
= 1−
VX ~i ( E X i (Y | X ~i ))
V (Y )
Crack model
parameter C
reliability-studies.vanderbilt.edu
Cementitious barrier PA
Multi-physics analysis
Diffusion of
Ions
Inputs
- Porosity
- Composition
- Modulus
- …..
Chemical Reactions
Damage
Progression
Damage
Accumulation
Calibration of chemical reaction
model parameters (equilibrium
constants)
t t)
Durability prediction
Vanderbilt University
reliability-studies.vanderbilt.edu
Extrapolation from Validation to Application
Bayes Network
m
Use for
• Calibration
C
e
b
a
• Validation
g
d
c
• Extrapolation
f
Extrapolation scenarios
MCMC techniques
Gibbs sampling
• Nominal values to Extreme values
• Test conditions to Use conditions
• Validation variables to Decision variables
• Components to System
Vanderbilt University
reliability-studies.vanderbilt.edu
UQ in system-level prediction
Level 1
Level 0
Level 2
System
level
Foam
Material
characterization
Component level
Sub-system
level
Joints
Hardware data and photos courtesy of Sandia National Laboratories
System complexity
Sources of uncertainty
Amount of real data
Vanderbilt University
Increases
Increase
Decreases
reliability-studies.vanderbilt.edu
Bayes Network Implementation
Foam
Y1 j
Y1f
X 1f
Y1f
X 1f
ε
ε1j
f
1
X 2f
Vanderbilt University
θ
j
X 2j
Y2j
Y2f
J = Joints
F = Foam
θ = Calibration parameters
ε = Error terms
X 1j
ε 2j
ε 2f
θf
Joints
Xs
Y = Experimental data
X = FEM prediction
1 - Level 1
2 - Level 2
S - System
reliability-studies.vanderbilt.edu
Likelihood Approach to Data Uncertainty
• Likelihood function
⎛ n
L(P) ∝ ⎜ ∏
⎜ i =1
⎝
bi
∫
ai
⎞
f X ( x | P )dx ⎟
⎟
⎠
interval data
⎛ m
⎞
⎜⎜ ∏ f X ( x i | P ) ⎟⎟
⎝ i =1
⎠
P – distribution parameters
m – point data size
n – interval data size
sparse point data
• Maximum Likelihood Estimate Æ Maximize L(P)
• To account for uncertainty in P Æ
f (P) =
L(P)
∫ L(P)
• Two approaches
– Family of distributions for X (for every sample of P Æ probability
distribution for X)
– Single
Si l di
distribution
t ib ti off X
f ( x ) = ∫ f ( x | P ) f ( P ) dP
• Can use non-parametric distributions
Vanderbilt University
reliability-studies.vanderbilt.edu
Risk Management: System Health Monitoring
• System integration
• Integrate reliability/risk methods with SHM
• Integrate diagnosis with prognosis
• Rapid diagnosis and prognosis
• Derive damage signatures
• Qualitative isolation, then damage quantification
• Uncertainty Quantification
• Quantify variability, uncertainty, errors
• Estimate Confidence in diagnosis/prognosis
Vanderbilt University
reliability-studies.vanderbilt.edu
Decision-Making Under Uncertainty
Optimization
MDA
A1
A2
Risk Analysis
• Various stages in life cycle Æ design, operations, maintenance
• Multiple objectives, MCDA, decision trees, utility-based formulations
• Multi-disciplinary systems
• Optimization for reliability and robustness
• Include both aleatory and epistemic uncertainties
• Dynamic,
y
network systems
y
• Critical facility protection – design of safeguards/detectors
• Transportation networks, supply networks, emergency response systems
• System of systems
• Multiple system linkages
• Homeland security, military, commercial applications
Vanderbilt University
reliability-studies.vanderbilt.edu
Fire Satellite System
Target latitude,
Target longitude
Target size
[Altit d ]
[Altitude]
Analyses
Multi-disciplinary uncertainty propagation
Design optimi
optimization
ation for reliabilit
reliability, rob
robustness
stness
Orbit period, eclipse period
Orbit
Power
P_ACS
_
Orbit Period,
Satellite velocity,
Max slewing angle
I_min, I_max
Attitude
[P_tot], [T_tot], [A_sat], [I_min], [I_max]
Vanderbilt University
reliability-studies.vanderbilt.edu
System of Systems Decision-Making Under Uncertainty
Decision
D
i i
making
• Risk-informed
• Design
Optimization
• Operations
• Non-linear
• Utility theory
• Stochastic
• Static/Dynamic
Modeling
• Coefficient based
• System dynamics
• Agent based
S
t model
d l
• Surrogate
• Monolithic
• Family of Systems
• System of Systems
SOS
• Fully rational
• Bounded rational
Human in the
loop
System
Complexity
• Information bonded
• Energy bonded
• Hybrid
• Aleatory
• Epistemic
Uncertainty
• Analysis
• Management
Risk
Types
yp of
Systems
Vanderbilt University
reliability-studies.vanderbilt.edu
Pandemic Influenza Risk Management
CIPDSS (LANL)
Vanderbilt University
reliability-studies.vanderbilt.edu
Conclusion
Continuing opportunities for methods development
y System risk assessment
System risk assessment
y
y
y
Risk variation with time and space
Dynamic, multi‐physics, systems of systems
Computational effort
Computational effort
y Decision‐making under uncertainty
y
y
y
Design, operations, maintenance, risk management
Data collection Model de elopment
Data collection, Model development
Embedding flexibility
y Include data uncertainty
y Sparse, noisy, qualitative, missing data, intervals, expert opinion
l
d
l
y Multi‐scale fusion of heterogeneous information
y Include model uncertainty
y
y Validation, calibration, error estimation, extrapolation
Vanderbilt University
reliability-studies.vanderbilt.edu