Bayesian Inversion for the WIPP groundwater flow
problem
Björn Sprungk
joint work:
O. Ernst (U Chemnitz), K. A. Cliffe, (U Nottingham)
Workshop on Multiscale Inverse Problems
Mathematics Institue, University of Warwick
June 17–19, 2013
FAKULTÄT FÜR MATHEMATIK
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
1 / 40
Outline
1
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
Obtaining the prior
Bayesian Inversion using MCMC
Bayesian Inversion via EnKF
2
Gauss-Newton (preconditioned) MH-MCMC methods
3
Conclusions
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
2 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Next
1
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
Obtaining the prior
Bayesian Inversion using MCMC
Bayesian Inversion via EnKF
2
Gauss-Newton (preconditioned) MH-MCMC methods
3
Conclusions
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
3 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
Next
1
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
Obtaining the prior
Bayesian Inversion using MCMC
Bayesian Inversion via EnKF
2
Gauss-Newton (preconditioned) MH-MCMC methods
3
Conclusions
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
4 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
UQ Problem
Radioactive Waste Repository Site Assessment
Waste Isolation Pilot Plant (WIPP)
Carlsbad, New Mexico
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
5 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
UQ Problem
Radioactive Waste Repository Site Assessment
Waste Isolation Pilot Plant (WIPP)
Carlsbad, New Mexico
Groundwater transport of
radionuclides in transmissive
Culebra layer
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
5 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
UQ Problem
Radioactive Waste Repository Site Assessment
Waste Isolation Pilot Plant (WIPP)
Carlsbad, New Mexico
Groundwater transport of
radionuclides in transmissive
Culebra layer
Uncertainty in hydraulic
conductivity
source: Sandia National Labs
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
5 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
UQ Problem
Radioactive Waste Repository Site Assessment
Waste Isolation Pilot Plant (WIPP)
Carlsbad, New Mexico
Groundwater transport of
radionuclides in transmissive
Culebra layer
Uncertainty in hydraulic
conductivity
Quantity of interest: travel time to
boundary of WIPP site
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
5 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
UQ Problem
Radioactive Waste Repository Site Assessment
Waste Isolation Pilot Plant (WIPP)
Carlsbad, New Mexico
ECDF of log travel time
20 000 MC samples
1
Groundwater transport of
radionuclides in transmissive
Culebra layer
0.9
0.8
0.7
Uncertainty in hydraulic
conductivity
0.6
0.5
0.4
0.3
M = 30
M = 100
M = 500
M = 1000
0.2
0.1
0
4
4.5
5
5.5
log t
B. Sprungk (TU Chemnitz)
6
Quantity of interest: travel time to
boundary of WIPP site
Approach: Model uncertainty (lack
of knowledge) stochastically.
Propagate random input data to
travel time.
Bayesian Inversion of WIPP
Warwick, June 2013
5 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
Groundwater Flow Model
Stationary Darcy flow
q = −K ∇p
q : Darcy flux
K : hydraulic conductivity
p : hydraulic head
mass conservation
div u = 0
u : pore velocity
q = φu
φ : porosity
transmissivity
T = Kb
b : aquifer thickness
Particle transport
ẋ(t) = −
T (x)
∇p(x)
bφ
x(0) = x0
x : particle position
x0 : release location
Quantity of interest s: log10 of travel time of particle to reach WIPP boundary,
in particular, its cumulative distribution function (cdf).
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
6 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
PDE with Random Coefficient
Mixed form of Darcy equations:
T −1 (x)u(x) + ∇p(x) = 0,
div u(x) = 0,
p|∂D = p0 ,
x ∈ D.
Model T as a random field (RF) T = T (x, ω), ω ∈ Ω, with respect to underlying
probability space (Ω, A, P).
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
7 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
PDE with Random Coefficient
Mixed form of Darcy equations:
T −1 (x)u(x) + ∇p(x) = 0,
div u(x) = 0,
p|∂D = p0 ,
x ∈ D.
Model T as a random field (RF) T = T (x, ω), ω ∈ Ω, with respect to underlying
probability space (Ω, A, P).
Assumptions:
T has finite mean and covariance
T (x) = E [T (x, ·)] ,
CovT (x, y) = E T (x, ·) − T (x) T (y, ·) − T (y) ,
x ∈ D,
x, y ∈ D.
T is lognormal, i.e., Z (x, ω) := log T (x, ω) is a Gaussian RF.
CovZ is stationary, isotropic, and of Matérn type.
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
7 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Obtaining the prior
Next
1
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
Obtaining the prior
Bayesian Inversion using MCMC
Bayesian Inversion via EnKF
2
Gauss-Newton (preconditioned) MH-MCMC methods
3
Conclusions
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
8 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Obtaining the prior
WIPP Data
6
x 10
above median
below median
WIPP site boundary
3.595
transmissivity measurements at 38
test wells
UTM Northing
3.59
head measurements at 33 test wells,
used to obtain boundary data via
statistical interpolation (kriging)
3.585
constant layer thickness of b = 8m
3.58
constant porosity of φ = 0.16
SANDIA Nat. Labs reports
3.575
[Caufman et al., 1990]
[La Venue et al., 1990]
3.57
6.05
6.1
6.15
UTM Easting
B. Sprungk (TU Chemnitz)
6.2
5
x 10
Bayesian Inversion of WIPP
Warwick, June 2013
9 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Obtaining the prior
Prior Probabilistic Model of Transmissivity
Merge transmissivity data with statistical model
Given: Measurements of log transmissivity (xj , κj ), j = 1, . . . , N
Assumptions:
Linear model κ(x) =
Pn
i=1
βi fi (x) for mean
Matérn covariance structure for fluctuations around mean
Procedure:
(1) {fi }ni=1 yield point estimates of parameters in Matérn covariance function via
restricted maximum likelihood estimation (REML).
(2) Krige RF log T : best linear prediction of κ(x) based on measurements (for
Gaussian RF coincides with conditioning on observations).
(3) Approximate log T by truncated Karhunen-Loève expansion.
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
10 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Obtaining the prior
Matérn Family of Covariance Kernels
σ2
c(x, y) = cθ (r ) = ν−1
2
Γ(ν)
√ √ ν
2 νr
2 νr
Kν
,
ρ
ρ
r = kx − yk2
Kν : modified Bessel function of order ν
Parameters θ = (σ 2 , ρ, ν)
σ 2 : variance
ρ : correlation length
ν : smoothness parameter
Special cases:
ν=
1
2
:
ν=1:
ν→∞:
√
c(r ) = σ 2 exp(− 2r /ρ)
c(r ) = σ 2 2rρ K1 2rρ
2
2
2
c(r ) = σ exp(−r /ρ )
exponential covariance
Bessel covariance
Gaussian covariance
Smoothness: Realizations Z (·, ω) are k times differentiable ⇔ ν > k.
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
11 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Obtaining the prior
Kriging
Best unbiased linear prediction
Given RF κ with known covariance and observations {κ(xj ) = κj }N
j=1 , approximate
by
linear
prediction
E κ|{κ(xj ) = κj }N
j=1
κ̂(x) = m0 (x) +
N
X
mj (x) κ(xj )
j=1
such that it is unbiased E [κ̂(x)] = E [κ(x)] and optimal
h
i
2
E (κ̂(x) − κ(x)) → min !
m0 ,...,mN
Variants:
Simple Kriging (assumes known mean)
Universal Kriging (assumes linear regression model for mean)
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
12 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Obtaining the prior
Kriging
WIPP results
We assume constant but unknown mean κ̄(x) ≡ β.
REML estimates for covariance: σ 2 = 18.9, ρ = 9865, ν = 0.59.
6
6
x 10
x 10
Universal Kriging Variance
18
−7
3.595
3.595
16
−8
−9
3.59
14
3.59
−11
−12
3.58
12
UTM Northing
UTM Northing
−10
3.585
3.585
10
3.58
8
−13
6
3.575
−14
3.575
4
−15
3.57
3.57
2
−16
6.05
6.1
6.15
UTM Easting
B. Sprungk (TU Chemnitz)
6.05
6.2
5
x 10
Bayesian Inversion of WIPP
6.1
6.15
UTM Easting
6.2
5
x 10
Warwick, June 2013
13 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Obtaining the prior
Parametrization of Input RF
Karhunen-Loève expansion
∞ p
X
λm φm (x) ξm (ω)
κ(x, ω) = κ̄(x) +
m=1
converges in L∞ (D) and L2P (Ω), where
{(λm , φm )}m∈N
eigenpairs of covariance operator
Z
(C φ)(x) =
φ(y) Covκ (x, y) dy,
{ξm }m∈N ⊂ L2P (Ω),
E [ξm ] = 0,
D
E [ξk ξm ] = δk,m .
Truncation after M terms yields RF κM with
∞
h
i
X
E kκ − κM k2L2 (D) =
λm .
m=M+1
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
14 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Obtaining the prior
WIPP KL Modes
Conditioned on 38 transmissivity observations
unkriged, m = 1, 2, 9, 16
kriged, m = 1, 2, 9, 16
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
15 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion using MCMC
Next
1
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
Obtaining the prior
Bayesian Inversion using MCMC
Bayesian Inversion via EnKF
2
Gauss-Newton (preconditioned) MH-MCMC methods
3
Conclusions
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
16 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion using MCMC
The Inverse Problem
Bayesian Approach
exp(−κ)u = −∇p,
div u = 0,
p|∂D = p0
Further reduction of uncertainty using head measurements:
Finite number of observations of p: y = Q(p) = Q(p(κ)) ∈ Rk .
Measurement noise: d = y + ε, ε ∼ N(0, Γ)
Prior measure µ0 for κ ∈ span{φ1 , . . . , φM } ⊂ L∞ (D) by measurements of κ
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
17 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion using MCMC
The Inverse Problem
Bayesian Approach
exp(−κ)u = −∇p,
div u = 0,
p|∂D = p0
Further reduction of uncertainty using head measurements:
Finite number of observations of p: y = Q(p) = Q(p(κ)) ∈ Rk .
Measurement noise: d = y + ε, ε ∼ N(0, Γ)
Prior measure µ0 for κ ∈ span{φ1 , . . . , φM } ⊂ L∞ (D) by measurements of κ
Bayes theorem yields conditional distribution µd of κ|d
dµd
∝ exp(Φ(κ)),
dµ0
Φ(κ) =
1
kd − G (κ)k2Γ−1 ,
2
G = Q ◦ p.
Goal: Compute cdf of QoI s(κ) according to κ ∼ µd .
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
17 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion using MCMC
Bayesian Inversion for WIPP
Sampling from the posterior
Markov Chain Monte Carlo (MCMC)
Construct Markov chain with stationary distribution µd .
Sample sequence usually highly correlated, need for subsampling.
Prior model for κ:
κM (x, ξ) = φ0 (x) +
M
X
φm (x) ξm ,
ξ = (ξ1 , . . . , ξm ) ∼ N(0, I ),
m=1
Parametrization: prior on ξ ∈ RM instead of κ ∈ L∞ (D)
µ0 ∼ N(0, I ) on RM ,
Data: head measurements d = (p(x1 ), . . . , p(xk )) ∈ Rk , here k = 33.
Due to high dimension M: pCN-MCMC proposed in [Cotter et. al, 2012]
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
18 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion using MCMC
WIPP: MH-MCMC
Preliminary Results for M = 1000
ξ1
5
4
3
2
1
0
−1
−2
−3
0
1000
2000
3000
4000
5000
iteration
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
19 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion using MCMC
WIPP: MH-MCMC
Preliminary Results for M = 1000
ξ
1
5
4
3
2
1
0
−1
−2
−3
0
1000
2000
3000
4000
5000
iteration/100
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
19 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion using MCMC
WIPP: MH-MCMC
Preliminary Results for M = 1000
Comparing pCN and standard Random Walk
ACRF of Y(1)
ACRF of Y(10)
ACRF of Y(25)
ACRF of Y(50)
1
1
1
1
0.8
0.8
0.8
0.8
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0
0
100
200
300
400
500
0
0
100
200
300
400
500
0
0
0.2
100
200
300
lag
lag
lag
ACRF of Y(100)
ACRF of Y(200)
ACRF of Y(300)
400
500
0
0
1
1
0.8
0.8
0.8
0.8
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.2
100
200
300
400
500
0
0
100
200
300
400
500
0
0
200
300
lag
lag
ACRF of Y(500)
ACRF of Y(600)
ACRF of Y(800)
400
500
0
0
1
0.8
0.8
0.8
0.8
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
100
200
300
lag
B. Sprungk (TU Chemnitz)
400
500
100
200
300
400
500
500
100
lag
Bayesian Inversion of WIPP
200
300
lag
100
200
300
400
500
lag
1
0
0
400
ACRF of Y(1000)
1
0
0
300
0.2
100
lag
1
0
0
200
lag
1
0
0
100
ACRF of Y(400)
1
400
500
0
0
RW−MCMC
pCN−MCMC
100
200
300
400
500
lag
Warwick, June 2013
19 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion using MCMC
WIPP: Posterior Data Fit
Preliminary Results for M = 1000
Prior pressure head data fit (mean and 0.05/0.95 quantiles)
945
940
predicted pressure
935
930
925
920
915
910
905
905
910
915
920
925
930
935
940
observed pressure
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
20 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion using MCMC
WIPP: Posterior Data Fit
Preliminary Results for M = 1000
Posterior pressure head data fit (mean and 0.05/0.95 quantiles)
940
predicted pressure
935
930
925
920
915
910
905
905
910
915
920
925
930
935
940
observed pressure
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
20 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion using MCMC
WIPP: Posterior Data Fit
Preliminary Results for M = 1000
−3
2.2
x 10
Posterior data misfit−RMSE
Relative RMSE
2
1.8
1.6
1.4
1.2
1 1
10
2
10
3
10
Length M of KLE
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
20 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion using MCMC
WIPP: Posterior Data Fit
Preliminary Results for M = 33
Posterior pressure head data fit (mean and 0.05/0.95 quantiles)
940
predicted pressure
935
930
925
920
915
910
905
905
910
915
920
925
930
935
940
observed pressure
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
20 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion using MCMC
WIPP: Posterior distribution of ξ
Preliminary Results for M = 1000
Empirical moments of posterior marginals
1.5
mean
1
0.5
0
−0.5
standard deviation
−1
0
10
1
2
10
10
3
10
0.9
0.8
0.7
0.6
0.5 0
10
1
2
10
10
3
10
dimension
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
21 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion using MCMC
WIPP: Posterior distribution of QoI
Preliminary Results for M = 1000
Particle trajectories according to prior
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
22 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion using MCMC
WIPP: Posterior distribution of QoI
Preliminary Results for M = 1000
Particle trajectories according to posterior
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
22 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion using MCMC
WIPP: Posterior distribution of QoI
Preliminary Results for M = 1000
ECDF of log travel time for prior and posterior
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
prior
0.2
posterior
0.1
0
4
4.5
5
5.5
6
6.5
log travel time
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
22 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion using MCMC
WIPP: Posterior distribution of QoI
Preliminary Results for M = 1000
ECDFs of log travel timel: prior (solid) and posterior (dashed)
1
0.9
0.8
0.7
0.6
M=10
0.5
M=33
M=50
0.4
M=100
0.3
M=300
M=500
0.2
M=1000
0.1
0
4
4.2
4.4
4.6
4.8
5
5.2
5.4
5.6
5.8
6
log travel time
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
22 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion via EnKF
Next
1
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
Obtaining the prior
Bayesian Inversion using MCMC
Bayesian Inversion via EnKF
2
Gauss-Newton (preconditioned) MH-MCMC methods
3
Conclusions
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
23 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion via EnKF
Inversion via EnKF
Background
EnKF derived from Kalman filter for state estimation for incompletely
observed (stochastic) nonlinear dynamics.
Yields linear update of state estimate and estimation error by linear update of
the ensemble
State estimation by mean of ensemble, estimation error by covariance of
ensemble.
Can be applied to time-independent problems, too, e.g. elliptic PDEs.
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
24 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion via EnKF
Inversion via EnKF
Background
EnKF derived from Kalman filter for state estimation for incompletely
observed (stochastic) nonlinear dynamics.
Yields linear update of state estimate and estimation error by linear update of
the ensemble
State estimation by mean of ensemble, estimation error by covariance of
ensemble.
Can be applied to time-independent problems, too, e.g. elliptic PDEs.
Recent application of linear Bayesian update for UQ for PDE models
[Matthies et. al, 2012]
ξ d (ω) = ξ(ω) + K(d − G (ξ(ω)) − ε(ω)),
where K = Covξ,G (ξ) [CovG (ξ) + Covε ]−1 ∈ RM×k .
⇒ Yields posterior random variable ξ d instead of posterior measure µd .
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
24 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion via EnKF
Inversion via EnKF
Interpretation in Bayesian (statistics) context
Set Y(ω) = G (ξ(ω)), D(ω) = Y(ω) + ε(ω) and
ϕ̂(D) = E[ξ] + K D − E[G (ξ)] .
ϕ̂(D) linear approximation of E[ξ|D] = ϕ∗ (D):
ϕ̂ = argminϕ∈span{1,D} E kξ − ϕ(D)k2
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
25 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion via EnKF
Inversion via EnKF
Interpretation in Bayesian (statistics) context
Set Y(ω) = G (ξ(ω)), D(ω) = Y(ω) + ε(ω) and
ϕ̂(D) = E[ξ] + K D − E[G (ξ)] .
ϕ̂(D) linear approximation of E[ξ|D] = ϕ∗ (D):
ϕ̂ = argminϕ∈span{1,D} E kξ − ϕ(D)k2
E[ξ d ] = ϕ̂(d) and Cov(ξ d ) = E[kξ − ϕ̂(D)k2 ] (independent of d)
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
25 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion via EnKF
Inversion via EnKF
Interpretation in Bayesian (statistics) context
Set Y(ω) = G (ξ(ω)), D(ω) = Y(ω) + ε(ω) and
ϕ̂(D) = E[ξ] + K D − E[G (ξ)] .
ϕ̂(D) linear approximation of E[ξ|D] = ϕ∗ (D):
ϕ̂ = argminϕ∈span{1,D} E kξ − ϕ(D)k2
E[ξ d ] = ϕ̂(d) and Cov(ξ d ) = E[kξ − ϕ̂(D)k2 ] (independent of d)
In particular, for rd := ϕ∗ (d) − ϕ̂(d)
Z
Cov(ξ d ) =
Cov(ξ|δ) + rδ rδ> pD (δ) dδ,
Rk
where pD density of P ◦ D−1 .
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
25 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion via EnKF
Inversion via EnKF
Results
EnKF using nk even batches d = (d1 . . . , d33 ) = (d̄1 , . . . , d̄nk ) for nk updates.
Posterior ECDFs of log travel timel
1
MCMC
EnKF (nk=1)
0.9
0.8
EnKF (nk=3)
0.7
EnKF (n =11)
k
0.6
0.5
0.4
0.3
0.2
0.1
0
3.8
4
4.2
4.4
4.6
4.8
5
log travel time
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
26 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion via EnKF
Inversion via EnKF
Results
Using Gaussian approximation with final ensemble mean and covariance for UQ.
Posterior ECDFs of log travel timel
1
0.9
MCMC
EnKF (nk=1)
0.8
EnKF (nk=3)
0.7
EnKF (nk=11)
0.6
0.5
0.4
0.3
0.2
0.1
0
3.8
4
4.2
4.4
4.6
4.8
5
log travel time
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
27 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion via EnKF
Inversion via EnKF
Results
Prior pressure head data fit (mean and 0.05/0.95 quantiles)
945
940
predicted pressure
935
930
925
920
915
910
905
905
910
915
920
925
930
935
940
observed pressure
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
28 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion via EnKF
Inversion via EnKF
Results
Posterior pressure head data fit (mean and 0.05/0.95 quantiles)
945
940
predicted pressure
935
930
925
920
915
910
905
905
910
915
920
925
930
935
940
observed pressure
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
29 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion via EnKF
Inversion via EnKF
Comparison with MCMC
Errors relative to MCMC
kκ(ξ MCMC ) − κ(ξ EnKF )kL2 (D)
kκ(ξ MCMC )kL2 (D)
k Cov(κ(ξ MCMC )) − Cov(κ(ξ EnKF ))kL2 (D)⊗L2 (D)
k Cov(κ(ξ MCMC ))kL2 (D)⊗L2 (D)
no. updates
1
3
11
MCMC
B. Sprungk (TU Chemnitz)
rel. error mean
0.5728
0.3333
0.3516
-
rel. error cov
0.2752
0.2578
0.2785
-
Bayesian Inversion of WIPP
rel. data misfit
0.0026
0.0018
0.0017
0.0011
Warwick, June 2013
30 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion via EnKF
Observations
Need M = 300 KL modes to explain the data well.
Main changes from prior to posterior in the first, say, 100 KL modes.
For very high KL modes basically no change.
Significant uncertainty reduction by incooperating head data (total variance
reduced by 25%).
Need to take into account M = 1000 KL modes for accurate estimation of
posterior cdf of QoI.
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
31 / 40
Bayesian Inversion for the WIPP groundwater flow problem
Bayesian Inversion via EnKF
Observations
Need M = 300 KL modes to explain the data well.
Main changes from prior to posterior in the first, say, 100 KL modes.
For very high KL modes basically no change.
Significant uncertainty reduction by incooperating head data (total variance
reduced by 25%).
Need to take into account M = 1000 KL modes for accurate estimation of
posterior cdf of QoI.
Perspectives:
Run inversion (MCMC) only in active dimensions where significant change
from prior to posterior.
Use surrogates for solving parametric PDE in these parameters for computing
Hastings ratio.
Estimate posterior cdf of QoI according to posterior in active and prior in
inactive dimensions.
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
31 / 40
Gauss-Newton (preconditioned) MH-MCMC methods
Next
1
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
Obtaining the prior
Bayesian Inversion using MCMC
Bayesian Inversion via EnKF
2
Gauss-Newton (preconditioned) MH-MCMC methods
3
Conclusions
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
32 / 40
Gauss-Newton (preconditioned) MH-MCMC methods
Motivation
Observation:
Sufficiently small prior variance yields approx. linear relationship between ξ and
p(ξ).
Thus, for µ0 ∼ N(0, Σ0 ), ε ∼ N(0, Γ), posterior is approximately Gaussian, too,
µd ≈ N ( K(d − G (0)), Σ0 − KLΣ0 ) ,
where L = ∇ξ G (0) and K = Σ0 L> (LΣ0 L> + Γ)−1 . [Cliffe & Jackson, 2001]
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
33 / 40
Gauss-Newton (preconditioned) MH-MCMC methods
Motivation
Observation:
Sufficiently small prior variance yields approx. linear relationship between ξ and
p(ξ).
Thus, for µ0 ∼ N(0, Σ0 ), ε ∼ N(0, Γ), posterior is approximately Gaussian, too,
µd ≈ N ( K(d − G (0)), Σ0 − KLΣ0 ) ,
where L = ∇ξ G (0) and K = Σ0 L> (LΣ0 L> + Γ)−1 . [Cliffe & Jackson, 2001]
For target µd ∼ N(0, Σ) optimal Random-Walk proposal is q(ξ, dη) ∼ N(ξ, s 2 Σ).
[Roberts & Rosenthal, 2001]
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
33 / 40
Gauss-Newton (preconditioned) MH-MCMC methods
Example
1D toy problem
−∇(exp(κ)∇p) = 0 in D = [0, 100],
κ(x, ξ) =
P10
m=1 cm ξm
p(0) = 0, p(100) = 100,
1[ m−1 , m ] (x/100),
10
10
Qp = (p(15), p(35), p(65), p(85))> ,
ξm ∼ N(0, 1) iid,
ε ∼ N(0, 0.01I4 ).
Synthetic data: ξ true draw from µ0
p
κ
1.1
100
90
1.05
80
1
70
0.95
60
0.9
50
40
0.85
30
0.8
20
0.75
0.7
10
0
20
40
60
80
100
0
0
x
B. Sprungk (TU Chemnitz)
20
40
60
80
100
x
Bayesian Inversion of WIPP
Warwick, June 2013
34 / 40
Gauss-Newton (preconditioned) MH-MCMC methods
Example
1D toy problem
−∇(exp(κ)∇p) = 0 in D = [0, 100],
p(0) = 0, p(100) = 100,
ACRF of ξ
5
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
standard RW
0.2
0.1
0
0
preconditioned RW
200
400
600
800
1000
lag
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
34 / 40
Gauss-Newton (preconditioned) MH-MCMC methods
Example
1D toy problem
−∇(exp(κ)∇p) = 0 in D = [0, 100],
p(0) = 0, p(100) = 100,
P
κtrue (x) = m ξtrue,m 1[ m−1 , m ] (x/100)
Synthetic data: ξ true ∼ N(1, 0.5 I10 ),
10
10
ACRF of ξ5
1
0.98
0.96
0.94
0.92
0.9
0.88
0.86
standard RW
0.84
0.82
0.8
0
preconditioned RW
200
400
600
800
1000
lag
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
34 / 40
Gauss-Newton (preconditioned) MH-MCMC methods
Improvement
Stochastic Gauss-Newton
Local linearization for Random-Walk proposal:
q(ξ, dη) ∼ N(ξ, s 2 Σ(ξ)),
>
−1
Σ(ξ) = Σ0 − Σ0 L>
ξ (Lξ Σ0 Lξ + Γ) Lξ Σ0 ,
where Lη = ∇ξ G (η).
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
35 / 40
Gauss-Newton (preconditioned) MH-MCMC methods
Improvement
Stochastic Gauss-Newton
Local linearization for Random-Walk proposal:
q(ξ, dη) ∼ N(ξ, s 2 Σ(ξ)),
>
−1
Σ(ξ) = Σ0 − Σ0 L>
ξ (Lξ Σ0 Lξ + Γ) Lξ Σ0 ,
where Lη = ∇ξ G (η).
−1
Note Σ(ξ) = (LT
is inverse Hessian of
ξ ΓLξ + Σ0 )
1
1
kd − G̃ (ξ)k2Γ−1 + kξ − ξ 0 k2Σ−1
0
2
2
with linearized G̃ (η) = G (ξ) + ∇ξ G (ξ)(η − ξ).
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
35 / 40
Gauss-Newton (preconditioned) MH-MCMC methods
Improvement
Stochastic Gauss-Newton
Local linearization for Random-Walk proposal:
q(ξ, dη) ∼ N(ξ, s 2 Σ(ξ)),
>
−1
Σ(ξ) = Σ0 − Σ0 L>
ξ (Lξ Σ0 Lξ + Γ) Lξ Σ0 ,
where Lη = ∇ξ G (η).
−1
Note Σ(ξ) = (LT
is inverse Hessian of
ξ ΓLξ + Σ0 )
1
1
kd − G̃ (ξ)k2Γ−1 + kξ − ξ 0 k2Σ−1
0
2
2
with linearized G̃ (η) = G (ξ) + ∇ξ G (ξ)(η − ξ).
Yields stochastic Gauss-Newton method (cf. [Martin et al., 2011])
s2
q(ξ, dη) ∼ N ξ − ∇ξ J(ξ) Σ(ξ), Σ(ξ) ,
2
where J(ξ) = 21 kd − G (ξ)k2Γ−1 + 12 kξ − ξ 0 k2Σ−1 .
0
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
35 / 40
Gauss-Newton (preconditioned) MH-MCMC methods
Remarks on stochastic Gauss-Newton (sGN)
−1
sGN is preconditioned MALA with preconditioner Σ(ξ) = (LT
ξ ΓLξ + Σ0 ) ,
cf. [Girolami & Calderhead, 2011].
Σ(ξ) positive semi-definite in contrast to true Hessian of J.
>
−1
Computing Σ0 − Σ0 L>
ξ (Lξ Σ0 Lξ + Γ) Lξ Σ0 requires only inverse of
k × k-matrix
∇ξ G (η) easily computable via adjoint method.
Since adjoint equations as original PDE, except for source term, they allow
for the same surrogate (e.g. stochastic collocation).
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
36 / 40
Gauss-Newton (preconditioned) MH-MCMC methods
Example
1D toy problem
−∇(exp(κ)∇p) = 0 in D = [0, 100],
Synthetic data: ξ true ∼ N(1, 0.5 I10 ),
p(0) = 0, p(100) = 100,
P
κtrue (x) = m ξtrue,m 1[ m−1 , m ] (x/100)
10
10
ACRF of ξ5
1
0.9
standard RW
0.8
GN−preconditioned RW
0.7
MALA
0.6
sGN (A(s) = 0.55)
sGN (A(s) = 0.25)
0.5
0.4
0.3
0.2
0.1
0
0
100
200
300
400
500
lag
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
37 / 40
Gauss-Newton (preconditioned) MH-MCMC methods
Example
1D toy problem
−∇(exp(κ)∇p) = 0 in D = [0, 100],
Synthetic data: ξ true ∼ N(1, 0.5 I10 ),
p(0) = 0, p(100) = 100,
P
κtrue (x) = m ξtrue,m 1[ m−1 , m ] (x/100)
10
MALA path in ξ5
GN−prec. RW path in ξ5
3
3
2
2
1
1
0
0
−1
−1
−2
−2
−3
0
2
4
6
iteration
B. Sprungk (TU Chemnitz)
10
8
10
−3
0
4
x 10
Bayesian Inversion of WIPP
2
4
6
iteration
8
10
4
x 10
Warwick, June 2013
37 / 40
Gauss-Newton (preconditioned) MH-MCMC methods
Example
1D toy problem
−∇(exp(3κ)∇p) = 0 in D = [0, 100], p(0) = 0, p(100) = 100,
P
Synthetic data: ξ true ∼ µ0 , κtrue (x) = m 3cm ξtrue,m 1[ m−1 , m ] (x/100)
10
10
ACRF of ξ5
1
0.95
0.9
0.85
standard RW
0.8
GN−preconditioned RW
MALA
0.75
sGN (A(s) = 0.57)
sGN (A(s) = 0.22)
0.7
0
100
200
300
400
500
lag
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
37 / 40
Conclusions
Next
1
Bayesian Inversion for the WIPP groundwater flow problem
Problem setting and motivation
Obtaining the prior
Bayesian Inversion using MCMC
Bayesian Inversion via EnKF
2
Gauss-Newton (preconditioned) MH-MCMC methods
3
Conclusions
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
38 / 40
Conclusions
Concluding Remarks
Bayesian Inversion / MCMC challenging for WIPP problem due to high
parameter dimension and high chain correlation.
Possible remedy: reduced set of active dimensions for Bayesian inversion,
advanced MCMC methods and surrogates for solving PDE.
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
39 / 40
Conclusions
Concluding Remarks
Bayesian Inversion / MCMC challenging for WIPP problem due to high
parameter dimension and high chain correlation.
Possible remedy: reduced set of active dimensions for Bayesian inversion,
advanced MCMC methods and surrogates for solving PDE.
sGN methods exploit geometrical structure of posterior as stochastic Newton
and Riemann manifold MCMC methods for possibly less cost
Extension and further analysis of sGN (and preconditioned RW) to infinite
dimensions to be done.
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
39 / 40
Conclusions
Concluding Remarks
Bayesian Inversion / MCMC challenging for WIPP problem due to high
parameter dimension and high chain correlation.
Possible remedy: reduced set of active dimensions for Bayesian inversion,
advanced MCMC methods and surrogates for solving PDE.
sGN methods exploit geometrical structure of posterior as stochastic Newton
and Riemann manifold MCMC methods for possibly less cost
Extension and further analysis of sGN (and preconditioned RW) to infinite
dimensions to be done.
Further goal: adaptive refinement of stochastic collocation surrogates for
MCMC (necessary if posterior locates in tails of prior).
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
39 / 40
Conclusions
Thank you for your attention!
B. Sprungk (TU Chemnitz)
Bayesian Inversion of WIPP
Warwick, June 2013
40 / 40