Well-Posed Bayesian Geometric Inverse Problems
Arising In Groundwater Flow
Andrew Stuart
Mathematics Institute
The University of Warwick
Multiscale Inverse Problems, June 17th–19th, 2013
Collaboration with Marco Iglesias (Warwick), Kui Lin (Fudan)
Funded by EPSRC and ERC
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
1 / 31
Outline
1
Introduction
2
Forward And Inverse Problem
3
Bayesian Framework
4
MCMC Method
5
Numerical Results
6
Conclusions
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
2 / 31
Introduction
Outline
1
Introduction
2
Forward And Inverse Problem
3
Bayesian Framework
4
MCMC Method
5
Numerical Results
6
Conclusions
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
3 / 31
Introduction
Introduction
References
D. A. White J. N. Carter.
History matching on the Imperial College fault model using parallel
tempering.
Computational Geosciences, 17(2013) 43–65.
M.Iglesias, K. Lin and A.M. Stuart
Well-posed Bayesian geometric inverse problems arising in
groundwater flow.
In preparation.
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
4 / 31
Forward And Inverse Problem
Outline
1
Introduction
2
Forward And Inverse Problem
3
Bayesian Framework
4
MCMC Method
5
Numerical Results
6
Conclusions
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
5 / 31
Forward And Inverse Problem
Forward And Inverse Problems
Groundwater Flow Model
pore pressure (head) : p(x)
permeability (hydraulic conductivity) : κ(x)
Single-Phase Darcy Flow:
find p ∈ H01 (D), given κ ∈ L∞ (D)
−∇ · (κ(x)∇p) = f ,
p = 0,
x ∈ D,
x ∈ ∂D,
Inverse Problem: find κ ∈ L∞ (D), given noisy yj
yj = `j (p) + ηj ,
ηj ∼ N(0, γ 2 ),
Mathematics Institute (Warwick)
`j ∈ H −1 (D).
Geometric Inverse Problems
Andrew Stuart
6 / 31
Forward And Inverse Problem
Geometric Model
Piecewise Constant Permeability
κ(x) =
n
X
κi χDi (x),
i=1
where
S
i
Di = D and Di ∩ Dj = ∅, ∀i 6= j.
a1
a2
a3
..
.
a1
b1
a2
b2
b3
b2
..
.
b3
..
.
an
..
.
an
bn
Figure : A multiple layers Model
Mathematics Institute (Warwick)
b1
a3
Geometric Inverse Problems
bn
Figure : A fault Model
Andrew Stuart
7 / 31
Forward And Inverse Problem
Forward and Observation Map
Geometry
a1
parameterization
b1
κ
1
unknowns: u = (κT , aT , bT , c)T
permeability : u → κ(x)
c
a2
κ2
a3
κ
b2
c
b3
3
Forward Operator
G:
u
R
d
7−→
κ(x)
∞
7−→
7−→ L (D) 7−→
p(x)
H01 (D)
7−→ y
7−→ RJ
Find u from y
y = G(u) + η.
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
8 / 31
Bayesian Framework
Outline
1
Introduction
2
Forward And Inverse Problem
3
Bayesian Framework
4
MCMC Method
5
Numerical Results
6
Conclusions
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
9 / 31
Bayesian Framework
Bayesian Approach
y = G(u) + η,
η ∼ N(0, Γ).
Prior distribution on u:
µ0 (du) = P(du).
Negative log likelihood on y |u:
Φ(u; y ) =
1
|y − G(u)|2Γ ,
2
1
where | · |Γ = |Γ− 2 · |.
Posterior distribution on u|y :
µy (du) = P(du|y ).
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
10 / 31
Bayesian Framework
Prior Modeling
Geometric prior π0G and π0slip
π0G : a, b ∼ uniformly in A := {x ∈ Rn−1 |
Pn−1
i=1
xi ≤ 1, xi ≥ 0}.
slip
π0 : c ∼ uniformly in [−C, C].
a, b, c are independent.
Permeability prior π0i on κi
π0i (x)
log-normal: κi = exp(ξi ), ξi drawn from Gaussian.
log-normal
uniform
uniform: κi drawn from (> 0) uniform distribution.
exponential
exponential: κi drawn from exponential distribution.
x
Prior on u ∈ U := (0, ∞)n × A2 × [−C, C]
π0 (u) =
n
Y
slip
π0i (κi ) × π0G (a)π0G (b)π0 (c).
i=1
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
11 / 31
Bayesian Framework
Well-Defined Posterior
Theorem (Bayes’ Theorem [4])
Assume that Φ(u; y ) : U × Y → R is measurable and,
Z
Z :=
exp(−Φ(u; y ))µ0 (du) > 0
U
Then the conditional distribution µy of u|y exists, µy µ0 and
dµy
1
= exp (−Φ(u; y )) .
dµ0
Z
We establish this theorem for our problem by proving continuity of G,
(and hence of Φ(·; y )) on a set of full µ0 measure.
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
12 / 31
Bayesian Framework
Continuity of G
Applying Bayes’ Theorem (see [3])
G is continuous on U,
µ0 (U) = 1 and Z > 0
=⇒
u → κ(x) → p(x)
uε
→
κε (x)
→
BUT
µy µ0
dµy
1
= exp (−Φ(u; y )) .
dµ0
Z
κ ∈ L∞ (D) → p ∈ H01 (D)
is Lipschitz.
u ∈ Rd → κ ∈ Lr (D)
continuous only for r < ∞.
pε (x)
−∇ · (κε (x)∇(pε − p)) = ∇ · ((κε − κ)∇p),
ε
p − p = 0,
Mathematics Institute (Warwick)
Geometric Inverse Problems
x ∈D
x ∈ ∂D.
Andrew Stuart
13 / 31
Bayesian Framework
Continuity of G
kpε − pkV ≤
1
Z
κmin
n
1 X ε
|κ − κi |2
κmin
i
i=1
a1
u→u
1
2
D

=
|κε − κ|2 |∇p|2 dx
Z
2
|∇p| dx +
Diε ∩Di
X
i6=j
b1
κ1
|κεi
− κj |
2
κ(x) → κ (x)
a2
κ2
b2 1
Figure : κ(x), κε (x)
Mathematics Institute (Warwick)
Geometric Inverse Problems
2
2
Diε ∩Dj
|∇p| dx  .
κε1 − κ1
ε
ε
1
Z
κε1 − κ2
κε2 − κ1
κε2 − κ2
Figure : κε (x) − κ(x)
Andrew Stuart
14 / 31
Bayesian Framework
Well-Poseness of Posterior
The Hellinger distance:

1
dHell (µ, µ0 ) = 
2
Z
U
r
dµ
−
dν
r
dµ0
dν
!2
1
2
dν  .
The Total Variation distance:
1
dTV (µ, µ ) =
2
0
Z dµ dµ0 dν.
−
dν U dν
Relationship:
1
1
√ dTV (µ, µ0 ) ≤ dHell (µ, µ0 ) ≤ dTV (µ, µ0 ) 2 .
2
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
15 / 31
Bayesian Framework
Well-Poseness of Posterior
µ (resp µ0 ) the posterior corresponding to data y (resp. y 0 ).
Theorem (Well-Posedness [3])
Assume that max{|y |, |y 0 |} < r . Then there are Ci = Ci (r ) such that:
(I) κi ∼ log-normal or κi ∼ uniform, then
dTV (µ, µ0 ) ≤ C1 |y − y 0 |,
dHell (µ, µ0 ) ≤ C2 |y − y 0 |.
(II) κi ∼ exponential, then
dTV (µ, µ0 ) ≤ C1 |y − y 0 |α ,
α
dHell (µ, µ0 ) ≤ C2 |y − y 0 | 2 ,
where 0 < α < 1.
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
16 / 31
Bayesian Framework
Well-Poseness of Posterior
Sketch of proof:
dTV (µ, µ0 ) ≤ I1 + I2 .
where
Z
1
exp (−Φ(u, y )) − exp −Φ(u, y 0 ) dµ0 (u),
I1 =
2Z U
Z
1 −1
0 −1
I2 = |Z − (Z ) |
exp −Φ(u, y 0 ) dµ0 (u).
2
U
And
I2 ≤ CI1 .
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
17 / 31
Bayesian Framework
Well-Poseness of Posterior
1
I1 =
2Z
Z
U
exp (−Φ(u, y )) − exp −Φ(u, y 0 ) dµ0 (u)
exp (−Φ(u, y )) − exp −Φ(u, y 0 ) ≤ Φ(u, y ) − Φ(u, y 0 ) ≤ M(r , u)|y − y 0 |.
(I): log-normal or uniform µ0 on u
Z
Z
0
|Φ(u; y ) − Φ(u, y )|dµ0 (u) ≤
M(r , u)dµ0 (u) |y − y 0 |.
U
U
(II): exponential µ0 on u
R
R
0 α
α
0 α
{|Φ(u;y )−Φ(u,y 0 )|≤1} |Φ(u; y ) − Φ(u, y )| dµ0 ≤ U M dµ0 |y − y | .
R
µ0 (|Φ(u; y ) − Φ(u, y 0 )| > 1) ≤ U M α dµ0 |y − y 0 |α .
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
18 / 31
MCMC Method
Outline
1
Introduction
2
Forward And Inverse Problem
3
Bayesian Framework
4
MCMC Method
5
Numerical Results
6
Conclusions
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
19 / 31
MCMC Method
Metropolis Hasting Method
General Metropolis-Hasting Algorithm: to sample π(u):
1
2
3
k = 0, initialize u (0) .
Propose v ∈ Rd from q(u (k ) , v ).
Set
u
4
(k +1)
=
v
with probability a(u (k ) , v )
,
(k
)
u
otherwise
k → k + 1 and repeat 2.
Here
a(u, v ) =
Mathematics Institute (Warwick)
π(v )q(v , u)
∧ 1.
π(u)q(u, v )
Geometric Inverse Problems
Andrew Stuart
20 / 31
MCMC Method
Prior Reversible Proposal
To sample π(u) = exp −Φ(u) π0 (u):
Choose q(u, v ) s.t.
π0 (u)q(u, v ) = π0 (v )q(v , u),
Then
a(u, v ) =
∀u, v ∈ Rd .
π(v )q(v , u)
∧ 1,
π(u)q(u, v )
π0 (v ) exp − Φ(v ) q(v , u)
∧ 1,
=
π0 (u) exp −Φ(u) q(u, v )
= exp Φ(u) − Φ(v ) ∧ 1.
Propose using prior; accept/reject according to model-data misfit.
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
21 / 31
MCMC Method
Two-Step Metropolis Hasting Method
Algorithm:
1
k = 0, initialize u (0) ∈ U.
2
3
4
Draw w ∈ Rd from p(u (k ) , w).
Propose v ∈ U defined by
w
w ∈U
v=
.
u (k ) w ∈
/U
Accept or reject v :
u (k +1) =
v
with probability a(u (k ) , v )
.
(k
)
u
otherwise
k → k + 1 and repeat 2.
Here
a(u, v ) = exp Φ(u; y ) − Φ(v ; y ) ∧ 1.
5
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
22 / 31
Numerical Results
Outline
1
Introduction
2
Forward And Inverse Problem
3
Bayesian Framework
4
MCMC Method
5
Numerical Results
6
Conclusions
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
23 / 31
Numerical Results
Two Layers Model
Truth and Prior distribution
Truth
1
6
5.5
0.9
5
0.8
7
8
9
parameter
a
b
κ+
κ−
4.5
0.7
4
y
0.6
3.5
0.5
3
4
0.4
5
6
2.5
2
0.3
1.5
0.2
true value
0.11
0.86
4
1
prior distribution
U[0, 1]
U[0, 1]
U[0.1, 6] or exp(1)
U[0.1, 6] or exp(1)
1
0.1
0
1
0
0.2
2
0.4
3
0.6
0.8
0.5
1
x
Random draws from prior
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
24 / 31
Numerical Results
Two Layers Model: Uniform Prior Permeability
6
1
a(n)
Truth
mean
standard deviation
0.6
5
4
k+(n)
a(n)
0.8
3
0.4
2
0.2
1
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
MCMC step
κ+(n)
Truth
mean
standard deviation
0.2
2
0.4
0.6
0.8
1
1.2
1.4
2
5
x 10
κ−(n)
Truth
mean
standard deviation
5
4
k−(n)
0.6
b(n)
Truth
mean
standard deviation
0.2
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
MAP
Mean
Truth
10
5
0
0
0.5
a
Mathematics Institute (Warwick)
1
1
0.2
2
0.4
0.6
15
10
MAP
Mean
Truth
5
0
0
0.5
0.8
1
1.2
1.4
1.6
MCMC step
5
x 10
+
posterior density of b
posterior density of a
15
2
1.8
MCMC step
3
1
15
10
MAP
Mean
Truth
5
0
0
b
Geometric Inverse Problems
5
κ+
posterior density of κ−
0.4
posterior density of κ
b(n)
1.8
6
1
0.8
0
1.6
MCMC step
5
x 10
1.8
2
5
x 10
15
MAP
Mean
Truth
10
5
0
0
5
κ−
Andrew Stuart
25 / 31
Numerical Results
Two Layers Model: Exponential Prior Permeability
0.8
a(n)
6
a(n)
Truth
mean
standard deviation
0.6
5
4
k+(n)
1
κ+ , κ− ∼ exp(λ), where λ = 1.
3
0.4
2
0.2
1
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
MCMC step
κ+(n)
Truth
mean
standard deviation
0.2
2
0.4
0.6
0.8
1
1.2
1.4
2
5
x 10
κ−(n)
Truth
mean
standard deviation
5
4
0.6
k−(n)
(n)
0.4
b
Truth
mean
standard deviation
0.2
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
MCMC step
1
1.8
5
0.5
a
Mathematics Institute (Warwick)
1
0.2
2
0.4
0.6
0.8
x 10
10
MAP
Mean
Truth
5
0
0
0.5
1
1
1.2
1.4
1.6
MCMC step
5
posterior density of κ+
MAP
Mean
Truth
10
0
0
2
15
posterior density of b
posterior density of a
15
3
15
10
posterior density of κ−
b(n)
1.8
6
1
0.8
0
1.6
MCMC step
5
x 10
MAP
Mean
Truth
5
0
0
2
b
Geometric Inverse Problems
4
κ+
6
1.8
2
5
x 10
15
MAP
Mean
Truth
10
5
0
0
2
4
6
κ−
Andrew Stuart
26 / 31
Numerical Results
Fault Model
Truth and Prior distribution
Truth
1
parameter
a1
a2
b1
b2
c
κ1
κ2
κ3
6
5.5
0.9
5
0.8
7
8
9
4.5
0.7
4
y
0.6
3.5
0.5
3
4
0.4
5
6
2.5
2
0.3
1.5
0.2
1
0.1
0
1
0
0.2
2
0.4
3
0.6
0.8
0.5
true value
0.39
0.35
0.18
0.6
0.15
4
1
1
prior distribution
a ∼ U(A)
b ∼ U(A)
U[−0.5, 0.5]
U[0.1, 6]
U[0.1, 6]
U[0.1, 6]
1
x
Random drawn from prior
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
27 / 31
Numerical Results
Fault Model
0.5
1
a(n)
1
Truth
mean
standard deviation
0.6
slip
a(n)
1
0.8
0
slip
Truth
mean
standard deviation
0.4
0.2
1
2
3
4
5
6
7
8
9
MCMC step
−0.5
10
b(n)
1
5
6
7
8
9
10
5
x 10
4
κ(n)
1
b(n)
1
4
5
Truth
mean
standard deviation
0.6
0.4
3
κ(n)
1
2
0.2
Truth
mean
standard deviation
1
2
3
4
5
6
7
8
posterior density of b1
MAP
Mean
Truth
6
4
2
0.5
1
a1
Mathematics Institute (Warwick)
2
3
4
MAP
Mean
Truth
4
2
0.5
1
5
6
7
8
9
MCMC step
5
6
0
0
1
10
x 10
8
1
8
9
posterior density of the slip
1
MCMC step
posterior density of a
3
6
0.8
0
0
2
MCMC step
1
0
1
5
x 10
8
6
10
5
x 10
8
posterior density of κ1
0
MAP
Mean
Truth
4
2
0
−0.5
b1
Geometric Inverse Problems
0
slip
0.5
6
MAP
Mean
Truth
4
2
0
0
2
4
6
κ1
Andrew Stuart
28 / 31
Conclusions
Outline
1
Introduction
2
Forward And Inverse Problem
3
Bayesian Framework
4
MCMC Method
5
Numerical Results
6
Conclusions
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
29 / 31
Conclusions
Conclusions
What we have shown:
Geometric permeability prior modeling
Well-definedness of posterior distribution (continuity argument)
Well-posedness of posterior distribution (not Lipschitz)
A tailored Metropolis-Hasting method
Mathematics Institute (Warwick)
Geometric Inverse Problems
Andrew Stuart
30 / 31
Conclusions
References
References
D. A. White J. N. Carter.
History matching on the Imperial College fault model using parallel tempering.
Computational Geosciences, 17(2013) 43–65.
S.L.Cotter, G.O.Roberts, A.M. Stuart, and D. White.
MCMC methods for functions: modifying old algorithms to make them faster.
Statistical Science, To Appear, arXiv:1202.0709.
M.Iglesias, K.Lin and A.M.Stuart
Well-Posed Bayesian Geometric Inverse Problems Arising In Groundwater Flow. In
preparation.
A.M. Stuart.
The Bayesian Approach to Inverse Problems.
Lecture Notes, arXiv:1302.6989
A.M. Stuart.
Inverse problems: a Bayesian perspective.
Acta Numerica, 19(2010) 451–559.
S. Vollmer.
Dimension-Independent MCMC Sampling for Elliptic Inverse Problems with Non-Gaussian
Priors
Preprint,
arXiv:1302.2213
Mathematics
Institute
(Warwick)
Geometric Inverse Problems
Andrew Stuart
31 / 31