Scaling Limits in Computational Bayesian Inversion
Claudia Schillings, Christoph Schwab
Warwick Centre for Predictive Modelling - 23. April 2015
research supported by the European Research Council under FP7 Grant AdG247277
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
1 / 18
Outline
1
Bayesian Inversion of Parametric Operator Equations
2
Sparsity of the Forward Solution
3
Sparsity of the Posterior
4
Sparse Quadrature
5
Numerical Results
6
Small Observation Noise Covariance
7
Summary
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
2 / 18
Inverse Problem
Physical Model
G(u) → δ
u parameter vector / parameter function
G the forward map modelling the physical process
δ result / observations
Forward Problem
Inverse Problem
Find the output δ for given
parameters u
Find the parameters u from
(noisy) observations δ
→ well-posed
→ ill-posed
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
3 / 18
Inverse Problem
Physical Model
G(u) → δ
u parameter vector / parameter function
G the forward map modelling the physical process
δ result / observations
G forward response operator
Forward Problem
Inverse Problem
Find the output δ for given
parameters u
Find the parameters u from
(noisy) observations δ
→ well-posed
→ ill-posed
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
3 / 18
Inverse Problem
Find the unknown data u ∈ X from noisy observations
δ = G(u) + η
Deterministic optimization problem
1
min kδ − G(u)k2 + R(u)
u 2
kδ − G(u)k potential / data misfit
R regularization term
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
3 / 18
Inverse Problem
Find the unknown data u ∈ X from noisy observations
δ = G(u) + η
Deterministic optimization problem
1
min kδ − G(u)k2 + R(u)
u 2
Large-scale, deterministic optimization problem
No quantification of the uncertainty in the unknown u
Proper choice of the regularization term R
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
3 / 18
Inverse Problem
Find the unknown data u ∈ X from noisy observations
δ = G(u) + η
Bayesian inverse problem
u, η, δ random variables / fields
Goal of computation: moments of system quantities under the posterior w.r. to
noisy data δ
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
3 / 18
Inverse Problem
Find the unknown data u ∈ X from noisy observations
δ = G(u) + η
Bayesian inverse problem
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
3 / 18
Inverse Problem
Find the unknown data u ∈ X from noisy observations
δ = G(u) + η
Bayesian inverse problem
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
3 / 18
Inverse Problem
Find the unknown data u ∈ X from noisy observations
δ = G(u) + η
Bayesian inverse problem
Quantification of uncertainty in u and system quantities
Well-posedness of the inverse problem
Incorporation of prior knowledge on the uncertain data u
Need of efficient approximations of the posterior
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
3 / 18
Inverse Problem
Find the unknown data u ∈ X from noisy observations
δ = G(u) + η
Bayesian inverse problem
MCMC methods
MAP estimators
Ad hoc algorithms (EnKF, SMC, GP,...)
Sparse deterministic approximations of posterior expectations
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
3 / 18
Bayesian Inverse Problems (Stuart 2010)
Find the unknown data u ∈ X from noisy observations
δ = G(u) + η,
Goal: Efficient estimation of system quantities from noisy observations
Infinite-dimensional parameter space
Fast convergence by exploiting sparsity
of the underlying forward problem
Suitable for application to a broad class
of forward problems
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
4 / 18
Bayesian Inverse Problems (Stuart 2010)
Find the unknown data u ∈ X from noisy observations
δ = G(u) + η,
Goal: Efficient estimation of system quantities from noisy observations
UQ in Nano Optics
Infinite-dimensional parameter space
Fast convergence by exploiting sparsity
of the underlying forward problem
Suitable for application to a broad class
of forward problems
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
4 / 18
Bayesian Inverse Problems (Stuart 2010)
Find the unknown data u ∈ X from noisy observations
δ = G(u) + η,
Goal: Efficient estimation of system quantities from noisy observations
UQ in Biochemical Networks
Infinite-dimensional parameter space
Fast convergence by exploiting sparsity
of the underlying forward problem
Suitable for application to a broad class
of forward problems
Source: Chen et al.
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
4 / 18
Bayesian Inverse Problems (Stuart 2010)
Find the unknown data u ∈ X from noisy observations
δ = G(u) + η,
Goal: Efficient estimation of system quantities from noisy observations
UQ in Groundwater Flow
Infinite-dimensional parameter space
1
0.9
Fast convergence by exploiting sparsity
of the underlying forward problem
0.8
0.7
0.6
0.5
Suitable for application to a broad class
of forward problems
0.4
0.3
0.2
y
0.1
1
C. Schillings (UoW)
Bayesian Inversion
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0x
WCPM - 23. Apr. 2015
4 / 18
Bayesian Inverse Problems (Stuart 2010)
Find the unknown data u ∈ X from noisy observations
δ = G(u) + η,
X separable Banach space
G : X 7→ X the forward map
Abstract Operator Equation
Given u ∈ X, find q ∈ X :
A(u; q) = F(u) in Y 0
with A(u; ·) ∈ L(X , Y 0 ), F : X 7→ Y 0 , X , Y reflexive Banach spaces,
a(v, w) := Y hw, AviY 0 ∀v ∈ X , w ∈ Y corresponding bilinear form
O : X 7→ RK bounded, linear observation operator
G : X 7→ RK uncertainty-to-observation map, G = O ◦ G
η ∈ RK the observational noise (η ∼ N (0, Γ))
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
4 / 18
Bayesian Inverse Problems (Stuart 2010)
Find the unknown data u ∈ X from noisy observations
δ = G(u) + η,
X separable Banach space
G : X 7→ X the forward map
Model Parametric Elliptic Problem
Given u ∈ X, find q ∈ X :
− ∇ · u∇q = f
in D,
q = 0 in ∂D
R
with weak formulation D u(x)∇q(x) · ∇w(x)dx = X hw, f iX 0 for all w ∈ X ,
X = Y = H01 (D), D ⊂ Rd bounded Lipschitz domain with Lipschitz boundary ∂D
O : X 7→ RK bounded, linear observation operator
G : X 7→ RK uncertainty-to-observation map, G = O ◦ G
η ∈ RK the observational noise (η ∼ N (0, Γ))
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
4 / 18
Bayesian Inverse Problems (Stuart 2010)
Find the unknown data u ∈ X from noisy observations
δ = G(u) + η,
X separable Banach space
G : X 7→ X the forward map
O : X 7→ RK bounded, linear observation operator
G : X 7→ RK uncertainty-to-observation map, G = O ◦ G
η ∈ RK the observational noise (η ∼ N (0, Γ))
Least squares potential Φ : X × RK → R
1
Φ(u; δ) :=
(δ − G(u))> Γ−1 (δ − G(u))
2
Reformulation of the forward problem with unknown stochastic input
data as an infinite dimensional, parametric deterministic problem
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
4 / 18
Bayesian Inverse Problems (Stuart 2010)
Parametric representation of the unknown u
X
u = u(y) := hui +
yj ψj ∈ X
j∈J
y = (yj )j∈J iid sequence of real-valued random variables yj ∼ U[−1, 1]
hui, ψj ∈ X
J finite or countably infinite index set
Prior measure on the uncertain input data
µ0 (dy) :=
O1
j∈J
2
λ1 (dyj ) .
N
1
(U, B) = [−1, 1]J ,
j∈J B [−1, 1] measurable space
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
4 / 18
Bayesian Inverse Problem
Theorem (Schwab and Stuart 2011)
Assume that G(u)
is bounded and continuous.
P
u=hui+ j∈J yj ψj
Then µδ (dy), the distribution of y ∈ U given δ, is absolutely continuous
with respect to µ0 (dy), and
dµδ
1
(y) = Θ(y)
dµ0
Z
with the parametric Bayesian posterior Θ given by
Θ(y) = exp −Φ(u; δ) ,
P
u=hui+
j∈J yj ψj
and the normalization constant
Z
Z=
Θ(y)µ0 (dy) .
U
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
5 / 18
Bayesian Inverse Problem
Expectation of a Quantity of Interest φ : X → S
Z
µδ
−1
E [φ(u)] = Z
exp −Φ(u; δ) φ(u)
P
u=hui+
U
with Z =
R
U
j∈J yj ψj
µ0 (dy) =: Z 0 /Z
exp(− 21 (δ − G(u))> Γ−1 (δ − G(u)) )µ0 (dy).
Reformulation of the forward problem with unknown stochastic input data
as an infinite dimensional, parametric deterministic problem
Parametric, deterministic representation of the derivative of the posterior
measure with respect to the prior µ0
Approximation of Z 0 and Z to compute the expectation of QoI under the
posterior given data δ
Efficient algorithm to approximate the conditional expectations given
the data with dimension-independent rates of convergence
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
5 / 18
Bayesian Inverse Problem
Expectation of a Quantity of Interest φ : X → S
Z
µδ
−1
E [φ(u)] = Z
exp −Φ(u; δ) φ(u)
P
u=hui+
U
with Z =
R
U
j∈J yj ψj
µ0 (dy) =: Z 0 /Z
exp(− 21 (δ − G(u))> Γ−1 (δ − G(u)) )µ0 (dy).
Exploiting sparsity in the parametric operator equation
Parameters belonging to a specified sparsity class
Analytic regularity of the parametric, deterministic Bayesian posterior
Parametric, deterministic Bayesian posterior belongs to the same
sparsity class
→ Sparsity of generalized pce + dimension-independent convergence
rates for Smolyak integration algorithms
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
5 / 18
Model Parametric Elliptic Problem
Z
u(x)∇q(x) · ∇w(x)dx =
X hw, f iX 0
for all w ∈ X
D
with D ⊂ Rd bounded Lipschitz domain with Lipschitz boundary ∂D, X = H01 (D), f ∈ L2 (D).
Structural Assumptions
u(x, y) = hui(x) +
X
yj ψj (x),
x∈X
j∈J
with hui ∈ L∞ (D) and (ψj )j∈J ⊂ L∞ (D), y = (y1 , y2 , . . . ) ∈ U = [−1, 1]J
Assumption 1 (UEA)
0 < umin ≤ u(x, y) ≤ umax < ∞
∀x ∈ D, y ∈ U
with 0 < umin ≤ umax < ∞.
Assumption 2
X
kψj kL∞ (D) ≤
j∈J
κ
hui
1 + κ min
with huimin = minx∈D hui(x) > 0 and κ > 0
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
6 / 18
Sparse Polynomial Chaos Approximations
Sparsity in the unknown coefficient function u,
i.e. if
P∞
j=1
kψj kpL∞ (D) < ∞ for 0 < p < 1,
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
7 / 18
Sparse Polynomial Chaos Approximations
Sparsity in the unknown coefficient function u,
P∞
j=1
kψj kpL∞ (D) < ∞ for 0 < p < 1,
(||ψ j||) j
(||ψ j||) j
(||ψ j||) j
(||ψ j||) j
1
0.9
0.8
summable, p=1
summable, p=1/2
summable, p=1/3
summable, p=1/4
0.7
0.6
||ψ j||
i.e. if
0.5
0.4
0.3
0.2
0.1
0
0
2
4
6
8
10
12
j
C. Schillings (UoW)
14
16
Bayesian Inversion
18
20
WCPM - 23. Apr. 2015
7 / 18
Sparse Polynomial Chaos Approximations
Sparsity in the unknown coefficient function u,
i.e. if
P∞
j=1
kψj kpL∞ (D) < ∞ for 0 < p < 1,
implies p-sparsity in the solution’s Taylor expansion,
∀y ∈ U :
q(y) =
P
C. Schillings (UoW)
ν∈F
τν yν ,
τν :=
1 ν
∂ q(y) |y=0
ν! y
Bayesian Inversion
,
F = {ν ∈ NN0 : |ν| < ∞}:
WCPM - 23. Apr. 2015
7 / 18
Sparse Polynomial Chaos Approximations
Sparsity in the unknown coefficient function u,
i.e. if
P∞
j=1
kψj kpL∞ (D) < ∞ for 0 < p < 1,
implies p-sparsity in the solution’s Taylor expansion,
∀y ∈ U :
q(y) =
P
ν∈F
τν yν ,
τν :=
1 ν
∂ q(y) |y=0
ν! y
,
F = {ν ∈ NN0 : |ν| < ∞}:
There exists a p-summable, monotone envelope t = {tν }ν∈F
i.e. tν := supµ≥ν kτν kX with C(p, q) := ktk`p (F ) < ∞ .
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
7 / 18
Sparse Polynomial Chaos Approximations
Sparsity in the unknown coefficient function u,
i.e. if
P∞
j=1
kψj kpL∞ (D) < ∞ for 0 < p < 1,
1
implies p-sparsity
in the solution’s Taylor expansion,
taylor coeff. (||τj||) j, 1D
∀y ∈ U :
0.9
P
q(y) =0.8 ν∈F τν yν ,
τν :=
1 ν
∂ q(y) |y=0
ν! y
monotone envelope (tj)j
F = {ν ∈ NN0 : |ν| < ∞}:
,
0.7
||τj||, t j
There exists a0.6
p-summable, monotone envelope t = {tν }ν∈F
0.5
i.e. tν := supµ≥ν 0.4
kτν kX with C(p, q) := ktk`p (F ) < ∞ .
0.3
0.2
0.1
0
0
2
4
6
8
10
12
14
j
C. Schillings (UoW)
Bayesian Inversion
16
18
20
WCPM - 23. Apr. 2015
7 / 18
Sparse Polynomial Chaos Approximations
Sparsity in the unknown coefficient function u,
i.e. if
P∞
j=1
kψj kpL∞ (D) < ∞ for 0 < p < 1,
implies p-sparsity in the solution’s Taylor expansion,
∀y ∈ U :
q(y) =
P
ν∈F
τν yν ,
τν :=
1 ν
∂ q(y) |y=0
ν! y
,
F = {ν ∈ NN0 : |ν| < ∞}:
There exists a p-summable, monotone envelope t = {tν }ν∈F
i.e. tν := supµ≥ν kτν kX with C(p, q) := ktk`p (F ) < ∞ .
and monotone ΛTN ⊂ F corresponding to the N largest terms of t with
X
sup q(y) −
τν yν ≤ C(p, t)N −(1/p−1) .
y∈U
ν∈ΛTN
X
A. Cohen, R. DeVore and Ch. Schwab. Analytic regularity and polynomial approximation of
parametric and stochastic elliptic PDEs, Analysis and Applications, 2010.
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
7 / 18
Sparse Polynomial Chaos Approximations
Idea of Proof
Holomorphic Extension
z 7→ q(z) ,
z ∈ Uρ := ⊗j≥1 {zj ≤ ρj }
holomorph with bound kq(z)kX ≤ Cδ for any positive sequence ρ = {ρj }j≥1 such that
P
j≥1 ρj |ψj (x)| ≤ hui(x) − δ, x ∈ D for some δ > 0.
Estimate on the Taylor Coefficients
kτν kX ≤ Cδ inf{ρ−ν :
X
ρj |ψj (x)| ≤ hui(x) − δ, x ∈ D}
j≥1
by recursive application of Cauchy’s formula
Construction of a particular ρ
{kψj kL∞ }j≥1 ∈ `p (N) ⇒ {tν }ν∈F ∈ `p (F )
Stechkin
X
sup q(y) −
τν yν y∈U
C. Schillings (UoW)
ν∈ΛTN
X
Bayesian Inversion
≤
X
ktν kX ≤ CN
− p1 +1
ν ∈Λ
/ TN
WCPM - 23. Apr. 2015
7 / 18
Sparsity of the Posterior
Theorem (ClS and Schwab 2013)
Assume that the unknown coefficient function u is p-sparse for
0 < p < 1. Then, the posterior density’s Taylor expansion is p-sparse
with the same p.
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
8 / 18
Sparsity of the Posterior
Theorem (ClS and Schwab 2013)
Assume that the unknown coefficient function u is p-sparse for
0 < p < 1. Then, the posterior density’s Taylor expansion is p-sparse
with the same p.
N-term Approximation Results
X
1
P
sup Θ(y) −
Θν Pν (y) ≤ N −s kθ P k`pm (F ) , s := − 1 .
p
X
y∈U
P
ν∈ΛN
Adaptive Smolyak quadrature algorithm with convergence rates
depending only on the summability of the parametric operator
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
8 / 18
Sparsity of the Posterior
Theorem (ClS and Schwab 2013)
Assume that the unknown coefficient function u is p-sparse for
0 < p < 1. Then, the posterior density’s Taylor expansion is p-sparse
with the same p.
Examples
Parametric initial value ODEs (Hansen & Schwab; Vietnam J. Math. 2013)
Affine-parametric, linear operator equations (ClS & Schwab; 2013)
Semilinear elliptic PDEs (Hansen & Schwab; Math. Nachr. 2013)
Elliptic multiscale problems (Hoang & Schwab; Analysis and Applications 2012)
Nonaffine holomorphic-parametric, nonlinear problems (Cohen, Chkifa &
Schwab; 2013)
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
8 / 18
Sparsity of the Posterior
Theorem (ClS and Schwab 2013)
Assume that the unknown coefficient function u is p-sparse for
0 < p < 1. Then, the posterior density’s Taylor expansion is p-sparse
with the same p.
Goal of computation
Expectation of a Quantity of Interest φ : X → S
Z
δ
Eµ [φ(u)] = Z −1
exp −Φ(u; δ) φ(u)
P
u=hui+
U
with Z =
R
U
j∈J yj ψj
µ0 (dy) =: Z 0 /Z
exp(− 21 (δ − G(u))> Γ−1 (δ − G(u)) )µ0 (dy).
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
8 / 18
Sparse Quadrature Operator
For any finite monotone set Λ ⊂ F, the quadrature operator is defined
by
X
XO
QΛ =
∆ν =
∆νj
ν∈Λ
ν∈Λ j≥1
with associated collocation grid
ZΛ = ∪ν∈Λ Z ν .
(Qk )k≥0 sequence of univariate quadrature formulas
∆j = Qj − Qj−1 , j ≥ 0 univariate quadrature difference operator
N
N
Qν = j≥1 Qνj , ∆ν = j≥1 ∆νj tensorized multivariate operators
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
9 / 18
Sparse Quadrature Operator
For any finite monotone set Λ ⊂ F, the quadrature operator is defined
by
X
XO
QΛ =
∆ν =
∆νj
ν∈Λ
ν∈Λ j≥1
with associated collocation grid
ZΛ = ∪ν∈Λ Z ν .
1
1
0
0
−1
−1
C. Schillings (UoW)
0
1
Bayesian Inversion
−1
−1
0
1
WCPM - 23. Apr. 2015
9 / 18
Convergence Rates for Adaptive Smolyak Integration
Theorem
Assume that the unknown coefficient function u is p-sparse for 0 < p < 1.
Then there exists a sequence (ΛN )N≥1 of monotone index sets ΛN ⊂ F
such that #ΛN ≤ N and
|Z − QΛN [Θ]| ≤ C1 N
R
with Z = U Θ(y)µ0 (dy), Θ(y) = exp −Φ(u; δ) − 1p +1
and,
P
u=hui+
kZ 0 − QΛN [Ψ]kX ≤ C2 N
R
with Z 0 = U Ψ(y)µ0 (dy), Ψ(y) = Θ(y)φ(u)
,
P
u=hui+
yψ
j∈J j j
− p1 +1
,
C1 , C2 > 0 independent of N.
yψ
j∈J j j
Remark: SAME index sets ΛN for BOTH, Z 0 and Z.
ClS and Schwab Sparsity in Bayesian Inversion of Parametric Operator Equations, 2013.
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
10 / 18
Uncertainty Quantification in Nano Optics
Goal: Quantification of the influence of defects in fabrication process
on the optical response of nano structures
Propagation of plane wave and its interaction with
scatterer described by Helmholtz equation (2D).
Stochastic shape of the scatterer
0 < ρmin ≤ ρ(ω, φ) ≤ ρmax ,
ω ∈ Ω, φ ∈ [0, 2π)
Collaboration with Ralf Hiptmair, Laura Scarabosio
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
11 / 18
Uncertainty Quantification in Nano Optics
Forward Problem
−∆u − k12 u = 0
−∆u −
k22 u
=0
in D1 (ω)
in D2 (ω)
u|+ − u|− = 0 on Γ(ω)
∂u ∂u −µ
= 0 on Γ(ω)
∂n +
∂n −
p
∂
lim
|x|
− ik1 (u(ω) − ui )(x) = 0,
|x|→∞
∂ |x|
ui (x) = eik1 ·x
Parametrization of the shape
r(ω, ϕ) = r0 (ϕ) +
C. Schillings (UoW)
64
X
1
1
y (ω) cos(kϕ) + ζ y2k+1 (ω) sin(kϕ),
ζ 2k
k
k
k=1
Bayesian Inversion
ϕ ∈ [0, 2π)
WCPM - 23. Apr. 2015
11 / 18
Numerical Results of Sparse Interpolation
Interpolation, dimension 64
1
10
Estimated error CC, nonadaptive strategy, zeta=2
Estimated error CC, nonadaptive strategy, zeta=3
Estimated error CC, nonadaptive strategy, zeta=4
0
10
−1
10
−2
10
−3
10
−4
10
−5
10
−6
10
0
10
1
2
10
10
3
10
#Λ
Figure: Estimated error curves of the interpolation error w.r. to the cardinality of the
index set ΛN based on the sequences CC with nonadaptive refinement, uniform
distribution of the parameters, dimension of the parameter space 64.
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
12 / 18
Concentration Effect of the Posterior for 0 < Γ 1
Model parametric elliptic problem
−div(u∇q) = f
in D := [0, 1] , q = 0
in ∂D ,
with f (x) = 100 · x and
u(x, y) = 0.15 + y1 ψ1 + y2 ψ2 ,
with J = 2, J = {1, 2}, ψ1 (x) = 0.1 sin(πx), ψ2 (x) = 0.025 cos(2πx) and with
yj ∼ U [−1, 1], j ∈ J .
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
13 / 18
Concentration Effect of the Posterior for 0 < Γ 1
For given (noisy) data δ,
δ = G(u) + η ,
we are interested in the behavior of the posterior
Θ(y) = exp −ΦΓobs (u; δ) P2
u=
with
ΦΓobs (u; δ) =
j=1 yj ψj
,
1
(δ − G(u))> Γ−1 (δ − G(u)) .
2
and variance of the noise Γ = λ · id
QoI φ is the solution of the forward problem at x = 0.5.
Observation operator O consists of 2 system responses at x = 0.25 and x = 0.75.
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
13 / 18
Concentration Effect of the Posterior for 0 < Γ 1
Contour plot of the posterior, 2 meas. at 0.25 and 0.75, 6=0.25
1
2
reference parameter
posterior
0.8
0.6
0.4
y2
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
-1
-0.5
0
0.5
1
y1
Figure: Contour plot of the posterior with observational noise λ = 0.252 .
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
13 / 18
Concentration Effect of the Posterior for 0 < Γ 1
Contour plot of the posterior, 2 meas. at 0.25 and 0.75, 6=0.5
1
2
reference parameter
posterior
0.8
0.6
0.4
y2
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
-1
-0.5
0
0.5
1
y1
Figure: Contour plot of the posterior with observational noise λ = 0.52 .
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
13 / 18
Concentration Effect of the Posterior for 0 < Γ 1
Contour plot of the posterior, 2 meas. at 0.25 and 0.75, 6=0.05
1
2
reference parameter
posterior
0.8
0.6
0.4
y2
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
-1
-0.5
0
0.5
1
y1
Figure: Contour plot of the posterior with observational noise λ = 0.052 .
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
13 / 18
Asymptotic Analysis
Theorem (ClS and Schwab 2014)
Assume that the parameter space is finite (possibly after dimension truncation) and G(·), δ are such that the assumptions of Laplace’s
method hold; in particular, the minimum y0 of
S(y) =
1
(δ − G(u))> (δ − G(u))
2
is nondegenerate.
Then, as Γ ↓ 0, the Bayesian estimate admits an asymptotic expansion
δ
Eµ [φ] =
ZΓ0
∼ a0 + a1 λ + a2 λ2 + ....
ZΓ
where a0 = φ(y0 ).
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
14 / 18
Curvature Rescaling Regularization
Removing the degeneracy in the integrand function
The maximizer y0 of the posterior measure Θ(y) is computed by
minimizing the potential
1
(δ − G(u))> (δ − G(u)) P2
.
2
u= j=1 yj ψj
using a trust-region Quasi-Newton approach with SR1 updates.
Diagonalize the approximated Hessian HSR1 = QMQ> and regularize the
integrand by the curvature rescaling transformation
y0 + Γ1/2 QM −1/2 z ,
C. Schillings (UoW)
Bayesian Inversion
z ∈ RJ .
WCPM - 23. Apr. 2015
15 / 18
Numerical Experiments
Curvature rescaling
Contour plot of the posterior, 2 meas. at 0.25 and 0.75, 6=0.25 2
1
0.8
transformed posterior
0.8
0.6
0.6
0.4
0.4
0.2
0.2
y2
y2
Contour plot of the transf. posterior, 2 meas. at 0.25 and 0.75, 6=0.25 2
1
reference parameter
posterior
optimal point
0
0
-0.2
-0.2
-0.4
-0.4
-0.6
-0.6
-0.8
-1
-1
-0.8
-0.5
0
y1
0.5
1
-1
-1
-0.5
0
0.5
1
y1
Figure: Contour plot of the posterior with observational noise Γobs = 0.252 · Id (left)
and contour plot of the transformed posterior (right).
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
16 / 18
Numerical Experiments
Curvature rescaling
Truncated domain, 2 meas. at 0.25 and 0.75, 6=0.25 2
1
Contour plot of the posterior, 2 meas. at 0.25 and 0.75, 6=0.25 2
1
reference parameter
posterior
optimal point
0.6
0.6
0.4
0.4
0.2
0.2
0
0
-0.2
-0.2
-0.4
-0.4
-0.6
-0.6
-0.8
-1
-1
transformed posterior
truncated domain
0.8
y2
y2
0.8
-0.8
-0.5
0
y1
0.5
1
-1
-1
-0.5
0
0.5
1
y1
Figure: Contour plot of the posterior with observational noise Γobs = 0.252 · Id (left)
and and truncated domain of integration of the rescaled Smolyak approach in the
original coordinate system (right).
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
16 / 18
Numerical Experiments
Curvature rescaling
2
2
10
orig. integrand
trans. integrand
0
10
−2
−2
10
10
−4
10
−6
10
−8
10
−10
10
−4
10
−6
10
−8
10
−10
10
−12
−12
10
10
−14
10
orig. integrand
trans. integrand
0
10
Estimated (absolute) error (Z’)
Estimated (absolute) error (normalization constant Z)
10
−14
0
10
1
2
10
10
3
10
10
0
10
#Λ
1
2
10
10
3
10
#Λ
Figure: Comparison of the estimated (absolute) error curves using the Smolyak
approach for the original integrand (gray) and the transformed integrand (black) for the
computation of ZΓobs (left) and ZΓ0 obs (right) with observational noise Γobs = 0.252 · Id.
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
16 / 18
Numerical Experiments
Curvature rescaling
Extension to lognormal case
ln(u(x, y)) = 0.15 + y1 ψ1 + y2 ψ2 ,
with J = 2, J = {1, 2}, ψ1 (x) = 0.1 sin(πx), ψ2 (x) = 0.025 cos(2πx) and with
yj ∼ N (0, 1), j ∈ J.
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
16 / 18
Numerical Experiments
Curvature rescaling
Extension to lognormal case
Contour plot of the posterior, 2 meas. at 0.25 and 0.75, , 6=0.01 2
3
Contour plot of the posterior, 2 meas. at 0.25 and 0.75, 6=1.0001
3
reference parameter
posterior
optimal point
2
1
y2
y2
1
0
0
-1
-1
-2
-2
-3
-3
reference parameter
posterior
optimal point
2
-2
-1
0
1
2
3
-3
-3
-2
y1
-1
0
1
2
3
y1
Figure: Contour plot of the posterior density with observational noise Γobs = 0.012 · Id
(left), and contour plot of the transformed posterior (right).
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
16 / 18
Numerical Experiments
Curvature rescaling
Extension to lognormal case
Contour plot of the posterior, 2 meas. at 0.25 and 0.75, , 6=0.01 2
3
Contour plot of the transformed posterior, 2 meas. at 0.25 and 0.75, 6=0.0001
6
reference parameter
posterior
optimal point
2
transformed posterior
quadrature points
4
2
y2
y2
1
0
0
-1
-2
-2
-4
-3
-3
-2
-1
0
1
2
3
-6
-6
-4
y1
-2
0
2
4
6
y1
Figure: Contour plot of the posterior density with observational noise Γobs = 0.012 · Id
(left), and contour plot of the transformed posterior (right).
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
16 / 18
Conclusions and Outlook
New class of sparse, adaptive quadrature methods for Bayesian
inverse problems for a broad class of operator equations
Dimension-independent convergence rates depending only on the
summability of the parametric operator
Numerical confirmation of the predicted convergence rates
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
17 / 18
Conclusions and Outlook
New class of sparse, adaptive quadrature methods for Bayesian
inverse problems for a broad class of operator equations
Dimension-independent convergence rates depending only on the
summability of the parametric operator
Numerical confirmation of the predicted convergence rates
Efficient treatment of small observation noise variance Γ
Development of preconditioning techniques to overcome the
convergence problems in the case Γ ↓ 0
Combination of optimization and sampling techniques
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
17 / 18
References
Chkifa A 2013 On the Lebesgue constant of Leja sequences for the unit disk Journal of Approximation Theory 166
176-200
Chkifa A, Cohen A and Schwab C 2013 High-dimensional adaptive sparse polynomial interpolation and applications to
parametric PDEs Foundations of Computational Mathematics 1-33
Cohen A, DeVore R and Schwab C 2011 Analytic regularity and polynomial approximation of parametric and stochastic
elliptic PDEs Analysis and Applications 9 1-37
Gerstner T and Griebel M 2003 Dimension-adaptive tensor-product quadrature Computing 71 65-87
Hansen M and Schwab C 2013 Sparse adaptive approximation of high dimensional parametric initial value problems
Vietnam J. Math. 1 1-35
Hansen M, Schillings C and Schwab C 2012 Sparse approximation algorithms for high dimensional parametric initial value
problems Proceedings 5th Conf. High Performance Computing, Hanoi
Schillings C and Schwab C 2013 Sparse, adaptive Smolyak algorithms for Bayesian inverse problems Inverse Problems
29 065011
Schillings C and Schwab C 2014 Sparsity in Bayesian inversion of parametric operator equations Inverse Problems 30
065007
Schillings C 2013 A Note on Sparse, Adaptive Smolyak Quadratures for Bayesian Inverse Problems Oberwolfach Report
10/1 239-293
Schillings C and Schwab C 2014 Scaling Limits in Computational Bayesian Inversion SAM Report 2014-26
Schwab C and Stuart A M 2012 Sparse deterministic approximation of Bayesian inverse problems Inverse Problems 28
045003
C. Schillings (UoW)
Bayesian Inversion
WCPM - 23. Apr. 2015
18 / 18