Predictive Coarse-Graining
M. Schöberl1 , N. Zabaras2,3 , P.S. Koutsourelakis1
1
Continuum Mechanics Group
Technische Universität München
3
2
Institute for Advanced Study
Technische Universität München
Warwick Centre of Predictive Modeling
University of Warwick
November 27, 2015
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
1 / 56
Motivation Coarse-Graining
Atomistic simulation for obtaining insights of
chemical and physical process of complex systems.
Difficulty
I
Complex interactions
I
Long-range interactions
I
Small time- and length-scales
→ Exceeding computational tractability
A coarse description allows us
I
to evaluate larger systems during larger time
intervals
I
to gain understanding of physics of the system.
WCPM Seminar
Predictive Coarse-Graining
Figure :
Coarse-graining water
November 27, 2015
2 / 56
Motivation
Approach
I
Describing system with less degrees of freedom
I
Determining optimal parameter set for given parametrization of a
coarse-grained potential
I
Leading to point estimates of its parametrization and thus also in predictions
I
Predictions performed on coarse scale
How can we quantify the uncertainty induced by a coarse
description and the loss of information?
How can we reconstruct fine configurations given a coarse
configuration?
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
3 / 56
General coarse-graining problem
Fine-scale degrees of freedom x ∈ M with M ⊂ Rn , n 1 in equilibrium
described by a Boltzmann-type PDF:
Fine-scale description
pf (x|β) =
I
Potential U(x)
I
Inverse Temperature β =
1
kb T
Partition function Z (β) =
R
I
WCPM Seminar
exp {−βU(x)}
Z (β)
, with temperature T
M
exp {−βU(x)} dx
Predictive Coarse-Graining
November 27, 2015
4 / 56
General coarse-graining problem
Coarse-scale description
pc (X|β) =
exp {−βUc (X, θ c )}
Zc (θ c , β)
I
Coarse variables X ∈ Mc , Mc ⊂ Rnc , nc n
I
Coarse-grained potential selected Ûc selected out of candidate potentials
Ûc ∈ Uc
I
Parametrization and shape selected by assuming Uc (X, θ c ) but any potential
possible
How are fine and coarse configurations connected?
Connecting fine variables x with coarse variables X with coarse-graining map
ξ : M −→ Mc
X = ξ(x)
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
5 / 56
How to determine the effective coarse potential Uc ?
Analytical solution for the effective coarse-grained potential:
Uc (X) = −β −1 ln
Z
exp {−βU(x)} δ(ξ(x) − X)dx
M
Potential of mean force with respect to coarse-grained coordinates ξ(x). →
intractable!
Numerical coarse-graining strategies:
I
iterative Boltzmann inversion [5]
I
inverse Monte Carlo [8]
I
force matching [6]
I
variational mean-field theory [9]
I
relative entropy [10]
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
6 / 56
The relative entropy method[10]
→ minimize a distance metric between pf (X|β) and pc (X|β) with the PDF
pf (X|β) sampling configuration x that maps to configuration X
Z
pf (X|β) ∝
pf (x|β)δ(ξ(x) − X)dx
M
Relative entropy method[10, 4, 2]
(or KL-divergence)
Z
KL(pf ||pc ) =
pf (X|β) ln
Mc
pf (X|β)
pc (X|β)
dX
Formulation in M:
Z
KL(pf ||pc ) =
pf (x|β) ln
M
I
pf (x|β)
pc (ξ(x)|β)
dx + Smap (ξ(x))
R
with Smap = M δ(ξ(x) − X)dx p (x|β) measures the information loss
f
induced by mapping ξ(x).
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
7 / 56
Relative entropy method
Smap (ξ(x)) is independent of Ûc , the minimization of the KL-divergence is
equivalent to minimization of
pf (x|β)
F(Ûc ) = ln
pc (ξ(x)|β) pf (x|β)
For any given m-dimensional parametrization θ c ∈ Θc ⊂ Rm of the potential, the
optimization problem follows:
θ ∗c = arg min F(Uc (X, θ c ))
θ c ∈Θc
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
8 / 56
Relative entropy method
Model Restrictions
I
Relative entropy method has no capability to predict fine-scale configurations
(no mapping coarse to fine)
I
I
Eg. for predicting correlations within fine configurations
Predicting quantities of interest (QoI) by evaluating on fine-scale
Z
hf (x)ipf (x|β) =
f (x)pf (x|β)dx
M
I
With coarse-grained description pc (X|β) predictions in Mc but not in M
Difficulty:
I
I
I
How to define observable g (X) for desired QoI in Mc ? R
Are you able to predict same QoI with hg (X)ipc (X|β) = Mc g (X)pc (X|β)dX
No quantification of epistemic uncertainty: determine Smap (ξ(x))
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
9 / 56
Proposed Model
Framework following generative probabilistic models and builds upon:
1
Coarse-scale PDF pc (X|θ c , β) ∝ exp {−βUc (X, θ c )}
I
I
2
Probabilistic mapping from coarse to fine pcf (x|X, θ cf )
I
Conditional PDF of x given the coarse variables X
Parametrization of the probabilistic mapping by θ cf ∈ Θcf ⊂ Rmcf
Deterministic mapping ξ(x) is not invertible (many to one map)
I
→ Probabilistic relation necessary
Smap not fixed for per definition. → Optimization with respect to θ cf possible.
I
I
3
Describing statistics of coarse variables X
Parametrized by θ c , no restrictions in shape of pc
Prior PDF for model parametrization
I
I
p(θ c )
p(θ cf )
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
10 / 56
Proposed Model
I
Proposed model describes a Bayesian perspective of coarse-graining
equilibrium atomistic ensembles
I
Model is data driven by N data points x(i) in the fine-scale, denoted as
x(1:N) = {x(1) , · · · , x(N) }
Given the data x(1:N) a posterior distribution on the model parameters θ cf , θ c and
the latent variables X(1:N) can be defined.
I
Coarse variables seen as latent
I
To each observable x(i) one latent Variable X(i) is assigned
I
X(i) represents pre-image of the fine-scale observation x(i)
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
11 / 56
Bayesian coarse-graining - posterior
Posterior distribution
p(θ cf , θ cf , X(1:N) |x(1:N) ) ∝ p(x(1:N) |θ cf , θ c , X(1:N) )p(θ cf , θ c , X(1:N) )
!
N
Y
(i)
(i)
(i)
pcf (x |θ cf , X )pc (X |θ c ) p(θ cf )p(θ c )
=
i=1
Use given data x(1:N) for inferring the the posterior
distribution or for determining MAP estimates of the
model parameters θ ∗cf , θ ∗c .
I
θc
θ cf
X(n)
x(n)
Bayesian formulation answers the questions of
I
I
model validation and
prediction
N
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
12 / 56
Bayesian coarse-graining
The posterior distribution leads to the predictive distribution for fine-scale
configurations x:
Predictive distribution
p(x|x(1:N) ) =
Z
pcf (x|X, θ cf )pc (X|θ c )p(θ c , θ cf |x(1:N) )dXdθ cf dθ cf
Simulate fine-scale system
2
Draw sample θ ∗c , θ ∗cf ∼ p(θ c , θ cf |x(1:N) )
Draw sample of coarse-scale description X∗ ∼ pc (X|θ ∗c )
3
Draw sample of fine-scale description x∗ ∼ pcf (x∗ |X∗ , θ ∗cf )
1
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
13 / 56
Bayesian coarse-graining: ensemble averages
Approximating ensemble averages by
Z
hf (x)ipf (x) =
f (x)pf (x)dx
M
= E [f (x)]≈E [f (x)|x(1:N) ]
Z
=
f (x)p(x|x(1:N) )dx
M
Z
= f (x)pcf (x|X, θ cf )pc (X|θ c )p(θ c , θ cf |x(1:N) )dxdXdθ c dθ cf
I
Error arising from discrepancy between E [f (x)] and E [f (x)|x(1:N) ]
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
14 / 56
Bayesian coarse-graining: coarse-scale model pc
Coarse-scale model
pc (X|θ cf ) =
exp {−βUc (X, θ c )}
Zc (θ c )
Requirements
I
Allow sufficient flexibility of pc (X)
I
Uc built from high-order interactions affording flexibility in pc (X)
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
15 / 56
Bayesian coarse-graining: reconstruction map pcf
Map from coarse to fine: pcf (x|X, θ cf )
I
Several fine-configurations map to same coarse-configuration
I
I
Probabilistic relation between a given coarse-configuration and a
fine-configuration takes account of it. [7]
Fast reconstruction ([7, 1]) of fine-scale configurations x given a
coarse-configuration X desired.
coarse coordinate X
Influence of pcf
I
What is expected from the coarse variables X in
terms of predicting the given data x(1:N) ?
I
Adjusting pc (X|θ c ) so that X agrees best with data
x(1:N) connected with probabilistic mapping pcf .
WCPM Seminar
Predictive Coarse-Graining
pcf (xi|X)
(...)
possible set of fine coordinates xi for given coarse coordinate X
Figure : Configurations x
given X
November 27, 2015
16 / 56
Optimization
For learning the model parameters θ c and θ cf the expectation maximization
algorithm is applied.
I
Point estimates for maximizing the likelihood or posterior distribution are
obtained.
I
Approximation of posterior on θ cf by Laplace (validated by full posterior
sampling)
MAP estimate
h
i
θ MAP = arg max log p(θ c , θ cf , x(1:N) )
θ
h
i
= arg max log p(x(1:N) |θ cf , θ cf ) + log p(θ c ) + log p(θ cf )
θ
I
In the following the ML-estimate is shown and will be extended by prior
distributions p(θ) in a next step by adding log-priors log p(θ c ) + log p(θ cf ).
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
17 / 56
Optimization
I
I
To obtain point estimates with MLE for the model parameters θc and θcf we
maximize the log-likelihood by using the EM algorithm.
The likelihood is defined as
p(x
(1:N)
Z
|θ c , θ cf ) =
p(x
(1:N)
Z
=
pcf (x
Z Y
N
=
,X
(1:N)
(1:N)
|X
|θ c , θ cf )dX
(1:N)
(i)
, θ cf )pc (X
(i)
(1:N)
(1:N)
|θ c )dX
(i)
pcf (x |X , θ cf )pc (X |θ c )dX
(1:N)
(1:N)
i=1
I
augmenting the log-likelihood with an arbitrary density q(X(1:N) )
log p(x
(1:N)
Z
|θ c , θ cf ) = log
q(X
Z
≥
q(X
= L(q(X
WCPM Seminar
(1:N)
(1:N)
(1:N)
)
) log
pcf (x(1:N) |X(1:N) , θ cf )pc (X(1:N) |θ c )
q(X(1:N) )
dX
pcf (x(1:N) |X(1:N) , θ cf )pc (X(1:N) |θ c )
q(X(1:N) )
(1:N)
dX
(1:N)
), θ c , θ cf )
Predictive Coarse-Graining
November 27, 2015
18 / 56
Expectation maximization
It holds the following decomposition for the lower bound:
L(q(X
(1:N)
Z
), θ c , θ cf ) =
q(X
(1:N)
= − KL(q(X
) log
(1:N)
p(x(1:N) |X(1:N) , θ cf )p(X(1:N) |θ c )
)||p(X
q(X(1:N) )
(1:N)
(1:N)
|x
dX
, θ cf , θ cf )) + log p(x
(1:N)
)
Decomposition of the log likelihood
log p(x
(1:N)
|θ c , θ cf ) = L(q(X
(1:N)
), θ c , θ cf ) + KL(q(X
(1:N)
)||p(X
(1:N)
|x
(1:N)
, θ c , θ cf )).
Maximizing L(q(X(1:N) ), θ c , θ cf ) is equal to minimizing
KL(q(X(1:N) )||p(X(1:N) |x(1:N) , θ cf , θ cf ))
p(X(1:N) |x(1:N) , θ cf , θ cf ) PDF over pre-images for X for the given data x(1:N) and
parameters θ.
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
19 / 56
Expectation maximization
For given parameters θ cf , θ c the lower bound is maximized if we choose
q(X(1:N) ) ∝ pcf (x(1:N) |X(1:N) , θ cf )pc (X(1:N) |θ c ).
Therefore,
⇒ KL(q(X(1:N) )||p(X(1:N) |x(1:N) , θ c , θ cf )) = 0.
Expectation maximization
1
2
Initial parameter setting for iteration t = 0: θ 0cf and θ 0c
E-step: q (t+1) = arg maxq L(q, θ t )
Involves MCMC to obtain samples X ∼ q(X(1:N) |x(1:N) , θ tcf , θ tcf )
3
M-step: θ t+1 = arg maxθ L(q t+1 , θ t )
4
t →t +1
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
20 / 56
Derivatives
First order derivatives


+
*
N
X
∂L
∂Uc (X, θ c )
∂Uc (X(i) , θ c )
= β N
−
∂θ c
∂θ c
∂θ c
pc (X(i) |θ t )
i=1
∂L
=
∂θ cf
*
1
pcf
(x(1:N) |X, θ
∂pcf (x(1:N) |X, θ cf )
∂θ cf
cf )

q(X(i) |x(i) ,θ t ,θ tc )
cf
c
+
q(X(1:N) |x(1:N) ,θ t ,θ tc )
cf
Second order derivatives
N
X
∂2L
=−β
∂θk ∂θl
i=1
*
∂ 2 Uc (X(i) , θ c )
∂θk ∂θl
+
*
+ βN
q(X(i) |x(i) ,θ 0 ,θ 0c )
cf
∂ 2 Uc (X, θ c )
∂θk ∂θl
+
pc (X|θ c )
∂Uc (X, θ c ) ∂Uc (X, θ c )
2
−β N
∂θk
∂θl
pc (X|θ c )
∂Uc (X, θ c )
∂Uc (X, θ c )
2
+β N
∂θk
∂θ
l
pc (X|θ c )
pc (X|θ c )
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
21 / 56
Derivatives
Second order derivatives
N
X
∂ 2 L(q(X(1:N) ), θ c , θ cf )
=
∂θ 2cf
i=1
WCPM Seminar
*
∂ 2 pcf (x(i) |X(i) , θ cf )
1
−
∂θ 2cf
pcf (x(i) |X(i) , θ cf )
pcf (x(i) |X(i) , θ cf )2
1
Predictive Coarse-Graining
∂pcf (x(i) |X(i) , θ cf )
∂θ cf
November 27, 2015
!
22 / 56
Robbins-Monro Optimization
Gradients are sample averages → noise afflicted.
Robbins-Monro Algorithm
θ t+1 = θ t + αt ∇θ L(θ t )
with
αt =
α
1
, with < ρ ≤ 1.
ρ
(t + A)
2
I
Convergence is guaranteed to the true optimum θ ∗ if infinite steps performed.
I
αt is a sequence of real numbers and has to fulfill:
∞
X
t=1
WCPM Seminar
αt = ∞ and
∞
X
αt2 < ∞
t=1
Predictive Coarse-Graining
November 27, 2015
23 / 56
Laplace Approximation
Approximating a density function p(z) =
1
Zf
(z) by
q(z) = N (z|z0 , Σ)
−1
with Σ = (−∇∇f (z)|z=0 )
and ∇f (z)|z=z0 = 0.
Figure : Laplace approximation[3]
For the given problem using a noniformative prior it follows
Laplace approximation
N
X
∂ 2 log p(θ cf |x(1:N) )
=
2
∂θ cf
i
WCPM Seminar
*
∂ 2 log pcf (x(i) |X(i) , θ cf )
∂θ 2cf
Predictive Coarse-Graining
+
q(X(i) )
November 27, 2015
24 / 56
Predictive uncertainty
Propagate uncertainty induced by mapping
1
−1
2
∂ log p(θ cf |x(1:N) )
θ icf ∼ N (θ MAP
,
Σ)
with
Σ
=
−
2
cf
∂θ
2
Xij ∼ pc (X|θ MAP
)
c
3
xijk ∼ pcf (x|Xij , θ icf )
cf
Predictice uncertainty of properties:
D
WCPM Seminar
E
f |θc ,θicf = =
Z
f (x)pcf (x|X, θ icf )pc (X|θ MAP
)dXdx
c
Predictive Coarse-Graining
November 27, 2015
25 / 56
Example Problem: Coarse-graining 1D Ising Model
I
Applying proposed method (’predCg’) for coarse-graining a one dimensional
Ising Model
I
Comparison between ’predCg’ and the deterministic formulation of
minimizing the KL-divergence (’relEntr’) by Shell (2008)
I
Assessed by predictive capabilities for magnetization and correlation
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
26 / 56
Ising Model - System Setting
The Ising-model regarded in the following discussion is one dimensional. It has the
properties:
I
System size nf in fine- and nc in coarse-scale
I
Level of coarse-graining lc =
I
Inverse temperature β
I
I
External field µ
Regarded interaction length Lf in fine- and Lc in coarse scale, including all
interactions k ≤ L
I
J0 = 1 overall interaction strength
WCPM Seminar
nf
nc
Predictive Coarse-Graining
November 27, 2015
27 / 56
Ising Model - Potential in Fine-Scale
Fine variables take the values xi ∈ {−1, 1} following
pf (x|β) ∝ exp(−βU(x, Jk , µ)):
Fine-scale potential
U(x, Jk , µ) = −
Lf
nf
X
X
1X
Jk
xi xj − µ
xi
2
k=1
|i−j|=k
i=1
with i, j ∈ {1, . . . , nf } having nf lattice sites.
I Maximal interactions of Lf sites apart are regarded in the potential.
I |i − j| = k interpreted as summation over neighbors k over all sites i sites
apart
I Jk , strength of the k-th interaction.
with Jk following a power law for a given overall strength J0 and exponent a,
Jk = LkK a with,
L
X
K = J0 L1−a
k −a
k=1
in order to normalize the interaction strength [1].
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
28 / 56
Ising Model - Potential in Coarse-Scale
Coarse variables take the values Xi ∈ {−1, 1}, pc ∝ exp{−βUc (X, θ c , µ)}
Coarse-scale potential
n
Uc (X, θ c , µ) = −
+
c
1 lin X
(θ
Xi
2
i=1
Lc
X
twop
θk
nc
X
trip
θmn
nc
X
XX
xi xi±m xi±m±n )
m=1 n=1
i=1
−µ
Xi Xj
|i−j|=k
k=1
+
X
Xi
i=1
with i, j ∈ {1, . . . , nc } and nc nf lattice sites.
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
29 / 56
Ising Model - Mapping Fine to Coarse (for relative entropy)
Each coarse variable Xr , r = {1, . . . , R} describes S fine-scale variables xr ,s with
s = {1, . . . , S}. The mapping from fine-scale variables xr ,s to coarse variable Xr is
defined as,
Fine-to-coarse mapping

PS
1

+1,
S Ps xr ,s > 0
S
1
Xr = −1,
s xr ,s < 0 .
S


U(−1, +1), otherwise
Figure : Mapping from fine-scale x to coarse-scale X
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
30 / 56
Ising Model - Mapping Coarse to Fine
Coarse-to-fine mapping pcf (x|X, θcf )
1+xr ,s Xr
2
pcf (xr ,s |Xr ) = θcf
(1 − θcf )
1+xr ,s Xr
2
Assumption: xi conditionally
independent for given X:
p(x
(1:N)
|X
(1:N)
, θ cf ) =
N
Y
(t)
(t)
pcf (x |X , θ cf )
i=1
=
N Y
R Y
S
Y

1+xri ,s Xri
2
cf
θ
(1 − θcf )
1−xri ,s Xri
2


i=1 r =1 s=1
1+xri ,s Xri
PN PR PS
r =1
s=1
i=1
2
= θcf
(1 − θcf )
WCPM Seminar
1−xri ,s Xri
PN PR PS
r =1
s=1
i=1
2
Figure : Probabilistic mapping form coarse to
fine: pcf (x|X, θ cf )
Predictive Coarse-Graining
November 27, 2015
31 / 56
Which properties to predict?
The magnetization m is given:
Magnetization in the fine-scale
Z
mfine =
n
1X
xi pf (x)dx
n
i
Magnetization calculated with proposed method (’predcg ’)
Z
mML,fine =
n
1X
xi pcf (x|X, θ cf )pc (X|θ c )p(θ cf , θ c |x(1:N) ) dθ cf dθ c dXdx
n
i
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
32 / 56
Quantity of Interest: Magnetization
Magnetization calculated with relative entropy method ‘relEntr ’
(deterministic mapping fine to coarse: X = ξ(x))
I
I
For each µ calculate optimal model parameter θ ∗cg, det
Since we want to compare the magnetization on the same (fine) scale a
mapping is introduced
I
I
I
if Xr = −1, select randomly a possible configuration xr ,s where either all
xr ,s = −1 or one is xr ,s = −1 and the other xr ,s = 1 (for given r : s ∈ {1 . . . lc })
if Xr = +1, select randomly a possible configuration xr ,s where either all
xr ,s = +1 or one is xr ,s = +1 and the other xr ,s = −1 (for given r : s ∈ 1, 2)
For a given coarse-state the selected fine-states xdet are used for calculating
the magnetization
Relative error in magnetization,
errmag =
WCPM Seminar
km − mtruth k
.
kmtruth k
Predictive Coarse-Graining
November 27, 2015
33 / 56
Quantity of Interest: Correlation
The correlation Rl of all sites being separated by l sites is measured as,
Rl =
n
1 X
< xi xj > .
n
|i−j|=l
Relative correlation error,
errcorr =
WCPM Seminar
kR − Rtruth k
.
kRtruth k
Predictive Coarse-Graining
November 27, 2015
34 / 56
Ising Model - ML Algorithm
1:
2:
3:
4:
5:
6:
7:
7:
8:
8:
9:
10:
11:
12:
13:
14:
14:
14:
for µ = µmin to µmax do
set initial fine-scale parameter: J0 , nf , µ
create data set: x(1:Sf ) ∼ p(x(1:Sf ) |J0 , µ, β) with Sf samples
n
set initial coarse-graining parameter: θ c 0 , θ cf0 , nc = f , µ
lc
θ tc ← θ c 0
t
θ cf ← θ cf0
while !(Convergence check passed) do
function E-Step
X ∼ q(X|x(1:N) , θ tc , θ tcf , µ) ∝ pcf (x(1:N) |X, θ tcf )pc (X|θ tc , µ) with Sq per data point x(i)
function M-Step
∂Uc (X,θ c )
∂L =−β ∂Uc (X,θ c )
+β
∂θ c
∂θ c
∂θ c
q(X|x(1:N) ,θ t ,θ ct )
pc (X|θ ct )
cf
*
+
∂pcf (x(1:N) |X,θ cf )
1
∂L =
∂θ cf
∂θ cf
pcf (x(1:N) |X,θ cf )
q(X|x,θ t ,θ ct )
cf
t +α
∂L
θ t+1
←
θ
θ c ∂θ c
c
c
t+1
t
∂L
θ
← θ cf + αθ
cf
cf ∂θ cf
end while
t
t
xpred ∼ p̃(xnew |θ c , θ cf ) with Spred ∗ Sc samples
function EvalMag(xpred , x(1:N) )
function EvalCorr(xpred , x(1:N) )
15: end for=0
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
35 / 56
Ising Model - Posterior and Prediction
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
36 / 56
Overview
1
Behavior of parameters with respect to µ
2
Influence of available data Sf for training the model
3
Level of coarse-graining
4
Aspects of coarse-grained models
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
37 / 56
Behavior of parameters with respect to µ
Constant attributes:
I
Sc = 200 (samples X ∼ pc (X|θ c ))
I
Sq = 50 (samples X ∼ q(X(i) |x(i) , θ c , θ cf ))
I
J0 = 1 (overall interaction strength)
I
Spred = 100Sc (samples for prediction step after learning the model)
Furthermore:
I
nf = 32, system size fine-scale
I
Lf = Lc = 1, in fine- and coarse scale nearest-neighbor interactions are
regarded
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
38 / 56
Behavior of parametrization with respect to µ
The potentials follows to,
nf
X
1 X
xi
U(J, µ) = − J
xi xj − µ
2
|i−j|=1
i=1
nc
X
X
1
Uc (θ c , µ) = − θc
Xi Xj − µ
Xi .
2
|i−j|=1
i=1
I
Learn two parameters θ c and θ cf of the mapping pcf
I
Learning for every evaluated µ: µ = [−4.5, 4.5] with a step size of 0.6
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
39 / 56
1
θc,predCg
θc,relEntr
2
θcf
θc
0.9
1
0.8
0
−4
−2
0
2
0.7
4
µ
Figure : θc : Parameter of potential Uc
WCPM Seminar
−4
−2
0
2
4
µ
Figure : θcf : Parameter of the mapping pcf (x|X, θcf )
Predictive Coarse-Graining
November 27, 2015
40 / 56
1
Truth
relEntr
predCg
hm(µ)i
0.5
0
−0.5
−1
−4
−2
0
2
4
µ
Figure : Magnetization
Key aspects
I
True magnetization based on data set and predicted by ’predCg’ coincides.
I
Method ’relEntr’ not able to predict data set due to missing mapping
capabilities from coarse to fine.
I
Parameters θ c and θ cf behave symmetrical with respect to µ = 0.
I
Stronger interaction θ c for bigger |µ| but also higher probability θ cf that the
coarse configuration reflects the fine configuration.
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
41 / 56
Behavior with respect to size of data-set
I
Learn θ c and θ cf at each evaluated µ
I
Given different size of data-set Sf with Sf ∈ {5, 10, 20, 50}.
I
Interaction length in fine-scale Lf = 10 with exponential decay of overall
strength J0 = 1.5
Potentials
Uf (J, µ) = −
Lf
nf
X
X
1X
Jk
xi xj − µ
xi
2
k
X
1
1
Xi Xj − θclin
Uc (θ c , µ) = − θc
2
2
|i−j|=1
WCPM Seminar
i=1
|i−j|=k
Predictive Coarse-Graining
nc
X
i
Xi − µ
nc
X
Xi .
i=1
November 27, 2015
42 / 56
1
1
0.5
0.5
hm(µ)i
hm(µ)i
Behavior with respect to size of data-set
0
−0.5
−1
0
−0.5
−2
0
−1
2
−2
µ
0
2
µ
Figure : Uncertainty in magnetization Sf = 5
99% - 1% conf. interval
95% - 5% conf. interval
Pred. mean
Truth
Data
Figure : Uncertainty in magnetization Sf = 10
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
43 / 56
1
1
0.5
0.5
hm(µ)i
hm(µ)i
Behavior with respect to size of data-set
0
−0.5
−1
0
−0.5
−2
0
−1
2
µ
0
2
µ
Figure : Uncertainty in magnetization Sf = 20
WCPM Seminar
−2
Figure : Uncertainty in magnetization Sf = 50
Predictive Coarse-Graining
November 27, 2015
44 / 56
Behavior with respect to size of data-set
0.8
0.1
errcorr
errmag
0.6
5 · 10−2
0.4
0.2
0
0
20
40
60
80
0
100
Sf
20
40
60
80
100
Sf
Figure : Relative mean error of magnetization
with respect to increasing amount of data Sf ,
Sf ∈ {5, 10, 20, 50, 100}
WCPM Seminar
0
Figure : Relative mean error of correlation with
respect to increasing amount of data Sf ,
Sf ∈ {5, 10, 20, 50, 100}
Predictive Coarse-Graining
November 27, 2015
45 / 56
Behavior with respect to size of data-set
1
I
Uncertainty decreasing with
increasing amount of data available
I
By using Sf = 5 predictions
resembling true magnetization well
Sf = 1
Sf = 5
0.8
Sf = 10
Sf = 20
0.6
Rl
Sf = 50
Sf = 100
0.4
Truth
0.2
0
0
5
10
15
l sites apart
Figure : Correlation Rl for µ = 0 with
Sf ∈ {5, 10, 20, 50, 100}
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
46 / 56
Predictability by various levels of coarse-gaining
I
Choose various level of coarse-graining lc
I
Mapping pcf (x|X, θ cf ) responsible for lc ∈ {2, 4, 8, 16} fine variables
I
Coarse-grained potential Uc (X, θ c ) as given before
lc = 2
lc = 4
lc = 8
lc = 16
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
47 / 56
1
1
0.5
0.5
hm(µ)i
hm(µ)i
Predictability by various levels of coarse gaining
0
−0.5
−1
0
−0.5
−2
0
−1
2
−2
µ
0
2
µ
Figure : Uncertainty in magnetization lc = 16
99% confidence interval
95% confidence interval
Posterior mean
Truth
Data
Figure : Uncertainty in magnetization lc = 8
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
48 / 56
1
1
0.5
0.5
hm(µ)i
hm(µ)i
Predictability by various levels of coarse gaining
0
−0.5
−1
0
−0.5
−2
0
−1
2
µ
0
2
µ
Figure : Uncertainty in magnetization lc = 4
WCPM Seminar
−2
Figure : Uncertainty in magnetization lc = 2
Predictive Coarse-Graining
November 27, 2015
49 / 56
Predictability by various levels of coarse gaining
1
I
Applying lc = 8 leads already to
good predictions in magnetization
I
Predicting correlations needs finer
resolution in coarse-scale
lc = 2
lc = 4
0.8
lc = 8
Rl
lc = 16
Truth
0.6
0.4
0.2
0
5
10
15
l sites apart
Figure : Correlation Rl at µ = 0 with
lc ∈ {2, 4, 8, 16}
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
50 / 56
Model Comparison
I
I
Assumption before: Model is given for potential Uc (X, θ c )
Using different order of interactions and interaction lengths
Linear
2nd order
Interaction length 2
3rd order
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
51 / 56
Model Comparison
errcorr
·10−2
4
0.1
5 · 10−2
0
0
100
010
001
110
011
101
111
121
131
112
2
100
010
001
110
011
101
111
121
131
112
errmag
6
Model
Model
Model [abc]
I
(a) a = 0 no linear term, a = 1 linear term
I
(b) b = 0 no 2nd order term, b = x 2nd order term up to interaction length
Lc = x
I
(c) c = 0 no third order term, c = 1 third order term on (nearest correlation)
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
52 / 56
Model Comparison
I
Linear term necessary to describe data
I
Second order interaction necessary
I
In correlation: errors decreasing for including 2nd order interactions with
higher interaction lengths Lc
I
No further improvement for longer ranged third order interactions
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
53 / 56
Outline
Numerical issues / efficiency
I
Use advanced sampling methods / optimization methods, using curvature
information (in progress)
I
Variational approximations
Coarse-Graining
I
Hierarchical coarse-graining
I
How to select mappings from coarse to fine?
I
Probit- or Logit-classification model, O model for coarse variables
Algorithmic
I
Approach with sparsity priors for model selection (in progress).
WCPM Seminar
Predictive Coarse-Graining
November 27, 2015
54 / 56
Bibliography I
S. Are, M. A. Katsoulakis, P. Plecháč, and L. R. Bellet.
Multibody Interactions in Coarse-Graining Schemes for Extended Systems.
SIAM Journal on Scientific Computing, 31(2):987–1015, 2008.
I. Bilionis and N. Zabaras.
A stochastic optimization approach to coarse-graining using a relative-entropy framework.
The Journal of chemical physics, 138(4):044313, Jan. 2013.
C. M. Bishop.
Pattern Recognition and Machine Learning.
Springer, 2006.
A. Chaimovich and M. S. Shell.
Coarse-graining errors and numerical optimization using a relative entropy framework.
The Journal of chemical physics, 134(9):094112, Mar. 2011.
F. Ercolessi and J. B. Adams.
Interatomic potentials from first-principles calculations: The force-matching method.
EPL (Europhysics Letters), 26(8):583, 1994.
S. Izvekov and G. A. Voth.
Multiscale coarse graining of liquid-state systems.
J. Chem. Phys., 123(13):134105, 2005.
M. Katsoulakis, P. Plecháč, and L. Rey-Bellet.
Numerical and statistical methods for the coarse-graining of many-particle stochastic systems.
Journal of Scientific Computing, 37(1):43–71, 2008.
A. P. Lyubartsev and A. Laaksonen.
Calculation of effective interaction potentials from radial distribution functions: A reverse Monte Carlo approach.
Phys. Rev. E, 52:3730–3737, Oct 1995.
Bibliography II
D. Ming and M. E. Wall.
Allostery in a coarse-grained model of protein dynamics.
Phys. Rev. Lett., 95:198103, Nov 2005.
M. S. Shell.
The relative entropy is fundamental to multiscale and inverse thermodynamic problems.
J. Chem. Phys., 129(14):144108, 2008.