Exchange-Correlation Functional with Uncertainty
Quantification Capabilities for Density Functional Theory
Manuel Aldegunde
M.A.Aldegunde-Rodriguez@Warwick.ac.uk
Warwick Centre of Predictive Modelling (WCPM)
The University of Warwick
October 27, 2015
http://www2.warwick.ac.uk/wcpm/
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
1 / 51
Outline
1
Introduction
2
Bayesian Linear Regression
3
Exchange Model Training
Data setup
Numerical results
4
Exchange Model Testing
Atomisation Energies
Bulk Properties
5
Summary
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
2 / 51
1
Introduction
2
Bayesian Linear Regression
3
Exchange Model Training
Data setup
Numerical results
4
Exchange Model Testing
Atomisation Energies
Bulk Properties
5
Summary
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
3 / 51
Introduction
Density Functional Theory (DFT) has become the most widespread
technique to study materials in a quantum-mehanical framework
It can provide a favourable trade-off between accuracy and
computation
It is commonly implemented using approximations to make it more
manageable:
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
4 / 51
Introduction
Density Functional Theory (DFT) has become the most widespread
technique to study materials in a quantum-mehanical framework
It can provide a favourable trade-off between accuracy and
computation
It is commonly implemented using approximations to make it more
manageable:
simplify physics, e. g., Born-Oppenheimer approximation
numerical approximations, e.g., pseudopotentials, PAW, ...
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
4 / 51
Introduction
Density Functional Theory (DFT) has become the most widespread
technique to study materials in a quantum-mehanical framework
It can provide a favourable trade-off between accuracy and
computation
It is commonly implemented using approximations to make it more
manageable:
simplify physics, e. g., Born-Oppenheimer approximation
numerical approximations, e.g., pseudopotentials, PAW, ...
One approximation is necessary:
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
4 / 51
Introduction
Density Functional Theory (DFT) has become the most widespread
technique to study materials in a quantum-mehanical framework
It can provide a favourable trade-off between accuracy and
computation
It is commonly implemented using approximations to make it more
manageable:
simplify physics, e. g., Born-Oppenheimer approximation
numerical approximations, e.g., pseudopotentials, PAW, ...
One approximation is necessary:
Exchange-Correlation Functional (unknown in general)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
4 / 51
Density Functional Theory
Approximates the ground state energy of a material system with
charge density n.
Minimisation of the energy functional E DFT [n] for a given system
Z
E DFT [n] = n(r)v (r) dr + T0 [n] + U[n] + E xc [n]
=E b [n] + E xc [n] = E b [n] + E x [n] + E c [n]
1
R. Jones et al. (1989), R. Jones (2015)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
5 / 51
Density Functional Theory
Approximates the ground state energy of a material system with
charge density n.
Minimisation of the energy functional E DFT [n] for a given system
Z
E DFT [n] = n(r)v (r) dr + T0 [n] + U[n] + E xc [n]
=E b [n] + E xc [n] = E b [n] + E x [n] + E c [n]
E xc [n] not known... Need to specify an approximation
1
R. Jones et al. (1989), R. Jones (2015)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
5 / 51
Density Functional Theory: Kohn-Sham method
Kohn and Sham (1965) introduced a methodology to solve the
equations: most common formulation nowadays
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
6 / 51
Density Functional Theory: Kohn-Sham method
Kohn and Sham (1965) introduced a methodology to solve the
equations: most common formulation nowadays
Solve a self-consistent problem using independent electrons in an
effective potential:
1 2
− 2 ∇ + veff (r) ψi (r) = i ψi (r)
R n(r0 ) 0 δE xc [n]
veff (r) = v (r) + |r−r
0 | dr + δn(r)
P
2
n(r) = i |ψi (r)|
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
6 / 51
DFT: Exchange-correlation energies
How to approximate E xc [n]?
R
We can write E xc [n] = nεxc (n; r) dr
nεxc (n; r): XC energy density
1
J. P. Perdew et al. (2001)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
7 / 51
DFT: Exchange-correlation energies
How to approximate E xc [n]?
R
We can write E xc [n] = nεxc (n; r) dr
nεxc (n; r): XC energy density
εxc (n; r) = εxc [n(r)]: LDA
1
J. P. Perdew et al. (2001)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
7 / 51
DFT: Exchange-correlation energies
How to approximate E xc [n]?
R
We can write E xc [n] = nεxc (n; r) dr
nεxc (n; r): XC energy density
εxc (n; r) = εxc [n(r)]: LDA
εxc (n; r) = εxc [n(r), ∇n(r)]: GGA
1
J. P. Perdew et al. (2001)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
7 / 51
DFT: Exchange-correlation energies
How to approximate E xc [n]?
R
We can write E xc [n] = nεxc (n; r) dr
nεxc (n; r): XC energy density
εxc (n; r) = εxc [n(r)]: LDA
εxc (n; r) = εxc [n(r), ∇n(r)]: GGA
εxc (n; r) = εxc [n(r), ∇n(r),
τ (r)]: meta-GGA
P0
2
τ (r) = 2 i 12 |∇ψi (r)|
1
J. P. Perdew et al. (2001)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
7 / 51
DFT: Exchange-correlation energies
How to approximate E xc [n]?
R
We can write E xc [n] = nεxc (n; r) dr
nεxc (n; r): XC energy density
εxc (n; r) = εxc [n(r)]: LDA
εxc (n; r) = εxc [n(r), ∇n(r)]: GGA
εxc (n; r) = εxc [n(r), ∇n(r),
τ (r)]: meta-GGA
P0
2
τ (r) = 2 i 12 |∇ψi (r)|
We can add exact exchange: E x [n] = − 12
P R
i
ψi∗ (r)ψi (r0 )
|r−r0 |
drdr0
But still need to approximate the correlation energy...
1
J. P. Perdew et al. (2001)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
7 / 51
DFT: Exchange-correlation energies
The double integral in the exact exchange makes it much more costly
We chose the meta-GGA framework to build our approximation,
Z
xc
E [n] = nεxc (n(r), ∇n(r), τ (r)) dr
We will focus on exchange energy only,
Z
E xc [n] = E c [n] + nεx (n(r), ∇n(r), τ (r)) dr
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
8 / 51
DFT: Exchange energy
We can transform the dependence on ∇n(r) and τ (r) into two
dimensionless parameters s and α:
s=
|∇n|
;
2(3π 2 )1/3 n4/3
α=
τ − τW
,
τ UEG
2
τ W = |∇n| /8n: Weizsäcker kinetic energy density
3
τ UEG = 10
(3π 2 )2/3 n5/3 : UEG kinetic energy density
Also, we can group all non-local contributions in the exchange
enhancement factor F x (s, α),
Z
Z
x
x
E [n] = nε (n, ∇n, τ ) dr = nεxUEG (n)F x (s, α) dr
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
9 / 51
Linear model for the enhancement factor
To specify a model for the exchange energy we just need to specify a
model for the enhancement factor
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
10 / 51
Linear model for the enhancement factor
To specify a model for the exchange energy we just need to specify a
model for the enhancement factor
We specify a linear model
X
F x (s, α) =
ξix φi (s, α) = (ξ x )T φ(s, α)
i
φi (s, α): basis functions
ξix : linear model coefficients
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
10 / 51
Linear model for the exchange energy
Given this model, the exchange energy becomes a linear model
x
x
E [n; ξ ] =
=
M
X
ξix
i=1
M−1
X
Z
nεxUEG (n)φi (s, α) dr
ξix E x [n; êi ] = (ξ x )T Ex [n; ê]
i=0
R
E x [n; êi ] = nεxUEG (n)φi (s, α) dr is the “basis exchange energy”,
which is obtained if we use ξ x = êi , i.e., only basis φi with ξi = 1,
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
11 / 51
Linear model for the exchange energy
How do we choose the basis functions φi (s, α)?
Follow selection in Wellendorff et al.
Physically based: inspired in previous non-empirical functionals
(PBEsol, MS)
Complete basis: 2D Legendre Polynomials
1
J. Wellendorff et al. (2014)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
12 / 51
Linear model for the exchange energy
How do we choose the basis functions φi (s, α)?
Follow selection in Wellendorff et al.
Physically based: inspired in previous non-empirical functionals
(PBEsol, MS)
Complete basis: 2D Legendre Polynomials
2D Legendre polynomials: argument in [-1, 1]
Rational transformations from s, α to the interval [-1, 1]
2s 2
−1
q + s2
(1 − α2 )3
tα (α) =
1 + α3 + α6
PBEsol → ts (s) =
MS →
1
J. Wellendorff et al. (2014)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
12 / 51
Linear model for the exchange energy
The final exchange enhancement model is
Ms X
Mα
X
F x (s, α) =
i
ξijx Pi (ts (s))Pj (tα (α))
j
The final exchange energy model is therefore
E x [n; ξ x ] =
Ms X
Mα
X
i
Manuel Aldegunde (WCPM)
ξijx
Z
nεxUEG (n)Pi (ts (s))Pj (tα (α)) dr
j
WCPM Seminar Series
October 27, 2015
13 / 51
Linear model for the exchange energy
The final exchange enhancement model is
Ms X
Mα
X
F x (s, α) =
i
ξijx Pi (ts (s))Pj (tα (α))
j
The final exchange energy model is therefore
E x [n; ξ x ] =
Ms X
Mα
X
i
ξijx
Z
nεxUEG (n)Pi (ts (s))Pj (tα (α)) dr
j
How to obtain the coefficients?
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
13 / 51
1
Introduction
2
Bayesian Linear Regression
3
Exchange Model Training
Data setup
Numerical results
4
Exchange Model Testing
Atomisation Energies
Bulk Properties
5
Summary
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
14 / 51
Bayesian linear regression
We want to account for uncertainty in the model
Classical least squares fitting gives a point estimate, not appropriate
A Bayesian model will give us probability distributions for the
coefficients
Uncertainty from a limited data set for the regression
Uncertainty from an inadequate model
p(ξ, β | t) ∝ L(t | x, ξ, β)p(ξ, β)
Example: Bayesian linear regression and ordinary least squares fit using a 10th order polynomial
2
1
0
1
0.0
1
0.5
1.0
1.5
2.0
2.5
3.0
C. Bishop (2006)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
15 / 51
Bayesian linear regression
We want to account for uncertainty in the model
Classical
leastBayesian
squares linear
fittingregression
gives a point
estimate,
appropriate
Example:
and ordinary
leastnot
squares
fit
A Bayesian
model
will
give
us
probability
distributions
for
the
using a 10th order polynomial
coefficients
2
Uncertainty from a limited data set for the regression
Uncertainty from an inadequate model
1
0
1
0.0
0.5
Manuel Aldegunde (WCPM)
1.0
1.5
WCPM Seminar Series
2.0
2.5
3.0
October 27, 2015
15 / 51
Some definitions
t = (t1 , t2 , . . . , tN )T : given data (experimental, simulation, mix)
n = (n1 , n2 , . . . , nN )T : input points (densities for DFT)
L(t | n, model): likelihood function
N (x | µ, v ): normal distribution on x with mean µ and variance v
G(x | α, β): gamma distribution on x with parameters α and β
St(x | µ, λ, ν): Student t-distribution on x with parameters µ, λ and
ν
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
16 / 51
Assumptions
The observed data t follow on average our model and have a noise ε
(includes model inaccuracy),
ti = (ξ x )T Ex [n; ê] + εi
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
17 / 51
Assumptions
The observed data t follow on average our model and have a noise ε
(includes model inaccuracy),
ti = (ξ x )T Ex [n; ê] + εi
The noise is assumed Gaussian with precision β = 1/v = 1/σ 2 and
uncorrelated, so that
ti ∼ N (t | (ξ x )T Ex [n; ê], β −1 )
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
17 / 51
Assumptions
The observed data t follow on average our model and have a noise ε
(includes model inaccuracy),
ti = (ξ x )T Ex [n; ê] + εi
The noise is assumed Gaussian with precision β = 1/v = 1/σ 2 and
uncorrelated, so that
ti ∼ N (t | (ξ x )T Ex [n; ê], β −1 )
The likelihood function is, therefore,
L(t | n, ξ, β) =
N
Y
N (ti | ξ T Ex [ni ; ê], β −1 )
i=1
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
17 / 51
Assumptions
The noise is assumed Gaussian with precision β = 1/v = 1/σ 2 and
uncorrelated, so that
ti ∼ N (t | (ξ x )T Ex [n; ê], β −1 )
The likelihood function is, therefore,
L(t | n, ξ, β) =
N
Y
N (ti | ξ T Ex [ni ; ê], β −1 )
i=1
We choose conjugate priors for ξ and β
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
17 / 51
Priors
Incorporate prior beliefs into the model
Depend on extra parameters: hyperparameters
Conjugate priors keep the posterior propability distribution in the
same family as the prior probability distribution
Prior on ξ: p(ξ | β, m0 , S0 ) = N (ξ | m0 , β −1 S0 )
Prior on β: p(β | a0 , b0 ) = G(β | a0 , b0 )
Joint prior: p(ξ, β) = p(ξ | β)p(β) = N (ξ | m0 , β −1 S0 )G(β | a0 , b0 )
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
18 / 51
Posterior
Probability of a set of parameters given the data
Proportional to the prior distribution of parameters
Proportional to the likelihood of the data
p(ξ, β | t) = R
L(t | x, ξ, β)p(ξ, β)
L(t | x, ξ, β)p(ξ, β) dξ dβ
=N (ξ | mN , β −1 SN )G(β | aN , bN )
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
19 / 51
Posterior
The parameters of the posterior depend on those of the prior and the
data,
h
i
−1
−1
T
T
S−1
=
S
+
Φ
Φ;
m
=
S
S
m
+
Φ
t
0
N
N
0
0
N
aN = a0 + N/2
1 T −1
−1
T
m0 S0 m0 − mT
S
m
+
t
t
bN = b0 +
N
N N
2
Φ is the design matrix
 x ∗
E [n1 ; ê0 ]

..
Φ=
.
···
..
.
∗ ; ê ] · · ·
E x [nN
0
Manuel Aldegunde (WCPM)
  x ∗ T
E x [n1∗ ; êM−1 ]
E [n1 ; ê]
 

..
..
=

.
.
∗ ; ê
E x [nN
M−1 ]
WCPM Seminar Series
∗ ; ê]T
Ex [nN
October 27, 2015
20 / 51
Posterior
The parameters of the posterior depend on those of the prior and the
data,
Φ is the design matrix
 x ∗
E [n1 ; ê0 ] · · ·

..
..
Φ=
.
.
x
∗
E [nN ; ê0 ] · · ·
  x ∗ T
E x [n1∗ ; êM−1 ]
E [n1 ; ê]

 
..
..

=
.
.
∗ ; ê
E x [nN
M−1 ]
∗ ; ê]T
Ex [nN
Rows: For a given data point, a vector with the “basis exchange
energies”
Columns: For a given basis function, its value for every data point
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
20 / 51
Predictive distribution
Once we have our probability distributions for model parameters,
what can we say about new data points?
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
21 / 51
Predictive distribution
Once we have our probability distributions for model parameters,
what can we say about new data points?
Prediction averaged over all possible parameters
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
21 / 51
Predictive distribution
Once we have our probability distributions for model parameters,
what can we say about new data points?
Prediction averaged over all possible parameters
Predictions given with probability distributions
Z
p(t̃ | ñ, t) = p(t̃ | ñ, ξ, β)p(ξ, β | t) dξ dβ
Z
= N (t̃ | ξ T Ex [ñ; ê], β −1 )N (ξ | mN , β −1 SN )G(β | aN , bN ) dξ dβ
= St(t̃ | µ, λ, ν).
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
21 / 51
Predictive distribution
The Student t-distribution St(t̃ | µ, λ, ν) has parameters
µ = Ex [ñ; ê]T mN
−1
aN λ=
1 + Ex [ñ; ê]T SN Ex [ñ; ê]
bN
ν = 2aN .
Its mean, variance and mode are
E[t̃] = µ;
ν>1
ν
1 + Ex [ñ; ê]T SN Ex [ñ; ê]
λ−1 =
;
ν−2
mode[β]
mode[t̃] = µ
cov[t̃] =
Manuel Aldegunde (WCPM)
WCPM Seminar Series
ν>2
October 27, 2015
22 / 51
Predictive distribution
Its mean, variance and mode are
E[t̃] = µ;
ν>1
ν
1 + Ex [ñ; ê]T SN Ex [ñ; ê]
λ−1 =
;
ν−2
mode[β]
mode[t̃] = µ
cov[t̃] =
ν>2
The variance of the prediction depends on the data point
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
22 / 51
Predictive distribution
If we make several predictions, they are correlated,
Z
p(t̃ | ñ, t) = p(t̃ | ñ, ξ, β)p(ξ, β | t) dξ dβ = St(t̃ | µ, Λ, ν)
The mean and covariance are
E[t̃] = Φ̃mN ;
I + Φ̃SN Φ̃
cov[t̃] =
mode[β]
T
Φ̃ is analogous to the design matrix where the rows are the points of
the predictions
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
23 / 51
Hyperparameters: Evidence approximation
One last point to be solved
How do we obtain the hyperparameters?
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
24 / 51
Hyperparameters: Evidence approximation
One last point to be solved
How do we obtain the hyperparameters?
We chose the evidence approximation: maximise the log of the
marginal likelihood (evidence function)
log p(t | m0 , S0 , a0 , b0 ) =
Z
=log p(t | ξ, β, m0 , S0 , a0 , b0 )p(ξ, β | m0 , S0 , a0 , b0 )dξdβ
|SN | N
Γ(aN )
1
log
− log(2π) + log
+
2
|S0 |
2
Γ(a0 )
+ a0 log(b0 ) − aN log(bN )
=E(m0 , S0 , a0 , b0 ) =
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
24 / 51
Hyperparameters: Evidence approximation
2
For a model with M parameters, m0 , S−1
0 , a0 and b0 have ∼ M
parameters
1
M. E. Tipping (2001), C. Bishop (2006)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
25 / 51
Hyperparameters: Evidence approximation
2
For a model with M parameters, m0 , S−1
0 , a0 and b0 have ∼ M
parameters
Using m0 = 0 and S0−1 = αI, we only have three hyperparameters
and we can easily find a maximum of the evidence function (Bayesian
ridge regression)
1
M. E. Tipping (2001), C. Bishop (2006)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
25 / 51
Hyperparameters: Evidence approximation
2
For a model with M parameters, m0 , S−1
0 , a0 and b0 have ∼ M
parameters
Using m0 = 0 and S0−1 = αI, we only have three hyperparameters
and we can easily find a maximum of the evidence function (Bayesian
ridge regression)
Using S−1
0 = diag (α0 , . . . , αM−1 ), we have M + 2 hyperparameters
(Relevance Vector Machine)
Induces sparsity (some of the αi go to infinity)
Only keeps relevant terms: automatic model selection
1
M. E. Tipping (2001), C. Bishop (2006)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
25 / 51
Hyperparameters: Evidence approximation
Algorithm 1 Hyperparameter optimisation for the RVM.
−1
1: S0 = diag (α0 , . . . , αM−1 ), m0 = 0.
2: Initialize αi from random numbers r ∈ (0, 1010 ).
3: repeat
4:
repeat
5:
for all i = 0, 1, . . . , M − 1 do
6:
Update αi as αinew = [S ] + 1aN [m ]2 .
N ii
7:
8:
9:
10:
11:
12:
bN
N i
Update SN , mN .
end for
until ∆αi < 10−5 % or αi > 1010
Update a0 , b0 with a Newton iteration.
Update SN , mN .
until ∆α, ∆a0 , ∆b0 < 10−4 %
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
26 / 51
Relevance Vector Machine: Example
Generate data from f (x) = sin(2πx) + ε
ε ∼ N (ε | 0, 0.12 )
Use 10 sine basis functions, sin(kπx); k = 0, 1, ..., 9
Fitted mN = [0, 0.003, 1.006, −0.016, 0, 0, 0, 0, 0, 0],
mode[σN ] = 0.097
Only three coefficients remain
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
27 / 51
Relevance Vector Machine: Example
Generate data from f (x) = sin(2πx) + ε
ε ∼ N (ε | 0, 0.12 )
1.5
Use 10 sine basis functions, sin(kπx); k = 0, 1, ..., 9
Fitted mN = [0, 0.003, 1.006, −0.016, 0, 0, 0, 0, 0, 0],
mode[σN ] = 0.097
1.0
Only three coefficients remain
0.5
0.0
0.5
1.0
1.5
1
0
Manuel Aldegunde (WCPM)
1
2
3
4
WCPM Seminar Series
5
6
7
October 27, 2015
27 / 51
1
Introduction
2
Bayesian Linear Regression
3
Exchange Model Training
Data setup
Numerical results
4
Exchange Model Testing
Atomisation Energies
Bulk Properties
5
Summary
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
28 / 51
1
Introduction
2
Bayesian Linear Regression
3
Exchange Model Training
Data setup
Numerical results
4
Exchange Model Testing
Atomisation Energies
Bulk Properties
5
Summary
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
29 / 51
Training data
The exchange energy cannot be measured
Absolute energies cannot be measured
How to train the model?
Easy to get from DFT simulations
Experimentally available
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
30 / 51
Training data
The exchange energy cannot be measured
Absolute energies cannot be measured
How to train the model?
Easy to get from DFT simulations
Experimentally available
Atomisation/cohesive energies
Given a system M = AnA BnB . . .,
Eat
Manuel Aldegunde (WCPM)
1
=
N
!
X
n I EI − EM
;
I = A, B, . . .
I
WCPM Seminar Series
October 27, 2015
30 / 51
Atomisation energies
Decomposing energies into components...
Eat
1
=
N
=
b
Eat
Manuel Aldegunde (WCPM)
!
X
b
x
c
nI (EIb + EIx + EIc ) − (EM
+ EM
+ EM
)
I
x
c
+ Eat
+ Eat
,
WCPM Seminar Series
October 27, 2015
31 / 51
Atomisation energies
Decomposing energies into components...
Using our exchange energy model...
"
#
X
x
x
x
T 1
nI E [ni ; ê] − E [nM ; ê]
Eat = ξ
N
I
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
31 / 51
Atomisation energies
Decomposing energies into components...
Using our exchange energy model...
The design matrix becomes...
1 P

x
x
T
N (PI ∈s1 nI E [ni ; ê] − E [ns1 ; ê])
x
x
T 
 1(
I ∈s2 nI E [ni ; ê] − E [ns2 ; ê]) 
N
Φ=

..


.
P
1
x
x
T
N ( I ∈sN nI E [ni ; ê] − E [nsN ; ê])
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
31 / 51
Atomisation energies
What is my data vector t?
No access to exchange energy directly
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
32 / 51
Atomisation energies
What is my data vector t?
No access to exchange energy directly
b and E c do not depend on our model...
But Eat
at
 exp
b [n ] − E c [n ]
Eat (s1 ) − Eat
1
at 1
 E exp (s2 ) − E b [n2 ] − E c [n2 ]
at
at
 at
t=
..

.





exp
b
c
Eat (sN ) − Eat [nN ] − Eat [nN ]
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
32 / 51
Atomisation energies
What is my data vector t?
No access to exchange energy directly
b and E c do not depend on our model...
But Eat
at
 exp
b [n ] − E c [n ]
Eat (s1 ) − Eat
1
at 1
 E exp (s2 ) − E b [n2 ] − E c [n2 ]
at
at
 at
t=
..

.





exp
b
c
Eat (sN ) − Eat [nN ] − Eat [nN ]
We have all we need for the regression
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
32 / 51
Indirect measurements
What if we want to add other data?
1
A. B. Alchagirov et al. (2001)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
33 / 51
Indirect measurements
What if we want to add other data?
Linear on the energy?
1
A. B. Alchagirov et al. (2001)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
33 / 51
Indirect measurements
What if we want to add other data?
Linear on the energy?
We can use it directly within our model
1
A. B. Alchagirov et al. (2001)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
33 / 51
Indirect measurements
What if we want to add other data?
Linear on the energy?
Not linear?
1
A. B. Alchagirov et al. (2001)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
33 / 51
Indirect measurements
What if we want to add other data?
Linear on the energy?
Not linear?
Transform into impact on energy if possible
Non-analytically tractable posterior otherwise
1
A. B. Alchagirov et al. (2001)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
33 / 51
Indirect measurements
What if we want to add other data?
Linear on the energy?
Not linear?
Example: equilibrium volume (V0 ), bulk modulus and pressure
derivative (B0 , B1 ) → E (V ) through equation of state
1
A. B. Alchagirov et al. (2001)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
33 / 51
Indirect measurements
Example: equilibrium volume (V0 ), bulk modulus and pressure
derivative (B0 , B1 ) → E (V ) through equation of state
1/3
2/3
V
V
V0
= γ T φ(V )
E (V ) = a + b 01/3 + c 02/3 + d
V
V
V
1
A. B. Alchagirov et al. (2001)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
33 / 51
Indirect measurements
Example: equilibrium volume (V0 ), bulk modulus and pressure
derivative (B0 , B1 ) → E (V ) through equation of state
1/3
2/3
V
V
V0
E (V ) = a + b 01/3 + c 02/3 + d
= γ T φ(V )
V
V
V
where

1
1 1
 3
2 1

 18 10 4
108 50 16
1



1
−E0


0
0

γ = 


0
9V0 B0 
0
27V0 B0 B1
A. B. Alchagirov et al. (2001)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
33 / 51
Indirect measurements
Example: equilibrium volume (V0 ), bulk modulus and pressure
derivative (B0 , B1 ) → E (V ) through equation of state
1/3
2/3
V
V
V0
= γ T φ(V )
E (V ) = a + b 01/3 + c 02/3 + d
V
V
V
From experimental V0 , B0 , B1 we can also obtain cohesive energies of
the strained material
1
A. B. Alchagirov et al. (2001)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
33 / 51
1
Introduction
2
Bayesian Linear Regression
3
Exchange Model Training
Data setup
Numerical results
4
Exchange Model Testing
Atomisation Energies
Bulk Properties
5
Summary
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
34 / 51
Data sets
Elementary solids
Molecules
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
35 / 51
Data sets
Elementary solids
20 cubic solids: 13 training + 7 testing
Extended using bulk properties (4 strains each)
Molecules
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
35 / 51
Data sets
Elementary solids
20 cubic solids: 13 training + 7 testing
Extended using bulk properties (4 strains each)
Molecules
G2/97 data set (small molecules): 120 training + 28 testing
Larger molecules from G3/99 only for testing
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
35 / 51
Model
Linear model with 10 × 10 terms
F x (s, α) =
9 X
9
X
ξijx Pi (ts (s))Pj (tα (α))
i=0 j=0
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
36 / 51
Model
Linear model with 10 × 10 terms
DFT simulations run
Using PBE functional
Cut-off energy of 800 eV
Monkhorst-Pack mesh (16 × 16 × 16)
Relaxing with a maximum force criterion of 0.05 eV/Å
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
36 / 51
Results
10
Order in s
8
6
4
2
0
22
Manuel Aldegunde (WCPM)
0
2
4
6
Order in α
WCPM Seminar Series
8
1.20
1.05
0.90
0.75
0.60
0.45
0.30
0.15
10 0.00
October 27, 2015
37 / 51
Enhancement factor with 1 and 2 σ intervals
1.20
1.8
1.15
Lieb-Oxford bound
1.10
1.6
This work
mBEEF
MS0/PBEsol
PBE
TPSS
MVS
1.4
Fx (s =0,α)
Fx (s,α =1)
Results
1.2
1.05
1.00
0.95
0.90
0.85
1.0
0
1
2
3
s ∝ |∇ |/n4/3
Manuel Aldegunde (WCPM)
4
5
0.800
WCPM Seminar Series
1
2
This work
mBEEF
MS0
TPSS
MVS
3
α =(τ−τW )/τUEG
October 27, 2015
4
5
38 / 51
1
Introduction
2
Bayesian Linear Regression
3
Exchange Model Training
Data setup
Numerical results
4
Exchange Model Testing
Atomisation Energies
Bulk Properties
5
Summary
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
39 / 51
1
Introduction
2
Bayesian Linear Regression
3
Exchange Model Training
Data setup
Numerical results
4
Exchange Model Testing
Atomisation Energies
Bulk Properties
5
Summary
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
40 / 51
Atomisation energies
Cohesive Energy [eV]
Prediction of atomisation energies in the 7 test solids
9
8
7
6
5
4
3
2
1
0
K
Manuel Aldegunde (WCPM)
Ca
V
Cu
C
WCPM Seminar Series
Al
Fe-FM
October 27, 2015
41 / 51
Atomisation energies
Prediction of atomisation energies in the 7 test solids
Error in predictions for solids and molecules
XC functional
This work
PBE
Error
MAE
MARE
MAE
MARE
G2/97-test
0.116
3.27
G2/97
0.103
1.46
0.703
5.09
EL20-test
0.243
8.56
EL20
0.0975
5.62
0.238
6.88
Table: Mean absolute error (in eV) and mean absolute relative error (in %)
of the predictions of atomisation energies using the average model for the
training sets containing molecules (G2/97) and solids (EL20).
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
41 / 51
Atomisation energies
Prediction of atomisation energies in the 7 test solids
Error in predictions for solids and molecules
XC functional
This work
PBE
Error
MAE
MARE
MAE
MARE
G2/97-test
0.116
3.27
G2/97
0.103
1.46
0.703
5.09
EL20-test
0.243
8.56
EL20
0.0975
5.62
0.238
6.88
G2/97 MAE better than TPSS (0.28 eV), BEEF-vdW (0.16 eV),
B3LYP (0.14 eV) or PBE0 (0.21 eV)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
41 / 51
Correlation functional
We assumed a fixed correlation energy functional
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
42 / 51
Correlation functional
We assumed a fixed correlation energy functional
What’s the impact of our choice?
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
42 / 51
We assumed a fixed correlation energy functional
What’s the impact of our choice?
Coefficients with PBEsol (left) and vPBE (right) correlations:
10
8
6
4
2
0
22
0
2
4
6
Order in α
Manuel Aldegunde (WCPM)
8
1.35
1.20
1.05
0.90
0.75
0.60
0.45
0.30
0.15
10 0.00
10
8
Order in s
Order in s
Correlation functional
6
4
2
0
22
WCPM Seminar Series
0
2
4
6
Order in α
8
1.35
1.20
1.05
0.90
0.75
0.60
0.45
0.30
0.15
10 0.00
October 27, 2015
42 / 51
Correlation functional
We assumed a fixed correlation energy functional
What’s the impact of our choice?
Errors:
C functional
PBE
PBEsol
vPBE
TPSS
Manuel Aldegunde (WCPM)
Error
MAE
MARE
MAE
MARE
MAE
MARE
MAE
MARE
G2/97-test
0.116
3.27
0.116
2.91
0.110
2.72
0.108
2.68
WCPM Seminar Series
G2/97
0.103
1.46
0.108
1.55
0.107
1.41
0.104
1.42
EL20-test
0.243
8.56
0.204
6.12
0.226
6.45
0.227
6.85
EL20
0.0975
5.62
0.172
4.98
0.184
5.17
0.190
5.53
October 27, 2015
42 / 51
1
Introduction
2
Bayesian Linear Regression
3
Exchange Model Training
Data setup
Numerical results
4
Exchange Model Testing
Atomisation Energies
Bulk Properties
5
Summary
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
43 / 51
Test set
SL20 test set including
13 elemental solids
I-VII, II-VI, III-V and IV-IV compounds
7 of them in the training set (elemental solids)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
44 / 51
Uncertainty propagation from DFT
Bulk properties are not obtained directly from DFT simulations
How to propagate the uncertainty?
We use a nested Monte Carlo approach
Sample model coefficients from the posterior distribution
Fit the EOS to the values from this X energy (Bayesian fit)
Sample coefficients from the fitting to calculate V0 , B0 , B1
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
45 / 51
Uncertainty propagation from DFT
Algorithm 2
Calculation of uncertainty for V0 and B0 .
1: Input: system s with unit cell (x1 , x2 , x3 ).
2: Input: N1max , N2max , the maximum iterations.
3: for 5 strains 0.95 ≤ σi ≤ 1.05 do
4:
Strain unit cell by σi : xα → σi xα , α = 1, 2, 3.
5:
Self-consistent simulation of strained system.
6:
Keep the self-consistent electron density ni∗ = n(σi ).
7: end for
8: N1 = 0
9: repeat
10:
Sample ξN1 , βN1 .
11:
Non self-consistent simulation with ξN1 , βN1 and ni∗ .
12:
N2 = 0
13:
repeat
14:
Sample γ N2 .
15:
Calculate V0 , B0 .
16:
until N2 = N2max
17:
N1 = N1 + 1
18: until N1 = N1max
19: Collect statistics on calculated V0 , B0 .
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
46 / 51
Results
Equilibrium lattice constants for SL20 materials
6.5
Lattice constant [ ]
6.0
5.5
5.0
4.5
4.0
3.5
3.0 Li Na Ca Sr Ba Al Cu Rh Pd Ag
Manuel Aldegunde (WCPM)
C
Si Ge SiC GaAs LiF LiCl NaF NaCl MgO
WCPM Seminar Series
October 27, 2015
47 / 51
Bulk modulus [GPa]
Results
Equilibrium bulk moduli for SL20 materials
500
400
300
200
100
0
100 Li Na Ca Sr Ba Al Cu Rh Pd Ag
Manuel Aldegunde (WCPM)
C
Si Ge SiC GaAs LiF LiCl NaF NaCl MgO
WCPM Seminar Series
October 27, 2015
47 / 51
Results
What if the DFT results have another error sources?
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
48 / 51
Results
What if the DFT results have another error sources?
For example, assume a numerical error with Gaussian distribution and
standard deviation 10 mV
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
48 / 51
Results
Equilibrium lattice constants for SL20 materials
Lattice constant [ ]
6.5
6.0
5.5
5.0
4.5
4.0
3.5
3.0 Li Na Ca Sr Ba Al Cu Rh Pd Ag C
Manuel Aldegunde (WCPM)
WCPM Seminar Series
Si Ge SiC GaAs LiF LiCl NaF NaCl MgO
October 27, 2015
48 / 51
Bulk modulus [GPa]
Results
Equilibrium bulk moduli for SL20 materials
600
500
400
300
200
100
0
100 Li Na Ca Sr Ba Al Cu Rh Pd Ag C
Manuel Aldegunde (WCPM)
WCPM Seminar Series
Si Ge SiC GaAs LiF LiCl NaF NaCl MgO
October 27, 2015
48 / 51
1
Introduction
2
Bayesian Linear Regression
3
Exchange Model Training
Data setup
Numerical results
4
Exchange Model Testing
Atomisation Energies
Bulk Properties
5
Summary
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
49 / 51
Summary
Bayesian framework to obtain an exchange energy functional
Use of a linear model
Coefficients of the model are random variables
Basis functions are fixed
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
50 / 51
Summary
Bayesian framework to obtain an exchange energy functional
Use of a relevance vector machine to find hyperparameters
Automatic selection of relevant basis functions (model selection)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
50 / 51
Summary
Bayesian framework to obtain an exchange energy functional
Use of a relevance vector machine to find hyperparameters
Obtained exchange energy from a simulation has an uncertainty
Limited data in the training (can be reduced to zero asymptotically
with more data)
Limited model space, meta-GGA (cannot be reduced to zero
asymptotically with more expansion terms)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
50 / 51
Summary
Bayesian framework to obtain an exchange energy functional
Use of a relevance vector machine to find hyperparameters
Obtained exchange energy from a simulation has an uncertainty
This uncertainty can be propagated to other derived quantities
Bulk properties (shown)
Band diagrams, phonon properties, transport coefficients, enrgy
barriers, etc.
Further models based on DFT results (e.g., cluster expansion for alloy
modelling)
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
50 / 51
Acknowledgments
Thank you for your attention!
Acknowledgments
EPSRC Strategic Package Project EP/L027682/1 for research at WCPM
Manuel Aldegunde (WCPM)
WCPM Seminar Series
October 27, 2015
51 / 51