Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Data Assimilation in an Incompressible
Viscous Fluid
Simon Cotter
Warwick University, March 20th 2009,
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Outline
1
Introduction
2
Prior and Posterior Measures on v0
3
Metropolis-Hastings in Function Space
4
Model Error
5
Summary
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Motivation
Huge Eulerian And Lagrangian
Data Sets
Large amounts of data
Blending with sophisticated
PDE models will lead to:
Better weather forecasting
Better understanding of
ocean flows
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Motivation
Huge Eulerian And Lagrangian
Data Sets
Large amounts of data
Blending with sophisticated
PDE models will lead to:
Better weather forecasting
Better understanding of
ocean flows
Simon Cotter
Example Of Lagrangian Data
Collection
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
General Setup
Observation operator G on input to the dynamical system
Finite dimensional data given by
y = G(u) + ξ,
ξ ∼ N (0, Σ)
Use MCMC to make inference about u
u can be
u = v0 initial condition of dynamical system
u = (v0 , f ) initial condition and time-dependent forcing
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
The (Navier) Stokes Equation
Stokes operator A = −P∆, where projection P puts us
into an appropriate divergence-free function space
For u ∈ H, (Navier) Stokes flow can also be given by an
ODE on H = L2 ∩ {Divergence-free fields}:
dv
+ Av + γB(v , v ) = Pf ,
dt
v (0) = v0 ∈ H.
Theory developed for γ = 0, 1. Numerics for γ = 0:
A fully spectral method can be implemented
Code much less computationally expensive
Given v0 and f , solution calculable by FFT. For now let
f ≡ 0.
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
The Observation Operators
Eulerian Case: GE (v0 )
Given a set of observation points in space
{xj }Jj=1 ⊂ T2 , and set of observations
times {tk }K
k =1 , then
GE (v0 , f ) = {v (xj , tk )}J,K
,
j,k=1
where v is the solution of the Stokes
equation with initial condition v0 .
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
The Observation Operators
Lagrangian Case: GL (u = v0 )
Given a set of initial positions for J passive
tracers {xj }Jj=1 ⊂ T2 , we consider the
paths of the tracers governed by the ODEs
Eulerian Case: GE (v0 )
Given a set of observation points in space
{xj }Jj=1 ⊂ T2 , and set of observations
times
{tk }K
k =1 ,
dt
zj (0)
then
GE (v0 , f ) =
dzj
{v (xj , tk )}J,K
,
j,k=1
where v is the solution of the Stokes
equation with initial condition v0 .
=
v (zj (t), t),
=
xj ,
∀t > 0,
where v is the solution of the Stokes
equation with initial condition v0 . Given a
set of observation times {tk }K
k=1 , we define
GL (v0 , f ) = {zj (tk )}J,K
.
j,k =1
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Observational Noise
As stated before, y ∈ R2JK satisfies
y = G(v0 , f ) + ξ,
ξ ∼ N (0, Σ).
Likelihood that y was created with v (x, 0) = v0 ∈ H:
1
P(y|v0 ) ∝ exp − ky − G(v0 , f )k2Σ .
2
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Outline
1
Introduction
2
Prior and Posterior Measures on v0
3
Metropolis-Hastings in Function Space
4
Model Error
5
Summary
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Bayes’ Theorem and the Posterior
Prior distribution µ0 = P(v0 ).
Likelihood:
1
P(y|v0 ) ∝ exp − ky − G(v0 , f )k2Σ .
2
Posterior distribution µ = P(v0 |y)
Bayes Theorem:
dµ
1
∝ exp − ky − G(v0 , f )k2Σ .
dµ0
2
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Prior Measure
Prior distribution: Gaussian random field
v0 ∼ N (0, A−α ),
α ∈ R+ .
Construct a sample v0 ∼ N (0, A−α ) via Karhunen-Loeve
expansion:
X −α/2
v0 =
λk
φk ξk , ξk ∼ N (0, 1)iid,
k∈K
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Absolute Continuity of the Posterior
Theorem
Assume that f ∈ L2 (0, T , H r ) for some r > 0. Define a
Gaussian measure µ0 on H, with mean vb ∈ H α and covariance
operator A−α . If for any α > 1, then the density
dµ
1
2
(v ) ∝ exp − |y − GE (v )|Σ
(1)
dµ0
2
is µ0 -a.s. non-zero, µ0 −measurable and µ0 integrable. Thus
the posterior measure µ on H, defined by the Radon-Nikodym
derivative (1), is absolutely continuous with respect to the prior
measure µ0 . a
a
Data Assimilation Problems In Fluid Mechanics: Bayesian Formulation In
Function Space SC,MD,JR,AS
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Outline
1
Introduction
2
Prior and Posterior Measures on v0
3
Metropolis-Hastings in Function Space
4
Model Error
5
Summary
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Metropolis-Hastings in Function Space
We seek to create a Markov chain of samples from the
posterior distribution. Given accepted state of the initial
condition u, we propose
v = (1 − β 2 )1/2 u + βw,
w ∼ µ0 = N (0, A−α ).
We accept this sample with probability given by
Metropolis-Hastings formula:
a(u, v ) = 1 ∧ exp(Φ(u) − Φ(v )),
Φ(·) =
1
kG(·) − yk2Σ .
2
Acceptance probability independent of β
Appropriate version of random walk in infinite dimensions
with Gaussian priors
Goldilocks’ principle
Adaptive burn-in to find sensible β
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
MCMC Results
9 paths
36 paths
100 paths
900 paths
(peaks at ! 1111.5)
Re(u0,1)
160
140
Probability Density
120
100
80
60
40
20
0
0.3
0.32
0.34
0.36
0.38
Simon Cotter
0.4
Re(u0,1)
0.42
0.44
0.46
0.48
0.5
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
MCMC Results
1200
1 Observation Time
10 Observation Times
50 Observation Times
100 Observation Times
Re(u0,1)
Probability Density
1000
800
600
400
200
0
0.42
0.425
0.43
0.435
Simon Cotter
0.44
Re(u0,1)
0.445
0.45
0.455
0.46
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Outline
1
Introduction
2
Prior and Posterior Measures on v0
3
Metropolis-Hastings in Function Space
4
Model Error
5
Summary
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
General Setup
Observation operator G on initial condition of system AND
forcing f (x, t)
Data given by
y = G(u = (v0 , f )) + ξ,
ξ ∼ N (0, Σ)
Use data to infer on v (0) = v0 , initial condition of system
AND forcing or “Model Error” f
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Metropolis-Hastings for Model Error
Very similar proposal:
w
v
2 1/2 u
=
(1
−
β
)
+
β
g
ψ ,
f
w
−α
ψ ∼ µ0 = N (0, A ) × ν0 (ψ)
ν0 is space-time GRF prior
Identical acceptance probability with Φ(·) = 21 kG(·) − yk2Σ :
a
u
u
v
v
,
=
1
∧
exp
Φ
−
Φ
g
g .
f
f
Need sufficient regularity in the prior to ensure we can
make sense of G with probability 1
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
What on Earth do these things look like?
u((x,y),t)
!((x,y),t)
1
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
0
0.2
0.4
0.6
t=0.000
0.8
1
Simon Cotter
0
0
0.2
0.4
0.6
t=0.000
0.8
1
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Results
−0.04
Re(!0,1)(t)
Re(f0,1)
−0.05
Time Average
−0.07
Re(!
0,1
)(t)
−0.06
−0.08
−0.09
−0.1
−0.11
0
0.1
0.2
0.3
0.4
Simon Cotter
0.5
Time t
0.6
0.7
0.8
0.9
1
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Eulerian Inference on Model Error
Theorem
Suppose our observation operator returns observations at a
sequence of times 0 < t1 ≤ t2 ≤ . . . ≤ tN ≤ T . Then given an
initial condition and forcing (v0 , η) ∈ H × L2 ((0, T ), H), there
exists an infinite number of alternative η 0 ∈ ×L2 ((0, T ), H) with
η 6= η 0 almost everywhere, such that
GE (v0 , η 0 ) = GE (v0 , η).
Can only infer on:
Z
tj
Fk (tj ) =
e−A(tj −s) η(s)ds
0
for each k ∈ {1 . . . N}.
Not applicable in Lagrangian case
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Results
250
9 Observation Stations
36 Observation Stations
100 Observation Stations
900 Observation Stations
Re(u0,1)
Probability Density
200
150
100
50
0
0.56
0.58
0.6
Re(u0,1)
Simon Cotter
0.62
0.64
0.66
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Results
350
9 Observation Stations
36 Observation Stations
100 Observation Stations
900 Observation Stations
Re(!0,1(0.5))
300
Probability Density
250
200
150
100
50
0
−0.05
0
0.05
Re(!0,1(0.5))
Simon Cotter
0.1
0.15
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Results
350
300
Probability Density
250
9 Observation Stations
36 Observation Stations
100 Observation Stations
900 Observation Stations
Re(F0,1(0.5))
200
150
100
50
0
0.02
0.025
0.03
0.035
0.04
0.045
Re(F (0.5))
Simon Cotter
0,1 Data
0.05
0.055
0.06
Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Results
0.07
9 Observation Stations
36 Observation Stations
100 Observation Stations
900 Observation Stations
Re(F0,1(t))
0.06
0.05
Re(F0,1(t))
0.04
0.03
0.02
0.01
0
−0.01
0
0.1
0.2
0.3
0.4
Simon Cotter
0.5
Time t
0.6
0.7
0.8
0.9
1
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Results
−1.2
−1.4
−1.6
2
0
log(L error in v )
−1.8
−2
−2.2
−2.4
−2.6
−2.8
−3
2
2.5
3
3.5
4
4.5
5
log(Number of Stations)
Simon Cotter
5.5
6
6.5
7
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Results
−0.3
−0.5
−0.6
−0.7
2
log(Relative L ([0,1],H) error of Fkt)
−0.4
−0.8
−0.9
−1
−1.1
2
2.5
3
3.5
4
4.5
5
log(Numb)er of Stations
Simon Cotter
5.5
6
6.5
7
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Outline
1
Introduction
2
Prior and Posterior Measures on v0
3
Metropolis-Hastings in Function Space
4
Model Error
5
Summary
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
Summary
Data assimilation formulated on function space
This leads to MCMC methods on function space
These sampling methods are robust under discretization
We can use data to infer on both the initial condition and
forcing of the system
We require minimum amounts of regularity in the prior
distribution to make sense of the observation operator G
MCMC methods allow us to sample from well-defined
posterior distributions
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
References/Acknowledgements
References
Data Assimilation Problems In Fluid Mechanics: Bayesian
Formulation In Function Space Simon Cotter, Masoumeh
Dashti, James Robinson, Andrew Stuart (Submitted to
Inverse Problems) available at www.simoncotter.com
Data Assimilation Problems In Fluid Mechanics: Sampling
Methods In Function Space, Simon Cotter, Masoumeh
Dashti, James Robinson, Andrew Stuart (In preparation)
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid
Introduction
Prior and Posterior Measures on v0
Metropolis-Hastings in Function Space
Model Error
Summary
References/Acknowledgements
References
Data Assimilation Problems In Fluid Mechanics: Bayesian
Formulation In Function Space Simon Cotter, Masoumeh
Dashti, James Robinson, Andrew Stuart (Submitted to
Inverse Problems) available at www.simoncotter.com
Data Assimilation Problems In Fluid Mechanics: Sampling
Methods In Function Space, Simon Cotter, Masoumeh
Dashti, James Robinson, Andrew Stuart (In preparation)
Acknowledgements
Thanks go to
Andrew Stuart, Masoumeh Dashti and James Robinson
Centre for Scientific Computing for use of cluster
EPSRC for SLC PhD funding
Simon Cotter
Data Assimilation in an Incompressible Viscous Fluid