Particle Filters in high dimensions
Peter Jan van Leeuwen and Mel Ades
Data-Assimilation Research Centre DARC
University of Reading
Lorentz Center 2011
Data assimilation: general formulation
Bayes theorem:
Solution is pdf!
NO INVERSION !!!
Parameter estimation:
with
Again, no inversion but a direct point-wise multiplication.
Nonlinear filtering: Particle filter
Use ensemble
with
the weights.
What are these weights?
• The weight is the normalised value of the
pdf of the observations given model state .
• For Gaussian distributed variables is is given
by:
• One can just calculate this value
• That is all !!!
Standard Particle filter
Not very efficient !
A closer look at the weights I
Probability space in large-dimensional systems is
‘empty’: the curse of dimensionality
u(x1)
u(x2)
T(x3)
A closer look at the weights II
Assume particle 1 is at 0.1 standard deviations s of M
independent observations.
Assume particle 2 is at 0.2 s of the M observations.
The weight of particle 1 will be
and particle 2 gives
A closer look at the weights III
The ratio of the weights is
Take M=1000 to find
Conclusion: the number of independent observations is
responsible for the degeneracy in particle filters.
Increased efficiency: proposal
density
Instead of drawing samples from p(x) we draw samples from
a proposal pdf q(x).
Use ensemble
with weights
Particle filter with
proposal transition
density
Barotropic vorticity equation
256 X 256 grid points
600 time steps
Typically q=1-3
Decorrelation time scale=
= 25 time steps
Observations
Every 4th gridpoint
Every 50th time step
24 particles
sigma_model=0.03
sigma_obs=0.01
Vorticity field standard particle filter
Truth
Ensemble mean PF
Vorticity field new particle filter
Truth
Ensemble mean
Posterior weights
Rank histogram:
How the truth ranks in the ensemble
Equivalent Weights Particle Filter
Recall:
Assume
Find the minimum for each particle by
perturbing each observation, gives
Equivalent Weights Particle filter
Calculate the full weight for each of these
Determine a target weight C that 80% of the particles
can reach and determine
in
such that
This is a line search, so doable.
Equivalent Weights Particle Filter
• This leads to 80% of the particle having
an almost equal weight, so no degeneracy
by construction!
• Example …
Gaussian pdf in high dimensions
Assume variables are identically independently distributed:
Along each if the axes it looks
like a standard Gaussian:
However, the probability mass as function of the distance to the
centre is given by:
d=100
d=400
r in standard deviation
d=900
The so-called
Important Ring
Why?
In distribution
Fisher has shown
So we find
Importance Ring
Experimental evidence, sums of d squared random numbers:
r
d
Given this what do these
efficient particles represent???
Conclusions
Particle filters with proposal transition density:
•
•
•
•
solve for fully nonlinear posterior pdf
very flexible, much freedom
scalable => high-dimensional problems
extremely efficient
• But what do they represent?
We need more people !
• In Reading only we expect to have 7 new
PDRA positions available in the this year
• We also have PhD vacancies
• And we still have room in the
Data Assimilation and Inverse Methods in
Geosciences MSc program