Monte Carlo Approximation
of Monte Carlo Filters
Adam M. Johansen et al.
Collaborators Include:
Arnaud Doucet, Axel Finke, Anthony Lee, Nick Whiteley
7th January 2014
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Context & Outline
Filtering in State-Space Models:
I
SIR Particle Filters [GSS93]
I
Rao-Blackwellized Particle Filters [AD02, CL00]
I
Block-Sampling Particle Filters [DBS06]
Exact Approximation of Monte Carlo Algorithms:
I
Particle MCMC [ADH10]
I
SMC2 [CJP13]
Approximating the RBPF
I
Approximated Rao-Blackwellized Particle Filters [CSOL11]
I
Exactly-approximated RBPFs [JWD12]
Approximating the BSPF
I
Local SMC [JD14]
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Particle MCMC
I
I
MCMC algorithms which employ SMC proposals [ADH10]
SMC algorithm as a collection of RVs
I
I
I
I
Construct MCMC algorithms:
I
I
I
I
I
Values
Weights
Ancestral Lines
With many auxiliary variables
Exactly invariant for distribution on extended space
Standard MCMC arguments justify strategy
SMC2 employs the same approach within an SMC setting.
What else does this allow us to do with SMC?
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Ancestral Trees
t=1
t=2
t=3
a13 =1
a43 =3
a12 =1
a42 =3
b23,1:3 =(1, 1, 2) b43,1:3 =(3, 3, 4) b63,1:3 =(4, 5, 6)
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
SMC Distributions
We’ll need the SMC Distribution:
M
an−L+2:n , xn−L+1:n , k; xn−L
ψn,L
"M
#
"
#
n
M
Y
Y
Y
i
a
p
=
q( xin−L+1 xn−L )
r(ap |wp−1 )
q xip |xp−1
r(k|wn )
i=1
i=1
p=n−L+2
and the conditional SMC Distribution:
ek
k
k
M
ek
e
b
,
k,
x
e
ψen,L
a
,
x
;
x
n−L
n−L+1:n−1
n−L+1:n
n−L+2:n
n−L+1:n
M
en−L+1:n , k; xn−L )
ψn,L
(e
an−L+2:n , x
"
= #
ebk
n
e
e
Q
bk
bn
n,n−L+1
n,p
n,p−1
k
e
e n)
e p−1 q x
q x
en−L+1 |xn−L
r bn,p |w
xp−1
r(k|w
ep |e
p=n−L+2
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
A (Rather Broad) Class of Hidden Markov Models
y1
x1
z1
x2
z2
y2
x3
z3
y3
I
I
I
Unobserved Markov chain {(Xn , Zn )} transition f .
Observed process {Yn } conditional density g.
Density:
p(x1:n , z1:n , y1:n ) = f1 (x1 , z1 )g(y1 |x1 , z1 )
n
Y
f (xi , zi |xi−1 , zi−1 )g(yi |xi , zi ).
i=2
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Formal Solutions
I
Filtering and Prediction Recursions:
p(xn , zn |y1:n−1 )g(yn |xn , zn )
p(xn , zn |y1:n ) = R
0
p(xn , zn0 |y1:n−1 )g(yn |x0n , zn0 )d(x0n , zn0 )
Z
p(xn+1 , zn+1 |y1:n ) = p(xn , zn |y1:n )f (xn+1 , zn+1 |xn , zn )d(xn , zn )
I
Smoothing:
p((x, z)1:n |y1:n ) ∝ p((x, z)1:n−1 |y1:n−1 )f ((x, z)n |(x, z)n−1 )g(yn |(x, z)n )
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
A Simple SIR Filter
Algorithmically, at iteration n:
I
I
i
Given {Wn−1
, (X, Z)i1:n−1 } for i = 1, . . . , N :
e Z)
e i
}.
Resample, obtaining { 1 , (X,
1:n−1
N
I
e Z)
e i )
Sample (X, Z)in ∼ qn (·|(X,
n−1
I
Weight
Wni ∝
e Z)
e i )g(yn |(X,Z)i )
f ((X,Z)in |(X,
n
n−1
e Z)
e i )
qn ((X,Z)i |(X,
n
n−1
Actually:
I
Resample efficiently.
I
Only resample when necessary.
I
...
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
A Rao-Blackwellized SIR Filter
Algorithmically, at iteration n:
I
X,i
i
i
Given {Wn−1
, (X1:n−1
, p(z1:n−1 |X1:n−1
, y1:n−1 )}
ei
ei
, p(z1:n−1 |X
Resample, obtaining { 1 , (X
I
For i = 1, . . . , N :
I
I
I
1:n−1 , y1:n−1 ))}.
1:n−1
N
ei )
Sample Xni ∼ qn (·|X
n−1
i
i
ei
← (X
Set X1:n
1:n−1 , Xn ).
i
ei
p(Xn
,yn |X
n−1 )
ei
qn (X i |X
)
I
Weight WnX,i ∝
I
i
Compute p(z1:n |y1:n , X1:n
).
n
n−1
Requires analytically tractable substructure.
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
An Approximate Rao-Blackwellized SIR Filter
Algorithmically, at iteration n:
I
I
I
X,i
i
i
Given {Wn−1
, (X1:n−1
, pb(z1:n−1 |X1:n−1
, y1:n−1 )}
ei
ei
, pb(z1:n−1 |X
Resample, obtaining { 1 , (X
1:n−1 , y1:n−1 ))}.
1:n−1
N
For i = 1, . . . , N :
I
I
ei )
Sample Xni ∼ qn (·|X
n−1
i
i
ei
← (X
Set X1:n
1:n−1 , Xn ).
i
ei
p
b(Xn
,yn |X
n−1 )
ei
qn (X i |X
)
I
Weight WnX,i ∝
I
i
Compute pb(z1:n |y1:n , X1:n
).
n
n−1
Is approximate; how does error accumulate?
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Exactly Approximated Rao-Blackwellized SIR Filter
At time n = 1
I Sample, X i ∼ q x ( ·| y1 ).
1
i,j
z ·| X i , y .
I Sample, Z
1 1
1 ∼q
I Compute and normalise the local weights
w1z X1i , Z1i,j
p(X i , y1 , Z i,j )
:= 1 1 , W1z,i,j
q z Z1i,j X1i , y1
define pb(X1i , y1 ) :=
:=
w1z X1i , Z1i,j
PM
z X i , Z i,k
w
1
1
k=1 1
M
1 X z i i,j w1 X1 , Z1 .
M
j=1
I
Compute and normalise the top-level weights
i
w1x X1
w1x X1i
pb(X1i , y1 )
x,i
, W1 := PN
:= x
.
x
k
q X1i |y1
k=1 w1 X1
At times n ≥ 2, resample and do essentially the same again. . .
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Toy Example: Model
We use a simulated sequence of 100
defined by the densities:
x1
µ(x1 , z1 ) =N
z1
xn
f (xn , zn |xn−1 , zn−1 ) =N
zn
g(yn |xn , zn ) =N yn ;
observations from the model
0
1 0
;
0
0 1
xn−1
1 0
;
,
zn−1
0 1
2
xn
σx 0
,
zn
0 σz2
Consider IMSE (relative to optimal filter) of filtering estimate of
first coordinate marginals.
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Mean Squared Filtering Error
Approximation of the RBPF
N = 10
−1
10
N = 20
N = 40
N = 80
−2
10
N = 160
0
10
1
10
Number of Lower−Level Particles, M
2
10
For σx2 = σz2 = 1.
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Computational Performance
0
Mean Squared Filtering Error
10
N = 10
N = 20
N = 40
N = 80
N = 160
−1
10
−2
10
−3
10
1
10
2
10
3
10
Computational Cost, N(M+1)
4
10
5
10
For σx2 = σz2 = 1.
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Computational Performance
N = 10
N = 20
N = 40
N = 80
N = 160
1.9
Mean Squared Filtering Error
10
1.8
10
1.7
10
1.6
10
1
10
2
10
3
10
Computational Cost, N(M+1)
4
10
5
10
For σx2 = 102 and σz2 = 0.12 .
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
What About Other HMMs / Algorithms?
Returning to:
x1
x2
x3
x4
x5
x6
y1
y2
y3
y4
y5
y6
I
Unobserved Markov chain {Xn } transition f .
I
Observed process {Yn } conditional density g.
Density:
I
p(x1:n , y1:n ) = f1 (x1 )g(y1 |x1 )
n
Y
f (xi |xi−1 )g(yi |xi ).
i=2
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Block Sampling: An Idealised Approach
At time n, given x1:n−1 ; discard xn−L+1:n−1 :
I
Sample from q(xn−L+1:n |xn−L , yn−L+1:n ).
I
Weight with
W (x1:n ) =
I
p(x1:n |y1:n )
p(x1:n−L |y1:n−1 )q(xn−L+1:n |xn−L , y1:n−L+1:n )
Optimally,
q(xn−L+1:n |xn−L , yn−L+1:n ) =p(xn−L+1:n |xn−L , yn−L+1:n )
p(x1:n−L |y1:n )
W (x1:n ) ∝
=p(yn |x1:n−L , yn−L+1:n−1 )
p(x1:n−L |y1:n−1 )
I
Typically intractable; auxiliary variable approach in [DBS06].
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Local Particle Filtering: Current Trajectories
4
3
2
1
0
−1
−2
−3
−4
0
2
4
6
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
8
10
,
12
14
16
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Local Particle Filtering: First Particle
4
3
2
1
0
−1
−2
−3
−4
0
2
4
6
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
8
10
,
12
14
16
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Local Particle Filtering: SMC Proposal
4
3
2
1
0
−1
−2
−3
−4
0
2
4
6
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
8
10
,
12
14
16
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Local Particle Filtering: CSMC Auxiliary Proposal
4
3
2
1
0
−1
−2
−3
−4
0
2
4
6
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
8
10
,
12
14
16
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Local SMC
I
Propose from:
⊗n−1
M
U1:M
(b1:n−2 , k)p(x1:n−1 |y1:n−1 )ψn,L
(an−L+2:n , xn−L+1:n , k; xn−L )
k
k
M
en−L+2:n−1 , x
en−L+1:n−1 ; xn−L ||bn−L+2:n−1 , xn−L+1:n−1
ψen−1,L−1 a
I
Target:
k̄
b
n,n−L+1:n
⊗n
U1:M
(b1:n−L , b̄k̄n,n−L+1:n−1 , k̄)p(x1:n−L , xn−L+1:n
|y1:n )
k̄
bn,n−L+1:n
k
k
M
k̄
e
ψn,L an−L+2:n , xn−L+1:n ; xn−L b̄n,n−L+1:n , xn−L+1:n
M
en−L+1:n−1 , k; xn−L ) .
ψn−1,L−1
(e
an−L+2:n−1 , x
I
Weight: Z̄n−L+1:n /Zen−L+1:n−1 .
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Stochastic Volatility Bootstrap Local SMC
I
Model:
f (xi |xi−1 ) =N φxi−1 , σ 2
g(yi |xi ) =N 0, β 2 exp(xi )
I
Top Level:
I
I
I
Local SMC proposal.
Stratified resampling when ESS< N/2.
Local SMC Proposal:
I
Proposal:
q(xt |xt−1 , yt ) = f (xt |xt−1 )
I
Weighting:
W (xt−1 , xt ) ∝
I
f (xt |xt−1 )g(yt |xt )
= g(yt |xt )
f (xt |xt−1 )
Resample residually every iteration.
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
SV Exchange Rata Data
Exchange Rate Data
5
4
Observed Values
3
2
1
0
−1
−2
−3
−4
0
100
200
300
400
500
600
700
800
900
n
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
SV Bootstrap Local SMC: M=100
N = 100, M = 100
100
Average Number of Unique Values
90
80
70
L=2
L=4
L=6
L = 10
60
50
40
30
20
10
0
0
100
200
300
400
500
600
700
800
900
n
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
SV Bootstrap Local SMC: M=1000
N = 100, M = 1000
100
Average Number of Unique Values
90
80
70
60
L=2
L=4
L=6
L = 10
L = 14
L = 18
50
40
30
20
10
0
0
100
200
300
400
500
600
700
800
900
n
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
SV Bootstrap Local SMC: M=10000
N=100, M=10,000
100
Average Number of Unique Values
90
80
70
60
50
L=2
L=4
L=6
L = 10
L = 14
L = 18
L = 22
L = 26
40
30
20
10
0
0
100
200
300
400
500
600
700
800
900
n
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Some Heuristics
Recent calculations suggest that, under appropriate assumptions,
at fixed cost (2L − 1) · M · N :
I
Optimal L is determined solely by the mixing of the HMM.
I
Optimal M is a linear function of L.
I
N can then be obtained from M, L and available budget.
In practice:
I
L can be chosen using pilot runs,
I
and M fine-tuned once L is chosen.
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
In Conclusion
I
SMC can be used hierarchically.
I
Software implementation is not difficult [Joh09, Zho13].
The Rao-Blackwellized particle filter can be approximated
exactly
I
I
I
I
I
I
Can reduce estimator variance at fixed cost.
Attractive for distributed/parallel implementation.
Allows combination of different sorts of particle filter.
Can be combined with other techniques for parameter
estimation etc..
The optimal block-sampling particle filter can be
approximated exactly
I
I
I
Requiring only simulation from the transition and evaluation of
the likelihood
Easy to parallelise
Low storage cost
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
References I
[AD02] C. Andrieu and A. Doucet. Particle filtering for partially observed Gaussian state space models. Journal
of the Royal Statistical Society B, 64(4):827–836, 2002.
[ADH10] C. Andrieu, A. Doucet, and R. Holenstein. Particle Markov chain Monte Carlo. Journal of the Royal
Statistical Society B, 72(3):269–342, 2010.
[CJP13] N. Chopin, P. Jacob, and O. Papaspiliopoulos. SMC2 . Journal of the Royal Statistical Society B,
75(3):397–426, 2013.
[CL00] R. Chen and J. S. Liu. Mixture Kalman filters. Journal of the Royal Statistical Society B,
62(3):493–508, 2000.
[CSOL11] T. Chen, T. Schön, H. Ohlsson, and L. Ljung. Decentralized particle filter with arbitrary state
decomposition. IEEE Transactions on Signal Processing, 59(2):465–478, February 2011.
[DBS06] A. Doucet, M. Briers, and S. Sénécal. Efficient block sampling strategies for sequential Monte Carlo
methods. Journal of Computational and Graphical Statistics, 15(3):693–711, 2006.
[GSS93] N. J. Gordon, S. J. Salmond, and A. F. M. Smith. Novel approach to nonlinear/non-Gaussian Bayesian
state estimation. IEE Proceedings-F, 140(2):107–113, April 1993.
[JD14] A. M. Johansen and A. Doucet. Hierarchical particle sampling for intractable state-space models.
CRiSM working paper, University of Warwick, 2014. In preparation.
[Joh09] A. M. Johansen. SMCTC: Sequential Monte Carlo in C++. Journal of Statistical Software, 30(6):1–41,
April 2009.
[JWD12] A. M. Johansen, N. Whiteley, and A. Doucet. Exact approximation of Rao-Blackwellised particle filters.
In Proceedings of 16th IFAC Symposium on Systems Identification, Brussels, Belgium, July 2012. IFAC,
IFAC.
[Zho13] Y. Zhou. vSMC: Parallel sequential Monte Carlo in C++. Mathematics e-print 1306.5583, ArXiv, 2013.
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters
Key Identity
M
ψn,L
(an−L+2:n , xn−L+1:n , k; xn−L )
k
M (ak
p(xn−L+1:n |xn−L , yn−L+1:n )ψen,L
n−L+2:n , xn−L+1:n , k; xn−L ||. . . )
"
#
k
k
n
Q
bn,n−L+1
bn,p bn
n,p−1
k
r(k|wn )
q xn−L+1 |xn−L
r bn,p |wp−1 q xp |xp−1
=
p=n−L+2
p(xn−L+1:n |xn−L , yn−L+1:n )
=Zbn−L+1:n /p(yn−L+1:n |xn−L )
Monte Carlo Approximationof Monte Carlo Filters
Adam M. Johansen
,
Introduction
Approximating the RBPF
Block Sampling Particle Filters