Monte Carlo Approximation of Monte Carlo Filters
Monte Carlo Approximation
of Monte Carlo Filters
Adam M. Johansen? , Arnaud Doucet‡ and Nick Whiteley†
? University
of Warwick Department of Statistics,
of Bristol School of Mathematics,
‡ University of Oxford Department of Statistics
† University
a.m.johansen@warwick.ac.uk
http://go.warwick.ac.uk/amjohansen/talks
January 8th, 2013
1
Monte Carlo Approximation of Monte Carlo Filters
Background
Outline
Context & Outline
Filtering in State-Space Models:
I
SIR Particle Filters [GSS93]
I
Rao-Blackwellized Particle Filters [AD02, CL00]
I
Block-Sampling Particle Filters [DBS06]
Exact Approximation of Monte Carlo Algorithms:
I
Particle MCMC [ADH10]
Approximating the RBPF
I
Approximated Rao-Blackwellized Particle Filters [CSOL11]
I
Exactly-approximated RBPFs [JWD12]
Approximating the BSPF
I
Local SMC [JD13]
2
Monte Carlo Approximation of Monte Carlo Filters
Background
Hidden Markov Models
The Structure of the Problem
30
25
20
sy/km
15
10
5
0
−5
−10
0
5
10
15
20
25
30
35
sx/km
3
Monte Carlo Approximation of Monte Carlo Filters
Background
Hidden Markov Models
Hidden Markov Models / State Space Models
x1
x2
x3
x4
x5
x6
y1
y2
y3
y4
y5
y6
I
Unobserved Markov chain {Xn } transition f .
I
Observed process {Yn } conditional density g .
I
Density:
p(x1:n , y1:n ) = f1 (x1 )g (y1 |x1 )
n
Y
f (xi |xi−1 )g (yi |xi ).
i=2
4
Monte Carlo Approximation of Monte Carlo Filters
Background
Hidden Markov Models
Motivating Examples
I
Tracking, e.g. ACV Model:
I
I
I
I
States: xn = [snx unx sny uny ]T
Dynamics: xn = Axn−1 + n
Observation: yn = Bxn + νn
Stochastic Volatility, e.g.:
I
I
I
States: xn is latent volatility.
Dynamics: f (xi |xi−1 ) = N φxi−1 , σ 2
Observations: g (yi |xi ) = N 0, β 2 exp(xi )
5
Monte Carlo Approximation of Monte Carlo Filters
Background
Hidden Markov Models
Solutions
I
Formally:
p(x1:n |y1:n ) = R
I
p(x1:n−1 |y1:n−1 )f (xn |xn−1 )g (yn |xn )
0
0
g (yn |xn0 )f (xn0 |xn−1 )p(xn−1
|y1:n−1 )dxn−1:n
Importance Sampling in The HMM Setting
I
I
I
Given p(x1:n |y1:n ) for n = 1, 2, . . . .
Choose qn (x1:n ) = qn (xn |x1:n−1 )qn−1 (x1:n−1 ).
Weight:
wn (x1:n ) ∝
p(x1:n |y1:n )
f (xn |xn−1 )g (yn |xn )
∝ wn−1 (x1:n−1 )
qn (x1:n )
qn (xn |xn−1 )
6
Monte Carlo Approximation of Monte Carlo Filters
Background
Hidden Markov Models
Sequential Importance Sampling – Prediction & Update
I
A first “particle filter”:
I
I
Simple default: qn (xn |xn−1 ) = f (xn |xn−1 ).
Importance weighting becomes:
wn (x1:n ) = wn−1 (x1:n−1 ) × g (yn |xn )
I
Algorithmically, at iteration n:
I
i
i
Given {Wn−1
, X1:n−1
} for i = 1, . . . , N:
I
I
i
Sample Xni ∼ f (·|Xn−1
) (prediction)
i
i
Weight Wn ∝ Wn−1 g (yn |Xni ) (update)
7
Monte Carlo Approximation of Monte Carlo Filters
Background
Hidden Markov Models
Resampling
I
Simplest approach:
iid
ei ∼
X
n
n
X
Algorithmically, at iteration n:
Wnj δX j
j=1
I
I
I
i
i
Given {Wn−1
, X1:n−1
}N
i=1 :
ei
Resample → { 1 , X
}.
I
For i = 1, . . . , N:
I
n
Replace {Wni , Xni }N
i=1
e i }N .
with { N1 , X
n i=1
Lower variance
options preferable.
N
1:n−1
I
ei )
Sample Xni ∼ qn (·|X
n−1
I
Weight Wni ∝
e i )g (yn |X i )
f (Xni |X
n−1
n
ei )
qn (X i |X
n
n−1
8
Monte Carlo Approximation of Monte Carlo Filters
Background
Hidden Markov Models
Iteration 2
8
7
6
5
4
3
2
1
0
−1
−2
1
2
3
4
5
6
7
8
9
10
9
Monte Carlo Approximation of Monte Carlo Filters
Background
Hidden Markov Models
Iteration 3
8
7
6
5
4
3
2
1
0
−1
−2
1
2
3
4
5
6
7
8
9
10
10
Monte Carlo Approximation of Monte Carlo Filters
Background
Hidden Markov Models
Iteration 4
8
7
6
5
4
3
2
1
0
−1
−2
1
2
3
4
5
6
7
8
9
10
11
Monte Carlo Approximation of Monte Carlo Filters
Background
Hidden Markov Models
Iteration 5
8
7
6
5
4
3
2
1
0
−1
−2
1
2
3
4
5
6
7
8
9
10
12
Monte Carlo Approximation of Monte Carlo Filters
Background
Hidden Markov Models
Iteration 6
8
7
6
5
4
3
2
1
0
−1
−2
1
2
3
4
5
6
7
8
9
10
13
Monte Carlo Approximation of Monte Carlo Filters
Background
Hidden Markov Models
Iteration 7
8
7
6
5
4
3
2
1
0
−1
−2
1
2
3
4
5
6
7
8
9
10
14
Monte Carlo Approximation of Monte Carlo Filters
Background
Hidden Markov Models
Iteration 8
8
7
6
5
4
3
2
1
0
−1
−2
1
2
3
4
5
6
7
8
9
10
15
Monte Carlo Approximation of Monte Carlo Filters
Background
Hidden Markov Models
Iteration 9
8
7
6
5
4
3
2
1
0
−1
−2
1
2
3
4
5
6
7
8
9
10
16
Monte Carlo Approximation of Monte Carlo Filters
Background
Hidden Markov Models
Iteration 10
8
7
6
5
4
3
2
1
0
−1
−2
1
2
3
4
5
6
7
8
9
10
17
Monte Carlo Approximation of Monte Carlo Filters
Background
Hidden Markov Models
Rao-Blackwellised Particle Filters
18
Monte Carlo Approximation of Monte Carlo Filters
Approximating the RBPF
A Class of Hidden Markov Models
A (Rather Broad) Class of Hidden Markov Models
y1
x1
z1
x2
z2
y2
x3
z3
y3
I
I
I
Unobserved Markov chain {(Xn , Zn )} transition f .
Observed process {Yn } conditional density g .
Density:
p(x1:n , z1:n , y1:n ) = f1 (x1 , z1 )g (y1 |x1 , z1 )
n
Y
f (xi , zi |xi−1 , zi−1 )g (yi |xi , zi ).
i=2
19
Monte Carlo Approximation of Monte Carlo Filters
Approximating the RBPF
A Class of Hidden Markov Models
Simple Solutions
I
Formally:
p((x, z)1:n |y1:n ) ∝ p((x, z)1:n−1 |y1:n−1 )f ((x, z)n |(x, z)n−1 )g (yn |(x, z)n )
I
An SIR Filter: Algorithmically, at iteration n:
I
I
i
Given {Wn−1
, (X , Z )i1:n−1 } for i = 1, . . . , N:
e, Z
e )i
Resample, obtaining { N1 , (X
1:n−1 }.
I
e, Z
e )in−1 )
Sample (X , Z )in ∼ qn (·|(X
I
Weight
Wni ∝
e ,Z
e )i )g (yn |(X ,Z )i )
f ((X ,Z )in |(X
n
n−1
e ,Z
e )i )
qn ((X ,Z )i |(X
n
n−1
20
Monte Carlo Approximation of Monte Carlo Filters
Approximating the RBPF
A Class of Hidden Markov Models
A Rao-Blackwellized SIR Filter
Algorithmically, at iteration n:
I
X ,i
i
i
Given {Wn−1
, (X1:n−1
, p(z1:n−1 |X1:n−1
, y1:n−1 )}
ei
ei
, p(z1:n−1 |X
Resample, obtaining { 1 , (X
I
For i = 1, . . . , N:
I
1:n−1
N
I
I
1:n−1 , y1:n−1 ))}.
ei )
Sample Xni ∼ qn (·|X
n−1
i
i
ei
Set X1:n
← (X
1:n−1 , Xn ).
ei )
p(Xni ,yn |X
n−1
ei )
qn (X i |X
I
Weight WnX ,i ∝
I
i
Compute p(z1:n |y1:n , X1:n
).
n
n−1
Requires analytically tractable substructure.
21
Monte Carlo Approximation of Monte Carlo Filters
Approximating the RBPF
Exact Approximation of the RBPF
An Approximate Rao-Blackwellized SIR Filter
Algorithmically, at iteration n:
I
I
I
X ,i
i
i
Given {Wn−1
, (X1:n−1
,b
p (z1:n−1 |X1:n−1
, y1:n−1 )}
ei
ei
Resample, obtaining { 1 , (X
,b
p (z1:n−1 |X
I
I
I
1:n−1
N
1:n−1 , y1:n−1 ))}.
For i = 1, . . . , N:
ei )
Sample Xni ∼ qn (·|X
n−1
i
i
ei
Set X1:n
← (X
1:n−1 , Xn ).
ei )
b
p (Xni ,yn |X
n−1
ei )
qn (X i |X
I
Weight WnX ,i ∝
I
i
Compute b
p (z1:n |y1:n , X1:n
).
n
n−1
Our proposal: use N “local” M-particle filters to provide
i ) and b
e i ).
b
p (z1:n |y1:n , X1:n
p (Xni , yn |X
n−1
Is approximate; how does error accumulate?
22
Monte Carlo Approximation of Monte Carlo Filters
Approximating the RBPF
Exact Approximation of the RBPF
How can this be justified?
I
As an extended space SIR algorithm.
I
Via unbiased estimation arguments.
Note also the M = 1 and M → ∞ cases.
How does this differ from CSOL11?
Principally in the local weights, benefits including:
I
Valid (N-consistent) for all M ≥ 1 rather than
(M, N)-consistent.
I
Computational cost O(MN) rather than O(M 2 N).
I
Only requires knowledge of joint behaviour of x or z; doesn’t
require say p(xn |xn−1 , zn−1 ).
23
Monte Carlo Approximation of Monte Carlo Filters
Approximating the RBPF
Exact Approximation of the RBPF
Toy Example: Model
We use a simulated sequence of 100 observations from the model
defined by the densities:
x1
0
1 0
µ(x1 , z1 ) =N
;
z1
0
0 1
xn
xn−1
1 0
f (xn , zn |xn−1 , zn−1 ) =N
;
,
zn
zn−1
0 1
2
xn
σx 0
g (yn |xn , zn ) =N yn ;
,
zn
0 σz2
Consider IMSE (relative to optimal filter) of filtering estimate of
first coordinate marginals.
24
Monte Carlo Approximation of Monte Carlo Filters
Approximating the RBPF
Exact Approximation of the RBPF
Mean Squared Filtering Error
Approximation of the RBPF
N = 10
−1
10
N = 20
N = 40
N = 80
−2
10
N = 160
0
10
1
10
Number of Lower−Level Particles, M
2
10
For σx2 = σz2 = 1.
25
Monte Carlo Approximation of Monte Carlo Filters
Approximating the RBPF
Exact Approximation of the RBPF
Computational Performance
0
Mean Squared Filtering Error
10
N = 10
N = 20
N = 40
N = 80
N = 160
−1
10
−2
10
−3
10
1
10
2
10
3
10
Computational Cost, N(M+1)
4
10
5
10
For σx2 = σz2 = 1.
26
Monte Carlo Approximation of Monte Carlo Filters
Approximating the RBPF
Exact Approximation of the RBPF
Computational Performance
N = 10
N = 20
N = 40
N = 80
N = 160
1.9
Mean Squared Filtering Error
10
1.8
10
1.7
10
1.6
10
1
10
2
10
3
10
Computational Cost, N(M+1)
4
10
5
10
For σx2 = 102 and σz2 = 0.12 .
27
Monte Carlo Approximation of Monte Carlo Filters
Approximating the RBPF
Exact Approximation of the RBPF
Local Particle Filters
28
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Model
What About Other HMMs / Algorithms?
Returning to:
x1
x2
x3
x4
x5
x6
y1
y2
y3
y4
y5
y6
I
Unobserved Markov chain {Xn } transition f .
I
Observed process {Yn } conditional density g .
Density:
I
p(x1:n , y1:n ) = f1 (x1 )g (y1 |x1 )
n
Y
f (xi |xi−1 )g (yi |zi ).
i=2
29
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Idealised Algorithms
Block Sampling: An Idealised Approach
At time n, given x1:n−1 ; discard xn−L+1:n−1 :
I
Sample from q(xn−L+1:n |xn−L , yn−L+1:n ).
I
Weight with
W (x1:n ) =
I
p(x1:n |y1:n )
p(x1:n−L |y1:n−1 )q(xn−L+1:n |xn−L , y1:n−L+1:n )
Optimally,
q(xn−L+1:n |xn−L , yn−L+1:n ) =p(xn−L+1:n |xn−L , yn−L+1:n )
p(x1:n−L |y1:n )
W (x1:n ) ∝
=p(yn |x1:n−L , yn−L+1:n−1 )
p(x1:n−L |y1:n−1 )
I
Typically intractable; auxiliary variable approach in [DBS06].
30
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Motivation
Motivating Example
I
Model:
f (xi |xi−1 ) =N φxi−1 , σ 2
g (yi |xi ) =N 0, β 2 exp(xi )
I
Simple Bootstrap SIR Algorithm:
I
Proposal:
q(xt |xt−1 , yt ) = f (xt |xt−1 )
I
Weighting:
W (xt−1 , xt ) ∝
I
I
f (xt |xt−1 )g (yt |xt )
= g (yt |xt )
f (xt |xt−1 )
Resample residually every iteration.
Is hierarchical SMC possible here?
31
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Motivation
Local Particle Filtering: First Particle
4
3
2
1
0
−1
−2
−3
−4
0
2
4
6
8
10
12
14
16
32
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Motivation
Local Particle Filtering: SMC Proposal
4
3
2
1
0
−1
−2
−3
−4
0
2
4
6
8
10
12
14
16
33
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Motivation
Local Particle Filtering: CSMC Auxiliary Proposal
4
3
2
1
0
−1
−2
−3
−4
0
2
4
6
8
10
12
14
16
34
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Exact Approximation of BSPFs / Local Particle Filtering
The Justification of Local SMC
I
Propose from:
⊗n−1
M
U1:M
(b1:n−2 , k)p(x1:n−1 |y1:n−1 )ψn,L
(an−L+2:n , xn−L+1:n , k; xn−L )
k
k
M
ψen−1,L−1 e
an−L+2:n−1 , e
xn−L+1:n−1 ; xn−L ||bn−L+2:n−1 , xn−L+1:n−1
I
Target:
b
k̄
n,n−L+1:n
⊗n
k̄
U1:M
(b1:n−L , b̄n,n−L+1:n−1
|y1:n )
, k̄)p(x1:n−L , x n−L+1:n
k̄
b n,n−L+1:n
k
k
k̄
M
e
ψn,L an−L+2:n , xn−L+1:n ; xn−L b̄n,n−L+1:n , x n−L+1:n
M
ψn−1,L−1
(e
an−L+2:n−1 , e
xn−L+1:n−1 , k; xn−L ) .
I
en−L+1:n−1 .
Weight: Z̄n−L+1:n /Z
35
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Stochastic Volatility Model
Stochastic Volatility Bootstrap Local SMC
I
Model:
f (xi |xi−1 ) =N φxi−1 , σ 2
g (yi |xi ) =N 0, β 2 exp(xi )
I
Top Level:
I
I
I
Local SMC proposal.
Stratified resampling when ESS< N/2.
Local SMC Proposal:
I
Proposal:
q(xt |xt−1 , yt ) = f (xt |xt−1 )
I
Weighting:
W (xt−1 , xt ) ∝
I
f (xt |xt−1 )g (yt |xt )
= g (yt |xt )
f (xt |xt−1 )
Resample residually every iteration.
36
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Stochastic Volatility Model
SV Simulated Data
Simulated Data
4
3
Observed Values
2
1
0
−1
−2
−3
0
100
200
300
400
500
600
700
800
900
n
37
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Stochastic Volatility Model
SV Bootstrap Local SMC: M=100
N = 100, M = 100
100
Average Number of Unique Values
90
80
70
L=2
L=4
L=6
L = 10
60
50
40
30
20
10
0
0
100
200
300
400
500
600
700
800
900
n
38
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Stochastic Volatility Model
SV Bootstrap Local SMC: M=1000
N = 100, M = 1000
100
Average Number of Unique Values
90
80
70
60
L=2
L=4
L=6
L = 10
L = 14
L = 18
50
40
30
20
10
0
0
100
200
300
400
500
600
700
800
900
n
39
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Stochastic Volatility Model
SV Bootstrap Local SMC: M=10000
N = 100, M = 10000
100
Average Number of Unique Values
90
80
70
60
50
L=2
L=4
L=6
L = 10
L = 14
L = 18
L = 22
L = 26
40
30
20
10
0
0
100
200
300
400
500
600
700
800
900
n
40
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Stochastic Volatility Model
SV Exchange Rata Data
Exchange Rate Data
5
4
Observed Values
3
2
1
0
−1
−2
−3
−4
0
100
200
300
400
500
600
700
800
900
n
41
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Stochastic Volatility Model
SV Bootstrap Local SMC: M=100
N = 100, M = 100
100
Average Number of Unique Values
90
80
70
L=2
L=4
L=6
L = 10
60
50
40
30
20
10
0
0
100
200
300
400
500
600
700
800
900
n
42
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Stochastic Volatility Model
SV Bootstrap Local SMC: M=1000
N = 100, M = 1000
100
Average Number of Unique Values
90
80
70
60
L=2
L=4
L=6
L = 10
L = 14
L = 18
50
40
30
20
10
0
0
100
200
300
400
500
600
700
800
900
n
43
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Stochastic Volatility Model
SV Bootstrap Local SMC: M=10000
N=100, M=10,000
100
Average Number of Unique Values
90
80
70
60
50
L=2
L=4
L=6
L = 10
L = 14
L = 18
L = 22
L = 26
40
30
20
10
0
0
100
200
300
400
500
600
700
800
900
n
44
Monte Carlo Approximation of Monte Carlo Filters
Block Sampling Particle Filters
Stochastic Volatility Model
SV Exchange Rata Data
Exchange Rate Data
5
4
Observed Values
3
2
1
0
−1
−2
−3
−4
0
100
200
300
400
500
600
700
800
900
n
45
Monte Carlo Approximation of Monte Carlo Filters
Summary
In Conclusion
I
I
I
SMC can be used hierarchically.
Software implementation is not difficult [Joh09].
The Rao-Blackwellized particle filter can be approximated
exactly
I
I
I
I
I
Can reduce estimator variance at fixed cost.
Attractive for distributed/parallel implementation.
Allows combination of different sorts of particle filter.
Can be combined with other techniques for parameter
estimation etc..
The optimal block-sampling particle filter can be
approximated exactly
I
I
I
Requiring only simulation from the transition and evaluation of
the likelihood
Easy to parallelise
Low storage cost
46
Monte Carlo Approximation of Monte Carlo Filters
Summary
References I
C. Andrieu and A. Doucet. Particle filtering for partially observed Gaussian state space models. Journal of
the Royal Statistical Society B, 64(4):827–836, 2002.
C. Andrieu, A. Doucet, and R. Holenstein. Particle Markov chain Monte Carlo. Journal of the Royal
Statistical Society B, 72(3):269–342, 2010.
R. Chen and J. S. Liu. Mixture Kalman filters. Journal of the Royal Statistical Society B, 62(3):493–508,
2000.
T. Chen, T. Schön, H. Ohlsson, and L. Ljung. Decentralized particle filter with arbitrary state
decomposition. IEEE Transactions on Signal Processing, 59(2):465–478, February 2011.
A. Doucet, M. Briers, and S. Sénécal. Efficient block sampling strategies for sequential Monte Carlo
methods. Journal of Computational and Graphical Statistics, 15(3):693–711, 2006.
N. J. Gordon, S. J. Salmond, and A. F. M. Smith. Novel approach to nonlinear/non-Gaussian Bayesian state
estimation. IEE Proceedings-F, 140(2):107–113, April 1993.
A. M. Johansen and A. Doucet. Hierarchical particle sampling for intractable state-space models. CRiSM
working paper, University of Warwick, 2013. In preparation.
A. M. Johansen. SMCTC: Sequential Monte Carlo in C++. Journal of Statistical Software, 30(6):1–41,
April 2009.
A. M. Johansen, N. Whiteley, and A. Doucet. Exact approximation of Rao-Blackwellised particle filters. In
Proceedings of 16th IFAC Symposium on Systems Identification. IFAC, 2012.
47