Supplement to “Variance estimation and allocation in the
particle filter”
Anthony Lee∗ and Nick Whiteley†
S1. Proof of Theorem 1
We define
Qp (xp−1 , dxp )
:= Gp−1 (xp−1 )Mp (xp−1 , dxp ),
Qp,n := Qp+1 · · · Qn ,
Qn,n := Id,
p≥1
0 ≤ p < n.
The following Lemma will be put to multiple uses in the proof of Theorem 1.
Lemma 3. Assume (C). Fix n ≥ 0 and ϕ ∈ L(X ). Then, for any r ≥ 1,
h
√
r i1/r
sup N E γnN (ϕ) − γn (ϕ)
< ∞,
N ≥1
√
sup
N ≥1
h
r i1/r
< ∞.
N E ηnN (ϕ) − ηn (ϕ)
Proof. Consider the decomposition:
γnN (ϕ) − γn (ϕ)
=
=
n
X
p=0
n
X
N
γpN Qp,n (ϕ) − γp−1
Qp−1,n (ϕ)
γpN (1)
p=0
1 X i
∆p,n ,
Np
i∈[Np ]
N
with the convention γ−1
Q−1,n (ϕ) = M0 Q0,n (ϕ) in the first equality, and
∆i0,n := Q0,n (ϕ)(ζpi ) − M0 Q0,n (ϕ),
∆ip,n = Qp,n (ϕ)(ζpi ) −
N
ηp−1
Qp−1,n (ϕ)
,
N
ηp−1 (Gp−1 )
1 ≤ p ≤ n.
Note that (∆i0,n )i∈[N0 ] are i.i.d. and zero-mean variables, and for each p ≥ 1, given σ(ζ0:p−1 ), the (∆ip,n )i∈[Np ] are
conditionally i.i.d. and zero-mean random
variables. Moreover, under (H), there exists a finite constant say Cn such
that supN ≥1 max1≤p≤n maxi∈[Np ] ∆ip,n < Cn and supN ≥1 max1≤p≤n γpN (1) < Cn . Applying these observations
together with the Minkowski and Burkholder–Davis–Gundy inequalities, there exists a finite constant Bn,r such
that
h
√
r i1/r
sup N E γnN (ϕ) − γn (ϕ)
N ≥1
r 1/r

X
N
1
i 

≤ sup N
E γp (1)
∆p,n Np
N ≥1
p=0
i∈[Np ]
r 1/r

s
n
√ X
1  X
i
2
E ≤ Bn,r sup N
(∆p,n ) 
< ∞.
N
p
N ≥1
i∈[Np ]
p=0
√
n
X
∗ Department
† School
of Statistics, University of Warwick
of Mathematics, University of Bristol
1
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
2
Applying this bound and Minkowski’s inequality to the decomposition
γ N (ϕ) γn (1) − γnN (1)
γ N (ϕ) − γn (ϕ)
ηnN (ϕ) − ηn (ϕ) = nN
+ n
γn (1)
γn (1)
γn (1)
gives
√
sup
N ≥1
h
r i1/r
N E ηnN (ϕ) − ηn (ϕ)
h
h
√
√
r i1/r
r i1/r
supx |ϕ(x)|
1
sup N E γnN (1) − γn (1)
sup N E γnN (ϕ) − γn (ϕ)
+
γn (1) N ≥1
γn (1) N ≥1
< ∞.
≤
Proof of Theorem 1. For part 1., the equality E[γnN (ϕ)] = γnN (ϕ) is established by Corollary 2. The expression for
E[γnN (ϕ)2 ] in part 1. follows from Corollary 1 upon taking mp,n (ϕ) := µbp (ϕ)/γn (1)2 with bp containing a single
“1” at location p, and noting that µ0 (ϕ) = γn (ϕ)2 .
Lemma 3 establishes the moment bounds in parts 2. and 3. To verify the expression for σγ2N in part 2., combine
n
the equality E[γnN (ϕ)] = γnN (ϕ) and the expression for E[γnN (ϕ)2 ] to give
γ N (ϕ)
N var n
γn (1)
n
X
N
N Y
1
−1
= −ηn (ϕ)
+ O(N ) +
mp,n (ϕ)
1−
+ O(N −1 )
dN
c
e
dN
c
e
dN
c
e
p
p
q
p=0
p=0
q6=p


n
X
Y
N 
1
=
mp,n (ϕ)
1−
− ηn (ϕ)2  + O(N −1 ).
dN
c
e
dN
c
e
p
q
p=0
2
n
X
q6=p
Taking N → ∞ gives σ̄γ2N , and the expression for σγ2N then follows.
n
n
It remains to verify the expression for ση2N in part 3. For the remainder of the proof, denote ϕ̄ := ϕ − ηn (ϕ).
n
Observe
γ N (1)
γ N (ϕ̄)
= ηnN (ϕ̄) 1 − n
ηnN (ϕ̄) − n
,
γn (1)
γn (1)
and so by Cauchy–Schwarz,
√
"
2 #1/2
N
N
(
ϕ̄)
γ
n
N E ηn (ϕ̄) −
γn (1) "
2 #1/2
N
N
(1)
γ
n
N E ηn (ϕ̄) 1 −
γn (1) "
4 #1/4
h
N
√
4 i1/4
γ
(1)
n
N
N E ηn (ϕ̄)
E 1 −
γn (1) √
=
≤
→ 0, as N → ∞,
(S1)
where the convergence to zero is a consequence
of Lemma 3. Rearranging
Minkowski’s inequality gives for any
h
i1/2 h
i1/2
h
i1/2 2
2
2
, so the convergence in (S1) implies
random variables X, Y , E |X − Y |
≥ E |X|
− E |Y |
"
2 #1/2 h
i
N
1/2
√
√
γ (ϕ̄) N E ηnN (ϕ̄)2
→ 0, as N → ∞,
− N E n
γn (1)
i.e., limN →∞ N E ηnN (ϕ̄)2 = σ̄γ2N (ϕ̄).
n
(S2)
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
3
S2. Algorithms for computing the estimates
i
In Algorithm 3, S0,n [i] = ψ0,n
(ϕ) for each i ∈ [N0 ] upon completion of step 2.
Algorithm 3 Computing VnN (ϕ)
1. Let S0,n be an array of length N0 , initialized to 0.
j
j
j
2. For j ∈ [Nn ], set S0,n [En
] ← S0,n [En
] + ϕ(ζn
)/Nn .
3. Set

mN
? (ϕ)
←
n
Y
p=0
4. Set
VnN
←
N (ϕ)2
ηn
−


X
Np   N
2
2
ηn (ϕ) −
S0,n [i]
.
Np − 1
i∈[N0 ]
mN
? (ϕ).
i
In Algorithm 4, Sp,n [i] = ψp,n
(ϕ) for each i ∈ [Np ] and each p ∈ {0, . . . , n} upon completion of step 2. Upon
completion of step 3, g0,p [i] = Ḡi0,p for each i ∈ [N0 ] and p ∈ {0, . . . , n − 1}. Finally, upon completion of step 6(b),
2
P
P
(1)
(2)
i
i
Rp [i] = j∈Opi ψp+1,n
and Rp [i] = j∈Opi ψp+1,n
for each i ∈ [Np ] and p ∈ {0, . . . , n − 1}.
N
(ϕ), and vnN (ϕ)
Algorithm 4 Computing each vp,n
1. Let Sn,n be an array of length Nn , such that Sn,n [i] =
1
i ).
ϕ(ζn
Nn
2. For each p = n − 1, . . . , 0:
(a) Let Sp,n be an array of length Np , initialized to 0. For j ∈ [Np+1 ], set Sp,n [Ajp ] ← Sp,n [Ajp ] + Sp+1,n [j].
3. For each p ∈ {0, . . . , n − 1}:
(a) Let tp be an array of length Np such that tp [i] = Gp (ζpi )/
P
j∈[Np ]
Gp (ζpj ).
(b) Let g0,p be an array of length N0 , initialized to 0. For j ∈ [Np ], set g0,p [Epj ] ← g0,p [Epj ] + tp [j].
Q
P
Np
n
N (ϕ)2 −
2 .
ηn
4. Set mN
? (ϕ) ←
p=0 Np −1
i∈[N0 ] S0,n [i]
iP
hQ
Nq
n
2 1−g
i
5. Set mN
0,n−1 [En ] .
n,n (ϕ) ← (Nn − 1)
i∈[Nn ] Sn,n [i]
q=0 N −1
q
6. For p ∈ {0, . . . , n − 1},
(1)
(a) Let Rp
(2)
and Rp
be arrays of length Np , initialized to 0.
(1)
(1)
N (ϕ)
vp,n
mN
p,n (ϕ)
(2)
(2)
(b) For j ∈ [Np+1 ], set Rp [Ajp ] ← Rp [Ajp ] + Sp+1,n [j] and Rp [Ajp ] ← Rp [Ajp ] + Sp+1,n [j]2 .
hQ
iP
Nq
(1) 2
(2)
n
1 − g0,p−1 [Epi ] ;
(c) If p ≥ 1, set mN
p,n (ϕ) ← (Np − 1)
i∈[Np ] Rp [i] − Rp [i]
q=0 Nq −1
hQ
iP
Nq
(1) 2
(2)
n
otherwise set mN
0,n (ϕ) ← (N0 − 1)
i∈[N0 ] R0 [i] − R0 [i] .
q=0 N −1
q
7. For p ∈ {0, . . . , n}, set
←
−
mN
? (ϕ).
N (ϕ) ←
Set vn
−1 N
p=0 cp vp,n (ϕ).
Pn
S3. Proofs for Section 3
S3.1. Conditional particle filters and lack of bias
We define M0 (dz0 ) :=
Q
i∈[N0 ]
M0 (dz0i ), and
i
i
ap−1
ap−1
Y Gp−1 (zp−1
)Mp (zp−1
, zpi )
Mp (zp−1 ; ap−1 , dzp ) :=
,
P
j
G
(z
)
p−1
p−1
j∈[N
]
p−1
i∈[N ]
p
p ≥ 1.
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
4
We can then describe the probabilistic evolution of the particle system by
P(a, dz) := M0 (dz0 )
n
Y
Mp (zp−1 ; ap−1 , dzp ).
p=1
We also define Gp (zp ) :=
1
Np
P
i∈[Np ]
Gp (zpi ) for p ∈ {0, . . . , n}. We define Q1 by
Q1 (k, a, dz) := P(a, dz)C1 (a, z; k),
which essentially defines the probability measure associated with the random variables (K1 , A, ζ) under P.
We now introduce the conditional particle filter construction of Andrieu, Doucet and Holenstein (2010). Let −kp
−k
denote the set [Np ] \ {kp } and k := k0:n . We define zp p := (zp1 , . . . , zpk−1 , zpk+1 , . . . , zpN ), zk := (z0k0 , . . . , znkn ) and
z−k := (z0−k0 , . . . , zn−kn ). We define a variant of Mp in which one ancestor index and particle is excluded
ai
−k
Mkpp (zp−1 ; ap−1p , dzp−kp )
Y
:=
i∈[Np ]\{kp
ai
p−1
p−1
Gp−1 (zp−1
)Mp (zp−1
, dzpi )
,
P
j
G
(z
)
p−1
p−1
j∈[N
]
p−1
}
with Mk00 (dz0−k0 ) := i∈[N0 ]\{k0 } M0 (dz0i ). With a fixed reference path zk in position k, we define the conditional
particle filter to be a Markov kernel defined by
#
" n
Y
k
−k
p
P̄1 (k, zk ; a, dz−k ) := Mk00 (dz0−k0 )
.
Mkpp (zp−1 ; ap−1p , dzp−kp )I kp−1 = ap−1
Q
p=1
This specifies a particular distribution for the particle system excluding zk , and the ancestor indices conditional
upon k and zk . We also define the Feynman–Kac measure on the path space
ˆ
γ n (A) :=
M0 (dx0 )
A
n
Y
Gp−1 (xp−1 )Mp (xp−1 , dxp ),
A ∈ X ⊗n+1 .
p=1
Finally, we define Q̄1 by
Q̄1 (k, a, dz) :=
k
I {k ∈ [N]} γ n (dz )
P̄1 (k, zk ; dz−k , a).
|[N]|
γn (1)
This essentially defines the probability measure associated with an alternative distribution for (K1 , A, ζ), where K1
1
1
1
is first sampled uniformly from [N], then ζ K ∼ γ n (·)/γn (1) and finally (A, ζ −K ) ∼ P̄1 (K1 , ζ K ; ·). We denote by
Ē1 expectations w.r.t. the law of this alternative process.
Lemma 4. For any ϕ ∈ L(X ),
h
i
K1
E γnN (1)ϕ(ζn n ) = γn (ϕ).
Proof of Lemma 4. It suffices to show that
"n−1
#
Y
Gp (zp ) Q1 (k, a, dz) = γn (1)Q̄1 (k, a, dz),
p=0
since it then follows that
We observe that
h
i
h
i
K1
K1
E γnN (1)ϕ(ζn n ) = Ē1 γn (1)f (ζn n ) = γn (f ).
ak
ak
p−1
p−1
Gp−1 (zp−1
)Mp (zp−1
, zpk ) k
−k
Mp (zp−1 ; ap−1 , dzp ) = P
Mp (zp−1 ; a−k
p−1 , dzp ).
j
G
(z
)
j∈[Np−1 ] p−1 p−1
(S3)
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
5
Hence,
"n−1
Y
#
Gp (zp ) Q1 (k, a, dz) =
"n−1
Y
p=0
#
Gp (zp ) P(a, dz)C1 (a, z; k)
p=0
"n−1
Y
#
n
Y
=
1
Nn
=
"n−1
#"
#
n
Y
1 Y
kp
−k0
k0
kp
−kp
Mp (zp−1 ; ap−1 , dzp )I kp−1 = ap−1
Gp (zp ) M0 (dz0 )
Nn p=0
p=1
Gp (zp ) M0 (dz0 )
p=0
p=1
·M0 (dz0k0 )
=
kp
Mp (zp−1 ; ap−1 , dzp )I kp−1 = ap−1
kp−1
kp−1
k
n
Y
Gp−1 (zp−1
)Mp (zp−1
, dzp p )
Np−1 Gp−1 (zp )
p=1
#
"
n
Y
1
kp
−k0
k0
kp
−kp
Mp (zp−1 ; ap−1 , dzp )I kp−1 = ap−1
M0 (dz0 )
|[N]|
p=1
·M0 (dz0k0 )
n
Y
k
k
p−1
p−1
Gp−1 (zp−1
)Mp (zp−1
, dzpkp )
p=1
1
P̄1 (k, zk ; a, dz−k )γ n (dz k )
|[N]|
= γn (1)Q̄1 (k, a, dz).
Corollary 2. For any ϕ ∈ L(X ), E γnN (ϕ) = γn (ϕ).
=
Proof. We obtain


E γnN (ϕ) = E γnN (1)
1
Nn
X
h
i
K1
ϕ(ζni ) = E γnN (1)ϕ(ζn n ) = γn (ϕ).
i∈[Nn ]
S3.2. Doubly conditional particle filters and proof of Lemma 1
The structure of the proof of Lemma 1 is very similar to the structure of the proof of Lemma 4. We define
Q2 (k1:2 , a, dz) := P(a, dz)C1,2 (a, z; k1:2 ),
which essentially defines the probability measure associated with the random variables (K1:2 , A, ζ) under P. We
Q
k1:2
−k1:2
define M00 (dz0 0 ) := i∈[N0 ]\{k1:2 } M0 (dz0i ),
0
ai
k1:2
−k1:2
−k1:2
Mpp (zp−1 ; ap−1p , dzp p )
ai
p−1
p−1
Gp−1 (zp−1
)Mp (zp−1
, dzpi )
,
:=
PN
j
j∈[Np−1 ] Gp−1 (zp−1 )
i∈[Np ]\{k1:2 }
Y
p
and the Markov kernel associated with a doubly conditional particle filter as
1:2
1:2
P̄2 (k1:2 , zk ; a, dz−k )
#
" n
Y k1:2
1:2
1:2
−kp
−kp
k01:2
−k01:2
p
)
Mp (zp−1 ; ap−1 , dzp
)
:= M0 (dz0
p=1
·
n
Y
p=1
1
I(kp−1
=
1
kp
ap−1
)
2
kp
2
1 2
2
1
I kp 6= kp , kp−1 = ap−1 + I kp = kp .
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
6
A doubly conditional particle filter was also used in Andrieu, Lee and Vihola (2013), but for a different purpose.
We also define the path space counterpart for each µb , b ∈ {0, 1}n+1 , by
ˆ
µb (A) :=
n
Y
M̃0b0 (dx1:2
0 )
A
A ∈ X ⊗2n+2 ,
bp
1:2
1:2
G̃p−1 (x1:2
p−1 )M̃p (xp−1 , dxp ),
p=1
Finally, we define Q̄2 by
1:2
Q̄2 (k
k1:2
I k1:2 ∈ [N]2 µb(k1:2 ) (dz )
1:2
1:2
P̄2 (k1:2 , zk ; a, dz−k ),
, a, dz) :=
2
|[N]|
µb(k1:2 ) (1)
where b : [N]2 → {0, 1}n+1 maps (k1 , k2 ) to the unique b ∈ {0, 1}n+1 such that (k1 , k2 ) ∈ I(b). Q̄2 essentially
defines the probability measure associated with an alternative distribution for (K1:2 , A, ζ), where K1:2 is first
1:2
1:2
1:2
sampled uniformly from [N]2 , then ζ K ∼ µb(K1:2 ) (·)/µb(K1:2 ) (1) and finally (A, ζ −K ) ∼ P̄2 (K1:2 , ζ K ; ·). We
denote by Ē2 expectations w.r.t. the law of this alternative process.
Proof of Lemma 1. It suffices to show that
"n−1
Y
#2
Gp (zp )
Q2 (k1:2 , a, dz) = µb(k1:2 ) (1)Q̄2 (k1:2 , a, dz),
p=0
since it then follows that
h
i
K1
K2
E II(b) (K1 , K2 )γnN (1)2 ϕ(ζn n , ζn n )
=
=
h
i
K1
K2
Ē2 II(b) (K1 , K2 )µb (1)ϕ(ζn n , ζn n )
2
X
1
µb (ϕ)
|[N]|
1
2
k ,k ∈I(b)
bp 1−bp
n Y
1
1
=
1−
µb (ϕ),
Np
Np
p=0
where the last equality follows from the fact that |I(b)| =
We first note that by application of (S3),
"n−1
Y
#2
Gp (zp )
Q2 (k1:2 , a, dz)
=
p=0
"n−1
Y
Qn
1−bp
p=0
Np (Np − 1)
.
#2
Gp (zp )
Q1 (k1 , a, dz)C2 (a, z, k1 ; k2 )
p=0
=
"n−1
Y
p=0
#
Gp (zp ) γn (1)Q̄1 (k1:2 , a, dz)C2 (a, z, k1 ; k2 ).
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
1
We observe that for any k1 , k2 ∈ [N] and zk ∈ Xn+1 , we have
k1
1
P̄1 (k , z ; a, dz
−k1
n Y
2
kp
2
1 2
2
1
)
I kp 6= kp , kp−1 = ap−1 + I kp = kp
p=1
"
−k01
k01
=
M (dz0
)
n
Y
#
−k1
−k1
(zp−1 ; ap−1p , dzp p )
1
kp
M
p=1
"
k01:2
=
M
−k1:2
(dz0 0 )
n
Y
n
Y
1
I(kp−1
=
1
kp
ap−1
)
2
kp
2
1 2
2
1
I kp 6= kp , kp−1 = ap−1 + I kp = kp
p=1
#
1:2
kp
M
−k1:2
−k1:2
(zp−1 ; ap−1p , dzp p )
p=1

2
2
2
kp−1
kp−1
kp
G
(z
)M
(z
,
dz
)
p
p−1
p
p−1
p−1
I kp2 6= kp1
+ I kp2 = kp1 
·
N
G
(z
)
p−1
p−1
p−1
p=0
·
n
Y

n
Y
1
2
kp
kp
1
2
I(kp−1
= ap−1
) I kp2 6= kp1 , kp−1
= ap−1
+ I kp2 = kp1
p=1
=
1:2
1:2
P̄2 (k1:2 , zk ; a, dz−k

2
2
2
kp−1
kp−1
kp
G
(z
)M
(z
,
dz
)
p
p−1
p
p−1
p−1
I kp2 6= kp1
+ I kp2 = kp1  .
)
N
G
(z
)
p−1
p−1
p−1
p=0
n
Y

It follows that
"n−1
Y
#
Gp (zp ) γn (1)Q̄1 (k1:2 , a, dz)C2 (a, z, k1 ; k2 )
p=0
=
"n−1
Y
p=0
#
Gp (zp )
1
1
1
1
γ n (dzk )P̄1 (k1 , zk ; a, dz−k )C2 (a, z, k1 ; k2 )
|[N]|
#
"n−1
Y
1:2
1:2
1
Gp (zp ) P̄2 (k1:2 , zk ; a, dz−k )
=
|[N]| p=0


2
2
2
kp−1
kp−1
kp
n
G
(z
)M
(z
,
dz
)
1 Y
p
p−1
p
p−1
p−1
I kp2 6= kp1
+ I kp2 = kp1 
·γ n (dzk )
N
G
(z
)
p−1
p−1
p−1
p=0


2
kp−1
n
2
G
(z
)
1 Y
p−1
k
p−1
p
2

I kp2 6= kp1 , kp−1
= ap−1
+ I kp2 = kp1
·
Nn p=1
Np−1 Gp−1 (zp−1 )
#
2 "
n
Y
1:2
1:2
1:2
1:2
1:2
1
kp−1
kp−1
kp
k01:2
b0
bp
=
M̃0 (dx0 )
G̃p−1 (xp−1 )M̃p (xp−1 , dxp ) P̄2 (k1:2 , zk ; a, dz−k )
|[N]|
p=1
2
1:2
1:2
1:2
1
=
µb(k1:2 ) (dzk )P̄2 (k1:2 , zk ; a, dz−k )
|[N]|
=
µb(k1:2 ) (1)Q̄2 (k1:2 , a, dz).
7
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
8
⊗2
⊗2
S3.3. Computationally efficient representations of µN
) and µN
)
0 (ϕ
bp (ϕ
Proof of Proposition 2. Recall the definitions of Ep , FN (p), Ḡ0,p and ψp,n in Section 2. For part 1., we observe that
⊗2
µN
)
0 (ϕ
Np
n
N
2
p=0 Np −1 γn (1)
X
k1
k2
C1,2 (A, ζ; k1:2 )ϕ(ζnn )ϕ(ζnn )
Q
=
k1:2 ∈I(0)
X
=
k1:2 ∈[N]2
" n
#
1
2
I(kn1 6= kn2 ) Y
kp
kp
k1
k2
1
2
1
2
I(kp−1 = Ap−1 , kp−1 = Ap−1 , kp−1 6= kp−1 ) ϕ(ζnn )ϕ(ζnn )
2
Nn
p=1
X
1
j
i
I(Eni 6= Enj )ϕ(ζni )ϕ(ζnj ) =
ψ0,n
(ϕ)ψ0,n
(ϕ)
2
Nn
i6=j∈[N0 ]
i,j∈[Nn ]

2
X
X
X
i
i
i
ψ0,n
(ϕ) −
ψ0,n
(ϕ)2 = ηnN (ϕ)2 −
ψ0,n
(ϕ)2 .
= 
X
=
i∈[N0 ]
i∈[N0 ]
i∈[N0 ]
For part 2., when p = n, we have
=
⊗2
µN
)
bn (ϕ
N
FN (n)γn (1)2
X
k1
k2
C1,2 (A, ζ; k1:2 )ϕ(ζnn )ϕ(ζnn )
k1:2 ∈I(bn )
1
k
i
i
X Gn−1 (ζn−1
1
)I(En−1
6= Enn )
I(kn1 = kn2 )
kn
2
ϕ(ζ
)
=
P
n
j
Nn2
j∈[Nn−1 ] Gn−1 (ζn−1 )
i∈[Nn−1 ]
k1:2 ∈[N]2
X 1 i
En
i 2
=
1
−
Ḡ
0,n−1 ϕ(ζn ) ,
Nn2
X
i∈[Nn ]
i
(ϕ) = ϕ(ζni )/Nn . When p = 0, we have
and we conclude by noting that ψn,n
=
⊗2
µN
)
b0 (ϕ
N
FN (0)γn (1)2
X
k1
k2
C1,2 (A, ζ; k1:2 )ϕ(ζnn )ϕ(ζnn )
k1:2 ∈I(b0 )
" n
#
I(k01 = k02 ) Y
kq1
kq2
k1
k2
1
2 1
2
=
I(kq 6= kq , kq−1 = Aq−1 , kq−1 = Aq−1 ) ϕ(ζnn )ϕ(ζnn )
2
N
n
q=1
k1:2 ∈[N]2

2
X
X
X j
X X
0
j
j
j

(ϕ) −
ψ1,n (ϕ)2 .
=
ψ1,n
(ϕ)ψ1,n
(ϕ) =
ψ1,n
X
i∈[N0 ] j6=j 0 ∈O0i
i∈[N0 ]
j∈O0i
j∈O0i
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
9
Finally when p ∈ {1, . . . , n − 1}, we have
⊗2
µN
)
bp (ϕ
=
FN (p)γnN (1)2
X
k1
k2
C1,2 (A, ζ; k1:2 )ϕ(ζnn )ϕ(ζnn )
k1:2 ∈I(bp )
X
=
k1:2 ∈[N]2
I(kp1 = kp2 )
Nn2
"
n
Y
I(kq1
6=
1
kq2 , kq−1
=
kq1
2
Aq−1
, kq−1
#
=
kq2
Aq−1
)
q=p+1
k1
·
i
i
Gp−1 (ζp−1
)I(Ep−1
6= Ep p )
P
j
j∈[Np−1 ] Gp−1 (ζp−1 )
i∈[Np−1 ]

0
Epi
j
j
ψp+1,n
(ϕ)ψp+1,n
(ϕ) 1 − Ḡ0,p−1
k1
k2
ϕ(ζnn )ϕ(ζnn )

X
=
X

i∈[N0 ]
j6=j 0 ∈Opi
X =
X
Ei
p
1 − Ḡ0,p−1


2
X j
 X j

ψp+1,n (ϕ) −
ψp+1,n (ϕ)2  .

j∈Opi
i∈[Np ]
j∈Opi
S4. Proof of Theorem 4
We define
1:2
1:2
1:2
1:2
bp
Q̃bpp (x1:2
p−1 , dxp ) := G̃p−1 (xp−1 )Mp (xp−1 , dxp ),
p ≥ 1.
Our analysis involves an induction argument on the number of time steps in the discrete-time Feynman–Kac
model, and in particular shows that convergence occurs for all models satisfying (G) and functions ϕ ∈ L(X ⊗2 )
simultaneously. We define
FbN0 (i0 , j0 ) := I(b0 = 0, i0 6= j0 )
N0
+ I(b0 = 1, i0 = j0 )N0 ,
N0 − 1
and for p ≥ 1,
FbN0:p (ip , jp )
:= I(bp = 0, ip 6= jp )
Np
i
j
F N (A p , A p )
Np − 1 b0:p−1 p−1 p−1
Np−1
+I(bp = 1, ip =
jp−1
Gp−1 (ζp−1
)
ip
N
Fb0:p−1 (Ap−1 , jp−1 ) PNp−1
.
jp )Np
j
jp−1 =1
j=1 Gp−1 (ζp−1 )
X
(S4)
For each p we have
N
2
µN
b0:p (ϕ) = γp (1)
X
ip ,jp ∈[Np ]
1 N
F (ip , jp )ϕ(ζpip ,jp ) =
Np2 b0:p
X
ip ,jp ∈[Np ]
1
W ip ,jp δζ ip ,jp ,
p
Np2 p
i ,j
where Wpp p := γpN (1)2 FbN0:p (ip , jp ). Let Fp := σ(A0 , . . . , Ap−1 , ζ0 , . . . , ζp ), p ≥ 1 and F0 := σ(ζ0 ). We show that
−1/2
if µN
rate for any ϕ ∈ L(X ⊗2 ) then µN
b0:p−1 (ϕ) approximates µb0:p−1 (ϕ) at a N
b0:p (ϕ) approximates µb0:p (ϕ) at
i,j
a N −1/2 rate for any ϕ ∈ L(X ⊗2 ). The first step is to show that the random variables Wp−1
necessarily satisfy a
certain regularity condition.
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
10
Lemma 5. For each N ≥ 1, let (W i,j )i,j∈[N ] be a collection of possibly dependent non-negative random variables.
Assume this sequence of collections of random variables satisfies, for any probability measure ν : B(X) → [0, 1] and
bounded ϕ ∈ L(X ⊗2 ),

2  21
X
√
 1

sup N E  2
W i,j ϕ(ζ i,j ) − ν ⊗2 (ϕ)  < +∞,
N
N
i,j∈[N ]
where ζ i ∼ ν i.i.d. Then, with S := {(i, j, i0 , j 0 ) ∈ [N ]4 : i0 , j 0 ∈
/ {i, j}},


X
0 0
1
W i,j W i ,j  < +∞.
sup E  3
N
N
0 0
{
(i,j,i ,j )∈S
Proof. Let ϕ(x, x0 ) := x + x0 and ν be defined by ν({1}) = ν({−1}) = 12 . Since ν ⊗2 (ϕ) = 0, we have

2  12
√
 1 X

W i,j ϕ(ζ i,j )  < +∞.
sup N E  2
N
N
i,j∈[N ]
We observe that
0
0
0
0
0
0
E(ϕ(ζ i , ζ j )ϕ(ζ i , ζ j )) = E(ζ i ζ i ) + E(ζ i ζ j ) + E(ζ j ζ i ) + E(ζ j ζ j ),
0
0
so that in particular if (i, j, i0 , j 0 ) ∈ S { then E(ϕ(ζ i , ζ j )ϕ(ζ i , ζ j )) ≥ 1. That is,

2 


X
X
0
0
1
1


W i,j ϕ(ζ i,j )  ≥ E  4
W i,j W i ,j  .
E  2
N
N
0 0
{
i,j∈[N ]
{i,j,i ,j }∈S
It follows that
 12

sup E 
N
1
N3
X
0
0
W i,j W i ,j 
√
=
sup
N
{i,j,i0 ,j 0 }∈S {
≤
 12

NE 
1
N4
X
0
0
W i,j W i ,j 
{i,j,i0 ,j 0 }∈S {

2  12
√
 1 X i,j

sup N E  2
W ϕ(ζ i,j )  < +∞,
N i,j
N
1
and we conclude by noting that supN f (N ) 2 < +∞ implies supN f (N ) < +∞.
Lemma 6. If, for any (Mq , Gq )q≥0 ,
√
sup N E
N
µN
b0:p−1 (ϕ)
2 21
− µb0:p−1 (ϕ)
< ∞,
then, with Sp−1 := {(i, j, i0 , j 0 ) ∈ [Np−1 ]4 : i0 , j 0 ∈
/ {i, j}},

X
 1
sup E  3
Np−1
N
{
0 0
ϕ ∈ L(X ⊗2 ),

i0 ,j 0 
i,j
Wp−1
Wp−1
 < +∞.
(i,j,i ,j )∈Sp−1
i,j
Proof. We apply Lemma 5, noting that Wp−1
is measurable w.r.t. σ(A0:p−2 , ζ0:p−2 ) and taking Mp−1 (x, ·) = ν(·)
for an arbitrary probability measure ν.
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
11
Lemmas 7 and 8 together provide important bounds that allow us to prove Theorem 4. Their proofs involve mainly
tedious manipulations involving properties of multinomial random variables, and can be found in Section S4.2.
h
i
Lemma 7. We can write, with E ∆N
p,bp (ip−1 , jp−1 ) | Fp−1 = 0,
1
X
N
bp
µN
b0:p (ϕ) − µb0:p−1 (Q̃p (ϕ)) =
ip−1 ,jp−1 ∈[Np−1 ]
i
2
Np−1
p−1
Wp−1
,jp−1
∆N
p,bp (ip−1 , jp−1 ),
(S5)
where
h
i2
PNp−1
j
G
(ζ
)
p−1
p
j=1

∆N
p,0 (ip−1 , jp−1 ) := 
Np (Np − 1)
and


jp−1
∆N
p,1 (ip−1 , jp−1 ) := Gp−1 (ζp−1 )


ip−1 ,jp−1
I(ip 6= jp )ϕ(ζpip ,jp ) − Q̃bpp (ϕ)(ζp−1
),
X
i
j
p−1
p−1
(ip ,jp )∈Op−1
×Op−1
PNp−1
j=1
j
Gp−1 (ζp−1
)
Np

X

ip−1 ,jp−1
).
ϕ(ζpip ,ip ) − Q̃bpp (ϕ)(ζp−1
i
p−1
ip ∈Op−1
Lemma 8. Let ϕ ∈ L(X ⊗2 ) be non-negative, and Sp−1 := {(i, j, i0 , j 0 ) ∈ [Np−1 ]4 : i0 , j 0 ∈
/ {i, j}}. Then,
0
1. For any (ip−1 , jp−1 , i0p−1 , jp−1
) ∈ Sp−1 ∩ I(bp−1 )2 ,
h
i
N
0
0
E ∆N
(i
,
j
)∆
(i
,
j
)
|
F
p−1 ≤ 0.
p,bp p−1 p−1
p,bp p−1 p−1
2. There exists C < ∞ such that
h
i
N
0
0
E ∆N
p,bp (ip−1 , jp−1 )∆p,bp (ip−1 , jp−1 ) | Fp−1 ≤ C,
0
{
for any (ip−1 , jp−1 , i0p−1 , jp−1
) ∈ Sp−1
∩ I(bp−1 )2 .
Proposition 4. If, for any (Mq , Gq )q≥0 ,
√
sup
NE
N
µN
b0:p−1 (ϕ)
2 21
− µb0:p−1 (ϕ)
< ∞,
ϕ ∈ L(X ⊗2 ).
Then, for any (Mq , Gq )q≥0 ,
√
sup N E
N
µN
b0:p (ϕ)
− µb0:p (ϕ)
2 21
< ∞,
ϕ ∈ L(X ⊗2 ).
Proof. We decompose ϕ into its positive and negative parts. That is ϕ = ϕ+ −ϕ− , where ϕ+ (x1:2 ) := max{0, ϕ(x1:2 )}
and ϕ− (x1:2 ) := | min{0, ϕ(x1:2 )}|. We can therefore write
E
=
E
≤
E
µN
b0:p (ϕ)
−
µN
b0:p (ϕ+ )
2 12
bp
µN
b0:p−1 (Q̃p (ϕ))
−
bp
µN
b0:p−1 (Q̃p (ϕ+ ))
N
bp
µN
b0:p (ϕ+ ) − µb0:p−1 (Q̃p (ϕ+ ))
+
bp
µN
b0:p−1 (Q̃p (ϕ− ))
2 12
+E
−
2 12
µN
b0:p (ϕ− )
bp
N
µN
b0:p−1 (Q̃p (ϕ− )) − µb0:p (ϕ− )
2 21
,
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
12
by application of Minkowski’s inequality. We bound the first term. It follows from Lemmas 7 and 8 that
2 N
bp
E µN
(ϕ
)
−
µ
(
Q̃
(ϕ
))
+
+
b0:p
b0:p−1
p

2 
X
1


ip−1 ,jp−1 N
= E 
Wp−1
∆p,bp (ip−1 , jp−1 ) 
2
Np−1
ip−1 ,jp−1 ∈[Np−1 ]

=

1
X
E
0
∈[Np−1 ]
ip−1 ,jp−1 ,i0p−1 ,jp−1
4
Np−1
i0p−1 ,j0 p−1 N
ip−1 ,jp−1
0
0

Wp−1
Wp−1
∆p,bp (ip−1 , jp−1 )∆N
p,bp (ip−1 , jp−1 )

=
E
1
ip−1 ,jp−1
Wp−1
Wp−1
4
Np−1
0
i0p−1 ,jp−1
X
0
ip−1 ,jp−1 ,i0p−1 ,jp−1
∈[Np−1 ]

h
i
N
0
0

E ∆N
p,bp (ip−1 , jp−1 )∆p,bp (ip−1 , jp−1 ) | Fp−1

≤


CE 
1
ip−1 ,jp−1
Wp−1
Wp−1
4
Np−1
0
i0p−1 ,jp−1
X
0
{
(ip−1 ,jp−1 ,i0p−1 ,jp−1
)∈Sp−1




≤
C

E
Np−1
1
ip−1 ,jp−1
Wp−1
Wp−1
3
Np−1
0
i0p−1 ,jp−1
X
0
{
(ip−1 ,jp−1 ,i0p−1 ,jp−1
)∈Sp−1

.
It then follows from Lemma 6 that
√
sup N E
N
µN
b0:p (ϕ+ )
2 12
−
bp
µN
b0:p−1 (Q̃p (ϕ+ ))
−
bp
µN
b0:p−1 (Q̃p (ϕ− ))
< +∞,
and an identical argument shows that
√
sup N E
N
so
µN
b0:p (ϕ− )
2 12
< +∞,
2 12
√
N
bp
(ϕ))
(ϕ)
−
µ
(
Q̃
sup N E µN
< +∞.
p
b0:p
b0:p−1
N
b
b
Now, since ϕ ∈ L(X ⊗2 ) implies Q̃pp (ϕ) ∈ L(X ⊗2 ) under (G), and µb0:p−1 (Q̃pp (ϕ)) = µb0:p (ϕ) we have by the
hypothesis in the statement
√
sup N E
N
bp
µN
b0:p−1 (Q̃p (ϕ))
2 21
− µb0:p (ϕ)
< +∞.
Therefore, by an application of Minkowski’s inequality we obtain that
2 12
√
sup N E µN
(ϕ)
−
µ
(ϕ)
b0:p
b0:p
N
( )
2 21
2 12
√
N
N
bp
N
bp
≤ sup N E µ̌b0:p (ϕ) − µb0:p−1 (Q̃p (ϕ))
+ E µb0:p−1 (Q̃p (ϕ)) − µb0:p (ϕ)
N
< +∞.
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
13
Proof of Theorem 4. The proof is by induction on n. In the case n = 0 we obtain,
X
1
µN
W0i0 ,j0 ϕ(ζ0i0 ,j0 )
b0 (ϕ) =
2
N0
i0 ,j0 ∈[N0 ]
1
N02
=
X
(i0 ,j0 )∈I(b0 )
X
=
N0b0
(i0 ,j0 )∈I(b0 )
N0
N0 − 1
1−b0
ϕ(ζ0i0 ,j0 )
1
ϕ(ζ0i0 ,j0 ).
|I(b0 )|
Let ϕ̄(x1 , x2 ) := ϕ(x1 , x2 ) − µb0 (ϕ), and ||ϕ̄|| := supx |ϕ̄(x)|. We obtain
h
2 i
2
E µN
= E µN
(ϕ)
−
µ
(ϕ)
b
b0 (ϕ̄) .
b0
0
In the case b0 = 1, we have
h
E

i
X
2
µN
= E
b0 (ϕ) − µb0 (ϕ)
i0 ∈[N0 ]
so supN
√
NE
h

1
1
1 E ϕ̄(X0 , X0 )2 ≤
||ϕ̄||,
ϕ̄(ζ0i0 ,i0 )2  =
2
N0
N0
N0
2 i 12
µN
(ϕ)
−
µ
(ϕ)
< +∞ for any ϕ ∈ L(X ⊗2 ). In the case b0 = 0, we obtain
b
0
b0

h
E
2 i
µN
b0 (ϕ) − µb0 (ϕ)

0
0
1
i
,j
= E
ϕ̄(ζ0i0 ,j0 )ϕ̄(ζ00 0 )
N02 (N0 − 1)2
0
0
i0 6=j0 ∈[N0 ] i0 6=j0 ∈[N0 ]


X
0
0
1
i
,j
i0 ,j0
)ϕ̄(ζ00 0 )
= E
2 (N − 1)2 ϕ̄(ζ0
N
0
0
0
0
{
X
X
(i0 ,j0 ,i0 ,j0 )∈S0 ∩I(b0 )
|S0{ ∩ I(b0 )|
N02 (N0 − 1)2
4N0 (N0 − 1)(N0 − 2) + 2N0 (N0 − 1)
= ||ϕ̄||2
N02 (N0 − 1)2
4(N0 − 2) + 2
4||ϕ̄||2
≤ ||ϕ̄||2
≤
,
N0 (N0 − 1)
N0
≤
so supN
√
NE
h
µN
b0 (ϕ) − µb0 (ϕ)
2 i 12
||ϕ̄||2
< +∞ for any ϕ ∈ L(X ⊗2 ). The result then follows by applying Proposition 4.
S4.1. Properties of multinomial random variables
All the results in Lemmas 9–11 can be obtained, after fairly tedious but straightforward calculations, using the
moment generating function of a multinomial distribution,
!n
k
X
ti
MX (t) =
pi e
,
i=1
and the fact that

E
m
Y
j=1

Xij  =
∂ m MX
(0).
∂ti1 · · · ∂tim
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
14
Lemma 9. Let (X1 , . . . , Xk ) be a Multinomial(n, p1 , . . . , pk ) random variable. Then
1. For any i ∈ [k], E [Xi ] = npi .
2. For distinct i, j ∈ [k], E [Xi Xj ] = n(n − 1)pi pj .
3. For any i ∈ [k], E [Xi (Xi − 1)] = n(n − 1)p2i .
Lemma 10. Let (X1 , . . . , Xk ) be a Multinomial(n, p1 , . . . , pk ) random variable. Then
1. For distinct i1 , i2 , i3 , i4 ∈ [k],
E [Xi1 Xi2 Xi3 Xi4 ]
=
4
Y
n!
pi
(n − 4)! j=1 j
≤
n2 (n − 1)2
4
Y
pij
j=1
=
E [Xi1 Xi2 ] E [Xi3 Xi4 ] .
2. For distinct i, j ∈ [k],
E[Xi (Xi − 1)Xj (Xj − 1)]
= n(n − 1)(n − 2)(n − 3)p2i p2j
≤ n2 (n − 1)2 p21 p22
= E[Xi (Xi − 1)]E [Xj (Xj − 1)] .
Lemma 11. Let (X1 , . . . , Xk ) be a Multinomial(n, p1 , . . . , pk ) random variable. Then
1. For any i ∈ [k],
E Xi2 = n(n − 1)p2i + npi .
2. For any i ∈ [k],
E Xi2 (Xi − 1)2 = n(n − 1)(n − 2)(n − 3)p4i + 4n(n − 1)(n − 2)p3i + 2n(n − 1)p2i .
3. For distinct i1 , i2 , i3 ∈ [k],
E Xi21 Xi2 Xi3 = n(n − 1)(n − 2)(n − 3)p2i1 pi2 pi3 + n(n − 1)(n − 2)pi1 pi2 pi3 .
4. For distinct i1 , i2 ∈ [k],
E Xi21 Xi22 = n(n − 1)(n − 2)(n − 3)p2i1 p2i2 + n(n − 1)(n − 2)(p2i1 pi2 + pi1 p2i2 ) + n(n − 1)pi1 pi2
Lemma 12. Let (X1 , . . . , Xk ) be a Multinomial(n, p1 , . . . , pk ) random variable where pi = S −1 gi , k/n ≤ c and
n > 4. Then there exists a constant C < ∞ such that, with ḡ := maxi∈{1,...,k} gi ,
1. For any i ∈ [k],
S2 2
E 2 Xi ≤ gi2 + C ḡ 2 .
n
2. For any i ∈ [k],
E
S4 2
2
X
(X
−
1)
≤ gi4 + C ḡ 4 .
i
n4 i
3. For distinct i1 , i2 , i3 ∈ [k],
S4
2
E 2
X Xi Xi ≤ gi21 gi2 gi3 + C ḡ 4 .
n (n − 1)2 i1 2 3
4. For distinct i1 , i2 ∈ [k],
S4
2
2
X X
≤ gi21 gi22 + C ḡ 4 .
E 2
n (n − 1)2 i1 i2
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
15
Proof. We use the properties from Lemma 11. For part 1.,
2
S
S2 n(n − 1)p2i + npi
E 2 Xi2
=
2
n
n
k
S
≤ gi2 + gi ≤ gi2 + ḡ 2 .
n
n
For part 2.,
S4
2
2
E 2
X (Xi − 1)
n (n − 1)2 i
S4
n(n − 1)(n − 2)(n − 3)p4i + 4n(n − 1)(n − 2)p3i + 2n(n − 1)p2i
=
2
2
n (n − 1)
4S(n − 2) 3
2S 2
≤ gi4 +
gi +
g2
n(n − 1)
n(n − 1) i
4k 4
k2
≤ gi4 +
ḡ + 2
ḡ 4
n
n(n − 1)
n 4
≤ gi4 + 4cḡ 4 + 2c2
ḡ .
n−1
For part 3.,
S4
2
E 2
X
X
X
i
i
n (n − 1)2 i1 2 3
S4
=
n(n − 1)(n − 2)(n − 3)p2i1 pi2 pi3 + n(n − 1)(n − 2)pi1 pi2 pi3
2
2
n (n − 1)
S(n − 2)
gi gi gi
≤ gi21 gi2 gi3 +
n(n − 1) 1 2 3
≤ gi21 gi2 gi3 + cḡ 4 .
For part 4.,
S4
2
2
X X
E 2
n (n − 1)2 i1 i2
S4
=
n(n − 1)(n − 2)(n − 3)p2i1 p2i2 + n(n − 1)(n − 2)(p2i1 pi2 + pi1 p2i2 ) + n(n − 1)pi1 pi2
2
2
n (n − 1)
S(n − 2) 2
S2
≤ gi21 gi22 +
(gi1 gi2 + gi1 gi22 ) +
gi gi
n(n − 1)
n(n − 1) 1 2
n 4
≤ gi21 gi22 + cḡ 4 + c2
ḡ .
n−1
The result follows by taking C = 4c +
4
3
· 2c2 since
n
n−1
≤ 43 .
0
{
) ∈ Sp−1
∩I(bp−1 )2 . Then there exists C < ∞
Corollary 3. Assume supx∈X Gp−1 (x) < ∞ and (ip−1 , jp−1 , i0p−1 , jp−1
Np−1
such that for any Np , Np−1 satisfying Np−1 /Np ≤ c and ζp−1 ∈ X
,
h

i
4
PNp−1
j
G
(ζ
)
0
0
p−1
p−1
j=1
ip−1
jp−1


ip−1
jp−1
0
E
| |Op−1
| − I{i0p−1 = jp−1
} | Fp−1  ≤ C,
|Op−1
| |Op−1
| − I{ip−1 = jp−1 } |Op−1
Np2 (Np − 1)2
and
h
0
jp−1

jp−1
Gp−1 (ζp−1
)Gp−1 (ζp−1 )E 
PNp−1
j=1
j
Gp−1 (ζp−1
)
Np2
i2

i0p−1

ip−1
|Op−1
||Op−1 | | Fp−1  ≤ C.
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
16
P
j
Proof. Let k := Np−1 , n := Np , S =
Gp−1 (ζp−1
) and (X1 , . . . , Xk ) be a Multinomial(n, p1 , . . . , pk ) random
−1
i
variable where pi = S Gp−1 (ζp−1 ) and n > 4. For the first expression, consider the case ip−1 = jp−1 . Then the
expression can be written as
S4
2
2
X (Xi − 1) ,
E 2
n (n − 1)2 i
and we conclude by combining part 2. of Lemma 12 with supx∈X Gp−1 (x) < ∞. Now consider the case ip−1 6= jp−1 .
Then the expression can be written as either
S4
2 2
E 2
X X ,
n (n − 1)2 i j
or in the form
S4
2
E 2
X Xi Xi ,
n (n − 1)2 i1 2 3
and we conclude by combining parts 3. and 4. of Lemma 12 with supx∈X Gp−1 (x) < ∞. The second expression can
be bounded by
2 2
S
sup Gp−1 (x) E 2 Xi2 ,
n
x∈X
and we conclude by combining the part 1. of Lemma 12 with supx∈X Gp−1 (x) < ∞.
S4.2. Expressions and bounds for ∆N
p,bp
Proof of Lemma 7. We obtain expressions for ∆N
p,bp for each of the cases bp ∈ {0, 1}, making use of (S4). In the case
bp = 0, we have
µN
b0:p (ϕ)
=
X
γpN (1)2
ip ,jp ∈[Np ]
=
=
N
γp−1
(1)2
1 N
F (ip , jp )ϕ(ζpip ,jp )
Np2 b0:p
hP
Np−1
1
X
j=1
FbN0:p−1 (ip−1 , jp−1 )
2
Np−1
ip−1 ,jp−1 ∈[Np−1 ]
hP
Np−1
1
ip−1 ,jp−1
Wp−1
2
Np−1
ip−1 ,jp−1 ∈[Np−1 ]
X
j=1
i2
j
Gp−1 (ζp−1
)
Np (Np − 1)
i2
j
Gp−1 (ζp−1
)
Np (Np − 1)
X
i
j
p−1
p−1
(ip ,jp )∈Op−1
×Op−1
X
i
I(ip 6= jp )ϕ(ζpip ,jp )
I(ip 6= jp )ϕ(ζpip ,jp ),
j
p−1
p−1
(ip ,jp )∈Op−1
×Op−1
from which the expression for ∆N
p,0 (ip−1 , jp−1 ) follows. We have
h
PNp−1
j=1

E
h

= E
i2
j
)
Gp−1 (ζp−1
Np (Np − 1)
PNp−1
j=1
i2
j
)
Gp−1 (ζp−1
Np (Np − 1)
i
p−1
= G̃p−1 (ζp−1
,jp−1

X
i

I(ip 6= jp )ϕ(ζpip ,jp ) | Fp−1 
j
p−1
p−1
(ip ,jp )∈Op−1
×Op−1
i
j
p−1
p−1
|Op−1
| |Op−1
| − I{ip−1
i
p−1
)M̃pbp (ϕ)(ζp−1
,jp−1


ip−1 ,jp−1
= jp−1 } M̃pbp (ϕ)(ζp−1
) | Fp−1 
),
where the last line follows from Lemma 9. Hence E ∆N
p,0 (ip−1 , jp−1 ) | Fp−1 = 0.
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
17
In the case bp = 1, we have
µN
b0:p (ϕ)
=
X
γpN (1)2
ip ,jp ∈[Np ]
=
X
γpN (1)2
ip ∈[Np ]
=
=
N
γp−1
(1)2
1 N
F (ip , jp )ϕ(ζpip ,jp )
Np2 b0:p
1 N
F (ip , ip )ϕ(ζpip ,ip )
Np2 b0:p
1
X
2
Np−1
ip−1 ,jp−1 ∈[Np−1 ]
1
X
2
Np−1
ip−1 ,jp−1 ∈[Np−1 ]
jp−1
FbN0:p−1 (ip−1 , jp−1 )Gp−1 (ζp−1
)
ip−1 ,jp−1
jp−1
Wp−1
Gp−1 (ζp−1
)
PNp−1
j=1
PNp−1
j
)
Gp−1 (ζp−1
Np
from which the expression for ∆N
p,1 (ip−1 , jp−1 ) follows. We have

PNp−1
j
X

jp−1
j=1 Gp−1 (ζp−1 )
E Gp−1 (ζp−1 )
Np
i
j=1
j
Gp−1 (ζp−1
)
Np
X
X
ϕ(ζpip ,ip )
i
p−1
ip ∈Op−1
ϕ(ζpip ,ip ),
ip−1
ip ∈Op−1


ϕ(ζpip ,ip ) | Fp−1 
p−1
ip ∈Op−1
=
jp−1
Gp−1 (ζp−1
)E
i
" PNp−1
j=1
j
Gp−1 (ζp−1
) ip−1 bp
ip−1 ,jp−1
|Op−1 |M̃p (ϕ)(ζp−1
) | Fp−1
Np
i
,jp−1
#
,j
p−1 p−1
),
)M̃pbp (ϕ)(ζp−1
N
where the last line follows from Lemma 9. Hence E ∆p,1 (ip−1 , jp−1 ) | Fp−1 = 0.
=
p−1
G̃p−1 (ζp−1
0
Proof of Lemma 8. For the first part, consider first the case bp = 0. Then, since (ip−1 , jp−1 , i0p−1 , jp−1
) ∈ Sp−1 ∩
2
I(bp−1 ) ,
N
0
0
E ∆N
p,0 (ip−1 , jp−1 )∆p,0 (ip−1 , jp−1 ) | Fp−1

h
i4
PNp−1
j
X
X


j=1 Gp−1 (ζp )
i0 ,j 0
I(ip 6= jp )ϕ(ζpp p ) | Fp−1 
I(ip 6= jp )ϕ(ζpip ,jp )
= E


2
2
N (Np − 1)
p
i
i0
j
p−1
p−1
(ip ,jp )∈Op−1
×Op−1
i
,j
i
,j
i0
j0
p−1
p−1
(i0p ,jp0 )∈Op−1
×Op−1
,j 0
p−1 p−1
p−1 p−1
−Q̃bpp (ϕ)(ζp−1
)Q̃bpp (ϕ)(ζp−1
)
h

i4
PNp−1
j
G
(ζ
)
0
p−1 p
j=1
i0p−1
jp−1


ip−1
jp−1
0
= E
| |Op−1
| − I{i0p−1 = jp−1
} | Fp−1 
|Op−1
| Op−1
− I{ip−1 = jp−1 } |Op−1
2
2
Np (Np − 1)
i0
,j 0
i
,j
i0
,j 0
p−1 p−1
p−1 p−1
p−1 p−1
p−1 p−1
·M̃pbp (ϕ)(ζp−1
)M̃pbp (ϕ)(ζp−1
) − Q̃bpp (ϕ)(ζp−1
)Q̃bpp (ϕ)(ζp−1
)
 h

i4
P
Np−1

j

G
(ζ
)
0
0
p−1
p
j=1
ip−1
jp−1


ip−1
jp−1
0
| |Op−1
| − I{i0p−1 = jp−1
} | Fp−1 
=
E
|Op−1
| Op−1
− I{ip−1 = jp−1 } |Op−1
2 (N − 1)2

N
p
p




0
0
i0p−1 ,jp−1
i0p−1 ,jp−1
ip−1 ,jp−1
ip−1 ,jp−1
−G̃p−1 (ζp−1
)G̃p−1 (ζp−1
) · M̃pbp (ϕ)(ζp−1
)M̃pbp (ϕ)(ζp−1
)


≤ 0,
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
18
where the final inequality follows from properties in Lemma 10. Now, in the case bp = 1, and again because
0
(ip−1 , jp−1 , i0p−1 , jp−1
) ∈ Sp−1 ∩ I(bp−1 )2 ,
N
0
0
E ∆N
p,1 (ip−1 , jp−1 )∆p,1 (ip−1 , jp−1 ) | Fp−1


hP
i2
Np−1
j
X
X
0


j=1 Gp−1 (ζp−1 )
jp−1
i0 ,i0
jp−1
ϕ(ζpp p ) | Fp−1 
= E
ϕ(ζpip ,ip )
Gp−1 (ζp−1 )Gp−1 (ζp−1 )

2
N
p
i
,j
i0
,j 0
i
,j
i0
,j 0
i0
i
p−1
ip ∈Op−1
p−1
i0p ∈Op−1
p−1 p−1
p−1 p−1
−Q̃bpp (ϕ)(ζp−1
)Q̃bpp (ϕ)(ζp−1
)

h
i
2
PNp−1
j
G
(ζ
)
0
0
0
p−1 p−1
j=1
jp−1
ip−1
i0p−1 ,jp−1


ip−1
jp−1
ip−1 ,jp−1
bp
bp
G
(ζ
)G
(ζ
)
M̃
||O
|
|
F
(ϕ)(ζ
)
M̃
(ϕ)(ζ
)
|O
= E

p−1
p−1
p−1
p
p
p−1
p−1
p−1
p−1
p−1
p−1
Np2
p−1 p−1
p−1 p−1
−Q̃bpp (ϕ)(ζp−1
)Q̃bpp (ϕ)(ζp−1
)
 h


i
2
PNp−1
j




G
(ζ
)
0
0
p−1
p−1
j=1
ip−1
ip−1


ip−1
ip−1
E
=
|O
||O
|
|
F
−
G
(ζ
)G
(ζ
)
p−1 
p−1 p−1
p−1 p−1
p−1
p−1
2


Np


j0
j
i
p−1
p−1
p−1
·Gp−1 (ζp−1
)Gp−1 (ζp−1
)M̃pbp (ϕ)(ζp−1
,jp−1
i0
p−1
)M̃pbp (ϕ)(ζp−1
0
,jp−1
)
≤ 0,
where the final inequality follows from properties in Lemma 10. For the second part, let ||ϕ|| = supx ϕ(x). In the
case bp = 0 we have
h
i
N
0
0
E ∆N
p,bp (ip−1 , jp−1 )∆p,bp (ip−1 , jp−1 ) | Fp−1

h
i4
PNp−1
j
G
(ζ
)
X
X
p−1 p


j=1
i0 ,j 0
I(ip 6= jp )ϕ(ζpp p ) | Fp−1 
I(ip 6= jp )ϕ(ζpip ,jp )
≤ E


2
2
N (Np − 1)
p
i
i0
j
p−1
p−1
(ip ,jp )∈Op−1
×Op−1
j0
p−1
p−1
(i0p ,jp0 )∈Op−1
×Op−1
h

i4
PNp−1
j
)
G
(ζ
0
0
p−1
p
j=1
ip−1
jp−1


ip−1
jp−1
0
≤ ||ϕ||2 E 
|Op−1
| |Op−1
| − I{ip−1 = jp−1 } |Op−1
| |Op−1
| − I{i0p−1 = jp−1
} | Fp−1 
Np2 (Np − 1)2
≤ C||ϕ||2 ,
by applying Corollary 3 to obtain the last inequality. Similarly, in the case bp = 1 we have
h
i
N
0
0
E ∆N
p,bp (ip−1 , jp−1 )∆p,bp (ip−1 , jp−1 ) | Fp−1


hP
i2
Np−1
j
X
X
0


j=1 Gp−1 (ζp−1 )
jp−1
i0 ,i0
jp−1
≤ E
ϕ(ζpip ,ip )
ϕ(ζpp p ) | Fp−1 
Gp−1 (ζp−1 )Gp−1 (ζp−1 )

2
N
p
h
≤
0
jp−1

jp−1
||ϕ||2 Gp−1 (ζp−1
)Gp−1 (ζp−1
)E 
≤
C||ϕ||2 ,
PNp−1
j=1
by applying Corollary 3 to obtain the last inequality.
i
p−1
ip ∈Op−1
i2
j
)
Gp−1 (ζp−1
Np2
i0
p−1
i0p ∈Op−1

i0p−1

ip−1
|Op−1
||Op−1
| | Fp−1 
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
19
S5. Proofs for Section 4
Proof of Proposition 3. For part 1., since γ̂n (ϕ) = γn (ϕ̂), Lemma 3 establishes the moment bounds. To obtain the
expression for σγ̂2N (ϕ), we have
n
n
X
v̂p,n (ϕ)
lim N var γ̂nN (ϕ) = lim N var γnN (ϕ̂) = γ̂n (1)2
.
N →∞
N →∞
cp
p=0
For part 2., applying the moment bound in part 1. and Minkowski’s inequality to the decomposition
γ̂ N (ϕ) − γ̂n (ϕ)
γ̂ N (ϕ) γ̂n (1) − γ̂nN (1)
η̂nN (ϕ) − η̂n (ϕ) = nN
+ n
γ̂n (1)
γ̂n (1)
γ̂n (1)
gives
√
sup
N ≥1
h
r i1/r
N E η̂nN (ϕ) − η̂n (ϕ)
h
h
√
√
r i1/r
r i1/r
supx |ϕ(x)|
1
+
sup N E γ̂nN (1) − γ̂n (1)
sup N E γ̂nN (ϕ) − γ̂n (ϕ)
γ̂n (1) N ≥1
γ̂n (1) N ≥1
< ∞.
≤
The expression for ση̂2N (ϕ) follows by combining this with an essentially identical line of arguments as in the proof
n
of Theorem 1.
Proof of Theorem 5. The results follow from Theorems 2 and 3. For the first part,
h
i
E γ̂nN (1)2 V̂nN (ϕ) = E γnN (1)2 VnN (ϕ̂) = var γnN (ϕ̂) = var γ̂nN (ϕ) .
p
For the second part, since σ̄γ̂2N (ϕ) = σ̄γ2N (ϕ̂)/ηn (Gn )2 , it follows that N V̂nN (ϕ) = N VnN (ϕ̂)/ηnN (Gn )2 → σ̄γ̂2N (ϕ)
p
n
n
n
p
since N VnN (ϕ̂) → σ̄γ2N (ϕ̂) and ηnN (Gn )2 → ηn (Gn )2 . The third part holds by the same reasoning as in the proof of
n
Theorem 2.
Proof of Theorem 6. For part 1.,
N
N
E γ̂nN (1)2 v̂p,n
(ϕ) = E γnN (1)2 vp,n
(ϕ̂) = γn (1)2 vp,n (ϕ̂) = γ̂n (1)2 v̂p,n (ϕ).
p
p
N
N
N
(ϕ − η̂nN (ϕ)) →
(ϕ̂)/ηnN (Gn )2 → v̂p,n (ϕ) and letting f = ϕ − η̂n (ϕ) we obtain v̂p,n
For part 2., we have v̂p,n
(ϕ) = vp,n
vp,n (fˆ)/ηn (Gn )2 = v̂p,n (f ). Part 3. follows from parts 1. and 2.
S6. Supplementary figures
References
Andrieu, C., Doucet, A. and Holenstein, R. (2010). Particle Markov chain Monte Carlo methods. Journal of
the Royal Statistical Society B 72 269–342.
Andrieu, C., Lee, A. and Vihola, M. (2013). Uniform ergodicity of the iterated conditional SMC and geometric
ergodicity of particle Gibbs samplers. arXiv preprint arXiv:1312.6432.
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
400
20
1.5
●
●
●
●
●
●
●
300
●
●
●
●
●
1.0
●
●
200
0.5
100
0.0
7
8
9
(a) ϕ ≡ 1
10
11
7
8
9
10
11
N (Id)
(b) ϕ = Id − η̂n
Figure 9: Estimated asymptotic variances N V̂n (ϕ) (blue dots and error bars for the mean ± one standard deviation)
against log2 N for the stochastic volatility example.
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
21
80
●
60
●
40
●
●
20
●
●
0
●
●
●
●
●● ●
●●
●● ●
0
●
●
●
●
● ●
●●●●●●● ●
25
●
●
●
●● ●●
● ●
●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
50
75
100
(a) ϕ ≡ 1
●
0.4
0.3
●
0.2
●
0.1
0.0
●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
0
25
50
75
● ●
● ●
●
100
(b) ϕ = Id − η̂n (Id)
N
Figure 10: Plot of v̂p,n
(ϕ) (blue dots and error bars for the mean ± one standard deviation) at each p ∈ {0, . . . , n}
in the stochastic volatility example, with N = 105 .
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
15
●
22
●
●
−2
14
●
●
13
●
●
−4
12
●
●
11
●
−6
●
●
10
−6
−4
−2
−6
−4
(a)
−2
(b)
Figure 11: Plots for the simple adaptive N particle filter estimates of γ̂n (1) for the stochastic volatility example.
Figure (a) plots the base 2 logarithm of the empirical variance of γ̂nN (1)/γ̂n (1) against log2 δ, with the straight line
y = x. Figure (b) plots log2 N against log2 δ, where N is the average number of particles used by the final particle
filter.
3.0
900
2.5
●
●
●
●
●
●
●
2.0
●
●
●
●
●
●
●
800
1.5
700
1.0
7
8
9
(a) ϕ ≡ 1
10
11
7
8
9
10
11
N (Id)
(b) ϕ = Id − ηn
Figure 12: Estimated asymptotic variances N VnN (ϕ) (blue dots and error bars for the mean ± one standard deviation) against log2 N for the SMC sampler example.
A. Lee and N. Whiteley/Supplement to “Variance estimation and allocation in the particle filter”
16
23
●
●
−2
●
15
●
●
14
●
−4
●
13
●
●
12
●
−6
●
−6
●
11
−4
(a)
−2
−6
−4
−2
(b)
Figure 13: Plots for the simple adaptive N particle filter estimates of ηn (Id) for the SMC sampler example. Figure
(a) plots the base 2 logarithm of the squared L2 error of ηnN (Id) against log2 δ, with the straight line y = x. Figure
(b) plots log2 N against log2 δ, where N is the average number of particles used by the final particle filter.