OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
AutoSPEC
Adaptive Spectral Estimation for Nonstationary
Time Series
DAVID S. S TOFFER
Department of Statistics
University of Pittsburgh
Ori Rosen
Sally Wood
UTEP
U.Melbourne
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
F OURIER T RANSFORM AND P ERIODOGRAM
Collect stationary time series {Xt ; t = 1, ..., n} with interest in cycles.
Rather than work with the data {Xt }, we transform it into the frequency domain:
Discrete Fourier Transformation (DFT)
Xt 7→ dj = n−1/2
Pn
t=1
Xt e−2πı̇t j/n
Periodogram (j= 0, 1, . . . , n − 1)
"
P (νj ) = |dj |2 =
#2 "
#2
n
n
1X
1X
j
j
Xt cos(2πt n
) +
Xt sin(2πt n
)
n t=1
n t=1
80
j cycles
.
n points
P (νj )
60
Xt
−6
0
−4
20
−2
40
0
2
4
That is, match (correlate) data with [co]sines oscillating at freqs νj =
0
20
40
60
80
100
0.0
Time
0.1
0.2
0.3
Frequency
DAVID S TOFFER
AUTO SPEC
for better segmentation
0.4
0.5
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
S PECTRAL D ENSITY
The periodogram P (νj:n ) = n−1/2
Pn
t=1
2
Xt exp(−2πı̇tνj:n ) is a sample
jn
n
concept. Its population counterpart is the (νj:n =
→ ν)
Spectral Density
f (ν) = lim E{P (νj:n )} =
n→∞
∞
X
γ(h) exp(2πı̇νh)
h=−∞
provided the limit exits (i.e.
|γ(h)| < ∞ where γ(h) = cov{Xt+h , Xt }). It
follows that f (ν) ≥ 0, f (1 + ν) = f (ν), f (ν) = f (−ν), and because
P
1/2
Z
γ(h) =
f (ν) exp(−2πı̇νh) dν
−1/2
The sample equivalent of the integral equation is:
n−1
X
P (j/n) n−1 = S 2 .
j=1
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
S PECTRAL D ENSITY
The periodogram P (νj:n ) = n−1/2
Pn
t=1
2
Xt exp(−2πı̇tνj:n ) is a sample
jn
n
concept. Its population counterpart is the (νj:n =
→ ν)
Spectral Density
f (ν) = lim E{P (νj:n )} =
n→∞
∞
X
γ(h) exp(2πı̇νh)
h=−∞
provided the limit exits (i.e.
|γ(h)| < ∞ where γ(h) = cov{Xt+h , Xt }). It
follows that f (ν) ≥ 0, f (1 + ν) = f (ν), f (ν) = f (−ν), and
P
Z
1/2
f (ν) dν = var(Xt )
[ = γ(0) ].
−1/2
The sample equivalent of the integral equation is:
n−1
X
P (j/n) n−1 = S 2 .
j=1
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
S PECTRAL D ENSITY
The periodogram P (νj:n ) = n−1/2
Pn
t=1
2
Xt exp(−2πı̇tνj:n ) is a sample
jn
n
concept. Its population counterpart is the (νj:n =
→ ν)
Spectral Density
f (ν) = lim E{P (νj:n )} =
n→∞
∞
X
γ(h) exp(2πı̇νh)
h=−∞
provided the limit exits (i.e.
|γ(h)| < ∞ where γ(h) = cov{Xt+h , Xt }). It
follows that f (ν) ≥ 0, f (1 + ν) = f (ν), f (ν) = f (−ν), and
P
Z
1/2
f (ν) dν = var(Xt )
[ = γ(0) ].
−1/2
The sample equivalent of the integral equation is:
n−1
X
P (j/n) n−1 = S 2 .
j=1
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
S OME E XAMPLES
2 δ h . The spectral density
WN: Wt is white noise if EWt = 0 and γ(h) = σw
0
f (ν) =
X
2
γ(h) exp(−2πı̇νh) = σw
− 1/2 ≤ ν ≤ 1/2,
is uniform (think of white light).
MA: Xt = Wt + .9Wt−1
AR: Xt = Xt−1 − .9Xt−2 + Wt
0.2
0.3
frequency
0.4
0.5
140
80
spectrum
40
20
1.0
0.5
0
0.0
0.1
60
2.0
1.5
spectrum
2.5
100
1.2
spectrum
1.0
0.8
0.6
0.0
AR
120
3.5
MA
3.0
1.4
White Noise
0.0
0.1
0.2
0.3
0.4
0.5
frequency
DAVID S TOFFER
AUTO SPEC
0.0
0.1
0.2
0.3
frequency
for better segmentation
0.4
0.5
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
S OME A SYMPTOTIC R ESULTS
cos(2πtνj:n )−ı̇ sin(2πtνj:n )
n
z
}|
{
X
d(νj:n ) = n−1/2
Xt exp(−2πı̇tνj:n )
t=1
= dc (νj:n ) − ı̇ ds (νj:n )
Under general conditions on {Xt } (n → ∞, νj:n → ν ):
dc (νj:n ) ∼ AN 0,
ds (νj:n ) ∼ AN 0,
1
2
1
2
f (ν)
f (ν)
dsc (νj:n ) ⊥ dsc (νk:n ) ∀ j, k (νk:n
→ ν 0 6= ν and terms not the same)
Pn (νj:n ) = d2c (νj:n ) + d2s (νj:n ),
thus
2Pn (νj:n )/f (ν) ⇒ χ22 ,
so
E[Pn (νj:n )] → f (ν), but var[Pn (νj:n )] → f 2 (ν) ←- BAD
One remedy? Kernel smooth for consistency:
Z
1/2
Pn (λ)Kn (ν − λ)dλ
fb(ν) =
−1/2
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
S OME A SYMPTOTIC R ESULTS
cos(2πtνj:n )−ı̇ sin(2πtνj:n )
n
z
}|
{
X
d(νj:n ) = n−1/2
Xt exp(−2πı̇tνj:n )
t=1
= dc (νj:n ) − ı̇ ds (νj:n )
Under general conditions on {Xt } (n → ∞, νj:n → ν ):
dc (νj:n ) ∼ AN 0,
ds (νj:n ) ∼ AN 0,
1
2
1
2
f (ν)
f (ν)
dsc (νj:n ) ⊥ dsc (νk:n ) ∀ j, k (νk:n
→ ν 0 6= ν and terms not the same)
Pn (νj:n ) = d2c (νj:n ) + d2s (νj:n ),
thus
2Pn (νj:n )/f (ν) ⇒ χ22 ,
so
E[Pn (νj:n )] → f (ν), but var[Pn (νj:n )] → f 2 (ν) ←- BAD
One remedy? Kernel smooth for consistency:
Z
1/2
Pn (λ)Kn (ν − λ)dλ
fb(ν) =
−1/2
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
S OME A SYMPTOTIC R ESULTS
cos(2πtνj:n )−ı̇ sin(2πtνj:n )
n
z
}|
{
X
d(νj:n ) = n−1/2
Xt exp(−2πı̇tνj:n )
t=1
= dc (νj:n ) − ı̇ ds (νj:n )
Under general conditions on {Xt } (n → ∞, νj:n → ν ):
dc (νj:n ) ∼ AN 0,
ds (νj:n ) ∼ AN 0,
1
2
1
2
f (ν)
f (ν)
dsc (νj:n ) ⊥ dsc (νk:n ) ∀ j, k (νk:n
→ ν 0 6= ν and terms not the same)
Pn (νj:n ) = d2c (νj:n ) + d2s (νj:n ),
thus
2Pn (νj:n )/f (ν) ⇒ χ22 ,
so
E[Pn (νj:n )] → f (ν), but var[Pn (νj:n )] → f 2 (ν) ←- BAD
One remedy? Kernel smooth for consistency:
Z
1/2
Pn (λ)Kn (ν − λ)dλ
fb(ν) =
−1/2
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
S OME A SYMPTOTIC R ESULTS
cos(2πtνj:n )−ı̇ sin(2πtνj:n )
n
z
}|
{
X
d(νj:n ) = n−1/2
Xt exp(−2πı̇tνj:n )
t=1
= dc (νj:n ) − ı̇ ds (νj:n )
Under general conditions on {Xt } (n → ∞, νj:n → ν ):
dc (νj:n ) ∼ AN 0,
ds (νj:n ) ∼ AN 0,
1
2
1
2
f (ν)
f (ν)
dsc (νj:n ) ⊥ dsc (νk:n ) ∀ j, k (νk:n
→ ν 0 6= ν and terms not the same)
Pn (νj:n ) = d2c (νj:n ) + d2s (νj:n ),
thus
2Pn (νj:n )/f (ν) ⇒ χ22 ,
so
E[Pn (νj:n )] → f (ν), but var[Pn (νj:n )] → f 2 (ν) ←- BAD
One remedy? Kernel smooth for consistency:
Z
1/2
Pn (λ)Kn (ν − λ)dλ
fb(ν) =
−1/2
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
S OME A SYMPTOTIC R ESULTS
cos(2πtνj:n )−ı̇ sin(2πtνj:n )
n
z
}|
{
X
d(νj:n ) = n−1/2
Xt exp(−2πı̇tνj:n )
t=1
= dc (νj:n ) − ı̇ ds (νj:n )
Under general conditions on {Xt } (n → ∞, νj:n → ν ):
dc (νj:n ) ∼ AN 0,
ds (νj:n ) ∼ AN 0,
1
2
1
2
f (ν)
f (ν)
dsc (νj:n ) ⊥ dsc (νk:n ) ∀ j, k (νk:n
→ ν 0 6= ν and terms not the same)
Pn (νj:n ) = d2c (νj:n ) + d2s (νj:n ),
thus
2Pn (νj:n )/f (ν) ⇒ χ22 ,
so
E[Pn (νj:n )] → f (ν), but var[Pn (νj:n )] → f 2 (ν) ←- BAD
One remedy? Kernel smooth for consistency:
Z
1/2
Pn (λ)Kn (ν − λ)dλ
fb(ν) =
−1/2
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
W HITTLE L IKELIHOOD
Given time series data x = (X1 , . . . , Xn ), for large n,
n−1
Y
1h
Pn (νk ) i
−n/2
L(f x) ≈ (2π)
exp − log f (νk ) +
,
2
f (νk )
k=0
νk = k/n, and k = 0, . . . , [n/2].
this part intentionally left blank
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
S TATIONARY C ASE
E STIMATION OF S PECTRA VIA S MOOTHING S PLINES
In the stationary case, let Pn (νk ) denote the periodogram. For large n,
approximately [recall 2Pn (νk:n )/f (ν) ⇒ χ22 ]
Pn (νk ) = f (νk )Uk
iid
where f (νk ) is the spectrum and Uk ∼ Gamma(1, 1).
Taking logs, we have a GLM
y(νk ) = g(νk ) + k
where y(νk ) = log Pn (νk ), g(νk ) = log f (νk ) and k are iid log(χ22 /2)s.
Want to fit the model with the constraint that g() is smooth.
Wahba (1980) suggested smoothing splines. This can be done in a
Bayesian framework.
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
S TATIONARY C ASE
E STIMATION OF S PECTRA VIA S MOOTHING S PLINES
In the stationary case, let Pn (νk ) denote the periodogram. For large n,
approximately [recall 2Pn (νk:n )/f (ν) ⇒ χ22 ]
Pn (νk ) = f (νk )Uk
iid
where f (νk ) is the spectrum and Uk ∼ Gamma(1, 1).
Taking logs, we have a GLM
y(νk ) = g(νk ) + k
where y(νk ) = log Pn (νk ), g(νk ) = log f (νk ) and k are iid log(χ22 /2)s.
Want to fit the model with the constraint that g() is smooth.
Wahba (1980) suggested smoothing splines. This can be done in a
Bayesian framework.
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
S TATIONARY C ASE
E STIMATION OF S PECTRA VIA S MOOTHING S PLINES
In the stationary case, let Pn (νk ) denote the periodogram. For large n,
approximately [recall 2Pn (νk:n )/f (ν) ⇒ χ22 ]
Pn (νk ) = f (νk )Uk
iid
where f (νk ) is the spectrum and Uk ∼ Gamma(1, 1).
Taking logs, we have a GLM
y(νk ) = g(νk ) + k
where y(νk ) = log Pn (νk ), g(νk ) = log f (νk ) and k are iid log(χ22 /2)s.
Want to fit the model with the constraint that g() is smooth.
Wahba (1980) suggested smoothing splines. This can be done in a
Bayesian framework.
DAVID S TOFFER
AUTO SPEC
for better segmentation
log(f
)
OVERTURE
11
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
−5
AR(2)
−10
0
0.05
0.1
0.2
0.25
ν
— true g = log f
• log Pn
4
0.15
0.3
0.35
0.4
0.45
0.5
- - - 95% credible intervals
2
log(f ) 0
22
−2
−4
−6
0
0.05
0.1
0.15
0.2
0.25
ν
0.3
0.35
frequency
DAVID S TOFFER
AUTO SPEC
for better segmentation
0.4
0.45
0.5
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
AR(2)
• log Pn
— true g = log f
- - - 95% credible intervals
frequency
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
H OW I T ’ S D ONE : P RIOR D ISTRIBUTIONS
Place a linear smoothing spline prior on log f (ν). Let
yn (νk ) = log Pn (νk ) and g(νk ) = log f (νk ).
g(νk ) = α0 + α1 νk + h(νk )
linear [α] + nonlinear [h()]
α0 ∼ N (0, σα2 ), α1 ≡ 0, since (∂g(ν)/∂ν)|ν=0 = 0.
h = Dβ , is a linear combination of basis functions where
√
h = (h(ν0 ), . . . , h(νn/2 ))0 , and the j th column of D is 2 cos(jπν),
ν = (ν0 , . . . , νn/2 )0 .
β ∼ N (0, τ 2 I)
τ 2 ∼ U (0, cτ 2 )
Wood, Jiang, Tanner (2002). Bayesian mixture of splines for ... nonparametric regression. Biometrika
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
H OW I T ’ S D ONE : P RIOR D ISTRIBUTIONS
Place a linear smoothing spline prior on log f (ν). Let
yn (νk ) = log Pn (νk ) and g(νk ) = log f (νk ).
g(νk ) = α0 + α1 νk + h(νk )
linear [α] + nonlinear [h()]
α0 ∼ N (0, σα2 ), α1 ≡ 0, since (∂g(ν)/∂ν)|ν=0 = 0.
h = Dβ , is a linear combination of basis functions where
√
h = (h(ν0 ), . . . , h(νn/2 ))0 , and the j th column of D is 2 cos(jπν),
ν = (ν0 , . . . , νn/2 )0 .
β ∼ N (0, τ 2 I)
τ 2 ∼ U (0, cτ 2 )
Wood, Jiang, Tanner (2002). Bayesian mixture of splines for ... nonparametric regression. Biometrika
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
H OW I T ’ S D ONE : P RIOR D ISTRIBUTIONS
Place a linear smoothing spline prior on log f (ν). Let
yn (νk ) = log Pn (νk ) and g(νk ) = log f (νk ).
g(νk ) = α0 + α1 νk + h(νk )
linear [α] + nonlinear [h()]
α0 ∼ N (0, σα2 ), α1 ≡ 0, since (∂g(ν)/∂ν)|ν=0 = 0.
h = Dβ , is a linear combination of basis functions where
√
h = (h(ν0 ), . . . , h(νn/2 ))0 , and the j th column of D is 2 cos(jπν),
ν = (ν0 , . . . , νn/2 )0 .
β ∼ N (0, τ 2 I)
τ 2 ∼ U (0, cτ 2 )
Wood, Jiang, Tanner (2002). Bayesian mixture of splines for ... nonparametric regression. Biometrika
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
H OW I T ’ S D ONE : P RIOR D ISTRIBUTIONS
Place a linear smoothing spline prior on log f (ν). Let
yn (νk ) = log Pn (νk ) and g(νk ) = log f (νk ).
g(νk ) = α0 + α1 νk + h(νk )
linear [α] + nonlinear [h()]
α0 ∼ N (0, σα2 ), α1 ≡ 0, since (∂g(ν)/∂ν)|ν=0 = 0.
h = Dβ , is a linear combination of basis functions where
√
h = (h(ν0 ), . . . , h(νn/2 ))0 , and the j th column of D is 2 cos(jπν),
ν = (ν0 , . . . , νn/2 )0 .
β ∼ N (0, τ 2 I)
τ 2 ∼ U (0, cτ 2 )
Wood, Jiang, Tanner (2002). Bayesian mixture of splines for ... nonparametric regression. Biometrika
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
H OW I T ’ S D ONE : P RIOR D ISTRIBUTIONS
Place a linear smoothing spline prior on log f (ν). Let
yn (νk ) = log Pn (νk ) and g(νk ) = log f (νk ).
g(νk ) = α0 + α1 νk + h(νk )
linear [α] + nonlinear [h()]
α0 ∼ N (0, σα2 ), α1 ≡ 0, since (∂g(ν)/∂ν)|ν=0 = 0.
h = Dβ , is a linear combination of basis functions where
√
h = (h(ν0 ), . . . , h(νn/2 ))0 , and the j th column of D is 2 cos(jπν),
ν = (ν0 , . . . , νn/2 )0 .
β ∼ N (0, τ 2 I)
τ 2 ∼ U (0, cτ 2 )
Wood, Jiang, Tanner (2002). Bayesian mixture of splines for ... nonparametric regression. Biometrika
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
H OW I T ’ S D ONE : P RIOR D ISTRIBUTIONS
Place a linear smoothing spline prior on log f (ν). Let
yn (νk ) = log Pn (νk ) and g(νk ) = log f (νk ).
g(νk ) = α0 + α1 νk + h(νk )
linear [α] + nonlinear [h()]
α0 ∼ N (0, σα2 ), α1 ≡ 0, since (∂g(ν)/∂ν)|ν=0 = 0.
h = Dβ , is a linear combination of basis functions where
√
h = (h(ν0 ), . . . , h(νn/2 ))0 , and the j th column of D is 2 cos(jπν),
ν = (ν0 , . . . , νn/2 )0 .
β ∼ N (0, τ 2 I)
τ 2 ∼ U (0, cτ 2 )
Wood, Jiang, Tanner (2002). Bayesian mixture of splines for ... nonparametric regression. Biometrika
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
S AMPLING S CHEME ∼ M ETROPOLIS -H ASTINGS (M-H)
The parameters
α0 , β and τ 2 are drawn from the posterior distribution
p(α0 , β, τ 2 y), where y = (yn (ν0 ), . . . , yn (νn/2 ))0 , using MCMC:
α0 and β are sampled jointly via an M-H step from
n 1 n−1
Xh
i
α0 + d0k β + exp yn (νk ) − α0 − d0k β
p(α0 , β τ 2 , y) ∝ exp −
2
k=0
o
α2
1
− 02 − 2 β 0 β ,
2σα
2τ
where d0k is the k th row of D .
τ 2 is sampled from the truncated inverse gamma distribution,
1
p(τ 2 β) ∝ (τ 2 )−J/2 exp − 2 β 0 β , τ 2 ∈ (0, cτ 2 ],
2τ
where J = number of frequencies.
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
S AMPLING S CHEME ∼ M ETROPOLIS -H ASTINGS (M-H)
The parameters
α0 , β and τ 2 are drawn from the posterior distribution
p(α0 , β, τ 2 y), where y = (yn (ν0 ), . . . , yn (νn/2 ))0 , using MCMC:
α0 and β are sampled jointly via an M-H step from
n 1 n−1
Xh
i
α0 + d0k β + exp yn (νk ) − α0 − d0k β
p(α0 , β τ 2 , y) ∝ exp −
2
k=0
o
α2
1
− 02 − 2 β 0 β ,
2σα
2τ
where d0k is the k th row of D .
τ 2 is sampled from the truncated inverse gamma distribution,
1
p(τ 2 β) ∝ (τ 2 )−J/2 exp − 2 β 0 β , τ 2 ∈ (0, cτ 2 ],
2τ
where J = number of frequencies.
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
Locally Stationary Time Series
P IECEWISE S TATIONARY
j−1
1
n1,m
ξ0,m
nj−1,m
ξ1,m . . .
m
j
ξj−1,m
nj,m
nm,m
ξj,m . . .
ξm,m
Suppose x = {X1 , . . . , Xn } is a time series with an unknown number of
Suppose x = (x1, . . . , xn)0 is a time series with an unknown number of
stationary segments.
locally stationary segments.
m: unknown number of segments (m = 1 means stationary)
nj,m :mnumber
of observations
the j th segment, nj,m ≥ tmin .
unknown
number of in
segments
ξj,m : location of the end of the j th segment, j = 0, . . . , m, ξ0,m ≡ 0 and
nξj,m number
of observations in the jth segment, nj,m ≥ tmin.
m,m ≡ n.
fj,m :ξj,m
spectral
densities
location
of the end of the jth segment, j = 0, . . . , m, ξ0,m ≡ 0
and ξm,m ≡ at
n.νkj = kj /nj,m , 0 ≤ kj ≤ nj,m − 1.
Pnj,m : periodograms
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
W HITTLE L IKELIHOOD
L(f1,m , . . . , fm,m x [data], ξ m
m
Y
j=1
(2π)−nj,m /2
[partition]
nj,m −1
Y
kj =0
)≈
n 1h
Pn (νk ) io
exp − log fj,m (νkj ) + j,m j
2
fj,m (νkj )
this space intentionally left blank
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
P RIOR D ISTRIBUTIONS
Priors on gj,m (ν) = log fj,m (ν), j = 1, . . . , m, as before.
Pr(ξj,m = t m) = 1/pjm , for j = 1, . . . , m − 1,
where pjm = n − ξj−1,m − (m − j + 1)tmin + 1 is the number of
available locations for split point ξj,m .
The prior on the number of segments
Pr(m = k) = 1/M,
DAVID S TOFFER
for
AUTO SPEC
k = 1, . . . , M.
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
S AMPLING S CHEME
LIFE GOES ON WITHIN MOVES AND WITHOUT MOVES
Within-model moves:
(location of end points)
Given m, ξk∗ ,m is proposed to be relocated.
The corresponding and β s are updated (absorb α0 s into β s).
These two steps are jointly accepted or rejected in a M-H step.
The τ 2 s are then updated in a Gibbs step.
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
Between-model moves:
I FALL TO P IECES
(number of segments)
mp = mc + 1†
Select a segment to split
Select a new split point in this segment.
Two new τ 2 s are formed from the current τ 2
Two new β s are drawn.
mp = mc − 1
Select a split point to be removed.
A single τ 2 is then formed from the current τ 2 s
A new β is proposed.
Accept or Reject in a M-H step.
† c=current,
p=proposed
DAVID S TOFFER
AUTO SPEC
for better segmentation
W E ’ RE SO SORRY, U NCLE ENSO
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
Between-Model Moves: mc → mp
Let θ m = {ξ m , τ 2m , β m } and suppose the chain is currently at (mc , θ cmc ).
p
p
p
p
We propose to move
toc (mc , θ mp ) by drawing (m , θ mp ) from a proposal
p p
density q(m , θ mp m , θ mc ) and accepting this draw with probability
(
p(mp , θ pmp |x) × q(mc , θ cmc
α = min 1,
p(mc , θ cmc |x) × q(mp , θ pmp
p p )
m , θ mp )
mc , θ cmc ) ,
where p(·) is the approximate likelihood. The M-H transition kernel is
composed of the q(mp |mc ) × α. These are essentially a likelihood ratios.
Thus the decision of whether or not to change m via the posterior is
essentially based on the likelihood ratio.
Within-model moves, relocation of end points, is similar.
... and model averaging!
... and all data used for estimation, not just segmented data!
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
Between-Model Moves: mc → mp
Let θ m = {ξ m , τ 2m , β m } and suppose the chain is currently at (mc , θ cmc ).
p
p
p
p
We propose to move
toc (mc , θ mp ) by drawing (m , θ mp ) from a proposal
p p
density q(m , θ mp m , θ mc ) and accepting this draw with probability
(
p(mp , θ pmp |x) × q(mc , θ cmc
α = min 1,
p(mc , θ cmc |x) × q(mp , θ pmp
p p )
m , θ mp )
mc , θ cmc ) ,
where p(·) is the approximate likelihood. The M-H transition kernel is
composed of the q(mp |mc ) × α. These are essentially a likelihood ratios.
Thus the decision of whether or not to change m via the posterior is
essentially based on the likelihood ratio.
Within-model moves, relocation of end points, is similar.
... and model averaging!
... and all data used for estimation, not just segmented data!
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
E XAMPLE
−2
−0.4
0
Xt
at
0.0
2
0.4
4
Consider two tvAR(1) models Xt = at Xt−1 + Wt for t = 1, . . . , 500
(blue) at = t/500 − .5 There is no optimal segmentation in this case.
(green) at = .5 sign(t − 250)
100
200
300
Time
400
500
0
100
200
300
Time
400
500
0
100
200
300
Time
400
500
0
100
200
300
Time
400
500
−3
−0.4
Xt
−1
at
0.0
1
0.4
3
0
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
In each case, m = 2 is the modal value [posteriors in
paper] on the
data) and
number of partitions.
Plotted
below
are
Pr(ξ
=
t
1,2
Pr(ξ1,2 < t data), where ξ1,2 is the change point when m = 2.
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
0.5
0.4
0.3
0.1
0
← time-varying
−0.1
−0.2
−0.3
0
0.1
1
0.2
0.8
0.6
0.3
0.4
0.4
Frequency
0.2
0.5
2
0
Time (rescaled)
1.5
1
Log(Power)
Log(Power)
0.2
0.5
0
−0.5
change-point →
−1
−1.5
0
0.1
1
0.2
0.8
0.6
0.3
0.4
0.4
Frequency
DAVID S TOFFER
AUTO SPEC
0.2
0.5
0
for better segmentation
Time (rescaled)
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
El Niño – Southern Oscillation
DAVID S TOFFER
AUTO SPEC
for better segmentation
W E ’ RE SO SORRY, U NCLE ENSO
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
DAVID S TOFFER
AUTO SPEC
for better segmentation
W E ’ RE SO SORRY, U NCLE ENSO
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
(a)
W E ’ RE SO SORRY, U NCLE ENSO
(b)
40
(c)
3
4
30
3
2
20
2
10
1
1
0
0
0
−10
−20
−1
−1
−2
−30
−2
−3
−40
−50
1880
1920
1960
Year
2000
−3
1950
1970
1990
2010
−4
1951
Year
1970
1990
Year
Plots of (a) SOI from 1876–2011; (b) Niño3.4 index from 1950-2011;
(c) DSLPA from 1951–2010.
DAVID S TOFFER
AUTO SPEC
for better segmentation
2010
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
The Posteriors – Pr(m = k x)
k
SOI
Niño3.4
DSLPA
1
2
3
0.95
0.05
0.00
0.93
0.07
0.00
0.99
0.01
0.00
DAVID S TOFFER
AUTO SPEC
for better segmentation
OVERTURE
I T ’ S A LL THE S AME
I FALL TO P IECES
W E ’ RE SO SORRY, U NCLE ENSO
The Posteriors – Pr(m = k x)
k
SOI
Niño3.4
DSLPA
1
2
3
0.95
0.05
0.00
0.93
0.07
0.00
0.99
0.01
0.00
DAVID S TOFFER
AUTO SPEC
for better segmentation