Detecting changes in second order structure within
oceanographic time series
Rebecca Killick
Joint work with Idris Eckley and Phil Jonathan
Lancaster University
March 26, 2012
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
1 / 27
Summary
Motivation
Changepoint recap
Locally Stationary Wavelet (LSW) model
Wavelet Likelihood Method for changes in autocovariance
Simulation Study
Oceanographic Example
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
2 / 27
Motivation
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
3 / 27
Cyclic mean
Rebecca Killick (Lancaster University)
0
2
Unknown dependence structure
changing over time
4
6
Interested in detecting start and
end of storm season
8
Wave Heights
1993
Changes in autocovariance
1994
March 26, 2012
1995
4 / 27
Changepoint recap
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
5 / 27
Changepoints
For data y1 , . . . , yn , if a changepoint exists at τ , then y1 , . . . , yτ differ
from yτ +1 , . . . , yn in some way.
There are many different types of changes.
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
6 / 27
Change in second order structure
The traditional hypothesis, for some lag, v ,
H0 : cov (X0 , X0−v ) = cov (X1 , X1−v ) = . . . = cov (Xn−1 , Xn−1−v ) = ρ0,v
H1 : ρ1,v = cov (X0 , X0−v ) = . . . = cov (Xτ , Xτ −v )
6= cov (Xτ +1 , Xτ +1−v ) = . . . = cov (Xn−1 , Xn−1−v ) = ρn,v .
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
7 / 27
Change in second order structure
The traditional hypothesis, for some lag, v ,
H0 : cov (X0 , X0−v ) = cov (X1 , X1−v ) = . . . = cov (Xn−1 , Xn−1−v ) = ρ0,v
H1 : ρ1,v = cov (X0 , X0−v ) = . . . = cov (Xτ , Xτ −v )
6= cov (Xτ +1 , Xτ +1−v ) = . . . = cov (Xn−1 , Xn−1−v ) = ρn,v .
Likelihood-Based test statistic
λ = max
τ
L(y1:n , τ |ρ1,v , ρn,v , θ)
.
L(y1:n |ρ0,v , θ)
Methods exist that make structural assumptions about the covariance to
get L(·), e.g. piecewise AR.
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
7 / 27
Locally Stationary Wavelet Model
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
8 / 27
The Model
Locally Stationary Wavelet Model
Xt,n =
XX
j
Wj
k
n
ψj,k−t ξjk .
k
Relevant assumptions:
Eξjk = 0, cov(ξjk , ξlm ) = δjl δkm
We assume that the ξjk ∼ Normal
Hence EXt = 0, i.e. remove any mean or trend before analysis
Bounded total variation condition on the Wj2 (·).
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
9 / 27
The Model
Locally Stationary Wavelet Model
Xt,n =
XX
j
Wj
k
n
ψj,k−t ξjk .
k
Reasons for using LSW:
Models the local structure as it changes over time
Has a more general covariance structure than other time series models
Piecewise constant covariance structure =⇒ p.c. spectrum
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
10 / 27
Wavelet Likelihood Method
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
11 / 27
The Hypothesis
The traditional hypothesis, for some lag, v ,
H0 : cov (X0 , X0−v ) = cov (X1 , X1−v ) = . . . = cov (Xn−1 , Xn−1−v ) = ρ0,v
H1 : ρ1,v = cov (X0 , X0−v ) = . . . = cov (Xτ , Xτ −v )
6= cov (Xτ +1 , Xτ +1−v ) = . . . = cov (Xn−1 , Xn−1−v ) = ρn,v .
is equivalent to
= γ0,j
∀j
H0 : Wj2 n0 = Wj2 n1 = . . . = Wj2 n−1
n
H1 : γ1,j = Wj2 n0 = . . . = Wj2 τn 6= Wj2 τ +1
= . . . = Wj2
n
n−1
n
= γn,j ,
for some j ∈ {1, 2, . . . , J = log2 n}.
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
12 / 27
The Likelihood
Xt,n =
XX
j
Wj
k
n
ψj,k−t ξjk .
k
If ξ are Gaussian then,
`(W |x) =
n
2
log 2π − 12 log |ΣW | − 12 x0 Σ−1
W x,
where the variance-covariance matrix, ΣW , has the following form:
XX
0
Wl2 m
ΣW (k, k 0 ) = cov (Xk , Xk 0 ) =
n ψl,m−k ψl,m−k .
l
Rebecca Killick (Lancaster University)
m
Changes in autocovariance
March 26, 2012
13 / 27
Changepoint test statistic
λτ =
max
J<τ <n−J
n o
0 −1
log Σ̂0 + x0 Σ̂−1
x
−
log
Σ̂
−
x
Σ̂
x
.
1
0
1
Here Σ̂0 and Σ̂1 have elements,
XX
Σ̂0 (k, k 0 ) =
γ̂0,l ψl,m−k ψl,m−k 0 ,
m
l
0
Σ̂1 (k, k ) =
XX
l
γ̂1,l ψl,m−k ψl,m−k 0 +
γ̂n,l ψl,m−k ψl,m−k 0 ,
m>τ
m≤τ
Rebecca Killick (Lancaster University)
X
Changes in autocovariance
March 26, 2012
14 / 27
Simulation Study
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
15 / 27
Simulations
We compare with
AutoPARM (AP) - Davis, Lee, Rodriguez, Yam (2006)
Piecewise AR models
Likelihood ratio test statistic
Minimum Description Length Penalty
CF - Cho, Fryzlewicz (2011)
LSW model with Gaussian innovations
Designed to stabilize variance prior to changepoint estimation
Non-parametric test statistic
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
16 / 27
Simulations
Important measures,
Number of changepoints identified
Location of changepoints
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
17 / 27
Simulation Results
Stationary AR Model
Xt = aXt−1 + t
a
no. cpts
0
1
≥2
WL
100
0
0
-0.7
CF
71
24
5
AP
100
0
0
Rebecca Killick (Lancaster University)
WL
100
0
0
-0.1
CF
89
11
0
for 1 ≤ t ≤ 1024.
AP
100
0
0
WL
100
0
0
Changes in autocovariance
0.4
CF
94
5
1
AP
100
0
0
WL
91
9
0
0.7
CF
92
7
1
March 26, 2012
AP
100
0
0
18 / 27
Simulation Results
Piecewise AR Models
B
C
D
no. of
cpts
0
1
2
3
≥4

if 1 ≤ t ≤ 512,
 0.9Xt−1 + t
1.68Xt−1 − 0.81Xt−2 + t
if 513 ≤ t ≤ 768,
Xt =

1.32Xt−1 − 0.81Xt−2 + t
if 769 ≤ t ≤ 1024,

if 1 ≤ t ≤ 400,
 0.4Xt−1 + t
−0.6Xt−1 + t
if 401 ≤ t ≤ 612,
Xt =

0.5Xt−1 + t
if 613 ≤ t ≤ 1024,
0.75Xt−1 + t
if 1 ≤ t ≤ 50,
Xt =
−0.5Xt−1 + t
if 51 ≤ t ≤ 1024.
WL
0
0
98
2
0
Model B
CF AP
0
0
0
0
70
94
27
6
3
0
Rebecca Killick (Lancaster University)
WL
0
0
94
6
1
Model C
CF AP
0
0
0
0
76 100
22
0
2
0
Changes in autocovariance
WL
4
94
2
0
0
Model D
CF AP
2
0
83 100
15
0
0
0
0
0
March 26, 2012
19 / 27
Simulation Results
Models
E
F
G

2
 1.399Xt−1 − 0.4Xt−2 + t , t ∼ N (0, 0.8 )
Xt =
0.999Xt−1 + t ,
t ∼ N (0, 1.22 )

0.699Xt−1 + 0.3Xt−2 + t t ∼ N (0, 1)

if 1 ≤ t ≤ 125,

 0.7Xt−1 + t + 0.6t−1

0.3Xt−1 + t + 0.3t−1
if 126 ≤ t ≤ 352,
Xt =
0.9Xt−1 + t
if 353 ≤ t ≤ 704,



0.1Xt−1 + t − 0.5t−1
if 705 ≤ t ≤ 1024.
t + 0.8t−1
if 1 ≤ t ≤ 128,
Xt =
t + 1.68t−1 − 0.81t−2
if 129 ≤ t ≤ 256,
no. of
cpts
0
1
2
3
≥4
WL
0
26
45
26
3
Model E
CF AP
0
0
9
9
75
33
15
31
1
27
Rebecca Killick (Lancaster University)
WL
0
20
22
35
23
Model F
CF AP
0
0
12
51
36
33
45
16
7
0
Changes in autocovariance
WL
0
99
1
0
0
if 1 ≤ t ≤ 400,
if 401 ≤ t ≤ 750,
if 751 ≤ t ≤ 1024.
Model G
CF AP
0
0
85 100
15
0
0
0
0
0
March 26, 2012
20 / 27
0.04
Density
0.02
0.03
0.004
0.00
0.000
0.01
0.002
Density
0.05
0.006
0.06
0.07
Simulation Results
0
200
400
600
800
1000
0
Changepoint Location
Green - AP
Rebecca Killick (Lancaster University)
100
150
200
250
Changepoint Location
Model F
Red- WL
50
Model G
Blue - CF
Changes in autocovariance
March 26, 2012
21 / 27
Oceanographic Example
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
22 / 27
Application to North Sea Wave Heights
6
8
Significant Wave Heights at a
Central North Sea location
0
2
4
March 1992 - December 1994
1993
Rebecca Killick (Lancaster University)
1994
1995
First difference to remove the
mean
Changes in autocovariance
March 26, 2012
23 / 27
−2
−1
0
1
2
3
Application to North Sea Wave Heights
1993
Black - WL
Red - AP
1994
1995
Blue - CF
Ocean Engineers when presented with the results preferred the WL
estimates.
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
24 / 27
Summary
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
25 / 27
Summary
Demonstrated why changes in second order structure are important
Developed a method that:
has less restrictive assumptions than existing likelihood-based methods
performs on par with existing likelihood-based methods for AR models
is useful in practice when the covariance structure is unknown
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
26 / 27
References
Cho, H. and Fryzlewicz, P. (2011) Multiscale and multilevel technique
for consistent segmentation of nonstationary time series Statistica
Sinica, 22:207229
Davis, R. A., Lee, T. C. M., and Rodriguez-Yam, G. A. (2006)
Structural break estimation for nonstationary time series models
JASA, 101:223239.
Killick, R., Eckley, I. A., Jonathan, P. (2012) Detecting changes in
second order structure within oceanographic time series In
Submission.
Rebecca Killick (Lancaster University)
Changes in autocovariance
March 26, 2012
27 / 27