Rob J Hyndman
Functional time series
with applications in demography
5. Forecasting functional time series via PLS
Outline
1 Functional Partial Least Squares
2 Application: French mortality rates
3 Application: Australian fertility rates
4 Forecast accuracy comparisons
5 Bootstrap intervals
6 Comparisons
7 References
Functional time series with applications in demography
5. Forecasting functional time series via PLS
2
Functional PLS
PCA components are designed to explain
historical variation. They do not necessarily
provide the best predictors.
Partial least squares extracts uncorrelated latent
components cores by maximizing the covariance
between predictors and response.
Response is st (x) and predictor is st−1 (x).
We want to predict st (x) using
Z
ŝT +1|T (x) = µ̂(x) +
[sT (x) − µ̂(x)] b(x, u) du
This is a functional ARH(1).
How to choose b(x, u)?
Functional time series with applications in demography
5. Forecasting functional time series via PLS
3
Functional PLS
PCA components are designed to explain
historical variation. They do not necessarily
provide the best predictors.
Partial least squares extracts uncorrelated latent
components cores by maximizing the covariance
between predictors and response.
Response is st (x) and predictor is st−1 (x).
We want to predict st (x) using
Z
ŝT +1|T (x) = µ̂(x) +
[sT (x) − µ̂(x)] b(x, u) du
This is a functional ARH(1).
How to choose b(x, u)?
Functional time series with applications in demography
5. Forecasting functional time series via PLS
3
Functional PLS
PCA components are designed to explain
historical variation. They do not necessarily
provide the best predictors.
Partial least squares extracts uncorrelated latent
components cores by maximizing the covariance
between predictors and response.
Response is st (x) and predictor is st−1 (x).
We want to predict st (x) using
Z
ŝT +1|T (x) = µ̂(x) +
[sT (x) − µ̂(x)] b(x, u) du
This is a functional ARH(1).
How to choose b(x, u)?
Functional time series with applications in demography
5. Forecasting functional time series via PLS
3
Functional PLS
PCA components are designed to explain
historical variation. They do not necessarily
provide the best predictors.
Partial least squares extracts uncorrelated latent
components cores by maximizing the covariance
between predictors and response.
Response is st (x) and predictor is st−1 (x).
We want to predict st (x) using
Z
ŝT +1|T (x) = µ̂(x) +
[sT (x) − µ̂(x)] b(x, u) du
This is a functional ARH(1).
How to choose b(x, u)?
Functional time series with applications in demography
5. Forecasting functional time series via PLS
3
Functional PLS
PCA components are designed to explain
historical variation. They do not necessarily
provide the best predictors.
Partial least squares extracts uncorrelated latent
components cores by maximizing the covariance
between predictors and response.
Response is st (x) and predictor is st−1 (x).
We want to predict st (x) using
Z
ŝT +1|T (x) = µ̂(x) +
[sT (x) − µ̂(x)] b(x, u) du
This is a functional ARH(1).
How to choose b(x, u)?
Functional time series with applications in demography
5. Forecasting functional time series via PLS
3
Functional PLS
PCA components are designed to explain
historical variation. They do not necessarily
provide the best predictors.
Partial least squares extracts uncorrelated latent
components cores by maximizing the covariance
between predictors and response.
Response is st (x) and predictor is st−1 (x).
We want to predict st (x) using
Z
ŝT +1|T (x) = µ̂(x) +
[sT (x) − µ̂(x)] b(x, u) du
This is a functional ARH(1).
How to choose b(x, u)?
Functional time series with applications in demography
5. Forecasting functional time series via PLS
3
Functional PLS
PCA components are designed to explain
historical variation. They do not necessarily
provide the best predictors.
Partial least squares extracts uncorrelated latent
components cores by maximizing the covariance
between predictors and response.
Response is st (x) and predictor is st−1 (x).
We want to predict st (x) using
Z
ŝT +1|T (x) = µ̂(x) +
[sT (x) − µ̂(x)] b(x, u) du
This is a functional ARH(1).
How to choose b(x, u)?
Functional time series with applications in demography
5. Forecasting functional time series via PLS
3
Functional weighted PLS
Outer relationship
∗
f̂ (x) =
∞
X
βk ψk (x),
and
k =1
∗
ĝ (x) =
∞
X
βk φk (x)
k =1
W = diagonal(w1 , . . . , wT ), wt = κ(1 − κ)T −t
f ∗ (x) = W [s∗1 (x), . . . , s∗T −1 (x)]0 and
g∗ (x) = W [s∗2 (x), . . . , s∗T (x)]0 are weighted
decentralized functional predictors and
responses
βk denotes common latent component scores
ψk (x) and φk (x) are latent components of
predictors and responses respectively
Functional time series with applications in demography
5. Forecasting functional time series via PLS
4
Functional weighted PLS
Outer relationship
∗
f̂ (x) =
∞
X
βk ψk (x),
and
k =1
∗
ĝ (x) =
∞
X
βk φk (x)
k =1
W = diagonal(w1 , . . . , wT ), wt = κ(1 − κ)T −t
f ∗ (x) = W [s∗1 (x), . . . , s∗T −1 (x)]0 and
g∗ (x) = W [s∗2 (x), . . . , s∗T (x)]0 are weighted
decentralized functional predictors and
responses
βk denotes common latent component scores
ψk (x) and φk (x) are latent components of
predictors and responses respectively
Functional time series with applications in demography
5. Forecasting functional time series via PLS
4
Functional weighted PLS
Outer relationship
∗
f̂ (x) =
∞
X
βk ψk (x),
and
k =1
∗
ĝ (x) =
∞
X
βk φk (x)
k =1
W = diagonal(w1 , . . . , wT ), wt = κ(1 − κ)T −t
f ∗ (x) = W [s∗1 (x), . . . , s∗T −1 (x)]0 and
g∗ (x) = W [s∗2 (x), . . . , s∗T (x)]0 are weighted
decentralized functional predictors and
responses
βk denotes common latent component scores
ψk (x) and φk (x) are latent components of
predictors and responses respectively
Functional time series with applications in demography
5. Forecasting functional time series via PLS
4
Functional weighted PLS
Outer relationship
∗
f̂ (x) =
∞
X
βk ψk (x),
and
k =1
∗
ĝ (x) =
∞
X
βk φk (x)
k =1
W = diagonal(w1 , . . . , wT ), wt = κ(1 − κ)T −t
f ∗ (x) = W [s∗1 (x), . . . , s∗T −1 (x)]0 and
g∗ (x) = W [s∗2 (x), . . . , s∗T (x)]0 are weighted
decentralized functional predictors and
responses
βk denotes common latent component scores
ψk (x) and φk (x) are latent components of
predictors and responses respectively
Functional time series with applications in demography
5. Forecasting functional time series via PLS
4
Functional weighted PLS
Z
βk =
Z
∗
f (x)wk (x)dx =
g∗ (x)mk (x) dx,
Compute βk iteratively:
1
2
Let f ∗0 (x) = f ∗ (x) and g∗0 (x) = g∗ (x)
(0)
Obtain wZZ
k (x) iteratively, starting with wk (x) = 1:
∗
∗
(i)
(i−1)
∗
∗
wk (x) =
wk (v)[f̂ k−1 (v)]0 ĝk−1 (u)[ĝk−1 (u)]0 f̂ k−1 (x) dv du
Z
f ∗ (x)wk (x)dx.
3
βk =
4
f̂ k (x) = (I − βk βk0 )f̂ k−1 (x)
5
ĝk (x) = (I − βk βk0 )ĝk−1 (x)
∗
∗
∗
∗
Functional time series with applications in demography
5. Forecasting functional time series via PLS
5
Functional weighted PLS
Z
βk =
Z
∗
f (x)wk (x)dx =
g∗ (x)mk (x) dx,
Compute βk iteratively:
1
2
Let f ∗0 (x) = f ∗ (x) and g∗0 (x) = g∗ (x)
(0)
Obtain wZZ
k (x) iteratively, starting with wk (x) = 1:
∗
∗
(i)
(i−1)
∗
∗
wk (x) =
wk (v)[f̂ k−1 (v)]0 ĝk−1 (u)[ĝk−1 (u)]0 f̂ k−1 (x) dv du
Z
f ∗ (x)wk (x)dx.
3
βk =
4
f̂ k (x) = (I − βk βk0 )f̂ k−1 (x)
5
ĝk (x) = (I − βk βk0 )ĝk−1 (x)
∗
∗
∗
∗
Functional time series with applications in demography
5. Forecasting functional time series via PLS
5
Functional weighted PLS
Z
βk =
Z
∗
f (x)wk (x)dx =
g∗ (x)mk (x) dx,
Compute βk iteratively:
1
2
Let f ∗0 (x) = f ∗ (x) and g∗0 (x) = g∗ (x)
(0)
Obtain wZZ
k (x) iteratively, starting with wk (x) = 1:
∗
∗
(i)
(i−1)
∗
∗
wk (x) =
wk (v)[f̂ k−1 (v)]0 ĝk−1 (u)[ĝk−1 (u)]0 f̂ k−1 (x) dv du
Z
f ∗ (x)wk (x)dx.
3
βk =
4
f̂ k (x) = (I − βk βk0 )f̂ k−1 (x)
5
ĝk (x) = (I − βk βk0 )ĝk−1 (x)
∗
∗
∗
∗
Functional time series with applications in demography
5. Forecasting functional time series via PLS
5
Functional weighted PLS
Z
βk =
Z
∗
f (x)wk (x)dx =
g∗ (x)mk (x) dx,
Compute βk iteratively:
1
2
Let f ∗0 (x) = f ∗ (x) and g∗0 (x) = g∗ (x)
(0)
Obtain wZZ
k (x) iteratively, starting with wk (x) = 1:
∗
∗
(i)
(i−1)
∗
∗
wk (x) =
wk (v)[f̂ k−1 (v)]0 ĝk−1 (u)[ĝk−1 (u)]0 f̂ k−1 (x) dv du
Z
f ∗ (x)wk (x)dx.
3
βk =
4
f̂ k (x) = (I − βk βk0 )f̂ k−1 (x)
5
ĝk (x) = (I − βk βk0 )ĝk−1 (x)
∗
∗
∗
∗
Functional time series with applications in demography
5. Forecasting functional time series via PLS
5
Functional weighted PLS
Z
βk =
Z
∗
f (x)wk (x)dx =
g∗ (x)mk (x) dx,
Compute βk iteratively:
1
2
Let f ∗0 (x) = f ∗ (x) and g∗0 (x) = g∗ (x)
(0)
Obtain wZZ
k (x) iteratively, starting with wk (x) = 1:
∗
∗
(i)
(i−1)
∗
∗
wk (x) =
wk (v)[f̂ k−1 (v)]0 ĝk−1 (u)[ĝk−1 (u)]0 f̂ k−1 (x) dv du
Z
f ∗ (x)wk (x)dx.
3
βk =
4
f̂ k (x) = (I − βk βk0 )f̂ k−1 (x)
5
ĝk (x) = (I − βk βk0 )ĝk−1 (x)
∗
∗
∗
∗
Functional time series with applications in demography
5. Forecasting functional time series via PLS
5
Functional weighted PLS
Computationally equivalent approach
Discretize s∗t (x) on a dense grid of q equally
spaced points.
Denote discretized s∗t (x) as T × q matrix G∗ and
let G = WG∗ .
Define Gk−1 and Fk−1 analogously
w1 (x) = largest eigenvector of G0 F
mk (x) = largest eigenvector of G0k−1 Fk−1
wk (x) = largest eigenvector of F0k−1 Gk−1
Functional time series with applications in demography
5. Forecasting functional time series via PLS
6
Functional weighted PLS
Computationally equivalent approach
Discretize s∗t (x) on a dense grid of q equally
spaced points.
Denote discretized s∗t (x) as T × q matrix G∗ and
let G = WG∗ .
Define Gk−1 and Fk−1 analogously
w1 (x) = largest eigenvector of G0 F
mk (x) = largest eigenvector of G0k−1 Fk−1
wk (x) = largest eigenvector of F0k−1 Gk−1
Functional time series with applications in demography
5. Forecasting functional time series via PLS
6
Functional weighted PLS
Computationally equivalent approach
Discretize s∗t (x) on a dense grid of q equally
spaced points.
Denote discretized s∗t (x) as T × q matrix G∗ and
let G = WG∗ .
Define Gk−1 and Fk−1 analogously
w1 (x) = largest eigenvector of G0 F
mk (x) = largest eigenvector of G0k−1 Fk−1
wk (x) = largest eigenvector of F0k−1 Gk−1
Functional time series with applications in demography
5. Forecasting functional time series via PLS
6
Functional weighted PLS
Computationally equivalent approach
Discretize s∗t (x) on a dense grid of q equally
spaced points.
Denote discretized s∗t (x) as T × q matrix G∗ and
let G = WG∗ .
Define Gk−1 and Fk−1 analogously
w1 (x) = largest eigenvector of G0 F
mk (x) = largest eigenvector of G0k−1 Fk−1
wk (x) = largest eigenvector of F0k−1 Gk−1
Functional time series with applications in demography
5. Forecasting functional time series via PLS
6
Functional weighted PLS
Computationally equivalent approach
Discretize s∗t (x) on a dense grid of q equally
spaced points.
Denote discretized s∗t (x) as T × q matrix G∗ and
let G = WG∗ .
Define Gk−1 and Fk−1 analogously
w1 (x) = largest eigenvector of G0 F
mk (x) = largest eigenvector of G0k−1 Fk−1
wk (x) = largest eigenvector of F0k−1 Gk−1
Functional time series with applications in demography
5. Forecasting functional time series via PLS
6
Functional weighted PLS
Computationally equivalent approach
Discretize s∗t (x) on a dense grid of q equally
spaced points.
Denote discretized s∗t (x) as T × q matrix G∗ and
let G = WG∗ .
Define Gk−1 and Fk−1 analogously
w1 (x) = largest eigenvector of G0 F
mk (x) = largest eigenvector of G0k−1 Fk−1
wk (x) = largest eigenvector of F0k−1 Gk−1
Functional time series with applications in demography
5. Forecasting functional time series via PLS
6
Functional weighted PLS
Functional autoregression coefficient:
b(x, u) =
∞
X
wk (u)φk (x)
k =1
By orthogonality of βk :
φk (x) = (βk0 βk )−1 βk0 ĝ∗ (x).
Therefore
b̂(x, u) =
K
X
∗
wk (u)(βk0 βk )−1 βk0 ĝ (x),
k =1
for some finite K.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
7
Functional weighted PLS
Functional autoregression coefficient:
b(x, u) =
∞
X
wk (u)φk (x)
k =1
By orthogonality of βk :
φk (x) = (βk0 βk )−1 βk0 ĝ∗ (x).
Therefore
b̂(x, u) =
K
X
∗
wk (u)(βk0 βk )−1 βk0 ĝ (x),
k =1
for some finite K.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
7
Functional weighted PLS
Functional autoregression coefficient:
b(x, u) =
∞
X
wk (u)φk (x)
k =1
By orthogonality of βk :
φk (x) = (βk0 βk )−1 βk0 ĝ∗ (x).
Therefore
b̂(x, u) =
K
X
∗
wk (u)(βk0 βk )−1 βk0 ĝ (x),
k =1
for some finite K.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
7
Functional weighted PLS
Forecasted curves:
Z
f̂t+1|t (x) = µ̂(x) +
[f̂t (u) − µ̂(u)] b̂(x, u) du.
For f̂t+h|t (x) where h > 1, apply iteratively.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
8
Outline
1 Functional Partial Least Squares
2 Application: French mortality rates
3 Application: Australian fertility rates
4 Forecast accuracy comparisons
5 Bootstrap intervals
6 Comparisons
7 References
Functional time series with applications in demography
5. Forecasting functional time series via PLS
9
French female mortality rates
−6
−4
45
60
75
−8
Log mortality rates
−2
0
15
30
1850
1900
1950
2000
Year
Functional time series with applications in demography
5. Forecasting functional time series via PLS
10
French female mortality rates
−4
−6
−8
Log death rate
−2
0
France: female death rates (1816−2012)
0
20
40
60
80
100
Age
Functional time series with applications in demography
5. Forecasting functional time series via PLS
10
0
20 40 60 80
0.2
20 40 60 80
20 40 60 80
0.5
0.5
1.0
1.0
1.5
Age
−1.0 −0.5
−0.5
βt,3
0.0
βt,2
20
10
βt,1
0
Age
0
1850
0.0
φ3(x)
−0.2 −0.1
0
Age
30
Age
1950
Year
Functional time series with applications in demography
1850
1950
Year
−1.5
20 40 60 80
0.1
0.1 0.2
φ2(x)
−0.3
−0.15
−0.20
−5
−6
0
−0.1
−0.10
φ1(x)
−3
−4
^ (x)
µ
−2
−0.05
−1
French mortality rate models
1850
1950
Year
5. Forecasting functional time series via PLS
11
w3(x)
0.0
0.1
0.1
0.0
−0.2
−0.2
−0.1
w2(x)
−0.10
0
20
40
0
20
40
60
80
100
60
80
100
0
20
40
0
20
40
60
80
100
60
80
100
0
20
40
0
20
40
60
80
100
60
80
100
φ3(x)
φ2(x)
0.0
−0.3
−0.2
−0.1
0.1
0.2
0.1
−0.05
φ1(x)
−0.10
−0.15
−0.20
0.2
0.3
−0.20
−0.15
w1(x)
−0.05
0.2
French mortality rate models
Age
Functional time series with applications in demography
Age
Age
5. Forecasting functional time series via PLS
11
French mortality rate models
Functional time series with applications in demography
5. Forecasting functional time series via PLS
11
0
French mortality rate forecasts
−4
−6
−8
Log death rates
−2
Weighted PCA
forecasts
0
20
40
60
80
100
Age
Functional time series with applications in demography
5. Forecasting functional time series via PLS
12
0
French mortality rate forecasts
−4
−6
−8
Log death rates
−2
Weighted PLS
forecasts
0
20
40
60
80
100
Age
Functional time series with applications in demography
5. Forecasting functional time series via PLS
12
Outline
1 Functional Partial Least Squares
2 Application: French mortality rates
3 Application: Australian fertility rates
4 Forecast accuracy comparisons
5 Bootstrap intervals
6 Comparisons
7 References
Functional time series with applications in demography
5. Forecasting functional time series via PLS
13
250
Australian fertility rates
35
41
47
150
100
0
50
Fertility rates
200
17
23
29
1920
1940
1960
1980
2000
Year
Functional time series with applications in demography
5. Forecasting functional time series via PLS
14
150
100
0
50
Fertility rate
200
250
Australian fertility rates
15
20
25
30
35
40
45
50
Age
Functional time series with applications in demography
5. Forecasting functional time series via PLS
14
35
45
0.3
φ3(x)
35
45
15
25
35
45
Age
150
0
βt,2
50
200
600
400
200
−100
0
βt,1
25
Age
−200
1920
0.0
−0.2
15
Age
βt,3
25
1960
2000
Year
Functional time series with applications in demography
1920
−50 0
15
−150
45
300
35
Age
100
25
−0.2
−0.35
0
15
0.1
0.2
0.2
0.1
φ2(x)
0.0
−0.15
−0.25
φ1(x)
100
50
^ (x)
µ
150
−0.05
0.3
Australian fertility rate models
1960
Year
2000
1920
1960
2000
Year
5. Forecasting functional time series via PLS
15
30
35
40
45
50
15
20
25
30
35
40
45
50
w3(x)
−0.1
15
20
25
30
35
40
45
50
15
20
25
30
35
40
45
50
φ3(x)
20
25
30
35
40
45
50
15
20
25
30
35
40
45
50
0.0
−0.2
−0.1
0.0
φ2(x)
0.1
0.2
0.3
0.2
φ1(x)
0.1
15
0.0 0.1 0.2 0.3 0.4
25
0.3
20
−0.3
−0.2 −0.1
0.00
15
0.1 0.2 0.3
0.2
0.1
w2(x)
0.0
0.20
0.10
w1(x)
0.30
0.3
Australian fertility rate models
Age
Functional time series with applications in demography
Age
Age
5. Forecasting functional time series via PLS
15
Australian fertility rate models
Functional time series with applications in demography
5. Forecasting functional time series via PLS
15
250
Australian fertility rate forecasts
150
100
0
50
Fertility rates
200
Weighted PCA
forecasts
15
20
25
30
35
40
45
50
Age
Functional time series with applications in demography
5. Forecasting functional time series via PLS
16
250
Australian fertility rate forecasts
150
100
0
50
Fertility rates
200
Weighted PLS
forecasts
15
20
25
30
35
40
45
50
Age
Functional time series with applications in demography
5. Forecasting functional time series via PLS
16
Outline
1 Functional Partial Least Squares
2 Application: French mortality rates
3 Application: Australian fertility rates
4 Forecast accuracy comparisons
5 Bootstrap intervals
6 Comparisons
7 References
Functional time series with applications in demography
5. Forecasting functional time series via PLS
17
Forecast accuracy comparisons
p
2
1 X
yt (xi ) − ŷt|t−1 (xi ) .
MSEt =
p
i=1
Averaged over last m years of observed
data.
For French female mortality, m = 30
For Australian fertility, m = 20
For comparison, compare random walk:
yt+1|t (x) = yt (x).
Functional time series with applications in demography
5. Forecasting functional time series via PLS
18
Forecast accuracy comparisons
p
2
1 X
yt (xi ) − ŷt|t−1 (xi ) .
MSEt =
p
i=1
Averaged over last m years of observed
data.
For French female mortality, m = 30
For Australian fertility, m = 20
For comparison, compare random walk:
yt+1|t (x) = yt (x).
Functional time series with applications in demography
5. Forecasting functional time series via PLS
18
Forecast accuracy comparisons
p
2
1 X
yt (xi ) − ŷt|t−1 (xi ) .
MSEt =
p
i=1
Averaged over last m years of observed
data.
For French female mortality, m = 30
For Australian fertility, m = 20
For comparison, compare random walk:
yt+1|t (x) = yt (x).
Functional time series with applications in demography
5. Forecasting functional time series via PLS
18
Forecast accuracy comparisons
p
2
1 X
yt (xi ) − ŷt|t−1 (xi ) .
MSEt =
p
i=1
Averaged over last m years of observed
data.
For French female mortality, m = 30
For Australian fertility, m = 20
For comparison, compare random walk:
yt+1|t (x) = yt (x).
Functional time series with applications in demography
5. Forecasting functional time series via PLS
18
Forecast accuracy comparisons
French female mortality rates
0.20
0.25
●
●
0.15
●
●
0.10
●
●
●
●
●
0.00
0.05
MSE(x1000)
●
FPCR
FPCRw
Functional time series with applications in demography
FPLSR
FPLSRw
RW
5. Forecasting functional time series via PLS
19
Forecast accuracy comparisons
MSE: French female mortality rates (x1000)
K
FPC
FPCw FPLSR FPLSRw
RW
1 0.5956 0.0293 0.5994
0.0607
2 0.0537 0.0310 0.0738
0.0288
3 0.0316 0.0310 0.0445
0.0288
4 0.0296 0.0311 0.0428
0.0288
5 0.0287 0.0311 0.0472
0.0297
6 0.0425 0.0311 0.0474
0.0291 0.0437
Functional time series with applications in demography
5. Forecasting functional time series via PLS
20
Forecast accuracy comparisons
●
10
12
14
Australian fertility rates
●
●
8
●
●
●
2
4
6
MSE
●
FPCR
FPCRw
Functional time series with applications in demography
FPLSR
FPLSRw
RW
5. Forecasting functional time series via PLS
21
Forecast accuracy comparisons
MSE: Australian fertility rates
K
FPC
FPCw
FPLSR
1 99.0611 16.7304 94.0311
2 56.3095
3.3019 54.3410
3 24.9330
3.2580 26.0428
4 15.6845
3.1995 19.7227
5
4.4495
3.2132
5.9299
6
3.4310
3.2123
4.9205
Functional time series with applications in demography
FPLSRw
RW
53.8186
17.5883
10.2599
4.4818
4.0573
2.9046 4.9800
5. Forecasting functional time series via PLS
22
Computation time
Weighted FPLSR more efficient than weighted
FPC as FPC requires many univariate time
series models.
Time to fit 100 replications:
Method Fertility data Mortality data
FPC
FPCw
FPLSR
FPLSRw
RW
34.1072
33.1424
0.4287
0.4537
0.0000
62.2797
60.8426
2.9184
3.1602
0.0002
(Intel Xeon 2.33GHz processor)
Functional time series with applications in demography
5. Forecasting functional time series via PLS
23
Computation time
Weighted FPLSR more efficient than weighted
FPC as FPC requires many univariate time
series models.
Time to fit 100 replications:
Method Fertility data Mortality data
FPC
FPCw
FPLSR
FPLSRw
RW
34.1072
33.1424
0.4287
0.4537
0.0000
62.2797
60.8426
2.9184
3.1602
0.0002
(Intel Xeon 2.33GHz processor)
Functional time series with applications in demography
5. Forecasting functional time series via PLS
23
Outline
1 Functional Partial Least Squares
2 Application: French mortality rates
3 Application: Australian fertility rates
4 Forecast accuracy comparisons
5 Bootstrap intervals
6 Comparisons
7 References
Functional time series with applications in demography
5. Forecasting functional time series via PLS
24
Sources of uncertainty
Functional PCA
1
smoothing error in estimating st (x)
2
error in estimating µ(x)
3
error in forecasting the scores βt,k
4
error in the model residuals et (x)
5
appropriateness of model
Functional time series with applications in demography
5. Forecasting functional time series via PLS
25
Sources of uncertainty
Functional PCA
1
smoothing error in estimating st (x)
2
error in estimating µ(x)
3
error in forecasting the scores βt,k
4
error in the model residuals et (x)
5
appropriateness of model
Functional time series with applications in demography
5. Forecasting functional time series via PLS
25
Sources of uncertainty
Functional PCA
1
smoothing error in estimating st (x)
2
error in estimating µ(x)
3
error in forecasting the scores βt,k
4
error in the model residuals et (x)
5
appropriateness of model
Functional time series with applications in demography
5. Forecasting functional time series via PLS
25
Sources of uncertainty
Functional PCA
1
smoothing error in estimating st (x)
2
error in estimating µ(x)
3
error in forecasting the scores βt,k
4
error in the model residuals et (x)
5
appropriateness of model
Functional time series with applications in demography
5. Forecasting functional time series via PLS
25
Sources of uncertainty
Functional PCA
1
smoothing error in estimating st (x)
2
error in estimating µ(x)
3
error in forecasting the scores βt,k
4
error in the model residuals et (x)
5
appropriateness of model
Functional time series with applications in demography
5. Forecasting functional time series via PLS
25
Sources of uncertainty
Functional PLS
1
smoothing error in estimating st (x)
2
error in estimating µ(x)
3
error in estimating b(x, u)
4
error in the model residuals et (x)
5
appropriateness of model
Functional time series with applications in demography
5. Forecasting functional time series via PLS
25
Bootstrap curves for FPC
Let ξˆt,k = β̂t,k − β̂t|t−1,k be 1-step errors of PC scores.
(`)
{ξ } sampled with replacement from {ξˆt,k }.
k
Simulate future sample paths of βT +h|T ,k using these
(`)
bootstrapped residuals: {βT +h|T ,k }.
{e(`) (x)} sampled with replacement from residual
functions {ê1 (x), . . . , êT (x)}.
{ε(`) } sampled with replacement from {ε̂1,i , . . . , ε̂T ,i }.
(`)
yT +h|T (xi )
= µ̂(xi ) +
K
X
(`)
βT +h|T ,k φk (xi ) + e(`) (xi ) + σ̂T +h (xi )ε(`)
k =1
Prediction intervals are produced from the bootstrap
variants using percentiles.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
26
Bootstrap curves for FPC
Let ξˆt,k = β̂t,k − β̂t|t−1,k be 1-step errors of PC scores.
(`)
{ξ } sampled with replacement from {ξˆt,k }.
k
Simulate future sample paths of βT +h|T ,k using these
(`)
bootstrapped residuals: {βT +h|T ,k }.
{e(`) (x)} sampled with replacement from residual
functions {ê1 (x), . . . , êT (x)}.
{ε(`) } sampled with replacement from {ε̂1,i , . . . , ε̂T ,i }.
(`)
yT +h|T (xi )
= µ̂(xi ) +
K
X
(`)
βT +h|T ,k φk (xi ) + e(`) (xi ) + σ̂T +h (xi )ε(`)
k =1
Prediction intervals are produced from the bootstrap
variants using percentiles.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
26
Bootstrap curves for FPC
Let ξˆt,k = β̂t,k − β̂t|t−1,k be 1-step errors of PC scores.
(`)
{ξ } sampled with replacement from {ξˆt,k }.
k
Simulate future sample paths of βT +h|T ,k using these
(`)
bootstrapped residuals: {βT +h|T ,k }.
{e(`) (x)} sampled with replacement from residual
functions {ê1 (x), . . . , êT (x)}.
{ε(`) } sampled with replacement from {ε̂1,i , . . . , ε̂T ,i }.
(`)
yT +h|T (xi )
= µ̂(xi ) +
K
X
(`)
βT +h|T ,k φk (xi ) + e(`) (xi ) + σ̂T +h (xi )ε(`)
k =1
Prediction intervals are produced from the bootstrap
variants using percentiles.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
26
Bootstrap curves for FPC
Let ξˆt,k = β̂t,k − β̂t|t−1,k be 1-step errors of PC scores.
(`)
{ξ } sampled with replacement from {ξˆt,k }.
k
Simulate future sample paths of βT +h|T ,k using these
(`)
bootstrapped residuals: {βT +h|T ,k }.
{e(`) (x)} sampled with replacement from residual
functions {ê1 (x), . . . , êT (x)}.
{ε(`) } sampled with replacement from {ε̂1,i , . . . , ε̂T ,i }.
(`)
yT +h|T (xi )
= µ̂(xi ) +
K
X
(`)
βT +h|T ,k φk (xi ) + e(`) (xi ) + σ̂T +h (xi )ε(`)
k =1
Prediction intervals are produced from the bootstrap
variants using percentiles.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
26
Bootstrap curves for FPC
Let ξˆt,k = β̂t,k − β̂t|t−1,k be 1-step errors of PC scores.
(`)
{ξ } sampled with replacement from {ξˆt,k }.
k
Simulate future sample paths of βT +h|T ,k using these
(`)
bootstrapped residuals: {βT +h|T ,k }.
{e(`) (x)} sampled with replacement from residual
functions {ê1 (x), . . . , êT (x)}.
{ε(`) } sampled with replacement from {ε̂1,i , . . . , ε̂T ,i }.
(`)
yT +h|T (xi )
= µ̂(xi ) +
K
X
(`)
βT +h|T ,k φk (xi ) + e(`) (xi ) + σ̂T +h (xi )ε(`)
k =1
Prediction intervals are produced from the bootstrap
variants using percentiles.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
26
Bootstrap curves for FPC
Let ξˆt,k = β̂t,k − β̂t|t−1,k be 1-step errors of PC scores.
(`)
{ξ } sampled with replacement from {ξˆt,k }.
k
Simulate future sample paths of βT +h|T ,k using these
(`)
bootstrapped residuals: {βT +h|T ,k }.
{e(`) (x)} sampled with replacement from residual
functions {ê1 (x), . . . , êT (x)}.
{ε(`) } sampled with replacement from {ε̂1,i , . . . , ε̂T ,i }.
(`)
yT +h|T (xi )
= µ̂(xi ) +
K
X
(`)
βT +h|T ,k φk (xi ) + e(`) (xi ) + σ̂T +h (xi )ε(`)
k =1
Prediction intervals are produced from the bootstrap
variants using percentiles.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
26
Bootstrap curves for FPC
Let ξˆt,k = β̂t,k − β̂t|t−1,k be 1-step errors of PC scores.
(`)
{ξ } sampled with replacement from {ξˆt,k }.
k
Simulate future sample paths of βT +h|T ,k using these
(`)
bootstrapped residuals: {βT +h|T ,k }.
{e(`) (x)} sampled with replacement from residual
functions {ê1 (x), . . . , êT (x)}.
{ε(`) } sampled with replacement from {ε̂1,i , . . . , ε̂T ,i }.
(`)
yT +h|T (xi )
= µ̂(xi ) +
K
X
(`)
βT +h|T ,k φk (xi ) + e(`) (xi ) + σ̂T +h (xi )ε(`)
k =1
Prediction intervals are produced from the bootstrap
variants using percentiles.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
26
Bootstrap curves for FPLS
Resample residuals and construct bootstrap
samples:
∗
f (`) (x) = µ̂(x) + f̂ (x) + e(`) (x),
∗
g(`) (x) = µ̂(x) + ĝ (x) + e(`) (x).
Construct weighted FPLSR model for each bootstrap
sample.
Compute bootstrapped forecast variants:
(`)
yT +1|T (xi )
Z
= µ̂(xi ) +
(fT (u) − µ̂(u))b̂(`) (xi , u)du + σ̂T +1 (xi )ε̂(`)
Prediction intervals are produced from the bootstrap
variants using percentiles.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
27
Bootstrap curves for FPLS
Resample residuals and construct bootstrap
samples:
∗
f (`) (x) = µ̂(x) + f̂ (x) + e(`) (x),
∗
g(`) (x) = µ̂(x) + ĝ (x) + e(`) (x).
Construct weighted FPLSR model for each bootstrap
sample.
Compute bootstrapped forecast variants:
(`)
yT +1|T (xi )
Z
= µ̂(xi ) +
(fT (u) − µ̂(u))b̂(`) (xi , u)du + σ̂T +1 (xi )ε̂(`)
Prediction intervals are produced from the bootstrap
variants using percentiles.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
27
Bootstrap curves for FPLS
Resample residuals and construct bootstrap
samples:
∗
f (`) (x) = µ̂(x) + f̂ (x) + e(`) (x),
∗
g(`) (x) = µ̂(x) + ĝ (x) + e(`) (x).
Construct weighted FPLSR model for each bootstrap
sample.
Compute bootstrapped forecast variants:
(`)
yT +1|T (xi )
Z
= µ̂(xi ) +
(fT (u) − µ̂(u))b̂(`) (xi , u)du + σ̂T +1 (xi )ε̂(`)
Prediction intervals are produced from the bootstrap
variants using percentiles.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
27
Bootstrap curves for FPLS
Resample residuals and construct bootstrap
samples:
∗
f (`) (x) = µ̂(x) + f̂ (x) + e(`) (x),
∗
g(`) (x) = µ̂(x) + ĝ (x) + e(`) (x).
Construct weighted FPLSR model for each bootstrap
sample.
Compute bootstrapped forecast variants:
(`)
yT +1|T (xi )
Z
= µ̂(xi ) +
(fT (u) − µ̂(u))b̂(`) (xi , u)du + σ̂T +1 (xi )ε̂(`)
Prediction intervals are produced from the bootstrap
variants using percentiles.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
27
Bootstrap curves for FPLS
Resample residuals and construct bootstrap
samples:
∗
f (`) (x) = µ̂(x) + f̂ (x) + e(`) (x),
∗
g(`) (x) = µ̂(x) + ĝ (x) + e(`) (x).
Construct weighted FPLSR model for each bootstrap
sample.
Compute bootstrapped forecast variants:
(`)
yT +1|T (xi )
Z
= µ̂(xi ) +
(fT (u) − µ̂(u))b̂(`) (xi , u)du + σ̂T +1 (xi )ε̂(`)
Prediction intervals are produced from the bootstrap
variants using percentiles.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
27
Bootstrap curves for FPLS
Resample residuals and construct bootstrap
samples:
∗
f (`) (x) = µ̂(x) + f̂ (x) + e(`) (x),
∗
g(`) (x) = µ̂(x) + ĝ (x) + e(`) (x).
Construct weighted FPLSR model for each bootstrap
sample.
Compute bootstrapped forecast variants:
(`)
yT +1|T (xi )
Z
= µ̂(xi ) +
(fT (u) − µ̂(u))b̂(`) (xi , u)du + σ̂T +1 (xi )ε̂(`)
Prediction intervals are produced from the bootstrap
variants using percentiles.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
27
Coverage probability
Empirical coverage probability of 95% intervals
1
mph
T
X
p
h X
X
1
(0.025)
ŷt+j|t (xi )
< yt+j (xi ) <
(0.975)
ŷt+j|t (xi )
t =T −m+1 j=1 i=1
(α)
ŷt+j|t (xi ) = α-quantile from bootstrap samples
m = smallest number of observations used to fit a
model
Method
FPC
FPLS
Fertility data
Mortality data
98.00%
96.86%
97.19%
97.23%
Functional time series with applications in demography
5. Forecasting functional time series via PLS
28
Coverage probability
Empirical coverage probability of 95% intervals
1
mph
T
X
p
h X
X
1
(0.025)
ŷt+j|t (xi )
< yt+j (xi ) <
(0.975)
ŷt+j|t (xi )
t =T −m+1 j=1 i=1
(α)
ŷt+j|t (xi ) = α-quantile from bootstrap samples
m = smallest number of observations used to fit a
model
Method
FPC
FPLS
Fertility data
Mortality data
98.00%
96.86%
97.19%
97.23%
Functional time series with applications in demography
5. Forecasting functional time series via PLS
28
Coverage probability
Empirical coverage probability of 95% intervals
1
mph
T
X
p
h X
X
1
(0.025)
ŷt+j|t (xi )
< yt+j (xi ) <
(0.975)
ŷt+j|t (xi )
t =T −m+1 j=1 i=1
(α)
ŷt+j|t (xi ) = α-quantile from bootstrap samples
m = smallest number of observations used to fit a
model
Method
FPC
FPLS
Fertility data
Mortality data
98.00%
96.86%
97.19%
97.23%
Functional time series with applications in demography
5. Forecasting functional time series via PLS
28
Coverage probability
Empirical coverage probability of 95% intervals
1
mph
T
X
p
h X
X
1
(0.025)
ŷt+j|t (xi )
< yt+j (xi ) <
(0.975)
ŷt+j|t (xi )
t =T −m+1 j=1 i=1
(α)
ŷt+j|t (xi ) = α-quantile from bootstrap samples
m = smallest number of observations used to fit a
model
Method
FPC
FPLS
Fertility data
Mortality data
98.00%
96.86%
97.19%
97.23%
Functional time series with applications in demography
5. Forecasting functional time series via PLS
28
Adjusted prediction intervals
Compute d(x) = difference between (1 − α/2) and
(α/2) quantiles of {f̂t+1 (x) − f̂t+1|t (x); t = m, . . . , T − 1}
Let [`h (x), uh (x)] be 100(1 − α)% h-step-ahead
prediction intervals obtained from bootstrap
methods.
Ideally, u1 (x) − `1 (x) = d(x).
Adjusted prediction interval
h
0.5{`h (x) + uh (x)} − {uh (x) − `h (x)}p(x),
i
0.5{`h (x) + uh (x)} + {uh (x) − `h (x)}p(x)
where p(x) = 0.5d(x)/[u1 (x) − `1 (x)].
For h = 1, these have same coverage as in-sample
intervals if distributions symmetric.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
29
Adjusted prediction intervals
Compute d(x) = difference between (1 − α/2) and
(α/2) quantiles of {f̂t+1 (x) − f̂t+1|t (x); t = m, . . . , T − 1}
Let [`h (x), uh (x)] be 100(1 − α)% h-step-ahead
prediction intervals obtained from bootstrap
methods.
Ideally, u1 (x) − `1 (x) = d(x).
Adjusted prediction interval
h
0.5{`h (x) + uh (x)} − {uh (x) − `h (x)}p(x),
i
0.5{`h (x) + uh (x)} + {uh (x) − `h (x)}p(x)
where p(x) = 0.5d(x)/[u1 (x) − `1 (x)].
For h = 1, these have same coverage as in-sample
intervals if distributions symmetric.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
29
Adjusted prediction intervals
Compute d(x) = difference between (1 − α/2) and
(α/2) quantiles of {f̂t+1 (x) − f̂t+1|t (x); t = m, . . . , T − 1}
Let [`h (x), uh (x)] be 100(1 − α)% h-step-ahead
prediction intervals obtained from bootstrap
methods.
Ideally, u1 (x) − `1 (x) = d(x).
Adjusted prediction interval
h
0.5{`h (x) + uh (x)} − {uh (x) − `h (x)}p(x),
i
0.5{`h (x) + uh (x)} + {uh (x) − `h (x)}p(x)
where p(x) = 0.5d(x)/[u1 (x) − `1 (x)].
For h = 1, these have same coverage as in-sample
intervals if distributions symmetric.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
29
Adjusted prediction intervals
Compute d(x) = difference between (1 − α/2) and
(α/2) quantiles of {f̂t+1 (x) − f̂t+1|t (x); t = m, . . . , T − 1}
Let [`h (x), uh (x)] be 100(1 − α)% h-step-ahead
prediction intervals obtained from bootstrap
methods.
Ideally, u1 (x) − `1 (x) = d(x).
Adjusted prediction interval
h
0.5{`h (x) + uh (x)} − {uh (x) − `h (x)}p(x),
i
0.5{`h (x) + uh (x)} + {uh (x) − `h (x)}p(x)
where p(x) = 0.5d(x)/[u1 (x) − `1 (x)].
For h = 1, these have same coverage as in-sample
intervals if distributions symmetric.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
29
Adjusted prediction intervals
Compute d(x) = difference between (1 − α/2) and
(α/2) quantiles of {f̂t+1 (x) − f̂t+1|t (x); t = m, . . . , T − 1}
Let [`h (x), uh (x)] be 100(1 − α)% h-step-ahead
prediction intervals obtained from bootstrap
methods.
Ideally, u1 (x) − `1 (x) = d(x).
Adjusted prediction interval
h
0.5{`h (x) + uh (x)} − {uh (x) − `h (x)}p(x),
i
0.5{`h (x) + uh (x)} + {uh (x) − `h (x)}p(x)
where p(x) = 0.5d(x)/[u1 (x) − `1 (x)].
For h = 1, these have same coverage as in-sample
intervals if distributions symmetric.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
29
Adjusted prediction intervals
Compute d(x) = difference between (1 − α/2) and
(α/2) quantiles of {f̂t+1 (x) − f̂t+1|t (x); t = m, . . . , T − 1}
Let [`h (x), uh (x)] be 100(1 − α)% h-step-ahead
prediction intervals obtained from bootstrap
methods.
Ideally, u1 (x) − `1 (x) = d(x).
Adjusted prediction interval
h
0.5{`h (x) + uh (x)} − {uh (x) − `h (x)}p(x),
i
0.5{`h (x) + uh (x)} + {uh (x) − `h (x)}p(x)
where p(x) = 0.5d(x)/[u1 (x) − `1 (x)].
For h = 1, these have same coverage as in-sample
intervals if distributions symmetric.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
29
Adjusted prediction intervals
Fertility data
95%
adj 95%
FPC
FPLS
98.00%
96.86%
95.86%
94.89%
Functional time series with applications in demography
Mortality data
95%
adj 95%
97.19%
97.23%
95.91%
94.95%
5. Forecasting functional time series via PLS
30
0
Adjusted prediction intervals
−4
−6
−8
Log mortality rates
−2
Observation
Prediction intervals
Adjusted prediction intervals
−10
FPC method
0
20
40
60
80
100
Age
Functional time series with applications in demography
5. Forecasting functional time series via PLS
31
0
Adjusted prediction intervals
−4
−6
−8
Log mortality rates
−2
Observation
Prediction intervals
Adjusted prediction intervals
−10
FPLS method
0
20
40
60
80
100
Age
Functional time series with applications in demography
5. Forecasting functional time series via PLS
31
Observation
Prediction intervals
Adjusted prediction intervals
FPC method
80
60
0
20
40
Fertility rates
100
120
140
Adjusted prediction intervals
15
20
25
30
35
40
45
50
Age
Functional time series with applications in demography
5. Forecasting functional time series via PLS
31
Observation
Prediction intervals
Adjusted prediction intervals
FPC method
80
60
0
20
40
Fertility rates
100
120
140
Adjusted prediction intervals
15
20
25
30
35
40
45
50
Age
Functional time series with applications in demography
5. Forecasting functional time series via PLS
31
Outline
1 Functional Partial Least Squares
2 Application: French mortality rates
3 Application: Australian fertility rates
4 Forecast accuracy comparisons
5 Bootstrap intervals
6 Comparisons
7 References
Functional time series with applications in demography
5. Forecasting functional time series via PLS
32
Comparisons
FPC advantages
1 allows more complex dynamics and higher
order components;
2 eases interpretability of dynamic changes by
separating out effects of a few orthogonal
components;
FPLS advantages
1 latent components more suitable for prediction
rather than variance decomposition.
2 Faster as there is no need to fit univariate time
series models
Both methods implemented in ftsa package for R.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
33
Comparisons
FPC advantages
1 allows more complex dynamics and higher
order components;
2 eases interpretability of dynamic changes by
separating out effects of a few orthogonal
components;
FPLS advantages
1 latent components more suitable for prediction
rather than variance decomposition.
2 Faster as there is no need to fit univariate time
series models
Both methods implemented in ftsa package for R.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
33
Comparisons
FPC advantages
1 allows more complex dynamics and higher
order components;
2 eases interpretability of dynamic changes by
separating out effects of a few orthogonal
components;
FPLS advantages
1 latent components more suitable for prediction
rather than variance decomposition.
2 Faster as there is no need to fit univariate time
series models
Both methods implemented in ftsa package for R.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
33
Comparisons
FPC advantages
1 allows more complex dynamics and higher
order components;
2 eases interpretability of dynamic changes by
separating out effects of a few orthogonal
components;
FPLS advantages
1 latent components more suitable for prediction
rather than variance decomposition.
2 Faster as there is no need to fit univariate time
series models
Both methods implemented in ftsa package for R.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
33
Comparisons
FPC advantages
1 allows more complex dynamics and higher
order components;
2 eases interpretability of dynamic changes by
separating out effects of a few orthogonal
components;
FPLS advantages
1 latent components more suitable for prediction
rather than variance decomposition.
2 Faster as there is no need to fit univariate time
series models
Both methods implemented in ftsa package for R.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
33
Comparisons
FPC advantages
1 allows more complex dynamics and higher
order components;
2 eases interpretability of dynamic changes by
separating out effects of a few orthogonal
components;
FPLS advantages
1 latent components more suitable for prediction
rather than variance decomposition.
2 Faster as there is no need to fit univariate time
series models
Both methods implemented in ftsa package for R.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
33
Comparisons
FPC advantages
1 allows more complex dynamics and higher
order components;
2 eases interpretability of dynamic changes by
separating out effects of a few orthogonal
components;
FPLS advantages
1 latent components more suitable for prediction
rather than variance decomposition.
2 Faster as there is no need to fit univariate time
series models
Both methods implemented in ftsa package for R.
Functional time series with applications in demography
5. Forecasting functional time series via PLS
33
Outline
1 Functional Partial Least Squares
2 Application: French mortality rates
3 Application: Australian fertility rates
4 Forecast accuracy comparisons
5 Bootstrap intervals
6 Comparisons
7 References
Functional time series with applications in demography
5. Forecasting functional time series via PLS
34
Selected references
Hyndman, Shang (2009). “Forecasting
functional time series (with discussion)”.
Journal of the Korean Statistical Society 38(3),
199–221.
Hyndman (2014). demography: Forecasting
mortality, fertility, migration and population
data.
cran.r-project.org/package=demography
Shang, Hyndman (2013). ftsa: Functional time
series analysis.
cran.r-project.org/package=ftsa
Functional time series with applications in demography
5. Forecasting functional time series via PLS
35