Rob J Hyndman
Functional time series
with applications in demography
1. Tools for functional time series analysis
Mortality rates
−2
−4
−8
−6
Log death rate
0
2
French: male mortality (1816)
0
20
40
60
80
100
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
2
Fertility rates
150
100
0
50
Fertility rate
200
250
Australia fertility rates (1921)
15
20
25
30
35
40
45
50
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
3
Outline
1 Functional time series
2 Functional principal components
3 Data visualization
4 References
Functional time series with applications in demography
1. Tools for functional time series analysis
4
Functional time series
(
yt (xi ) = gλ (zt (xi )) =
log[zt (xi )]
if λ = 0;
λ−1 ztλ (xi ) − 1 otherwise.
= st (xi ) + σt (xi )εt,i
zt (xi ) is observed data for age xi in year t,
i = 1, . . . , N, t = 1, . . . , T.
λ chosen so that εt,i ∼ NID(0, 1).
We assume st (x) is a smooth function of x.
We need to estimate st (x) from the data for
x1 < x < xN .
We want to forecast whole curve zt (x) for
t = T + 1, . . . , T + h.
Functional time series with applications in demography
1. Tools for functional time series analysis
5
Functional time series
(
yt (xi ) = gλ (zt (xi )) =
log[zt (xi )]
if λ = 0;
λ−1 ztλ (xi ) − 1 otherwise.
= st (xi ) + σt (xi )εt,i
zt (xi ) is observed data for age xi in year t,
i = 1, . . . , N, t = 1, . . . , T.
λ chosen so that εt,i ∼ NID(0, 1).
We assume st (x) is a smooth function of x.
We need to estimate st (x) from the data for
x1 < x < xN .
We want to forecast whole curve zt (x) for
t = T + 1, . . . , T + h.
Functional time series with applications in demography
1. Tools for functional time series analysis
5
Functional time series
(
yt (xi ) = gλ (zt (xi )) =
log[zt (xi )]
if λ = 0;
λ−1 ztλ (xi ) − 1 otherwise.
= st (xi ) + σt (xi )εt,i
zt (xi ) is observed data for age xi in year t,
i = 1, . . . , N, t = 1, . . . , T.
λ chosen so that εt,i ∼ NID(0, 1).
We assume st (x) is a smooth function of x.
We need to estimate st (x) from the data for
x1 < x < xN .
We want to forecast whole curve zt (x) for
t = T + 1, . . . , T + h.
Functional time series with applications in demography
1. Tools for functional time series analysis
5
Functional time series
(
yt (xi ) = gλ (zt (xi )) =
log[zt (xi )]
if λ = 0;
λ−1 ztλ (xi ) − 1 otherwise.
= st (xi ) + σt (xi )εt,i
zt (xi ) is observed data for age xi in year t,
i = 1, . . . , N, t = 1, . . . , T.
λ chosen so that εt,i ∼ NID(0, 1).
We assume st (x) is a smooth function of x.
We need to estimate st (x) from the data for
x1 < x < xN .
We want to forecast whole curve zt (x) for
t = T + 1, . . . , T + h.
Functional time series with applications in demography
1. Tools for functional time series analysis
5
Functional time series
(
yt (xi ) = gλ (zt (xi )) =
log[zt (xi )]
if λ = 0;
λ−1 ztλ (xi ) − 1 otherwise.
= st (xi ) + σt (xi )εt,i
zt (xi ) is observed data for age xi in year t,
i = 1, . . . , N, t = 1, . . . , T.
λ chosen so that εt,i ∼ NID(0, 1).
We assume st (x) is a smooth function of x.
We need to estimate st (x) from the data for
x1 < x < xN .
We want to forecast whole curve zt (x) for
t = T + 1, . . . , T + h.
Functional time series with applications in demography
1. Tools for functional time series analysis
5
Smoothing functional time series
(
yt (xi ) = gλ (zt (xi )) =
log(zt (xi ))
if λ = 0;
λ−1 ztλ (xi ) − 1
otherwise.
= st (xi ) + σt (xi )εt,i
Estimate st (x) using penalized regression spline
with a large number of knots.
For mortality data, use λ = 0
and constrain st (x) to be monotonic for x > 50.
For fertility data, use λ = 0.4
and constrain st (x) to be concave.
Fit is weighted with wt (xi ) = σt−2 (xi ).
Functional time series with applications in demography
1. Tools for functional time series analysis
6
Smoothing functional time series
(
yt (xi ) = gλ (zt (xi )) =
log(zt (xi ))
if λ = 0;
λ−1 ztλ (xi ) − 1
otherwise.
= st (xi ) + σt (xi )εt,i
Estimate st (x) using penalized regression spline
with a large number of knots.
For mortality data, use λ = 0
and constrain st (x) to be monotonic for x > 50.
For fertility data, use λ = 0.4
and constrain st (x) to be concave.
Fit is weighted with wt (xi ) = σt−2 (xi ).
Functional time series with applications in demography
1. Tools for functional time series analysis
6
Smoothing functional time series
(
yt (xi ) = gλ (zt (xi )) =
log(zt (xi ))
if λ = 0;
λ−1 ztλ (xi ) − 1
otherwise.
= st (xi ) + σt (xi )εt,i
Estimate st (x) using penalized regression spline
with a large number of knots.
For mortality data, use λ = 0
and constrain st (x) to be monotonic for x > 50.
For fertility data, use λ = 0.4
and constrain st (x) to be concave.
Fit is weighted with wt (xi ) = σt−2 (xi ).
Functional time series with applications in demography
1. Tools for functional time series analysis
6
Smoothing functional time series
(
yt (xi ) = gλ (zt (xi )) =
log(zt (xi ))
if λ = 0;
λ−1 ztλ (xi ) − 1
otherwise.
= st (xi ) + σt (xi )εt,i
Estimate st (x) using penalized regression spline
with a large number of knots.
For mortality data, use λ = 0
and constrain st (x) to be monotonic for x > 50.
For fertility data, use λ = 0.4
and constrain st (x) to be concave.
Fit is weighted with wt (xi ) = σt−2 (xi ).
Functional time series with applications in demography
1. Tools for functional time series analysis
6
Smoothing functional time series
Mortality
Dt (xi ) = number of deaths at age xi in year t.
Et (xi ) = total population aged xi on June 30 in year t.
mt (xi ) = Dt (xi )/Et (xi ) = observed mortality rate.
µt (xi ) = “true” mortality rate.
Dt (xi ) ∼ Poisson(Et (xi )µt (xi ))
1
E[mt (xi )] = µt (x) and V[mt (xi )] = µt (xi )E−
t (xi ).
Functional time series with applications in demography
1. Tools for functional time series analysis
7
Smoothing functional time series
Mortality
Dt (xi ) = number of deaths at age xi in year t.
Et (xi ) = total population aged xi on June 30 in year t.
mt (xi ) = Dt (xi )/Et (xi ) = observed mortality rate.
µt (xi ) = “true” mortality rate.
Dt (xi ) ∼ Poisson(Et (xi )µt (xi ))
1
E[mt (xi )] = µt (x) and V[mt (xi )] = µt (xi )E−
t (xi ).
Functional time series with applications in demography
1. Tools for functional time series analysis
7
Smoothing functional time series
Mortality
Dt (xi ) = number of deaths at age xi in year t.
Et (xi ) = total population aged xi on June 30 in year t.
mt (xi ) = Dt (xi )/Et (xi ) = observed mortality rate.
µt (xi ) = “true” mortality rate.
Dt (xi ) ∼ Poisson(Et (xi )µt (xi ))
1
E[mt (xi )] = µt (x) and V[mt (xi )] = µt (xi )E−
t (xi ).
Functional time series with applications in demography
1. Tools for functional time series analysis
7
Smoothing functional time series
Mortality
Dt (xi ) = number of deaths at age xi in year t.
Et (xi ) = total population aged xi on June 30 in year t.
mt (xi ) = Dt (xi )/Et (xi ) = observed mortality rate.
µt (xi ) = “true” mortality rate.
Dt (xi ) ∼ Poisson(Et (xi )µt (xi ))
1
E[mt (xi )] = µt (x) and V[mt (xi )] = µt (xi )E−
t (xi ).
Taylor series approx
V[gλ (X)] = σX2 [g0λ (µX )]2
V[log(X)] = σX2 /µ2X
Functional time series with applications in demography
1. Tools for functional time series analysis
7
Smoothing functional time series
Mortality
Dt (xi ) = number of deaths at age xi in year t.
Et (xi ) = total population aged xi on June 30 in year t.
mt (xi ) = Dt (xi )/Et (xi ) = observed mortality rate.
µt (xi ) = “true” mortality rate.
Dt (xi ) ∼ Poisson(Et (xi )µt (xi ))
1
E[mt (xi )] = µt (x) and V[mt (xi )] = µt (xi )E−
t (xi ).
σ 2 (xi ) = V(log[mt (xi )])
1
≈ µt (xi )E−
µt (x)−2
(
x
)
i
t
= µt (xi )−1 Et (xi )−1
Taylor series approx
V[gλ (X)] = σX2 [g0λ (µX )]2
V[log(X)] = σX2 /µ2X
Functional time series with applications in demography
1. Tools for functional time series analysis
7
Smoothing functional time series
Fertility
Bt (xi ) = number of births to women aged xi in year t.
Et (xi ) = total population aged xi on June 30 in year t.
ft (xi ) = Bt (xi )/Et (xi ) = observed fertility rate.
µt (xi ) = “true” fertility rate.
Bt (xi ) ∼ Pn(Et (xi )µt (xi ))
1
E[ft (xi )] = µt (x) and V[ft (xi )] = µt (xi )E−
t (xi ).
Functional time series with applications in demography
1. Tools for functional time series analysis
8
Smoothing functional time series
Fertility
Bt (xi ) = number of births to women aged xi in year t.
Et (xi ) = total population aged xi on June 30 in year t.
ft (xi ) = Bt (xi )/Et (xi ) = observed fertility rate.
µt (xi ) = “true” fertility rate.
Bt (xi ) ∼ Pn(Et (xi )µt (xi ))
1
E[ft (xi )] = µt (x) and V[ft (xi )] = µt (xi )E−
t (xi ).
Functional time series with applications in demography
1. Tools for functional time series analysis
8
Smoothing functional time series
Fertility
Bt (xi ) = number of births to women aged xi in year t.
Et (xi ) = total population aged xi on June 30 in year t.
ft (xi ) = Bt (xi )/Et (xi ) = observed fertility rate.
µt (xi ) = “true” fertility rate.
Bt (xi ) ∼ Pn(Et (xi )µt (xi ))
1
E[ft (xi )] = µt (x) and V[ft (xi )] = µt (xi )E−
t (xi ).
Functional time series with applications in demography
1. Tools for functional time series analysis
8
Smoothing functional time series
Fertility
Bt (xi ) = number of births to women aged xi in year t.
Et (xi ) = total population aged xi on June 30 in year t.
ft (xi ) = Bt (xi )/Et (xi ) = observed fertility rate.
µt (xi ) = “true” fertility rate.
Bt (xi ) ∼ Pn(Et (xi )µt (xi ))
1
E[ft (xi )] = µt (x) and V[ft (xi )] = µt (xi )E−
t (xi ).
Taylor series approx
V[gλ (X)] = σX2 [g0λ (µX )]2
= σX2 µ2Xλ−2
Functional time series with applications in demography
1. Tools for functional time series analysis
8
Smoothing functional time series
Fertility
Bt (xi ) = number of births to women aged xi in year t.
Et (xi ) = total population aged xi on June 30 in year t.
ft (xi ) = Bt (xi )/Et (xi ) = observed fertility rate.
µt (xi ) = “true” fertility rate.
Bt (xi ) ∼ Pn(Et (xi )µt (xi ))
1
E[ft (xi )] = µt (x) and V[ft (xi )] = µt (xi )E−
t (xi ).
σ 2 (xi ) = V(gλ [ft (xi )])
1
= µt (xi )E−
(
x
)
µt (x)2λ−2 Taylor series approx
i
t
= µt (xi )2λ−1 Et (xi )−1
≈ ft (xi )2λ−1 Et (xi )−1
Functional time series with applications in demography
V[gλ (X)] = σX2 [g0λ (µX )]2
= σX2 µ2Xλ−2
1. Tools for functional time series analysis
8
Smoothing functional time series
2
France: male death rates (1821)
−2
−4
−8
−6
Log death rate
0
●●●●
●
●
●
●
● ●●●●●
●
●
●●
●
●●●●●●●●●●
●●
●●
●
●●
●●●
●●
●●
●
●●●
●
●
●
●
●●●
●
●●●
●●●●●
●
●●
●●●
●
●
●●●●●
●
●●●●●●● ●●●●●●●●●●●●
●
●●
●●
●
●
●●
● ●●
●●●
0
20
40
60
80
100
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
9
Smoothing functional time series
2
France: male death rates (1944)
−2
−4
−8
−6
Log death rate
0
●
● ●
●● ● ●
●● ● ● ●●●●●●●●●●
●●●
●●
●
●●
●●
●●●
●●●
●
●
●●●
●●●
●●●
●●●●●
●●●
●
●
●
●●●
●●
●
●●●
●●●●
●
●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●
●●●●●●●●●●●
0
20
40
60
80
100
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
9
Smoothing functional time series
2
France: male death rates (2012)
−2
−4
−8
−6
Log death rate
0
●
●
●
●
● ●●
●
●
●●
●●● ●
●●●
●●
●
●●
●●
●●
●●
●
●
●●
●●
●●
●
●
●●●
●●●
●●●●
●●●●
●
●
●●
●●
●●●
●●
●
●
●●
●●●
●●
●●
●●●●●
●●●●●●●
●●●●●●
●
●
●
●●
●
●
●●
●
●●
● ●●●● ●
0
20
40
60
80
100
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
9
Smoothing functional time series
−2
−4
−8
−6
Log death rate
0
2
France: male death rates (1816−2012)
0
20
40
60
80
100
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
10
Smoothing functional time series
−2
−4
−8
−6
Log death rate
0
2
France: male death rates (1816−2012)
0
20
40
60
80
100
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
10
Smoothing functional time series
150
●
● ●
●
● ●
●
●
● ●
●
●
●
●
●
●
100
Fertility rate
200
250
Australia fertility rates (1921)
● ●
●
●
50
●
●
●
●
●
●
●
0
●
●
15
●
●
20
25
30
35
40
● ●
● ●
45
50
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
11
Smoothing functional time series
200
250
Australia fertility rates (1950)
● ● ● ●
●
●
●
150
●
●
●
●
●
●
100
Fertility rate
●
●
●
●
●
●
●
50
●
●
●
●
0
●
●
15
●
●
20
25
30
35
40
●
● ●
● ● ● ●
45
50
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
11
Smoothing functional time series
150
● ● ●
●
●
100
Fertility rate
200
250
Australia fertility rates (2009)
●
●
●
●
●
●
●
●
●
50
●
●
●
●
●
●
●
●
●
●
0
●
●
● ●
15
20
25
30
35
40
● ●
● ● ● ● ●
45
50
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
11
Smoothing functional time series
150
100
0
50
Fertility rate
200
250
Australia fertility rates (1921−2009)
15
20
25
30
35
40
45
50
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
12
Smoothing functional time series
150
100
0
50
Fertility rate
200
250
Australia fertility rates (1921−2009)
15
20
25
30
35
40
45
50
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
12
Outline
1 Functional time series
2 Functional principal components
3 Data visualization
4 References
Functional time series with applications in demography
1. Tools for functional time series analysis
13
Functional principal components
yt (xi ) = st (xi ) + σt (xi )εt,i ,
st (x) = µ(x) +
T −1
X
βt,k φk (x)
k =1
1
2
3
Estimate smooth functions st (x) using weighted
penalized regression splines.
Compute µ(x) as s̄(x) across years.
Compute βt,k and φk (x) using functional
principal components.
Functional time series with applications in demography
1. Tools for functional time series analysis
14
µ(x)
−2
−4
−8
−6
Log death rate
0
2
France: male death rates (1816−2012)
0
20
40
60
80
100
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
15
µ(x)
150
100
0
50
Fertility rate
200
250
Australia fertility rates (1921−2009)
15
20
25
30
35
40
45
50
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
16
Functional principal components
(Ramsay and Silverman, 1997,2002).
In FDA, each principal component is specified
by a weight function φk (x).
The PC scoresZ for each year are given by
βk,t =
φk (x) [ŝt (x) − s̄(x)] dx
The aim is to:
1
2
3
Find the weight function φ1 (x) that maximizes the
variance
of β1,t subject to the constraint
R 2
φi (x)dx = 1.
Find the weight function φR2 (x) that maximizes the
2
Rvariance of β2,t such that φi (x)dx = 1 and
φ1 (x)φ2 (x)dx = 0.
Find the weight function φ3 (x) that . . .
Functional time series with applications in demography
1. Tools for functional time series analysis
17
Functional principal components
(Ramsay and Silverman, 1997,2002).
In FDA, each principal component is specified
by a weight function φk (x).
The PC scoresZ for each year are given by
βk,t =
φk (x) [ŝt (x) − s̄(x)] dx
The aim is to:
1
2
3
Find the weight function φ1 (x) that maximizes the
variance
of β1,t subject to the constraint
R 2
φi (x)dx = 1.
Find the weight function φR2 (x) that maximizes the
2
Rvariance of β2,t such that φi (x)dx = 1 and
φ1 (x)φ2 (x)dx = 0.
Find the weight function φ3 (x) that . . .
Functional time series with applications in demography
1. Tools for functional time series analysis
17
Functional principal components
(Ramsay and Silverman, 1997,2002).
In FDA, each principal component is specified
by a weight function φk (x).
The PC scoresZ for each year are given by
βk,t =
φk (x) [ŝt (x) − s̄(x)] dx
The aim is to:
1
2
3
Find the weight function φ1 (x) that maximizes the
variance
of β1,t subject to the constraint
R 2
φi (x)dx = 1.
Find the weight function φR2 (x) that maximizes the
2
Rvariance of β2,t such that φi (x)dx = 1 and
φ1 (x)φ2 (x)dx = 0.
Find the weight function φ3 (x) that . . .
Functional time series with applications in demography
1. Tools for functional time series analysis
17
Functional principal components
(Ramsay and Silverman, 1997,2002).
In FDA, each principal component is specified
by a weight function φk (x).
The PC scoresZ for each year are given by
βk,t =
φk (x) [ŝt (x) − s̄(x)] dx
The aim is to:
1
2
3
Find the weight function φ1 (x) that maximizes the
variance
of β1,t subject to the constraint
R 2
φi (x)dx = 1.
Find the weight function φR2 (x) that maximizes the
2
Rvariance of β2,t such that φi (x)dx = 1 and
φ1 (x)φ2 (x)dx = 0.
Find the weight function φ3 (x) that . . .
Functional time series with applications in demography
1. Tools for functional time series analysis
17
Functional principal components
(Ramsay and Silverman, 1997,2002).
In FDA, each principal component is specified
by a weight function φk (x).
The PC scoresZ for each year are given by
βk,t =
φk (x) [ŝt (x) − s̄(x)] dx
The aim is to:
1
2
3
Find the weight function φ1 (x) that maximizes the
variance
of β1,t subject to the constraint
R 2
φi (x)dx = 1.
Find the weight function φR2 (x) that maximizes the
2
Rvariance of β2,t such that φi (x)dx = 1 and
φ1 (x)φ2 (x)dx = 0.
Find the weight function φ3 (x) that . . .
Functional time series with applications in demography
1. Tools for functional time series analysis
17
Functional principal components
(Ramsay and Silverman, 1997,2002).
In FDA, each principal component is specified
by a weight function φk (x).
The PC scoresZ for each year are given by
βk,t =
φk (x) [ŝt (x) − s̄(x)] dx
The aim is to:
1
2
3
Find the weight function φ1 (x) that maximizes the
variance
of β1,t subject to the constraint
R 2
φi (x)dx = 1.
Find the weight function φR2 (x) that maximizes the
2
Rvariance of β2,t such that φi (x)dx = 1 and
φ1 (x)φ2 (x)dx = 0.
Find the weight function φ3 (x) that . . .
Functional time series with applications in demography
1. Tools for functional time series analysis
17
Functional principal components
The optimal basis functions
Approximate st (x) using
st (x) = s̄(x) +
K
X
βt,k φk (x) + rt (x)
k =0
The basis function φk (x) which minimizes
T
MISE =
1X
Z
rt2 dx is the kth principal component
T
t =1
(computed recursively).
Functional time series with applications in demography
1. Tools for functional time series analysis
18
Functional principal components
The optimal basis functions
Approximate st (x) using
st (x) = s̄(x) +
K
X
βt,k φk (x) + rt (x)
k =0
The basis function φk (x) which minimizes
T
MISE =
1X
Z
rt2 dx is the kth principal component
T
t =1
(computed recursively).
Functional time series with applications in demography
1. Tools for functional time series analysis
18
Functional principal components
Computationally equivalent approach
Let s∗t (x) = st (x) − s̄(x).
Discretize s∗t (x) on a dense grid of q equally spaced
points.
Denote discretized s∗t (x) as T × q matrix G.
SVD of G = ΦΛΨ0 where φk (x) is kth column of Φ.
βt,k is (t , k )th element of GΦ.
The basis functions are orthogonal.
This means the coefficients series are also
uncorrelated with each other. i.e., Corr(β̂t,i , β̂t,j ) = 0
for i 6= j. However, Corr(β̂t,i , β̂s,j ) 6= 0 in general for
t 6= s and i 6= j.
Functional time series with applications in demography
1. Tools for functional time series analysis
19
Functional principal components
Computationally equivalent approach
Let s∗t (x) = st (x) − s̄(x).
Discretize s∗t (x) on a dense grid of q equally spaced
points.
Denote discretized s∗t (x) as T × q matrix G.
SVD of G = ΦΛΨ0 where φk (x) is kth column of Φ.
βt,k is (t , k )th element of GΦ.
The basis functions are orthogonal.
This means the coefficients series are also
uncorrelated with each other. i.e., Corr(β̂t,i , β̂t,j ) = 0
for i 6= j. However, Corr(β̂t,i , β̂s,j ) 6= 0 in general for
t 6= s and i 6= j.
Functional time series with applications in demography
1. Tools for functional time series analysis
19
Functional principal components
Computationally equivalent approach
Let s∗t (x) = st (x) − s̄(x).
Discretize s∗t (x) on a dense grid of q equally spaced
points.
Denote discretized s∗t (x) as T × q matrix G.
SVD of G = ΦΛΨ0 where φk (x) is kth column of Φ.
βt,k is (t , k )th element of GΦ.
The basis functions are orthogonal.
This means the coefficients series are also
uncorrelated with each other. i.e., Corr(β̂t,i , β̂t,j ) = 0
for i 6= j. However, Corr(β̂t,i , β̂s,j ) 6= 0 in general for
t 6= s and i 6= j.
Functional time series with applications in demography
1. Tools for functional time series analysis
19
Functional principal components
Computationally equivalent approach
Let s∗t (x) = st (x) − s̄(x).
Discretize s∗t (x) on a dense grid of q equally spaced
points.
Denote discretized s∗t (x) as T × q matrix G.
SVD of G = ΦΛΨ0 where φk (x) is kth column of Φ.
βt,k is (t , k )th element of GΦ.
The basis functions are orthogonal.
This means the coefficients series are also
uncorrelated with each other. i.e., Corr(β̂t,i , β̂t,j ) = 0
for i 6= j. However, Corr(β̂t,i , β̂s,j ) 6= 0 in general for
t 6= s and i 6= j.
Functional time series with applications in demography
1. Tools for functional time series analysis
19
Functional principal components
Computationally equivalent approach
Let s∗t (x) = st (x) − s̄(x).
Discretize s∗t (x) on a dense grid of q equally spaced
points.
Denote discretized s∗t (x) as T × q matrix G.
SVD of G = ΦΛΨ0 where φk (x) is kth column of Φ.
βt,k is (t , k )th element of GΦ.
The basis functions are orthogonal.
This means the coefficients series are also
uncorrelated with each other. i.e., Corr(β̂t,i , β̂t,j ) = 0
for i 6= j. However, Corr(β̂t,i , β̂s,j ) 6= 0 in general for
t 6= s and i 6= j.
Functional time series with applications in demography
1. Tools for functional time series analysis
19
Functional principal components
Computationally equivalent approach
Let s∗t (x) = st (x) − s̄(x).
Discretize s∗t (x) on a dense grid of q equally spaced
points.
Denote discretized s∗t (x) as T × q matrix G.
SVD of G = ΦΛΨ0 where φk (x) is kth column of Φ.
βt,k is (t , k )th element of GΦ.
The basis functions are orthogonal.
This means the coefficients series are also
uncorrelated with each other. i.e., Corr(β̂t,i , β̂t,j ) = 0
for i 6= j. However, Corr(β̂t,i , β̂s,j ) 6= 0 in general for
t 6= s and i 6= j.
Functional time series with applications in demography
1. Tools for functional time series analysis
19
Functional principal components
Computationally equivalent approach
Let s∗t (x) = st (x) − s̄(x).
Discretize s∗t (x) on a dense grid of q equally spaced
points.
Denote discretized s∗t (x) as T × q matrix G.
SVD of G = ΦΛΨ0 where φk (x) is kth column of Φ.
βt,k is (t , k )th element of GΦ.
The basis functions are orthogonal.
This means the coefficients series are also
uncorrelated with each other. i.e., Corr(β̂t,i , β̂t,j ) = 0
for i 6= j. However, Corr(β̂t,i , β̂s,j ) 6= 0 in general for
t 6= s and i 6= j.
Functional time series with applications in demography
1. Tools for functional time series analysis
19
Functional principal components
Eigenvector approach
Let V = (T − 1)−1 G0 G be m × m sample covariance
matrix of G.
Let ΦK = [φ1 , . . . , φK ] consist of the first K
eigenvectors of V where K ≤ T − 1. The (i, j)th
element of ΦK is φi (xj∗ ).
Robust versions possible using robust “covariance”
estimation.
Functional time series with applications in demography
1. Tools for functional time series analysis
20
Functional principal components
Eigenvector approach
Let V = (T − 1)−1 G0 G be m × m sample covariance
matrix of G.
Let ΦK = [φ1 , . . . , φK ] consist of the first K
eigenvectors of V where K ≤ T − 1. The (i, j)th
element of ΦK is φi (xj∗ ).
Robust versions possible using robust “covariance”
estimation.
Functional time series with applications in demography
1. Tools for functional time series analysis
20
Functional principal components
Eigenvector approach
Let V = (T − 1)−1 G0 G be m × m sample covariance
matrix of G.
Let ΦK = [φ1 , . . . , φK ] consist of the first K
eigenvectors of V where K ≤ T − 1. The (i, j)th
element of ΦK is φi (xj∗ ).
Robust versions possible using robust “covariance”
estimation.
Functional time series with applications in demography
1. Tools for functional time series analysis
20
Functional principal components
0.20
20 40 60 80
0
20 40 60 80
Age
0
Coefficient 3
4
−2
−2
0
2
Coefficient 2
1
6
10
8
2
Age
5
0
−5
0.10
−0.10
0
Age
Coefficient 1
0.00
Basis function 3
0.1
20 40 60 80
−15 −10
French
male
mortality
0.0
Basis function 2
−0.2
0
Age
−1
20 40 60 80
−0.1
0.20
0.15
0.10
Basis function 1
0.00
0
0.05
−2
−3
−4
−5
−6
Mean
0.2
Interaction
−1
Main effects
1850
1950
Time
Functional time series with applications in demography
1850
1950
1850
Time
1. Tools for functional time series analysis
1950
Time
21
Functional principal components
0
20 40 60 80
0.3
0.2
0.1
Basis function 3
0.0
0
20 40 60 80
Age
Age
1.0
4
0.0
−1.0
Coefficient 3
2
0
−2
Coefficient 2
−2.0
−4
−6
−15
Robust
PCA
20 40 60 80
5
0
−5
Coefficient 1
−10
French
male
mortality
0
Age
10
Age
−0.2
20 40 60 80
0.10
−0.10
−0.05
0
0.00
Basis function 2
0.15
Basis function 1
0.05
−1
−2
−3
Mean
−4
−6
−5
Interaction
0.25
Main effects
1850
1950
Time
Functional time series with applications in demography
1850
1950
1850
Time
1. Tools for functional time series analysis
1950
Time
21
Functional principal components
0.2
0.1
−0.1
Basis function 3
0.2
0.1
Basis function 2
0.0
0.3
0.2
Basis function 1
−0.1
0.0
35
45
15
35
45
15
25
35
45
Age
1960
2000
Time
Functional time series with applications in demography
0
1920
5
0
10
20
10
10 20 30 40
1920
25
Age
−10 0
Coefficient 1
Age
−30
Australian
fertility
25
Coefficient 3
15
−5
45
Coefficient 2
35
Age
−10
25
−20
15
−0.3
5
0.1
10 15 20 25 30
0.3
Interaction
0
Mean
Main effects
1960
2000
1920
Time
1. Tools for functional time series analysis
1960
2000
Time
22
Functional principal components
Robust
PCA
45
0.1
15
45
15
2000
Time
Functional time series with applications in demography
1920
35
45
10
Coefficient 3
10
20
1960
25
Age
0
10 20 30
−5
Coefficient 1
35
Age
−10 0
1920
25
5
35
Age
0
25
−30
Australian
fertility
−0.3
−0.1
15
−10
45
Coefficient 2
35
Age
−10
25
−20
15
−0.1
Basis function 3
0.2
0.1
Basis function 2
0.0
0.2
0.1
0.0
5
Basis function 1
0.3
0.3
0.2
Interaction
0
Mean
10 15 20 25 30
Main effects
1960
2000
1920
Time
1. Tools for functional time series analysis
1960
2000
Time
22
Outline
1 Functional time series
2 Functional principal components
3 Data visualization
4 References
Functional time series with applications in demography
1. Tools for functional time series analysis
23
French male mortality rates
−4
−6
−8
Log death rate
−2
0
France: male death rates (1816−2012)
0
20
40
60
80
100
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
24
French male mortality rates
−4
−6
Rainbow plot
−8
Log death rate
−2
0
France: male death rates (1816−2012)
Order of curves indicated by
rainbow colors.
0
20
40
60
80
100
Other orderings are possible.
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
24
Order by functional depth
Febrero, Galeano and Gonzalez-Manteiga (2007)
proposed:
Z
ot =
D(yt (x)) dx
where D(yt (x)) is a univariate depth measure for
each x.
ot provides an ordering of curves by “functional
depth”.
Problem: may not detect shape outliers.
Functional time series with applications in demography
1. Tools for functional time series analysis
25
Order by functional depth
Febrero, Galeano and Gonzalez-Manteiga (2007)
proposed:
Z
ot =
D(yt (x)) dx
where D(yt (x)) is a univariate depth measure for
each x.
ot provides an ordering of curves by “functional
depth”.
Problem: may not detect shape outliers.
Functional time series with applications in demography
1. Tools for functional time series analysis
25
Bivariate functional depth
Alternative: Apply bivariate depth measures to
first two PC scores.
Plot βt,2 vs βt,1
å Each point in scatterplot represents one curve.
å Outliers show up in bivariate score space.
å Curves can be ordered by bivariate depth.
Functional time series with applications in demography
1. Tools for functional time series analysis
26
Bivariate functional depth
Alternative: Apply bivariate depth measures to
first two PC scores.
Plot βt,2 vs βt,1
å Each point in scatterplot represents one curve.
å Outliers show up in bivariate score space.
å Curves can be ordered by bivariate depth.
Functional time series with applications in demography
1. Tools for functional time series analysis
26
Bivariate functional depth
Alternative: Apply bivariate depth measures to
first two PC scores.
Plot βt,2 vs βt,1
å Each point in scatterplot represents one curve.
å Outliers show up in bivariate score space.
å Curves can be ordered by bivariate depth.
Functional time series with applications in demography
1. Tools for functional time series analysis
26
Robust PC scores
Scatterplot of first two PC scores
●●
●
●
●
●
●
●●
5
●
−5
0
PC score 2
10
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●● ● ● ●
●
● ●●
●
●●
●●
●●
●
●
● ● ●●
●
●
●
●●● ●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●● ●
● ●●
●
●● ● ●●
● ● ●●
●●
●
●
●
●●
●●
●
●
●
●
●
●
● ●
−5
●
●
●
●
●
●
●
●
●● ●
●
● ●
● ●
● ●●● ●
●●●
●● ● ●
●
●
●
0
5
10
15
PC score 1
Functional time series with applications in demography
1. Tools for functional time series analysis
27
Robust PC scores
Scatterplot of first two PC scores
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
5
10
●
●
−5
0
PC score 2
●●
●
●
●
●
●
●●
WW1
WW2
1800s
2000s
●
●
● ●
●● ● ● ●
●
● ●●
●
●●
●●
●●
●
●
● ● ●●
●
●
●
●●● ●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●● ●
● ●●
●
●● ● ●●
● ● ●●
●●
●
●
●
●●
●●
●
●
●
●
●
●
● ●
−5
●
●
●
●
●
●
●
●
●● ●
●
● ●
● ●
● ●●● ●
●●●
●● ● ●
●
●
●
0
5
10
15
PC score 1
Functional time series with applications in demography
1. Tools for functional time series analysis
27
Robust PC scores
5
10
5
0
−5
Comp.2
−5
Comp.1
15
PC scores
1850
1900
1950
2000
Time
Functional time series with applications in demography
1. Tools for functional time series analysis
27
French male mortality rates
1
0
−1
−2
Differenced log death rate
2
France: differenced male death rates (1816−2012)
0
20
40
60
80
100
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
28
Robust PC scores
2
−2 0
4
0
−4
Comp.2
Comp.1
4
PC scores on differences
1850
1900
1950
2000
Time
Functional time series with applications in demography
1. Tools for functional time series analysis
29
Robust PC scores
Scatterplot of first two PC scores on differences
6
●
4
●
●
●
2
●
●
●
●
0
●
●
●
●
●
● ●●●●
●
● ●●●●
●
●
●●●●● ●
●●
●
●
●●
●
●
●●●
●
●
●
●●●
●●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●● ● ●
● ●
●● ●
●
●
●
●●
●
●
● ●
●
● ● ●●● ●
●●
●
●
−2
PC score 2
● ●
●
●
−4
●●
●
●
−2
0
2
4
PC score 1
Functional time series with applications in demography
1. Tools for functional time series analysis
29
Robust PC scores
Scatterplot of first two PC scores on differences
6
●
4
●
●
●
2
●
●
●
●
0
●
●
●
●
●
● ●●●●
●
● ●●●●
●
●
●●●●● ●
●●
●
●
●●
●
●
●●●
●
●
●
●●●
●●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●● ● ●
● ●
●● ●
●
●
●
●●
●
●
● ●
●
● ● ●●● ●
●●
●
●
−2
PC score 2
● ●
●
●
−4
●
●
●
WW1
WW2
1800s
2000s
●
●●
●
●
−2
0
2
4
PC score 1
Functional time series with applications in demography
1. Tools for functional time series analysis
29
Halfspace location depth
The halfspace depth of a point q:
(Due to Hotelling, 1929; Tukey, 1975)
For each closed halfspace that contains q,
count number of observations not in halfspace.
The minimum over all halfspaces is the depth
of that point.
The median is the point with maximum depth
(not generally unique).
Any point outside convex hull of the data has
depth zero.
Functional time series with applications in demography
1. Tools for functional time series analysis
30
Halfspace location depth
The halfspace depth of a point q:
(Due to Hotelling, 1929; Tukey, 1975)
For each closed halfspace that contains q,
count number of observations not in halfspace.
The minimum over all halfspaces is the depth
of that point.
The median is the point with maximum depth
(not generally unique).
Any point outside convex hull of the data has
depth zero.
Functional time series with applications in demography
1. Tools for functional time series analysis
30
Halfspace location depth
The halfspace depth of a point q:
(Due to Hotelling, 1929; Tukey, 1975)
For each closed halfspace that contains q,
count number of observations not in halfspace.
The minimum over all halfspaces is the depth
of that point.
The median is the point with maximum depth
(not generally unique).
Any point outside convex hull of the data has
depth zero.
Functional time series with applications in demography
1. Tools for functional time series analysis
30
Ordering by halfspace depth
−4
−6
−8
Log death rate
−2
0
France: male death rates (1816−2012)
0
20
40
60
80
100
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
31
Bivariate bagplot
Due to Rousseeuw, Ruts & Tukey (Am.Stat. 1999).
Rank points by halfspace location depth.
Display median, 50% convex hull and outer
convex hull (with 99% coverage if bivariate
1914
normal).
6
●
●
●
1943
●
0
2
●●
1944
18711918
●
●
1854
1870
1850
1920 ●
−2
PC score 2
4
1940
●
−4
1946 ●●
1941
1945
1919
−4
−2
0
●
1872
●
●
2
4
PC score 1
Functional time series with applications in demography
1. Tools for functional time series analysis
32
Bivariate bagplot
Due to Rousseeuw, Ruts & Tukey (Am.Stat. 1999).
Rank points by halfspace location depth.
Display median, 50% convex hull and outer
convex hull (with 99% coverage if bivariate
normal).
Boundaries contain all curves inside bags.
95% CI for median curve also shown.
0
●
6
1914
●
1850
1854
1870
1871
1872
1914
1918
1919
●
−8
−2
1850
1920 ●
−4
●
Log death rate
1943
−6
●
0
2
●●
1944
18711918
●
●
1854
1870
1946 ●●
1941
1945
−4
PC score 2
4
−2
1940
1919
−4
−2
0
2
●
1872
●
●
4
PC score 1
Functional time series with applications in demography
0
20
40
60
1920
1940
1941
1943
1944
1945
1946
80
100
Age
1. Tools for functional time series analysis
32
−4
−6
1850
1854
1870
1871
1872
1914
1918
1919
−8
Log death rate
−2
0
Functional bagplot
0
20
40
60
80
1920
1940
1941
1943
1944
1945
1946
100
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
33
−4
−6
−8
Log death rate
−2
0
Functional bagplot
0
1850, 1854: ?
1870–1871: Franco-Prussian war
1914–1919: WW1
1939–1945: WW2
20
40
60
1850
1854
1870
1871
1872
1914
1918
1919
80
1920
1940
1941
1943
1944
1945
1946
100
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
33
Kernel density estimate
Scatterplot of first two PC scores on differences
6
●
4
●
●
●
2
●
●
●
●
0
●
●
●
●
●
● ●●●●
●
● ●●●●
●
●
●●●●● ●
●●
●
●
●●
●
●
●●●
●
●
●
●●●
●●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●● ● ●
● ●
●● ●
●
●
●
●●
●
●
● ●
●
● ● ●●● ●
●●
●
●
−2
PC score 2
● ●
●
●
−4
●●
●
●
−2
0
2
4
PC score 1
Functional time series with applications in demography
1. Tools for functional time series analysis
34
Kernel density estimate
Functional time series with applications in demography
1. Tools for functional time series analysis
34
Ordering by bivariate density
−4
−6
−8
Log death rate
−2
0
France: male death rates (1816−2012)
0
20
40
60
80
100
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
35
Functional HDR boxplot
Bivariate HDR boxplot due to Hyndman (1996).
Rank points by value of kernel density estimate.
Display mode, 50% and (usually) 99% highest
density regions (HDRs) and mode.
●
6
1914
93% outer region
●
2
●
1944 1918
●
●
1854
1870
●
1943
●
0
●
1850
1920 ●
−2
PC score 2
4
1940
●
−4
1946 ●●
1941
1945
1919 ●
−4
−2
0
2
●
1872
●
4
PC score 1
Functional time series with applications in demography
1. Tools for functional time series analysis
36
Functional HDR boxplot
●
6
1914
0
Bivariate HDR boxplot due to Hyndman (1996).
Rank points by value of kernel density estimate.
Display mode, 50% and (usually) 99% highest
density regions (HDRs) and mode.
Boundaries contain all curves inside HDRs.
93% outer region
●
●
0
●
1850
1854
1870
1872
1914
1918
1919
●
−8
−2
1850
1920 ●
−4
1943
Log death rate
●
−6
2
●
1944 1918
●
●
1854
1870
1946 ●●
1941
1945
1919 ●
−4
PC score 2
4
−2
1940
−4
−2
0
2
●
1872
●
4
PC score 1
Functional time series with applications in demography
0
20
40
60
1920
1940
1941
1943
1944
1945
1946
80
100
Age
1. Tools for functional time series analysis
36
−4
−6
1850
1854
1870
1872
1914
1918
1919
−8
Log death rate
−2
0
Functional HDR boxplot
0
20
40
60
80
1920
1940
1941
1943
1944
1945
1946
100
Age
Functional time series with applications in demography
1. Tools for functional time series analysis
37
Outline
1 Functional time series
2 Functional principal components
3 Data visualization
4 References
Functional time series with applications in demography
1. Tools for functional time series analysis
38
Selected references
Hyndman, Shang (2010). “Rainbow plots,
bagplots and boxplots for functional data”.
Journal of Computational and Graphical
Statistics 19(1), 29–45
Hyndman, Ullah (2007). “Robust forecasting of
mortality and fertility rates: A functional data
approach”. Computational Statistics & Data
Analysis 51(10), 4942–4956.
Hyndman (2014). demography: Forecasting
mortality, fertility, migration and population
data.
cran.r-project.org/package=demography
Functional time series with applications in demography
1. Tools for functional time series analysis
39