Rob J Hyndman
Functional time series
with applications in demography
6. Coherent functional forecasting
Outline
1 Forecasting groups
2 Automatic ARFIMA forecasting
3 Coherent cohort life expectancy forecasts
4 Coherent forecasts for J > 2 groups
5 Forecasting state mortality
6 References
Functional time series with applications in demography
6. Coherent functional forecasting
2
The problem
Let st,j (x) be the smoothed mortality rate for age x
in group j in year t.
Groups may be males and females.
Groups may be states within a country.
Expected that groups will behave similarly.
Coherent forecasts do not diverge over time.
Existing functional models do not impose
coherence.
Functional time series with applications in demography
6. Coherent functional forecasting
3
The problem
Let st,j (x) be the smoothed mortality rate for age x
in group j in year t.
Groups may be males and females.
Groups may be states within a country.
Expected that groups will behave similarly.
Coherent forecasts do not diverge over time.
Existing functional models do not impose
coherence.
Functional time series with applications in demography
6. Coherent functional forecasting
3
The problem
Let st,j (x) be the smoothed mortality rate for age x
in group j in year t.
Groups may be males and females.
Groups may be states within a country.
Expected that groups will behave similarly.
Coherent forecasts do not diverge over time.
Existing functional models do not impose
coherence.
Functional time series with applications in demography
6. Coherent functional forecasting
3
The problem
Let st,j (x) be the smoothed mortality rate for age x
in group j in year t.
Groups may be males and females.
Groups may be states within a country.
Expected that groups will behave similarly.
Coherent forecasts do not diverge over time.
Existing functional models do not impose
coherence.
Functional time series with applications in demography
6. Coherent functional forecasting
3
The problem
Let st,j (x) be the smoothed mortality rate for age x
in group j in year t.
Groups may be males and females.
Groups may be states within a country.
Expected that groups will behave similarly.
Coherent forecasts do not diverge over time.
Existing functional models do not impose
coherence.
Functional time series with applications in demography
6. Coherent functional forecasting
3
The problem
Let st,j (x) be the smoothed mortality rate for age x
in group j in year t.
Groups may be males and females.
Groups may be states within a country.
Expected that groups will behave similarly.
Coherent forecasts do not diverge over time.
Existing functional models do not impose
coherence.
Functional time series with applications in demography
6. Coherent functional forecasting
3
Forecasting the coefficients
yt,x = st (x) + σt (x)εt,x ,
st (x) = µ(x) +
K
X
βt,k φk (x) + rt (x)
k =1
We use ARIMA or ETS models for each
coefficient {β1,j,k , . . . , βn,j,k }.
The ARIMA models are non-stationary for the
first few coefficients (k = 1, 2). All ETS models
are non-stationary.
Non-stationary forecasts will diverge. Hence
the mortality forecasts are not coherent.
Functional time series with applications in demography
6. Coherent functional forecasting
4
Forecasting the coefficients
yt,x = st (x) + σt (x)εt,x ,
st (x) = µ(x) +
K
X
βt,k φk (x) + rt (x)
k =1
We use ARIMA or ETS models for each
coefficient {β1,j,k , . . . , βn,j,k }.
The ARIMA models are non-stationary for the
first few coefficients (k = 1, 2). All ETS models
are non-stationary.
Non-stationary forecasts will diverge. Hence
the mortality forecasts are not coherent.
Functional time series with applications in demography
6. Coherent functional forecasting
4
Forecasting the coefficients
yt,x = st (x) + σt (x)εt,x ,
st (x) = µ(x) +
K
X
βt,k φk (x) + rt (x)
k =1
We use ARIMA or ETS models for each
coefficient {β1,j,k , . . . , βn,j,k }.
The ARIMA models are non-stationary for the
first few coefficients (k = 1, 2). All ETS models
are non-stationary.
Non-stationary forecasts will diverge. Hence
the mortality forecasts are not coherent.
Functional time series with applications in demography
6. Coherent functional forecasting
4
0.2
0.15
φ3(x)
−0.15
−0.1
0.0
−0.05
φ2(x)
0.1
0.05
0.20
0.15
φ1(x)
0.10
0.05
µ(x)
−8 −7 −6 −5 −4 −3 −2 −1
Male fts model
0
20 40 60 80
0
20 40 60 80
0
20 40 60 80
Age
0
20 40 60 80
Age
Age
−25
βt3
−1
−15
−2
−15
−10
βt2
βt1
−5
−5
0
0
0
5
1
Age
1950
2050
Time
Functional time series with applications in demography
1950
2050
1950
Time
6. Coherent functional forecasting
2050
Time
5
20 40 60 80
0.0
0
20 40 60 80
−0.2
φ3(x)
0
20 40 60 80
Age
0
20 40 60 80
Age
Age
βt3
−1.0 −0.5
−10
−30
1950
2050
Time
Functional time series with applications in demography
0.0
0.5
0
−5
βt2
−10
−20
βt1
0
1.0
5
10
Age
−0.4
−0.15
−8
0
−0.05
φ2(x)
0.05
0.15
0.05
0.10
φ1(x)
−4
−6
µ(x)
−2
Female fts model
1950
2050
1950
Time
6. Coherent functional forecasting
2050
Time
6
Australian mortality forecasts
−2
−4
−10
−8
−6
Log death rate
−4
−6
−10
−8
Log death rate
−2
0
(b) Females
0
(a) Males
0
20
40
60
80
100
Age
Functional time series with applications in demography
0
20
40
60
80
100
Age
6. Coherent functional forecasting
7
Mortality product and ratios
Key idea
Model the geometric mean and the mortality ratio
instead of the individual rates for each sex
separately.
pt (x) =
q
st,M (x)st,F (x) and
rt (x) =
q
st,M (x) st,F (x).
Product and ratio are approximately
independent
Ratio should be stationary (for coherence) but
product can be non-stationary.
Functional time series with applications in demography
6. Coherent functional forecasting
8
Mortality product and ratios
Key idea
Model the geometric mean and the mortality ratio
instead of the individual rates for each sex
separately.
pt (x) =
q
st,M (x)st,F (x) and
rt (x) =
q
st,M (x) st,F (x).
Product and ratio are approximately
independent
Ratio should be stationary (for coherence) but
product can be non-stationary.
Functional time series with applications in demography
6. Coherent functional forecasting
8
Mortality product and ratios
Key idea
Model the geometric mean and the mortality ratio
instead of the individual rates for each sex
separately.
pt (x) =
q
st,M (x)st,F (x) and
rt (x) =
q
st,M (x) st,F (x).
Product and ratio are approximately
independent
Ratio should be stationary (for coherence) but
product can be non-stationary.
Functional time series with applications in demography
6. Coherent functional forecasting
8
Mortality product and ratios
Key idea
Model the geometric mean and the mortality ratio
instead of the individual rates for each sex
separately.
pt (x) =
q
st,M (x)st,F (x) and
rt (x) =
q
st,M (x) st,F (x).
Product and ratio are approximately
independent
Ratio should be stationary (for coherence) but
product can be non-stationary.
Functional time series with applications in demography
6. Coherent functional forecasting
8
Product data
−2
−4
−6
−8
Log of geometric mean death rate
0
Australia: product data
0
20
40
60
80
100
Age
Functional time series with applications in demography
6. Coherent functional forecasting
9
Ratio data
3
2
1
Sex ratio of rates: M/F
4
Australia: mortality sex ratio (1921−2009)
0
20
40
60
80
100
Age
Functional time series with applications in demography
6. Coherent functional forecasting
10
Model product and ratios
pt (x) =
q
st,M (x)st,F (x) and
log[pt (x)] = µp (x) +
rt (x) =
K
X
q
st,M (x) st,F (x).
βt,k φk (x) + et (x)
k =1
L
log[rt (x)] = µr (x) +
X
γt,` ψ` (x) + wt (x).
`=1
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d, q).
No restrictions for βt,1 , . . . , βt,K .
Forecasts: sn+h|n,M (x) = pn+h|n (x)rn+h|n (x)
sn+h|n,F (x) = pn+h|n (x) rn+h|n (x).
Functional time series with applications in demography
6. Coherent functional forecasting
11
Model product and ratios
pt (x) =
q
st,M (x)st,F (x) and
log[pt (x)] = µp (x) +
rt (x) =
K
X
q
st,M (x) st,F (x).
βt,k φk (x) + et (x)
k =1
L
log[rt (x)] = µr (x) +
X
γt,` ψ` (x) + wt (x).
`=1
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d, q).
No restrictions for βt,1 , . . . , βt,K .
Forecasts: sn+h|n,M (x) = pn+h|n (x)rn+h|n (x)
sn+h|n,F (x) = pn+h|n (x) rn+h|n (x).
Functional time series with applications in demography
6. Coherent functional forecasting
11
Model product and ratios
pt (x) =
q
st,M (x)st,F (x) and
log[pt (x)] = µp (x) +
rt (x) =
K
X
q
st,M (x) st,F (x).
βt,k φk (x) + et (x)
k =1
L
log[rt (x)] = µr (x) +
X
γt,` ψ` (x) + wt (x).
`=1
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d, q).
No restrictions for βt,1 , . . . , βt,K .
Forecasts: sn+h|n,M (x) = pn+h|n (x)rn+h|n (x)
sn+h|n,F (x) = pn+h|n (x) rn+h|n (x).
Functional time series with applications in demography
6. Coherent functional forecasting
11
Model product and ratios
pt (x) =
q
st,M (x)st,F (x) and
log[pt (x)] = µp (x) +
rt (x) =
K
X
q
st,M (x) st,F (x).
βt,k φk (x) + et (x)
k =1
L
log[rt (x)] = µr (x) +
X
γt,` ψ` (x) + wt (x).
`=1
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d, q).
No restrictions for βt,1 , . . . , βt,K .
Forecasts: sn+h|n,M (x) = pn+h|n (x)rn+h|n (x)
sn+h|n,F (x) = pn+h|n (x) rn+h|n (x).
Functional time series with applications in demography
6. Coherent functional forecasting
11
20 40 60 80
0.10
0
20 40 60 80
0
20 40 60 80
Age
Age
4
Age
2
βt3
−0.5
βt2
1920
1980
2040
Year
Functional time series with applications in demography
−4
−20
−1.5
−2
0
5
0
−5
−10
βt1
0.00
φ3(x)
−0.10
−0.20
−0.1
0
Age
−0.2
20 40 60 80
0.5 1.0
0
0.0
φ2(x)
0.10
0.05
φ1(x)
−4
−8
−6
µP(x)
0.1
−2
0.15
0.2
Product model
1920
1980
2040
1920
Year
6. Coherent functional forecasting
1980
2040
Year
12
20 40 60 80
0.1 0.2 0.3
0.20
20 40 60 80
φ3(x)
−0.1
−0.3
0.00
0
0
Age
20 40 60 80
0
Age
0.4
1.0
1980
2040
Year
Functional time series with applications in demography
−2.0
βt3
−1.0
−0.4 −0.2
βt2
0.0
0.2
0.2
0.0
βt1
−0.2
−0.6
1920
20 40 60 80
Age
0.4
Age
0.0
0
0.10
φ2(x)
0.1
φ1(x)
0.0
−0.1
0.2
0.1
µR(x)
0.3
0.2
0.4
Ratio model
1920
1980
2040
1920
Year
6. Coherent functional forecasting
1980
2040
Year
13
−4
−6
−8
−10
Log of geometric mean death rate
−2
Product forecasts
0
20
40
60
80
100
Age
Functional time series with applications in demography
6. Coherent functional forecasting
14
2
1
Sex ratio: M/F
3
4
Ratio forecasts
0
20
40
60
80
100
Age
Functional time series with applications in demography
6. Coherent functional forecasting
15
Coherent forecasts
−4
−10
−8
−6
Log death rate
−6
−10
−8
Log death rate
−4
−2
(b) Females
−2
(a) Males
0
20
40
60
80
100
Age
Functional time series with applications in demography
0
20
40
60
80
100
Age
6. Coherent functional forecasting
16
Ratio forecasts
6
5
4
3
0
1
2
Sex ratio of rates: M/F
5
4
3
2
0
1
Sex ratio of rates: M/F
6
7
Coherent forecasts
7
Independent forecasts
0
20
40
60
80
100
Age
Functional time series with applications in demography
0
20
40
60
80
100
Age
6. Coherent functional forecasting
17
Life expectancy forecasts
Life expectancy difference: F−M
6
5
Number of years
8080
70
70
4
7575
Age
85 85
7
90 90 95
95
Life expectancy forecasts
1920
1960
2000
2040
Year
Functional time series with applications in demography
1960
1980
2000
2020
Year
6. Coherent functional forecasting
18
Life expectancy forecasts
a. USA
b. Canada
8
8
6
6
4
Years
Years
0
25
50
0
4
25
50
2
2
75
75
85
0
85
0
95
1960
1980
2000
95
2020
1960
1980
Year
c. UK
8
6
0
Years
Years
6
25
50
2
0
25
4
50
2
75
75
85
0
2020
d. France
8
4
2000
Year
85
95
0
1960
1980
2000
2020
Year
Functional time series with applications in demography
95
1960
1980
2000
2020
Year
6. Coherent functional forecasting
19
Li-Lee method
Li & Lee (Demography, 2005) method is a special
case of our approach.
st,j (x) = µj (x) + βt φ(x) + γt,j ψj (x) + et,j (x)
where f is unsmoothed log mortality rate, βt is a
random walk with drift and γt,j is AR(1) process.
No smoothing.
Only one basis function for each part,
Random walk with drift very limiting.
AR(1) very limiting.
The γt,j coefficients will be highly correlated
with each other, and so independent models
are not appropriate.
Functional time series with applications in demography
6. Coherent functional forecasting
20
Li-Lee method
Li & Lee (Demography, 2005) method is a special
case of our approach.
st,j (x) = µj (x) + βt φ(x) + γt,j ψj (x) + et,j (x)
where f is unsmoothed log mortality rate, βt is a
random walk with drift and γt,j is AR(1) process.
No smoothing.
Only one basis function for each part,
Random walk with drift very limiting.
AR(1) very limiting.
The γt,j coefficients will be highly correlated
with each other, and so independent models
are not appropriate.
Functional time series with applications in demography
6. Coherent functional forecasting
20
Li-Lee method
Li & Lee (Demography, 2005) method is a special
case of our approach.
st,j (x) = µj (x) + βt φ(x) + γt,j ψj (x) + et,j (x)
where f is unsmoothed log mortality rate, βt is a
random walk with drift and γt,j is AR(1) process.
No smoothing.
Only one basis function for each part,
Random walk with drift very limiting.
AR(1) very limiting.
The γt,j coefficients will be highly correlated
with each other, and so independent models
are not appropriate.
Functional time series with applications in demography
6. Coherent functional forecasting
20
Li-Lee method
Li & Lee (Demography, 2005) method is a special
case of our approach.
st,j (x) = µj (x) + βt φ(x) + γt,j ψj (x) + et,j (x)
where f is unsmoothed log mortality rate, βt is a
random walk with drift and γt,j is AR(1) process.
No smoothing.
Only one basis function for each part,
Random walk with drift very limiting.
AR(1) very limiting.
The γt,j coefficients will be highly correlated
with each other, and so independent models
are not appropriate.
Functional time series with applications in demography
6. Coherent functional forecasting
20
Li-Lee method
Li & Lee (Demography, 2005) method is a special
case of our approach.
st,j (x) = µj (x) + βt φ(x) + γt,j ψj (x) + et,j (x)
where f is unsmoothed log mortality rate, βt is a
random walk with drift and γt,j is AR(1) process.
No smoothing.
Only one basis function for each part,
Random walk with drift very limiting.
AR(1) very limiting.
The γt,j coefficients will be highly correlated
with each other, and so independent models
are not appropriate.
Functional time series with applications in demography
6. Coherent functional forecasting
20
Outline
1 Forecasting groups
2 Automatic ARFIMA forecasting
3 Coherent cohort life expectancy forecasts
4 Coherent forecasts for J > 2 groups
5 Forecasting state mortality
6 References
Functional time series with applications in demography
6. Coherent functional forecasting
21
ARFIMA model
Non-seasonal ARIMA model
φp (B)(1 − B)d yt = c + θq (B)εt
p, d and q are integer.
An ARFIMA model is identical except that d > − 12 can
be non-integer.
(1 − B)d is given by the binomial expansion (|d| < 12 )
d
(1 − B) = 1 +
∞
X
πj
j=1
− 12 < d < 0
0 < d < 12
d ≥ 12
j!
j
B
πj =
j
Y
(k − 1 − d)
k =1
yt is stationary with “anti-persistence”
yt is stationary with “long memory”
yt is non-stationary
Functional time series with applications in demography
6. Coherent functional forecasting
22
ARFIMA model
Non-seasonal ARIMA model
φp (B)(1 − B)d yt = c + θq (B)εt
p, d and q are integer.
An ARFIMA model is identical except that d > − 12 can
be non-integer.
(1 − B)d is given by the binomial expansion (|d| < 12 )
d
(1 − B) = 1 +
∞
X
πj
j=1
− 12 < d < 0
0 < d < 12
d ≥ 12
j!
j
B
πj =
j
Y
(k − 1 − d)
k =1
yt is stationary with “anti-persistence”
yt is stationary with “long memory”
yt is non-stationary
Functional time series with applications in demography
6. Coherent functional forecasting
22
ARFIMA model
Non-seasonal ARIMA model
φp (B)(1 − B)d yt = c + θq (B)εt
p, d and q are integer.
An ARFIMA model is identical except that d > − 12 can
be non-integer.
(1 − B)d is given by the binomial expansion (|d| < 12 )
d
(1 − B) = 1 +
∞
X
πj
j=1
− 12 < d < 0
0 < d < 12
d ≥ 12
j!
j
B
πj =
j
Y
(k − 1 − d)
k =1
yt is stationary with “anti-persistence”
yt is stationary with “long memory”
yt is non-stationary
Functional time series with applications in demography
6. Coherent functional forecasting
22
ARFIMA model identification
Long-memory ARFIMA model
φp (B)(1 − B)d yt = c + θq (B)εt
p and q are integer, 0 < d < 21 .
d can be estimated by MLE if p and q are known
(Haslett & Raftery, 1989).
So we set p = 2 and q = 0, and estimate d
constrained to (0, 0.5).
Then fix d, and select p and q using
auto.arima().
Automated in arfima().
Functional time series with applications in demography
6. Coherent functional forecasting
23
ARFIMA model identification
Long-memory ARFIMA model
φp (B)(1 − B)d yt = c + θq (B)εt
p and q are integer, 0 < d < 21 .
d can be estimated by MLE if p and q are known
(Haslett & Raftery, 1989).
So we set p = 2 and q = 0, and estimate d
constrained to (0, 0.5).
Then fix d, and select p and q using
auto.arima().
Automated in arfima().
Functional time series with applications in demography
6. Coherent functional forecasting
23
ARFIMA model identification
Long-memory ARFIMA model
φp (B)(1 − B)d yt = c + θq (B)εt
p and q are integer, 0 < d < 21 .
d can be estimated by MLE if p and q are known
(Haslett & Raftery, 1989).
So we set p = 2 and q = 0, and estimate d
constrained to (0, 0.5).
Then fix d, and select p and q using
auto.arima().
Automated in arfima().
Functional time series with applications in demography
6. Coherent functional forecasting
23
ARFIMA model identification
Long-memory ARFIMA model
φp (B)(1 − B)d yt = c + θq (B)εt
p and q are integer, 0 < d < 21 .
d can be estimated by MLE if p and q are known
(Haslett & Raftery, 1989).
So we set p = 2 and q = 0, and estimate d
constrained to (0, 0.5).
Then fix d, and select p and q using
auto.arima().
Automated in arfima().
Functional time series with applications in demography
6. Coherent functional forecasting
23
20 40 60 80
0.1 0.2 0.3
0.20
20 40 60 80
φ3(x)
−0.1
−0.3
0.00
0
0
Age
20 40 60 80
0
Age
0.4
1.0
1980
2040
Year
Functional time series with applications in demography
−2.0
βt3
−1.0
−0.4 −0.2
βt2
0.0
0.2
0.2
0.0
βt1
−0.2
−0.6
1920
20 40 60 80
Age
0.4
Age
0.0
0
0.10
φ2(x)
0.1
φ1(x)
0.0
−0.1
0.2
0.1
µR(x)
0.3
0.2
0.4
Ratio model
1920
1980
2040
1920
Year
6. Coherent functional forecasting
1980
2040
Year
24
Outline
1 Forecasting groups
2 Automatic ARFIMA forecasting
3 Coherent cohort life expectancy forecasts
4 Coherent forecasts for J > 2 groups
5 Forecasting state mortality
6 References
Functional time series with applications in demography
6. Coherent functional forecasting
25
Life expectancy (recap)
m(x) = mortality rate at age x.
Life expectancy at birth
Z
e0 =
∞
x
Z
m(u)du dx
exp
0
0
Approximated using life table methods.
Iterate for x = 0, 1, . . . ,, starting with `0 = 1:
Variations
for x = 0
and upper
age group.
qx = mx /(1 + 0.5mx )
Prob of death at age x
dx = `x qx
Propn deaths at age x
`x+1 = `x − dx
Propn survive to age x
Lx = `x − 0.5dx
Propn survive to age x + 0.5
Approximate life expectancy at birth
e0 =
∞
X
Lx
x=0
Functional time series with applications in demography
6. Coherent functional forecasting
26
Life expectancy (recap)
m(x) = mortality rate at age x.
Life expectancy at birth
Z
e0 =
∞
x
Z
m(u)du dx
exp
0
0
Approximated using life table methods.
Iterate for x = 0, 1, . . . ,, starting with `0 = 1:
Variations
for x = 0
and upper
age group.
qx = mx /(1 + 0.5mx )
Prob of death at age x
dx = `x qx
Propn deaths at age x
`x+1 = `x − dx
Propn survive to age x
Lx = `x − 0.5dx
Propn survive to age x + 0.5
Approximate remaining life expectancy at age u
eu =
∞
X
Lx
x=u
Functional time series with applications in demography
6. Coherent functional forecasting
26
Life expectancy (recap)
log[mt (xi )] = st (xi ) + σt (xi )εt,i ,
st (x) = µ(x) +
K
X
βt,k φk (x) + rt (x)
k =1
Non-coherent cohort life expectancy
Computed from ms+x (x) for a given s.
Combine observed ms+x (x) where s + x ≤ T with
forecast ms+x (x) for s + x > T.
Compute e∗0,s .
Prediction intervals by simulation
rt (x) resampled
εt,i ∼ N(0, 1)
βt,k simulated from ARIMA model
Functional time series with applications in demography
6. Coherent functional forecasting
27
Cohort life expectancy
−5
e
Log death rat
−10
2150
2100
2050
ar
Ye
80
60
2000
40
20
1950
Age
0
Functional time series with applications in demography
6. Coherent functional forecasting
28
Cohort life expectancy
−5
e
Log death rat
−10
2150
Need to do something like this
80
but using the product/ratio
60
approach40instead.
2100
2050
ar
Ye
2000
20
1950
Age
0
Functional time series with applications in demography
6. Coherent functional forecasting
28
Simulate future mortality rates
pt (x) =
q
st,M (x)st,F (x) and
log[pt (x)] = µp (x) +
rt (x) =
K
X
q
st,M (x) st,F (x).
βt,k φk (x) + et (x)
k =1
L
log[rt (x)] = µr (x) +
X
γt,` ψ` (x) + wt (x).
`=1
{γt,` } and {βt,k } simulated.
{et (x)} and {wt (x)} bootstrapped.
Generate many future sample paths for st,M (x)
and st,F (x) to estimate uncertainty in eu .
Functional time series with applications in demography
6. Coherent functional forecasting
29
Simulate future mortality rates
pt (x) =
q
st,M (x)st,F (x) and
log[pt (x)] = µp (x) +
rt (x) =
K
X
q
st,M (x) st,F (x).
βt,k φk (x) + et (x)
k =1
L
log[rt (x)] = µr (x) +
X
γt,` ψ` (x) + wt (x).
`=1
{γt,` } and {βt,k } simulated.
{et (x)} and {wt (x)} bootstrapped.
Generate many future sample paths for st,M (x)
and st,F (x) to estimate uncertainty in eu .
Functional time series with applications in demography
6. Coherent functional forecasting
29
Simulate future mortality rates
pt (x) =
q
st,M (x)st,F (x) and
log[pt (x)] = µp (x) +
rt (x) =
K
X
q
st,M (x) st,F (x).
βt,k φk (x) + et (x)
k =1
L
log[rt (x)] = µr (x) +
X
γt,` ψ` (x) + wt (x).
`=1
{γt,` } and {βt,k } simulated.
{et (x)} and {wt (x)} bootstrapped.
Generate many future sample paths for st,M (x)
and st,F (x) to estimate uncertainty in eu .
Functional time series with applications in demography
6. Coherent functional forecasting
29
Simulate future mortality rates
pt (x) =
q
st,M (x)st,F (x) and
log[pt (x)] = µp (x) +
rt (x) =
K
X
q
st,M (x) st,F (x).
βt,k φk (x) + et (x)
k =1
L
log[rt (x)] = µr (x) +
X
γt,` ψ` (x) + wt (x).
`=1
{γt,` } and {βt,k } simulated.
{et (x)} and {wt (x)} bootstrapped.
Generate many future sample paths for st,M (x)
and st,F (x) to estimate uncertainty in eu .
Functional time series with applications in demography
6. Coherent functional forecasting
29
Cohort life expectancy
40
35
30
20
25
Remaining life expectancy
45
50
Australia: cohort life expectancy at age 50
1920
1940
1960
1980
2000
2020
2040
2060
Year
Functional time series with applications in demography
6. Coherent functional forecasting
30
Complete code
library(demography)
# Read data
aus <- hmd.mx("AUS","username","password","Australia")
# Smooth data
aus.sm <- smooth.demogdata(aus)
#Fit model
aus.pr <- coherentfdm(aus.sm)
# Forecast
aus.pr.fc <- forecast(aus.pr, h=100)
# Compute life expectancies
e50.m.aus.fc <- flife.expectancy(aus.pr.fc, series="male",
age=50, PI=TRUE, nsim=1000, type="cohort")
e50.f.aus.fc <- flife.expectancy(aus.pr.fc, series="female",
age=50, PI=TRUE, nsim=1000, type="cohort")
Functional time series with applications in demography
6. Coherent functional forecasting
31
Forecast accuracy evaluation
34
32
26
28
30
Remaining years
36
38
40
Australian female cohort e50
1920
1940
1960
1980
2000
Year
Functional time series with applications in demography
6. Coherent functional forecasting
32
Forecast accuracy evaluation
Functional time series with applications in demography
6. Coherent functional forecasting
33
Forecast accuracy evaluation
Compute age 50 remaining cohort life
expectancy with a rolling forecast
origin beginning in 1921.
Compare against actual cohort life
expectancy where available.
Compute 80% prediction interval actual
coverage.
Functional time series with applications in demography
6. Coherent functional forecasting
34
Forecast accuracy evaluation
Compute age 50 remaining cohort life
expectancy with a rolling forecast
origin beginning in 1921.
Compare against actual cohort life
expectancy where available.
Compute 80% prediction interval actual
coverage.
Functional time series with applications in demography
6. Coherent functional forecasting
34
Forecast accuracy evaluation
Compute age 50 remaining cohort life
expectancy with a rolling forecast
origin beginning in 1921.
Compare against actual cohort life
expectancy where available.
Compute 80% prediction interval actual
coverage.
Functional time series with applications in demography
6. Coherent functional forecasting
34
Forecast accuracy evaluation
1.0
1.2
Mean Absolute Forecast Errors
0.6
0.2
0.4
Years
0.8
Female
0.0
Male
1 2 3 4 5
10
15
20
25
Forecast horizon
Functional time series with applications in demography
6. Coherent functional forecasting
35
Forecast accuracy evaluation
60
40
0
20
Percentage coverage
80
100
80% prediction interval coverage
1 2 3 4 5
10
15
20
25
Forecast horizon
Functional time series with applications in demography
6. Coherent functional forecasting
35
Outline
1 Forecasting groups
2 Automatic ARFIMA forecasting
3 Coherent cohort life expectancy forecasts
4 Coherent forecasts for J > 2 groups
5 Forecasting state mortality
6 References
Functional time series with applications in demography
6. Coherent functional forecasting
36
Coherent forecasts for J groups
1/J
pt (x) = [st,1 (x)st,2 (x) · · · st,j (x)]
and
rt,j (x) = st,j (x) pt (x),
log[pt (x)] = µp (x) +
K
X
βt,k φk (x) + et (x)
k =1
L
log[rt,j (x)] = µr ,j (x) +
X
γt,l,j ψl,j (x) + wt,j (x).
l =1
pt (x) and all rt,j (x)
are approximately
independent.
Functional time series with applications in demography
Ratios satisfy constraint
rt,1 (x)rt,2 (x) · · · rt,J (x) = 1.
log[st,j (x)] = log[pt (x)rt,j (x)]
6. Coherent functional forecasting
37
Coherent forecasts for J groups
1/J
pt (x) = [st,1 (x)st,2 (x) · · · st,j (x)]
and
rt,j (x) = st,j (x) pt (x),
log[pt (x)] = µp (x) +
K
X
βt,k φk (x) + et (x)
k =1
L
log[rt,j (x)] = µr ,j (x) +
X
γt,l,j ψl,j (x) + wt,j (x).
l =1
pt (x) and all rt,j (x)
are approximately
independent.
Functional time series with applications in demography
Ratios satisfy constraint
rt,1 (x)rt,2 (x) · · · rt,J (x) = 1.
log[st,j (x)] = log[pt (x)rt,j (x)]
6. Coherent functional forecasting
37
Coherent forecasts for J groups
1/J
pt (x) = [st,1 (x)st,2 (x) · · · st,j (x)]
and
rt,j (x) = st,j (x) pt (x),
log[pt (x)] = µp (x) +
K
X
βt,k φk (x) + et (x)
k =1
L
log[rt,j (x)] = µr ,j (x) +
X
γt,l,j ψl,j (x) + wt,j (x).
l =1
pt (x) and all rt,j (x)
are approximately
independent.
Functional time series with applications in demography
Ratios satisfy constraint
rt,1 (x)rt,2 (x) · · · rt,J (x) = 1.
log[st,j (x)] = log[pt (x)rt,j (x)]
6. Coherent functional forecasting
37
Coherent forecasts for J groups
1/J
pt (x) = [st,1 (x)st,2 (x) · · · st,j (x)]
and
rt,j (x) = st,j (x) pt (x),
log[pt (x)] = µp (x) +
K
X
βt,k φk (x) + et (x)
k =1
L
log[rt,j (x)] = µr ,j (x) +
X
γt,l,j ψl,j (x) + wt,j (x).
l =1
pt (x) and all rt,j (x)
are approximately
independent.
Functional time series with applications in demography
Ratios satisfy constraint
rt,1 (x)rt,2 (x) · · · rt,J (x) = 1.
log[st,j (x)] = log[pt (x)rt,j (x)]
6. Coherent functional forecasting
37
Coherent forecasts for J groups
1/J
pt (x) = [st,1 (x)st,2 (x) · · · st,j (x)]
and
rt,j (x) = st,j (x) pt (x),
log[pt (x)] = µp (x) +
K
X
βt,k φk (x) + et (x)
k =1
L
log[rt,j (x)] = µr ,j (x) +
X
γt,l,j ψl,j (x) + wt,j (x).
l =1
pt (x) and all rt,j (x)
are approximately
independent.
Functional time series with applications in demography
Ratios satisfy constraint
rt,1 (x)rt,2 (x) · · · rt,J (x) = 1.
log[st,j (x)] = log[pt (x)rt,j (x)]
6. Coherent functional forecasting
37
Coherent forecasts for J groups
log[st,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ]
= µj (x) +
K
X
βt,k φk (x) +
k =1
L
X
γt,`,j ψ`,j (x) + zt,j (x)
`=1
µj (x) = µp (x) + µr ,j (x) is group mean
zt,j (x) = et (x) + wt,j (x) is error term.
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d, q).
No restrictions for βt,1 , . . . , βt,K .
Functional time series with applications in demography
6. Coherent functional forecasting
38
Coherent forecasts for J groups
log[st,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ]
= µj (x) +
K
X
βt,k φk (x) +
k =1
L
X
γt,`,j ψ`,j (x) + zt,j (x)
`=1
µj (x) = µp (x) + µr ,j (x) is group mean
zt,j (x) = et (x) + wt,j (x) is error term.
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d, q).
No restrictions for βt,1 , . . . , βt,K .
Functional time series with applications in demography
6. Coherent functional forecasting
38
Coherent forecasts for J groups
log[st,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ]
= µj (x) +
K
X
βt,k φk (x) +
k =1
L
X
γt,`,j ψ`,j (x) + zt,j (x)
`=1
µj (x) = µp (x) + µr ,j (x) is group mean
zt,j (x) = et (x) + wt,j (x) is error term.
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d, q).
No restrictions for βt,1 , . . . , βt,K .
Functional time series with applications in demography
6. Coherent functional forecasting
38
Coherent forecasts for J groups
log[st,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ]
= µj (x) +
K
X
βt,k φk (x) +
k =1
L
X
γt,`,j ψ`,j (x) + zt,j (x)
`=1
µj (x) = µp (x) + µr ,j (x) is group mean
zt,j (x) = et (x) + wt,j (x) is error term.
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d, q).
No restrictions for βt,1 , . . . , βt,K .
Functional time series with applications in demography
6. Coherent functional forecasting
38
Coherent forecasts for J groups
log[st,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ]
= µj (x) +
K
X
βt,k φk (x) +
k =1
L
X
γt,`,j ψ`,j (x) + zt,j (x)
`=1
µj (x) = µp (x) + µr ,j (x) is group mean
zt,j (x) = et (x) + wt,j (x) is error term.
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d, q).
No restrictions for βt,1 , . . . , βt,K .
Functional time series with applications in demography
6. Coherent functional forecasting
38
Outline
1 Forecasting groups
2 Automatic ARFIMA forecasting
3 Coherent cohort life expectancy forecasts
4 Coherent forecasts for J > 2 groups
5 Forecasting state mortality
6 References
Functional time series with applications in demography
6. Coherent functional forecasting
39
Forecasting state mortality
Functional time series with applications in demography
6. Coherent functional forecasting
40
Forecasting state mortality
Northern Territory
and Australian
Capital Territory
omitted due to
missing data.
Functional time series with applications in demography
6. Coherent functional forecasting
40
Forecasting state mortality
Northern Territory
and Australian
Capital Territory
omitted due to
missing data.
Data for 1950 to
2003 obtained
from Australian
Demographic
Data Bank.
Functional time series with applications in demography
6. Coherent functional forecasting
40
Forecasting state mortality
0.18
0.12
φ2(x)
φ1(x)
µ(x)
0.14
−4
1.6%
0.5%
0.1
0.1
0.10
φ3(x)
−2
0.2
96.5%
0.16
0.0
0.08
−6
−0.1
−0.1
0.06
0.04
−8
−0.2
−0.2
0
20 40 60 80
0
20 40 60 80
Age (x)
0.0
0
20 40 60 80
Age (x)
0
20 40 60 80
Age (x)
Age (x)
1.5
0.5
0
0.5
−5
0.0
βt3
βt2
βt1
Forecasts for
pt (x)
1.0
0.0
−0.5
−0.5
−10
−1.0
−1.0
1960
2000
Time (t)
Functional time series with applications in demography
−1.5
1960
2000
1960
Time (t)
6. Coherent functional forecasting
2000
Time (t)
41
Forecasting state mortality
37.0%
0.3
0.00
13.5%
0.2
9.2%
0.3
0.1
0.2
−0.10
0.0
ψ2(x)
µ(x)
ψ1(x)
0.2
0.1
−0.15
0.0
ψ3(x)
−0.05
−0.1
0.1
−0.2
0.0
−0.3
−0.1
−0.4
−0.20
0
20 40 60 80
0
20 40 60 80
Age (x)
0
20 40 60 80
Age (x)
0
20 40 60 80
Age (x)
0.5
Age (x)
0.6
0.4
0.4
0.0
−0.5
γt3
γt2
0.2
γt1
Forecasts for
rt,1 (x)
— Victoria
0.2
0.0
0.0
−0.2
−0.2
−1.0
−0.4
−0.4
1960
2000
Time (t)
Functional time series with applications in demography
1960
2000
1960
Time (t)
6. Coherent functional forecasting
2000
Time (t)
41
Forecasting state mortality
Independent forecasts
80
70
70
75
Years
80
85
VIC
NSW
QLD
SA
WA
TAS
75
Years
85
Coherent forecasts
1960
1980
2000
2020
Year
Functional time series with applications in demography
1960
1980
2000
2020
Year
6. Coherent functional forecasting
42
Forecasting state mortality
VIC Coherent
VIC Independent
NSW Coherent
NSW Independent
QLD Coherent
QLD Independent
SA Coherent
SA Independent
WA Coherent
WA Independent
TAS Coherent
TAS Independent
0.7
0.6
MSFE
0.5
0.4
0.3
0.2
0.1
0
5
10
15
20
25
30
Forecast horizon
Functional time series with applications in demography
6. Coherent functional forecasting
42
Outline
1 Forecasting groups
2 Automatic ARFIMA forecasting
3 Coherent cohort life expectancy forecasts
4 Coherent forecasts for J > 2 groups
5 Forecasting state mortality
6 References
Functional time series with applications in demography
6. Coherent functional forecasting
43
Selected references
Hyndman, Booth, Yasmeen (2013). “Coherent
mortality forecasting: the product-ratio method
with functional time series models”.
Demography 50(1), 261–283.
Hyndman (2014). demography: Forecasting
mortality, fertility, migration and population
data.
cran.r-project.org/package=demography
Functional time series with applications in demography
6. Coherent functional forecasting
44