Demographic forecasting
using functional data
analysis
Rob J Hyndman
Joint work with: Heather Booth, Han Lin Shang,
Shahid Ullah, Farah Yasmeen.
Demographic forecasting using functional data analysis
1
Mortality rates
−4
−8
−6
Log death rate
−2
0
France: male mortality (1816)
0
20
40
60
80
100
Age
Demographic forecasting using functional data analysis
2
Fertility rates
150
100
0
50
Fertility rate
200
250
Australia: fertility rates (1921)
15
20
25
30
35
40
45
50
Age
Demographic forecasting using functional data analysis
3
Outline
1
A functional linear model
2
Bagplots, boxplots and outliers
3
Functional forecasting
4
Forecasting groups
5
Population forecasting
6
References
Demographic forecasting using functional data analysis
4
Outline
1
A functional linear model
2
Bagplots, boxplots and outliers
3
Functional forecasting
4
Forecasting groups
5
Population forecasting
6
References
Demographic forecasting using functional data analysis
A functional linear model
5
Some notation
Let yt,x be the observed (smoothed) data in period t
at age x, t = 1, . . . , n.
yt,x = ft (x) + σt (x)εt,x
K
X
ft (x) = µ(x) +
βt,k φk (x) + et (x)
k =1
Estimate ft (x) using penalized regression splines.
Estimate µ(x) as me(di)an ft (x) across years.
Estimate βt,k and φk (x) using (robust) functional
principal components.
iid
iid
εt,x ∼ N(0, 1) and et (x) ∼ N(0, v(x)).
Demographic forecasting using functional data analysis
A functional linear model
6
Some notation
Let yt,x be the observed (smoothed) data in period t
at age x, t = 1, . . . , n.
yt,x = ft (x) + σt (x)εt,x
K
X
ft (x) = µ(x) +
βt,k φk (x) + et (x)
k =1
Estimate ft (x) using penalized regression splines.
Estimate µ(x) as me(di)an ft (x) across years.
Estimate βt,k and φk (x) using (robust) functional
principal components.
iid
iid
εt,x ∼ N(0, 1) and et (x) ∼ N(0, v(x)).
Demographic forecasting using functional data analysis
A functional linear model
6
Some notation
Let yt,x be the observed (smoothed) data in period t
at age x, t = 1, . . . , n.
yt,x = ft (x) + σt (x)εt,x
K
X
ft (x) = µ(x) +
βt,k φk (x) + et (x)
k =1
Estimate ft (x) using penalized regression splines.
Estimate µ(x) as me(di)an ft (x) across years.
Estimate βt,k and φk (x) using (robust) functional
principal components.
iid
iid
εt,x ∼ N(0, 1) and et (x) ∼ N(0, v(x)).
Demographic forecasting using functional data analysis
A functional linear model
6
Some notation
Let yt,x be the observed (smoothed) data in period t
at age x, t = 1, . . . , n.
yt,x = ft (x) + σt (x)εt,x
K
X
ft (x) = µ(x) +
βt,k φk (x) + et (x)
k =1
Estimate ft (x) using penalized regression splines.
Estimate µ(x) as me(di)an ft (x) across years.
Estimate βt,k and φk (x) using (robust) functional
principal components.
iid
iid
εt,x ∼ N(0, 1) and et (x) ∼ N(0, v(x)).
Demographic forecasting using functional data analysis
A functional linear model
6
Some notation
Let yt,x be the observed (smoothed) data in period t
at age x, t = 1, . . . , n.
yt,x = ft (x) + σt (x)εt,x
K
X
ft (x) = µ(x) +
βt,k φk (x) + et (x)
k =1
Estimate ft (x) using penalized regression splines.
Estimate µ(x) as me(di)an ft (x) across years.
Estimate βt,k and φk (x) using (robust) functional
principal components.
iid
iid
εt,x ∼ N(0, 1) and et (x) ∼ N(0, v(x)).
Demographic forecasting using functional data analysis
A functional linear model
6
0.2
40
60
80
100
−0.1
−0.2
0.05
20
0.00
0
0.0
φ2(x)
0.15
0.10
φ1(x)
0.1
0.20
−1
−2
−3
−4
−5
−6
0
20
40
60
80
100
0
20
Age
40
60
80
100
Age
βt2
2
−5
βt1
4
0
5
6
10
8
Age
−2
0
−15 −10
µ(x)
French mortality components
1850
1900
1950
2000
1850
t
Demographic forecasting using functional data analysis
1900
1950
2000
t
A functional linear model
7
French mortality components
0
20
40
Age
60
80
100
Residuals
1850
1900
1950
2000
Year
Demographic forecasting using functional data analysis
A functional linear model
7
Outline
1
A functional linear model
2
Bagplots, boxplots and outliers
3
Functional forecasting
4
Forecasting groups
5
Population forecasting
6
References
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
8
French male mortality rates
−4
−6
−8
Log death rate
−2
0
France: male death rates (1900−2009)
0
20
40
60
80
100
Age
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
9
French male mortality rates
0
France: male death rates (1900−2009)
−4
−6
−8
Log death rate
−2
War years
0
20
40
60
80
100
Age
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
9
French male mortality rates
0
France: male death rates (1900−2009)
−4
−6
−8
Log death rate
−2
War years
0
20
Aims
1 “Boxplots” for functional data
60 detecting
80
100
240 Tools for
outliers
in
Age
functional data
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
9
Robust principal components
Let {ft (x)}, t = 1, . . . , n, be a set of curves.
1 Apply a robust principal component algorithm
n−1
X
βt,k φk (x)
ft (x) = µ(x) +
k =1
µ(x) is median curve
{φk (x)} are principal components
{βt,k } are PC scores
2
Plot βi ,2 vs βi ,1
å Each point in scatterplot represents one curve.
å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
10
Robust principal components
Let {ft (x)}, t = 1, . . . , n, be a set of curves.
1 Apply a robust principal component algorithm
n−1
X
βt,k φk (x)
ft (x) = µ(x) +
k =1
µ(x) is median curve
{φk (x)} are principal components
{βt,k } are PC scores
2
Plot βi ,2 vs βi ,1
å Each point in scatterplot represents one curve.
å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
10
Robust principal components
Let {ft (x)}, t = 1, . . . , n, be a set of curves.
1 Apply a robust principal component algorithm
n−1
X
βt,k φk (x)
ft (x) = µ(x) +
k =1
µ(x) is median curve
{φk (x)} are principal components
{βt,k } are PC scores
2
Plot βi ,2 vs βi ,1
å Each point in scatterplot represents one curve.
å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
10
Robust principal components
Let {ft (x)}, t = 1, . . . , n, be a set of curves.
1 Apply a robust principal component algorithm
n−1
X
βt,k φk (x)
ft (x) = µ(x) +
k =1
µ(x) is median curve
{φk (x)} are principal components
{βt,k } are PC scores
2
Plot βi ,2 vs βi ,1
å Each point in scatterplot represents one curve.
å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
10
Robust principal components
Let {ft (x)}, t = 1, . . . , n, be a set of curves.
1 Apply a robust principal component algorithm
n−1
X
βt,k φk (x)
ft (x) = µ(x) +
k =1
µ(x) is median curve
{φk (x)} are principal components
{βt,k } are PC scores
2
Plot βi ,2 vs βi ,1
å Each point in scatterplot represents one curve.
å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
10
Robust principal components
Let {ft (x)}, t = 1, . . . , n, be a set of curves.
1 Apply a robust principal component algorithm
n−1
X
βt,k φk (x)
ft (x) = µ(x) +
k =1
µ(x) is median curve
{φk (x)} are principal components
{βt,k } are PC scores
2
Plot βi ,2 vs βi ,1
å Each point in scatterplot represents one curve.
å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
10
Robust principal components
Let {ft (x)}, t = 1, . . . , n, be a set of curves.
1 Apply a robust principal component algorithm
n−1
X
βt,k φk (x)
ft (x) = µ(x) +
k =1
µ(x) is median curve
{φk (x)} are principal components
{βt,k } are PC scores
2
Plot βi ,2 vs βi ,1
å Each point in scatterplot represents one curve.
å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
10
Robust principal components
Let {ft (x)}, t = 1, . . . , n, be a set of curves.
1 Apply a robust principal component algorithm
n−1
X
βt,k φk (x)
ft (x) = µ(x) +
k =1
µ(x) is median curve
{φk (x)} are principal components
{βt,k } are PC scores
2
Plot βi ,2 vs βi ,1
å Each point in scatterplot represents one curve.
å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
10
Robust principal components
Let {ft (x)}, t = 1, . . . , n, be a set of curves.
1 Apply a robust principal component algorithm
n−1
X
βt,k φk (x)
ft (x) = µ(x) +
k =1
µ(x) is median curve
{φk (x)} are principal components
{βt,k } are PC scores
2
Plot βi ,2 vs βi ,1
å Each point in scatterplot represents one curve.
å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
10
Robust principal components
Scatterplot of first two PC scores
6
●
●
●
●
4
●
●
●
2
●
●●●
●●● ● ● ●
●
●
●●●● ●● ●
●
●
●
●● ●
●
● ●
●
●●
●●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ● ●
●● ●
●
●
● ●● ●●
●● ●
●
● ●●
●●●
●●
●
●
●
●
●●
●●●
●
●
●
●
●
●
−2
0
PC score 2
●
−10
−5
0
5
10
15
PC score 1
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
11
Robust principal components
Scatterplot of first two PC scores
6
● ●
1915
1914
1916●
1918●
1943●
2
1919●
●●●
●●● ● ● ●
●
●
●●●● ●● ●
●
●
●
●● ●
●
● ●
●
●●
●●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ● ●
●● ●
1945●
1942●
1941●●●●●●●●●●● ●
●
●
●●
●
●
●●●
●
● ●
●●
●
●
●●
●
●
●
−2
0
PC score 2
4
●
1944
1917●
●
1940
−10
−5
0
5
10
15
PC score 1
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
11
Functional bagplot
Bivariate bagplot due to Rousseeuw et al. (1999).
Rank points by halfspace location depth.
Display median, 50% convex hull and outer
convex hull (with 99% coverage if bivariate
normal).
6
● ●
1915
1914
●
●
1916●
1918●
●
●
●
1944
1917●
1940●
●
4
●
●
1919●
2
PC score 2
●
1943●
●
●
●●● ● ●
●
●●
● ●
● ●
●
●
●
1945●
●
● ●
●
●
●
● ●● ● ●
●
●
●●
●
●
●
● ●
●
●
●●
●
● ●● ●
●
● ● ●
●●
● ●●
●
0
● ● ●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
● ● ●●
●
●
●
●
●
●
●
●
−2
●●
●
●
●
●
−10
−5
0
5
10
15
PC score 1
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
12
Functional bagplot
0
Bivariate bagplot due to Rousseeuw et al. (1999).
Rank points by halfspace location depth.
Display median, 50% convex hull and outer
convex hull (with 99% coverage if bivariate
normal).
Boundaries contain all curves inside bags.
95% CI for median curve also shown.
6
● ●
1915
1914
●
●
1916●
1918●
●
−2
●
●
1944
1917●
1940●
●
4
●
2
●
●
●●● ● ●
●
●●
● ●
● ●
●
●
1945●
●
● ●
●
●
●
● ●● ● ●
●
●
●●
●
●
●
● ●
●
●
●●
●
● ●● ●
●
● ● ●
●●
● ●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
● ●●●
●
●
●
●
●
●
●
1914
1915
1916
1917
−8
0
● ● ●●
●
●
−4
Log death rate
●
1919●
−6
1943●
●
●
−2
PC score 2
●
−10
−5
0
5
10
15
PC score 1
Demographic forecasting using functional data analysis
0
20
40
60
1918
1940
1943
1944
80
100
Age
Bagplots, boxplots and outliers
12
−4
−6
1914
1915
1916
1917
−8
Log death rate
−2
0
Functional bagplot
0
20
40
60
80
1918
1940
1943
1944
100
Age
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
13
Functional HDR boxplot
6
Bivariate HDR boxplot due to Hyndman (1996).
Rank points by value of kernel density estimate.
Display mode, 50% and (usually) 99% highest
density regions (HDRs) and mode.
99% outer region
●
●
●
●
●
4
●
●
1919●
2
PC score 2
●
1943●
●
●
●●● ● ●
●
●●
● ●
● ●
●
●
●
●
● ●
●
●
●
● ●● ● ●
●
●
●●
●
●
●
● ●
●
●
●●
●
● ●● ●
●
● ● ●
●●
● ●●
●
o
0
● ● ●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
● ● ●●
●
●
●
●
●
●
●
●
−2
●●
●
●
●
●
●
●
−10
−5
0
PC score 1
Demographic forecasting using functional data analysis
5
10
15
Bagplots, boxplots and outliers
14
Functional HDR boxplot
6
Bivariate HDR boxplot due to Hyndman (1996).
Rank points by value of kernel density estimate.
Display mode, 50% and (usually) 99% highest
density regions (HDRs) and mode.
● ●
1915
1914
90% outer region
●
●
1916●
1918●
●
●
●
1944
1917●
1940●
●
4
●
●
1919●
2
PC score 2
●
1943●
●
●
●●● ● ●
●
●●
● ●
● ●
●
●
●
1945●
1942●
●
● ●
●
●
●
● ●● ● ●
●
●
●●
●
●
●
● ●
●
●
●●
●
● ●● ●
●
● ● ●
●●
● ●●
●
o
0
● ● ●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
● ● ●●
●
●
●
●
●
●
●
●
−2
●●
●
●
●
●
●
●
−10
−5
0
PC score 1
Demographic forecasting using functional data analysis
5
10
15
Bagplots, boxplots and outliers
14
Functional HDR boxplot
6
0
Bivariate HDR boxplot due to Hyndman (1996).
Rank points by value of kernel density estimate.
Display mode, 50% and (usually) 99% highest
density regions (HDRs) and mode.
Boundaries contain all curves inside HDRs.
● ●
1915
1914
90% outer region
●
●
1916●
1918●
●
−2
●
●
1944
1917●
●
4
●
1940●
2
●
●
●●● ● ●
●
●●
● ●
● ●
●
●
●
1945●
1942●
●
● ●
●
●
●
● ●● ● ●
●
●
●●
●
●
●
● ●
● ●
●●
● ●●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●●● ●
●
●
o
●
●
●
● ●●
●
● ●●●
●
●
●
●
●
●
1914
1915
1916
1917
1918
1919
●
●
−8
0
● ● ●●
●
●
−4
Log death rate
●
1919●
−6
1943●
●
−2
PC score 2
●
−10
−5
0
5
10
15
PC score 1
Demographic forecasting using functional data analysis
0
20
40
60
1940
1943
1944
1945
1948
80
100
Age
Bagplots, boxplots and outliers
14
−4
−6
1914
1915
1916
1917
1918
1919
−8
Log death rate
−2
0
Functional HDR boxplot
0
20
40
60
80
1940
1943
1944
1945
1948
100
Age
Demographic forecasting using functional data analysis
Bagplots, boxplots and outliers
15
Outline
1
A functional linear model
2
Bagplots, boxplots and outliers
3
Functional forecasting
4
Forecasting groups
5
Population forecasting
6
References
Demographic forecasting using functional data analysis
Functional forecasting
16
Functional time series model
yt,x = ft (x) + σt (x)εt,x
K
X
ft (x) = µ(x) +
βt,k φk (x) + et (x)
k =1
The eigenfunctions φk (x) show the main
regions of variation.
The scores {βt,k } are uncorrelated by
construction. So we can forecast each βt,k
using a univariate time series model.
Outliers are treated as missing values.
Univariate ARIMA models are used for
forecasting.
Demographic forecasting using functional data analysis
Functional forecasting
17
Functional time series model
yt,x = ft (x) + σt (x)εt,x
K
X
ft (x) = µ(x) +
βt,k φk (x) + et (x)
k =1
The eigenfunctions φk (x) show the main
regions of variation.
The scores {βt,k } are uncorrelated by
construction. So we can forecast each βt,k
using a univariate time series model.
Outliers are treated as missing values.
Univariate ARIMA models are used for
forecasting.
Demographic forecasting using functional data analysis
Functional forecasting
17
Functional time series model
yt,x = ft (x) + σt (x)εt,x
K
X
ft (x) = µ(x) +
βt,k φk (x) + et (x)
k =1
The eigenfunctions φk (x) show the main
regions of variation.
The scores {βt,k } are uncorrelated by
construction. So we can forecast each βt,k
using a univariate time series model.
Outliers are treated as missing values.
Univariate ARIMA models are used for
forecasting.
Demographic forecasting using functional data analysis
Functional forecasting
17
Functional time series model
yt,x = ft (x) + σt (x)εt,x
K
X
ft (x) = µ(x) +
βt,k φk (x) + et (x)
k =1
The eigenfunctions φk (x) show the main
regions of variation.
The scores {βt,k } are uncorrelated by
construction. So we can forecast each βt,k
using a univariate time series model.
Outliers are treated as missing values.
Univariate ARIMA models are used for
forecasting.
Demographic forecasting using functional data analysis
Functional forecasting
17
Functional time series model
yt,x = ft (x) + σt (x)εt,x
K
X
ft (x) = µ(x) +
βt,k φk (x) + et (x)
k =1
The eigenfunctions φk (x) show the main
regions of variation.
The scores {βt,k } are uncorrelated by
construction. So we can forecast each βt,k
using a univariate time series model.
Outliers are treated as missing values.
Univariate ARIMA models are used for
forecasting.
Demographic forecasting using functional data analysis
Functional forecasting
17
Forecasts
yt,x = ft (x) + σt (x)εt,x
K
X
ft (x) = µ(x) +
βt,k φk (x) + et (x)
k =1
Demographic forecasting using functional data analysis
Functional forecasting
18
Forecasts
yt,x = ft (x) + σt (x)εt,x
K
X
ft (x) = µ(x) +
βt,k φk (x) + et (x)
k =1
E[yn+h ,x | y] = µ̂(x) +
K
X
β̂n+h ,k φ̂k (x)
k =1
K
X
Var[yn+h ,x | y] = σ̂µ2 (x) +
vn+h ,k φ̂k2 (x) + σt2 (x) + v(x)
k =1
where vn+h ,k = Var(βn+h ,k | β1,k , . . . , βn,k )
and y = [y1,x , . . . , yn,x ].
Demographic forecasting using functional data analysis
Functional forecasting
18
Forecasting the PC scores
Interaction
0.1
0.0
Basis function 2
−0.2
−0.1
0.15
0.10
Basis function 1
0.05
−4
−6
0
20
40
60
80
100
0
20
40
60
80
100
0
20
40
Age
●
●●
60
80
100
Age
●
2
10 15
Age
●
●
●
−2
Coefficient 2
●
●
●
−4
0
−10
Coefficient 1
5
0
●
●
−6
●
−20
Mean
−2
0.20
0.2
0
Main effects
●
●
●
1900
1940
1980
2020
1900
Year
Demographic forecasting using functional data analysis
1940
1980
2020
Year
Functional forecasting
19
Forecasts of ft (x)
−4
−6
−10
−8
Log death rate
−2
0
France: male death rates (1900−2009)
0
20
40
60
80
100
Age
Demographic forecasting using functional data analysis
Functional forecasting
20
Forecasts of ft (x)
−4
−6
−10
−8
Log death rate
−2
0
France: male death rates (1900−2009)
0
20
40
60
80
100
Age
Demographic forecasting using functional data analysis
Functional forecasting
20
Forecasts of ft (x)
−4
−6
−10
−8
Log death rate
−2
0
France: male death forecasts (2010−2029)
0
20
40
60
80
100
Age
Demographic forecasting using functional data analysis
Functional forecasting
20
Forecasts of ft (x)
−4
−6
−10
−8
Log death rate
−2
0
France: male death forecasts (2010 & 2029)
80% prediction intervals
0
20
40
60
80
100
Age
Demographic forecasting using functional data analysis
Functional forecasting
20
Fertility application
150
100
0
50
Fertility rate
200
250
Australia fertility rates (1921−2009)
15
20
25
30
35
40
45
50
Age
Demographic forecasting using functional data analysis
Functional forecasting
21
0.25
0.05
−0.3
15
20
25
30
35
40
45
50
0.15
Φ2(x)
0.0
Φ1(x)
Μ
5
0
−0.1
10
0.1
15
0.2
Fertility model
15
20
25
30
35
40
45
50
15
20
25
Age
30
35
40
45
50
Age
Βt2
−10
−4 −2
0
2
0
−5
Βt1
4
5
6
8
10
Age
1970
1980
1990
2000
2010
1970
1980
t
Demographic forecasting using functional data analysis
1990
2000
t
Functional forecasting
22
2010
Fertility model
30
15
20
25
Age
35
40
45
Residuals
1970
1980
1990
2000
Year
Demographic forecasting using functional data analysis
Functional forecasting
23
Fertility model
Interaction
30
35
40
45
0.25
0.05
0.15
Basis function 2
0.0
50
15
20
25
30
35
40
45
50
15
20
25
30
35
40
45
50
Age
−4 −2
0
5
Coefficient 2
6
8
10 15 20 25
Age
Coefficient 1
Age
4
25
2
20
0
15
−10
Mean
−0.1
−0.3
0
5
10
Basis function 1
0.1
15
0.2
Main effects
1970
1990
2010
2030
1970
Year
Demographic forecasting using functional data analysis
1990
2010
2030
Year
Functional forecasting
24
Forecasts of ft (x)
150
100
0
50
Fertility rate
200
250
Australia fertility rates (1921−2009)
15
20
25
30
35
40
45
50
Age
Demographic forecasting using functional data analysis
Functional forecasting
25
Forecasts of ft (x)
150
100
0
50
Fertility rate
200
250
Australia fertility rates (1921−2009)
15
20
25
30
35
40
45
50
Age
Demographic forecasting using functional data analysis
Functional forecasting
25
Forecasts of ft (x)
150
100
0
50
Fertility rate
200
250
Australia fertility rates: 2010−2029
15
20
25
30
35
40
45
50
Age
Demographic forecasting using functional data analysis
Functional forecasting
25
Forecasts of ft (x)
Australia fertility rates: 2010 and 2029
150
100
0
50
Fertility rate
200
250
80% prediction intervals
15
20
25
30
35
40
45
50
Age
Demographic forecasting using functional data analysis
Functional forecasting
25
Outline
1
A functional linear model
2
Bagplots, boxplots and outliers
3
Functional forecasting
4
Forecasting groups
5
Population forecasting
6
References
Demographic forecasting using functional data analysis
Forecasting groups
26
The problem
Let ft,j (x) be the smoothed mortality rate for age x in
group j in year t.
Groups may be males and females.
Groups may be states within a country.
Expected that groups will behave similarly.
Coherent forecasts do not diverge over time.
Existing functional models do not impose
coherence.
Demographic forecasting using functional data analysis
Forecasting groups
27
The problem
Let ft,j (x) be the smoothed mortality rate for age x in
group j in year t.
Groups may be males and females.
Groups may be states within a country.
Expected that groups will behave similarly.
Coherent forecasts do not diverge over time.
Existing functional models do not impose
coherence.
Demographic forecasting using functional data analysis
Forecasting groups
27
The problem
Let ft,j (x) be the smoothed mortality rate for age x in
group j in year t.
Groups may be males and females.
Groups may be states within a country.
Expected that groups will behave similarly.
Coherent forecasts do not diverge over time.
Existing functional models do not impose
coherence.
Demographic forecasting using functional data analysis
Forecasting groups
27
The problem
Let ft,j (x) be the smoothed mortality rate for age x in
group j in year t.
Groups may be males and females.
Groups may be states within a country.
Expected that groups will behave similarly.
Coherent forecasts do not diverge over time.
Existing functional models do not impose
coherence.
Demographic forecasting using functional data analysis
Forecasting groups
27
The problem
Let ft,j (x) be the smoothed mortality rate for age x in
group j in year t.
Groups may be males and females.
Groups may be states within a country.
Expected that groups will behave similarly.
Coherent forecasts do not diverge over time.
Existing functional models do not impose
coherence.
Demographic forecasting using functional data analysis
Forecasting groups
27
The problem
Let ft,j (x) be the smoothed mortality rate for age x in
group j in year t.
Groups may be males and females.
Groups may be states within a country.
Expected that groups will behave similarly.
Coherent forecasts do not diverge over time.
Existing functional models do not impose
coherence.
Demographic forecasting using functional data analysis
Forecasting groups
27
Forecasting the coefficients
yt,x = ft (x) + σt (x)εt,x
K
X
ft (x) = µ(x) +
βt,k φk (x) + et (x)
k =1
We use ARIMA models for each coefficient
{β1,j ,k , . . . , βn,j ,k }.
The ARIMA models are non-stationary for the
first few coefficients (k = 1, 2)
Non-stationary ARIMA forecasts will diverge.
Hence the mortality forecasts are not coherent.
Demographic forecasting using functional data analysis
Forecasting groups
28
Forecasting the coefficients
yt,x = ft (x) + σt (x)εt,x
K
X
ft (x) = µ(x) +
βt,k φk (x) + et (x)
k =1
We use ARIMA models for each coefficient
{β1,j ,k , . . . , βn,j ,k }.
The ARIMA models are non-stationary for the
first few coefficients (k = 1, 2)
Non-stationary ARIMA forecasts will diverge.
Hence the mortality forecasts are not coherent.
Demographic forecasting using functional data analysis
Forecasting groups
28
Forecasting the coefficients
yt,x = ft (x) + σt (x)εt,x
K
X
ft (x) = µ(x) +
βt,k φk (x) + et (x)
k =1
We use ARIMA models for each coefficient
{β1,j ,k , . . . , βn,j ,k }.
The ARIMA models are non-stationary for the
first few coefficients (k = 1, 2)
Non-stationary ARIMA forecasts will diverge.
Hence the mortality forecasts are not coherent.
Demographic forecasting using functional data analysis
Forecasting groups
28
20 40 60 80
0.2
φ3(x)
−0.1
0.0
0.1
0.1
0
−0.3
0
−0.1 0.0
φ2(x)
0.10
0.05
−6
φ1(x)
−4
−2
0.15
0.2
Australian male death rates
−8
20 40 60 80
0
20 40 60 80
Age
0
20 40 60 80
Age
Age
−2.5
1960
2000
Year
Demographic forecasting using functional data analysis
0.0
−1.0
−0.5
−1.5
βt3
βt2
−5
βt1
−0.5
0
0.5
0.5
5
Age
−10
µM(x)
Male fts model
1960
2000
Year
Forecasting groups
1960
2000
Year
29
20 40 60 80
20 40 60 80
0.1
20 40 60 80
0
0.4
0.5
Age
βt3
−0.8
−1.0
−0.4
βt2
0.0
0.0
5
Demographic forecasting using functional data analysis
20 40 60 80
Age
0
βt1
−5
−10
2000
Year
0.0
φ3(x)
−0.2
0
Age
1960
−0.1
0.2
−0.1
0
Age
−0.5
0
0.0
φ2(x)
0.10
0.05
−6
φ1(x)
−4
0.1
0.15
−2
Australian female death rates
−8
µF(x)
Female fts model
1960
2000
Year
Forecasting groups
1960
2000
Year
30
Australian mortality forecasts
−4
−10
−8
−6
Log death rate
−4
−6
−10
−8
Log death rate
−2
(b) Females
−2
(a) Males
0
20
40
60
80
100
Age
Demographic forecasting using functional data analysis
0
20
40
60
80
100
Age
Forecasting groups
31
Mortality product and ratios
Key idea
Model the geometric mean and the mortality ratio
instead of the individual rates for each sex
separately.
q
q
.
pt (x) = ft,M (x)ft,F (x) and rt (x) = ft,M (x) ft,F (x).
Product and ratio are approximately
independent
Ratio should be stationary (for coherence) but
product can be non-stationary.
Demographic forecasting using functional data analysis
Forecasting groups
32
Mortality product and ratios
Key idea
Model the geometric mean and the mortality ratio
instead of the individual rates for each sex
separately.
q
q
.
pt (x) = ft,M (x)ft,F (x) and rt (x) = ft,M (x) ft,F (x).
Product and ratio are approximately
independent
Ratio should be stationary (for coherence) but
product can be non-stationary.
Demographic forecasting using functional data analysis
Forecasting groups
32
Mortality product and ratios
Key idea
Model the geometric mean and the mortality ratio
instead of the individual rates for each sex
separately.
q
q
.
pt (x) = ft,M (x)ft,F (x) and rt (x) = ft,M (x) ft,F (x).
Product and ratio are approximately
independent
Ratio should be stationary (for coherence) but
product can be non-stationary.
Demographic forecasting using functional data analysis
Forecasting groups
32
Mortality product and ratios
Key idea
Model the geometric mean and the mortality ratio
instead of the individual rates for each sex
separately.
q
q
.
pt (x) = ft,M (x)ft,F (x) and rt (x) = ft,M (x) ft,F (x).
Product and ratio are approximately
independent
Ratio should be stationary (for coherence) but
product can be non-stationary.
Demographic forecasting using functional data analysis
Forecasting groups
32
Mortality rates
−4
−6
−8
Log death rate
−2
Australia gmean mortality: 1950
0
20
40
60
80
100
Age
Demographic forecasting using functional data analysis
Forecasting groups
33
Mortality rates
3.0
2.5
2.0
1.0
1.5
sex ratio: M/F
3.5
4.0
Australia mortality sex ratio 1950
0
20
40
60
80
100
Age
Demographic forecasting using functional data analysis
Forecasting groups
34
Model product and ratios
q
pt (x) = ft,M (x)ft,F (x) and
log[pt (x)] = µp (x) +
log[rt (x)] = µr (x) +
K
X
k =1
L
X
q
rt (x) =
.
ft,M (x) ft,F (x).
βt,k φk (x) + et (x)
γt,` ψ` (x) + wt (x).
`=1
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Forecasts: fn+h |n,M (x) = pn+h |n (x)r.n+h |n (x)
fn+h |n,F (x) = pn+h |n (x) rn+h |n (x).
Demographic forecasting using functional data analysis
Forecasting groups
35
Model product and ratios
q
pt (x) = ft,M (x)ft,F (x) and
log[pt (x)] = µp (x) +
log[rt (x)] = µr (x) +
K
X
k =1
L
X
q
rt (x) =
.
ft,M (x) ft,F (x).
βt,k φk (x) + et (x)
γt,` ψ` (x) + wt (x).
`=1
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Forecasts: fn+h |n,M (x) = pn+h |n (x)r.n+h |n (x)
fn+h |n,F (x) = pn+h |n (x) rn+h |n (x).
Demographic forecasting using functional data analysis
Forecasting groups
35
Model product and ratios
q
pt (x) = ft,M (x)ft,F (x) and
log[pt (x)] = µp (x) +
log[rt (x)] = µr (x) +
K
X
k =1
L
X
q
rt (x) =
.
ft,M (x) ft,F (x).
βt,k φk (x) + et (x)
γt,` ψ` (x) + wt (x).
`=1
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Forecasts: fn+h |n,M (x) = pn+h |n (x)r.n+h |n (x)
fn+h |n,F (x) = pn+h |n (x) rn+h |n (x).
Demographic forecasting using functional data analysis
Forecasting groups
35
Model product and ratios
q
pt (x) = ft,M (x)ft,F (x) and
log[pt (x)] = µp (x) +
log[rt (x)] = µr (x) +
K
X
k =1
L
X
q
rt (x) =
.
ft,M (x) ft,F (x).
βt,k φk (x) + et (x)
γt,` ψ` (x) + wt (x).
`=1
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Forecasts: fn+h |n,M (x) = pn+h |n (x)r.n+h |n (x)
fn+h |n,F (x) = pn+h |n (x) rn+h |n (x).
Demographic forecasting using functional data analysis
Forecasting groups
35
20 40 60 80
0.1
0.2
0
20 40 60 80
20 40 60 80
0
Age
βt3
βt2
0.2
−0.5
0.6
5
0
βt1
−1.5
−5
Demographic forecasting using functional data analysis
−0.6
−2.5
2000
Year
20 40 60 80
Age
−10
1960
0.0
φ3(x)
−0.1
−0.2
0
Age
0.5
Age
−0.2
0
0.0
−0.2
0.05
−0.1
φ2(x)
0.10
φ1(x)
0.1
0.15
−2
−4
−6
−8
µP(x)
Product model
1960
2000
Year
Forecasting groups
1960
2000
Year
36
20 40 60 80
0
0.4
0.2
0.1
φ3(x)
20 40 60 80
0
20 40 60 80
Age
0
20 40 60 80
Age
Age
1960
2000
Year
Demographic forecasting using functional data analysis
0.1
1960
2000
Year
Forecasting groups
−0.3
−0.6
−0.1
βt3
−0.2 0.0
βt2
−0.5
βt1
0.0
0.2
0.3
0.4
0.5
Age
−0.2
−0.05
0.0
0.05
φ2(x)
0.00
−0.10
0
0.3
0.25
0.15
0.20
φ1(x)
0.10
0.5
0.4
0.3
0.2
0.1
µR(x)
Ratio model
1960
2000
Year
37
−4
−6
−8
−10
Log of geometric mean death rate
−2
Product forecasts
0
20
40
60
80
100
Age
Demographic forecasting using functional data analysis
Forecasting groups
38
3.0
2.5
2.0
1.0
1.5
Sex ratio: M/F
3.5
4.0
Ratio forecasts
0
20
40
60
80
100
Age
Demographic forecasting using functional data analysis
Forecasting groups
39
Coherent forecasts
−4
−10
−8
−6
Log death rate
−4
−6
−10
−8
Log death rate
−2
(b) Females
−2
(a) Males
0
20
40
60
80
100
Age
Demographic forecasting using functional data analysis
0
20
40
60
80
100
Age
Forecasting groups
40
Ratio forecasts
3.5
3.0
2.5
1.0
1.5
2.0
Sex ratio of rates: M/F
3.0
2.5
2.0
1.0
1.5
Sex ratio of rates: M/F
3.5
4.0
Coherent forecasts
4.0
Independent forecasts
0
20
40
60
80
100
Age
Demographic forecasting using functional data analysis
0
20
40
60
80
100
Age
Forecasting groups
41
Life expectancy forecasts
Life expectancy difference: F−M
4
70
2
75
Age
Number of years
80
6
85
8
Life expectancy forecasts
1960
1980
2000
2020
Year
Demographic forecasting using functional data analysis
1960
1980
2000
2020
Year
Forecasting groups
42
Coherent forecasts for J groups
and
pt (x) = [ft,1 (x)ft,2 (x) · · · ft,J (x)]1/J
.
rt,j (x) = ft,j (x) pt (x),
log[pt (x)] = µp (x) +
K
X
βt,k φk (x) + et (x)
k =1
L
X
log[rt,j (x)] = µr,j (x) +
γt,l ,j ψl ,j (x) + wt,j (x).
l =1
pt (x) and all rt,j (x)
are approximately
independent.
Demographic forecasting using functional data analysis
Ratios satisfy constraint
rt,1 (x)rt,2 (x) · · · rt,J (x) = 1.
log[ft,j (x)] = log[pt (x)rt,j (x)]
Forecasting groups
43
Coherent forecasts for J groups
and
pt (x) = [ft,1 (x)ft,2 (x) · · · ft,J (x)]1/J
.
rt,j (x) = ft,j (x) pt (x),
log[pt (x)] = µp (x) +
K
X
βt,k φk (x) + et (x)
k =1
L
X
log[rt,j (x)] = µr,j (x) +
γt,l ,j ψl ,j (x) + wt,j (x).
l =1
pt (x) and all rt,j (x)
are approximately
independent.
Demographic forecasting using functional data analysis
Ratios satisfy constraint
rt,1 (x)rt,2 (x) · · · rt,J (x) = 1.
log[ft,j (x)] = log[pt (x)rt,j (x)]
Forecasting groups
43
Coherent forecasts for J groups
and
pt (x) = [ft,1 (x)ft,2 (x) · · · ft,J (x)]1/J
.
rt,j (x) = ft,j (x) pt (x),
log[pt (x)] = µp (x) +
K
X
βt,k φk (x) + et (x)
k =1
L
X
log[rt,j (x)] = µr,j (x) +
γt,l ,j ψl ,j (x) + wt,j (x).
l =1
pt (x) and all rt,j (x)
are approximately
independent.
Demographic forecasting using functional data analysis
Ratios satisfy constraint
rt,1 (x)rt,2 (x) · · · rt,J (x) = 1.
log[ft,j (x)] = log[pt (x)rt,j (x)]
Forecasting groups
43
Coherent forecasts for J groups
and
pt (x) = [ft,1 (x)ft,2 (x) · · · ft,J (x)]1/J
.
rt,j (x) = ft,j (x) pt (x),
log[pt (x)] = µp (x) +
K
X
βt,k φk (x) + et (x)
k =1
L
X
log[rt,j (x)] = µr,j (x) +
γt,l ,j ψl ,j (x) + wt,j (x).
l =1
pt (x) and all rt,j (x)
are approximately
independent.
Demographic forecasting using functional data analysis
Ratios satisfy constraint
rt,1 (x)rt,2 (x) · · · rt,J (x) = 1.
log[ft,j (x)] = log[pt (x)rt,j (x)]
Forecasting groups
43
Coherent forecasts for J groups
and
pt (x) = [ft,1 (x)ft,2 (x) · · · ft,J (x)]1/J
.
rt,j (x) = ft,j (x) pt (x),
log[pt (x)] = µp (x) +
K
X
βt,k φk (x) + et (x)
k =1
L
X
log[rt,j (x)] = µr,j (x) +
γt,l ,j ψl ,j (x) + wt,j (x).
l =1
pt (x) and all rt,j (x)
are approximately
independent.
Demographic forecasting using functional data analysis
Ratios satisfy constraint
rt,1 (x)rt,2 (x) · · · rt,J (x) = 1.
log[ft,j (x)] = log[pt (x)rt,j (x)]
Forecasting groups
43
Coherent forecasts for J groups
log[ft,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ]
K
L
X
X
= µj (x) +
βt,k φk (x) +
γt,`,j ψ`,j (x) + zt,j (x)
k =1
`=1
µj (x) = µp (x) + µr,j (x) is group mean
zt,j (x) = et (x) + wt,j (x) is error term.
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Demographic forecasting using functional data analysis
Forecasting groups
44
Coherent forecasts for J groups
log[ft,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ]
K
L
X
X
= µj (x) +
βt,k φk (x) +
γt,`,j ψ`,j (x) + zt,j (x)
k =1
`=1
µj (x) = µp (x) + µr,j (x) is group mean
zt,j (x) = et (x) + wt,j (x) is error term.
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Demographic forecasting using functional data analysis
Forecasting groups
44
Coherent forecasts for J groups
log[ft,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ]
K
L
X
X
= µj (x) +
βt,k φk (x) +
γt,`,j ψ`,j (x) + zt,j (x)
k =1
`=1
µj (x) = µp (x) + µr,j (x) is group mean
zt,j (x) = et (x) + wt,j (x) is error term.
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Demographic forecasting using functional data analysis
Forecasting groups
44
Coherent forecasts for J groups
log[ft,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ]
K
L
X
X
= µj (x) +
βt,k φk (x) +
γt,`,j ψ`,j (x) + zt,j (x)
k =1
`=1
µj (x) = µp (x) + µr,j (x) is group mean
zt,j (x) = et (x) + wt,j (x) is error term.
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Demographic forecasting using functional data analysis
Forecasting groups
44
Coherent forecasts for J groups
log[ft,j (x)] = log[pt (x)rt,j (x)] = log[pt (x)] + log[rt,j ]
K
L
X
X
= µj (x) +
βt,k φk (x) +
γt,`,j ψ`,j (x) + zt,j (x)
k =1
`=1
µj (x) = µp (x) + µr,j (x) is group mean
zt,j (x) = et (x) + wt,j (x) is error term.
{γt,` } restricted to be stationary processes:
either ARMA(p, q) or ARFIMA(p, d , q).
No restrictions for βt,1 , . . . , βt,K .
Demographic forecasting using functional data analysis
Forecasting groups
44
Outline
1
A functional linear model
2
Bagplots, boxplots and outliers
3
Functional forecasting
4
Forecasting groups
5
Population forecasting
6
References
Demographic forecasting using functional data analysis
Population forecasting
45
Demographic growth-balance equation
Demographic growth-balance equation
Pt+1 (x + 1) = Pt (x) − Dt (x, x + 1) + Gt (x, x + 1)
Pt+1 (0) = Bt
− Dt (B , 0)
+ Gt (B , 0)
x = 0, 1, 2, . . . .
Pt (x) = population of age x at 1 January, year t
Bt = births in calendar year t
Dt (x, x + 1) = deaths in calendar year t of persons aged x at
the beginning of year t
Dt (B , 0) = infant deaths in calendar year t
Gt (x, x + 1) = net migrants in calendar year t of persons aged
x at the beginning of year t
Gt (B , 0) = net migrants of infants born in calendar year t
Demographic forecasting using functional data analysis
Population forecasting
46
Demographic growth-balance equation
Demographic growth-balance equation
Pt+1 (x + 1) = Pt (x) − Dt (x, x + 1) + Gt (x, x + 1)
Pt+1 (0) = Bt
− Dt (B , 0)
+ Gt (B , 0)
x = 0, 1, 2, . . . .
Pt (x) = population of age x at 1 January, year t
Bt = births in calendar year t
Dt (x, x + 1) = deaths in calendar year t of persons aged x at
the beginning of year t
Dt (B , 0) = infant deaths in calendar year t
Gt (x, x + 1) = net migrants in calendar year t of persons aged
x at the beginning of year t
Gt (B , 0) = net migrants of infants born in calendar year t
Demographic forecasting using functional data analysis
Population forecasting
46
Key ideas
Build a stochastic functional model for each
of mortality, fertility and net migration.
Treat all observed data as functional (i.e.,
smooth curves of age) rather than discrete
values.
Use the models to simulate future sample
paths of all components giving the entire age
distribution at every year into the future.
Compute future births, deaths, net migrants.
and populations from simulated rates.
Combine the results to get age-specific
stochastic population forecasts.
Demographic forecasting using functional data analysis
Population forecasting
47
Key ideas
Build a stochastic functional model for each
of mortality, fertility and net migration.
Treat all observed data as functional (i.e.,
smooth curves of age) rather than discrete
values.
Use the models to simulate future sample
paths of all components giving the entire age
distribution at every year into the future.
Compute future births, deaths, net migrants.
and populations from simulated rates.
Combine the results to get age-specific
stochastic population forecasts.
Demographic forecasting using functional data analysis
Population forecasting
47
Key ideas
Build a stochastic functional model for each
of mortality, fertility and net migration.
Treat all observed data as functional (i.e.,
smooth curves of age) rather than discrete
values.
Use the models to simulate future sample
paths of all components giving the entire age
distribution at every year into the future.
Compute future births, deaths, net migrants.
and populations from simulated rates.
Combine the results to get age-specific
stochastic population forecasts.
Demographic forecasting using functional data analysis
Population forecasting
47
Key ideas
Build a stochastic functional model for each
of mortality, fertility and net migration.
Treat all observed data as functional (i.e.,
smooth curves of age) rather than discrete
values.
Use the models to simulate future sample
paths of all components giving the entire age
distribution at every year into the future.
Compute future births, deaths, net migrants.
and populations from simulated rates.
Combine the results to get age-specific
stochastic population forecasts.
Demographic forecasting using functional data analysis
Population forecasting
47
Key ideas
Build a stochastic functional model for each
of mortality, fertility and net migration.
Treat all observed data as functional (i.e.,
smooth curves of age) rather than discrete
values.
Use the models to simulate future sample
paths of all components giving the entire age
distribution at every year into the future.
Compute future births, deaths, net migrants.
and populations from simulated rates.
Combine the results to get age-specific
stochastic population forecasts.
Demographic forecasting using functional data analysis
Population forecasting
47
The available data
In most countries, the following data are
available:
Pt (x) =
Et (x) =
Bt (x) =
Dt (x) =
population of age x at 1 January, year t
population of age x at 30 June, year t
births in calendar year t to females of age x
deaths in calendar year t of persons of age x
From these, we can estimate:
mt (x) = Dt (x)/Et (x) = central death rate in
calendar year t;
ft (x) = Bt (x)/EtF (x) = fertility rate for females of
age x in calendar year t.
Demographic forecasting using functional data analysis
Population forecasting
48
The available data
In most countries, the following data are
available:
Pt (x) =
Et (x) =
Bt (x) =
Dt (x) =
population of age x at 1 January, year t
population of age x at 30 June, year t
births in calendar year t to females of age x
deaths in calendar year t of persons of age x
From these, we can estimate:
mt (x) = Dt (x)/Et (x) = central death rate in
calendar year t;
ft (x) = Bt (x)/EtF (x) = fertility rate for females of
age x in calendar year t.
Demographic forecasting using functional data analysis
Population forecasting
48
The available data
In most countries, the following data are
available:
Pt (x) =
Et (x) =
Bt (x) =
Dt (x) =
population of age x at 1 January, year t
population of age x at 30 June, year t
births in calendar year t to females of age x
deaths in calendar year t of persons of age x
From these, we can estimate:
mt (x) = Dt (x)/Et (x) = central death rate in
calendar year t;
ft (x) = Bt (x)/EtF (x) = fertility rate for females of
age x in calendar year t.
Demographic forecasting using functional data analysis
Population forecasting
48
Net migration
We need to estimate migration data based on
difference in population numbers after
adjusting for births and deaths.
Demographic growth-balance equation
Gt (x, x + 1) = Pt+1 (x + 1) − Pt (x) + Dt (x, x + 1)
Gt (B , 0) = Pt+1 (0)
− Bt
+ Dt (B , 0)
x = 0, 1, 2, . . . .
Note: “net migration” numbers also include errors
associated with all estimates. i.e., a “residual”.
Demographic forecasting using functional data analysis
Population forecasting
49
Net migration
We need to estimate migration data based on
difference in population numbers after
adjusting for births and deaths.
Demographic growth-balance equation
Gt (x, x + 1) = Pt+1 (x + 1) − Pt (x) + Dt (x, x + 1)
Gt (B , 0) = Pt+1 (0)
− Bt
+ Dt (B , 0)
x = 0, 1, 2, . . . .
Note: “net migration” numbers also include errors
associated with all estimates. i.e., a “residual”.
Demographic forecasting using functional data analysis
Population forecasting
49
Net migration
We need to estimate migration data based on
difference in population numbers after
adjusting for births and deaths.
Demographic growth-balance equation
Gt (x, x + 1) = Pt+1 (x + 1) − Pt (x) + Dt (x, x + 1)
Gt (B , 0) = Pt+1 (0)
− Bt
+ Dt (B , 0)
x = 0, 1, 2, . . . .
Note: “net migration” numbers also include errors
associated with all estimates. i.e., a “residual”.
Demographic forecasting using functional data analysis
Population forecasting
49
Net migration
We need to estimate migration data based on
difference in population numbers after
adjusting for births and deaths.
Demographic growth-balance equation
Gt (x, x + 1) = Pt+1 (x + 1) − Pt (x) + Dt (x, x + 1)
Gt (B , 0) = Pt+1 (0)
− Bt
+ Dt (B , 0)
x = 0, 1, 2, . . . .
Note: “net migration” numbers also include errors
associated with all estimates. i.e., a “residual”.
Demographic forecasting using functional data analysis
Population forecasting
49
Net migration
4000
2000
−2000
0
Net migration
6000
8000
Australia: male net migration (1973−2008)
0
20
40
60
80
100
Age
Demographic forecasting using functional data analysis
Population forecasting
50
Net migration
4000
2000
−2000
0
Net migration
6000
8000
Australia: female net migration (1973−2008)
0
20
40
60
80
100
Age
Demographic forecasting using functional data analysis
Population forecasting
50
Stochastic population forecasts
Component models
Data: age/sex-specific mortality rates, fertility
rates and net migration.
Models: Functional time series models for
mortality (M/F), fertility and net migration (M/F)
assuming independence between components
and coherence between sexes.
Generate random sample paths of each
component conditional on observed data.
Use simulated rates to generate Bt (x),
DtF (x, x + 1), DtM (x, x + 1) for t = n + 1, . . . , n + h ,
assuming deaths and births are Poisson.
Demographic forecasting using functional data analysis
Population forecasting
51
Stochastic population forecasts
Component models
Data: age/sex-specific mortality rates, fertility
rates and net migration.
Models: Functional time series models for
mortality (M/F), fertility and net migration (M/F)
assuming independence between components
and coherence between sexes.
Generate random sample paths of each
component conditional on observed data.
Use simulated rates to generate Bt (x),
DtF (x, x + 1), DtM (x, x + 1) for t = n + 1, . . . , n + h ,
assuming deaths and births are Poisson.
Demographic forecasting using functional data analysis
Population forecasting
51
Stochastic population forecasts
Component models
Data: age/sex-specific mortality rates, fertility
rates and net migration.
Models: Functional time series models for
mortality (M/F), fertility and net migration (M/F)
assuming independence between components
and coherence between sexes.
Generate random sample paths of each
component conditional on observed data.
Use simulated rates to generate Bt (x),
DtF (x, x + 1), DtM (x, x + 1) for t = n + 1, . . . , n + h ,
assuming deaths and births are Poisson.
Demographic forecasting using functional data analysis
Population forecasting
51
Stochastic population forecasts
Component models
Data: age/sex-specific mortality rates, fertility
rates and net migration.
Models: Functional time series models for
mortality (M/F), fertility and net migration (M/F)
assuming independence between components
and coherence between sexes.
Generate random sample paths of each
component conditional on observed data.
Use simulated rates to generate Bt (x),
DtF (x, x + 1), DtM (x, x + 1) for t = n + 1, . . . , n + h ,
assuming deaths and births are Poisson.
Demographic forecasting using functional data analysis
Population forecasting
51
Simulation
Demographic growth-balance equation used to get
population sample paths.
Demographic growth-balance equation
Pt+1 (x + 1) = Pt (x) − Dt (x, x + 1) + Gt (x, x + 1)
Pt+1 (0) = Bt
− Dt (B , 0)
+ Gt (B , 0)
x = 0, 1, 2, . . . .
10000 sample paths of population Pt (x), deaths
Dt (x) and births Bt (x) generated for
t = 2004, . . . , 2023 and x = 0, 1, 2, . . . ,.
This allows the computation of the empirical
forecast distribution of any demographic
quantity that is based on births, deaths and
population numbers.
Demographic forecasting using functional data analysis
Population forecasting
52
Simulation
Demographic growth-balance equation used to get
population sample paths.
Demographic growth-balance equation
Pt+1 (x + 1) = Pt (x) − Dt (x, x + 1) + Gt (x, x + 1)
Pt+1 (0) = Bt
− Dt (B , 0)
+ Gt (B , 0)
x = 0, 1, 2, . . . .
10000 sample paths of population Pt (x), deaths
Dt (x) and births Bt (x) generated for
t = 2004, . . . , 2023 and x = 0, 1, 2, . . . ,.
This allows the computation of the empirical
forecast distribution of any demographic
quantity that is based on births, deaths and
population numbers.
Demographic forecasting using functional data analysis
Population forecasting
52
Simulation
Demographic growth-balance equation used to get
population sample paths.
Demographic growth-balance equation
Pt+1 (x + 1) = Pt (x) − Dt (x, x + 1) + Gt (x, x + 1)
Pt+1 (0) = Bt
− Dt (B , 0)
+ Gt (B , 0)
x = 0, 1, 2, . . . .
10000 sample paths of population Pt (x), deaths
Dt (x) and births Bt (x) generated for
t = 2004, . . . , 2023 and x = 0, 1, 2, . . . ,.
This allows the computation of the empirical
forecast distribution of any demographic
quantity that is based on births, deaths and
population numbers.
Demographic forecasting using functional data analysis
Population forecasting
52
Forecasts of TFR
2500
2000
TFR
3000
3500
Forecast Total Fertility Rate
1920
1940
1960
1980
2000
2020
Year
Demographic forecasting using functional data analysis
Population forecasting
53
Population forecasts
2009
150
100
2009
50
0
50
100
Age
Female
0 10 20 30 40 50 60 70 80 90 100
Male
0 10 20 30 40 50 60 70 80 90 100
Age
Forecast population: 2028
150
Population ('000)
Forecast population pyramid for 2028, along with
80% prediction intervals. Dashed: actual population
pyramid for 2009.
Demographic forecasting using functional data analysis
Population forecasting
54
Population forecasts
10
12
Population (millions)
14
16
Total females
●
●
●
●
●
8
●
●●
●●
●●
●
●●
●●
●●
●●
●●
●●
●●
●
●
●●
●●
●●
●
●●
●
●●
●
●
1980
1990
2000
2010
2020
2030
Year
Demographic forecasting using functional data analysis
Population forecasting
55
Population forecasts
10
12
Population (millions)
14
16
Total males
●
●
●
●
8
●●
●●
●●
●
●
●●
●●
●●
●
●
●●
●●
●●
●
●●
●●
●●
●●
●●
●
●
●●
●
●
1980
1990
2000
2010
2020
2030
Year
Demographic forecasting using functional data analysis
Population forecasting
55
Old-age dependency ratio
0.20
0.10
0.15
ratio
0.25
0.30
Old−age dependency ratio forecasts
1920
1940
1960
1980
2000
2020
Year
Demographic forecasting using functional data analysis
Population forecasting
56
Advantages of stochastic simulation approach
Functional data analysis provides a way of
forecasting age-specific mortality, fertility and
net migration.
Stochastic age-specific cohort-component
simulation provides a way of forecasting many
demographic quantities with prediction
intervals.
No need to select combinations of assumed
rates.
True prediction intervals with specified
coverage for population and all derived
variables (TFR, life expectancy, old-age
dependencies, etc.)
Demographic forecasting using functional data analysis
Population forecasting
57
Advantages of stochastic simulation approach
Functional data analysis provides a way of
forecasting age-specific mortality, fertility and
net migration.
Stochastic age-specific cohort-component
simulation provides a way of forecasting many
demographic quantities with prediction
intervals.
No need to select combinations of assumed
rates.
True prediction intervals with specified
coverage for population and all derived
variables (TFR, life expectancy, old-age
dependencies, etc.)
Demographic forecasting using functional data analysis
Population forecasting
57
Advantages of stochastic simulation approach
Functional data analysis provides a way of
forecasting age-specific mortality, fertility and
net migration.
Stochastic age-specific cohort-component
simulation provides a way of forecasting many
demographic quantities with prediction
intervals.
No need to select combinations of assumed
rates.
True prediction intervals with specified
coverage for population and all derived
variables (TFR, life expectancy, old-age
dependencies, etc.)
Demographic forecasting using functional data analysis
Population forecasting
57
Advantages of stochastic simulation approach
Functional data analysis provides a way of
forecasting age-specific mortality, fertility and
net migration.
Stochastic age-specific cohort-component
simulation provides a way of forecasting many
demographic quantities with prediction
intervals.
No need to select combinations of assumed
rates.
True prediction intervals with specified
coverage for population and all derived
variables (TFR, life expectancy, old-age
dependencies, etc.)
Demographic forecasting using functional data analysis
Population forecasting
57
Outline
1
A functional linear model
2
Bagplots, boxplots and outliers
3
Functional forecasting
4
Forecasting groups
5
Population forecasting
6
References
Demographic forecasting using functional data analysis
References
58
Selected references
Hyndman, Shang (2010). “Rainbow plots, bagplots and boxplots for
functional data”. Journal of Computational and Graphical Statistics
19(1), 29–45
Hyndman, Ullah (2007). “Robust forecasting of mortality and fertility
rates: A functional data approach”. Computational Statistics and
Data Analysis 51(10), 4942–4956
Hyndman, Shang (2009). “Forecasting functional time series (with
discussion)”. Journal of the Korean Statistical Society 38(3),
199–221
Shang, Booth, Hyndman (2011). “Point and interval forecasts of
mortality rates and life expectancy : a comparison of ten principal
component methods”. Demographic Research 25(5), 173–214
Hyndman, Booth (2008). “Stochastic population forecasts using
functional data models for mortality, fertility and migration”.
International Journal of Forecasting 24(3), 323–342
Hyndman, Booth, Yasmeen (2012). “Coherent mortality forecasting:
the product-ratio method with functional time series models”.
Demography, to appear
Hyndman (2012). demography: Forecasting mortality, fertility,
migration and population data.
cran.r-project.org/package=demography
Demographic forecasting using functional data analysis
References
59
Selected references
Hyndman, Shang (2010). “Rainbow plots, bagplots and boxplots for
functional data”. Journal of Computational and Graphical Statistics
19(1), 29–45
Hyndman, Ullah (2007). “Robust forecasting of mortality and fertility
rates: A functional data approach”. Computational Statistics and
Data Analysis 51(10), 4942–4956
Hyndman, Shang (2009). “Forecasting functional time series (with
discussion)”. Journal of the Korean Statistical Society 38(3),
199–221
Shang, Booth, Hyndman (2011). “Point and interval forecasts of
mortality rates and life expectancy : a comparison of ten principal
component methods”. Demographic Research 25(5), 173–214
Hyndman, Booth (2008). “Stochastic population forecasts using
functional data models for mortality, fertility and migration”.
International Journal of Forecasting 24(3), 323–342
Hyndman, Booth, Yasmeen (2012). “Coherent mortality forecasting:
the product-ratio method with functional time series models”.
Demography, to appear
Hyndman (2012). demography: Forecasting mortality, fertility,
migration and population data.
cran.r-project.org/package=demography
å Papers and R code:
robjhyndman.com
å Email: Rob.Hyndman@monash.edu
Demographic forecasting using functional data analysis
References
59