Rob J Hyndman Functional time series with applications in demography 4. Connections, extensions and applications Outline 1 Yield curves 2 Electricity prices 3 Dynamic updating with partially observed functions 4 Functional ARH models 5 References Functional time series with applications in demography 4. Connections, extensions and applications 2 Example: Yield curves 10 5 Yield 15 US Treasury Bonds 0 50 100 150 Functional time series with applications in demography 200 250 300 4. Connections, extensions and applications 350 3 Example: Yield curves US Treasury bonds 8 Jan 1989 6 Jan 1991 Jan Jan 1995 1993 Jan 1987 Jan 1997 Jan 2001 Jan 1999 4 Bid−ask midpoint average 10 Jan 1985 0 100 Functional time series with applications in demography 200 300 4. Connections, extensions and applications 400 3 Example: Yield curves 9 10 11 Jan 1985 Jan 1980 8 Jan 1990 Jan 1975 Jan Jan 1970 1995 7 Bid−ask midpoint average 12 US Treasury bonds 6 Jan 2000 0 100 Functional time series with applications in demography 200 300 4. Connections, extensions and applications 400 3 Model for yield curves PC3 0.15 PC2 50 150 250 350 0.05 0 50 150 250 350 0 50 150 250 Term Term Term B1 B2 B3 4 350 −8 −50 −10 −4 0 0 0 5 50 2 100 10 15 0 −0.05 0.060 −0.20 0.064 −0.10 0.068 0.00 0.072 PC1 1970 1980 1990 2000 1970 1980 Month Functional time series with applications in demography 1990 Month 2000 1970 1980 1990 2000 Month 4. Connections, extensions and applications 4 Forecasts of coefficients ARIMA forecasts (with first differencing). 80% prediction intervals shown in yellow. Forecasts from ARIMA(4,1,0) Forecasts from ARIMA(1,1,2) 1970 1980 1990 2000 0 1970 1980 Month Functional time series with applications in demography 1990 Month 2000 −8 −6 −4 −2 B3 B2 −100 −10 −50 0 0 B1 10 2 50 4 20 100 Forecasts from ARIMA(0,1,3) 1970 1980 1990 2000 Month 4. Connections, extensions and applications 5 2 year forecasts for yield curves 4 3 2 1 Bid−ask midpoint average 5 Yield curve forecasts: 2003−2004 0 50 100 150 200 250 300 350 Term Functional time series with applications in demography 4. Connections, extensions and applications 6 Nelson-Siegel models Functional time series model yt,x = µ(x) + K X βt,k φk (x) + rt (x) + εt,x k =1 Nelson-Siegel model yt,x = β1,t + β2,t 1−e−λt x λt x + β3,t e−λt x + rt (x) Well-behaved at long maturities. λt is usually fixed and pre-specified. Useful for describing yield curves, and for estimating yield for unobserved x. Coefficients highly collinear making interpretation and forecasting difficult. Functional time series with applications in demography 4. Connections, extensions and applications 7 Nelson-Siegel models Functional time series model yt,x = µ(x) + K X βt,k φk (x) + rt (x) + εt,x k =1 Nelson-Siegel model yt,x = β1,t + β2,t 1−e−λt x λt x + β3,t e−λt x + rt (x) Well-behaved at long maturities. λt is usually fixed and pre-specified. Useful for describing yield curves, and for estimating yield for unobserved x. Coefficients highly collinear making interpretation and forecasting difficult. Functional time series with applications in demography 4. Connections, extensions and applications 7 Nelson-Siegel models Functional time series model yt,x = µ(x) + K X βt,k φk (x) + rt (x) + εt,x k =1 Nelson-Siegel model yt,x = β1,t + β2,t 1−e−λt x λt x + β3,t e−λt x + rt (x) Well-behaved at long maturities. λt is usually fixed and pre-specified. Useful for describing yield curves, and for estimating yield for unobserved x. Coefficients highly collinear making interpretation and forecasting difficult. Functional time series with applications in demography 4. Connections, extensions and applications 7 Nelson-Siegel models Functional time series model yt,x = µ(x) + K X βt,k φk (x) + rt (x) + εt,x k =1 Nelson-Siegel model yt,x = β1,t + β2,t 1−e−λt x λt x + β3,t e−λt x + rt (x) Well-behaved at long maturities. λt is usually fixed and pre-specified. Useful for describing yield curves, and for estimating yield for unobserved x. Coefficients highly collinear making interpretation and forecasting difficult. Functional time series with applications in demography 4. Connections, extensions and applications 7 Nelson-Siegel models Functional time series model yt,x = µ(x) + K X βt,k φk (x) + rt (x) + εt,x k =1 Nelson-Siegel model yt,x = β1,t + β2,t 1−e−λt x λt x + β3,t e−λt x + rt (x) Well-behaved at long maturities. λt is usually fixed and pre-specified. Useful for describing yield curves, and for estimating yield for unobserved x. Coefficients highly collinear making interpretation and forecasting difficult. Functional time series with applications in demography 4. Connections, extensions and applications 7 Nelson-Siegel models Functional time series model yt,x = µ(x) + K X βt,k φk (x) + rt (x) + εt,x k =1 Nelson-Siegel model yt,x = β1,t + β2,t 1−e−λt x λt x + β3,t e−λt x + rt (x) Well-behaved at long maturities. λt is usually fixed and pre-specified. Useful for describing yield curves, and for estimating yield for unobserved x. Coefficients highly collinear making interpretation and forecasting difficult. Functional time series with applications in demography 4. Connections, extensions and applications 7 Diebold-Li model Functional time series model yt,x = µ(x) + K X βt,k φk (x) + rt (x) + εt,x k =1 Nelson-Siegel reparameterization yt,x = β1,t + β2,t 1−e−λt x λt x + β3,t 1−e−λt x λt x − e−λt x + rt (x) This parameterization has lower collinearity than the original. Provides interpretable coefficients (level, slope, curvature) All coefficients usually forecast with univariate AR(1) models. Functional time series with applications in demography 4. Connections, extensions and applications 8 Diebold-Li model Functional time series model yt,x = µ(x) + K X βt,k φk (x) + rt (x) + εt,x k =1 Nelson-Siegel reparameterization yt,x = β1,t + β2,t 1−e−λt x λt x + β3,t 1−e−λt x λt x − e−λt x + rt (x) This parameterization has lower collinearity than the original. Provides interpretable coefficients (level, slope, curvature) All coefficients usually forecast with univariate AR(1) models. Functional time series with applications in demography 4. Connections, extensions and applications 8 Diebold-Li model Functional time series model yt,x = µ(x) + K X βt,k φk (x) + rt (x) + εt,x k =1 Nelson-Siegel reparameterization yt,x = β1,t + β2,t 1−e−λt x λt x + β3,t 1−e−λt x λt x − e−λt x + rt (x) This parameterization has lower collinearity than the original. Provides interpretable coefficients (level, slope, curvature) All coefficients usually forecast with univariate AR(1) models. Functional time series with applications in demography 4. Connections, extensions and applications 8 Diebold-Li model Functional time series model yt,x = µ(x) + K X βt,k φk (x) + rt (x) + εt,x k =1 Nelson-Siegel reparameterization yt,x = β1,t + β2,t 1−e−λt x λt x + β3,t 1−e−λt x λt x − e−λt x + rt (x) This parameterization has lower collinearity than the original. Provides interpretable coefficients (level, slope, curvature) All coefficients usually forecast with univariate AR(1) models. Functional time series with applications in demography 4. Connections, extensions and applications 8 Diebold-Li model Functional time series model yt,x = µ(x) + K X βt,k φk (x) + rt (x) + εt,x k =1 Nelson-Siegel reparameterization yt,x = β1,t + β2,t 1−e−λt x λt x + β3,t 1−e−λt x λt x − e−λt x + rt (x) This parameterization has lower collinearity than the original. Provides interpretable coefficients (level, slope, curvature) All coefficients usually forecast with univariate AR(1) models. Functional time series with applications in demography 4. Connections, extensions and applications 8 Diebold-Li model Functional time series with applications in demography 4. Connections, extensions and applications 9 Diebold-Li model x= Functional time series with applications in demography 4. Connections, extensions and applications 10 Diebold-Li model Functional time series with applications in demography 4. Connections, extensions and applications 11 Diebold-Li model Functional time series with applications in demography 4. Connections, extensions and applications 11 Bowsher-Meeks model Functional time series model yt,x = st (x) + σt (x)εt,x , st (x) = µ(x) + K X βt,k φk (x) + rt (x) k =1 Bowsher-Meeks model y t ,x = J X γt,j ξj (x) + σt (x)εt,x j=1 {ξj (x)} are spline terms; ∆γt+1 = A(Bγt − µ) + Ψ∆γt + et is a cointegrated VAR with et ∼ N(0, Ω). Functional time series with applications in demography 4. Connections, extensions and applications 12 Bowsher-Meeks model Functional time series model yt,x = st (x) + σt (x)εt,x , st (x) = µ(x) + K X βt,k φk (x) + rt (x) k =1 Bowsher-Meeks model y t ,x = J X γt,j ξj (x) + σt (x)εt,x j=1 {ξj (x)} are spline terms; ∆γt+1 = A(Bγt − µ) + Ψ∆γt + et is a cointegrated VAR with et ∼ N(0, Ω). Functional time series with applications in demography 4. Connections, extensions and applications 12 Bowsher-Meeks model Functional time series model yt,x = st (x) + σt (x)εt,x , st (x) = µ(x) + K X βt,k φk (x) + rt (x) k =1 Bowsher-Meeks model y t ,x = J X γt,j ξj (x) + σt (x)εt,x j=1 {ξj (x)} are spline terms; ∆γt+1 = A(Bγt − µ) + Ψ∆γt + et is a cointegrated VAR with et ∼ N(0, Ω). Functional time series with applications in demography 4. Connections, extensions and applications 12 Bowsher-Meeks model Functional time series model yt,x = st (x) + σt (x)εt,x , st (x) = µ(x) + K X βt,k φk (x) + rt (x) k =1 Bowsher-Meeks model y t ,x = J X γt,j ξj (x) + σt (x)εt,x j=1 {ξj (x)} are spline terms; ∆γt+1 = A(Bγt − µ) + Ψ∆γt + et is a cointegrated VAR with et ∼ N(0, Ω). Functional time series with applications in demography 4. Connections, extensions and applications 12 Hays-Shen-Huang model Functional time series model yt,x = st (x) + σt (x)εt,x , st (x) = µ(x) + K X βt,k φk (x) + rt (x) k =1 Functional dynamic factor model st (x) = K X βt,k φk (x) + rt (x) k =1 p β t ,k − µ k = X ψr ,k (βt−r ,k − µk ) + vt,k r =1 Equations estimated simultaneously using penalized likelihood (analogous to PCA but taking account of autocorrelations). Functional time series with applications in demography 4. Connections, extensions and applications 13 Hays-Shen-Huang model Functional time series model yt,x = st (x) + σt (x)εt,x , st (x) = µ(x) + K X βt,k φk (x) + rt (x) k =1 Functional dynamic factor model st (x) = K X βt,k φk (x) + rt (x) k =1 p β t ,k − µ k = X ψr ,k (βt−r ,k − µk ) + vt,k r =1 Equations estimated simultaneously using penalized likelihood (analogous to PCA but taking account of autocorrelations). Functional time series with applications in demography 4. Connections, extensions and applications 13 Hays-Shen-Huang model Functional time series model yt,x = st (x) + σt (x)εt,x , st (x) = µ(x) + K X βt,k φk (x) + rt (x) k =1 Functional dynamic factor model st (x) = K X βt,k φk (x) + rt (x) k =1 p β t ,k − µ k = X ψr ,k (βt−r ,k − µk ) + vt,k r =1 Equations estimated simultaneously using penalized likelihood (analogous to PCA but taking account of autocorrelations). Functional time series with applications in demography 4. Connections, extensions and applications 13 Hays-Shen-Huang model Functional time series with applications in demography 4. Connections, extensions and applications 14 Compare FPCA PC3 0.15 PC2 50 150 250 350 0.05 0 50 150 250 350 0 50 150 250 Term Term Term B1 B2 B3 4 350 −8 −50 −10 −4 0 0 0 5 50 2 100 10 15 0 −0.05 0.060 −0.20 0.064 −0.10 0.068 0.00 0.072 PC1 1970 1980 1990 2000 1970 1980 Month Functional time series with applications in demography 1990 Month 2000 1970 1980 1990 2000 Month 4. Connections, extensions and applications 15 Outline 1 Yield curves 2 Electricity prices 3 Dynamic updating with partially observed functions 4 Functional ARH models 5 References Functional time series with applications in demography 4. Connections, extensions and applications 16 Electricity prices Price Time of day Functional time series with applications in demography Days 4. Connections, extensions and applications 17 Electricity prices Price set on the previous day, so each curve known a day ahead. Price is a smooth function of time of day, but varies from day to day. Price So a functional time series model is appropriate. Time of day Functional time series with applications in demography Days 4. Connections, extensions and applications 17 Electricity prices Price set on the previous day, so each curve known a day ahead. Price is a smooth function of time of day, but varies from day to day. Price So a functional time series model is appropriate. Time of day Functional time series with applications in demography Days 4. Connections, extensions and applications 17 Electricity prices Price set on the previous day, so each curve known a day ahead. Price is a smooth function of time of day, but varies from day to day. Price So a functional time series model is appropriate. Time of day Functional time series with applications in demography Days 4. Connections, extensions and applications 17 Electricity prices Functional time series models allows for multiple seasonality: time of day, day of week and time of year. Time of day is handled by the functions while day of week and time of year are handled via the PC scores. Different time of day patterns (e.g., weekdays and weekends) can be handled via different PCs. Functional time series with applications in demography 4. Connections, extensions and applications 18 Electricity prices Functional time series models allows for multiple seasonality: time of day, day of week and time of year. Time of day is handled by the functions while day of week and time of year are handled via the PC scores. Different time of day patterns (e.g., weekdays and weekends) can be handled via different PCs. Functional time series with applications in demography 4. Connections, extensions and applications 18 Electricity prices Functional time series models allows for multiple seasonality: time of day, day of week and time of year. Time of day is handled by the functions while day of week and time of year are handled via the PC scores. Different time of day patterns (e.g., weekdays and weekends) can be handled via different PCs. Functional time series with applications in demography 4. Connections, extensions and applications 18 Electricity prices Variation explained 4% 74% 3% 10% Eigenfunctions Functional time series with applications in demography 4. Connections, extensions and applications 19 Outline 1 Yield curves 2 Electricity prices 3 Dynamic updating with partially observed functions 4 Functional ARH models 5 References Functional time series with applications in demography 4. Connections, extensions and applications 20 Dynamic updating We split a seasonal univariate time series into sections of one year. yt (x) = observed value in year t and season x. Functional time series with applications in demography 4. Connections, extensions and applications 21 Dynamic updating We split a seasonal univariate time series into sections of one year. 26 24 22 20 Sea surface temperature 28 yt (x) = observed value in year t and season x. 1950 1960 1970 Functional time series with applications in demography 1980 1990 2000 4. Connections, extensions and applications 2010 21 Dynamic updating We split a seasonal univariate time series into sections of one year. 26 24 22 20 Sea surface temperature 28 yt (x) = observed value in year t and season x. 2 4 Functional time series with applications in demography 6 8 10 12 4. Connections, extensions and applications 21 26 24 22 20 Sea surface temperature 28 Sea surface temperatures 2 4 6 8 10 12 Month Functional time series with applications in demography 4. Connections, extensions and applications 22 28 Sea surface temperatures 26 an indicator of El Niño 22 24 å Some outliers identified and removed å Problem: how to forecast using a partially observed year? 20 Sea surface temperature å Monthly SST: 1950–2008, used as 2 4 6 8 10 12 Month Functional time series with applications in demography 4. Connections, extensions and applications 22 28 Sea surface temperatures 26 an indicator of El Niño 22 24 å Some outliers identified and removed å Problem: how to forecast using a partially observed year? 20 Sea surface temperature å Monthly SST: 1950–2008, used as 2 4 6 8 10 12 Month Functional time series with applications in demography 4. Connections, extensions and applications 22 28 Sea surface temperatures 26 an indicator of El Niño 22 24 å Some outliers identified and removed å Problem: how to forecast using a partially observed year? 20 Sea surface temperature å Monthly SST: 1950–2008, used as 2 4 6 8 10 12 Month Functional time series with applications in demography 4. Connections, extensions and applications 22 Dynamic updating: solution 1 p Season 1 Redefine the year 0 n n+1 Years Functional time series with applications in demography 4. Connections, extensions and applications 23 Dynamic updating: solution 1 p m Season 1 Redefine the year 0 n n+1 Years Functional time series with applications in demography 4. Connections, extensions and applications 23 p m Season 1 p m−1 Dynamic updating: solution 1 0 n n+1 n+2 Years Functional time series with applications in demography 4. Connections, extensions and applications 23 p m Season 1 p m−1 Dynamic updating: solution 1 0 n n+1 n+2 Years Functional time series with applications in demography 4. Connections, extensions and applications 23 p m Season 1 p m−1 Dynamic updating: solution 1 0 n n+1 n+2 Years Functional time series with applications in demography 4. Connections, extensions and applications 23 p m Season 1 p m−1 Dynamic updating: solution 1 0 n n+1 n+2 Years Functional time series with applications in demography 4. Connections, extensions and applications 23 Dynamic updating: solution 2 p m Season 1 OLS regression For each month, regress the month against the partial principal components. 0 n n+1 Years Functional time series with applications in demography 4. Connections, extensions and applications 24 Dynamic updating: solution 2 p m Season 1 OLS regression For each month, regress the month against the partial principal components. 0 n n+1 Years Functional time series with applications in demography 4. Connections, extensions and applications 24 Dynamic updating: solution 3 p m Season 1 Ridge regression For each month, regress the month against the partial principal components. 0 n n+1 Years Functional time series with applications in demography 4. Connections, extensions and applications 25 Dynamic updating Mean squared error Computed on a rolling forecast origin Univariate methods Dynamic updating MP RW SARIMA FTS BM OLS RR 0.69 1.45 0.98 0.74 0.69 1.04 0.48 MP = mean predictor (mean of prior data) RW = random walk SARIMA = seasonal ARIMA model FTS = functional time series BM = block moved OLS = ordinary least squares RR = ridge regression Functional time series with applications in demography 4. Connections, extensions and applications 26 Outline 1 Yield curves 2 Electricity prices 3 Dynamic updating with partially observed functions 4 Functional ARH models 5 References Functional time series with applications in demography 4. Connections, extensions and applications 27 Functional ARH model Autoregressive Hilbertian process of order 1 ARH(1) model Z ft (x) − µ(x) = [ft−1 (x) − µ(x)]θ(x, y) dy + et (x) = θ[ft−1 (x) − µ(x)] + et (x) RR θ2 (x, y) dx dy < 1 for stationarity. Bosq (2000) and Horváth & Kokoszka (2012) provide basic theory. Estimating θ(x, y) is very difficult to do well. Functional time series with applications in demography 4. Connections, extensions and applications 28 Functional ARH model Autoregressive Hilbertian process of order 1 ARH(1) model Z ft (x) − µ(x) = [ft−1 (x) − µ(x)]θ(x, y) dy + et (x) = θ[ft−1 (x) − µ(x)] + et (x) RR θ2 (x, y) dx dy < 1 for stationarity. Bosq (2000) and Horváth & Kokoszka (2012) provide basic theory. Estimating θ(x, y) is very difficult to do well. Functional time series with applications in demography 4. Connections, extensions and applications 28 Functional ARH model Autoregressive Hilbertian process of order 1 ARH(1) model Z ft (x) − µ(x) = [ft−1 (x) − µ(x)]θ(x, y) dy + et (x) = θ[ft−1 (x) − µ(x)] + et (x) RR θ2 (x, y) dx dy < 1 for stationarity. Bosq (2000) and Horváth & Kokoszka (2012) provide basic theory. Estimating θ(x, y) is very difficult to do well. Functional time series with applications in demography 4. Connections, extensions and applications 28 Functional ARH model ARH(p) model ft (x) − µ(x) = θ1 [ft−1 (x) − µ(x)] + · · · + θp [ft−p (x) − µ(x)] + et (x) θj () are linear functions on a Hilbert space et (x) is H white noise FPCA decompostion ft (x) = µ(x) + ∞ X βt,k φk (x) k =1 å Equivalence: βt is a VAR(p) process if ft (p) is an ARH(p). Functional time series with applications in demography 4. Connections, extensions and applications 29 Functional ARH model ARH(p) model ft (x) − µ(x) = θ1 [ft−1 (x) − µ(x)] + · · · + θp [ft−p (x) − µ(x)] + et (x) θj () are linear functions on a Hilbert space et (x) is H white noise FPCA decompostion ft (x) = µ(x) + ∞ X βt,k φk (x) k =1 å Equivalence: βt is a VAR(p) process if ft (p) is an ARH(p). Functional time series with applications in demography 4. Connections, extensions and applications 29 Functional ARH model ARH(p) model ft (x) − µ(x) = θ1 [ft−1 (x) − µ(x)] + · · · + θp [ft−p (x) − µ(x)] + et (x) θj () are linear functions on a Hilbert space et (x) is H white noise FPCA decompostion ft (x) = µ(x) + ∞ X βt,k φk (x) k =1 å Equivalence: βt is a VAR(p) process if ft (p) is an ARH(p). Functional time series with applications in demography 4. Connections, extensions and applications 29 Functional ARH model ARH(p) model ft (x) − µ(x) = θ1 [ft−1 (x) − µ(x)] + · · · + θp [ft−p (x) − µ(x)] + et (x) θj () are linear functions on a Hilbert space et (x) is H white noise FPCA decompostion ft (x) = µ(x) + ∞ X βt,k φk (x) k =1 å Equivalence: βt is a VAR(p) process if ft (p) is an ARH(p). Functional time series with applications in demography 4. Connections, extensions and applications 29 Functional ARH model ARH(p) model ft (x) − µ(x) = θ1 [ft−1 (x) − µ(x)] + · · · + θp [ft−p (x) − µ(x)] + et (x) θj () are linear functions on a Hilbert space et (x) is H white noise FPCA decompostion ft (x) = µ(x) + ∞ X βt,k φk (x) k =1 å Equivalence: βt is a VAR(p) process if ft (p) is an ARH(p). Functional time series with applications in demography 4. Connections, extensions and applications 29 Functional ARH model I have never seen a convincing application of ARH models to real data. But the tools are nearly ready for real work. The FPCA approach allows much more sophisticated time series dynamics with different processes for different coefficients. The FPCA approach is not based on a stochastic model, and does not necessarily give consistent estimates (e.g., when {ft (x)} is not stationary). Functional time series with applications in demography 4. Connections, extensions and applications 30 Functional ARH model I have never seen a convincing application of ARH models to real data. But the tools are nearly ready for real work. The FPCA approach allows much more sophisticated time series dynamics with different processes for different coefficients. The FPCA approach is not based on a stochastic model, and does not necessarily give consistent estimates (e.g., when {ft (x)} is not stationary). Functional time series with applications in demography 4. Connections, extensions and applications 30 Functional ARH model I have never seen a convincing application of ARH models to real data. But the tools are nearly ready for real work. The FPCA approach allows much more sophisticated time series dynamics with different processes for different coefficients. The FPCA approach is not based on a stochastic model, and does not necessarily give consistent estimates (e.g., when {ft (x)} is not stationary). Functional time series with applications in demography 4. Connections, extensions and applications 30 Functional ARH model I have never seen a convincing application of ARH models to real data. But the tools are nearly ready for real work. The FPCA approach allows much more sophisticated time series dynamics with different processes for different coefficients. The FPCA approach is not based on a stochastic model, and does not necessarily give consistent estimates (e.g., when {ft (x)} is not stationary). Functional time series with applications in demography 4. Connections, extensions and applications 30 Outline 1 Yield curves 2 Electricity prices 3 Dynamic updating with partially observed functions 4 Functional ARH models 5 References Functional time series with applications in demography 4. Connections, extensions and applications 31 Selected references Nelson & Siegel (1987) Parsimonious modeling of yield curves. J Business. Diebold & Li (2006) Forecasting the term structure of government bond yields. J Econometrics. Bowsher & Meeks (2008) The dynamics of economic functions: modeling and forecasting the yield curve. JASA. Hays, Shen & Huang (2012) Functional dynamic factor models with application to yield curve forecasting. Ann Appl Stat. Sen & Klüppelberg (2012) Time series of functional data. Tech report, TU Munich. Shang & Hyndman (2011) Nonparametric time series forecasting with dynamic updating. Mathematics and Computers in Simulation. Bosq (2000) Linear processes in function spaces. Springer. Horváth & Kokoszka (2012) Inference for functional data with applications. Springer. Springer Functional time series with applications in demography 4. Connections, extensions and applications 32