Module 11
Transfer Function Models
Class notes for Statistics 451: Applied Time Series
Iowa State University
Copyright 2016 W. Q. Meeker.
May 7, 2016
5h 40min
11 - 1
Module 11
Segment 1
Introduction to Transfer Function Time Series Models
11 - 2
Transfer Function Models
• Single “response” time series and one or more “explanatory” (or “input”) time series.
• Allows the use of explanatory variables to explain variability
in the “response” time series.
• Similar to regression analysis (sometimes called “dynamic
regression analysis”).
• For forecasting, most effective when the explanatory time
series is a “leading indicator” for the response time series
• For multiple response with feedback, needed generalization
is “multiple time series.”
11 - 3
Gas Furnace Input Rate (cu. ft/min)
0
-1
-2
Std. Cu ft/sec
1
2
3
Coded Gas Rate (input)
0
50
100
150
200
250
300
Time
11 - 4
50
55
60
Gas Furnace Percent CO2 in Outlet Gas
0
50
100
150
200
250
300
11 - 5
Gas Furnace Input Rate (cu. ft/min)
Coded Gas Rate (input)
w
-2
-1
0
1
2
3
w= Std. Cu ft/sec
0
50
100
150
200
250
300
time
ACF
0.0
-1.0
2.5
3.0
ACF
0.5
1.0
3.5
Range-Mean Plot
10
20
30
2.0
Lag
0.5
-1.0
0.0
1.0
Partial ACF
1.0
1.5
PACF
0.5
Range
0
-1.0
-0.5
0.0
Mean
0.5
1.0
1.5
0
10
20
30
Lag
11 - 6
Gas Furnace Percent CO2 in Outlet Gas
CO2 (output)
50
w
55
60
w= Percent CO2
0
50
100
150
200
250
300
time
ACF
6
0.0
-1.0
ACF
0.5
8
1.0
Range-Mean Plot
0
10
20
30
Range
Lag
0.5
0.0
-1.0
2
Partial ACF
1.0
4
PACF
48
50
52
54
Mean
56
58
0
10
20
30
Lag
11 - 7
Innovation to Realization Filter
Innovations
Realization
Process or Model
at
ψ(B)
Zt
Model:
Zt = ψ(B)at
= (1 + Bψ1 + B2ψ2 + · · · )at
Zt = at + ψ1at−1 + ψ2at−2 + · · ·
11 - 8
Realization to Residual Filter
Realization
Zt
Residuals
Specified Model
at
π (B)
Model:
1
Zt = π(B)Zt
at =
ψ(B)
= (1 − Bπ1 − B2π2 − · · · )Zt
at = Zt − π1Zt−1 − π2Zt−2 − · · ·
11 - 9
Dynamic Regression Model for Effect of Advertising
on Sales
at
ψ(B)
nt
Advertising
xt
ν(B)
ut
Sales
yt
Model:
yt = ν(B)xt + ψ(B)at
11 - 10
Transfer Function (Dynamic Regression) Model
at
ψ(B)
nt
Input
xt
ν(B)
ut
Output
yt
Model:
yt = u t + n t
= ν(B)xt + ψ(B)at
= (ν0 + ν1B + ν2B2 + · · · )xt + (1 + ψ1B + ψ2B2 + · · · )at
= ν0xt + ν1xt−1 + ν2xt−2 + · · ·
+ψ1at−1 + ψ2at−2 + · · · + at
11 - 11
Notes on the Transfer Function Model
• Input xt can, depending on the application, be either controlled
or stochastic.
• Input xt can, depending on the application, be continuous,
discrete, or binary (as in the intervention models).
• There can be more than one input variable.
11 - 12
Module 11
Segment 2
Univariate Models for the Gas Furnace Input Rate and
Percent CO2 in Outlet Gas
11 - 13
Some Dynamic Regression Models
for Percent CO2 in Outlet Gas
AR(9):
yt = θ0 + φ1yt−1 + φ2yt−2 + · · · + φ9yt−9 + at
Regression on present and past values of input
yt = θ0 + ν0xt + ν1xt−1 + · · · + νk xt−9 + at
Regression on past output as well as present and past values
of input:
yt = θ0 + ν0xt + ν1xt−1 + · · · + ν9xt−9
+φ1yt−1 + φ2yt−2 + · · · + φ9yt−9 + at
11 - 14
Univariate AR(5) Model for Gas Furnace Percent CO2
in Outlet Gas, Part 1
CO2 (output)
Model: Component 1 :: ar: 1 2 3 4 5 on w= Percent CO2
0.5
-0.5
0.0
Residuals
1.0
1.5
Residuals vs. Time
0
50
100
150
200
250
300
Time
Residual ACF
1.0
Residuals vs. Fitted Values
1.5
•
•
0.0
•
•
•
•
45
50
0.5
••
•
•
55
Fitted Values
•
•
•
•
0.0
•
ACF
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
••
• •
•
•
• •
• • • •
•
• •
•• •
••
•
•
•
• •
•
•
•• • • • • • • • ••
•
••
•
•
•
•
•
•
• •
•
•
•
•
• • • •
•
••
• • •• •• • •• • • ••••••• • • •
•
•
•
•
•
•
• • •
•
•
• • •
• • • •••
•• • • • • • • •
•
•
•
•
•
•
•
•
•
• •
•
•
•• • • •
•
•• ••
•
•
•• • • •
•
• •
••
• • ••
•
•••
• •• ••• •
•••
• • •• • •• •• • •
•
•• • •
••
•
• •
• • •••
•
• ••
•
• • • • •
•
•
•
•
•
• •
• •
•
•
• •
•
•
•
•
•
••
•
•
•
•
•
•
•
•• •
•
•
•
•
-0.5
0.5
•
-0.5
Residuals
•
•
••
•
•
•
-1.0
1.0
•
60
0
10
20
30
Lag
11 - 15
Univariate AR(5) Model for Gas Furnace Percent CO2
in Outlet Gas, Part 2
CO2 (output)
Model: Component 1 :: ar: 1 2 3 4 5 on w= Percent CO2
••
••
• •
• •
• •
•
••
••
•
••
•• •
•
• •
•
• ••
• •
•
•
•• • •
•
•
•••
•
•
•
•
• ••••
•
•
•
••
•
• •
•••
•
•
•
•
•
•
•
•
••
•
• • •
••
• •
•
• •
•
•
••
••
••
•
•••• • •
••
• •
••
••
•
• •
•
•
••••
•
• •
•••
•
•
•
•
••••••
•
•
•
•
•
•
•
•
•
•
•
•••••
•
•
••
•
•
•
•
•
•
•
••••
• •
•
•••
•
•••••• •
•
•
•
••
•
•
•
• ••
•
•
•
•
••
•
•
•
•
•
•
•
•
•• ••
••
• •
•••
•
•
•
•
•••
•
•
•
•
•
•••
•
•••
• •
•
•
••
•••
••
•
••
•
•
• •
•
• •
••
• •
••
•
• •
•
• •
••
•• •••••
•
•
••
• •
• •
•
••
•
•
••• •
••
• •
•
•
•
•
•••
• •
•
••
•• •
• • •
••
• • ••
•••
100
150
200
250
300
Index
3
Normal Probability Plot
0
Normal Scores
1
2
•
-1
50
-2
0
-3
55
50
45
Percent CO2
60
65
Actual Values * * * Fitted Values * * * Future Values
* * 95 % Prediction Interval * *
•
•
•
•
••
••
•••
••• •
•
•
•
••••
•••
•••
••••
•••••
•
•••••
•••
•••
•••
•
•
•
•
•••
•••
••••
•••
•
•
••
••••
•••
•••
•••
•
•
•
•
•••
•••
•••
••
•
•
-0.5
0.0
0.5
1.0
•
1.5
Residuals
11 - 16
Univariate AR(4) Model for Gas Furnace Input Rate,
Part 1
Coded Gas Rate (input)
Model: Component 1 :: ar: 1 2 3 4 on w= Std. Cu ft/sec
0.0
-1.0
-0.5
Residuals
0.5
1.0
Residuals vs. Time
0
50
100
150
200
250
300
Time
Residual ACF
1.0
1.0
Residuals vs. Fitted Values
•
0.5
0.5
•
•
•
•
•• •
• ••
•
0.0
•
ACF
0.0
-0.5
•
••
••
•
•
•
-0.5
•
••
• •
•• •
• •
•
•
••
••
•
• •
• •
•••• • ••• •••
• ••• •••••
•• • •• • ••
••
•
•
•
•
•• •
•
•
•
•
•
• • • ••• •• •
••
•
•
•• • ••••••• •••••••••• ••••• • ••••• • • ••
• ••• •••• • •••• ••
•
• •
• •• • • • • •
•
• •
• • ••••• • •• •• • •• ••• • ••• ••• ••• • •• •
•• • ••••
•• •
•
• • • •• • • •• • •• • ••••••• •• •• • ••• • •• •
• • ••
•
•
••
• • • • ••
•
•
•
•
•
•
•
•
• • •• • •
•
•
•
•
• ••
• •
•
•
•
•
•
-1.0
•
•
-1.0
Residuals
•
-2
0
Fitted Values
2
0
10
20
30
Lag
11 - 17
Univariate AR(4) Model for Gas Furnace Input Rate,
Part 2
Coded Gas Rate (input)
Model: Component 1 :: ar: 1 2 3 4 on w= Std. Cu ft/sec
4
Actual Values * * * Fitted Values * * * Future Values
* * 95 % Prediction Interval * *
••
••••• ••
•
• •
•
•••
•
•
•• •
•••
•
•
•
•••
• • • ••
• • • •
••
•••
-2
0
• •••
• •
••
•••
• ••• ••
•• • •
•
••••
•••
•
•
• • •••••
••••
• •
••••
••••
•••
• •
•
•
•
•
•
•••••
••
• •
•
•
• •
•
•
• •
•
••
•••
•
• •
•
•••
•
•••
••
•••
•
•
•
••
•
•••
••••• •
• •
•
•• •
•••••
•••
•
• •
• •
••••
•
•
•
•••• •
• • •
•
• •
•
••
•
••
•
••
•
••••••
•
••••
•
•••
••
•••
•• • •• ••• ••••••••••••••••••••••
•••
••
• •
•
•
• •
•••
•••
-4
100
150
200
250
300
Index
3
Normal Probability Plot
0
Normal Scores
1
2
•
-1
50
-2
0
•
-3
Std. Cu ft/sec
2
•••
•
•
• •••••
•
•
•
•
•
• •
••••
•
••••
• •
•• •
••
•
••
•
•
• •
•
•
•••
•
•
••
•
• •
••
•••
••
•
•••
•••
•••
•••
•
•
•
•
•••
••
•••
•••
•
•
•
•••
•••
•••
•••
••••
•••
•••••
••
•••
•
•••
••
•••
•
•
•
-1.0
-0.5
0.0
0.5
1.0
Residuals
11 - 18
Module 11
Segment 3
The Cross-Correlation Function and an Introduction
to Transfer Function Time Series Model Identification
11 - 19
Cross Correlation Function
The cross covariance function
γxy (k) = E[(xt − µx)(yt+k − µy )]
describes, for k > 0, the relationship between future values
of y and current values of x.
The cross correlation function
γxy (k)
ρxy (k) =
σxσy
gives the correlation between xt and yt+k (or, equivalently,
between xt−k and yt).
To see how present values of x might be related to past
values of y, use
ρxy (−k) = ρyx(k)
11 - 20
Data for Computing
Sample Cross Correlations Between xt and yt+k
t
1
2
3
4
5
6
7
8
9
..
294
295
296
xt
-0.109
0.000
0.178
0.339
0.373
0.441
0.461
0.348
0.127
..
0.017
-0.182
-0.262
yt
53.8
53.6
53.5
53.5
53.4
53.1
52.7
52.4
52.2
..
57.8
57.3
57.0
yt+1
53.6
53.5
53.5
53.4
53.1
52.7
52.4
52.2
52.0
..
57.3
57.0
NA
yt+2
53.5
53.5
53.4
53.1
52.7
52.4
52.2
52.0
52.0
..
57.0
NA
NA
yt+3
53.5
53.4
53.1
52.7
52.4
52.2
52.0
52.0
52.4
..
NA
NA
NA
yt+4
53.4
53.1
52.7
52.4
52.2
52.0
52.0
52.4
53.0
..
NA
NA
NA
yt+5
53.1
52.7
52.4
52.2
52.0
52.0
52.4
53.0
54.0
..
NA
NA
NA
11 - 21
Sample Cross Correlation Function (CCF)
The sample cross correlation function
ρbxy (k) =
γbxy (k)
Sx Sy
gives the sample correlation between xt and yt+k (or, equivalently, between xt−k and yt).
To see how xt is correlated with past values yt, use
ρbxy (−k) = ρbyx(k)
11 - 22
Cross Correlation Function Between Gas Furnace
Input Rate and Percent CO2 in Outlet Gas
Cross Correlation
0.0
-0.5
-1.0
CCF
0.5
1.0
Between gasrx.d and gasry.d with model null.model
-20
-10
0
10
20
Lag
11 - 23
Notes on Cross Correlation Function
• Both xt and yt must be stationary for cross correlations to
be defined.
• To judge whether a sample CCF value is significantly different from 0, use
t=
where
Sρbxy (k) =
ρbxy (k)
Sρbxy (k)
s
1
≈
n−k
s
1
n
Compare with standard normal quantiles (signal if outside
±2).
• As with ACF and PACF, commonly used tests and standard
errors for the CCF are only approximate.
• The sample CCF is difficult to interpret unless either xt or
yt is white noise.
11 - 24
Cross Correlation Function Between
Residuals of Gas Furnace Input Rate and
Residuals of Percent CO2 in Outlet Gas
Cross Correlation
0.0
-0.5
-1.0
CCF
0.5
1.0
Between gasrx.d and gasry.d with model gasrx.model and gasry.model
-20
-10
0
10
20
Lag
11 - 25
Interpreting the Cross Covariance Function
The transfer function model can be written as
yt+k = ν0xt+k + ν1xt+k−1 + ν2xt+k−2 + · · · + νk xt + nt+k
For illustration, assume that µy = 0 and µx = 0. Multiplying
the model by xt,taking expectations, and assuming xt and nt
are independent gives the lag-k cross covariance function as
γxy (k) = E(yt+k xt)
= ν0E(xt+k xt) + ν1E(xt+k−1xt) + ν2E(xt+k−2xt)
+ · · · + νk E(xtxt) + E(nt+k xt)
= ν0γk + ν1γk−1 + ν2γk−2 + · · · + νk γ0
where the γk values here are the autocovariances of the xt
time series. If xt is white noise, then γk = 0 for k 6= 0, and
the cross covariance function at lag k simplifies to
γxy (k) = E(yt+k xt) = νk γ0 = νk σx2
11 - 26
Module 11
Segment 4
Prewhitening for Transfer Function Time Series Model
Identification
11 - 27
Prewhitening Transfer Function Inputs
yt = ν(B)xt + nt
If αt is a “white-noise” process, the model for xt is
xt = ψx(B)αt =
θx(B)
αt
φx(B)
or
αt = πx(B)xt =
φx(B)
xt
θx(B)
Filter yt (i.e., find the “residuals” of yt using the xt model)
βt = πx(B)yt =
φx(B)
yt
θx(B)
Because πx(B) is the “wrong” model for yt, we do not expect
the βt series to be white noise. Multiplying the original model
by πx(B) gives
πx(B)yt = πx(B)ν(B)xt + πx(B)nt
βt = ν(B)αt + πx(B)nt
which is a white-noise input process, allowing easy identification of ν(B).
11 - 28
Use of a Prewhitening Filter to Identify ν(B)
nt
xt
ν(B)
ut
yt
π x (B) n t
at
π x (B) x t
αt
π x (B) y t
ν(B)
βt
yt = ν(B)xt + nt
πx(B)yt = πx(B)ν(B)xt + πx(B)nt
βt = ν(B)αt + πx(B)nt
11 - 29
Cross Correlation Function Between Prewhitened Gas
Furnace Input Rate and Percent CO2 in Outlet Gas
Cross Correlation
0.0
-0.5
-1.0
CCF
0.5
1.0
Between gasrx.d and gasry.d with model gasrx.model
-20
-10
0
10
20
Lag
11 - 30
Module 11
Segment 5
Transfer Function Model for the Gas Furnace Percent
CO2 and Forecasting with a Transfer Function Model
11 - 31
Cross Correlation Function Between Prewhitened Gas
Furnace Input Rate and Percent CO2 in Outlet Gas
Cross Correlation
0.0
-0.5
-1.0
CCF
0.5
1.0
Between gasrx.d and gasry.d with model gasrx.model
-20
-10
0
10
20
Lag
11 - 32
Transfer Function Model for Gas Furnace Percent
CO2 in Outlet Gas
yt = ν(B)xt + nt
= B2ν5(B)xt +
1
at
φ5(B)
φ5(B)yt = φ5(B)B2 ν5(B)xt + at
(1 − φ1B + · · · + φ5B5)yt = (1 − φ1B + · · · + φ5B5)B2×
(ν1B + · · · + ν5B5)xt + at
yt = φ1yt−1 + · · · + φ5yt−5 + (ν1B3 − ν1φ1B4 + . . .
ν5B7 − ν5φ5B12)xt + at
yt = φ1yt−1 + · · · + φ5yt−5 + ν1xt−3 − ν1φ1xt−4 + . . .
ν5xt−7 − ν5φ5xt−12 + at
This model has 10 parameters (plus σa).
11 - 33
Transfer Function Model for Gas Furnace Percent
CO2 in Outlet Gas, Part 1
CO2 (output)
Model: Component 1 :: ar: 1 2 3 on w= Percent CO2 regr variables= cbind
0.5
-0.5
0.0
Residuals
1.0
1.5
Residuals vs. Time
0
50
100
150
200
250
300
Time
Residual ACF
1.0
1.5
Residuals vs. Fitted Values
1.0
•
0.5
•
••
•
••
• • ••
•
45
50
•
•
•
•
•
•
•
•
55
Fitted Values
• •
ACF
•
•
0.0
• •
•
•
•
•
•
•
••
• •
• • •
• •• •• • • • •
• •
• • •• •
• ••
•
• ••
• •• • • • •
•
•
•
• •• •• ••• • • • • •
•
•
•
•
•
••
••• • •
•
• ••
•••
•
•
••
• • •• • •
••
• • •••• ••
•
•• • ••••• • • • • • • • •• • • • • •• •• • •• • •• •• • •• • • •• • ••• • • • ••
•• • • •• • ••• •
•
••• • • • • •
• • •• • • • •••• • • •• •••
•
•
•
•
•
•
•
• •
•
• • • • • • • •• • •
• •
• • •
•
•
•
•
•
•• •
• •
•
•
•
•
•
•
•
•
•
•
•
•
•
• ••
••
-0.5
0.0
•
•
•
-1.0
0.5
•
-0.5
Residuals
•
•
60
0
10
20
30
Lag
11 - 34
Transfer Function Model for Gas Furnace Percent
CO2 in Outlet Gas, Part 2
CO2 (output)
Model: Component 1 :: ar: 1 2 3 on w= Percent CO2 regr variables= cbind
••
••
• •
• •
• •
•
••
••
•
••
•• •
•
• •
•
• ••
• •
•
•
•• • •
•
•
•••
•
•
•
• •••
•
•
•
•
••
•
• •
•
•
•
•
•
•
•
•
••
••
•
• • •
••
• •
•
• •
•
•
••
••
•
•
•••• • •
• •
••
••
•
• •
•
•
•
•
• •
•••
•
•
•
•
•
•
•
•
•
•
•
•
•
•
••
••
•••
•
•
•
•
••••
•
••••
• •
•
•••
••••••••••••••••
•••••• •
•
•
•
••
•
•
•
• ••
•
•
•
•
••
•
•
•
•
•
•
•
•
•• ••
••
• •
•••
•
•
•
•
•••
•
•
•
•
•
•••
•
•
•••
•
•
•
•
••
•
••
•
••
•
•
•
• •
•
• •
••
• •
••
•
• •
•
• •
••
•• •••••
•
•
••
• •
• •
•
••
•
••• ••
••
• •
•
•
•
•
•••
• •
•
••
•• •
• • •
••
• • ••
•••
100
150
200
250
300
Index
3
Normal Probability Plot
0
Normal Scores
1
2
•
-1
50
-2
0
•
-3
55
50
45
Percent CO2
60
65
Actual Values * * * Fitted Values * * * Future Values
* * 95 % Prediction Interval * *
•
•
••
••
•••
•• •
•
•
•
•
•••
••
•••
••
••
•
•
•••
•••
•••
•••
•••
•
•••
••
•••
•••
•••
•••
•••
••
••••
•
•
•••
•••
••
•••
••
•
•
•
-0.5
0.0
0.5
1.0
1.5
Residuals
11 - 35
Univariate AR(5) Model for Gas Furnace Percent CO2
in Outlet Gas, Part 2
CO2 (output)
Model: Component 1 :: ar: 1 2 3 4 5 on w= Percent CO2
••
••
• •
• •
• •
•
••
••
•
••
•• •
•
• •
•
• ••
• •
•
•
•• • •
•
•
•••
•
•
•
•
• ••••
•
•
•
••
•
• •
•••
•
•
•
•
•
•
•
•
••
•
• • •
••
• •
•
• •
•
•
••
••
••
•
•••• • •
••
• •
••
••
•
• •
•
•
••••
•
• •
•••
•
•
•
•
••••••
•
•
•
•
•
•
•
•
•
•
•
•••••
•
•
••
•
•
•
•
•
•
•
••••
• •
•
•••
•
•••••• •
•
•
•
••
•
•
•
• ••
•
•
•
•
••
•
•
•
•
•
•
•
•
•• ••
••
• •
•••
•
•
•
•
•••
•
•
•
•
•
•••
•
•••
• •
•
•
••
•••
••
•
••
•
•
• •
•
• •
••
• •
••
•
• •
•
• •
••
•• •••••
•
•
••
• •
• •
•
••
•
•
••• •
••
• •
•
•
•
•
•••
• •
•
••
•• •
• • •
••
• • ••
•••
100
150
200
250
300
Index
3
Normal Probability Plot
0
Normal Scores
1
2
•
-1
50
-2
0
-3
55
50
45
Percent CO2
60
65
Actual Values * * * Fitted Values * * * Future Values
* * 95 % Prediction Interval * *
•
•
•
•
••
••
•••
••• •
•
•
•
••••
•••
•••
••••
•••••
•
•••••
•••
•••
•••
•
•
•
•
•••
•••
••••
•••
•
•
••
••••
•••
•••
•••
•
•
•
•
•••
•••
•••
••
•
•
-0.5
0.0
0.5
1.0
•
1.5
Residuals
11 - 36
Forecasting Transfer Function Output with Fixed
(Known) Input xt (so that ut is deterministic)
at
ψ(B)
nt
Input
xt
ν(B)
ut
Output
yt
• Compute ut from known xt and ν(B) from ut = ν(B)xt .
• Use the univariate method to find a prediction interval for
nt .
• Because yt = ut + nt, forecasts and intervals for yt can be
obtained by adding ut to the forecasts and intervals for nt.
11 - 37
Forecasting Transfer Function Output with Stochastic
Input xt (so that ut is stochastic)
αt
at
ut
U (B)
ψ(B)
nt
αt
ψx (B)
U (B) =
xt
ν(B)
Output
ut
yt
ψx (B) ν(B)
yt = u t + n t
= u(B)αt + ψ(B)at
2
Var[en (l)] = σα
l−1
X
j=1
2
u2
j + σa
l−1
X
ψj2
j=1
11 - 38
Module 11
Segment 6
Strategy for Transfer Function Modeling and the
Impulse Response Function
11 - 39
Strategy for Identifying a Transfer Function Model
• Model yt alone as a base-line for comparison and as a preliminary noise model
• Model xt alone
• Use the xt model to “filter” both xt and yt; examine the
CCF
• Identify tentative dynamic regression model from CCF
• Fit and check the tentative dynamic regression model
• For a multiple input transfer function model, add new variables by using new xt to help explain the residuals from the
current model.
11 - 40
Transfer Function (Dynamic Regression) Model
Non-Parsimonious Form
at
ψ(B)
nt
Input
xt
ν(B)
ut
Output
yt
Model:
yt = u t + n t
= ν(B)xt + ψ(B)at
= (ν0 + ν1B + ν2B2 + · · · )xt + (1 + ψ1B + ψ2B2 + · · · )at
= ν0xt + ν1xt−1 + ν2xt−2 + · · ·
+ψ1at−1 + ψ2at−2 + · · · + at
11 - 41
Parsimonious Transfer Function Models
at
θ q (B)
φ p (B)
υ (B)
Input
xt
ωs (B)
nt
ut
δ r (B)
ψ (B)
Output
yt
yt = ν(B)xt + nt
b ωs (B)
θq (B)
= B
xt +
at
δr (B)
φp(B)
where d is the delay, the polynomial sizes s, r, q, and p are
small (0, 1 or 2) and s + r + q + p is typically between 3 and
6.
11 - 42
The Impulse Response Function
• The ψ weights for an ARIMA model gives an impulse-response function
(IRF) and can be used to
◮ Check for stationarity (ψ weights must die down rapidly)
◮ Derive and compute forecast error variances
◮ Compare ARIMA models with different forms (ARIMA
models with different forms but similar IRFs are similar)
◮ Characterize ARIMA models
• The ν weights for the
ν(B) = ν0 + ν1B + ν2B2 + . . .
filter gives an impulse-response function. Often we can find
a more parsimonius filter of form ωs(B)/δr (B) that has an
IRF similar to that of ν(B).
11 - 43
One-Term Impulse Response Function,
Three Period Delay
0
5
10
lag
15
20
s = 0, r = 0, b = 3
ωs(B)
ω0(B)
b
3
ut = B
xt = B
xt
δr (B)
δ0(B)
= B3ω0 xt = ω0 xt−3
= 1.0 xt−3
11 - 44
Two-Term Impulse Response Function,
Three Period Delay
0
5
10
lag
15
20
s = 1, r = 0, b = 3
ωs(B)
ω1(B)
b
3
ut = B
xt = B
xt
δr (B)
δ0(B)
= B3(ω0 − ω1B) xt = ω0 xt−3 − ω1xt−4
= 1.0 xt−3 + 1.3xt−4
11 - 45
Three-Term Impulse Response Function,
Three Period Delay
0
5
10
lag
15
20
s = 2, r = 0, b = 3
ωs(B)
ω2(B)
b
3
ut = B
xt = B
xt
δr (B)
δ0(B)
= B3(ω0 − ω1B − ω2B2) xt = ω0 xt−3 − ω1xt−4 − ω2xt−5
= 1.0 xt−3 + 1.3xt−4 + 1.65xt−5
11 - 46
Ten-Term Impulse Response Function,
Three Period Delay
0
5
10
lag
15
20
s = 9, r = 0, b = 3
ut = B3(ω0 xt − ω1 xt−1 − ω2 xt−2 − ω3 xt−3 − ω4 xt−4
− ω5 xt−5 − ω6 xt−6 − ω7 xt−7 − ω8 xt−8 − ω9 xt−9)
= 0.50 xt−3 + 1.05 xt−4 + 1.52 xt−5 + 0.76 xt−6 + 0.38 xt−7
+ 0.19 xt−8 + 0.095 xt−9 + 0.048 xt−10 + 0.024 xt−11 + 0.012 xt−12
11 - 47
Parsimonious Impulse Response Function,
Three Period Delay
0
5
10
lag
15
20
s = 2, r = 1, b = 3
ω2(B)
ωs(B)
b
3
ut = B
xt = B
xt
δr (B)
δ1(B)
0.50 + 0.8B + 1.0B2
ω0 − ω1 B − ω2 B2
3
xt =
xt−3
=B
1 − δ1 B
1 − 0.50B
11 - 48
Parsimonious Damped Sin Wave Impulse Response
Function
0
5
10
lag
15
20
s = 0, r = 2, b = 0
ω0(B)
xt =
xt
ut = B
δr (B)
δ2(B)
b ωs (B)
=
ω0
1
x
=
xt
t
2
2
1 − δ1 B − δ2 B
1 − 1.0 B + 0.55 B
11 - 49
Module 11
Segment 7
Transfer Function Model for the Gas Furnace Percent
CO2 Example and Software for Parsimonious Transfer
Function Models
11 - 50
Cross Correlation Function Between Prewhitened Gas
Furnace Input Rate and Percent CO2 in Outlet Gas
Cross Correlation
0.0
-0.5
-1.0
CCF
0.5
1.0
Between gasrx.d and gasry.d with model gasrx.model
-20
-10
0
10
20
Lag
11 - 51
Parsimonious Transfer Function Model for Gas
Furnace Percent CO2 in Outlet Gas
s = 2, r = 1, p = 2, q = 0, b = 3
yt = θ0 + Bb
ωs(B)
θq (B)
xt +
at
δr (B)
φp(B)
ω2(B)
1
3
= θ0 + B
xt +
at
δ1(B)
φ2(B)
1
(ω0 − ω1B − ω2B2)
3
xt +
at
= θ0 + B
2
1 − δ1 B
1 − φ1 B − φ2 B
1
ω0 − ω1B − ω2B2
= θ0 +
xt−3 +
at
2
1 − δ1 B
1 − φ1 B − φ2 B
This model has 7 parameters (plus σa).
11 - 52
Gas Furnace Percent CO2 in Outlet Gas
JMP Transfer Function Model Parameter Estimates
Parameter Estimates
Variable
Term
Factor Lag
Input Gas RateNum0,0
0
0
Input Gas RateNum1,1
1
1
Input Gas RateNum1,2
1
2
Input Gas RateDen1,1
1
1
Output CO2 AR1,1
1
1
Output CO2 AR1,2
1
2
Intercept
0
0
Estimate Std Error t Ratio Prob>|t|
-0.53110 0.0737809 -7.20 <.0001*
0.37992 0.1016023
3.74 0.0002*
0.51798 0.1085541
4.77 <.0001*
0.54907 0.0391928 14.01 <.0001*
1.52713 0.0467279 32.68 <.0001*
-0.62883 0.0494715 -12.71 <.0001*
53.36276 0.1374725 388.17 <.0001*
11 - 53
Parsimonious Transfer Function Model for Gas
Furnace Percent CO2 in Outlet Gas
0
5
10
lag
15
20
s = 2, r = 1, p = 2, q = 0, b = 2
b ωs (B)
θq (B)
yt = θ0 + B
xt +
at
δr (B)
φp(B)
1
−0.53 − 0.38B − 0.52B2
xt−3 +
at
= 53.4 +
2
1 − 0.55B
1 − 1.53B + 0.63B
11 - 54
Gas Furnace Percent CO2 in Outlet Gas
JMP Transfer Function Model Residuals
Residuals
Residual Value
1.5
1.0
0.5
0.0
-0.5
-1.0
0
59
118
177
236
295
Row
Lag AutoCorr -.8-.6-.4-.2 0 .2 .4 .6 .8 Ljung-Box Q p-Value
0
1.0000
.
.
1
0.0304
0.2763 0.5992
2
0.0572
1.2595 0.5327
3 -0.0697
2.7202 0.4368
4 -0.0530
3.5701 0.4673
5 -0.0563
4.5302 0.4759
6
0.1183
8.7877 0.1859
7
0.0286
9.0376 0.2500
11 - 55
Parsimonious Transfer Function Model for Gas
Furnace Percent CO2 in Outlet Gas
Unscrambled Model
ω0B3 − ω1B4 − ω2B5
1
yt =
xt +
at
2
1 − δ1 B
1 − φ1 B − φ2 B
(1 − δ1B)(1 − φ1B − φ2B2)yt = (1 − φ1B − φ2B2)×
(ω0B3 − ω1B4 − ω2B5) xt+
(1 − δ1B) at
The unscrambled form of the model has the form
yt = f (current and past xt, past yt, past at )
with a finite number of lags. This form is useful for computing forecasts.
11 - 56
Software for Fitting
Parsimonious Transfer Function Models
• R package TSA (in CRAN)
• SCA (www.scausa.com)
• SAS Econometrics and Time Series (ETS) (www.sas.com)
• STATA (www.stata.com)
All of these packages also support univariate ARIMA models.
11 - 57