Assessing Model Stability Using
Recursive Estimation and Recursive Residuals
Our forecasting procedure cannot be expected to
produce good forecasts if the forecasting model
that we constructed was stable over the sample
period and will remain stable over the forecast
period.
If the model’s parameters are different during
the forecast period than they were during the
sample period, then the model we estimated will
not be very useful , regardless of how well it was
estimated. And, if the model’s parameters were
unstable over the sample period, then model was
not even a good representation of how the series
evolved over the sample period.
The focus of this lecture is on assessing the
stability of the model parameters over the
sample period. Is there evidence that one or
more of the model’s parameters changed at one
or more points in the sample period?
Note: We have already proposed one strategy to
account for changes in the trend parameters that
are believed to have occurred at known points in
time – introduce dummy variables.
Consider the model of Y that combines the trend
and AR(p) components into the following form:
Yt =β0+ β1t + β2t2 +…+βsts +φ1Yt-1+…+φpYt-p+εt
where the ε’s are WN(0,σ2).
Note for emphasis – the issue is whether the β’s and
φ’s are constant over the sample over whether they
are varying over sample. (We assumed the former in
constructing our forecasts.)
We will propose using results from applying the
recursive estimation method to evaluate
parameter stability over the sample period t =
1,…,T.
Fit the model (by OLS) for t = p+1,…,T*:
T* = p+1+s+p = 2p+s+1
T* = 2p+s+2
T* = 2p+s+3
…
T* = T
Recursive Parameter Estimates
The recursive estimation yield parameter
estimates for each T*:
ˆi ,T * and ˆ j ,T *
for i = 1,..,s, j = 1,…,p and T* = 2p+s+1,…,T.
If the model is stable over time then what we
should find is that as T* increases the recursive
parameter estimates should stabilize at some
level. (See for example the upper right panel in
Figure 9.15 in the text.)
A model parameter is unstable if it does not
appear to stabilize as T* increases or if there
appears to be a sharp break in the behavior of
the sequence before and after some T*. (See, for
example the upper right panel in Figure 9.16 in
the text.)
Computing Recursive Coefficients in EViews
and an Application to the HEPI series –
I fit the HEPI series, 1961-2004, to the quadratic
trend model with AR(2) errors:
Yt =β0+ β1t + β2t2 +φ1Yt-1+φ2Yt-2+εt
From the regression output box, select “View”,
“Stability Tests”, “Recursive Estimates”, and
“Recursive Coefficients”. (You will have an
option of which subset of the coefficients you
want to look at.)
The output will be a graph of each of the
recursive coefficient estimates (and a 95-percent
confidence interval, i.e., a two-standard error
band) as T* changes.
0 .8
0 .3
0 .6
0 .2
0 .4
0 .2
0 .1
0 .0
-0 .2
0 .0
-0 .4
-0 .6
-0 .1
75
80
85
90
95
Re c u rs i v e C(1 ) Es ti m a te s
00
75
± 2 S.E.
60
2 .5
40
2 .0
20
1 .5
0
1 .0
-2 0
0 .5
-4 0
80
85
90
95
Re c u rs i v e C(2 ) Es ti m a te s
00
± 2 S.E.
0 .0
75
80
85
90
95
Re c u rs i v e C(3 ) Es ti m a te s
00
± 2 S.E.
75
80
85
90
95
Re c u rs i v e C(4 ) Es ti m a te s
00
± 2 S.E.
1 .0
0 .5
0 .0
-0 .5
-1 .0
-1 .5
-2 .0
-2 .5
75
80
85
90
95
Re c u rs i v e C(5 ) Es ti m a te s
00
± 2 S.E.
C(1) = coefficient on t, β1 ; C(2) = coefficient on t2, β2;
C(3) = intercept, β0;
C(4) = coefficient on hepi(-1), φ1;
C(5)=coefficient on hepi(-2), φ2
Conclusion?
Recursive Residuals and the CUSUM Test
Visual examination of the graphs of the
recursive parameter estimates can be useful in
evaluating the stability of the model. It would be
useful to have a formal statistical test that we
could apply to test the null hypothesis of model
stability. The CUSUM test, which is based on
the residuals from the recursive estimates,
provides such a test.
The idea – We calculate a statistic, called the
CUSUM statistic, for each t. Under the null
hypothesis, the statistic is drawn from a
distribution, called the CUSUM distribution. If,
the calculated CUSUM statistics appear to be
too large to have been drawn from the CUSUM
distribution, we reject the null hypothesis (of
model stability).
More details 1. Let et+1,t denote the one-step-ahead forecast
error associated with forecasting Yt+1 based
on the model fit for over the sample period
ending in period t. These are called the
recursive residuals.
et+1,t = Yt+1 – Yt+1,t
 Yt 1  [ˆ0,t  ˆ1,t (t  1)  ...  ˆ s,t (t  1)s  ˆ1,tYt  ...  ˆ p,tYt  p 1 ]
where the t subscripts on the estimated
parameters refers to the fact that they were
estimated based on a sample whose last
observation was in period t.
2. Let σ1,t denote the standard error of the
one-step ahead forecast of Y formed at time
t, i.e,
σ1,t = sqrt(var(et+1,t))
Define the standardized recursive
residuals, wt+1,t, according to
wt+1,t = et+1,t/σ1,t
Fact: Under our maintained assumptions,
including model homogeneity,
wt+1,t ~ i.i.d. N(0,1).
Note that there will be a set of standardized
recursive residuals for each sample.
3. The CUSUM (cumulative sum) statistics
are defined according to:
t
CUSUM t   wi 1,i
i k
for t = k,k+1,…,T-1, where k = 2p+s+1 is
the minimum sample size for which we can
fit the model.
Under the null hypothesis, the CUSUMt
statistic is drawn from from a CUSUM(t-k)
distribution. The CUSUM(t-k) distribution
is a symmetric distribution centered at 0. Its
dispersion increases as t-k increases.
We reject the null hypothesis at the 5%
significance level if CUSUMt is below the
2.5-percentile or above the 97.5-percentile
of the CUSUM(t-k) distribution.
See Figures 9.15 and 9.16 from the text.
Computing the CUSUM Statistics in EViews
and an Application to the HEPI series –
As above, I fit the HEPI series, 1961-2004, to
the quadratic trend model with AR(2) errors:
Yt =β0+ β1t + β2t2 +φ1Yt-1+φ2Yt-2+εt
From the regression output box, select “View”,
“Stability Tests”, “Recursive Estimates”, and
“CUSUM Test”.
The output will be a graph of the CUSUM
statistics and bands representative the bounds of
the critical region for a test at the five-percent
significance level.
20
10
0
-10
-20
70
75
80
CUSUM
85
90
95
5% Signific anc e
00