Model Evaluation in a Forecast-dependent
Policy Context
Comments on the Oslo Conference
David F. Hendry∗
Economics Department, Oxford University
May 11, 2005
∗
I have drawn freely on several of my working and published papers.
2
1 Introduction
These notes address some of the main themes of the
papers and discussions, and seek to clarify a little of
what is known about some of the issues.
Section 2 considers the gap between macro-economic
theory models, and empirical econometric findings;
§3 notes the prevalence of non-stationarity in
economic time series;
§4 discusses its impact on the role of ceteris paribus;
§5 on models of expectations; and
§6 draws some implications for economic forecasting.
Sections 7, 8 and 9 briefly consider impulse response
analyses; DSGEs; and the roles of a suite of models.
3
2 Macroeconomics and macro-econometrics
Gap persists between macro-economic theory models,
and empirical econometric findings: see Pagan (2003).
Six main reasons.
(a) Much data variability is due to factors
that are absent from economic theories.
In economic data, non-stationarity rules;
forecast failure is quite common.
Stationarity requires all moments and/or joint distributions
to be time invariant.
Two counter-examples: unit roots and structural breaks.
§3 addresses empirically;
here we note some major UK events.
4
Institutional knowledge and economic history essential:
World War I; 1920–21 flu’ epidemic; 1926 general strike;
great depression; World War II; 1970s oil crises.
Huge legislative, social, and technological changes.
Also major financial innovations:
telegraph; cheques; credit cards; ATMs;
deregulation of banking sector and building societies.
Recent shifts include:
savings behaviour from changes in demography,
legislation on pensions, health-care provision in old age;
consumption decisions from changes in credit rationing.
Many policy regime shifts: end of gold standard,
breakdown of Bretton Woods, inflation targeting, etc.
5
(b) Economic theory evolves:
yet much empirical work imposes existing theory.
Rapidly invalidated by advances in understanding.
Unless theory is complete, correct, and immutable,
best viewed as a cognitive guide.
Major advances in many theory areas will continue,
so evidence is ‘theory dependent’:
a drawback not an advantage.
Ahistorical theory confronts non-stationary evidence.
6
(c) All economic theories incomplete:
rely on implicit ceteris paribus.
Useful for theoretical reasoning:
not an acceptable basis for empirical modelling.
Cet. par. invalid even for strongly exogenous variables,
when potential effects are intrinsically non-constant.
Need ‘minor influence’ theorems instead:
theory for why omitted factors can be neglected.
Or perhaps, even evidence....
Alternative: embed theory model in general system:
expect models with many, rather than few, factors.
7
(d) Huge range of ‘macro-economic theories’ based on:
constrained optimization;
heterogeneity of endowments;
incomplete information;
imperfect-knowledge expectations;
dynamic stochastic general equilibrium (DSGE) models;
‘new macroeconomics’ based in asymmetric information.
No uniquely accepted paradigm:
so must test between tightly constrained theories.
8
(e) No clear theoretical guidance on:
forms of utility functions;
all relevant and irrelevant variables;
functional forms of relationships;
exogeneity status;
dynamic reactions;
aggregation over heterogeneous agents; and
sources of non-stationarities, especially breaks.
At best one transform can deliver a constant relation.
9
(f) Gaps common across research arenas,
but differ in form.
Micro-economics–low R2s: individual heterogeneity.
Most variability not accounted for by theory models.
Doubt that all major factors have been included.
Panels–most variance from persistent ‘individual effects’,
removed by differencing or deviations.
That micro variability is due to heterogeneity, conflicts
with ‘representative’ agent theories for macro behaviour.
None of above suggests ‘single-agent’ theory dominant.
Deconstruct Pagan trade-off.
10
Theory
consistency
Perfection
1.0
0.8
DSGE?
preferences?
0.6
0.4
← non−
rejection
cut off
← Conflicting theories
0.2
VAR?
0.0
0.2
Figure 1
0.4
0.6
0.8
Empirical
coherency
1.0
Revised Pagan trade-off diagram.
11
Four difficulties undermine Pagan’s claimed trade-off:
(1) Unique, universally-agreed theory needed for theory
‘consistency’ to be meaningful, or even to draw the axes:
else trade-off measure needed for every theory class.
(2) Theories evolve, delivering new insights,
requiring re-definition of ‘consistency’ each time.
(3) Even fixed theories do not imply unique models:
it matters how theory models are implemented empirically
(marriage or data rape?) & how expectations are treated.
(4) Empirical coherence not unique either: alternative data
measures exist; revisions occur; selection criteria differ.
12
Figure suggests non-evidential preferences could legitimize
falsified theory. In danger of ceasing to be scientific.
Pagan accords DSGEs the highest theory ‘consistency’:
members of equilibrium-correction class, so
shifts in equilibrium means induce forecast failure.
May reject theory model because shifts not modelled.
Theory and evidence should be symbiotic:
conflict should be of major concern.
Solution—conduct research in a progressive framework:
successively encompassing, congruent empirical models,
describing reliable and well-measured data,
suggested and consolidated by relevant theories.
13
3 Non-stationarities
Non-stationary if moments or distribution change.
(1) Unit roots. Induce stochastic trends.
Many theories deliver unit roots:
arbitrage pricing, efficient markets; Euler equations.
Easily removed by differencing–if that is only issue.
Equilibrium theories predict cointegration:
reduced-rank condition on agents’ decision functions.
Often a property of constrained optimization theories.
But data transforms crucial:
cointegration not invariant under many non-linear
transformations.
14
2
∆P
∆M
300
200
1
100
0
1850
10
1900
1950
2000
∆Y
0
1850
0.04
5
0.02
0
0.00
1850
1900
1950
2000
1850
1900
1950
2000
1900
1950
2000
∆R L
Figure 2 Changes in prices, money, GDP and RL.
15
Figure 2 plots changes in:
prices P , money M , GDP Y , and long-term interest rates RL.
Changes in all 4 series highly non-stationary:
yet many Euler-based equations in differences of levels.
Obvious how non-stationary levels must be.
Log changes denoted by ∆p, ∆m, ∆y, ∆rL:
their means and variances changed across epochs (fig.3).
16
0.2
∆p
0.2
∆m
25−year means
0.1
0.1
0.0
0.0
−0.1
1850
0.10
1900
1950
2000
1850
∆y
1900
1950
2000
1950
2000
∆r L
0.2
0.05
0.00
0.0
−0.05
−0.10
1850
−0.2
1900
1950
2000
Figure 3
1900
∆p, ∆m, ∆y , ∆rL.
17
(2) Structural breaks.
Many varieties, but location shifts are key:
shifts in non-zero mean components.
Even growth rates are non-stationary (fig.4);
as are unemployment (fig.5) and hours (fig.6),
where both means and variances change.
Location shifts can be attenuated by differencing:
transforms to less pernicious ‘blips’.
Potential to remove by co-breaking:
but requires no breaks in those relations.
18
20−year mean real income growth per capita
20−year mean inflation
0.10
0.02
0.05
0.01
0.00
0.00
1850
0.03
1900
1950
2000
20−year mean real consumption growth
1850
1900
1950
2000
20−year mean population growth
0.0100
0.02
0.0075
0.01
0.0050
0.00
1850
1900
1950
2000
1850
1900
1950
2000
Figure 4 20-year mean UK growth rates, 1850–2000.
19
0.100
unemployment rate
mean
0.15
upward mean shift →
0.075
0.10
0.050
0.05
0.025
1880
1890
1900
1910
1920
0.15
0.15
0.10
downward
0.10
mean shift
and
lower variance
0.05
↓
0.05
1880
1900
1920
1940
1960
Figure 5
1980
1880
1900
1920
1940
upward mean shift
and higher variance
↓
1880 1900 1920 1940 1960 1980 2000
UK unemployment shifts.
20
65
Average hours worked per week
60
55
50
45
40
1880
Figure 6
1900
1920
1940
1960
1980
2000
UK average hours per week data: stationary?.
21
4 Ceteris paribus in a non-stationary world
In theory, cet. par. impounds strongly exogenous forces:
but unacceptable empirically under non-stationarity.
‘If nothing interferes .... a regularity is observed’.
In non-stationary processes something will interfere.
Instead, need ‘minor influence’ results:
why omitted factors are negligible.
Suggests embedding theory model in general system:
anticipate models with many, rather than few, factors.
Can impose theory restrictions on that system.
22
5 Models of expectations
Theories of expectations face realities of forecasting.
‘Rational’ expectations (RE):
conditional expectation given available information (It):
re = E [y
yt+1
t+1 | It] .
Avoids arbitrage opportunities or unnecessary losses.
Expectations are instrumental—
agents should equate marginal costs with benefits:
leads to ‘economically rationally expectations’ (ERE).
RE assumes free information, computing power, and
discovery of form of E [yt+1|It].
Even use of mean assumes symmetry.
(1)
23
Model-consistent expectations:
further imposes invalid model on expectations.
ERE still assumes agents can do relevant calculations.
(1) should be written as:
e =E
yt+1
t+1 [yt+1 | It] =
Z
yt+1ft+1 (yt+1 | It) dyt+1.
(2)
e even unbiased for y
Only then is yt+1
t+1.
But (2) requires crystal balls for future ft+1(yt+1|It).
se ,
Best an agent can do is form ‘sensible expectation’, yt+1
forecasting ft+1(·) by fbt+1(·):
se =
yt+1
Z
yt+1fbt+1 (yt+1 | It) dyt+1.
If {yt} not time-invariant, no good rules for fbt+1(yt+1|It).
(3)
24
If moments of ft+1(yt+1|It) alter,
fbt+1(·) = ft(·) not a good choice, yet underlies RE.
Agents cannot know how It enters ft(·)
under non-time-invariance.
Ask them: if expectations matter collect data.
Known cases poor:
based on UK index bonds, out for almost 4 years;
Norway currently close to target, not to actual inflation.
Agents may well adopt ‘robust forecasts’ as in §9.
25
6 Implications for economic forecasting
Forecast failures confirm these arguments:
either economists uniquely fail own assumptions;
or assumptions are incorrect.
Forecast failure mainly due to location shifts.
Most location shifts seem unanticipated ex ante:
explains forecast failure in equilibrium-correction models,
which are majority of theories and models in economics.
Unanticipated does not entail unpredictable:
more effort to forecast breaks.
Zero-mean changes don’t harm forecasts, pace Lucas;
but could damage policy analyses.
Pernicious for impulse response analyses.
26
“The inability of relatively unstructured, estimated models to predict well for
any length of time outside their estimation period seemed to indicate that
small-sample econometric problems were perhaps more fundamental than
had been appreciated and that too much attention had been paid to capturing the idiosyncrasies of particular samples. There had been a systematic
tendency towards over-fitting equations and too little attention to capturing
(Coletti, Hunt, Rose and Tetlow,
1996, p.14, cited by Pagan, 2003).
the underlying economics.”
No aspect of this assertion can be rigorously established
in a non-stationary world.
Unmodelled location shifts easily explain:
but then conclusion is a non sequitur.
Briefly review structure of forecasting to see this.
27
First, problems learning data density DX1T (·|θ):
(1) specification of the set of relevant variables {xt},
(2) measurement of the xs,
(3) formulation of DX1T (·),
(4) modelling of the relationships,
(5) estimation of θ, and
(6) properties of DX1T (·) determine ‘intrinsic’ uncertainty,
all of which introduce in-sample uncertainties.
Next, problems involving forecast density:
(7) properties of DXT +1 (·) determine forecast uncertainty,
T +H
(8) which grows as H increases,
(9) especially for integrated data,
(10) increased by changes in DXT +1 (·) or θ.
T +H
These 10 issues structure analysis of forecasting.
28
Stationary scalar first-order autoregressive example:
h
i
xt = ρxt−1 + vt where vt ∼ IN 0, σv2 and |ρ| < 1.
With ρ known and constant, forecast from xT is:
DX1 (·)
T
x
bT +1|T = ρxT
implies DXT +1 (·), producing unbiased forecast:
T +1
i
h
E xT +1 − x
bT +1|T | xT = E [(ρ − ρ) xT + vT ] = 0,
with smallest possible variance determined by DX1T (·):
Thus:
h
i
V xT +1 − x
bT +1|T = σv2.
2
DXT +1 (·) = IN ρxT , σv .
T +1
Issues (1)–(10) ‘assumed away’.
(4)
29
Now consider potential problems:
(1) Specification incomplete if (e.g.) xt vector, not scalar.
(2) Measurement incorrect if (e.g.) observe xet not xt.
(3) Formulation inadequate if (e.g.) intercept needed.
(4) Modelling wrong if (e.g.) selected ρxt−2.
(5) Estimating ρ adds bias, (ρ − E[bρ])xT , and variance V[bρ]x2T .
2
(6) Properties of D(vt) = IN 0, σv determine V[xt].
2
(7) Assumed vT +1 ∼ IN 0, σv but V[vT +1] could differ.
PH h−1
1−ρ2H 2
(8) Multi-step forecast error h=1 ρ vT +h has V = 1−ρ2 σv .
(9) If ρ = 1 have trending forecast variance Hσv2.
(10) If ρ changes could experience forecast failure.
Must be prepared for risks from (1)–(10).
Figure 7 shows densities of bρ, σb2v , and tρ=0 for ρ = 0.8.
30
2
xt
^
D(ρ)
4
1
3
0
2
−1
1
−2
0
10
20
30
40
^2 )
D(σ
v
0.15
0.2
0.4
0.6
0.8
1.0
D(tρ=0)
0.15
0.10
0.10
0.05
0.05
2.5
0.0
5.0
7.5
10.0
Figure 7
12.5
15.0
17.5
20.0
0
5
10
15
20
Estimated AR1 ρ = 0.8, T = 40, σv2 = 10.
31
All well behaved, matching theory:
Slightly skew, reflecting dynamic biases.
Figure 8 panels a, b show 1-, 5-step forecasts from (4).
Problems hardly disastrous.
Small increase in uncertainty from estimating ρ
(bands) versus knowing (bars);
forecast intervals grow quite slowly as H increases.
Panels c, d show 1-, 5-step forecasts after break in ρ.
Little noticeable impact from halving ρ at T = 40.
Chow constancy test hardly ever rejects false null.
All forecasts within ex ante intervals.
32
1−step forecasts, known ρ versus estimated ρ 5−step forecasts, constant parameters
20
20
10
10
0
0
known ρ
xt
estimated ρ
full variance
xt
error variance only
40
45
40
20
constancy test under null
constancy test under alternative
1.00
0.75
45
5−step forecasts, changed ρ
5−step forecasts
x t for changed ρ
10
5% significance line
0.50
0
0.25
0.0
2.5
5.0
7.5
10.0
12.5
40
45
Figure 8 Forecasts from AR1, break in ρ = 0.4 at T = 40.
33
But, slight change to model:
h
i
xt = α + ρxt−1 + vt where vt ∼ IN 0, σv2 and |ρ| < 1.
Everything else the same, but α = 10.
Little change in estimation distributions or forecasts:
until non-constant ρ, for same size and time of break.
Then – catastrophe.
Constancy test rejects almost 100%;
forecast failure obvious: all 5-step intervals incorrect.
Change due to effect on E[xt].
In first case E[xt] = 0 before and after shift in ρ.
In second, E[xt] = α/(1 − ρ): shifts markedly from 50 to 17.
34
x t =10+ρxt−1 +v t
60
Constancy test
0.02
50
40
0.01
30
20
0
10
20
30
40
60
0
25
50
75
100
125
150
60
1−step forecasts
xt
50
40
50
40
30
30
20
20
5−step forecasts
xt
10
40
Figure 9
45
40
45
Forecasts from AR1 with intercept & break in ρ.
35
All models in this class are equilibrium correction:
so fail systematically if E[·] changes are unmodelled.
If data fall (rise), multi-step forecasts up (down),
as model equilibrium corrects to old equilibrium mean:
completely misleading for policy.
Serious problems when a model is mis-specified
for a non-time-invariant DGP–
see Norges Bank forecasts below.
Outcome depends on ‘magnitudes’ of deterministic terms,
yet rarely addressed in any economic theory.
Lesson: forecasts not robust to such mis-specifications.
36
Must write conditional expectation as:
bT +h|T = ET +h[xT +h|XT ].
x
Fine if stationary: ET +h = ET .
But paradox if Dxt (·) not constant:
need to know whole future distribution to derive forecast.
Empirically-relevant forecast theory needs to allow for:
model mis-specified for DGP,
parameters estimated from inaccurate observations,
on an integrated-cointegrated system,
which intermittently alters unexpectedly,
from structural breaks.
Better economic theory could reduce first:
but not if merely imposing restrictions on a VAR.
37
7 Impulse response analyses
VAR dynamic changes not easily detected.
Disastrous for impulse-response analyses.
Estimates are weighted averages across regimes:
impulses will not represent policy responses.
We consider a large parameter shift, from:
Φ=
0.50 −0.20
−0.20 −0.25
to
Φ∗ =
0.50 0.20
0.20 0.25
.
(5)
38
1
ChowB:0.05
ChowB:0.01
.9
.8
.7
.6
.5
.4
.3
.2
.1
10
20
30
Figure 10
40
50
60
70
80
90
100
110
120
Constancy-test rejections for I(0) break.
39
.01
Regime1
Regime 2
Mixed
.0075
.002
.001
.005
0
.0025
-.001
0
5
10
0
5
10
0
5
10
.01
.002
.001
.005
0
0
0
5
10
Figure 11 Impulse-responses in an I(0) VAR.
40
Cointegrated system
.01
Regime 1
Regime 2
Mixed regime
.008
.004
.003
.002
.006
.001
.004
0
0
5
10
.006
0
5
10
5
10
.01
.009
.004
.008
.002
.007
.006
0
5
10
0
Figure 12 Impulse responses in an I(1) VAR.
41
Many other problems with impulse-responses:
Describe dynamic properties of estimated model,
not necessarily dynamic characteristics of variables.
Even if model is congruent, encompassing, and invariant to
extensions of information
residuals structural only if model coincides with DGP:
could reflect measurement and specification mistakes.
Most orthogonalized impulses violate weak exogeneity.
Consider bivariate normal in (yt, zt):
yt
zt
∼ IN2
µy
σyy σyz
.
,
µz
σyz σzz
(6)
‘Impulse’ to zt on yt is objective of analysis, with σyz 6= 0.
42
Two ‘Choleski’ factorizations: first use order in (6).
Conditional-marginal factorization is:
yt|zt
zt
∼ IN2
µy + β (zt − µz )
σyy − βσyz 0
,
µz
0
σzz
(7)
where β = σyz /σzz .
Perturbation to {zt} from µz or z,t not identifiable,
yet effect on yt need not be same.
Let agents’ optimization behaviour be:
µy = γµz ,
(e.g., µy , µz = permanent consumption, income), from (7):
yt|zt
zt
∼ IN2
γµz + βz,t
σyy − βσyz 0
,
.
µz
0
σzz
(8)
43
Unless γ = β , it matters which element of {zt} perturbed.
But γ = β is condition for weak exogeneity of zt for β in
conditional model.
If holds, IRs not dependent on which element is perturbed.
In converse Choleski factorization:
yt
zt|yt
∼ IN2
µy
0
, σyy
0 σzz − λσyz
µz + λ y t − µy
when γ = β so λ 6= 1/β by non-degeneracy,
and an incorrect reaction will be inferred.
λ = σyz /σyy
Finally, in closed systems ‘modelling’ policy variables,
impulse-responses require constant instrument process.
(9)
44
8 DSGEs
Deconstruct its 4 main components–
a] economic theory based on intertemporal optimization;
b] identities;
c] ad hoc rigidities;
d] ‘exogenous’ shock processes.
Many aspects debatable, especially c] and d];
additional difficulties of ‘identification’.
But even a] not clear:
consider usual ‘Euler’ equation consumption relation.
Agents smooth – or do they? See Figure 13.
±10% changes Q on Q in real consumers’ expenditure:
dwarf changes in real personal disposable income.
45
Quarterly changes
Levels
12.25
12.00
0.1
log(income) →
∆log(C) ↓
11.75
0.0
11.50
11.25
← log(consumer expenditure)
11.00
1970
1980
1990
−0.1
2000
1970
1980
1990
2000
Consumption to income ratio
Annual changes
0.10
∆log(Y) →
← ∆4 log(Y)
0.750
← C/Y
0.725
0.05
0.700
0.00
← ∆4 log(C)
1970
1980
1990
2000
0.675
1970
1980
1990
2000
Figure 13 UK consumption data: smoothing?.
46
Even annual changes in real consumers’ expenditure less
smooth than in real income over 1982–1992:
explanation seems to be major financial deregulation.
Such non-stationarities large component of data variance:
must be modelled for coherent ‘structural’ explanation.
Data measurement process also can change:
see behaviour of real disposable income from 1997.
Huge increase in seasonal variance from self-assessment
taxe payments moved to January and July.
Data movements reflect more than just economics.
Get clearer picture of economics if model such changes
(e.g., impute taxes evenly over quarters):
suggests advantages to symbiosis not conflict, pace Pagan.
47
c] Price and wage stickiness; capacity utilization;
adjustment costs; habit persistence.
All very sensible, and necessary to ‘fit’ data,
but hardly compelling or ‘deep theory’.
d] Was Industrial Revolution a demand or supply shock?
Certainly large & persistent–yet case for former strong:
demand for fine porcelain; bigger iron canons;
better clothes all induced technology changes.
Much progress on all 4 issues though many old lessons lost.
Some convergence: New Keynesian-Classical model?
Potential route ahead: embed DSGE theories
in congruent encompassing general statistical model
(ongoing research with James Reade using Gets).
48
Key methodology issue to be addressed:
Gets versus simple-to-general (SitG).
Major difficulties with SitG:
no valid constructive use of test rejections;
no unique path of generalization;
more generally, science painfully slow–as has to be SitG.
But econometric modelling need not be.
Evaluation takes many forms generically:
agreement on being essential–but not what and how.
Central: look for evidence against, not ‘confirmation’.
Tests of congruency, encompassing, theory consistency.
Care if using forecast performance (Clements and Hendry, 2003).
Consider ‘counterfactuals’ for policy evaluation.
49
9 Suite of models
Thick modelling all the rage: horses for courses.
The objective matters–as does congruency with evidence.
Can prove more information cannot worsen predictability.
Basis for higher frequency data, and more (e.g., factors).
Hard to prove much more in a non-stationary world:
cannot prove more information cannot worsen forecasts.
Issues of what and how to ‘pool’: recent performance;
‘select best’; weighted average; or reveal divergences.
Always keep a robust forecast device in the set.
Fig.2 from Ragnar Nymoen illustrates dangers otherwise.
Contrast his fig.3.
IR 1/02
IR 2/02
0.04
0.03
0.03
forecast
0.02
0.02
0.01
0.01
actual
2001
2002
2003
2004
2005
2006
2007
2001
0.03
0.03
0.02
0.02
0.01
0.01
2002
2003
2004
2005
2006
2007
2003
2004
2005
2006
2007
2003
2004
2005
2006
2007
IR 1/03
IR 3/02
2001
2002
2003
2004
2005
2006
2007
IR 2/03
2001
0.03
2002
IR 3/03
0.03
0.02
0.02
0.01
0.01
0.00
2001
0.04
2002
2003
2004
2005
2006
2007
2003
2004
2005
2006
2007
2001
2002
IR 1/04
0.03
0.02
0.01
0.00
2001
2002
Figure 2: Norges Bank’s inflation forecasts, 90% confidence regions and the
actual rate of inflation.
14
0.050
"IR 1/02"
"IR 2/02"
0.050
forecast
0.025
0.025
actual
0.000
0.000
2001
2002
2003
2004
2001
2002
2003
2004
2002
2003
2004
2002
2003
2004
"IR 1/03"
"IR 3/02"
0.04
0.02
0.02
0.00
0.00
2001
0.03
2002
2003
2004
2001
"IR 2/03"
"IR 3/03"
0.02
0.02
0.01
0.01
0.00
0.00
2001
0.02
2002
2003
2004
2002
2003
2004
2001
"IR 1/04"
0.01
0.00
2001
Figure 3: Dynamic forecasts from an econometric model of inflation, where we
have emulated the (real time) forecast situation of the Norges Bank inflation
forecasts. The forecasted variable is the annual change in the logarithm of the
20are shown as bars.
CPI-ATE. Forecast confidence intervals
50
‘Naive’ forecasting devices are not what they seem
Example: using ∆xT to forecast.
Consider an in-sample VEqCM DGP:
0
∆xT = γ + α β xT −1 − µ + ΨzT + vT ,
(10)
where zt denotes many omitted effects. Forecast period:
∗
0
∗
∗
∗
∆xT +i = γ + α (β ) xT +i−1 − µ + Ψ∗zT +i + vT +i.
(11)
A VEqCM in xt is used for forecasting:
0
b xT +i−1 − µ
b+α
b β
b .
∆b
xT +i|T +i−1 = γ
All main sources of forecast error occur given (11):
stochastic and deterministic breaks;
omitted variables; inconsistent parameters;
estimation uncertainty; innovation errors.
(12)
51
Contrast using sequence of ∆xT +i−1 to forecast:
∆e
xT +i|T +i−1 = ∆xT +i−1.
(13)
Because of (11), ∆xT +i−1 is (i > 1):
∗
∗
∗
0
∗
∆xT +i−1 = γ + α (β ) xT +i−2 − µ + Ψ∗zT +i−1 + vT +i−1.
(14)
Thus, ∆xT +i−1 reflects all the effects needed:
all parameter changes; no omitted variables;
with no estimation needed.
Two drawbacks: unwanted presence of vT +i−1 in (14),
which doubles innovation error variance;
and all variables lagged one extra period,
which adds ‘noise’ of I(−1) effects.
52
Clear trade-off.
But easy to see why ∆xT +i−1 may win.
Let ∆xT +i − ∆exT +i|T = uT +i, then:
∗
∗
∗
0
∗
uT +i = γ + α (β ) xT +i−1 − µ + Ψ∗zT +i + vT +i
∗
∗
∗
0
∗
∗
− γ + α (β ) xT +i−2 − µ + Ψ zT +i−1 + vT +i−1
= α∗(β ∗)0∆xT +i−1 + Ψ∗∆zT +i + ∆vT +i.
All terms in last line I(−1), so ‘noisy’,
but no systematic failure;
no levels omitted variables;
& no estimation uncertainty.
Estimate α∗(β ∗)0∆xT +i−1 by αβ0∆xT +i−1 from VEqCM.
Same logic applies to DSGE forecasts.
(15)
53
References
Clements, M. P., and Hendry, D. F. (2003). Evaluating a model by forecast performance. Unpublished
paper, Economics Department, University of Warwick.
Coletti, D., Hunt, B., Rose, D., and Tetlow, R. (1996). The Bank of Canada’s new quarterly projection
model, part 3: The dynamic model QPM. Technical report 75, Bank of Canada, Ottawa.
Pagan, A. R. (2003). Report on modelling and forecasting at the Bank of England. Bank of England
Quarterly Bulletin, Spring.