Comparing Predictive Accuracy
Author(s): Francis X. Diebold and Roberto S. Mariano
Source: Journal of Business & Economic Statistics, Vol. 13, No. 3, (Jul., 1995), pp. 253-263
Published by: American Statistical Association
Stable URL: http://www.jstor.org/stable/1392185
Accessed: 07/05/2008 12:16
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=astata.
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.
JSTOR is a not-for-profit organization founded in 1995 to build trusted digital archives for scholarship. We enable the
scholarly community to preserve their work and the materials they rely upon, and to build a common research platform that
promotes the discovery and use of these resources. For more information about JSTOR, please contact support@jstor.org.
http://www.jstor.org
@ 1995AmericanStatisticalAssociation
Journalof Business&Economic
Statistics,July1995,Vol.13, No.3
ComparingPredictive
Accuracy
Francis X. DIEBOLD
of Pennsylvania,
PA19104-6297,and
of Economics,University
Philadelphia,
Department
MA 02138
NationalBureauof EconomicResearch,Cambridge,
RobertoS.
MARIANO
PA 19104-6297
of Pennsylvania,
of Economics,University
Philadelphia,
Department
of no difference
intheaccuracyof
tests of thenullhypothesis
Weproposeandevaluateexplicit
forecasts.Incontrastto previously
twocompeting
developedtests, a widevarietyof accuracy
theloss function
neednotbe quadratic
andneednoteven
measurescanbe used(inparticular,
nonzeromean,seriallycorrelated,
andforecasterrorscan be non-Gaussian,
be symmetric),
correlated.Asymptotic
and exactfinite-sample
tests are proposed,
and contemporaneously
evaluated,andillustrated.
KEYWORDS:Economicloss function;Exchangerates;Forecastevaluation;Forecasting;
tests;Signtest.
Nonparametric
in all of the sciPredictionis of fundamental
importance
ences, includingeconomics. Forecastaccuracyis of obvious importanceto usersof forecastsbecauseforecastsare
used to guide decisions.Forecastaccuracyis also of obvious importanceto producersof forecasts,whose reputations(andfortunes)rise and fall with forecastaccuracy.
to
of forecastaccuracyarealso of importance
Comparisons
economistsmoregenerallywho are interestedin discriminatingamongcompetingeconomichypotheses(models).
andmodeladequacyare inextricaPredictiveperformance
failure
impliesmodelinadequacy.
bly linked-predictive
Giventhe obviousdesirabilityof a formalstatisticalproone is struckby
cedurefor forecast-accuracy
comparisons,
aretypically
the casualmannerin whichsuchcomparisons
carriedout. The literaturecontainsliterallythousandsof
comparisons;almostwithoutexception,
forecast-accuracy
of
forecastaccuracyareexamined,withno
estimates
point
their
assess
samplinguncertainty.Onreflection,
attemptto
of
the reasonfor the casualapproachis clear: Correlation
forecasterrorsacrossspaceandtime,as well as severaladof forecast
makesformalcomparison
ditionalcomplications,
et
al.
and
difficult.
(1972) Howrey,Klein,
Dhrymes
accuracy
andMcCarthy(1974),for example,offeredpessimisticassessmentsof thepossibilitiesforformaltesting.
testsof thenull
Inthisarticlewe proposewidelyapplicable
in
the
of
two
difference
of
no
accuracy
competing
hypothesis
forecasts.Ourapproachis similarin spiritto thatof Vuong
(1989)in the sensethatwe proposemethodsfor measuring
betweenmodels
of divergences
andassessingthesignificance
is
based
anddata.Ourapproach,
however,
directlyonpredicandwe entertaina wideclassof accuracy
tiveperformance,
measuresthatuserscantailorto particular
decision-making
situations.This is important
because,as is well known,realisticeconomicloss functionsfrequentlydo not conform
to stylized textbookfavoriteslike mean squaredpredictionerror(MSPE).[Forexample,LeitchandTanner(1991)
253
andChinnandMeese (1991) stresseddirectionof change,
CumbyandModest(1987)stressedmarketandcountrytiming, McCullochandRossi (1990), andWest,Edison,and
Cho(1993)stressedutility-based
criteria,andClementsand
new
a
Hendry(1993)proposed
accuracymeasure,the generalizedforecast-error
secondmoment.]Moreover,we allow
forforecasterrorsthatarepotentiallynon-Gaussian,
nonzero
and
correlated.
mean,seriallycorrelated, contemporaneously
Weproceedby detailingourtestprocedures
in Section1.
to
Then,in Section2, we reviewthe smallextantliterature
for
the
evaluprovidenecessarybackground
finite-sample
ationof ourtestsin Section3. In Section4 we providean
illustrative
andin Section5 we offerconclusions
application,
anddirectionsforfutureresearch.
1. TESTINGEQUALITY
OF FORECAST
ACCURACY
Considertwo forecasts,{i}yir, and {rj}i,, of the time
series{yt}L. Let the associatedforecasterrorsbe {ej,}r1
and{ejt}rl. Wewishto assessthe expectedloss associated
witheachof theforecasts(oritsnegative,accuracy).
Of great
andalmostalwaysignored,is the factthatthe
importance,
economicloss associatedwitha forecastmaybe poorlyassessedby theusualstatisticalmetrics.Thatis, forecastsare
usedto guidedecisions,andthe loss associatedwitha forecasterrorof a particular
signandsize is induceddirectlyby
thenatureof thedecisionproblemathand.Whenoneconsiders the varietyof decisionsundertaken
by economicagents
decisions,inventoryguidedby forecasts(e.g.,risk-hedging
stockingdecisions,policydecisions,advertising-expenditure
decisions,public-utility
decisions,etc.),it is clear
rate-setting
thatthe loss associatedwitha particular
forecasterroris in
functionof theerrorand,evenif symgeneralanasymmetric
metric,certainlyneednot conformto stylizedtextbookexampleslikeMSPE.
254
Journalof Business&Economic
Statistics,
July1995
Thus, we allow the time-t loss associated with a forecast (say i) to be an arbitraryfunctionof the realizationand
prediction, g(y,,'it). In many applications,the loss function will be a direct function of the forecast error;that is,
g(y,, Y,) = g(eit). To economize on notation,we write g(eit)
from this point on, recognizing that certain loss functions
(like direction-of-change)do not collapse to g(eit,) form, in
which case the full g(y,,Yt) form would be used. The null
hypothesis of equal forecast accuracy for two forecasts is
E[g(eit)] = E[g(ej)], or E[dt] = 0, where dt,
[g(eit) - g(ejt)]
is the loss differential. Thus, the "equalaccuracy"null hypothesis is equivalentto the null hypothesisthatthe population mean of the loss-differentialseries is 0.
1.1 An AsymptoticTest
Considera samplepath {d,}1l of a loss-differentialseries.
If the loss-differentialseriesis covariancestationaryandshort
memory, then standardresults may be used to deduce the
asymptoticdistributionof the sample mean loss differential.
We have
,t) -
Vi(-
N(O,21rfd(O)),
where
S= TE[g(eit) - g(ejt)]
is the sample mean loss differential,
+
fd(O)=
ZE
T=--00
d(Tr)
is the spectraldensity of the loss differentialat frequency0,
yd(7) = E[(dt - lz)(d,_, - z)] is the autocovarianceof the
loss differentialat displacementr, and t is the population
mean loss differential. The formulaforfd(O)shows that the
correctionfor serial correlationcan be substantial,even if
the loss differentialis only weakly serially correlated,due to
cumulationof the autocovarianceterms.
Because in largesamplesthe samplemeanloss differential
is
d approximatelynormally distributedwith mean /t and
variance27rfd(O)/T,
the obviouslarge-sampleN(O, 1) statistic
for testing the null hypothesis of equal forecastaccuracyis
S
2
=
27rfd(?)
wherefd(0) is a consistent estimateofAfd(0).
Following standardpractice, we obtain a consistent estimate of 2lrfd(0) by taking a weighted sum of the available
sample autocovariances,
27fd(0) =
E
r=-(T-
1)
"
1 S(T)
r)
where
1
To motivate a choice of lag window and truncationlag
that we have often found useful in practice, recall the familiar result that optimalk-step-aheadforecast errorsare at
most (k - 1)-dependent.In practicalapplications,of course,
(k - 1)-dependencemay be violated for a varietyof reasons.
Nevertheless,it seems reasonableto take (k - 1)-dependence
as a reasonablebenchmarkfor a k-step-aheadforecast error
(and the assumptionmay be readily assessed empirically).
This suggests the attractivenessof the uniform,or rectangular, lag window,definedby
S(T)
=1
=0
for
S(T)
otherwise.
<1
(k - 1)-dependenceimplies thatonly (k - 1) sample autocovariancesneed be used in the estimationof fd(O)because all
the others are 0, so S(T) = (k - 1). This is legitimate (i.e.,
the estimatoris consistent)under(k - 1)-dependenceso long
as a uniformwindow is used because the uniform window
assigns unit weight to all includedautocovariances.
Because the Dirichletspectralwindow associatedwith the
rectangularlag window dips below 0 at certainlocations,the
resultingestimatorof thespectraldensityfunctionis notguaranteedto be positive semidefinite.The large positive weight
nearthe originassociatedwith the Dirichletkernel,however,
makes it unlikely to obtain a negative estimate offd(0). In
applications,in the rareevent thata negativeestimatearises,
we treat it as 0 and automaticallyreject the null hypothesis of equal forecast accuracy.If it is viewed as particularly
importantto impose nonnegativityof the estimated spectral
density, it may be enforcedby using a Bartlettlag window,
with correspondingnonnegativeFejerspectralwindow,as in
the work of Newey andWest (1987), at the cost of havingto
increasethe truncationlag "appropriately"
with sample size.
Otherlag windows and truncationlag selection procedures
are of coursepossible as well. Andrews(1991), for example,
suggested using a quadraticspectral lag window, together
with a "plug-in"automaticbandwidthselection procedure.
1.2 ExactFinite-SampleTests
Sometimes only a few forecast-errorobservations are
available in practice. One approachin such situations is
to bootstrapour asymptotictest statistic, as done by Mark
(1995). Ashley's (1994) workis also very muchin thatspirit.
Little is knownaboutthe first-orderasymptoticvalidityof the
bootstrapin this situation, however, let alone higher-order
asymptoticsor actualfinite-sampleperformance.Therefore,
it is useful to have availableexact finite-sampletests of predictive accuracy, to complement the asymptotic test presented previously. Two powerful such tests are based on
the observed loss differentials(the sign test) or their ranks
(Wilcoxon's signed-ranktest). [These tests are standard,so
our discussion is terse. See, for example, Lehmann(1975)
for details.]
T
t=IrI+1
1(7r/S(T))is the lag window, and S(T) is the truncationlag.
1.2.1 The Sign Test. The null hypothesis is a zeromedian loss differential:med(g(e,) - g(ef,)) = 0. Note that
the null of a zero-medianloss differentialis not the same
DieboldandMariano:
Predictive
Comparing
Accuracy
as the null of zero difference between median losses; that
is, med(g(ei,) - g(ejt)) med(g(ei,)) - med(g(ej,)). For that
reason, the null differs slightly in spiritfrom thatassociated
with our earlier discussed asymptotictest statistic S1, but it
neverthelesshas an intuitiveandmeaningfulinterpretation-namely,thatP(g(ei,) > g(ej,)) = P(g(ei,) < g(ej,)).
If, however, the loss differential is symmetrically distributed,then the null hypothesis of a zero-medianloss differential corresponds precisely to the earlier null because
in that case the median and mean are equal. Symmetryof
the loss differential will obtain, for example, if the distributions of g(ei,) and g(ej,) are the same up to a location
shift. Symmetryis ultimatelyan empiricalmatterandmay be
assessed using standardprocedures.We have found roughly
symmetric loss-differential series to be quite common in
practice.
The constructionandintuitionof a test statisticarestraightforward.Assuming thatthe loss-differentialseries is iid (and
we shall relax that assumptionshortly), the numberof positive loss-differentialobservationsin a sample of size T has
the binomial distributionwith parametersT and 1 underthe
null hypothesis. The test statisticis thereforesimply
T
S2
I+(d),
where
I+(d,)= 1 if dt > 0
= 0 otherwise.
Significance may be assessed using a table of the cumulative binomial distribution.In large samples, the studentized
version of the sign-test statisticis standardnormal:
S2 =
S2 -
.5T
5T
a
N(O, 1).
1.2.2 Wilcoxon's Signed-RankTest. A related distribution-freeprocedurethatrequiressymmetryof the loss differential(but can be more powerfulthanthe sign test in that
case) is Wilcoxon's signed-ranktest. We again assume for
the moment that the loss-differentialseries is iid. The test
statistic is
the loss functionneed not be quadraticand need not even be
symmetricor continuous.
Second, a varietyof realisticfeaturesof forecasterrorsare
readily accommodated.The forecast errorscan be nonzeromean, non-Gaussian, and contemporaneously correlated.
Allowance for contemporaneouscorrelation,in particular,is
importantbecausethe forecastsbeing comparedareforecasts
of the same economic time series and because the information sets of forecastersarelargelyoverlappingso thatforecast
errorstend to be stronglycontemporaneouslycorrelated.
Moreover, the asymptotic test statistic S, can of course
handle a serially correlatedloss differential. This is potentially importantbecause, as discussed earlier,even optimal
forecasterrorsareseriallycorrelatedin general. Serialcorrelation presentsmore of a problemfor the exact finite-sample
test statisticsS2 and S3 and their asymptoticcounterpartsS2a
and S3a because the elements of the set of all possible rearrangementsof the sample loss differential series are not
equally likely when the data are serially correlated,which
violates the assumptionson which such randomizationtests
are based. Nevertheless, serial correlationmay be handled
via Bonferronibounds,as suggested in a differentcontextby
Campbell and Ghysels (1995). Under the assumptionthat
the forecasterrorsand hence the loss differentialare (k - 1)dependent,each of the following k sets of loss differentials
will be free of serial correlation: {dij, i,dj, 1+kij,1+2k
,..
. . .} .....
{dij,2,dij,2+k,
dij,2+2k,
d,k, dii,k, dij,3k
,.... Thus, a
test with size boundedby a can be obtainedby performing
k tests, each of size a/k, on each of the k loss-differential
sequences and rejectingthe null hypothesis if the null is rejected for any of the k samples. Finally, it is interestingto
note that, in multistepforecast comparisons, forecast-error
serial correlationmay be a "commonfeature,"in the terminology of Engle and Kozicki (1993), because it is induced
largelyby the fact thatthe forecasthorizonis longer thanthe
intervalat which the dataare sampledand may thereforenot
be presentin loss differentialseven if presentin the forecast
errorsthemselves. This possibility can of course be checked
empirically.
2. EXTANT
TESTS
T
S3 = ZI+(d,) rank(Id,I),
the sum of the ranks of the absolute values of the positive
observations.The exact finite-samplecritical values of the
test statistic are invariant to the distribution of the loss
differential-it need be only zero-meanandsymmetric-and
have been tabulated. Moreover, its studentized version is
asymptoticallystandardnormal,
S _a=
1.3
S3 -
T(T+I)
4
/T(T+1)(2T+I)
24
a
N(O,1).
Discussion
Here we highlight some of the virtues and limitationsof
our tests. First, as we have stressedrepeatedly,our tests are
valid for a very wide class of loss functions. In particular,
255
In this section we provide a brief descriptionof three existing tests of forecast accuracy that have appearedin the
literatureand will be used in our subsequentMonte Carlo
comparison.
2.1
The Simple F Test: A Naive Benchmark
If (1) loss is quadraticand(2) the forecasterrorsare(a) zero
mean, (b) Gaussian,(c) serially uncorrelated,or (d) contemporaneouslyuncorrelated,then the null hypothesis of equal
forecast accuracycorrespondsto equal forecast error variances [by (1) and(2a)], andby (2b)-(2d), the ratio of sample
varianceshas the usual F distributionunderthe null hypothesis. More precisely,the test statistic
F=
=eje
256
Journalof Business&Economic
Statistics,
July1995
is distributedas F(T, T), where the forecasterrorseries have
been stackedinto the (T x 1) vectorsei and ei.
Test statistic F is of little use in practice, however, because the conditions requiredto obtain its distributionare
too restrictive. Assumption (2d) is particularlyunpalatable
for reasons discussed earlier. Its violation producescorrelation between the numeratorand denominatorof F, which
will not then have the F distribution.
A consistentestimatorof E is
=
Z
'=-S(T)
[1 - T [X]
MGN =is distributedas Student'st with T - 1 df, where
x'z
+ '(r)7(r)],
where
T
=
2.2 The Morgan-Granger-Newbold
Test
The contemporaneouscorrelation problem led Granger
and Newbold (1977) to apply an orthogonalizingtransformation due to Morgan(1939-1940) that enables relaxation
of Assumption(2d). Let x, = (ei,+ ej,) andz, = (ei, - ej,),and
let x = (e, + ej) and z = (ei - el). Then, underthe maintained
Assumptions(1) and (2a)-(2c), the null hypothesisof equal
forecastaccuracyis equivalentto zero correlationbetweenx
and z (i.e., p. = 0) and the test statistic
(r7(r)
t=-r+1
(-r)
otherwise,
T1
t=r+1
otherwise,
= •z(-r)
Sr)
= -
t
1T
1=
4
S=
andthe truncationlag S(T) growswith the samplesize butat a
slowerrate. Alternatively,following Diebold andRudebusch
(1991), one may use the closely related covariance matrix
estimator,
S(T)
" Z
"
[U(-r)?n(-r) + --r)-(r)]
-=-S(T)
(e.g., see Hogg and Craig 1978, pp. 300-303).
Let us now consider relaxing the Assumptions (1) and
(2a)-(2c) underlyingthe Morgan-Granger-Newbold(MGN)
test. It is clear that the entire frameworkdepends crucially
on the assumption of quadraticloss (1), which cannot be
relaxed. The remainingassumptions,however,can be weakened in varyingdegrees; we shall considerthem in turn.
First, it is not difficultto relax the unbiasednessAssumption (2a), while maintainingAssumptions(1), (2b), and (2c).
Second, the normality Assumption (2b) may be relaxed,
while maintaining (1), (2a), and (2c), at the cost of substantialtediuminvolvedwith accountingfor the higher-order
momentsthatthenenterthe distributionof the samplecorrelation coefficient (e.g., see Kendalland Stuart1979, chap. 26).
Finally, the no-serial-correlationAssumption (2c) may be
relaxed in addition to the no-contemporaneous-correlation
Assumption (2d) while maintaining(1), (2a), and (2b), as
discussed in Subsection2.3.
Eitherway, the test statisticis
MR =
Under the null hypothesisand the maintainedAssumptions
(1), (2a), and (2b), MR (Meese-Rogoff) is asymptotically
distributedas standardnormal.
It is easy to show that,if the null hypothesisand Assumptions (1), (2a), (2b), and (2c) are satisfied,then all termsin E
are0 except -y(0) and'y(O) so thatMR coincides asymptotically with MGN. It is interestingto note also thatreformulation of the test in termsof correlationratherthan covariance
would have enabled Meese and Rogoff to dispense with the
normalityassumptionbecause the sample autocorrelations
are asymptoticallynormaleven for non-Gaussiantime series
(e.g., Brockwell and Davis 1992, pp. 221-222).
2.4 AdditionalExtensions
In Subsection 2.3, we consideredrelaxationof Assumptions (2a)-(2c), one at a time, while consistentlymaintaining
Assumption(1) and consistentlyrelaxing Assumption (2d).
UnderAssumptions(1), (2a), and (2b), Meese and Rogoff
Simultaneousrelaxationof multipleassumptionsis possible
(1988) showed that
within the MGN orthogonalizingtransformationframework
but much more tedious. The distributiontheoryrequiredfor
joint relaxationof (2b) and (2c), for example, is complicated
where 7 = x'z/T, C = -7•)___o['(7)'Y(7) + %YZ(7-)Y(7)], by the presenceof fourth-ordercumulantsin the distribution
of the the sampleautocovariances,as shown, for example,by
-Y(7) = cov(x,,z,,), Y%(7)= cov(z,,x,_,), %,(r) =
Hannan(1970, p. 209) and Mizrach (1991). More imporcov(x,,x,_,), and'y(7) = coy(z,, z,_,). This is a well-known
result(e.g., Priestley 1981, pp. 692-693) for the distribution
tantly,however,any procedurebased on the MGN orthogoof the sample cross-covariancefunction, cov(•(s), z(u)),
is inextricablywed to the assumption
nalizingtransformation
of quadraticloss.
specializedto a displacementof 0.
2.3
The Meese-Rogoff
Test
DieboldandMariano:
Predictive
Comparing
Accuracy
CARLOANALYSIS
3. MONTE
standardizationamountsto dividingthe t(6) randomvariable
byV\72.
3.1 ExperimentalDesign
We evaluate the finite-sample size of test statistics F,
MGN, MR, S1, S2, S2a, S3, and S3aunder the null hypothesis and variousof the maintainedassumptions.The design
includes a variety of specifications of forecast-errorcontemporaneouscorrelation, forecast-errorserial correlation,
and forecast-errordistributions.To maintainapplicabilityof
all test statistics for comparisonpurposes,we use quadratic
loss; that is, the null hypothesis is an equality of MSPE's.
We emphasize again, however,that an importantadvantage
of test statistics S1, S2, S2a, S3, and S3ain substantiveeconomic applications-and one not shared by the others-is
their direct applicabilityto analyses with nonquadraticloss
functions.
Consider first the case of Gaussian forecast errors. We
draw realizations of the bivariate forecast-errorprocess,
with varying degrees of contemporaneousand
{eit,ejt}11,
serial correlation in the generated forecast errors. This is
achieved in two steps. First, we build in the desired degree of contemporaneouscorrelationby drawinga (2 x 1)
forecast errorinnovationvector u, from a bivariatestandard
normal distribution,ut, N(02,12), and then premultiplying by the Choleski factor of the desired contemporaneous innovationcorrelationmatrix.Let the desiredcorrelation
matrixbe
'
R=
pE [0, 1).
1
Then the Choleski factoris
1
0
P vp -p
?
Thus, the transformed(2 x 1) vector v, = Pu, , N(02, R).
This operationis repeatedT times, yielding {vi,,
Vj,}l-.
Second, (moving average)MA(1) serial correlation(with
parameter0) is introducedby taking
ei,
[etJ
ej,
1
_
L
0
0
257
Vit
- t.
L Vt
I.2
t=,1...
T.
1+2L
J
We use Vo= 0. Multiplicationby (1 + 02)-1/2 is done to keep
the unconditionalvariancenormalizedto 1.
We consider sample sizes of T = 8, 16, 32, 64, 128, 256,
and 512, contemporaneouscorrelationparametersof p = 0,
.5, and .9, and MA parametersof 0 = 0, .5, .9. Simple calculationsrevealthatp is not only the correlationbetween•vand
v, but also the correlationbetween the forecasterrorsei and
ei so thatvaryingthe correlationof vyandv, through[0, .9] effectively variesthe correlationof the observedforecasterrors
throughthe same range.
We also considernon-Gaussianforecasterrors.Thedesign
is the same as for the Gaussiancase describedpreviouslybut
driven by fat-tailed variates (uit,u;*)'[ratherthan (ui,, uj,)'],
which are independentstandardizedt randomvariableswith
6 df. The variance of a t(6) randomvariableis 3/2. Thus,
Throughout,we performtests at the ac = .1 level. When
using the exact sign andsigned-ranktests, restrictionof nominal size to precisely 10%is impossible (withoutintroducing
randomization),so we use the obtainableexact size closest
to 10%,as specified in the tables. We performat least 5,000
Monte Carloreplications. The truncationlag is set at 1, reflectingthe fact thatthe experimentis designed to mimic the
comparisonof two-step-aheadforecast errors, with associated MA(1) structure.
3.2 Results
Results appearin Tables 1-6, which show the empirical
size of the varioustest statisticsin cases of GaussianandnonGaussianforecast errorsas the degree of contemporaneous
correlation,the degree of serial correlation,and sample size
are varied.
Let us first discuss the case of Gaussian forecast errors.
The resultsmay be summarizedas follows:
1. F is correctlysized in the absence of both contemporaneous and serial correlationbut is missized in the presence
of either contemporaneousor serial correlation. Serial
correlationpushes empiricalsize above nominal size, but
contemporaneouscorrelationpushesempiricalsize drastically below nominalsize. In combination,andparticularly
for largep and 0, contemporaneouscorrelationdominates
and F is undersized.
2. MGN is designed to remainunaffectedby contemporaneous correlationand thereforeremains correctly sized so
long as 0 = 0. Serialcorrelation,however,pushes empirical size above nominalsize.
3. As expected,MR is robustto contemporaneousand serial
correlationin large samples, but it is oversized in small
samples in the presence of serial correlation.The asymptotic distributionobtainsratherquickly,however,resulting
in approximatelycorrectsize for T > 64.
The
4.
behaviorof S1 is similarto that of MR. S1 is robustto
contemporaneousand serial correlationin large samples,
but it is oversized in small samples, with nominal and
empiricalsize converginga bit more slowly than for MR.
5. The Bonferronibounds associated with S2 and S3 work
well, with nominaland empiricalsize in close agreement
throughout.Moreover,the asymptoticson which S2a and
S3adependobtainquickly.
Now consider the case of non-Gaussianforecast errors.
The strikingand readilyapparentresult is that F, MGN, and
MR aredrasticallymissized in largeas well as small samples.
S1, S2o,and S3a, on the other hand, maintainapproximately
correctsize for all but the very small sample sizes. In those
cases, S2 and S3 continue to performwell. The results are
well summarizedby Figure 1, p. 261, which charts the dependenceof F, MGN, MR, and S1on T for the non-Gaussian
case with p = 9 = .5.
258
Journalof Business &EconomicStatistics,July1995
Table 1. EmpiricalSize UnderQuadraticLoss, TestStatisticF
Gaussian
T
p
8=.0
Fat-tailed
8=.5
8=.9
8=.0
8=.5
8=.9
8
8
8
.0
.5
.9
9.85
7.02
.58
12.14
9.49
1.26
14.10
11.42
1.86
14.28
9.61
.57
15.76
11.64
1.13
17.21
13.02
1.79
16
16
16
.0
.5
.9
9.83
7.30
.47
12.97
10.11
.99
14.85
11.89
1.55
16.47
11.14
.34
18.59
13.55
.70
19.78
14.94
1.13
32
32
32
.0
.5
.9
9.88
6.98
.23
12.68
9.50
.55
14.34
11.22
1.00
18.06
21.30
.01
19.55
21.00
.07
20.35
21.37
.23
64
64
64
.0
.5
.9
9.71
6.48
.16
13.05
9.25
.47
14.62
10.62
.79
29.84
23.48
.02
29.72
23.93
.12
29.96
24.15
.29
128
128
128
.0
.5
.9
10.30
7.01
.16
13.41
10.13
.50
14.99
11.64
.74
30.34
24.89
.11
30.95
25.01
.44
31.26
25.16
.73
256
256
256
.0
.5
.9
10.01
7.37
.19
13.05
10.31
.51
14.65
11.78
.80
31.07
25.48
.51
31.12
25.45
1.13
31.24
25.70
1.44
512
512
512
.0
.5
.9
10.22
7.53
.18
13.51
10.16
.50
15.25
11.49
.85
31.45
26.35
.81
32.38
26.92
1.58
32.60
16.95
2.06
NOTE: T is sample size, p is the contemporaneouscorrelationbetweenthe innovationsunderlyingthe forecasterrors,and 0 is the
coefficientof the MA(1)forecasterror.Alltests are at the 10%level. 10,000MonteCarloreplications
are performed.
Table 2. EmpiricalSize UnderQuadraticLoss, TestStatisticMGN
Gaussian
Fat-tailed
T
p
8 =.0
8 =.5
0 =.9
8 =.0
8 =.5
0 =.9
8
8
8
.0
.5
.9
10.19
9.96
9.75
14.14
14.66
14.53
17.94
18.61
18.67
18.10
16.00
11.76
21.89
20.51
16.31
25.65
24.19
20.00
16
16
16
.0
.5
.9
10.07
9.56
10.02
14.34
14.37
14.70
17.54
17.95
18.20
20.33
37.15
12.01
24.54
36.18
16.76
27.08
25.66
19.81
32
32
32
.0
.5
.9
9.89
10.08
9.59
15.04
15.11
15.32
18.00
17.95
18.25
22.94
20.23
12.75
26.32
23.76
17.78
28.72
26.20
20.54
64
64
64
.0
.5
.9
10.09
9.95
10.26
15.37
15.18
15.67
17.99
18.15
18.49
24.56
21.10
12.98
28.15
25.18
18.09
30.00
27.28
20.53
128
128
128
.0
.5
.9
9.96
10.23
10.11
15.09
15.07
15.05
17.59
17.48
18.05
26.47
23.62
14.34
29.50
26.82
18.89
30.94
28.51
21.56
256
256
256
.0
.5
.9
10.28
10.60
10.11
15.62
16.02
15.48
18.37
18.44
17.91
27.39
23.81
14.15
30.74
28.38
19.43
32.46
30.31
22.03
512
512
512
.0
.5
.9
10.12
10.05
9.90
15.34
14.96
15.09
17.68
17.66
17.53
27.64
24.10
14.78
30.55
27.40
19.16
32.14
29.28
21.49
NOTE: T is sample size, p is the contemporaneous correlation between the innovations underlying the forecast errors, and 0 is the
coefficient of the MA(1) forecast error. All tests are at the 10% level. 10,000 Monte Carlo replications are performed.
Dieboldand Mariano:ComparingPredictiveAccuracy
Table 3. EmpiricalSize UnderQuadraticLoss, TestStatisticMR
Gaussian
T
p
0=.0
Fat-tailed
0 =.5
0=.9
0= .0
0 =.5
0=.9
8
8
8
.0
.5
.9
9.67
9.50
9.66
19.33
19.00
19.51
22.45
22.07
22.85
16.16
14.81
11.23
25.26
24.50
21.28
27.62
26.99
24.14
16
16
16
.0
.5
.9
9.62
10.02
10.04
13.92
13.88
13.82
14.72
14.96
14.94
19.94
17.70
11.76
22.56
21.04
15.68
23.06
21.26
16.70
32
32
32
.0
.5
.9
9.96
9.68
9.86
10.98
11.46
11.62
11.12
11.66
11.96
22.78
19.78
12.42
22.86
20.32
13.54
21.72
20.14
13.46
64
64
64
.0
.5
.9
10.32
9.84
9.58
11.02
10.56
10.58
11.04
10.64
10.34
24.50
21.44
13.38
22.60
19.48
13.38
21.58
18.84
13.20
128
128
128
.0
.5
.9
9.78
10.02
10.76
10.54
11.04
11.28
10.44
11.18
11.38
25.86
22.76
13.44
22.90
20.26
13.52
21.54
19.44
12.92
256
256
256
.0
.5
.9
10.04
10.32
9.92
9.90
9.92
10.16
9.58
9.82
10.34
27.16
24.00
13.38
23.74
20.50
12.70
22.70
19.18
12.24
512
512
512
.0
.5
.9
9.94
9.52
9.80
10.48
10.56
9.82
10.56
10.48
9.88
26.92
23.56
13.96
23.40
20.52
12.98
21.78
19.36
12.74
NOTE: T is sample size, p is the contemporaneouscorrelationbetweenthe innovationsunderlyingthe forecasterrors,and 0 is the
coefficientof the MA(1)forecasterror.Alltests are at the 10%level. Atleast 5,000 MonteCarloreplicationsare performed.
Table 4. EmpiricalSize UnderQuadraticLoss, TestStatisticS,
Gaussian
Fat-tailed
p
0= .0
0= .5
0= .9
0= .0
0= .5
0= .9
8
8
8
.0
.5
.9
31.39
31.37
31.08
31.10
30.39
30.19
31.03
29.93
30.18
31.62
31.21
31.18
29.51
29.71
30.12
29.07
29.36
29.75
16
16
16
.0
.5
.9
20.39
20.43
20.90
19.11
19.52
19.55
18.94
18.86
19.59
19.26
19.57
20.15
18.50
17.67
18.38
18.32
17.63
18.16
32
32
32
.0
.5
.9
12.42
13.32
12.60
12.28
13.22
13.38
12.18
12.94
13.22
11.30
11.54
11.16
11.64
10.66
11.22
11.56
10.84
11.50
64
64
64
.0
.5
.9
12.47
12.76
12.21
12.11
12.49
12.23
11.94
12.35
12.03
12.44
12.10
13.00
11.62
12.26
12.36
11.36
12.10
12.16
128
128
128
.0
.5
.9
11.72
11.44
11.76
11.94
11.72
11.26
12.04
11.60
11.34
11.48
10.84
11.50
10.72
10.96
10.66
10.28
10.96
10.86
256
256
256
.0
.5
.9
11.11
10.90
10.69
10.65
10.39
10.79
10.66
10.48
10.75
12.06
12.16
11.51
11.67
11.46
11.59
11.79
11.60
11.16
512
512
512
.0
.5
.9
11.15
10.90
10.31
10.67
10.39
10.09
10.63
10.49
10.05
10.06
9.94
10.12
9.46
9.66
10.12
9.62
9.76
10.06
T
NOTE: T is sample size, p is the contemporaneous correlation between the innovations underlying the forecast errors, and 0 is the
coefficient of the MA(1) forecast error. All tests are at the 10% level. At least 5,000 Monte Carlo replications are performed.
259
260
Journalof Business & EconomicStatistics,July1995
Table 5. EmpiricalSize UnderQuadraticLoss, TestStatisticsS2 and S2a
Gaussian
T
p
Fat-tailed
0=.0
0 = .5
0=.9
23.94
23.08
22.92
23.46
24.80
23.26
23.34
23.06
22.86
13.46
14.22
13.08
S2, nominalsize = 14.08%
13.26
13.14
13.46
12.92
13.84
13.28
13.62
13.70
12.86
13.06
13.24
13.06
13.76
13.62
13.20
14.36
14.36
14.68
S2, nominalsize = 15.36%
14.52
14.28
14.06
13.94
14.62
13.46
14.54
15.08
14.94
14.32
14.36
14.76
14.30
15.02
14.52
9.68
9.52
9.40
10.36
10.06
8.98
10.44
10.00
10.02
12.22
12.06
12.06
12.20
11.94
10.76
11.42
11.44
11.40
0=.0
0=.5
0=.9
S2, nominalsize = 25%
8
8
8
16
16
16
32
32
32
.0
.5
.9
.0
.5
.9
.0
.5
.9
22.24
22.14
22.24
22.48
23.46
23.02
22.38
22.16
22.66
S2a, nominal size = 10%
64
64
64
.0
.5
.9
9.72
9.66
10.84
9.92
10.34
9.46
9.42
9.68
10.34
S2a, nominal size = 10%
128
128
128
.0
.5
.9
11.62
11.66
11.22
11.62
11.62
11.72
11.84
11.90
11.28
NOTE: T is sample size, p is the contemporaneouscorrelationbetweenthe innovationsunderlyingthe forecasterrors,and 0 is the
are performed.
coefficientof the MA(1)forecasterror.Atleast 5,000 MonteCarloreplications
Table 6. EmpiricalSize UnderQuadraticLoss, TestStatisticsS3 and S,
Gaussian
T
p
9=.0
0=.5
Fat-tailed
0=.9
0=.0
0=.5
0=.9
23.26
23.42
24.26
23.34
23.86
23.32
21.96
22.88
23.34
10.16
10.54
10.58
10.42
10.94
10.96
9.84
10.34
10.64
9.90
10.40
10.46
10.00
10.64
9.96
9.98
10.30
10.70
9.64
9.58
9.92
9.24
8.82
9.78
8.84
8.78
10.00
9.82
10.08
9.28
9.04
9.24
9.22
8.46
9.20
9.26
S3, nominalsize = 25%
8
8
8
16
16
16
.0
.5
.9
.0
.5
.9
22.50
22.98
23.16
10.62
10.38
10.64
22.92
22.26
22.36
22.90
23.06
24.24
S3, nominalsize = 10.92%
10.06
10.40
10.92
10.32
10.18
9.62
S3, nominalsize = 10.12%
32
32
32
.0
.5
.9
10.72
10.56
10.92
64
64
64
.0
.5
.9
9.38
9.80
9.90
128
128
128
.0
.5
.9
9.94
9.52
9.46
10.28
10.00
10.44
9.30
10.02
10.30
S3a, nominalsize = 10%
9.54
9.16
10.02
9.66
9.24
9.68
S3a, nominal size = 10%
9.70
10.00
9.64
9.12
9.32
9.42
NOTE: T is sample size, p is the contemporaneouscorrelationbetweenthe innovationsunderlyingthe forecasterrors,and 0 is the
coefficientof the MA(1)forecasterror.Atleast 5,000 MonteCarloreplicationsare performed.
Dieboldand Mariano:ComparingPredictiveAccuracy
261
1.25
30
1.00
MGN
43
N 25
rn
0.75
S0.50
0.25
a
S150
S0.00
-0.25
8
16
32
64
128
256
78
512
Sample Size
1.
Case;
Size,
Figure Empirical FourTestStatistics:Fat-Tailed
We shall illustrate the practicaluse of the tests with an
application to exchange-rateforecasting. The series to be
forecast,measuredmonthly,is the three-monthchangein the
nominal dollar/Dutchguilder end-of-monthspot exchange
rate(in U.S. cents, noon, New Yorkinterbank),from 1977.01
to 1991.12. We assess two forecasts, the "no change" (0)
forecast associated with a random-walkmodel and the forecast implicit in the three-monthforwardrate (the difference
between the three-monthforwardrate and the spot rate).
The actual and predictedchanges are shown in Figure 2.
The random-walkforecast, of course, is just constant at 0,
whereas the forwardmarketforecast moves over time. The
movements in both forecasts, however, are dwarfedby the
realized movementsin exchange rates.
We shall assess the forecasts'accuracyunderabsoluteerror
loss. In termsof point estimates,the random-walkforecastis
more accurate. The mean absoluteerrorof the random-walk
forecast is 1.42, as opposed to 1.53 for the forwardmarket
forecast;as one hearsso often, "Therandomwalk wins." The
82
84
Time
86
88
90
Figure3. LossDifferential
walk).
(forward--random
Theta= Rho = .5.
4. AN EMPIRICAL
EXAMPLE
80
loss-differentialseries is shown in Figure 3, in which no obvious nonstationaritiesarevisually apparently.Approximate
stationarityis also supportedby the sample autocorrelation
function of the loss differential,shown in Figure 4, which
decays quickly.
Because the forecastsare three-step-ahead,our earlierargumentssuggest the need to allow for at least two-dependent
forecast errors, which may translateinto a two-dependent
loss differential. This intuitionis confirmedby the sample
autocorrelationfunctionof the loss differential,in which sizable and significantsample autocorrelationsappearat lags 1
and 2 and nowhereelse. The Box-Pierce X2test of jointly
zero autocorrelationsat lags 1 through 15 is 51.12, which
is highly significantrelative to its asymptotic null distribution of X'5. Conversely,the Box-Pierce X2 test of jointly
zero autocorrelationsat lags 3 through 15 is 12.79, which is
insignificantrelativeto its null distributionof x'.
We now proceedto test the null of equal expected loss. F,
MGN, andMR areinapplicablebecause one or more of their
0.5
5.0
0.4
2.5
0.3
0
r
-2.5
S0.0
-5.0
77 78 79 80 81 82 83 84 85 86 87 88 89 90 91
Time
Figure2. Actualand PredictedExchange-Rate
Changes. The
solidlineis theactualexchange-rate
change.Theshortdashedline
is thepredictedchangefromthe random-walk
model,andthelong
dashedlineis thepredicted
rate.
changeimpliedby theforward
-0.2
2
4
6
8
Displacement
Autocorrelations.
ThefirsteightsamFigure4. LossDifferential
aregraphed,togetherwithBartlett's
pleautocorrelations
approximate
95%confidenceinterval.
262
Journalof Business&Economic
Statistics,
July1995
maintainedassumptionsareexplicitly violated. We therefore
focus on our test statisticS1, setting the truncationlag at two
in light of the preceding discussion. We obtain S1 = -1.3,
implying a p value of .19. Thus, for the sample at hand, we
do not reject at conventionallevels the hypothesis of equal
expectedabsoluteerror-the forwardrateis not a statistically
significantlyworse predictorof the futurespot ratethanis the
currentspot rate.
5. CONCLUSIONSAND DIRECTIONS
FOR
FUTURERESEARCH
We have proposed several tests of the null hypothesisof
equal forecast accuracy.We allow the forecast errorsto be
non-Gaussian,nonzero mean, serially correlated,and contemporaneouslycorrelated.Perhaps most importantly,our
tests are applicableundera very wide varietyof loss structures.
We hasten to add that comparisonof forecast accuracyis
but one of many diagnostics that should be examinedwhen
comparingmodels. Moreover,the superiorityof a particular model in terms of forecast accuracydoes not necessarily
imply thatforecastsfrom othermodels containno additional
information.That, of course, is the well-knownmessage of
the forecast combinationand encompassingliteratures;see,
for example, Clemen (1989), Chong andHendry(1986), and
Fairand Shiller (1990).
Several extensions of the results presentedhere appearto
be promisingdirectionsfor futureresearch. Some are obvious, such as generalizationto comparisonof more thantwo
forecasts or, perhapsmost generally, multiple forecasts for
each of multiplevariables. Othersare less obvious andmore
interesting.We shall list just a few:
1. Our frameworkmay be broadenedto examine not only
whetherforecast loss differentialshave nonzeromean but
also whether other variables may explain loss differentials. For example, one could regress the loss differential
not only on a constant but also on a "stage of the business cycle" indicatorto assess the extentto whichrelative
predictiveperformancediffers over the cycle.
2. The ability to formally compare predictive accuracy
afforded by our tests may prove useful as a modelspecification diagnostic, as well as a means to test
both nested and nonnested hypotheses under nonstandardconditions, in the traditionof Ashley, Granger,and
Schmalensee (1980) and Marianoand Brown (1983).
3. Explicit accountmay be takenof the effects of uncertainty
associatedwith estimatedmodelparameterson the behavior of the test statistics, as shown by West (1994).
Let us provide some examples of the ideas sketchedin 2.
First, consider the development of a test of exclusion restrictions in time series regression that is valid regardless
of whetherthe data are stationaryor cointegrated. The desirability of such a test is apparentfrom works like those
of Stock and Watson (1989), Christianoand Eichenbaum
(1990), Rudebusch(1993), and Todaand Phillips (1993), in
which it is simultaneouslyapparentthat (a) it is difficultto
determinereliably the integrationstatus of macroeconomic
time seriesand(b)theconclusionsofmacroeconometricstudies are often criticallydependenton the integrationstatusof
the relevanttime series. One may proceed by noting that
tests of exclusion restrictionsamountto comparisonsof restrictedand unrestrictedsums of squares. This suggests estimatingthe restrictedand unrestrictedmodels using partof
the availabledata and then using our test of equality of the
mean squarederrorsof the respective one-step-aheadforecasts.
As a second example, it would appearthat our test is applicable in nonstandardtesting situations, such as when a
nuisanceparameteris not identifiedunderthe null. This occurs, for example, when testing for the appropriatenumber
of states in Hamilton's(1989) Markov-switchingmodel. In
spite of the fact thatstandardtests are inapplicable,certainly
the null and alternativemodels may be estimated and their
out-of-sampleforecastingperformancecomparedrigorously,
as shown by Engel (1994).
In closing, we note that this article is part of a largerresearchprogramaimed at doing model selection, estimation,
prediction,and evaluationusing the relevant loss function,
whateverthatloss functionmaybe. This articlehas addressed
evaluation. Granger(1969) and Christoffersenand Diebold
(1994) addressedprediction. These results, together with
those of Weiss andAndersen(1984) and Weiss (1991, 1994)
on estimationunderthe relevantloss functionwill make feasible recursive,real-time,prediction-basedmodel selection
underthe relevantloss function.
ACKNOWLEDGMENTS
We thank the editor, associate editor, and two referees
for constructivecomments.Seminarparticipantsat Chicago,
Cornell,the FederalReserve Board,London School of Economics, Maryland,the Model ComparisonSeminar,Oxford,
Pennsylvania,Pittsburgh,and Santa Cruz provided helpful
input, as did Rob Engle, Jim Hamilton, Hashem Pesaran,
IngmarPrucha,PeterRobinson,and Ken West, but all errors
are ours alone. Portions of this article were written while
the first authorvisited the Financial Markets Group at the
London School of Economics, whose hospitality is gratefully acknowledged. Financial supportfrom the National
Science Foundation,the Sloan Foundation,and the University of PennsylvaniaResearchFoundationis gratefully acknowledged. Ralph Bradley,Jos6 A. Lopez, and Gretchen
Weinbachprovidedresearchassistance.
[ReceivedMarch1994. RevisedDecember 1994.]
REFERENCES
Andrews, D. W. K. (1991), "Heteroskedasticityand AutocorrelationConsistent CovarianceMatrixEstimation,"Econometrica,59, 817-858.
Ashley, R. (1994), "PostsampleModel Validationand InferenceMade Feasible," unpublishedmanuscript,VirginiaPolytechnic Institute,Dept. of
Economics.
Ashley, R., Granger,C. W.J., and Schmalensee,R. (1980), "Advertisingand
Aggregate Consumption:An Analysis of Causality,"Econometrica,48,
1149-1167.
DieboldandMariano:
Predictive
Accuracy
Comparing
263
Brockwell, P.J., and Davis, R. A. (1992), TimeSeries: Theoryand Methods
Kendall,M., andStuart,A. (1979), TheAdvancedTheoryofStatistics(Vol. 2,
(2nded.),NewYork:Springer-Verlag.
of theFederalBudget
B., andGhysels,E. (1995),"IstheOutcome
Campbell,
Review
Assessment,"
ProcessUnbiasedandEfficient?A Nonparametric
Lehmann, E. L. (1975), Nonparametrics: Statistical Methods Based on
of Economics and Statistics, 77, 17-31.
on Currency
Forecasts:Is
Chinn,M., andMeese,R. A. (1991),"Banking
Universityof
manuscript,
unpublished
Changein MoneyPredictable?"
Schoolof Business.
California,
Berkeley,Graduate
of Linear
Evaluation
Chong,Y Y.,andHendry,D. F. (1986),"Econometric
MacroeconomicModels,"Reviewof EconomicStudies,53, 671-690.
M. (1990),"UnitRootsin RealGNP:Do
L., andEichenbaum,
Christiano,
We Know, and Do We Care?"Carnegie-RochesterConferenceSeries on
Public Policy, 32, 7-61.
Under
Prediction
Christoffersen,
P., andDiebold,F. X. (1994),"Optimal
AsymmetricLoss,"TechnicalWorkingPaper167, NationalBureauof
MA.
EconomicResearch,Cambridge,
Forecasts:A ReviewandAnnotated
Clemen,R. T. (1989), "Combining
4thed.),NewYork:OxfordUniversity
Press.
Ranks,SanFrancisco:Holden-Day.
ForecastEvaluation:
Leitch,G., andTanner,J. E. (1991), "Econometric
ErrorMeasures,"
ProfitsVersusthe Conventional
AmericanEconomic
Review,81, 580-590.
R. S., andBrown,B. W.(1983),"Prediction-Based
TestforMisMariano,
in NonlinearSimultaneous
Systems,"in Studiesin Econospecification
metrics, TimeSeries and MultivariateStatistics,Essays in Honor of T W
Anderson,eds. T. Amemiya,S. Karlin,andL. Goodman,New York:
AcademicPress,pp. 131-151.
RatesandFundamentals:
Evidenceon LongMark,N. (1995),"Exchange
HorizonPredictability,"
AmericanEconomicReview, 85, 201-218.
andUtilityMcCulloch,
R., andRossi,P.E. (1990),"Posterior,
Predictive,
to Testingthe Arbitrage
Journalof
BasedApproaches
PricingTheory,"
Financial Economics,28, 7-38.
Bibliography"(with discussion), InternationalJournalof Forecasting,5,
559-583.
Meese,R. A., andRogoff,K. (1988),"Wasit Real? TheExchangeRateInterestDifferentialRelationOverthe ModernFloating-Rate
Period,"
of ComparClements,M. P.,andHendry,D. T.(1993),"OntheLimitations
Journalof Forecast(withdiscussion),
ingMeanSquareForecastErrors"
ing, 12,617-676.
forMarket
TimingAbilCumby,R. E., andModest,D. M.(1987),"Testing
Journalof FinancialEcofor ForecastEvaluation,"
ity: A Framework
nomics,19, 169-189.
Outputwith
Diebold,F. X., andRudebusch,G. D. (1991), "Forecasting
the CompositeLeadingIndex: An Ex AnteAnalysis,"Journalof the
in L12,"
Mizrach,B. (1991),"Forecast
Comparison
manuscript,
unpublished
of Pennsylvania,
Wharton
School,University
Dept.of Finance.
Morgan,W. A. (1939-1940),"ATestfor Significanceof the Difference
in a SampleFroma NormalBivariatePopuBetweentheTwoVariances
lation,"Biometrika,
31, 13-19.
Newey, W., and West, K. (1987), "A Simple, PositiveSemi-Definite,
andAutocorrelation
ConsistentCovariance
Matrix,"
Heteroskedasticity
AmericanStatistical Association, 86, 603-610.
Dhrymes,P. J., Howrey,E. P.,Hymans,S. H., Kmenta,J., Leamer,E. E.,
V. (1972),
Quandt,R. E., Ramsey,J. B., Shapiro,H. T., andZarnowitz,
for Evaluationof Econometric
"Criteria
Models,"Annalsof Economic
and Social Measurement,1, 291-324.
Engel,C. (1994), "Canthe MarkovSwitchingModelForecastExchange
Rates"Journal of InternationalEconomics,36, 151-165.
JourforCommonFeatures,"
Engle,R. F, andKozicki,S. (1993),"Testing
nal of Business & EconomicStatistics, 11, 369-395.
in ForeInformation
Fair,R. C., and Shiller,R. J. (1990), "Comparing
casts From Econometric Models," American Economic Review, 80,
375-389.
Costof Error
Witha Generalized
Granger,C. W. J. (1969), "Prediction
Function,"OperationalResearchQuarterly,20, 199-207.
EconomicTime
Granger,C. W. J., andNewbold,P. (1977), Forecasting
Series,Orlando,FL:AcademicPress.
Hamilton,J. D. (1989), "ANew Approachto the EconomicAnalysisof
TimeSeriesandthe BusinessCycle,"Econometrica,
57,
Nonstationary
357-384.
Hannan,E. J. (1970),MultipleTimeSeries,NewYork:JohnWiley.
Hogg, R. V., andCraig,A. T. (1978), Introductionto MathematicalStatistics
(4thed.), NewYork:MacMillan.
M. D. (1974),"Noteson TestHowrey,E. P.,Klein,L. R., andMcCarthy,
of Econometric
Models,"International
ing the PredictivePerformance
EconomicReview, 15, 366-383.
Journal of Finance, 43, 933-948.
Econometrica,55, 703-708.
Priestley,M. B. (1981), SpectralAnalysis and TimeSeries, New York:Aca-
demicPress.
Trendin U.S. RealGNP,"
AmerG. D. (1993),"TheUncertain
Rudebusch,
ican EconomicReview,83, 264-272.
the Evidenceon
Stock,J. H., andWatson,M. W. (1989), "Interpreting
Money-IncomeCausality,"Journalof Econometrics,40, 161-181.
andCausalToda,H.Y, andPhillips,P.C.B.(1993),"Vector
Autoregression
ity,"Econometrica,61, 1367-1393.
RatioTestsforModelSelectionandNonVuong,Q.H.(1989),"Likelihood
nestedHypotheses,"
Econometrica,
57, 307-334.
Estimation
andForecasting
in Dynamic
Weiss,A. A. (1991),"Multi-step
Models,"Journalof Econometrics,48, 135-149.
TimeSeriesModelsUsingthe RelevantCost
(1994),"Estimating
Function,"
unpublished
manuscript,
Universityof SouthernCalifornia,
Dept.of Economics.
A. P.(1984),"Estimating
Models
Weiss,A. A., andAndersen,
Forecasting
Journalof theRoyal
Criterion,"
UsingtheRelevantForecastEvaluation
StatisticalSociety, Ser. A, 137, 484-487.
InferenceAbout PredictiveAbility,"
West, K. D. (1994), "Asymptotic
SSRIWorkingPaper9417, Universityof Wisconsin-Madison,
Dept.of
Economics.
West,K.D.,Edison,H.J.,andCho,D. (1993),"AUtility-Based
Comparison
of SomeModelsof ExchangeRateVolatility,"
Journalof International
Economics,35, 23-46.