A Forecast Rationality Test that Allows for Loss Function Asymmetries
Andrea A. Naghi
Department of Economics, University of Warwick
I
I
I
I
I
Forecast rationality is tested under different assumptions regarding the
forecaster’s underlying loss function
Instrument
A symmetric loss function may not be flexible enough to capture the loss
structures that forecasters face. An asymmetric loss function, could be more
representative for the forecaster’s intentions; see Zellner (1986), Christoffersen
and Diebold (1997), Elliott, Komunjer and Timmermann (2005, 2008; EKT
hereafter), Patton and Timmermann (2007a), Komunjer and Owyang (2007)
R, P
R=250, P=150
R=250, P=200
R=250, P=250
R=300, P=200
EKT provide a forecast rationality testing framework based on a general class of
loss functions that allows for a parametrization of the asymmetry in the loss
function
a0=-5
0.0482
0.0494
0.0509
0.0482
a0=-3
0.1326
0.1331
0.1346
0.1321
a0=-1
0.3461
0.3470
0.3488
0.3484
a0 = 3
0.8621
0.8606
0.8608
0.8647
a0=1
0.6532
0.6540
0.6516
0.6522
a0 = 5
0.7769
0.7769
0.7751
0.7745
However, as I show in this paper, their methodology is loss function sensitive.
Also, the EKT test is based on the assumption that the forecasts were generated
using a linear model. Thus, their test may not detect nonlinear dependencies
This paper proposes an new test for forecast rationality that allows for an
asymmetric loss function, relaxes the assumption that the forecaster’s loss
belongs to the parametrization of EKT and allows for nonlinear dependencies
between the forecast error and the information set
a0=-5
0.0870
0.0930
0.1010
0.0870
a0=-3
0.0610
0.0580
0.0710
0.0670
a0=-1
0.0360
0.0450
0.0320
0.0380
a0 = 3
0.2130
0.2950
0.3260
0.3070
a0=1
0.0510
0.0400
0.0310
0.0420
a0 = 5
0.3950
0.5130
0.5890
0.4880
Empirical Size and Power Comparison
Linex and EKT-Loss Graphs
3
J-Stat
g(x) = x
The EKT Framework
-4
-3
-2
-1
General Class of Loss Functions:
MT Stat
2
0
1
2
3
4
δ = 0.2
δ = 0.5
δ=1
δ=2
5
-1
p
L1 (t+h; p, α) = [α + (1 − 2α) · 1(t+h < 0)] · |t+h|
-2
0.186
0.244
0.242
0.310
0.436
0.662
0.592
0.800
The shape parameter α describes the degree of asymmetry in the forecaster’s
loss function


0
TX
+τ −1
1
p0−1
−1
vt [1(êt+1 < 0) − α̂T ]|êt+1|
J=
Ŝ
T
t=τ


TX
+τ −1
p0−1
2

vt [1(êt+1 < 0) − α̂T ]|êt+1|
∼ χd−1
Figure 1: Red L2(t+1; a) = exp(a · t+1) − a · t+1 − 1
with a = 1
Blue: L1(t+1; p, α) = [α + (1 − 2α) · 1(t+1 ≤ 0)] · |t+1|p
with p = 2, α = 0.65
3
g(x) = arctan(x)
1
-5
-4
-3
-2
-1
0
1
2
3
4
5
-1
New Forecast Rationality Test
0.4630
0.4759
0.4651
0.4715
0.4683
0.0488
0.0484
0.0483
0.0483
0.0483
-0.7586
-0.4966
-0.7216
-0.5890
-0.6570
1.2149
0.5967
0.2062
0.0640
1.6501
2.71
2.71
2.71
2.71
4.60
Case 1
Case 2
Case 3
Case 4
Case 5
Consumption
Case 1
Case 2
Case 3
Case 4
Case 5
0.5541
0.5676
0.5626
0.5610
0.5621
0.0471
0.0463
0.0460
0.0459
0.0458
1.1480
1.4601
1.3622
1.3312
1.3549
0.4478
2.6596
0.9246
0.6171
0.9456
2.71
2.71
2.71
2.71
4.60
0.2760
0.3075
0.2815
0.3057
0.2732
0.0502
0.0522
0.0503
0.0519
0.0498
-4.4603
-3.6884
-4.3453
-3.7425
-4.5532
5.9011
0.7563
4.1666
0.3205
6.0723
2.71
2.71
2.71
2.71
4.60
MT Test for Forecast Rationality Based on Median Forecasts
Info Set
δ = 0.2
δ = 0.5
δ=1
δ=2
0.296
0.302
0.298
0.320
0.610
0.724
0.762
0.812
δ = 0.2
δ = 0.5
δ=1
δ=2
0.290
0.324
0.362
0.430
0.616
0.736
0.856
0.816
δ=0
0.104
0.116
-2
Idea: asymmetric preferences imply an unconditional bias of the forecast error
but not a conditional bias
H0 : E(t+1|Wt ) = E(t+1)
H1 : E(t+1|Wt ) 6= E(t+1)
H0 : E[(t+1 − E(t+1))|Wt ] = 0
H1 : Pr [E[(t+1 − E(t+1))|Wt ] = 0] < 1
Conditional moment type test in the spirit of Bierens (1982,1990), de Jong
(1996):
MT = supγ∈Γ|mT (γ)|
mT (γ) =
√1
T
t=0
(êt+1 − e) w
P
t−1 0
j=0 γj Φ(Wt−j )
2014 NBER-NSF Time Series Conference September 26-27, 2014
Figure 2: Red: L2(t+1; a) = exp(a · t+1) − a · t+1 − 1
with a = 3
Blue: L1(t+1; p, α) = [α + (1 − 2α) · 1(t+1 ≤ 0)] · |t+1|p
with p = 2, α = 0.86
Test Stat
BootCV at 5%
BootCV at 10%
Case 1
Case 2
Case 3
Case 4
Case 5
0.0229
0.0224
0.0200
0.0164
0.0235
0.0581
0.0439
0.0457
0.0554
0.0470
0.0466
0.0439
0.0391
0.0493
0.0370
Case 1
Case 2
Case 3
Case 4
Case 5
0.0316
0.0321
0.0322
0.0336
0.0308
0.0340
0.0257
0.0419
0.0319
0.0315
0.0303
0.0219
0.0302
0.0281
0.0305
Case 1
Case 2
Case 3
Case 4
Case 5
0.0152
0.0114
0.0171
0.0123
0.0220
0.0378
0.0397
0.0512
0.0410
0.0540
0.0338
0.0347
0.0400
0.0377
0.0471
Price Index
-3
PT −1
CV at 10%
Case 1
Case 2
Case 3
Case 4
Case 5
g(x) = exp(x)
2
t=τ
I
J-Stat
GNP/GDP
-3
I
t-Stat
1
-5
I
StdErr
Price Index
Table 2: Rejection frequencies for the J-test, forecast evaluation under a misspecified loss
R/ P
R=250, P=150
R=250, P=200
R=250, P=250
R=300, P=200
α
GNP/GDP
Table 1: Average GMM estimates for α across 1000 MC, obtained under the misspecified loss
2
I
J Test for Forecast Rationality Based on Median Forecasts
The Effect of a Misspecified Loss Function in Forecast Evaluation
Introduction and Motivation
Consumption
0
0
DGP : Yt = θ Wt + δg(φ Wt ) + Ut
T = 500, M = 1000
Concluding Remarks
New forecast rationality test that allows for asymmetric preferences, without the necessity to
assume any particular functional form for the forecaster’s loss function. It is a conditional
moment type test in the spirit of Bierens (1982,1990)
I Monte Carlo evidence shows that the forecast rationality test of EKT (2005, 2008) is loss
function sensitive. The proposed test has higher power than the J-test in the presence of
nonlinear dependencies between the forecast error and the info. set used in forecasting
I Empirical application using data from the Survey of Professional Forecasters (SPF)
I
Email: a.a.naghi@warwick.ac.uk
Acknowledgments
Part of this paper was written while the author was visiting the UCSD Economics Department,
whose warm hospitality is gratefully acknowledged. I would like to thank Michael Clements,
Valentina Corradi, Ivana Komunjer and Jeremy Smith for their helpful comments and
suggestions. Financial support from the ESRC is thankfully acknowledged.
Web: http://www.wawick.ac.uk/anaghi