Threshold volatility models: forecasting
performance
Márquez M.D.1 , Muñoz M.P.2 , Martı́-Recober M.2 , and Acosta L.M.2
1
2
Department of Business Economics, Univ. Autònoma de Barcelona, Barcelona,
Spain, mariadolores.marquez@uab.es
Department of Statistics and Operational Research, Univ. Politècnica de
Catalunya, Barcelona, Spain,
{pilar.munyoz,manuel.marti-recober,lesly.acosta}@upc.edu
Summary. The aim of this paper is to compare the forecasting performance of competing volatility models, in order to capture the asymmetric effect in the volatility.
We focus on examining the relative out-of-sample forecasting ability of the models
(SETAR-TGARCH and SETAR-THSV), which contain the introduction of regimes
based on thresholds in the mean equation and volatility equation, compared to the
GARCH model and SV model. For each model, we consider two cases: Gaussian and
t-Student measurement noise distribution. An important problem when evaluating
the predictive ability of volatility models is that the “true” underlying process is
not observable and thus a proxy must be defined for the unobservable volatility. To
attain our proposal, the proxy volatility measure and the loss function must also be
decided to ensure a correct ranking of models.
Our empirical application suggests the following results: when time series include
leverage effects on the mean, the introduction of threshold in the mean and variance
equations produces more accurate predictions. If the leverage in the mean is not
important, then the SVt is flexible enough to beat the threshold models.
Key words: Volatility, Threshold Models, Bayesian estimation
1 Aims and Background
Volatility is understood as a measure of the random component of a return time
series. Modelling and forecasting volatility is crucial in financial markets (option
pricing, risk management and portfolio management). Models that try to explain the
volatility of return series have become exceedingly complex in recent years. However,
volatility models are not merely theoretical constructs; the success of these models
depend on their capability to capture and predict the features of empirical data.
Many authors now focus their interest on the study of volatility forecasting (see
Patton [Pat05] and the references therein).
Threshold models [Ton90] are an interesting alternative for modelling both returns and volatilities. The fundamental idea behind these models is the introduction
1542
Márquez M.D., Muñoz M.P., Martı́-Recober M., and Acosta L.M.
of regimes based on thresholds, thus allowing the analysis of complex stochastic
systems from simple subsystems. In this group, the self-exciting threshold autoregressive model (SETAR) is defined on the basis of an autoregressive (AR) process
in each regimen, and the variable that governs the switching regime is the delayed
process variable. Cao and Tsay [CT92] showed that this new kind of models provided
better forecasts than the previous family of conditional heteroscedastic models.
Li and Li [LL96] and Chan [Cha05] propose to combine SETAR models with
Threshold GARCH (SETAR-TGARCH) to explain nonlinearity in terms of the mean
and variance. So et al. [SLL02] combine SETAR models with Threshold SV model3 .
These authors also study the forecasting performance of those models.
The aim of this paper is to compare the forecasting performance of competing models. We focus on examining the relative forecasting ability of the SETARTGARCH model and the SETAR-THSV model compared to the GARCH model and
SV model, with the main problem consisting of evaluating the predictive ability of
volatility models being that the “true” underlying volatility process is not observable
and a proxy must be defined for the unobservable volatility. To attain our proposal,
the proxy volatility measure and the loss function must also be decided to ensure a
correct ranking of models.
This paper is organized as follows: Section 2 presents the SETAR-TGARCH and
SETAR-THSV models. In Section 3, we discuss the choice of the volatility proxy
and loss function. Section 4 focuses on empirical findings, and finally in Section 5
we present the concluding remarks.
2 Models
The conditional heteroscedastic models (GARCH (1,1), SV) are the non-linear time
series models most commonly used in the finance literature. Generally, these two
models cannot capture the asymmetry of volatility [ANV99]. This characteristic,
known as the leverage effect, is related to the asymmetric behavior of the market, in
the sense that it is more volatile after a continuous decrease in prices than after a rise
(both of the same magnitude). The non-linear SETAR models, however, allow the
asymmetries in the mean (non-linearity in the mean of the returns) to be captured.
The former models belong to the class of “first-generation models”.
To capture the asymmetric effect in the volatility, the introduction of thresholds
in the volatility equation has been proposed, thus obtaining the Threshold GARCH
(TGARCH) model or Threshold Stochastic Volatility (THSV) model. If we try to
explain both kinds of non-linearity, in terms of the mean and in variance, the SETAR
model is combined with the previous ones, obtaining the SETAR-TGARCH model
and the SETAR-THSV model, which are known as “second-generation models” (see
Table 3).
These second-generation models capture the main features of volatility. The
difference is that the SETAR-THSV is more flexible than the SETAR-TGARCH to
capture kurtosis though the estimation procedure is computationally more costly;
see Appendix A for a complete model formulation.
3
SV: Stochastic Volatility; THSV: Threshold Stochastic Volatility; TGARCH:
Threshold GARCH.
Threshold volatility models: forecasting performance
1543
2.1 Parameters estimation
In this work, the identification and estimation of the SETAR model was done using
the methodology proposed by Tsay [Tsa89] and the algorithm designed by Márquez
(2002) [Mar02] that improves the identification and estimation processes. This algorithm was developed in Fortran 77 and the estimation is based on the conditional
least-squares whereas the model order selection is done automatically minimizing
the AIC (Akaike Information Criteria). The parameters estimation for the GARCH,
TGARCH and SETAR-TGARCH models was based on maximizing the conditional
likelihood function with BFGS algorithm.
The SV model and other models based on it were formulated into a state-space
representation. To estimate the unknown parameters we choose to resort to Bayesian
estimation because the likelihood function cannot be expressed in closed form. An
usual approach is to introduce the unknown parameters as part of the state vector
(xt , θ)′ and then to obtain the a posteriori PDF p(xt , θ|Dt )4 as a new observation yt
arrives. Herein we estimate this a posteriori PDF using our modified version of the
SIR (Sampling Importance Resampling) filter described in Muñoz et al. [MMM04];
this procedure avoids the use of optimization algorithms. We developed a specific
code in the R language (http://cran.r-project.org) for the models exposed in
Appendix A.
For all models, a simulation study was carried out using as true parameters values
those already published in the literature. This is done in order to assess the accuracy
and stability of our implemented algorithms and thus guarantying the quality of our
results.
3 Comparison of different volatility models
In this section, we focus on the procedure for comparing conditional volatility forecasts, based on three steps:
Step 1. First of all, as volatility is an unobservable characteristic of the return
time series, we need to choose a proxy for it. Several proxies are used in the
financial literature as a proxy of σt2 , for instance: absolute value of returns
|rt |, realized volatility (which is the sum of squared intradaily returns), and
squared returns rt2 . All of them exhibit advantages and disadvantages [AB98,
HL06, Pat05, Poo05]. We choose to use one of the volatility proxies proposed in
the Risk Metrics Manual 1995, the so called rolling daily volatility (RDV). The
RDV is defined as a moving window obtained from the standard deviation of
the returns:
σ̂tRDV
v
u i=t+(k−1)/2
u
X
u1
=t
(ri − r k )2 ,
k
(1)
i=t−(k−1)/2
where k is the lag length of the rolling window, in days, and r k is the sample
mean of the k returns used.
4
Dt means the set of all the observations until time t, Dt = {y1 , ..., yt }
1544
Márquez M.D., Muñoz M.P., Martı́-Recober M., and Acosta L.M.
Step 2. Second, a loss function must be chosen among those used by several authors, in order to compare the goodness-of-fit of the one-step-ahead forecast
obtained from each one of the volatility models used in this paper to the rolling
daily volatility defined by Eq. (1).
We propose to use the following quadratic function:
2
RDV
RDV 2
L σ̂t+1
, σ̂t+1/t = (σ̂t+1
) − (σ̂t+1/t )2 ,
(2)
where σ̂t+1/t is the one-step-ahead forecast obtained from the volatility model.
Step 3. Finally, the out-of-sample predictive accuracy of the different models is
statistically tested using the Diebold and Mariano’s sign test [DM95], used
for comparing forecast errors of different models. Following Poon [Poo05], let
j
i
N
N
{σ̂t+1/t
}N
t=1 and {σ̂t+1/t }t=1 be two sets of forecasts for the volatility {σt+1 }t=1
from models i and j respectively. The loss function defined in Eq. (2) is used to
calculate the loss differential as:
RDV 2
i
dt+1 = (σ̂t+1
) − (σ̂t+1/t
)2
2
RDV 2
j
− (σ̂t+1
) − (σ̂t+1/t
)2
2
(3)
This loss differential is simply the difference between the two forecasts errors
obtained from models i and j, respectively. In this case, we test the null hypothesis:
H0 : E(dt ) = 0
vs.
E(dt ) 6= 0
(4)
Assuming that dt ∼ iid, then the test statistic is:
S=
N
X
I+ (dt )
where
I+ (dt ) =
t=1
1 if dt > 0
0 otherwise
(5)
Following Diebold and Mariano [DM95], the large sample studentized version of
an exact finite sample test, the sign test, is asymptotically normal.
S − 0.5N
∼ N (0, 1).
Sa = √
0.25N
(6)
4 Empirical results
We considered two time series:
• The IBEX 35 is the official index returns of the Spanish Stock Exchange continuous market. This data set consists of 3932 observations of IBEX and spans
from January 2, 1990 through May 10, 2005. The first 3539 observations are
used in the estimation process and 393 in the one-step-ahead forecasting.
• The S&P 500 composite index returns spans from October 10, 1990 through
December 3, 2003 (3085 observations). The first 2785 observations are used for
the parameters estimation and the last 300 for the one-step-ahead forecasting.
Threshold volatility models: forecasting performance
1545
4.1 Main features of the series
The two data sets exhibit the main features of financial returns series: heavy tails,
heteroscedasticity, volatility clusters and the leverage effect.
In order to capture the behavior of the return time series as well as the main
features of the volatility, we analyzed as in Muñoz et al. [MMM05] the following
models: GARCH, SV, SETAR-TGARCH (1, 1) and SETAR-THSV. For each model
we consider two cases, when {ut } is Gaussian or follows a standardized t-Student
distribution.
The validation of the different models is based on the standardized observations
(called residuals). These are defined as ût = rt /σ̂t , where σ̂t is the estimated volatility obtained by replacing the estimated parameters considered in the equation that
defines volatility. In particular, we use the Box-Ljung statistics of the residuals to
check the goodness-of-fit of the equation relating to the mean; then we applied the
test to the squared residuals, to validate the equation that models volatility. The
skewness and kurtosis coefficients and the residuals normal probability plot will allow
validating the assumptions about the distribution of the random noise variables. The
models that pass the validation procedure are: GARCHt, SVt, SETAR-TGARCHt
(all with t-Student innovations) and the SETAR-THSV.
4.2 Comparing forecasting performance
For each model which passes the validation procedure and for each data set, the onestep-ahead is calculated, then the rolling daily volatility (RDV) using Eq. (1). We
consider k = 11, roughly half a month of activity. To choose the k value is a delicate
matter, because a large value smoothes the volatility series too much whereas a small
value introduces too much noise. In fact, we only decided on this value after several
tests Next, the loss differential is calculated from Eq. (3) and finally, the sign test
is done (see Eq. (6)). The results for this test are:
Table 1. Diebold’s and Mariano’s sign test (z-values)
IBEX35
SETAR- SETARSVt TGARCHt THSV
S&P500
GARCHt 7.31
SVt
SETARTGARCHt
6.80
4.041
5.77
3.63
2.09
GARCHt
SVt
SETARTGARCHt
SETAR- SETARSVt TGARCHt THSV
7.28
-3.76
-8.57
2.82
-2.11
6.34
Table 2 shows that all the values are significant at the 0.95 confidence level, indicating that all the models are not equivalent with regard to forecasting performance.
Once the estimated models are proven to be statistically different, we then select
the model that best fits our series. In order to select the “best” model as the model
which gives the most accurate forecasts, we use the mean square error between the
rolling window volatility and the one-step-ahead forecast, for each model i:
1546
Márquez M.D., Muñoz M.P., Martı́-Recober M., and Acosta L.M.
M SE i =
N
2
1 X
RDV 2
i
(σ̂t+1
) − (σ̂t+1/t
)2
N t=1
(7)
Table 2 shows that the forecasting ability of the models for the IBEX35 is similar,
when taking into account the MSE statistics, with the best model being the SETARTHSV. For the S&P500 series, the SVt model is superior to the others.
Table 2. Ranking of models
MSE
IBEX35
S&P500
RDV/
RDV/ RDV/
RDV/
GARCHt SVt SETAR-TGARCHt SETAR-THSV
0.068
3.364
0.060
2.216
0.065
3.308
0.057
3.470
We also evaluate the forecasting performance in sample, obtaining results similar
to the out of sample. In Table 3, we show the estimates coefficients of the SETARTHSV model for the IBEX35 return series. The leverage effect in the mean is clear
in this time series. Returns display asymmetric behavior in response to negative lags
shocks. This model captures the change in the level of stochastic volatility, volatility
increases (α̂1 > 0) after negative shocks (bad news) and decreases (α̂2 < 0) after
positive shocks (good news). The model displays a high persistence β̂, which does
not switch between the regimes.
The best model for S&P 500 is the simplest one, the stochastic volatility model
with error term following a t distribution (SVt ). The leverage effect for this data
is not relevant in the return and neither in the volatility. More important is the
introduction of the heavy error distribution for the error term. The persistence
parameter β̂ is 0.99 (st.dev.: 0.003) and the estimated degrees of freedom ν̂ are 8.73
(0.82) signaling significant departures from the normal model. Moreover, α̂ = −0.01
(0.11) and σ̂η = 0.193 (0.018).
Table 3. IBEX35: Estimated coefficients of the SETAR-THSV model
SETAR
THSV
First regime
Second regime
φ̂13
φ̂18
φ̂115
φ̂24
φ̂26
φ̂28
φ̂211
φ̂213
= −0.0507 (0.0259)
=
0.0602 (0.0258)
=
0.0977 (0.0263)
T = 0.000
() Std. dev.
First regime
1
Second regime
2
= −0.0478 (0.0214) α̂ = −0.079 (0.009) α̂ = −0.037 (0.002)
= −0.0436 (0.0219)
=
0.0454 (0.0220)
=
0.0507 (0.0217)
=
0.0529 (0.0217)
β̂ = 0.962 (0.002)
σ̂η = 0.172 (0.014)
Threshold volatility models: forecasting performance
1547
5 Conclusions and future work
In this paper we have compared the forecasting performance of competing models
for two return time series: IBEX 35 and SP&5005 . We have obtained a different
best model for each case. For the IBEX 35, the introduction of threshold in the
mean and variance equations produces more accurate predictions. In the S&P500
return series, however, the leverage in the mean is not important and thus the
SVt is flexible enough to beat the threshold models. Empirical evidence applied to
eight international financial market indices, including the G-7 countries supports the
hypothesis of thresholds nonlinearity in both the mean and also in volatility [CSG05].
Our results go in this line though further research is needed.
References
[AB98]
Andersen, T.G., Bollerslev, T.: Answering the skeptics: Yes, standard
volatility models do provide accurate forecasts. Int Econ Rev, 39, 885–
905 (1998)
[ANV99] Anderson, H., Nam, K., Vahid, F. Asymmetric nonlinear smooth transition GARCH models. In: Rothman, P. (ed) Non Linear Time Series
Analysis of Economic and Financial Data, Kluwer Academic Press, 191–
207 (1999)
[Cha05] Chan, Y.H.: Bayesian comparison of Asymmetric Heterocedastic Models.
MA Thesis, Feng Chia University, Taichung, Taiwan (2005)
[CSG05] Chen, W.S., So, M.K., Gerlach, R.H.: Assessing and testing for threshold
nonlinearity in stock returns. Aust NZ J Stat 47(4), 473–488 (2005)
[CT92]
Cao, C.Q., Tsay, R.S.: Nonlinear time-series analysis of stock volatilities.
J Appl Econom, Supplement 1, S165–S185 (1992)
[DM95] Diebold, F.X., Mariano, R.: Comparing predictive accuracy. J Bus Econ
Stat, 13, 253–265 (1995)
[HL06]
Hansen, P.R., Lunde, A.: Consistent ranking of volatility models. J Econometrics, 131, 97–121 (2006)
[LL96]
Li, C.W., Li, D.K.: On a double-threshold autoregressive heterocedastic
time series models. J Appl Econom, 11, 253–274 (1996)
[Mar02] Márquez, M.D.: Modelo SETAR aplicado a la volatilidad de la rentabilidad de las acciones: algoritmos para su identificación. PhD Thesis, Universitat Politècnica de Catalunya, Barcelona, Spain (2002)
[MMM04] Muñoz, M.P., Márquez, M.D., Martı́ Recober, M., Villazón, C., Acosta,
L.:. Stochastic volatility and TAR-GARCH models: Evaluation based on
simulations and financial time series. In: Antoch, J. (Ed.) COMPSTAT
2004 - Proceedings in Computational Statistics 16th Symposium Prague,
Czech Republic, Physica-Verlag, Springer, ISBN: 3-7908-1554-3 (2004)
[MMM05] Muñoz, M.P., Márquez, M.D., Martı́ Recober, M., Villazón, C.: A walk
through non-linear models in financial series. 3rd World Conference on
Computational Statistics and Data Analysis, Limassol, Cyprus (2005)
5
The Rolling daily volatility was used as volatility proxy: an important decision
since the quality of the comparison greatly depends on the choice of the proxies
1548
Márquez M.D., Muñoz M.P., Martı́-Recober M., and Acosta L.M.
[Pat05]
[Poo05]
[SLL02]
[Ton90]
[Tsa89]
Patton, A.J.: Volatility forecast evaluation and comparison using imperfect volatility proxies. Econometric Society. 2005 World Congress, London
(2005)
Poon, S.H.: A Practical Guide to Forecasting Financial Market Volatility.
John Wiley and Sons (2005)
So, M.P., Li, W.K., Lam, K.: A threshold stochastic volatility. J Forecasting 21, 473–500 (2002)
Tong, H.: Non linear time series: A dynamical system approach. Oxford
University Press (1990)
Tsay, R.S.: Testing and modelling threshold autoregressive processes.
JASA 84, 231–240 (1989)
Model formulation
Table 4. Summary of first and second-generation models
First-generation
Return Equation
Volatility Equation
GARCH/GARCHt
rt = σt ut , ut ∼ N ID(0, 1)
or ut ∼ tv
σt2 = γ0 +
c
P
2
γi γt−i
+
i=1
d
P
2
λi σt−i
i=1
SV/SVt
rt = σt ut , ut ∼ N ID(0, 1)
or ut ∼ tv
ln(σt2 ) = α + β ln(σt2 ) + ση ηt , ηt ∼ N ID(0, 1)
Second-generation
Return Equation
Volatility Equation
8
p
P
>
1
>
φ1i rt−i + yt1 , rt−1 < T
< φ0 +
SETAR-TGARCH/
i=1
r =
q
P
SETAR-TGARCHt t >
> 2
φ2i rt−i + yt2 , rt−1 ≥ T
: φ0 +
i=1
σt2 = γ01 +
σt2 = γ02 +
c
P
i=1
f
P
i=1
2
γi1 γt−i
+
2
γi2 γt−i
+
d
P
i=1
g
P
i=1
2
λ1i σt−i
, rt−1 < T
2
λ2i σt−i
, rt−1 ≥ T
yti = σti ut , ut ∼ N ID(0, 1) or ut ∼ tv
SETAR-THSV/
SETAR-THSVt
8
p
P
>
1
>
φ1i rt−i + yt1 , rt−1 < T
< φ0 +
rt =
>
2
>
: φ0 +
i=1
q
P
i=1
φ2i rt−i + yt2 , rt−1 ≥ T
yti = σti ut , ut ∼ N ID(0, 1) or ut ∼ tv
ln(σt2 ) = α1 + β ln(σt2 ) + ση ηt , rt−1 < T
ln(σt2 ) = α2 + β ln(σt2 ) + ση ηt , rt−1 ≥ T