advertisement

CONSTRUCTION OF STATIONARY TIME SERIES VIA THE GIBBS SAMPLER WITH APPLICATION TO VOLATILITY MODELS Michael K Pitt and Stephen G Walker No 595 WARWICK ECONOMIC RESEARCH PAPERS DEPARTMENT OF ECONOMICS Construction of Stationary Time Series via the Gibbs Sampler with Application to Volatility Models MICHAEL K PITT and STEPHEN G WALKER University of Warwick and University of Bath M.K.Pitt@warwick.ac.uk s.g.walker@maths.bath.ac.uk Abstract In this paper, we provide a method for modelling stationary time series. We allow the family of marginal densities for the observations to be specied. Our approach is to construct the model with a specied marginal family and build the dependence structure around it. We show that the resulting time series is linear with a simple autocorrelation structure. In particular, we present an original application of the Gibbs sampler. We illustrate our approach by tting a model to time series count data with a marginal Poisson-gamma density. Some key words: ARCH, Exponential family, GARCH, Gibbs sampler, Filtering, Markov chains, Markov chain Monte Carlo , Stochastic Volatility 1 1 INTRODUCTION In this paper we focus on a general method for constructing stationary time series models with marginals of choice. The construction of such time series is based on the Gibbs sampler. Although principally used for Bayesian inference (see, for example, Smith & Roberts (1993)), we intend to use the Gibbs sampler to design the model. Dynamic models are frequently derived in terms of the conditional updates. Whilst this construction is appealing, the marginal density (in cases where the time series is strictly stationary) of the observations belongs to a family of densities which is typically unknown. Our construction allows this family to be specied. A key paper on the construction of stationary time series with specied marginal densities is Lawrence & Lewis (1985) and the debate about the starting point for modelling time series appears in the discussion of Lawrence & Lewis (1985). There is also an interesting discussion of stationary time series in Joe (1996) and in McDonald & Zucchini (1997, Chapter 1). The current state of the art in the construction of such models, from a probabilistic perspective, may be found in Jorgensen & Song (1998). In a manner similar to Lawrence & Lewis (1985), we consider strictly stationary time series with a known marginal density and with linear expectations. Statistically this has several advantages. We can assign the correct stationary density to the initial observation, leading to ecient likelihood estimation. The linear expectations also enable the autocorrelations of the series to be easily obtained. In addition, point forecasting, several steps ahead, is also possible due to the linear expectations. The marginal interpretation of the model, particularly if exogenous covariates are included, is greatly facilitated by the fact that the marginal density of our observations is known. As for standard Gaussian dynamic models, the introduction of time dependence provides a direct generalisation of the assumptions of independence whilst retaining a specied density. These advantages will be made more explicit later in the paper. In applications, there may be reasons for assuming a particular marginal density. Lawrence & Lewis (1985) consider a marginal Gamma density for wind speeds, as this appears to t the data (once detrended) well marginally. For stationary processes, this forms a coherent modelling strategy as the marginal density may be suggested by a histogram and the dynamics, suggested by the correlogram, may then be incorporated into the model. In nancial applications, the marginal density of stock and exchange rate returns is of interest. It is often argued, see for instance Campbell et al. (1997, Chapter 1), that whilst the rst few moments (including the second moment) of returns are nite, many of the higher moments may not be. In particular, Praetz (1972) and Blattberg & Gonedes (1974) both model returns as independently identically distributed arguing that the scaled Student-t distribution is appropriate as a marginal model 2 for both stock price indices and individual stock prices. This marginal density will be exploited in Sections 25 and 3. The structure of this paper is as follows. In Section 2 we derive rst-order autoregressive processes with specied marginal families. These processes may be used to directly model our observations in an analogous way to Gaussian autoregressive models. Alternatively, we may use the process to specify a subordinate or latent structure. The models therefore belong to the class of non-Gaussian state space models, see West & Harrison (1997). Within this section, we consider general autoregressive results for the exponential dispersion family, together with two examples from this family: Poisson and gamma marginals. We also consider inverse-gamma marginals. The stochastic volatility example, a non-Gaussian state space model, is considered in Section 25. In Section 3, we examine the construction of models of ARCH(1)1 type. For the changing variance model, we obtain a closed form update f (yt+1 jyt ). The model is linear in yt2 and the marginal density of y is a scaled Student-t distribution. Other models of ARCH(1) structure are examined and a general result for the exponential family is established in Section 33. Section 4 deals with models of GARCH(1,1) type, allowing direct feedback of the observations from the preceding time step. Finally, in Section 5, a non-Gaussian state space model is proposed for modelling a series of dependent counts. This model, introduced in Section 2, is estimated and assessed using ecient Markov chain Monte Carlo sampling and recent particle ltering methods respectively. Finally we conclude in Section 6. Throughout this paper, we provide an innovative approach to introducing strictly stationary processes. Some of these processes, such as the Poisson process of Section 22 are known but most, to our knowledge, are not. In particular, we attempt to unify the derivation of such models. Before considering the rst-order autoregressive processes of Section 2, we outline our approach in the following section, Section 11, by examining some general properties of the Gibbs sampler. In particular, we show the standard Gaussian AR(1) process may be derived via the Gibbs sampler. 11 Gibbs sampler In this paper we use the Gibbs sampler as a tool to construct models. The Gibbs sampler has been widely used in recent years for Bayesian inference, the literature beginning with Gelfand & Smith (1990). Reviews of the Gibbs sampler may be found in Tierney (1994), Chib & Greenberg (1995) and Gilks et al. (1996). Suppose we wish to generate observations y1 ; y2 ; : : : from a Markov chain which starts in 1 Autoregressive conditional heteroskedastic models, see Engle (1982). 3 equilibrium and has stationary density fY (y). In order to do this we introduce an auxiliary variable x and consider the joint density fX;Y (x; y) = fY (y)fX jY (xjy): Clearly the marginal density for y is fY (y). Also, (x; y) : fY jX (yjx) = fX;Y f (x) X If we generate y1 fY (y), then x1 fX jY (xjy1 ), y2 fY jX (yjx1 ) and so on, then f(xt ; yt )g will yield a stationary sequence from fX;Y (x; y). In particular, fyt g will form a stationary Markov sequence from fY (y). The sequence of sampling recursively from each conditional density is known as the Gibbs sampler. It may be seen that the transition density of our Markov chain for y is Z f (yt+1 jyt ) = fY jX (yt+1 jx)fX jY (xjyt ) dx: In this paper, we construct a Markov chain for yt which leads to plausible time series models, which may then be estimated and tested. We are regarding the yt as observations. The \auxiliary" variable x plays no role in the model, except as a tool for obtaining a transition density f (yt+1 jyt), which ensures a marginal density of a specied form. The usual properties of the Gibbs sampler remain; in particular, the Markov chain is reversible. As a simple example, let us consider the Gaussian density for our marginal. Thus we wish to form a Markov chain yt with stationary density fY (y ) = N (y; ; 2 ). We introduce an auxiliary variable x and let fX jY (xjy) = N (x; y; 1). Thus fX;Y (x; y) = fY (y) fX jY (xjy) = N (y; ; 2 ) N (x; y; 1): Therefore, (11) fY jX (yjx) = N y; x= + (1 ; ); =2 ; where = 2 2 =(1 + 2 2 ). If we consider a Gibbs sampler generating y1 fY (y), x1 fX jY (xjy1 ), y2 fY jX (yjx1 ) and so on, then the transition density for y is f (yt+1 jyt ) = Z fY jX (yt+1jx)fX jY (xjyt ) dx = N yt+1 ; (1 ; ) + yt ; (1 ; 2 )2 : That is, yt+1 = (1 ; ) + yt + "t ; 4 (12) ; ; where "t N 0; (1 ; 2 )2 and y1 N ; 2 . This is clearly the standard Gaussian AR(1) process. In this case, our approach of constructing a stationary Markov chain for a Gaussian target density is not necessary. However, the construction of stationary autoregressive processes without the use of the Gibbs sampler would be dicult for many of the examples we consider later on. 2 AUTOREGRESSIVE PROCESSES In this section, we present some examples of the construction of stationary Markov chains via the Gibbs sampler. We consider models of the form yt+1 = (1 ; ) + yt + "t ; (21) where "t is a martingale dierence sequence. We show that the model in (21) may be used to generate any distribution which is a member of the exponential family. The general approach for the exponential dispersion family is discussed in Section 21. Two examples which lie in this family, the Poisson and Gamma, are then considered. We also focus on the inverse Gamma marginal in Section 24. 21 Exponential dispersion family Recently, Joe (1996) developed and constructed an AR(1) model with marginal distribution a member of the innitely divisible convolution-closed exponential family (see also Jorgensen & Song (1998)). Here we demonstrate the construction has a Gibbs sampler representation. Consider the exponential dispersion model with density function fY (y; ; ) = c(y; ) expfy ; ()g; where y 2 IR and 2 IR+ . We introduce an auxiliary variable x, as in (11), and dene the joint density fX;Y (x; y) = fY (y; ; )fX jY (x; 1 ; 2 ; y) ; where fX jY (x; 1 ; 2 ; y) = c(x; 1c)(cy(y; ;) x; 2 ) and 1 = , 2 = (1 ; ) with 0 < < 1. The full conditional for fX jY (j) is given above, and we write a random variable from this conditional density as A(y; ), which is a `thinned' variable (see Joe (1996); Jorgensen & Song (1998), for further details). The full conditional for fY jX (j) can be obtained from the constructive form y =x+ 5 where , independent of x, has density function fY (; ; (1 ; )). Consequently, the Gibbs sampler can be constructed via yt = At (yt;1 ; ) + t ; which is precisely the construction of Joe (1996). According to Joe (1996), EfAt (y; )g = y so E(yt+1 jyt ) = yt + (1 ; )0 (); and the autocorrelation function is given by r . In this respect, it is seen that our approach for constructing rst-order stationary processes based on the Gibbs sampler generalises the approach of Joe (1996). 22 Poisson marginals Here we consider the joint density, through the introduction of an auxiliary variable x, given by fY;X (y; x) = fY (y )fX jY (xjy ) = Po(y; ) bi(x; y; p); where bi represents the binomial distribution and Po the Poisson distribution. Hence, constructively, yt+1 = x +(p), where p = 1 ; p and () denotes a Poisson random variable with mean . Hence, if B (n; p) denotes a binomial random variable with parameters n and p, then yt+1 = B (yt ; p) + (p): This model has an easy to interpret form as a \simple-death with immigration" process. If the population at time t is yt , the population at time t +1 is the sum of the remaining alive members of the population from yt , that is B (yt ; p), and the new members who survive a unit of time, which is the (p) random variable. 23 Gamma marginals Here we look for a stationary sequence with a gamma marginal distribution, say ga(y; ; ). We consider the joint density fY;X (y; x) = fY (y)fX jY (xjy ) = ga(y; ; ) Po(x; y): Then yt+1 ga( + (yt ); + ); 6 where, again, () denotes a Poisson random variable with mean . Therefore, E(yt+1 jyt ) = (1 ; ) + yt ; where = =( + ) and = = . It is immediate that the autocorrelation function of yt at lag is . 24 Inverse Gamma marginals Here we consider a stationary sequence with an inverse-gamma marginal distribution, Ig(y; ; ). Note, this is not a member of the exponential family. Nevertheless, we can introduce a conjugate style auxiliary variable, x. Consider the joint density, fY;X (y; x) = fY (y)fX jY (xjy ) = Ig(y; ; ) ga(x; ; y;1 ): We obtain fY jX (yt+1 jx) = Ig(yt+1 ; + ; + x) and hence t E(yt+1 jyt ) = ++y ; 1 = (1 ; ) + yt; where = =( + ; 1) and = =( ; 1). and is the mean of the marginal distribution. The autocorrelation function of yt is provided by ( ) = . Consequently, we have a rst order autoregressive model which is linear in expectation and has inverse-gamma marginals. This provides a useful evolution equation for the variance in a stochastic volatility model. 25 State space form It is possible to use our autoregressive models as latent or subordinated processes for the evolution of a parameter. These models are therefore of a state space type, see for example West & Harrison (1997) and Harvey (1993). Examples of such models include the stochastic volatility model, see Shephard (1996), and the conditional Poisson models considered by Durbin & Koopman (1997). Specically, we are considering models of the type yt f (yt jst ) st+1 f (st+1 jst ); so that the observation yt is independent conditional upon the corresponding state st . Here our notation has changed. We keep y as the observation variable. Now s is the state parameter, is unobserved, and for the models we consider, is a Markov process of the kind considered in Section 2.1, where it was labelled y. We shall illustrate by examining the construction of a 7 stochastic volatility model. In Section 5, we consider the estimation of a state space model for count data. The discrete time stochastic volatility model, see Jacquier et al. (1994) and Shephard (1996), has been used to model the clustering of variance over time for returns. The model assumes that the log variance follows a Gaussian autoregressive process. A notable exception to this assumption in continuous time is given by Barndor-Nielsen & Shephard (2001). The implicit assumption of the standard model is that marginally the variance is log-normally distributed. The competing GARCH models have linear dynamics in the variance but the stationary density is unknown. To illustrate the latent models obtainable from constructions described in Section 2, let us consider the volatility model, writing s = 2 , yt N (0; t2 ) t2+1 = (1 ; ) + t2 + t ; where the evolution for t2 is as dened in Section 24. In fact, we use a special case of inversegamma model, Ig(=2; 2 =2). The point here is that marginally yt t , a scaled Student-t distribution. Since yt N (0; t2 ); we have yt2 = t2 "2t , where "2t 21 . Thus yt2 = t2 + vt ; where vt = yt2 ; t2 = t2 ("2t ; 1)2 . Consequently, we may obtain the autocorrelation function of yt2 , see Harvey (1993, page 31), as ( ) = (1 + 2 =2 ) ; = 2=( + 2 ; 2), 12 = 2 4 =f( V ; 2)2 ( 1 where ; 4)g and V2 = 2 4 =f( ; 4)( ; 2)g, leading to ( ) = =( ; 1), provided > 4, the requirement for the variance of t2 to exist. The autocorrelation function clearly has an interpretable form. As the degrees of freedom ! 1, thinner tails (until yt is marginally Gaussian), becomes smaller, as does the baseline multiplier ( ; 1);1 . As becomes smaller, the baseline autocorrelation becomes larger. The autocorrelation function in y2 is equivalent to an ARMA(1,1) model. This is the same as for the highly successful GARCH(1,1) model. However, our model is more akin to the stochastic volatility models, in the sense that we have a separate noise term in the evolution of the variance. In the stochastic volatility (1) model, the marginal distribution however is a mixture of a normal with the log-variance being normal. 26 Estimation The estimation of the autoregressive processes is not particularly dicult from a Bayesian perspective, since we can sample the auxiliary variables given the parameters and the data. Then we can sample the parameters given both the auxiliary variables and the data. Hence, we can us 8 Markov chain Monte Carlo methods for sampling from posterior distributions. The estimation methods for non-Gaussian state space models, however, are non-trivial. The posterior correlation between the states and the parameters can cause Markov chain Monte Carlo methods to mix poorly. Blocking methods, such as those used in Shephard & Pitt (1997), which sample large parts of the state space simultaneously can, to some extent, circumvent such problems. For example, in the stochastic volatility case is Section 2.3, we would wish to sample from the conditional posterior of a block of states, say f (t ; : : : ; t+k jt;1 ; t+k+1 ; ; yt ; : : : ; yt+k ), where = (; ; )0 : In Shephard & Pitt (1997), a Gaussian proposal formed by taking a second order Taylor expansion around the mode of this conditional posterior is used a an independence Metropolis-Hastings proposal. However, due to the fact that the state evolution is no longer Gaussian in the models we are now considering, it is unlikely that large blocks would be accepted frequently. The classical estimation approach via importance sampling (Durbin & Koopman (1997)) would encounter similar problems. Therefore, we employ a dierent Markov chain Monte Carlo strategy in this paper. This is illustrated in Section 5 for a count data which arises from a Poisson distribution whose intensity parameter is determined by a gamma autoregression. 3 ARCH(1)-TYPE MODELS Here we consider the ARCH(1) class of models, rst introduced for changing variance by Engle (1982). It is generally assumed that the conditional distribution of the measurement, y, is known, conditional on the parameter , for all time points t, say f (yt jt ). We consider a Markov chain of the type shown by the top graph of Figure 1, where 2 replaces . Recent ideas concerning Markov chains of ARCH type, particularly for modelling volatility appear in Barndor-Nielsen (1997) and Vidoni (1998). In neither of these cases is there a known stationary form for the evolution. The simplest ARCH model is the ARCH(1). This has an explicit conditional density update, given by yt+1 N (0; a + byt2 ); a > 0; b 0; the parameter constraints ensuring that the variance remains positive. This can be considered in two stages, where yt N (0; t2 ) and t2+1 = a + byt2 . This updating allows the dependence representation shown by the top graph of Figure 1. However, it should be emphasised that the update for t2+1 jyt is deterministic rather than stochastic. The attractive statistical features of the ARCH class of models is that the likelihood is available directly via the prediction decomposition. Estimation and testing are therefore quite straightforward. The stationary density for the ARCH(1) model, when it exists, is unknown. Nelson (1990) showed that the condition necessary 9 for covariance stationarity for the ARCH(1) model is b < 3:5622. The ARCH class is generally thought not to allow for sucient kurtosis to adequately model the marginal density of nancial data; see, for example, Bollerslev (1987), and the discussion of Shephard (1996, Section 1.2.2). Bollerslev (1987) notes from the literature that \the general conclusion to emerge from most of these studies is that speculative price changes : : : are well described by a unimodal symmetric distribution with fatter tails than the normal". In particular, Praetz (1972) and Blattberg & Gonedes (1974), argue that the scaled Student-t distribution is appropriate as a marginal model for both stock price indices and individual stock prices. Bollerslev (1987) introduced a Studentt for the conditional density of the ARCH model. However, this of course, does not lead to a marginal Student-t distribution. The ARCH(1) volatility model provides the rst example of our approach for a class of models for which the conditional density f (yt jt ) is regarded as known. After consideration of a duration model of this type, we shall, in Section 33, provide a version for the exponential family. In the general form of the ARCH(1) model we consider here we have our observation yt depending only upon a time varying parameter st = t . We now form our model in a similar manner to the methodology of Section 11. We consider the joint density General approach f;Y (; y) = fY j (yj)f(): In this case, the marginal density for y from this joint density is simply fY (y), where fY (y) = Z f (yj)f ()d: Y j Let us suppose we also have the form of conditional density, f (yj)f () fjY (jy) = Y j f (y) : Y As in Section 11 we consider the Gibbs sampler approach as generating y0 fY (y), 1 fjY (jy0 ), y1 fY j(yj1 ) and so on. Then f(t ; yt )g will yield a stationary sequence from f;Y (; y). In particular fyt g will yield a stationary sequence from fY (y). This principle forms the basis of the following results. 31 Volatility model We consider the joint density f (y; 2 ) = f (yj2 ) f (2 ) = N (y; 0; 2 ) Ig(2 ; =2; 2 =2): 10 Marginally, fY (y) = t (y; 0; 2 ), a scaled Student-t distribution. We also have the conditional density f (2 jy ) = Ig(2 ; =2 + 1=2; 2 =2 + y2 =2). Following the Gibbs sampler construction, if 02 Ig(=2; 2 =2) and then we cycle through the two full conditionals, for t = 1; : : : ; n, we have constructed a stationary process for yt , the unconditional density being a scaled Student-t; that is, marginally yt = t . This Gibbs sequence can itself be viewed as a model with a feedback interpretation2 . Since f: : : ; yt ; t2 ; yt+1 ; : : :g is a Markov chain, we can integrate t2 out to calculate the transition density Z f (yt+1 jyt) = f (yt+1 jt2 )f (t2jyt ) dt2 ; obtaining s 2 2 yt+1 = yt1++ (31) s +1 ; where s +1 t +1 , a Student-t random variable with + 1 degrees of freedom. In the original ARCH(1) model of Engle (1982) the s +1 random variable is replace with a standard normal random variable. Our model is therefore dierent from the ARCH(1) model of Engle (1982). However, our model arises as a restricted version of the extension provided by Bollerslev (1987). The t-ARCH(1) of Bollerslev (1987) is given by q yt+1 = a + byt2 sm ; where sm tm , a Student-t random variable with m degrees of freedom. Our t-ARCH model therefore arises by setting a = 2 =(1 + ), b = 1=(1 + ) and constraining bm = 1. So a single restriction on the model of Bollerslev (1987) provides a known, scaled t marginal density. This new model is linear in yt2 . We have 2 2 E(yt2+1 jyt2 ) = (y(t+;1) ) ; which implies the autocorrelation function of yt2 is ( ) = ( ; 1); . 32 Duration model Models for explaining times between successive trades have recently been of interest, due to the amount of data currently available on intra-daily market activity. The autoregressive conditional duration model of Engle & Russell (1998) has been introduced to explain the slowly changing mean structure of durations between trades. This work parallels the GARCH literature for changing volatility. Here we consider a Gibbs structure for constructing an observation driven model of this type. 2 See the top graph in Figure 1. This gives the dependency structure for our ARCH(1) model. Note the arrows indicate the direction of dependence. These graphs are known as directed acyclic graphs (DAGs) in the statistical community. 11 The standard assumption made in this context is that yi (the time between the ith and the (i ; 1)th trade) is exponential with parameter i . The i (as for the variance in the volatility model) is assumed to evolve over time. In the non-time varying context, it is often assumed the i arise from a gamma distribution, ga(; ) (see, for example, Lancaster (1990)). We will therefore take this as the marginal process for fi g. Now consider the joint density fY;(y; ) = fY j(yj)f() = Ep(y; ) ga(; ; ); where Ep denotes the exponential distribution. The conditional density if given y is ga( + 1; + y). The Gibbs sampler runs over the two conditional densities for i+1 and for yi+1 , iteratively. Further, it is clear that + yi E(yi+1 jyi ) = E(i;1 +1 jyi ) = ; similar to the specication of Engle & Russell (1998). We can integrate out , to obtain f (yi+1 jyi ) = Gg(yi+1 ; + 1; + yi; 1); where Gg represents the gamma-gamma distribution (see, for example, Bernardo & Smith (1994)), allowing classical inference for the two parameters and via prediction decomposition. 33 Exponential family More generally, we consider the situation when the conditional density of yt given t is a member of the exponential family; f (yt ; t) = c(yt ) exp fytt ; M (t )g ; so that E(yt jt ) = (t ) = M 0 (t ). We choose f (t ) to be a member of the standard conjugate family, see Gutierrez-Pena & Smith (1997); f (t) = exp fst ; M (t ) + (s; )g ; where > 0. Under mild regularity conditions (Diaconis & Ylvisaker (1979)), Ef(t )g = s=. Consequently, E(yt+1 jyt ) = s1 ++ yt since f (t+1 jyt) = expf(s + yt )t+1 ; (1 + )M (t+1 ) + (s + yt; 1 + )g 12 and E (yt+1 jt+1 ) = (t+1 ). It is clear that we can calculate f (yt+1 jyt ). As f: : : ; yt ; t+1 ; yt+1 ; : : :g is a Markov chain we obtain, f (yt+1 jyt ) = f (ytf+1(jt+1jy)f; y(t+1)jyt) t+1 t t+1 (yt+1 ) expf(s + yt ; 1 + )g = cexp f(s + yt + yt+1; 2 + )g and marginally fY (y) = exp f(s; )g c(yt ) expf;(s + yt ; 1 + )g. 34 Estimation Estimation for all models of the above type is reasonably straightforward via prediction decomposition, as f: : : ; yt ; yt+1 ; yt+2 ; : : :g is a Markov chain. Denoting the unknown xed parameters by , we have n Y f (y; ) = f (y1; ) f (yt jyt;1; ): t=2 The log-likelihood may be maximised with respect to via usual numerical procedures. Note that likelihood inference is made more statistically ecient as we explicitly have the initial density f (y1 j ) in the likelihood. This is a problem for ARCH and GARCH models as the unconditional density is unknown. 4 GARCH(1,1)-TYPE MODELS The GARCH(1,1), generalised ARCH, model of Bollerslev (1986), is one of the most widely used for modelling volatility. The ARMA type structure allows sucient memory for most volatility models applied to asset and exchange rate returns. There are many papers on the properties of the standard GARCH(1,1) model. A recent look at the existence of marginal moments and conditions for stationarity of the GARCH model is to be found in He & Tersvirta (1999). The conditions for strict stationarity are quite complicated and the marginal density, if one exists, is unknown. This is not the case for the models introduced in this section, which may usefully be thought of as GARCH processes with the addition of a (heteroskedastic) error term in the parameter evolution. The GARCH(1,1) model diers from the state space formulation of Section 25 in that observations can feedback directly into the prediction of the next observation. The GARCH(1,1) model can be applied not only to volatility models, of course, but also to durations, see Engle & Russell (1998), count models and so on. Indeed, the GARCH model provides a general formation for the evolution of an unobserved parameter. The standard GARCH(1,1) model for volatility, 13 Bollerslev (1986), is yt N (0; t2 ) t2+1 = a + byt2 + ct2 : Our models will dier from the above structure in that, in keeping with the spirit of this paper, the marginal density of yt and t2 will be kept xed and known. In the following subsections we consider the GARCH model for volatility, count data and the exponential family respectively. 41 Volatility model Although the models introduced here are similar to GARCH(1,1), we are not able write down f (yt+1 ; t2 ; yt ) explicitly, as we can for the standard GARCH(1,1) model. Our aim is to nd a model for which marginally yt is Student-t and, if yt conditional on t2 is N (0; t2 ), then, for some a, b, c, E(t2+1 jyt ; t2 ) = a + byt2 + ct2 : Of course, we still need t2 to be a stationary Ig(=2; =2) process. Following the development of the Gibbs sampler in Section 3, we introduce the joint density f (y; ; z) = N (y; 0; 2 ) Ig(2 ; =2; 2 =2) ga(z ; ; ;2 ); (41) where z is an auxiliary variable; making no dierence to our marginal process, but allowing longer range dependence. This GARCH model is based on a Gibbs sampler driven by the full conditionals f (yj2 ), f (z j2 ) and f (2 jy; z ). Adding time subscripts to explicitly model the process (illustrated in the middle graph of Figure 1) we update (from time t to t + 1) as f (t2+1 jyt ; zt ), f (zt+1 jt2+1 ) and f (yt+1 jt2+1 ). This satises our requirements, since we obtain 2 2 2 E t2+1 jyt ; t2 = yt + +2+;21t : We have three parameters ; and . The rst two represent aspects of the marginal density, while again represents persistence. When = 0, we obtain the same variance evolution as our t-ARCH model. As ;! 1, it is clear we obtain increased dependence. It is immediate that we have yt = t as the marginal density for observations. An approach to estimation of this model is outlined in the Appendix (Section 72). The update structure is given in the bottom graph of Figure 1. 42 Poisson model In order to show the generality of this method, let us consider a GARCH(1,1) structure for a conditional Poisson model. We start with a joint density f (y; ; z) = Po(y; ) ga(; a; b) Po(z; ); 14 so here the marginal density for y is Poisson-gamma, Pg(a; b; 1), see Bernardo & Smith (1994, page 119). Adding time subscripts to explicitly model the process (illustrated in the middle graph of Figure 1 with replacing 2 ) we consider updating (from time t to t + 1) via f (t+1 jyt ; zt ), f (zt+1 jt+1 ) and f (yt+1 jt+1 ). We obtain f (t+1 jyt ; zt ) = ga(t+1 ; yt + a + zt ; 1 + b + ); so f (t+1 jyt ; t ) follows the evolution t+1 ga(t+1 ; yt + a + (t ); 1 + b + ): Therefore, we have t E(t+1 jyt ; t ) = a 1++ytb++ : 43 Exponential family We now rely on the joint density f (yj)f ()f (zj): More generally, we consider the situation when the conditional density of yt given t is from exponential family; f (yt; t) = c(yt ) exp fyt t ; M (t )g ; so E(yt jt ) = (t ) = M 0 (t ). As previously, we choose the mixing density f (t) to be a member of the standard conjugate family, f (t) = exp fst ; M (t ) + (s; )g : For the GARCH version, we introduce f (zt; t ) = c (zt ) exp fzt t ; M (t )g : Then and therefore f (t+1 jzt ; yt ) / expf(s + yt + zt )t+1 ; (1 + + )M (t+1 )g (t ) : E(yt+1 jyt ; t ) = E Ef(t+1 )jyt ; zt g = s +1y+t + + The stochastic volatility version follows by running the Gibbs sampler over f (z; ) and drawing f (yj) at each iteration, see the bottom graph of Figure 1, replacing 2 by . In this case, we have (t ) : E(t+1 jt ) = s + + 15 5 APPLICATION: POISSON-GAMMA MODEL We shall illustrate the application of our model and the associated methods by examining the data set considered by McDonald & Zucchini (1997, page 194-195). The data consists of weekly rearm homicides in Cape Town from 1 January 1986 to 31 December 1991 (313 observations in total), see Figure 4. These models are in the form of small counts. We shall assume that the observations are Poisson, conditional upon a parameter which evolves according to the gamma autoregressive process of Section 23. Formally we have, yt Po(xt ); t = 1; : : : ; n xt+1 ga( + zt ; + ); zt Po(xt ): (51) Marginally, xt ga(; ) so the mean and variance of xt are = = and 2 = = 2 respectively. The persistence parameter is = =( + ). Of course, t E(xt+1 jxt ) = ++x = (1 ; ) + xt : We may write yt = xt + vt ; where vt = yt ; xt = (xt ) ; xt , an uncorrelated zero mean process. Therefore the autocorrelation function of yt is given by ( ) = 1 + : 51 Estimation Procedure Maximum likelihood for the model (51) is dicult due to the non-conjugate structure of the model. Quasi-maximum likelihood procedures are easier to apply. For instance, we can linearise the above system directly, obtaining, yt V2 = ; = xt + vt ; xt+1 = (1 ; ) + xt + t; 2 = 2 (1 ; 2 ); (52) where = = and 2 = = 2 . We could then treat the martingale terms vt and t as if they were independent Gaussian, N (0; V2 ) and N (0; 2 ). This would then allow maximum likelihood of the resulting state space form model to be carried out via the Kalman lter, see for example West & Harrison (1997). However, this approach would lead to biased estimation. For this reason, we choose a Bayesian Markov chain Monte Carlo approach for estimation of the parameters = (; ; )0 and the underlying state x = (x1 ; : : : ; xn )0 : We obtain stationary samples from the posterior density f (; xjy). A thorough review of Markov chain Monte Carlo is given, for example, in Gilks et al. (1996). Here we outline the main steps of the Markov chain 16 Monte Carlo algorithm, relegating the details to the Appendix (Section 7.1). We retain our auxiliary variables z = (z1 ; : : : ; zn;1 )0 in the Markov chain Monte Carlo analysis. The sampling scheme works by iterating the following steps: x f (xjz; ; y) : f (xjz; ; y) = f (x1jz1 ; ; y1 )f (xnjzn;1 ; ; yn) nY ;1 t=2 f (xtjzt ; zt;1 ; ; yt): These univariate densities may be simulated from directly as they are of gamma form, see Appendix (Section 7.1). z f (zjx; ; y) : f (z jx; ; y) = nY ;1 t=1 f (zt jxt ; xt+1 ; ): The univariate distributions may be bounded by a Poisson distribution allowing an ecient accept-reject algorithm, see Appendix (Section 7.1). f (jz; x; y) : f (jz; x; y) / f (z; xj)f () = f (x1 j) nY ;1 t=1 f (xt+1jzt ; )f (zt jxt ; )f (); where f () is the prior. This is the full conditional posterior for : A Student-t distribution is used as a proposal for each univariate element of = (; ; )0 . This proposal is accepted or rejected as an independence Metropolis-Hastings candidate, see Appendix (Section 7.1). ; x f (; xjz; y) : We have f (; xjz; y) = f (xjz; ; y)f (jz ; y) / f (yjx)f (z; xj)f (); Therefore, f (jz ; y) / f (yjxf )(fx(jz;z; x;jy))f () : This last move allows reduced conditioning, eectively integrating out x, and is necessary to permit a reasonably ecient Markov chain Monte Carlo scheme. Again, a Student-t distribution is used as a proposal within an independence Metropolis-Hastings scheme. 17 52 Results Let us denote the three parameter model given by (51) as Pga(; ; ). We shall estimate this model for the data set of weekly rearm homicides described above. However, prior to this two simple alternative models were considered. In particular a simple one parameter Poisson model, Po(), is considered together with a static Poisson model with gamma heterogeneity, Pg(; ). The maximum likelihood results for these models are given in Table 1. The diagnostics of the table will be examined in the following section. Note that there is a rise of 67:9 in the log-likelihood in going from the basic Po() model to the more general Pg(; ) distribution. As these are nested models, the usual 2 test is highly signicant. Therefore, heterogeneity (or mixing) is present. To estimate our Pga(; ; ) model we transform to (; 2 ; ). For brevity, we display the resulting samples , the mean of xt , resulting from our Markov chain Monte Carlo scheme in Figure 2. The Markov chain Monte Carlo sampler was run over 60 000 iterations. The scheme was ecient as can be seen from the, almost, independent draws of . The marginal posteriors for ( , ;)0 are given in Figure 2. The corresponding means and covariance matrix are given in Table 1. Figure 3 displays the marginal posteriors for (; 2 ; ), perhaps a more interpretable transformation. It can be seen that there is clear evidence of persistence in the time series. There is no appreciable marginal posterior mass for below 0:65, however this parameter is also not very close to 1, indicating that shocks are not highly persistent. The particle lter of Pitt & Shephard (1999) was applied to the time series. Representing Ft = (y1; : : : ; yt )0, the observations up to and including that of time t, the particle lter draws samples from the ltering density f (xt jFt ; b). We x b at the posterior mean. In Figure 4, we display the ltered mean over time together with the posterior mean under the smoothing density f (xt jFn ); obtained from the Markov chain Monte Carlo output. It can be seen from the top graph that both the ltered and smoothed estimates of the state follow the data closely. In addition, the bottom graph shows that the ltered and smoothed state means are in alignment. This is reassuring as a heuristic diagnostic since the ltered mean should be an unbiased estimate of the smoothed mean only if the model is correct. The log-likelihood for our model may be eciently estimated as a bi-product of the particle lter, see Pitt & Shephard (2000). We have an estimate of our one-step ahead prediction density, M X 1 f (yt+1jFt ; ) = M f (yt+1jxkt+1 ); b b k=1 where xkt f (xt jFt ; b) and xkt+1 f (xt+1 jxkt ; b), k = 1; : : : ; M . We may now estimate the 18 log-likelihood as, log L(b) = log f (y1; : : : ; yn jb) = nX ;1 t=0 log fb(yt+1 jFt ; b): This allows the calculation of the log-likelihood to be evaluated at the Bayesian mean b. In Table 1 we show the log-likelihood of our model. Interpreting classically, the likelihood ratio test indicates that the temporal parameter, ; is highly signicant compared with the nested Pg(; ) model, for which = 0. This conrms the Bayesian result of the importance of the time series aspect. 53 Diagnostics We now look for evidence of departures from our model. In particular, we look at tests under the null hypothesis that the model is correct, specically that our observations at time t have associated density f (yt jFt;1 ; b). We have a discrete distribution function on the random variable Yt , so F (yt ) = P (Yt yt jFt;1 ; b). We are denoting our observations as yt and the corresponding random variable as Yt . We shall suppress the conditioning upon Ft;1 ; b at this juncture since it does not aect our discussion. Suppose we sample ut from f (utjyt ) = ( 1 Pr(yt ) ; F (yt ; 1) < ut F (yt ) otherwise; 0; (53) where Pr(yt ) = F (yt ) ; F (yt ; 1). Note that the regions [F (yt ; 1), F (yt )] provide a partition of [0; 1], for yt = 0; 1; 2; : : : . Under the hypothesis that yt F (yt ), we have that f (ut) = X1 yt =0 f (ut jyt ) Pr(yt ) = 1; ut 2 [0; 1]; as we have a partition of the space of ut . Therefore, if our model, and parameters, are correct, ut U(0; 1), independently, over time. This forms the basis of our diagnostic tests. Any departure from our model, either marginally or dependence through time should be apparent by examining these residuals. We restrict ourselves to two tests. Firstly, we transform our ut via nt = ;1 (ut ). Under our null hypothesis these should be independently standard Gaussian. This allows us to perform a standard Portmanteau test for autocorrelation. Clearly, this allows us to test whether we have modelled temporal dependence adequately. Secondly, we perform a 2 test for goodness of t on the ut 's, comparing with a standard uniform. In order to sample ut we need to calculate F (yt jb). This is can be explicitly calculated for the Po() model and for the Pg(; ) distribution. Before commenting on the results, the details of estimating F (yt+1 jFt ; b) for our Pga(; ; ) time series model should be outlined. From the 19 output of the particle lter which we recall yields samples from f (xt jFt ; b); the ltering density, we have M X F (yt+1 jxkt+1 ); Fb (yt+1jFt ; b) = M1 k=1 where xkt f (xt jFt ; b) and xkt+1 f (xt+1 jxkt ; b), k = 1; : : : ; M . Noting that F (yt+1 jxkt+1 ) is just the distribution function for a Poisson, since yt Po(xt ); we arrive at ecient estimates of the prediction distribution. This allows us to sample ut via (53) for t = 1; : : : ; n. The details out of the way, we can look at the results. The last two columns of Table 1 give the goodness of t statistic for the three models and the Portmanteau statistic respectively. Corresponding to the table we have Figure 5, showing the Gaussian residuals, together with their correlogram and the standard quantile plot for the three models. It is clear from Table 1 that the simple independent Poisson model fails both tests at high signicance. The independent Poisson-gamma model fails to capture dependence but does provide a good overall t, indicated by the fact that at the 5% level the goodness of t test is passed. This suggests that the marginal model of Poisson-gamma is a sensible choice. Our three parameter Poisson-gamma time series model passes both tests with large p-values. The residuals in Figure 5 indicate hardly any evidence of correlation over time and the quantile plot is close to linear. This indicates that marginally and conditionally our model ts the data well. Poisson model Po() Par ML 2:6198 V ar 0:00837 log L ;719:36 GOF 219 (p) 77:09 (0) Q 215 (p) 281:51 (0:0) Poisson-gamma model Pg(; ) 2:1959 0:1108 0:0422 0:8382 0:0422 0:0180 ;651:5 28:40 (0:076) 216:93 (0:0) Poisson-gamma stationary model Pga(; ; ) 2:7769 0:28254 0:1006 0:5855 1:0040 0:1006 0:0460 0:17188 6:1773 0:5855 0:17188 6:3061 ;617:07 21:88 (0:290) 14:393 (0:496) Table 1: Estimation results for the rearms homicide dataset: The goodness of t statistics and the Portmanteau tests on the correlogram are reported for all three models. 20 6 CONCLUSIONS In this paper we have shown how it is possible to assign a xed marginal density to a time series model and to build the dependence structure around this. We have considered three dierent classes of models; the ARCH(1), latent AR(1) and GARCH(1,1) classes. There is a degree of exibility in this approach as we may choose which marginal family is most appropriate for the problem under consideration. The incorporation of dependence whilst retaining the marginal density clearly provides a natural generalisation over treating observations as independent. In addition, the specication of the unconditional density means that the assignment of a density for our initial observation is trivial. This can be more problematic for the observation driven models such as the GARCH(1,1) for which the unconditional density is unknown. The models obtained in this paper are all linear as the expectation of future observations is linear as a function of the previous parameters and observations. This lends interpretability to the models and allows direct evaluation of the autocorrelation. The inclusion of covariates has not been considered in this paper but is quite straightforward. In the example considered in Section 5 it was found that a simple three parameter state space model provided a very good t to the data, both marginally and temporally. The Markov chain Monte Carlo estimation procedure and the use of the particle lter provides straightforward estimation and evaluation for models of this type. 61 Acknowledgements We wish to thank a referee for constructive comments on a previous version of the paper. References Barndorff-Nielsen, O. E. (1997). Normal inverse Gaussian distributions and stochastic volatility modelling. Scand. J. Statist 24, 1{14. Barndorff-Nielsen, O. E. & Shephard, N. (2001). Non-gaussian ou based models and some of their uses in nancial economics. J. R. Statist. Soc. B forthcoming. Bernardo, J. M. & Smith, A. F. M. (1994). Bayesian Theory. John Wiley, Chichester. Blattberg, R. C. & Gonedes, N. J. (1974). A comparison of the stable and student distributions as statistical models for stock prices. J. Business 47, 244{280. Bollerslev, T. (1986). Generalised autoregressive conditional heteroskedasticity. J. Econometrics 51, 307{327. Reprinted as pp. 42{60 in Engle, R.F.(1995), ARCH: Selected Readings, Oxford: Oxford University Press. 21 Bollerslev, T. (1987). A conditional heteroskedastic time series model for speculative prices and rates of return. Rev. Economics and Statistics 69, 542{47. Campbell, J. Y., Lo, A. W., & MacKinlay, A. C. (1997). The Econometrics of Financial Markets. Princeton University Press, Princeton, New Jersey. Chib, S. & Greenberg, E. (1995). Understanding the Metropolis-Hastings algorithm. The American Statistican 49, 327{35. Diaconis, P. & Ylvisaker, D. (1979). Conjugate prior for exponential families. Ann. Statist. 7, 269{281. Durbin, J. & Koopman, S. J. (1997). Monte Carlo maximum likelihood estimation of nonGaussian state space model. Biometrika 84, 669{84. Engle, R. F. (1982). Autoregressive conditional heteroskedasticity with estimates of the variance of the United Kingdom ination. Econometrica 50, 987{1007. Reprinted as pp. 1{23 in Engle, R.F.(1995), ARCH: Selected Readings, Oxford: Oxford University Press. Engle, R. F. & Russell, J. R. (1998). Forecasting transaction rates: the autoregressive conditional duration model. Econometrica 66, 1127{1162. Gelfand, A. E. & Smith, A. F. M. (1990). Sampling-based approaches to calculating marginal densities. J. Am. Statist. Assoc. 85, 398{409. Gilks, W. K., Richardson, S., & Spiegelhalter, D. J. (1996). Markov Chain Monte Carlo in Practice. Chapman & Hall, London. Gutierrez-Pena, E. & Smith, A. F. M. (1997). Exponential and Bayesian Conjugate Families: Reviews and Extensions. Test 6, 1{90. Harvey, A. C. (1993). Time Series Models. Harvester Wheatsheaf, Hemel Hempstead, 2nd edition. He, C. & Tersvirta, T. (1999). Properties of moments of a family of garch processes. J. Econometrics 92, 173{192. Jacquier, E., Polson, N. G., & Rossi, P. E. (1994). Bayesian analysis of stochastic volatility models (with discussion). J. Business and Economic Statist. 12, 371{417. Joe, H. (1996). Time series models with univariate margins in the convolution-closed innitely divisible class. Journal of Applied Probability 33, 664{677. Jorgensen, B. & Song, P. (1998). Stationary time series models with exponential dispersion model margins. Journal of Applied Probability 35, 78{92. Lancaster, T. (1990). The Econometric Analysis of Transition Data. Cambridge University Press, Cambridge. Lawrence, A. J. & Lewis, P. A. W. (1985). Modelling and residual analysis of nonlinear 22 autoregressive time series in exponential variables. J. R. Statist. Soc. B 47, 165{202. McDonald, I. & Zucchini, W. (1997). Hidden Markov and Other Models for Discrete-valued Time Series. Chapman & Hall, London. Nelson, D. B. (1990). Stationarity and persistence in the GARCH(1,1) model. Econometric Theory 6, 318{334. Reprinted as pp. 176{192 in Engle, R.F.(1995), ARCH: Selected Readings, Oxford: Oxford University Press. Pitt, M. & Shephard, N. (2000). Auxiliary variable based particle lters. In Doucet, A., de Freitas, J. F. G., & Gordon, N., editors, Sequential Monte Carlo Methods in Practice. Cambridge University Press. Pitt, M. K. & Shephard, N. (1999). Filtering via simulation based on auxiliary particle lters. J. American Statistical Association 94, 590{599. Praetz, P. (1972). The distribution of share price changes. J. Business 45, 49{55. Ripley, B. D. (1987). Stochastic Simulation. Wiley, New York. Shephard, N. (1996). Statistical aspects of ARCH and stochastic volatility. In Cox, D. R., Hinkley, D. V., & Barndor-Nielson, O. E., editors, Time Series Models in Econometrics, Finance and Other Fields, pages 1{67. Chapman & Hall, London. Shephard, N. & Pitt, M. K. (1997). Likelihood analysis of non-Gaussian measurement time series. Biometrika 84, 653{67. Smith, A. F. M. & Roberts, G. (1993). Bayesian computations via the Gibbs sampler and related Markov Chain Monte Carlo methods. J. R. Statist. Soc. B 55, 3{23. Tierney, L. (1994). Markov Chains for exploring posterior distributions (with discussion). Ann. Statist. 21, 1701{62. Vidoni, P. (1998). Proper dispersion state space models for stochastic volatility. University of Udine, working paper. West, M. & Harrison, J. (1997). Bayesian Forecasting and Dynamic Models. SpringerVerlag, New York, 2 edition. 7 APPENDIX 71 Markov chain Monte Carlo for Poisson-gamma model. Here we describe the details of the Markov chain Monte Carlo procedure of Section 51. Recall we have yt Po(xt ); t = 1; : : : ; n; xt+1 ga( + zt ; + ); zt Po(xt ): 23 Our full data likelihood is therefore, f (y; z; xj) / f (y1jx1 )f (x1j) nY ;1 t=1 f (yt+1jxt+1 )f (xt+1 jzt ; )f (zt jxt ; ): From this the following conditional steps can be derived. Step 1: Sampling from f (xjz; ; y). We obtain the closed forms, f (xt jzt ; zt;1 ; ) = ga (xt jyt + zt + zt;1 + ; 1 + + 2) ; f (x1 jz1 ; ) = ga (x1 jyt + z1 + ; 1 + + ) ; f (xnjzn;1 ; ) = ga (xn jyn + zn;1 + ; 1 + + ) : Sampling from f (z jx; ; y). An accept-reject scheme is used for our discrete valued auxiliary variables zt . We obtain Step 2: f (zt jxt ; xt+1 ; ) / f (xt+1 jzt ; )f (ztjxt ; ): We get the simplication, log f (zt jxt ; xt+1 ; ) = c + zt log(t ) ; log(zt !) ; log ;( + zt ); where log(t ) = log(xt ) + log( + ) + log(xt+1 ): Let us denote l(zt ) = ; log ;( + zt ): Then we can form an ecient accept-reject algorithm, see Ripley (1987), as follows, noting that l(zt ) is concave (and can therefore be bounded by a straight line from above), log f (zt jxt ; xt+1 ; ) = c + zt log(t ) ; log(zt !) + l(zt ) c + zt log(t ) ; log(zt !) + l(zbt ) + l0 (zbt )(zt ; zbt ) = k ; t + zt log(t ) ; log(zt !) = k + log Po(zt ; t ); where log t = log(t ) + l0 (zbt ) and we take the expansion point as zbt = (xt + xt+1 )=2. We therefore sample from Po(t ) until acceptance with probability log Pr(accepting zt ) = l(zt ) ; l(zbt ) ; l0 (zbt )(zt ; zbt ): In practice we found that this proposal was accepted more than 99% of the time. 24 Sampling from f (jz; x; y): In order to perform parameter sampling eciently we transform from = (; ; )0 to e = (; 2 ; )0 where = ; and 2 = 2 and the persistence parameter, = + . This transformation is done to reduce the dependency between the parameters. We place our weak prior on ;e f (e). Step 3: f (ejz; x; y) / f (z; xje)f (e) = f (e)f (x1 je) nY ;1 t=1 (71) f (xt+1jzt ; e)f (zt jxt ; e): We sample the elements of e in turn drawing from f (j2 ; ; z; x; y), f (2 j; ; z; x; y) and f (j; 2 ; z; x; y), noting that each of these distributions can be derived, up to proportionality, from (71). In each case we iterate to the mode of the log of (71), recording the mode as m, and the second derivative of the log of (71) l00 (m). We propose by drawing from a t-distribution with degrees of freedom 5; mean m and variance ;1=l00 (m). We accept with probability given by the usual independence Metropolis Hastings expression. In practice, we nd that moves are frequently accepted, yielding an ecient algorithm. Sampling from f (; xjz ; y): Despite the fact that the proposed moves in the above step are frequently accepted, it is still the case that, due to high posterior correlation between the states, x and z , and the parameters, e, the resulting Markov chain Monte Carlo method mixes quite slowly. To combat this we eectively integrate out the x states and obtain samples from f (;e xjz ; y). This is possible as we know the form of f (xjz; e; y); given previously as the product of Gamma densities. We have, Step 4: f (;e xjz ; y) = f (xjz; e; y)f (ejz; y) / f (yjx)f (z; xje)f (e): Therefore, f (ejz; y) / f (yjx)f (z; exj)f () : f (xjz; ; y) We iterate through the elements of e in turn drawing from f (; xj2 ; ; z ; y), f (2 ; xj; ; z ; y) and f (; xj; 2 ; z ; y). The Metropolis scheme is the same in each case so we shall detail sampling from f (; xj2 ; ; z ; y). We have f (; xj2 ; ; z ; y) = f (xjz; ; 2 ; ; y)f (j2 ; ; z ; y). Note that f (j2 ; ; z ; y) / f (ejz; y), given above. Let us denote l() = log f (j2 ; ; z ; y). We iterate to the mode of l(), m; recording l00 (m). We sample our proposal p g(jm; v), a t-density with mean m, variance v = ;1=l00 (m) and degrees of freedom 5. We then sample xp f (xjz; ; 2 ; ; y). Letting and e 25 e x represent out current values, we accept the proposed values, p and xp with log probability, log P (; x ! p ; xp ) = l(p ) ; log g(p jm; v) ; l() + log g(jm; v). We nd that this independence Metropolis proposal accepts moves with high probability. 72 Estimation for GARCH Processes The estimation procedure for the GARCH models is aided by the conditional updating structure of the latent variables. Thus the Gibbs sampling scheme can be directly applied. We propose an Markov chain Monte Carlo procedure which allows blocking for our GARCH structure models of Section 4. Note that in this case we had f (y; ; z) = N (y; 0; 2 ) Ig(2 ; =2; 2 =2) ga(z ; ; ;2 ); from which all our conditional densities arise. In order carry out a general Markov chain Monte Carlo method we actually retain the auxiliary structure to perform estimation, rather than attempting to integrate variables out analytically. Let us denote the unknown parameters by . Then we can perform the following sweeps on a system, see middle graph in Figure 1; 1. for t = 1; : : : ; n sample zt from f (zt jt2 ; t2+1 ; yt ) / f (zt jt2 )f (t2+1 jzt ; yt ): 2. For t = 1; : : : ; n sample t2 from f (t2 jyt ; yt;1 ; zt ; zt;1 ) / f (yt jt2 )f (t2 jzt;1 ; yt;1 )f (zt jt2 ): 3. Sample from f (jy; z; 2 ) / f ()f (z1 j)f (y1 j) n Y t=2 f (yt jt2 ; )f (t2jzt;1 ; yt;1; )f (zt jt2 ; ): The dependence upon the xed parameters has been suppressed from the notation of steps (1) and (2). Note that in steps (1) and (2) we are, in fact, sampling directly from the reduced conditionals f (z j2 ) and f (2 jy; z ) where z = (z1 ; : : : ; zn )0 and 2 = (12 ; : : : ; n2 )0 . Fortunately the conditional independence structure means that this task reduces to univariate draws which are usually straightforward. Indeed the densities in (2) will usually be of standard form and the densities of (1) can generally be sampled from eciently using accept-reject sampling. The procedure in (3) may be speeded up by using the reduced conditional densities f (yt jzt ) and f (zt+1 jzt ; yt ) given for the specic forms of Section 4. In this case we use the following density for step (3); n Y f (jy; z ) / f ()f (z1j)f (y1 j) f (ytjzt ; )f (zt jzt;1 ; ): t=2 26 Note that f () is the prior for , whilst f (z1 j), f (y1 j) are the stationary densities for the rst latent variable and initial observation respectively. For large data sets a Gaussian proposal on these parameters (inside a Metropolis step) should work well as the posterior will be asymptotically Normal. 73 GARCH volatility model To illustrate how steps (1) and (2) of the above Gibbs procedure can be carried out eciently in practice we illustrate by examining the GARCH structure for volatility. In this case step (2) may be carried out directly as f (2 jyt ; yt;1 ; zt ; zt;1 ) = Ig t t2 j 12 ( + 4 + 2); 21 ( 2 + yt2 + yt2;1 + 2zt + 2zt;1 ) : Step (1) proceeds in the following manner. Let us examine the log-density log f (zt jt2 ; t2+1 ; yt ) = c + log f (zt jt2 ) + log f (t2+1 jzt ; yt ): 2 2 Letting a = 2 =2 + yt2 =2, b = + 1=2 + =2 and d = 2+2+1+1 , we have t t t t log f (zt jt2 ; t2+1 ; yt ) = c ; dzt + ( ; 1) log zt + b log(zt + a); where support is on zt 0. By changing variable to x = zt + a, x > a, we have log f (xjt2 ; t2+1 ; yt ) = c ; dx + ( ; 1) log(x ; a) + b log x c ; dx + ( ; 1) log x + b log x = k + log g(x). We may sample easily from the density g(x) which is ga(x; + b; d), truncated to be greater a, then accepting the sample x with probability exp[( ; 1)(log(x ; a) ; log x))]: This is the rejection algorithm, see Ripley (1987). We may then transform back to get zt = x ; a, yielding a direct sample from the required posterior. 27 y y 2 1 σ σ 2 1 2 2 3 y y 2 1 σ σ 2 1 z1 2 2 σ 2 3 z2 y y 2 1 σ z1 σ 2 σ 2 1 2 2 σ z2 Figure 1: Dependency structure for the ARCH(1), GARCH and Stochastic Volatility models. 28 2 3 4.5 .75 4 3.5 f( v | y ) .5 3 .25 2.5 2 0 1000 2000 3000 4000 1 2 3 4 5 6 2 .2 1.5 f( beta | y ) .15 1 f( alpha | y ) .1 .5 .05 .5 1 1.5 2 0 5 10 15 Figure 2: Top left: 60000 draws from the posterior of . Top right: The posterior of . Bottom left: The posterior of . Bottom right: The posterior of . 1.5 f( mu | y ) 1 .5 2 2.25 2.5 2.75 3 3.25 .5 3.5 3.75 4 4.25 4.5 f( sigma2 | y ) .25 1 2 3 4 5 6 .7 .75 7 8 9 10 f( rho | y ) 5 .55 .6 .65 .8 .85 .9 .95 Figure 3: Top: The posterior of . Middle: The posterior of 2 . Bottom: The posterior of . 29 15 10 5 0 25 50 75 100 125 150 175 200 225 250 275 300 0 25 50 75 100 125 150 175 200 225 250 275 300 10 7.5 5 2.5 Figure 4: Top: The number of rearm homicides per week in Cape Town from 1 January 1986 to 31 December 1991, together with (mean) smoothed and ltered estimates of the underlying Poisson parameter. Original Series: Mean=2.617, variance=6.568. Bottom: The mean of the ltered state (solid line) and the mean of the smoothed state (dashed line). Correlogram 1 5 1 .5 0 0 100 200 300 Q plot .5 0 1 10 Correlogram 20 30 1 .25 Q plot .5 .75 1 .25 Q plot .5 .75 1 .25 .5 .75 1 2.5 0 0 .5 -2.5 0 100 200 300 0 1 5 10 Correlogram 20 30 1 2.5 0 0 .5 -2.5 0 100 200 300 0 10 20 30 Figure 5: left to right, L: The nt residuals against time. M: The correlogram of the nt residuals. R: The quantile plot of the ut residuals. TOP: Po model. MIDDLE: Pg model. BOTTOM: Pga time series model. 30