Time Series Econometrics:
Some Basic Concepts
Reference : Gujarati, Chapters 21
Course Incharge:
Prof. Dr. Himayatullah Khan
Time Series Data
• One of the important and frequent types of data used in empirical
analysis.
• But it poses several challenges to econometricians/practitioners.
E.g.
1. Empirical work based on time series data assumes that the
underlying time series is stationary.
2. Autocorrelation: because the underlying time series data is
non-stationary.
3. Spurious/nonsense regression: a very high R2 and significant
regression coefficients (though there is no meaningful
relationship between the two variables)
Key Concepts
1. Stochastic Processes
i.
ii.
iii.
Stationarity Processes
Purely Random Processes
Non-stationary Processes
2. Random Walk Models
i.
ii.
3.
4.
5.
6.
Random Walk with Drift
Random Walk without Drift
Unit Root Stochostic Processes
Deterministic and Stochastic Trends
The Phenomenon of Spurious Regression
Tests of Stationarity/non-stationarity
i.
ii.
Graphical Method
Unit Root Tests
1)
Stochastic Processes
• Stochastic (Random) Process: collection of random
variables ordered in time.
o NOTATIONS: Let Y a random variable, Y(t) if continuous
(e.g. electrocardiogram), and Yt if discrete (e.g. GDP, PDI,
etc.).
o Now, If we let Y represent GDP, then we can have Y1, Y2, Y3,
... , Y20 where the subscript 1 denotes the 1st observation (i.e.
GDP for the 1st quarter of 1st year) and the subscript 20
denotes the last observation (i.e. GDP for the 4th quarter of 5th
year).
(1) Stochastic Processes
• Stationary Stochastic Processes: A stochastic process is said
to be stationary/ weakly /covariance/2nd-order stationary if:
o Its mean and variance are constant over time, and
o The value of the covariance between the two time periods depends only
on the distance/lag between the two time periods and not the actual time at
which the covariance is computed.
o E.g. let’s Yt be a stochastic process, then;
– Mean:
E(Yt ) = µ
…………………………………………..
– Variance:
var (Yt ) = E(Yt − µ)2 = σ2 ………………………………..
– Covariance:
γk = E[(Yt − µ)(Yt +k − µ)] ……………..…………
(1)
(2)
(3)
• Where γk, the covariance (or auto-covariance) at lag k,
• If k = 0, we obtain γ0, which is simply the variance of Y (= σ2); if k = 1,
γ1 is the covariance between two adjacent values of Y
Why are Stationary Time Series so
Important?
• Because if a time series is non-stationary, we can study its
behavior only for the time period under consideration, and as a
consequence, it is not possible to generalize it to other time
periods.
• Therefore, for the purpose of forecasting, such (non-stationary)
time series may be of little practical value.
• Non-stationary Stochastic Processes: Although our
interest is in stationary time series, one often encounters non-
stationary time series
• A non-stationary time series will have a time-varying mean or a
time-varying variance or both.
White Noise Processes
• We call a stochastic process(time series) purely random/white noise process
if it has zero mean, constant variance σ2, and is serially uncorrelated
i.e. [ut ∼ IIDN(0, σ2)].
• Note: Here onward, in all equations the assumption of “white noise” will be
applicable on ut .
2) Random Walk Model (RWM)
• The classic example of non-stationary time series is
the Random Walk Model (RWM).
• It is often said that asset prices, such as stock prices
or exchange rates, follow a random walk (i.e. nonstationary).
• Types of Random Walks:
a) Random Walk Without Drift:
i.e. no constant/intercept term and
b) Random Walk With Drift
i.e. a constant term is present
a) Random Walk without Drift
• The time series Yt is said to be a random walk without drift, if
Yt = Yt −1 + ut ……. (4)
• Here, the value of Y at time (t) is equal to its value at time (t − 1)
plus a random shock; thus it is an AR(1) model.
• Believers in the Efficient Capital Market Hypothesis argue
that stock prices are essentially random and therefore there is no
scope for profitable speculation in the stock market:
• If one could predict tomorrow’s price on the basis of today’s
price, we would all be millionaires.
Further Explanation
• Now from Yt = Yt −1 + ut……. (4) we can write:
Y1 = Y0 + u1
=>
Y2 = Y1 + u2 = Y0 + u1 + u2
=>
Y3 = Y2 + u3 = Y0 + u1 + u2 + u3 and so on…
• In general, if the process started at some time 0 with a value of Y0, we have:
Yt = Y0 +∑ut ……..……………………….(5)
• Therefore,
E(Yt ) = E(Y0 + ∑ut) = Y0 (why?) ………… (6)
Because, ut is “white noise”
• In like fashion, it can be shown that: var (Yt ) = tσ2 …… (7)
• Thus, the mean of Y is equal to its starting value, which is constant, but as t
increases, its variance increases indefinitely (thus violating the condition of
stationarity).
• In short, the RWM without drift is a non-stationary stochastic process.
Further Explanation (cont.)
• Now, if you write Yt = Yt −1 + ut….. (4) as
(Yt − Yt−1) = ∆Yt = ut ……. (8)
• It shows that, while Yt is non-stationary, its 1st
difference is stationary.
• In other words, the 1st differences of a random walk
time series are stationary.
b) Random Walk with Drift
• Let’s modify, Yt = Yt −1 + ut…………….(4) as follows:
Yt = δ + Yt−1 + ut….....………….(9)
where δ is the drift parameter.
• The name drift comes from the fact that if we write
the preceding equation as:
Yt − Yt−1 = ∆Yt = δ + ut ………….
(10)
• It shows that Yt drifts upward/downward, depending
on δ being positive/negative.
Further Explanation
• Note that model Yt = δ + Yt−1 + ut ………….. (9) is also an AR(1) model.
• Following the procedure discussed for Random Walk Without Drift,
it can be shown that for the random walk with drift model (9),
E(Yt ) = Y0 + t · δ ……….. (11)
var (Yt ) = tσ2 …………….... (12)
• Here, again for RWM with drift the mean as well as the variance
increases over time, again violating the conditions of stationarity.
• In short, RWM, with or without drift, is a non-stationary stochastic
process.
• The random walk model is an example of what is known in the
literature as a Unit Root Process.
3) Unit Root Stochastic Process
• Let’s write the RWM Yt = Yt −1 + ut….. (4) as:
Yt = ρYt−1 + ut − 1
-1 ≤ ρ ≤ 1 ……. (13)
• If ρ = 1, (13) becomes a RWM (without drift).
• If ρ is in fact 1, we face what is known as the unit root problem
(non-stationarity); as the variance of Yt is not stationary.
• The name unit root is due to the fact that ρ = 1.
• Thus the terms non-stationarity, random walk, and unit root can
be treated as synonymous.
• If, however, |ρ| ≤ 1, then the time series Yt is stationary in the
sense we have defined it.
• Note: Unit Root Stochastic Process will be further explained in
Unit Root Test of Stationarity.
Trend Stationary (TS) and Difference Stationary (DS)
Stochastic Processes
• Deterministic Trend: if the trend in a time series is completely
predictable
• Stochastic Trend: if it is not predictable
• E.g. consider the following model of the time series Yt .
Yt = β1 + β2t + β3Yt−1 + ut ………………….(14)
• Now we have the following possibilities:
1. Pure Random Walk: If in (14) β1 = 0, β2 = 0, β3 = 1, we get:
Yt = Yt−1 + ut …………(15)
• Which is nothing but a RWM without drift and is therefore non-stationary.
• But note that, if we write (2.2) as
∆Yt = (Yt − Yt−1) = ut ………… (8)
• It becomes stationary, as noted before.
• Hence, a RWM without drift is a Difference Stationary Process (DSP).
Trend Stationary (TS) and Difference Stationary (DS)
Stochastic Processes (cont.)
2. Random Walk With Drift: If in Yt = β1 + β2t + β3Yt−1 + ut ……(14)
β1 ≠ 0, β2 = 0, β3 = 1, we get:
Yt = β1 + Yt−1 + ut ……..(16)
• which is a random walk with drift and is therefore, non-stationary.
• If we write it as: (Yt − Yt−1) = ∆Yt = β1 + ut ……..(16a)
• this means Yt will exhibit a positive (β1 > 0) or negative (β1 < 0)
trend.
• Such a trend is called a Stochastic Trend.
• Equation (16a) is a DSP process because the non-stationarity in Yt
can be eliminated by taking first differences of the time series.
Trend Stationary (TS) and Difference Stationary (DS)
Stochastic Processes (cont.)
3. Deterministic Trend: If in Yt = β1 + β2t + β3Yt−1 + ut ……(14) β1 ≠
0, β2 ≠ 0, β3 = 0, we obtain:
Yt = β1 + β2t + ut ………..(17)
• Which is called a trend stationary process (TSP).
• Although the mean of Yt is β1 + β2t, which is not constant, its
variance ( = σ2) is.
• Once the values of β1 and β2 are known, the mean can be forecast
perfectly.
• Therefore, if we subtract the mean of Yt from Yt, the resulting
series will be stationary, hence the name trend stationary.
Trend Stationary (TS) and Difference Stationary (DS)
Stochastic Processes (cont.)
4. Random Walk With Drift and Deterministic Trend: If in Yt = β1
+ β2t + β3Yt−1 + ut ……(14) β1 ≠ 0, β2 ≠ 0, β3 = 1, we obtain:
Yt = β1 + β2t + Yt−1 + ut ……….…..(18)
• We have a random walk with drift and a deterministic trend,
which can be seen if we write this equation as:
∆Yt = β1 + β2t + ut ……….……….(18a)
• Which means that Yt is non-stationary.
5. Deterministic Trend With Stationary AR(1) Component: If in
Yt = β1 + β2t + β3Yt−1 + ut ……(14) β1 ≠ 0, β2 ≠ 0, β3 < 1, then we
get:
Yt = β1 + β2t + β3Yt−1 + ut ..……….(19)
• Which is stationary around the deterministic trend.
The Phenomenon of Spurious Regression
• Stationary Time Series are important, consider the following
two random walk models:
Yt = Yt−1 + ut ………….. (20)
Xt = Xt−1 + vt ………….. (21)
• Where we generated 500 observations of ut from ut ∼ N(0, 1)
and 500 observations of vt from vt ∼ N(0, 1) and assumed that
the initial values of both Y and X were zero.
• We also assumed that ut and vt are serially uncorrelated as well as
mutually uncorrelated.
• Both these time series are non-stationary; i.e. they are I(1) or
exhibit stochastic trends.
The Phenomenon of Spurious
Regression (cont.)
• Suppose we regress Yt on Xt .
• Since Yt and Xt are uncorrelated I(1) processes, the
R2 from the regression of Y on X should tend to zero;
that is, there should not be any relationship between
the two variables.
• But wait till you see the regression results:
The Phenomenon of Spurious
Regression (cont.)
• As you can see, the coefficient of X is highly
statistically significant, and, although the R2 value is
low, it is statistically significantly different from zero.
• From these results, you may be tempted to conclude
that there is a significant statistical relationship between
Y and X, whereas a priori there should be none.
• This is the phenomenon of spurious/non-sense
regression, first discovered by Yule.
The Phenomenon of Spurious
Regression (cont.)
• Yule showed that (spurious) correlation could
persist in non-stationary time series even if the
sample is very large.
• That there is something wrong in the preceding
regression is suggested by the extremely low
Durbin–Watson d value, which suggests very
strong first-order autocorrelation.
The Phenomenon of Spurious Regression
(cont.)
• R2 > d is a good rule of thumb to suspect that the
estimated regression is spurious, as in the example
above.
• This example is a strong reminder that one should be
extremely cautious of conducting regression analysis
based on time series that exhibit stochastic trends.
Tests of Stationarity
Presented by Qazi Yasir Ayub
Tests of Stationarity
• In practice we face two important questions:
– How do we find out if a given time series is stationary
or not?
– Is there a way that it can be made stationary?
• Prominently discussed tests in the literature are:
– Graphical Analysis
– The Unit Root Test
1. Graphical Analysis
• Before pursuing a formal test, it is always
advisable to plot the time series under study
• E.g. take the GDP time series.
• You will see that over the period of study GDP
has been increasing (i.e. showing an upward
trend)
• This perhaps suggests that the GDP series is not
stationary (also more or less true of the other
economic time series).
2-
The Unit Root Test
• The widely popular test of stationarity is the unit root
test.
• We start with: Yt = ρYt−1 + ut − 1 ≤ ρ ≤ 1 (1.1)
Where ut is a white noise error term.
• We know that if ρ = 1 (i.e. in the case of the unit root)
(1.1) becomes a Random Walk Model Without Drift,
(a non-stationary stochastic process).
• Therefore, why not simply regress Yt on its lagged value
Yt−1 and find out if the estimated ρ is statistically equal
to 1? If it is, then Yt is non-stationary.
2- The Unit Root Test (cont.)
• Procedure:
Yt = ρYt−1 + ut
(1.1)
=>
Yt − Yt−1 = ρYt−1 − Yt−1 + ut
(4.1)
=>
Yt − Yt−1 = Yt−1 (ρ − 1)+ ut
or
=>
Yt − Yt−1 = (ρ − 1)Yt−1 + ut
which can be alternatively written as:
=>
∆Yt = δYt−1 + ut
(4.2)
where δ = (ρ − 1) and ∆= first-difference (Yt − Yt−1 )
2- The Unit Root Test (cont.)
• Hence, to estimate (4.2) and test the H0: δ = 0.
• If δ = 0, then ρ = 1 (i.e. unit root/time series under
consideration is non-stationary) and (21.9.2) will
become ∆Yt = (Yt − Yt−1) = ut (4.3)
• Since ut is a white noise error term, it is stationary,
which means that the first differences of a random
walk time series are stationary.
• Now let’s turn to the estimation of (4.2).
2- The Unit Root Test (cont.)
• We have to take the first differences of Yt and regress
them on Yt−1 and see if the estimated slope co-efficient in
this regression (δ) is zero or not.
• If it is zero, we conclude that Yt is non-stationary.
• But if it is negative, we conclude that Yt is stationary.
• The only question is which test we use to find out if the
estimated co-efficient of Yt−1 in (4.2) is zero or not?
• Unfortunately, under the null hypothesis that δ = 0 (i.e., ρ
= 1), the t value of the estimated coefficient of Yt−1 does
not follow the t distribution even in large samples; i.e. it
does not have an asymptotic normal distribution.
What is the alternative?
• Dickey and Fuller have shown that under the null
hypothesis that δ = 0, the estimated t-value of the
coefficient of Yt−1 in (4.2) follows the τ (tau)
statistic.
• These authors have computed the critical values of
the tau statistic on the basis of Monte Carlo
simulations.
• Interestingly, if the hypothesis that δ = 0 is rejected
(i.e. the time series is stationary), we can use the
usual t test.
DF Test
• The actual procedure of implementing the DF test
involves several decisions.
• In discussing the nature of the unit root process in
Sections 21.4 and 21.5, we noted that a random
walk process may have no drift, or it may have drift
or it may have both deterministic and stochastic
trends.
• To allow for the various possibilities, the DF test is
estimated in three different forms, i.e. under three
different null hypotheses.
• Yt is a random walk: Yt = δYt−1 + ut (4.2)
• Yt is a random walk with drift: Yt = β1 + δYt−1 + ut (4.4)
• Yt is a random walk with drift around a stochastic
trend: Yt = β1 + β2t + δYt−1 + ut (4.5)
Where t is the time or trend variable.
• In each case, the null hypothesis is that δ = 0; i.e.
there is a unit root—the time series is nonstationary.
• The alternative hypothesis is that δ is less than
zero; i.e. the time series is stationary.
• If the null hypothesis is rejected, it means that Yt is
a stationary time series with zero mean in the case
of (4.2), that Yt is stationary with a nonzero mean
[= β1/(1 − ρ)] in the case of (4.4), and that Yt is
stationary around a deterministic trend in (4.5).
• It is extremely important to note that the critical values of the
tau test to test the hypothesis that δ = 0, are different for each
of the preceding three specifications of the DF test, which
can be seen clearly from Appendix D, Table D.7.
• Moreover, if, say, specification (4.4) is correct, but we
estimate (4.2), we will be committing a specification error,
whose consequences we already know from Chapter 13.
• The same is true if we estimate (4.4) rather than the true
(4.5).
• Of course, there is no way of knowing which specification is
correct to begin with.
• Some trial and error is inevitable, data mining
notwithstanding.
• The actual estimation procedure is as follows:
• Estimate (4.2), or (4.3), or (4.4) by OLS; divide the estimated
coefficient of Yt−1 in each case by its standard error to
compute the (τ) tau statistic; and refer to the DF tables (or
any statistical package).
• If the computed absolute value of the tau statistic (|τ |)
exceeds the DF or MacKinnon critical tau values, we reject
the hypothesis that δ = 0, in which case the time series is
stationary.
• On the other hand, if the computed |τ | does not exceed
the critical tau value, we do not reject the null hypothesis, in
which case the time series is non-stationary.
• Make sure that you use the appropriate critical τ values.
• Let us return to the U.S. GDP time series. For
this series, the results of the three regressions
(4.2), (4.4), and (4.5) are as follows:
• The dependent variable in each case is Yt =
GDPt
•
•
•
•
•
•
•
GDPt = 0.00576GDPt−1…… (4.6)
t = (5.7980) R2 =−0.0152 d = 1.34
GDPt = 28.2054 − 0.00136GDPt−1…… (4.7)
t = (1.1576) (−0.2191) R2 = 0.00056 d = 1.35
GDPt = 190.3857 + 1.4776t − 0.0603GDPt−1
t = (1.8389) (1.6109) (−1.6252)…….. (4.8)
R2 = 0.0305 d = 1.31
• Our primary interest here is in the t ( = τ) value of
the GDPt−1 coefficient.
• The critical 1, 5, and 10 percent τ values for
model (4.6) are −2.5897, −1.9439, and −1.6177,
respectively, and are −3.5064, −2.8947, and
−2.5842 for model (4.7) and −4.0661, −3.4614,
and −3.1567 for model (21.3.8).
• As noted before, these critical values are different
for the three models.
• Before we examine the results, we have to decide
which of the three models may be appropriate.
• We should rule out model (4.6) because the
coefficient of GDPt−1, which is equal to δ is positive.
• But since δ = (ρ − 1), a positive δ would imply that ρ>
1.
• Although a theoretical possibility, we rule this case out
because in this case the GDP time series would be
explosive.
• That leaves us with models (4.7) and (4.8). In both
cases the estimated δ coefficient is negative, implying
that the estimated ρ is less than 1.
• For these two models, the estimated ρ values are
0.9986 and 0.9397, respectively.
• The only question now is if these values are statistically
significantly below 1 for us to declare that the GDP
time series is stationary.
• For model (4.7) the estimated τ value is −0.2191, which in
absolute value is below even the 10 percent critical value of
−2.5842.
• Since, in absolute terms, the former is smaller than the latter,
our conclusion is that the GDP time series is not stationary.
• The story is the same for model (4.8).
• The computed τ value of −1.6252 is less than even the 10
percent critical τ value of −3.1567 in absolute terms.
• Therefore, on the basis of graphical analysis, the
correlogram, and the Dickey–Fuller test, the conclusion is
that for the quarterly periods of 1970 to 1991, the U.S. GDP
time series was non-stationary; i.e., it contained a unit root.