FN3142 QUANTITATIVE FINANCE
CHAP 3: BASIC TIME SERIES CONCEPTS
Essential Reading: Chap 7 & 8 of Elements of Forecasting, (Diebold)
(only the parts that overlap with these notes)
Further Reading: Chapter 2 of Analysis of Financial Time Series (Tsay)
FOR THIS CHAPTER…
AIMS
• Introduce standard measures for
predictability of time series,
autocovariance & autocorrelation.
• Introduce the ‘white noise’
processes, a class of time series
that are not predictable.
• Present the ‘law of iterated
expectations’ and illustrate its use
in time series analysis
LEARNING OUTCOMES
1. Able to describe the various
forms of white noise processes
used in the analysis of financial
data
2. Able to use the ‘law of iterated
expectations’ to derive
unconditional means from
conditional means
3. Able to apply these tools to a
simple AR(1) process
REPRESENTATION OF TIME SERIES (TS)
Sample of observations on a TS is denoted as 𝑦1 , 𝑦2 , 𝑦3 , ⋯ , 𝑦𝑇
Observations before the start of the sample: ⋯ , 𝑦−3 , 𝑦−2 , 𝑦−1 , 𝑦0
Observations beyond the end of the sample: 𝑦𝑇+1 , 𝑦𝑇+2 , 𝑦𝑇+3 , ⋯
Combining the sequence of observations on this TS, we obtain …
𝑦𝑡 ∞
𝑡=−∞
= ⋯ , 𝑦−3 , 𝑦−2 , 𝑦−1 , 𝑦0 , 𝑦1 , 𝑦2 , 𝑦3 , ⋯ , 𝑦𝑇 , 𝑦𝑇+1 , 𝑦𝑇+2 , 𝑦𝑇+3 , ⋯
Observed data
COVARIANCE STATIONARY TIME SERIES
A time series 𝑌𝑡 is said to be covariance stationary if it fulfils the following conditions:
1. 𝜇𝑡 = 𝔼 𝑌𝑡 = 𝜇 ∀𝑡
• the unconditional mean is the same at any moment of time
2. 𝜎𝑡2 = 𝕍𝑎𝑟 𝑌𝑡 = 𝜎 2 ∀𝑡
• the unconditional mean is the same at any moment of time
3. 𝛾𝑡,𝑡−𝑗 = ℂ𝑜𝑣 𝑌𝑡 , 𝑌𝑡−𝑗 = 𝛾𝑗 ∀𝑡 and 𝑗
• Covariance depends only on the time difference, not the time origin
• E.g. 𝛾6,0 = ℂ𝑜𝑣 𝑌6 , 𝑌0 = 𝛾6 = 𝛾27,21 = 𝛾10,4
Stationary process vs non-stationary process
Source: https://www.statisticshowto.com/stationarity/
AUTOCOVARIANCE & AUTOCORRELATION
• The j-th order autocovariance of a stationary time series 𝑌𝑡 is defined as
𝛾𝑗 = ℂ𝑜𝑣 𝑌𝑡 , 𝑌𝑡−𝑗 = 𝔼 𝑌𝑡 − 𝜇 𝑌𝑡−𝑗 − 𝜇
= 𝔼 𝑌𝑡 , 𝑌𝑡−𝑗 − 𝜇 2
Setting 𝑗 = 0 produces 𝛾0 = ℂ𝑜𝑣 𝑌𝑡 , 𝑌𝑡 = 𝕍𝑎𝑟 𝑌𝑡 = 𝜎 2
Variance and autocovariance are scale-dependent: they have units that equal the square of the units
for 𝑌𝑡
• The j-th order autocorrelation of a stationary time series 𝑌𝑡 is defined as
ℂ𝑜𝑣 𝑌𝑡 , 𝑌𝑡−𝑗
𝛾𝑗
𝜌𝑗 = ℂ𝑜𝑟𝑟 𝑌𝑡 , 𝑌𝑡−𝑗 =
=
∈ −1,1
𝕍 𝑌𝑡
𝛾0
with 𝜌0 = ℂ𝑜𝑟𝑟 𝑌𝑡 , 𝑌𝑡 =
𝕍
𝕍
𝑌𝑡
𝑌𝑡
= 1.
If a TS 𝑌𝑡 has 𝛾𝑗 ≠ 0 for some 𝑗 ≠ 0, then the TS is said to be serially correlated (or autocorrelated)
LAW OF ITERATED EXPECTATION (LIE)
Let 𝐼1 and 𝐼2 be two related information set satisfying 𝐼1 ⊆ 𝐼2
• 𝐼2 is bigger than 𝐼1 (𝐼2 contains more information 𝐼1 )
• In general, I0 ⊆ 𝐼𝑡 ⊆ 𝐼𝑡+1 ∀𝑡 (information grows with time)
By the law of iterated expectation (LIE), 𝔼 𝔼 𝑌 𝐼2 ห𝑰𝟏
Note the final landmark, i.e. information set at the end → the ultimate result is the expectation
conditional on information set 𝐼1 (the smaller info set)
LIE is very helpful to calculate conditional expectations, e.g.
𝔼𝑡 𝔼𝑡+1 𝑌𝑡+2 = 𝔼 𝔼 𝑌𝑡+2 ȁ𝐼𝑡+1 𝐼𝑡
= 𝔼 𝑌𝑡+2 ȁ𝐼𝑡
An unconditional expectation is equivalent to conditional expectation on an ‘empty’ or null
information set, which is smaller than any non-empty information set, Φ →𝔼 𝑌𝑡+2 ȁ𝐼0 = 𝔼 𝑌𝑡+2
RULES FOR EXPECTATIONS, VARIANCES & COVARIANCES
Let X, Y and Z be 3 scalar random variables, and
let a, b, c & d be constants. Then
𝔼 𝑎 + 𝑏𝑋 + 𝑐𝑌 = 𝑎 + 𝑏𝔼 𝑋 + 𝑐𝔼 𝑌
𝕍𝑎𝑟 𝑎 + 𝑏𝑋 = 𝕍𝑎𝑟 𝑏𝑋 = 𝑏 2 𝕍𝑎𝑟 𝑋
𝕍𝑎𝑟 𝑎 + 𝑏𝑋 + 𝑐𝑌 = 𝕍𝑎𝑟 𝑏𝑋 + 𝑐𝑌
= 𝕍𝑎𝑟 𝑏𝑋 + 𝕍𝑎𝑟 𝑐𝑌 + 2ℂ𝑜𝑣 𝑏𝑋, 𝑐𝑌
= 𝑏 2 𝕍𝑎𝑟 𝑋 + 𝑐 2 𝕍𝑎𝑟 𝑌 + 2𝑏𝑐ℂ𝑜𝑣 𝑋, 𝑌
Example: Consider two stocks X & Y, whose
returns follow normal distribution: 𝑋~𝑁 1,2 and
𝑌~𝑁 2,3 . Consider 2 portfolios, A & B with the
following allocation:
1
1
𝐴= 𝑋+ 𝑌
2
2
3
1
𝐵= 𝑋+ 𝑌
4
4
Let 𝑼 = 𝐴, 𝐵 ′. Find the mean vector, covariance
matrix and correlation matrix of U.
ℂ𝑜𝑣 𝑎 + 𝑏𝑋, 𝑐𝑌 + 𝑑𝑍 = ℂ𝑜𝑣 𝑏𝑋, 𝑐𝑌 + ℂ𝑜𝑣 𝑏𝑋, 𝑑𝑍
= 𝑏𝑐ℂ𝑜𝑣 𝑋, 𝑌 + 𝑏𝑑ℂ𝑜𝑣 𝑋, 𝑍
© 2022 Singapore Institute of Management Group Limited
WHITE NOISE & OTHER INNOVATION SERIES
In a linear regression problem / equation: 𝑦𝑡 = 𝛽0 + 𝛽1 𝑥1𝑡 + 𝛽2 𝑥2𝑡 + ⋯ + 𝛽𝑝 𝑥𝑝𝑡 + 𝑢𝑡 we assume
𝑢𝑡 ~𝑁 0, 𝜎 2 and ℂ𝑜𝑟𝑟 𝑢𝑡 , 𝑢𝑠 = 0 ∀𝑠 ≠ 𝑡
In TS analysis, the error terms, denoted 𝜀𝑡 , are assumed to not be correlated to each other,
ℂ𝑜𝑟𝑟 𝜀𝑡 , 𝜀𝑡−𝑗 = 0 ∀𝑗 ≠ 0
3 types of noise: A TS is called a _______ white noise (WN), denote as 𝜀𝑡
1. Simple WN: No serial correlation
2. i.i.d WN: No serial correlation, no serial dependence and identical distribution (no shape
assumed)
3. Gaussian WN: No serial correlation, no serial dependence & identical Gaussian distribution
(bell-shaped error)
Simple
WN
𝜀𝑡 ~𝑊𝑁 0, 𝜎
2
0 mean WN
with constant
variance 𝜎 2
WN
𝜀𝑡 ~i.i.d. WN(0, 𝜎 2 )
0 mean i.i.d
WN with
constant
variance 𝜎 2
Conditions
Conditions
1. 𝐶𝑜𝑟𝑟 𝜀𝑡 , 𝜀𝑡−𝑗 = 0
1. 𝐶𝑜𝑟𝑟 𝜀𝑡 , 𝜀𝑡−𝑗 = 0
2. 𝜀𝑡 independent of
𝜀𝑡−𝑗 ∀𝑗 ≠ 0
2. 𝔼 𝜀𝑡 = 0
No serial correlation
3. 𝜀𝑡 ~𝐹 ∀𝑡 where F is
some distribution
No shape assumed
Gaussian
WN
𝜀𝑡 ~i.i.d. 𝑁(0, 𝜎 2 )
Error terms follow
normal distribution
with 0 mean, variance
𝜎2
Conditions
1. 𝐶𝑜𝑟𝑟 𝜀𝑡 , 𝜀𝑡−𝑗 = 0
2. 𝜀𝑡 independent of
𝜀𝑡−𝑗 ∀𝑗 ≠ 0
3. 𝜀𝑡 ~𝑁(0, 𝜎 2 )
Shape assumed
0-corr ≠independent
Hence we need the
2nd condition in WN &
Gaussian WN
Reason: Correlation
only captures linear
dependency.
2 RVs with 0
correlation may
depend on each other
in non-linear fashion
TRY THIS
Any time series 𝑌𝑡+1 may be decompose into its conditional mean, 𝔼𝑡 𝑌𝑡+1 and a
remainder process, 𝜀𝑡+1 . Show that
𝑌𝑡+1 = 𝔼𝑡 𝑌𝑡+1 + 𝜀𝑡+1
a) 𝜀𝑡 has 0 conditional mean on the information set available at time 𝑡
b) 𝜀𝑡 is a zero-mean white noise process
c) 𝜀𝑡+1 is uncorrelated with the conditional mean term, 𝔼𝑡 𝑌𝑡+1
The above 3 questions are meant to prove the properties of white noise, so that we can be
sure that the remainder process is an error term in the process
APPLICATION TO AR(1) PROCESS
Consider a TS process 𝑌𝑡 defined as follows:
𝑌𝑡 = 𝜙𝑌𝑡−1 + 𝜀𝑡 ,
𝜀𝑡 ~𝑊𝑁 0, 𝜎 2 and 𝜙 < 1
The above equation is referred to as an autoregressive process of order 1, denoted AR(1)
•
Autoregressive / AR(1)= The variable 𝑌𝑡 is regressed onto itself (hence the ‘auto’),
lagged by 1 period (hence the ‘first-order’).
The TS has 2 parameters: 𝝓 (1st order AR coefficient) and 𝝈𝟐 (variance of white noise 𝜀𝑡 )
When asked for a particular property of 𝑌𝑡 , it should be given as the function of 𝜙 and 𝜎 2
𝒀𝒕 ~AR(1): 𝒀𝒕 = 𝝓𝒀𝒕−𝟏 + 𝜺𝒕 with 𝜀~𝑊𝑁 0, 𝜎 2
1. What is the unconditional mean of 𝑌𝑡 ?
2. What is the unconditional variance of 𝑌𝑡 ?
3. What is the 1st order autocovariance & autocorrelation of 𝑌𝑡 ?
4. What is the 2nd order autocovariance & autocorrelation of 𝑌𝑡 ?
5. What is the jth order autocovariance of 𝑌𝑡 ?
i)
Unconditional mean of 𝑌𝑡 ⟹ 𝔼 𝑌𝑡
𝔼 𝑌𝑡 = 𝔼 𝜙𝑌𝑡−1 + 𝜀𝑡 = 𝔼 𝜙𝑌𝑡−1 + 𝔼 𝜀𝑡
= 𝜙𝔼 𝑌𝑡−1 + 0
As 𝑌𝑡 is assumed to be stationary, 𝔼 𝑌𝑡 = 𝔼 𝑌𝑡−1 = 𝜇. From the above equation,
𝔼 𝑌𝑡 = 𝜙𝔼 𝑌𝑡−1
𝜇 = 𝜙𝜇
𝜇 − 𝜙𝜇 = 0
𝜇 1−𝜙 =0
Hence, the unconditional mean,𝔼 𝑌𝑡 = 𝜇 = 0 because 𝜙 < 1 by definition.
ii) Unconditional variance of 𝑌𝑡 ,𝕍𝑎𝑟 𝑌𝑡 (denoted as 𝛾0 )
𝛾0 = 𝕍𝑎𝑟 𝑌𝑡
= 𝕍𝑎𝑟 𝜙𝑌𝑡−1 + 𝜀𝑡
= 𝜙 2 𝕍𝑎𝑟 𝑌𝑡−1 + 𝕍𝑎𝑟 𝜀𝑡 + 2ℂ𝑜𝑣 𝜙𝑌𝑡−1 , 𝜀𝑡
= 𝜙 2 𝕍𝑎𝑟 𝑌𝑡−1 + 𝜎 2 + 0
Stationarity assumption of 𝑌𝑡 implies that 𝕍𝑎𝑟 𝑌𝑡 doesn’t vary with time
⟹ 𝛾0 = 𝜙 2 𝛾0 + 𝜎 2
𝛾0 − 𝜙 2 𝛾0 = 𝜎 2
𝛾0 1 − 𝜙 2 = 𝜎 2
𝜎2
⟹ 𝕍𝑎𝑟 𝑌𝑡 = 𝛾0 =
1 − 𝜙2
iii) The 1st order autocovariance & autocorrelation of 𝑌𝑡
ℂ𝑜𝑣 𝑌𝑡 , 𝑌𝑡−1 = ℂ𝑜𝑣 𝜙𝑌𝑡−1 + 𝜀𝑡 , 𝑌𝑡−1
= ℂ𝑜𝑣 𝜙𝑌𝑡−1 , 𝑌𝑡−1 + ℂ𝑜𝑣 𝜀𝑡 , 𝑌𝑡−1
= 𝜙𝕍𝑎𝑟 𝑌𝑡−1 + 0
= 𝜙𝛾0
ℂ𝑜𝑣 𝑌𝑡 , 𝑌𝑡−1
𝜌1 ≡ 𝐶𝑜𝑟𝑟 𝑌𝑡 , 𝑌𝑡−1
𝜎2
=𝜙
1 − 𝜙2
ℂ𝑜𝑣 𝑌𝑡 , 𝑌𝑡−1
𝜙𝛾0
=
=
=𝜙
𝕍𝑎𝑟 𝑌𝑡
𝛾0
Equipped with only the AR(1) equation, i.e. 𝑌𝑡 = 𝜙𝑌𝑡−1 + 𝜀𝑡 , confirm that the 2nd order
𝜎2
2
2.
autocovariance is 𝛾2 = 𝜙
&
autocorrelation
of
𝑌
is
𝜌
=
𝜙
𝑡
2
2
1−𝜙
LET’S TRY THIS
Consider the AR(2) process: 𝑧𝑡 = 𝛼0 + 𝛼1 𝑧𝑡−1 + 𝛼2 𝑧𝑡−2 + 𝜀𝑡 where 𝜀𝑡 is a zero-mean white noise
process with variance 𝜎 2, and assume 𝛼1 , 𝛼2 and 𝛼1 + 𝛼2 < 1, which together ensure 𝑧𝑡 is
covariance stationary.
a) Calculate the conditional and unconditional means of 𝑧𝑡 , i.e. 𝔼𝑡−1 𝑧𝑡 and 𝔼 𝑧𝑡 .
b) Let us now set 𝛼2 = 0. Calculate the conditional and unconditional variances of 𝑧𝑡 .
c) Keeping 𝛼2 = 0, derive the autocovariance and autocorrelation functions of this process for all lags
as functions of the parameters 𝛼1 and 𝜎.
Suppose now that 𝛼2 ≠ 0, and let us denote the autocovariance at lag k by 𝛾𝑘 ≡ ℂ𝑜𝑣 𝑧𝑡 , 𝑧𝑡−𝑘 .
d) Using the AR(2) equation, write down a recursive formula for 𝛾𝑘 , i.e. express 𝛾𝑘 as a function of
𝛾𝑘−1 , 𝛾𝑘−2 and the model parameters.
e) Apply this recursive formula for 𝑘 = 1 and 𝑘 = 0, and explain how to solve for the whole
autocovariance function 𝛾𝑘 𝑘≥0 . Note: No need to derive the exact values! Hint: Think about what 𝛾−1
and 𝛾−2 mean?
f) Can a linear transformation of the 𝑧𝑡 process be represented by an AR(1) process? That is, do
appropriate 𝜅, 𝛿0 and 𝛿1 constants exist such that the process defined as 𝑦𝑡 = 𝑧𝑡 + 𝜅𝑧𝑡−1 satisfies 𝑦𝑡 =
𝛿0 + 𝛿1 𝑦𝑡−1 + 𝜖𝑡 where 𝜖𝑡 is a zero-mean white noise process? Explain.