Dates for term tests
1. Friday, February 5
2. Friday, March 5
3. Friday, March 26
The Moving Average Time series of order q, MA(q)
Let {xt|t T} be defined by the equation.
x t u t 1u t 1 2 u t 2
qutq
where {ut|t T} denote a white noise time series
with variance s2.
Then {xt|t T} is called a Moving Average time series of
order q. (denoted by MA(q))
The mean value for an MA(q) time series
E xt
The autocovariance function for an MA(q) time series
s
s h
qh
2
i i h
i0
0
if i q
iq
The autocorrelation function for an MA(q) time series
q h
s h i i h
h
i0
s 0
0
q
2
i
i0
if i q
iq
Comment
The autocorrelation function for an MA(q) time series
q h
s h i i h
h
i0
s 0
0
q
2
i
i0
if i q
iq
“cuts off” to zero after lag q.
1.2
1
q
0.8
0.6
0.4
0.2
0
0
-0.2
-0.4
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
The Autoregressive Time series of order p, AR(p)
Let {xt|t T} be defined by the equation.
x t 1 x t 1 2 x t 2 p x t p u t
where {ut|t T} is a white noise time series with
variance s2.
Then {xt|t T} is called a Autoregressive time series of
order p. (denoted by AR(p))
The mean value of a stationary AR(p) series
E xt
1 1 2 p
The Autocovariance function s(h) of a stationary AR(p) series
Satisfies the equations:
s 0 1s 1 p s p s
Y u le W alk er
E q u a tion s
2
s 1 1s 0 p s p 1
s 2 1s 1 p s p 2
s p 1s p 1 p s 0
and
s h 1s h 1 p s h p
for h > p
The Autocorrelation function (h) of a stationary AR(p) series
Satisfies the equations:
1 11 p p 1
2 1 1 p p 2
p 1 p 1 p 1
with
h 1 h 1 p h p
and s 0
s
1 1 1
2
p p
for h > p
or:
h
h
1
1
1
h c1 c 2 c p
r
r
r
1
2
p
h
where r1, r2, … , rp are the roots of the polynomial
x 1 1x p x
p
x
x
x
1 1 1
r
r
r
1
2
p
and c1, c2, … , cp are determined by using the starting
values of the sequence (h).
Conditions for stationarity
Autoregressive Time series of
order p, AR(p)
If 1 = 1 and = 0.
i.e.
x t 1 x t 1 u t
The value of xt increases in magnitude and ut
eventually becomes negligible.
The time series {xt|t T} satisfies the equation:
x t 1 x t 1
The time series {xt|t T} exhibits deterministic
behaviour.
For a AR(p) time series, consider the polynomial
x 1 1x p x
p
x
x
x
1 1 1
r
r
r
1
2
p
with roots r1, r2 , … , rp
then {xt|t T} is stationary if |ri| > 1 for all i.
If |ri| < 1 for at least one i then {xt|t T} exhibits
deterministic behaviour.
If |ri| ≥ 1 and |ri| = 1 for at least one i then {xt|t T}
exhibits non-stationary random behaviour.
since:
h
h
1
1
1
h c1 c 2 c p
r
r
r
1
2
p
h
and |r1 |>1, |r2 |>1, … , | rp | > 1 for a stationary
AR(p) series then
lim h 0
h
i.e. the autocorrelation function, (h), of a stationary
AR(p) series “tails off” to zero.
1.2
1
0.8
0.6
0.4
0.2
0
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Special Cases: The AR(1) time
Let {xt|t T} be defined by the equation.
x t 1 x t 1 u t
Consider the polynomial
x
x 1 1 x 1
r1
with root r1= 1/1
1. {xt|t T} is stationary if |r1| > 1 or |1| < 1 .
2. If |ri| < 1 or |1| > 1 then {xt|t T} exhibits
deterministic behaviour.
3. If |ri| = 1 or |1| = 1 then {xt|t T} exhibits nonstationary random behaviour.
Special Cases: The AR(2) time
Let {xt|t T} be defined by the equation.
x t 1 x t 1 2 x t 2 u t
Consider the polynomial
x
x
x 1 1 x 2 x 1 1
r1
r2
2
where r1 and r2 are the roots of (x)
1. {xt|t T} is stationary if |r1| > 1 and |r2| > 1 .
This is true if 1+2 < 1 , 2 –1 < 1 and 2 > -1.
These inequalities define a triangular region for
1 and 2.
2. If |ri| < 1 or |1| > 1 then {xt|t T} exhibits
deterministic behaviour.
3. If |ri| ≥ 1 for i = 1,2 and |ri| = 1 for at least on i then
{xt|t T} exhibits non-stationary random behaviour.
Patterns of the ACF and PACF of AR(2) Time Series
In the shaded region the roots of the AR operator are complex
2
The Mixed Autoregressive Moving Average
Time Series of order p,q
The ARMA(p,q) series
The Mixed Autoregressive Moving Average Time
Series of order p, ARMA(p,q)
Let 1, 2, … p , 1, 2, … p , denote p +
q +1 numbers (parameters).
Let {ut|t T} denote a white noise time series with
variance s2.
– independent
– mean 0, variance s2.
Let {xt|t T} be defined by the equation.
x t 1 x t 1 2 x t 2 p x t p
u t 1u t 1 2 u t 2 q u t q
Then {xt|t T} is called a Mixed AutoregressiveMoving Average time series - ARMA(p,q) series.
Mean value, variance,
autocovariance function,
autocorrelation function of an
ARMA(p,q) series
Similar to an AR(p) time series, for certain
values of the parameters 1, …, p an
ARMA(p,q) time series may not be stationary.
An ARMA(p,q) time series is stationary if the
roots (r1, r2, … , rp ) of the polynomial
(x) = 1 – 1x – 2x2 - … - p xp
satisfy | ri| > 1 for all i.
Assume that the ARMA(p,q) time series {xt|t T}
is stationary:
Let = E(xt). Then
E x t 1 E x t 1 2 E x t 2 p E x t p
E u t 1 E u t 1 2 E u t 2 q E u t q
1 2 p
0 1 0 2 0 q 0
1
or
1
2 p
E xt
1 1 2
p
The Autocovariance function, s(h), of a stationary
mixed autoregressive-moving average time series {xt|t
T} be determined by the equation:
x t 1 x t 1 2 x t 2 p x t p
u t 1u t 1 2 u t 2 q u t q
now 1 1 2 p
Thus
x t 1 x t 1 p x t p
u t 1u t 1 2 u t 2 q u t q
Hence
s h E x t h x t
E 1 x t h 1 p x t h p
u t h 1u t h 1 2 u t h 2 q u t h q x t
1 E x t h 1 x t p E x t h p x t
E u t h x t 1 E u t h 1 x t q E u t h q x t
1s h 1 p s h p
s ux h 1s ux h 1 q s ux h q
where
s ux h E u t h x t
note
s ux h E u t h x t
E u t h 1 x t 1 p x t p
u t 1u t 1 2 u t 2 q u t q
1 E u t h x t 1 p E u t h x t p
E u t h u t 1 E u t h u t 1 q E u t h u t q
1s ux h 1 p s ux h p
s uu h 1s uu h 1 q s uu h q
where
and
s ux h E u t h x t 0 if h 0 .
s 2 if h 0 .
s uu h E u t h u t
0 if h 0 .
We need to calculate:
s ux 0 , s ux 1, , s ux q
note
s ux h 1s ux h 1 p s ux h p
s uu h 1s uu h 1 q s uu h q
and
s 2 if h 0 .
s uu h
0 if h 0 .
s ux h 0 if h 0 .
s ux 0 s
2
s ux 1 s ux 0 s
2
s
2
s
s ux 2 s ux 1 2s ux 0 2s
s
2
2s
2
2
s
2
2s
2 2 s
2
2
etc
2
The autocovariance function s(h) satisfies:
s h 1s h 1 p s h p
s ux h 1s ux h 1 q s ux h q
For h = 0, 1. … , q:
s 0 1s 1 p s p s ux 0 1s ux 1 q s ux q
s 1 1s 0 p s p 1 1s ux 0 q s ux q 1
s q 1s q 1 p s q p q s ux 0
for h > q:
s h 1s h 1 p s h p
We then use the first (p + 1) equations to determine:
s(0), s(1), s(2), … , s(p)
We use the subsequent equations to determine:
s(h) for h > p.
Example:The autocovariance function, s(h), for an
ARMA(1,1) time series:
s h 1s h 1 s ux h 1s ux h 1
For h = 0, 1:
s 0 1s 1 s ux 0 1s ux 1
s 1 1s 0 1s ux 0
or
s 0 1s 1 s 1 1 1 s
2
s 1 1s 0
for h > 1:
1s
2
s h 1s h 1
2
Substituting s(0) into the second equation we get:
s 1 1 1s 1 s 1 1 1 s
2
or
s 1
1 1 1 1 1
1 1
2
s
2
s
2
1
2
Substituting s(1) into the first equation we get:
s 0 1
1 1 1 1 1
1 1
2
s s 1 1 1 s
2
2
1 1 1 1 1 1 1 1 1 1 1 1 1
2
2
1 1
2
1 1 2 1 1
2
1 1
2
s
2
2
s
2
for h > 1:
s h 1s h 1
s 2 1s 1 1
s 3 1s 2
2
1
1 1 1 1 1
1 1
2
s
1 1 1 1 1
1 1
2
s
2
2
s h 1s h 1
h 1
1
1 1 1 1 1
1 1
2
s
2
The Backshift Operator B
Consider the time series {xt : t T} and Let M denote
the linear space spanned by the set of random variables
{xt : t T}
(i.e. all linear combinations of elements of {xt : t T}
and their limits in mean square).
M is a vector space
Let B be an operator on M defined by:
Bxt = xt-1.
B is called the backshift operator.
Note:
1. B c x
1
t1
c 2 xt2 c k xtk
c1 Bx t1 c 2 Bx t 2 c k Bx t k
c1 x t1 1 c 2 x t 2 1 c k x t k 1
2. We can also define the operator Bk with
Bkxt = B(B(...Bxt)) = xt-k.
3. The polynomial operator
p(B) = c0I + c1B + c2B2 + ... + ckBk
can also be defined by the equation.
p(B)xt = (c0I + c1B + c2B2 + ... + ckBk)xt .
= c0Ixt + c1Bxt + c2B2xt + ... + ckBkxt
= c0xt + c1xt-1 + c2xt-2 + ... + ckxt-k
4. The power series operator
p(B) = c0I + c1B + c2B2 + ...
can also be defined by the equation.
p(B)xt = (c0I + c1B + c2B2 + ... )xt
= c0Ixt + c1Bxt + c2B2xt + ...
= c0xt + c1xt-1 + c2xt-2 + ...
5. If p(B) = c0I + c1B + c2B2 + ... and q(B) = b0I + b1B
+ b2B2 + ... are such that
p(B)q(B) = I
i.e. p(B)q(B)xt = Ixt = xt
than q(B) is denoted by [p(B)]-1.
Other operators closely related to B:
1. F = B-1 ,the forward shift operator, defined
by Fxt = B-1xt = xt+1
and
2. D = I - B ,the first difference operator,
defined by Dxt = (I - B)xt = xt - xt-1 .
The Equation for a MA(q) time series
xt= 0ut + 1ut-1 +2ut-2 +... +qut-q +
can be written
xt= (B) ut +
where
(B) = 0I + 1B +2B2 +... +qBq
The Equation for a AR(p) time series
xt= 1xt-1 +2xt-2 +... +pxt-p + + ut
can be written
(B) xt= + ut
where
(B) = I - 1B - 2B2 -... - pBp
The Equation for a ARMA(p,q) time series
xt= 1xt-1 +2xt-2 +... +pxt-p + + ut + 1ut-1
+2ut-2 +... +qut-q
can be written
(B) xt= (B) ut +
where
(B) = 0I + 1B +2B2 +... +qBq
and
(B) = I - 1B - 2B2 -... - pBp
Some comments about the Backshift
operator B
1. It is a useful notational device, allowing us to
write the equations for MA(q), AR(p) and
ARMA(p, q) in a very compact form;
2. It is also useful for making certain
computations related to the time series
described above;
The partial autocorrelation
function
A useful tool in time series analysis
The partial autocorrelation function
Recall that the autocorrelation function of an AR(p)
process satisfies the equation:
x(h) = 1x(h-1) + 2x(h-2) + ... +px(h-p)
For 1 ≤ h ≤ p these equations (Yule-Walker) become:
x(1) = 1 + 2x(1) + ... +px(p-1)
x(2) = 1x(1) + 2 + ... +px(p-2)
...
x(p) = 1x(p-1)+ 2x(p-2) + ... +p.
In matrix notation:
1
x 1
2
x 1
x
x p x p 1
x 1
1
x p 2
x p 1 1
x p 2 2
1
p
These equations can be used to find 1, 2, … , p,
if the time series is known to be AR(p) and the
autocorrelation x(h)function is known.
If the time series is not autoregressive the
equations can still be used to solve for 1, 2, … ,
p, for any value of p 1.
In this case
p
p
p
1 , 2 , , p
are the values that minimizes the mean square
error:
M .S . E . E ( x t x )
p
i 1
p
i
( x t i
x )
2
Definition: The partial auto correlation function
at lag k is defined to be:
kk
(k )
k
1
x 1
x 1
x 1
1
x 2
x k 1
x k 2
x k
1
x 1
x k 1
x 1
1
x k 2
x k 2
1
x k 1
Comment:
The partial auto correlation function, kk is
determined from the auto correlation function,
(h)
Some more comments:
1. The partial autocorrelation function at lag k,
kk, can be interpreted as a corrected
autocorrelation between xt and xt-k conditioning
on the intervening variables xt-1, xt-2, ... ,xt-k+1 .
2. If the time series is an AR(p) time series than
kk = 0 for k > p
3. If the time series is an MA(q) time series than
x(h) = 0 for h > q
A General Recursive Formula for
Autoregressive Parameters and the
Partial Autocorrelation function
(PACF)
Let
, , , ,
k
1
k
2
k
3
k
k
kk
denote the autoregressive parameters of order k
satisfying the Yule Walker equations:
2 k 1
k
1
k
2
k
3
1 k 2
k
1
k
2
k
3
k
k
k
k
2
k 1 k 2 k 3
k
1
k
2
k
3
k
k
k
Then it can be shown that:
k
k j k 1 j
k
j 1
k 1
k 1 k 1, k 1
k
1 j j
k
j 1
and
k 1
j
k
j
k 1, k 1
k
k j 1
j 1, 2 , , k
Proof:
The Yule Walker equations:
1 2 2 3 k 1 k
k
k
k
k
1 2 1 3 k 2 k 2
k
k
k
k
k 1 1 k 2 2 k 3 3 k k
k
k
k
k
In matrix form:
1
k 1
1
k 2
or
1
k
Ρ
k 1
k 1 1k
k
k 2 2
2
k
1 k k
Ρ β ρ
k
1
k 2
k
k
k 1
β Ρ
k
k 1
ρ
k
1k
k
k 2
2
k
k
2
, β
and ρ
k
1
k
k
The equations for 1k 1 , 2k 1 , 3k 1 , , kk11
1
k
1
k 1
k 1k 1
k 1
k 1 2
2
k 1
1 k 1 k 1
k 1, k 1
or
A ρ β 1k 1 ρ k
1 k 1, k 1 k 1
Ρk
k
A ρ
k
0
0
A
1
where
β
k 1
1
k 1
1
0
0
0
,
k 1
2
,
The matrix A reverses order
1
0
0
k 1
3
and
, ,
k 1
k
The equations may be written
k 1
k
Ρ β1
ρ
k
k 1, k 1 A ρ ρ
k
k
k 1
A β 1 k 1, k 1 k 1
Ρ
k 1
Multiplying the first equations by
β
or
k 1
1
k 1, k 1 Ρ
β
k 1
1
k 1
β k 1, k 1
k 1
β k 1, k 1 A β
k
k 1
Aρ
A Ρ ρ
β k 1, k 1 Ρ
k
k
Aρ Ρ
k
k 1
k
ρ β
k
k
k
k
Substituting this into the second equation
ρ A β
k
k
k 1, k 1 A β
k
k 1, k 1
k 1
or
k k
k
k
k 1, k 1 1 ρ β k 1 ρ A β
and
k 1, k 1
k 1 ρ
Aβ
1 β ρ
k
k
k
k
Hence
k
k k 1 j
k
j
k 1
k 1
k 1, k 1
j 1
k
1 j j
k
j 1
and
β
k 1
β k 1, k 1 A β
k
k
or
k 1
j
k
j
k 1, k 1
k
k j 1
j 1, 2 , , k
Some Examples
Example 1: MA(1) time series
Suppose that {xt|t T} satisfies the following
equation:
xt = 12.0 + ut + 0.5 ut – 1
where {ut|t T} is white noise with s = 1.1.
Find:
1. The mean of the series,
2. The variance of the series,
3. The autocorrelation function.
4. The partial autocorrelation function.
Solution
Now {xt|t T} satisfies the following equation:
xt = 12.0 + ut + 0.5 ut – 1
Thus:
1. The mean of the series,
= 12.0
The autocovariance function for an MA(1) is
1 s
2
s h s
0
2
1
2
1 0.5 2 1.1 2
h0
2
h 1
0.5 1.1
h 1
0
h0
1.5125
h 1 0.605
h 1 0
h0
h 1
h 1
Thus:
2. The variance of the series,
s(0) = 1.5125
and
3. The autocorrelation function is:
1
s h 0.605
h
1.5125
s 0
0
h0
1
h 1 0.4
h 1 0
h0
h 1
h 1
4.
The partial auto correlation function at lag k is defined to be:
kk
Thus
22
(k )
k
11
(2)
2
1
1
1
1
1
2
k 1
k 2
k
1
1
k 1
1
1
k 2
k 1
k 2
1
(1)
1
1
1
1
2
1
1
1
1
1
1
1 0.4
2
1
2
1 1
2
0.4
2
1 0.4
2
0.16
0.84
.19048
33
44 4
(4)
(3)
3
55 5
1
1
1
1
0.4
0.4
1
1
2
0.4
1
0
2
1
3
0
0.4
0
1
1
2
1
1
2
1
1
.4
0
1
.4
1
.4
1
0
.4
1
0.064
1
1
2
1
1
.4
0
.4
1
1
1
2
.4
1
.4
0
2
1
1
3
0
.4
1
0
3
2
1
4
0
0
.4
0
1
1
2
3
1
.4
0
0
1
1
1
2
.4
1
.4
0
2
1
1
1
0
.4
1
.4
3
2
1
1
0
0
.4
1
(5)
0.01024
0.4368
0.0234
0.0941
0.68
0.0256
0.5456
0.0469
66 0.0117, 77 0.0059, 88 0.0029, 99 0.0015
10 ,10 0.0007, 11,11 0.0004, 12 ,12 0.00029
Graph: Partial Autocorrelation function kk
1 .2
1
0 .8
0 .6
0 .4
0 .2
0
0
-0 .2
-0 .4
1
2
3
4
5
6
7
8
9
10
11
Exercise: Use the recursive method to calculate kk
k
k j k 1 j
k
k 1
k 1 k 1, k 1
j 1
k
1 j j
k
j 1
and
k 1
j
k
j
k 1, k 1
k
k j 1
w e start w ith 1,1 1
1
1
j 1, 2,
, k
Exercise: Use the recursive method to calculate kk
2 ,2
2
2
and
2
1
1 1
1
1 1
1
0.4
1 0.4
2.2
2
1
2
1
1
1
1
2
.19048
j 1
1 .1 9 0 4 8 0 .4
1.19048 0.4 .0.476192
3 1 2 1 1
2
3 3,3
3
2
1 1 2
2
1
2
2
0.0941,
etc
Example 2: AR(2) time series
Suppose that {xt|t T} satisfies the following equation:
xt = 0.4 xt – 1 + 0.1 xt – 2 + 1.2 + ut
where {ut|t T} is white noise with s = 2.1.
Is the time series stationary?
Find:
1. The mean of the series,
2. The variance of the series,
3. The autocorrelation function.
4. The partial autocorrelation function.
1. The mean of the series
1 1 2
1.2
1 0.4 0.1
2.4
3. The autocorrelation function.
Satisfies the Yule Walker equations
1 1 1 2 1 0.4 0.1 1
2 1 1 2 1 0.4 1 0.1
then h 1 1 h 2 h 1 0.4 h 0.1 h 1
w here h h
hence
1
0.4
0.4444
0.9
2 0.4
0.4
0.1 2.778
0.9
then h 1 1 h 2 h 1 0.4 h 0.1 h 1
w here h h
h
0
1
2
3
4
5
6
1.0000 0.4444 0.2778 0.1556 0.0900 0.0516 0.0296
h
7
8
9
10
11
12
13
0.0170 0.0098 0.0056 0.0032 0.0018 0.0011 0.0006
h
h
2. the variance of the series
s 0
s
2
1 1 1 2 1
2.1
2
1 0.4 0.4444 0.1 0.2778
5.7522
4. The partial autocorrelation function.
1,1 1 0.4444
2 ,2
1
1
1
0.4444
1
2
0.4444
.2778
1
1
1
0.4444
1
1
0.4444
1
0.1000
3 ,3
1
1
1
1
0.4444
0.4444
1
1
2
0.4444
1
0.2778
2
1
3
0.2778
0.4444
0.1556
1
1
2
1
0.4444
0.2778
1
1
1
0.4444
1
0.4444
2
1
1
0.2778
0.4444
1
in fact k , k 0
0
for k 3
The partial autocorrelation function of an AR(p) time
series “cuts off” after p.
Example 3: ARMA(1, 2) time series
Suppose that {xt|t T} satisfies the following equation:
xt = 0.4 xt – 1 + 3.2 + ut + 0.3 ut – 1 + 0.2 ut – 1
where {ut|t T} is white noise with s = 1.6.
Is the time series stationary?
Find:
1. The mean of the series,
2. The variance of the series,
3. The autocorrelation function.
4. The partial autocorrelation function.
0
You can add this document to your study collection(s)
Sign in Available only to authorized usersYou can add this document to your saved list
Sign in Available only to authorized users(For complaints, use another form )