Module 5
Stationary Time Series Models Part 2
AR and ARMA Models and Their Properties
Class notes for Statistics 451: Applied Time Series
Iowa State University
Copyright 2015 W. Q. Meeker.
February 14, 2016
21h 0min
Mean the AR(p) Model
5-1
+ at )
at ∼ nid(0, σa2)
+ · · · + φpZt−p
Model: Zt = θ0 + φ1Zt−1 + · · · + φpZt−p + at,
+ φ1Zt−1
Mean: µZ ≡ E(Zt)
E(Zt) = E(θ0
+ (φ1 + · · · + φp)E(Zt)
= E(θ0) + φ1E(Zt−1) + · · · + φpE(Zt−p) + E(at)
= θ0
0
= 1−φ θ−···−φ
p
1
5-3
The last step requires several algebraic steps, but is straightforward.
•
•
••
•
100
150
200
• •
• •• • •
••
• • •• •
•
•
•
• • • • • •••
••
•• •
•
• •
• •
••
•
• • ••
•
•
•
•• •
•
•
• •
••
••
•
•
•
•
••
••
•
• • • • • ••
•
•
•••
•
••
• • ••
• • •
•
•
• ••
•
•
•
• •• • •
•
••
•
••
•
•• • •• •
•
• •
•
•
••
•
•
•
•
•
••
•••
• •
••
•• •
•
•
•
•••
•
••
•
•••
•
•
•
••
•
• •••
• •
•
••
•
•• •
•
•
•
•
• •
•
••
•
• • •• • • ••
• • •• •
•• • • • • •
•
• • • • ••
•
•
•
•
50
250
5-5
Simulated AR(1) Data with φ1 = 0.9, σa = 1
b Z limits
Showing ±2 × σ
plot(arima.sim(n=250, model=list(ar=0.90)),
ylab=””)
•
•
•• •
• •
•
•
0
Time
Module 5
Segment 1
AR(p) Mean and AR(1) Properties
at ∼ nid(0, σa2)
Variance of the AR(1) Model
Model: Zt = θ0 + φ1Zt−1 + at,
Variance: γ0 ≡ Var(Zt) ≡ E[(Zt − µZ )2] = E(Ż 2)
σa2
2
1−φ1
2γ
= φ1
0
=
Var(Zt) =
Var(at )
2
1 − φ1
2 Ż 2
+ at2)]
= E[(φ1
t−1 + 2φ1Żt−1 at
= φ2E(Ż 2 ) + 2φ1E(Żt−1at ) + E(at2)
1
t−1
+ 0
+ σa2
= E[(φ1Żt−1 + at)2]
γ0 = E(Żt2)
or
•
•
•
• • ••
••
•• •
••
•
•
•
•
•
•
50
•
•
100
•
•
•
••
150
•
••
200
•
•
•
•
•
•
•
•
••
•
•
•
•
••• •
••
•
•
•
•
•
•
•
•
•
••
••
•
•
•
••
•
•
•
•
• • • ••
••
•
•
•• •
• •
•
•
• • ••
•
• • ••
•
• •
• •• •
• •
• • •
• •
•
•
•
•
• •
•
•
•
• •
••
•
•
•
••• •
•
•
•
•
••
•
•• ••
•
• • • ••
•
• ••
•• • • •
• •
•
•• •
• • • • •• • • •• •
•
•
•
•
• •
•
•
•
•
• •
•
•
•
• • •
••
•• •••
• • • ••
•
• • • •
• • •
•
•• •
•
•
•
•
• •
•
•
0
Time
250
5-2
5-4
5-6
Simulated AR(1) Data with φ1 = −0.9, σa = 1
b Z limits
Showing ±2 × σ
plot(arima.sim(n=250, model=list(ar=-0.90)),
ylab=””)
4
2
0
-2
-4
4
2
0
-2
-4
-6
-8
-10
100
2
Lag
ar: -0.95
True ACF
10
ar: -0.95
True PACF
10
w= Data
time
150
0
0
Simulated data
4
15
15
200
5
5
ACF
10
Lag
PACF
10
Lag
250
20
20
15
15
300
20
20
0
0
-1.5
0
0
10
-2
10
-1.0
5
5
20
Range-Mean Plot
0
Mean
20
Range-Mean Plot
Mean
-0.5
2
30
4
30
Lag
ar: 0.95
True ACF
10
Lag
ar: 0.95
True PACF
10
w= Data
40
15
15
20
20
0
5
5
50
0
Simulated data
time
w= Data
time
40
0
0
5
5
Simulated data
0.0
50
60
ACF
10
Lag
PACF
10
Lag
Lag
10
PACF
Lag
10
ACF
60
15
20
20
70
15
5 - 10
15
20
20
5 - 12
15
70
Simulated Realization (AR(1), φ1 = −0.95, n = 75)
Graphical Output from Function iden
-4
Simulated Realization (AR(1), φ1 = 0.95, n = 75)
Graphical Output from Function iden
5-8
True ACF and PACF for AR(1) Model with φ1 = 0.95
1.0
Autocovariance and Autocorrelation Functions
for the AR(1) Model
2γ
= φ1(φ1γ0) = φ1
0
= φ1γ0
Autocovariance: γk ≡ Cov(Zt, Zt+k ) ≡ E(ŻtŻt+k )
γ1 ≡ E(ŻtŻt+1) = E[Żt(φ1Żt + at+1)]
+ 0
= φ1E(Żt2) + E(Żtat+1)
= φ1γ0
Thus ρ1 = γγ1 = φ1.
0
γ
+ 0
= φ1E(ŻtŻt+1) + E(Żtat+2)
= φ1γ1
2.
= φ1
50
0
Range-Mean Plot
Mean
True ACF
True PACF
0.0
-1.0
1.0
0.0
-1.0
γ2 ≡ E(ŻtŻt+2) = E[Żt(φ1Żt+1 + at+2)]
Thus ρ2 =
γ2
γ0
5-7
k for −1 < φ <
In general, for the AR(1) model, ρk = γk = φ1
1
0
1.
5
5
Lag
5-9
True ACF and PACF for AR(1) Model with φ1 = −0.95
0
0
-2
Simulated Realization (AR(1), φ1 = 0.95, n = 300)
Graphical Output from Function iden
0
-4
5 - 11
1.0
0.5
0.0
-1.0
1.0
0.5
0.0
-1.0
1.0
0.5
6
4
2
0
-2
-4
-6
7
6
5
4
3
2
ACF
Partial ACF
ACF
Partial ACF
0.0
-1.0
1.0
0.5
0.0
-1.0
1.0
1
5
1.0
0.0
-1.0
1.0
0.0
-1.0
w
Range
w
Range
0
-5
16
14
12
10
8
6
4
ACF
Partial ACF
0.5
0.0
-1.0
1.0
0.5
0.0
-1.0
6
4
2
True ACF
True PACF
w
Range
0
-2
-4
-6
6
5
4
3
2
-0.2
Mean
0.0
Range-Mean Plot
50
0.2
100
w= Data
time
150
0
0
5
5
Simulated data
0.4
200
Lag
10
PACF
Lag
10
ACF
250
15
15
20
20
300
Simulated Realization (AR(1), φ1 = −0.95, n = 300)
Graphical Output from Function iden
0
-0.4
5 - 13
Using the Geometric Power Series to Re-express the
AR(1) Model as an Infinite MA
(1 − φ1B)Żt = at
Żt = (1 − φ1B)−1 at
2 2
= (1 + φ1B + φ1
B + · · · )at
2a
= φ1at−1 + φ1
t−2 + · · · + at
Thus the AR(1) can be expressed as an infinite MA model.
5 - 15
If −1 < φ1 < 1, then the weight on the old residuals is decreasing with age. This is the condition of “stationarity” for
an AR(1) model.
at ∼ nid(0, σa2)
Notes on the AR(1) model
Zt = φ1Zt−1 + at,
• Because ρ1 = φ1, we can estimate φ1 by φb1 = ρb1.
• The root of (1 − φ1B) = 0 is B = 1/φ1 and thus if −1 <
φ1 < 1, then the root is outside [−1, 1] so that AR(1) will
be stationary.
• φ1 = 1 implies Zt = Zt−1 + at, the “random walk” model.
• If φ1 > 1, then Zt is explosive.
Module 5
Segment 2
AR(1) Stationarity Conditions
5 - 14
Using the Back-substitution to Re-express the
AR(1) Model as an Infinite MA
Żt = φ1Żt−1 + at
Żt−1 = φ1Żt−2 + at−1
Żt−2 = φ1Żt−3 + at−2
Żt−3 = φ1Żt−4 + at−3
2a
3
Żt = φ1at−1 + φ1
t−2 + φ1 at−3 + · · · + at
Substituting, successively, Żt−1, Żt−2, Żt−3 . . ., shows that
0
0
5
5
Lag
ar: 0.99999
True ACF
10
Lag
ar: 0.99999
True PACF
10
15
15
20
20
True ACF and PACF for AR(1) Model with
φ1 = 0.99999
5 - 18
5 - 16
This methods works, more generally, for higher-order AR(p)
models, but the algebra is tedious.
1.0
1.0
0.5
5
• If φ1 < −1, then Zt is oscillating explosive.
5 - 17
True ACF
True PACF
0.0
-1.0
1.0
0.0
-1.0
ACF
Partial ACF
0.0
-1.0
1.0
0.5
0.0
-1.0
w
Range
0
-5
14
12
10
8
6
4
0
-136
10
Mean
-132
30
-128
60
ACF
10
Lag
PACF
10
Lag
15
15
70
20
20
50
-150
Range-Mean Plot
Mean
-155
100
-145
time
150
0
5
5
200
Lag
10
PACF
Lag
10
ACF
250
φ2 − φ1 < 1
φ2 + φ1 < 1
→
→
−1 < φ2 < 1
and defines the AR(2) triangle.
φ2 < 1 − φ1
φ2 < 1 + φ1
−1 < φ2 < 1
15
20
20
5 - 24
5 - 22
5 - 20
15
300
Simulated Realization (AR(1), φ1 = 0.99999, n = 300)
Graphical Output from Function iden
0
-170
0
w= Data
50
5
5
Segment 3
Module 5
-140
Simulated data
40
0
0
-140
-150
-160
Both roots lying outside of the “unit-circle” implies
Checking the Stationarity of an AR(2) Model
Simple Rule
AR(2) Stationarity and Model Properties
-160
w= Data
time
q
2 + 4φ
φ1
2
-165
Simulated data
−φ1 ±
2φ2
z = x + iy
x2 + y 2 > 1
−1. A root is “outside of the unit circle” if
q
1.0
0.5
20
Range-Mean Plot
-134
-130
Simulated Realization (AR(1), φ1 = 0.99999, n = 75)
Graphical Output from Function iden
-138
5 - 19
Re-expressing the AR(p) Model as an Infinite MA
∞
X
k=1
ψk at−k + at
More generally, any AR(p) model can be expressed as
1
at = ψ(B)at
φp(B)
φp(B)Żt = at
Żt =
= ψ1at−1 + ψ2at−2 + · · · + at =
|ψj | < ∞
Values of ψ1, ψ2, . . . depend on φ1, . . . , φp. For the AR model
to be stationary, the ψj values should not remain large as j
gets large. Formally, the stationarity condition is met if
∞
X
j=1
5 - 21
Stationarity of an AR(p) model can be checked by finding the
roots of the polynomial φp(B) ≡ (1 − φ1B − φ2B2 − · · · − φpBp).
All p roots must lie outside of the “unit-circle.”
Checking the Stationarity of an AR(2) Model
B=
Stationarity of an AR(2) model can be checked by finding the
roots of the polynomial (1 − φ1B − φ2B2) = 0. Both roots
must lie outside of the “unit-circle.”
√
Roots have the form
where i =
|z| =
w
Range
-128
This method works for any p, but for p > 2 it is best to use
numerical methods to find the p roots.
5 - 23
ACF
Partial ACF
0.0
-1.0
1.0
0.5
0.0
-1.0
1.0
0.5
-170
8
6
4
2
ACF
Partial ACF
0.0
-1.0
1.0
0.5
0.0
-1.0
w
Range
-132
-136
-140
4
3
2
+ at+1)]
Autocovariance and Autocorrelation Functions
for the AR(2) Model
+ φ2Żt−1
+ φ2E(ŻtŻt−1) + E(Żtat+1)
γ1 ≡ E(ŻtŻt+1) = E[Żt(φ1Żt
= φ1E(Żt2)
+ φ2γ1
+ 0
+ E(Żtat+2)
+ at+2)]
+ 0
= φ1γ0
γ2 ≡ E(ŻtŻt+2) = E[Żt(φ1Żt+1 + φ2Żt
+ φ2γ0
= φ1E(ŻtŻt+1) + φ2E(Żt2)
= φ1γ1
+ φ2
+ φ2ρ1
Lag
ar: 1, -0.95
True ACF
10
ar: 1, -0.95
True PACF
10
Lag
w= Data
Simulated data
time
150
0
0
15
15
200
5
5
ACF
10
Lag
PACF
10
250
20
20
15
15
300
20
20
5 - 27
5 - 25
Then using ρk = γk /γ0 and ρ0 = 1, gives the AR(2) ACF
ρ2 = φ1ρ1
100
8
5 - 29
1.0
0.0
-1.0
1.0
0.0
0
0
0
5
5
20
Range-Mean Plot
-4
30
-2
Lag
ar: 0.78, 0.2
True ACF
10
Lag
ar: 0.78, 0.2
True PACF
10
w= Data
40
15
15
20
20
5
50
0
5
Simulated data
time
0
60
ACF
10
Lag
PACF
10
Lag
20
5 - 26
15
20
5 - 28
15
Simulated Realization
(AR(2), φ1 = 0.78, φ2 = 0.2, n = 75)
Graphical Output from Function iden
10
-6
Mean
70
True ACF and PACF for AR(2) Model with
φ1 = 0.78, φ2 = 0.2
-0.4
-0.2
10
Mean
0.2
20
Range-Mean Plot
0.0
0.4
0.6
30
w= Data
40
0
5
5
Simulated data
time
0
50
Lag
10
PACF
Lag
10
ACF
60
15
20
20
5 - 30
15
70
Simulated Realization (AR(2), φ1 = 1, φ2 = −0.95, n = 75)
Graphical Output from Function iden
-8
0
-1.0
True ACF
True PACF
w
ρ1 = φ1
5
5
True ACF and PACF for AR(2) Model with
φ1 = 1, φ2 = −0.95
0
0
50
Range-Mean Plot
4
6
Simulated Realization
(AR(2), φ1 = 0.78, φ2 = 0.2, n = 300)
Graphical Output from Function iden
2
Lag
1.0
0.5
0.0
-1.0
ρ3 = φ1ρ2
+ φ2ρ1
..
..
..
ρk = φ1ρk−1 + φ2ρk−2
0
0
Mean
1.0
0.5
0.0
-1.0
1.0
0.5
0.0
-1.0
-2
-4
-6
-8
-10
6
5
10
5
1.0
0.0
-1.0
1.0
0.0
-1.0
4
3
2
1
ACF
Partial ACF
ACF
Partial ACF
1.0
Range
w
Range
0
-5
-10
20
15
10
5
1.0
0.5
0.0
-1.0
ACF
Partial ACF
0.5
0.0
-1.0
1.0
0.5
0.0
-1.0
10
8
6
True ACF
True PACF
w
Range
4
2
0
-2
7
6
5
4
3
2
0
-0.5
0.5
100
w= Data
time
150
0
5
5
200
0
Simulated data
1.0
ACF
10
Lag
PACF
10
Lag
250
20
300
15
20
5 - 31
15
Simulated Realization
(AR(2), φ1 = 1, φ2 = −0.95, n = 300)
Graphical Output from Function iden
50
Range-Mean Plot
0.0
1.0
0.5
8
6
4
φ1 + ρ1φ2
φ2
φ1 + ρ1φ2 + ρ2φ3
φ3
φ2 + ρ1φ3
ρ3 = ρ2φ1 + ρ1φ2 +
ρ2 = ρ1φ1 +
ρ1 =
AR(3) Yule-Walker Equations
ρ2 = ρ1φ1 +
ρ1 =
AR(2) Yule-Walker Equations
ρ1 = φ1
AR(1) Yule-Walker Equation
Yule-Walker Equations (Correlation Form)
5 - 35
5 - 33
AR(2) Triangle Showing ACF and PACF Functions
Mean
ACF
Partial ACF
0.0
-1.0
1.0
0.5
0.0
-1.0
w
Range
2
0
-2
-4
-6
12
10
8
6
4
5 - 32
AR(2) Triangle Relating ρ1 and ρ2 to φ1 and φ2
Module 5
Segment 4
Yule-Walker Equations (Correlation Form)
φ1 + ρ1φ2 + ρ2φ3 + ρ3φ4
φ4
+ ρ1φ4
5 - 34
Yule-Walker Equations and Their Applications and
AR(p) Variance
ρ1 =
AR(4) Yule-Walker Equations
+
φ3
φ2 + ρ1φ3 + ρ2φ4
+ ρ1φ2
ρ2 = ρ1φ1 +
ρ3 = ρ2φ1
ρ4 = ρ3φ1 + ρ2φ2 + ρ1φ3 +
For given ρ1, . . . , ρ4, this is four linear equations with four
unknowns.
5 - 36
AR(p) Yule-Walker Equations (Correlation Form)
ρ1 = φ1 + ρ1φ2 + ρ2φ3 + · · · + ρp−1φp
ρ2 = ρ1φ1 + φ2 + ρ1φ3 + · · · + ρp−2φp
..
ρp = ρp−1φ1 + ρp−2φ2 + ρp−3φ3 + · · · + φp
For given ρ1, . . . , ρp, this is p linear equations with p unknowns.
+ · · · + φpŻt−p
+ at
5 - 37
Solving the Yule Walker equations for an AR(p) for φp gives
the PACF value φpp.
+ φ2Żt−2
Variance of the AR(p) model
Żt = φ1Żt−1
Multiplying through by Żt gives
+ φ2γ2 + · · · + φpγp
+ σa2
Żt2 = φ1ŻtŻt−1 + φ2ŻtŻt−2 + · · · + φpŻtŻt−p + Żtat
= Var(Zt ) = E(Żt2) = φ1γ1
Taking expectations and noting that E(Żtat ) = σa2 gives
γ0
Then dividing through by γ0 and solving for γ0 gives
σ2
γ0 = 1−φ ρ −φ aρ ···−φpρp
1 1
2 2
Note:
5 - 39
Applications of the Yule-Walker Equations
• Provides the true ACF of an AR(p) model (given φ1, φ2, . . . , φp,
compute ρ1, ρ2, . . . ρp recursively)
• Provides estimate of φ1, φ2, . . . , φp of an AR(p) model (given
sample ACF values ρb1, ρb2, . . . , ρbp, substitute in Y-W equations and solve the set of simultaneous equations for φb1, φb2, . . . , φbp )
• Provides the sample PACF φb1,1, φb2,2, . . . , φbkk for any realization.
(substitute ρb1, ρb2, . . . , ρbk into the AR(k) Y-W equations and
solve for φbk ). Same as Durbin’s formula.
ARMA(1,1) Model Properties
Segment 5
Module 5
5 - 40
5 - 38
• Provides the true PACF φ1,1, φ2,2, . . . , φkk for any ARMA model.
(substitute ρ1, ρ2, . . . , ρk into the AR(k) Y-W equations and
solve for φk ). Same as Durbin’s formula.
0
0
5
5
True ACF
Lag
ma: -0.95 ar: 0.95
10
True PACF
Lag
ma: -0.95 ar: 0.95
10
15
15
20
20
5 - 42
True ACF and PACF for ARMA(1, 1) Model with
φ1 = 0.95, θ1 = −0.95
1.0
E(Żtat) = E(φ1Żt−1at + φ2Żt−2at + · · · + φpŻt−pat + at2) = σa2
Properties of the ARMA(1, 1) Model
= θ0 + θ1(B)at
Zt = θ0 + φ1Zt−1 − θ1at−1 + at
φ1(B)Zt
Noting that µZ = E(Zt) = θ0/(1 − φ1)
= θ1(B)at
Żt = φ1Żt−1 − θ1at−1 + at
φ1(B)Żt
Omitting derivations,
2
γ0 = Var(Zt ) = [(1 − 2φ1θ1 + θ12)/(1 − φ1
)]σa2
ρ1 = (1 − φ1θ1)(φ1 − θ1)/(1 + θ12 − 2φ1θ1)
= φ1ρk−1
ρ2 = φ1ρ1
ρk
5 - 41
True ACF
True PACF
0.0
-1.0
1.0
0.0
-1.0
0
0
0
10
10
-0.2
10
20
Range-Mean Plot
-5
Mean
20
Range-Mean Plot
0.0
Mean
20
-0.5
Range-Mean Plot
-1.0
0
0.0
30
30
30
0.5
w= Data
40
0
0
Simulated data
time
w= Data
time
40
0
0
Simulated data
0.4
w= Data
40
Simulated data
time
0
0
50
50
50
5
5
5
5
5
5
60
ACF
10
Lag
PACF
10
Lag
60
ACF
10
Lag
PACF
10
Lag
60
ACF
10
Lag
PACF
10
Lag
15
15
15
15
15
15
70
70
70
20
20
5 - 43
20
20
5 - 45
20
20
5 - 47
0
0
0
0
0
0
5
5
5
5
5
5
True ACF
Lag
ma: 0.95 ar: -0.95
10
True PACF
Lag
ma: 0.95 ar: -0.95
10
True ACF
Lag
ma: 0.5 ar: 0.9
10
Lag
ma: 0.5 ar: 0.9
True PACF
10
True ACF
Lag
ma: -0.5 ar: -0.9
10
Lag
ma: -0.5 ar: -0.9
True PACF
10
15
15
20
20
15
15
20
20
15
15
20
20
5 - 48
True ACF and PACF for ARMA(1, 1) Model with
φ1 = −0.9, θ1 = −0.5
5 - 46
True ACF and PACF for ARMA(1, 1) Model with
φ1 = 0.9, θ1 = 0.5
5 - 44
True ACF and PACF for ARMA(1, 1) Model with
φ1 = −0.95, θ1 = 0.95
1.0
Simulated Realization
(ARMA(1, 1), φ1 = 0.95, θ1 = −0.95, n = 75)
Graphical Output from Function iden
-10
-1.5
Simulated Realization
(ARMA(1, 1), φ1 = 0.9, θ1 = 0.5, n = 75)
Graphical Output from Function iden
0.2
Simulated Realization
(ARMA(1, 1), φ1 = −0.95, θ1 = 0.95, n = 75)
Graphical Output from Function iden
-0.4
-2.0
Mean
True ACF
True PACF
0.0
-1.0
1.0
0.0
-1.0
1.0
0.0
-1.0
1.0
0.0
-1.0
1.0
True ACF
True PACF
True ACF
True PACF
1.0
0.5
1.0
0.5
0.0
-1.0
1.0
0.0
-1.0
ACF
Partial ACF
0.0
-1.0
1.0
0.5
0.0
-1.0
1.0
0.5
0.0
-1.0
1.0
0.5
0.0
-1.0
ACF
Partial ACF
ACF
Partial ACF
0.0
-1.0
1.0
0.5
0.0
-1.0
w
Range
5
0
-5
-10
14
12
10
8
6
4
10
5
0
-5
-10
20
15
10
5
2
1
0
w
Range
w
Range
-1
-2
-3
-4
3.0
2.8
2.6
2.4
2.2
0
10
20
Range-Mean Plot
0.2
0.4
30
w= Data
time
40
0
0
5
5
Simulated data
0.6
50
Lag
10
PACF
Lag
10
ACF
60
15
20
20
5 - 51
5 - 49
15
70
Simulated Realization
(ARMA(1, 1), φ1 = −0.9, θ1 = −0.5, n = 75)
Graphical Output from Function iden
0.0
∞
X
j=1
j−1
φ1
∞
X
j=0
ψj at−j
0 ≡ 1.
(φ1 − θ1) for j > 0, ψ0 = 1, and φ1
(φ1 − θ1)at−j + at =
j−1
5 - 53
ARMA(1,1) model is white noise (trivial model) if φ1 = θ1.
ARMA(1,1) model is stationary if −1 < φ1 < 1.
where ψj = φ1
=
2
(φ1 − θ1)at−3 + · · · + at
= (φ1 − θ1)at−1 + φ1(φ1 − θ1)at−2 + φ1
2 2
= (1 − θ1B)(1 + φ1B + φ1
B + · · · )at
= (1 − θ1B)(1 − φ1B)−1 at
(1 − θ1B)
at
(1 − φ1B)
(1 − φ1B)Żt = (1 − θ1B)at
Stationarity of an ARMA(1, 1) Model
ARMA(1,1) Square Showing
ACF and PACF Functions
Mean
1.0
0.5
4
Żt =
ACF
Partial ACF
0.0
-1.0
1.0
0.5
0.0
-1.0
w
Range
2
0
-2
7
6
5
4
3
2
Segment 6
Module 5
5 - 50
ARMA(1,1) Square Relating ρ1 and ρ2 to φ1 and θ1
(1 − φ1B)
Żt
(1 − θ1B)
(1 − φ1B)Żt = (1 − θ1B)at
Invertibility of an ARMA(1, 1) Model
5 - 52
Stationarity and Invertibility of the ARMA(1,1) Model
at =
= (1 − φ1B)(1 − θ1B)−1 Żt
= (1 − φ1B)(1 + θ1B + θ12B2 + · · · )Żt
= Żt + (θ1 − φ1)Żt−1 + θ1(θ1 − φ1)Żt−2 + θ12(θ1 − φ1)Żt−3 + · · ·
∞
X
j=1
j−1
θ1
(φ1 − θ1)Żt−j + at =
∞
X
j=1
πj Żt−j + at
(φ1 − θ1), j > 0 and π0 = 1.
5 - 54
Żt = (φ1 − θ1)Żt−1 + θ1(φ1 − θ1)Żt−2 + θ12(φ1 − θ1)Żt−3 + · · · + at
=
j−1
where πj = θ1
ARMA(1,1) model is invertible if −1 < θ1 < 1.
5
5
True ACF
Lag
ma: 0.9 ar: 0.9
10
ma: 0.9 ar: 0.9
True PACF
10
15
15
20
20
ψ1 − φ1 = −θ1 → ψ1 = −θ1 + φ1
5 - 55
ψ2 − ψ1φ1 − φ2 = −θ2 → ψ2 = −θ2 + ψ1φ1 + φ2
0 → ψ3 = ψ2φ1 + ψ1φ2
.
0 → ψk = ψk−1φ1 + ψk−2φ2
0
-0.1
20
Range-Mean Plot
0.1
0.2
30
0.3
w= Data
40
5
50
0
5
Simulated data
time
0
60
ACF
10
Lag
PACF
10
20
70
15
20
5 - 56
15
Simulated Realization
(ARMA(1, 1), φ1 = 0.9, θ1 = 0.9, n = 75)
Graphical Output from Function iden
10
0.0
Lag
ARMA Identification Exercises
Segment 7
Module 5
5 - 60
5 - 58
Given φp(B) and θq (B), solve for π(B), giving π1, π2, . . . .
φp(B) = π(B)θq (B)
• Similar method for expressing an ARMA(p, q) model as an
infinite AR
• Given φp(B) and θq (B), solve for ψ(B), giving ψ1, ψ2, . . . .
φp(B)ψ(B) = θq (B)
Żt = [φp (B)]−1 θq (B)at = ψ(B)at
φp(B)Żt = θq (B)at
Expressing an ARMA(p, q) Model as an Infinite MA
Mean
1.0
0.5
2
1
True ACF and PACF for ARMA(1, 1) Model with
φ1 = 0.9, θ1 = 0.9
0
0
Lag
Properties of ARMA(p, q) model
φp(B)Zt = θ0 + θq (B)at
φp(B)Żt = θq (B)at
(1 − φ1B − φ2B2 − · · · − φp Bp)Żt = (1 − θ1B − θ2B2 − · · · − θq Bq )at
0
• µZ ≡ E(Zt ) = 1−φ θ−···−φ
p
1
• An ARMA(p, q) model is stationary if all of the roots of
φp(B) lie outside of the unit circle.
• An ARMA(p, q) model is invertible if all of the roots of θq (B)
lie outside of the unit circle.
5 - 57
Expressing and ARMA(p, q) Model as an Infinite MA
φ2(B)ψ(B) = θ2(B)
Using ARMA(2,2) as a particular example
+ψ2B2
+ψ3B3 + · · ·
(1 − φ1B − φ2B2)(1 + ψ1B + ψ2B2 + ψ3B3 + · · · ) = (1 − θ1B − θ2B2)
1 + ψ1 B
Multiplying out gives
−φ2B2 −ψ1φ2B3 − · · · = 1 − θ1B − θ2B2
−φ1B −ψ1φ1B2 −ψ2φ1B3 − · · ·
B:
Equating terms with the same power of B gives
B2 :
B3 :
ψ3 − ψ2φ1 − ψ1φ2 =
.
.
Bk : ψk − ψk−1φ1 − ψk−2φ2 =
5 - 59
ACF
Partial ACF
0.0
-1.0
1.0
0.5
0.0
-1.0
w
Range
0
-1
-2
-3
5
4
3
2
1
True ACF
True PACF
1.0
0.0
-1.0
1.0
0.0
-1.0
1
2
0
3
4
Range-Mean Plot
2
Mean
5
20
30
4
6
30
9
10
10
10
11
ACF
12
20
Lag
PACF
20
13
14
30
30
15
5 - 61
20
1
Lag
ACF
20
Lag
PACF
20
Lag
30
30
30
30
5 - 63
-20
2
-10
Mean
Range-Mean Plot
30
10
40
time
0
0
10
10
50
Lag
20
PACF
Lag
20
ACF
Simulated Time Series #6
Graphical Output from Function iden
3
-2
4
5
Range-Mean Plot
-1
Mean
Mean
Range-Mean Plot
2
4
6
7
time
w= decibles
8
9
0
10
10
11
0
10
Simulated Time Series #8
1
10
w= Pressure
time
0
0
10
10
Simulated Time Series #13
6
12
13
ACF
20
Lag
PACF
20
Lag
Lag
20
PACF
Lag
20
ACF
14
Simulated Time Series #13
Graphical Output from Function iden
0
Simulated Time Series #8
Graphical Output from Function iden
0
w= Sales
0
0
40
10
ACF
20
Lag
PACF
20
Lag
-3
Simulated Time Series #6
8
time
w= Deviation
time
10
10
-4
w= Sales
10
0
0
Simulated Time Series #11
0.2
50
0
Simulated Time Series #5
7
6
w= Deviation
time
0
0
Simulated Time Series #7
40
10
Simulated Time Series #7
Graphical Output from Function iden
Range-Mean Plot
10
Mean
0.1
1.0
0.5
1.0
0.5
0
0.0
5 - 65
ACF
Partial ACF
0.0
-1.0
1.0
0.5
0.0
-1.0
1.0
0.5
0.0
-1.0
1.0
0.5
0.0
-1.0
ACF
Partial ACF
ACF
Partial ACF
0.0
-1.0
1.0
0.5
0.0
-1.0
-10
Mean
Range-Mean Plot
-0.1
20
10
0
-10
-20
-30
-40
35
30
25
15
20
10
4
2
0
-2
-4
-6
6
5
4
3
2
6
4
2
Range
w
Range
Simulated Time Series #11
Graphical Output from Function iden
-0.2
20
Simulated Time Series #5
Graphical Output from Function iden(simsta5.d)
20
-20
-0.3
w
Range
w
1.0
0.5
1.0
0.5
0
-2
5
4
3
2
ACF
Partial ACF
0.0
-1.0
1.0
0.0
0.5
-1.0
1.0
0.5
0.0
-1.0
1.0
0.5
0.0
-1.0
ACF
Partial ACF
ACF
Partial ACF
0.0
-1.0
1.0
0.5
0.0
-1.0
w
Range
20
10
0
-10
25
20
15
10
60
40
20
0
-20
-40
100
80
60
40
20
4
w
Range
w
Range
2
0
-2
-4
-6
12
10
8
6
4
30
30
30
30
30
30
15
20
60
5 - 62
16
5 - 64
5 - 66
-1.0
Mean
0.5
Range-Mean Plot
0.0
1.0
10
w= Pressure
time
0
0
10
10
Simulated Time Series #15
1.5
1.0
0.5
3
2
1
Lag
20
PACF
Lag
20
ACF
Simulated Time Series #15
Graphical Output from Function iden
-0.5
ACF
Partial ACF
0.0
-1.0
1.0
0.5
0.0
-1.0
w
Range
0
-1
-2
4.0
3.5
3.0
2.5
2.0
30
30
20
5 - 67