Module 5
Stationary Time Series Models Part 2
AR and ARMA Models and Their Properties
Class notes for Statistics 451: Applied Time Series
Iowa State University
Copyright 2015 W. Q. Meeker.
February 14, 2016
21h 0min
5-1
Module 5
Segment 1
AR(p) Mean and AR(1) Properties
5-2
Mean the AR(p) Model
Model: Zt = θ0 + φ1Zt−1 + · · · + φpZt−p + at,
at ∼ nid(0, σa2)
Mean: µZ ≡ E(Zt)
E(Zt) = E(θ0
+ φ1Zt−1
+ · · · + φpZt−p
+ at )
= E(θ0) + φ1E(Zt−1) + · · · + φpE(Zt−p) + E(at)
= θ0
+ (φ1 + · · · + φp)E(Zt)
0
= 1−φ θ−···−φ
p
1
The last step requires several algebraic steps, but is straightforward.
5-3
Variance of the AR(1) Model
Model: Zt = θ0 + φ1Zt−1 + at,
at ∼ nid(0, σa2)
Variance: γ0 ≡ Var(Zt) ≡ E[(Zt − µZ )2] = E(Ż 2)
γ0 = E(Żt2)
= E[(φ1Żt−1 + at)2]
2
= E[(φ2
1 Żt−1 + 2φ1Żt−1 at
+ a2
t )]
2
2
= φ2
1 E(Żt−1 ) + 2φ1E(Żt−1 at ) + E(at )
= φ2
1 γ0
=
+ 0
+ σa2
σa2
1−φ2
1
or
Var(at )
Var(Zt) =
1 − φ2
1
5-4
Simulated AR(1) Data with φ1 = 0.9, σa = 1
b Z limits
Showing ±2 × σ
plot(arima.sim(n=250, model=list(ar=0.90)),
ylab=””)
••
•
•
••
•
•
•
•
•
• •
••
••
•
•
• • ••
•
••
• • •• •
••
•• • • • • •
• •
•
•
• • • • ••
•
•
•
•••••
•
•
•
•
•
••
•
• •••
• •
•
-6
-4
-2
0
2
4
• •
• •• • •
••
• • •• •
••
•
•
•
• • • • ••
• • ••
•
•
•
•
•• • • ••
• • ••
•
•
•
•• •
•
•
••
•• • ••••• •
•
••
••
•
•
• ••
••
•
•
•
•
••
• • ••
• • •
•
•
•
•
•
•
•
••
• •• • •
•
• •
••
•
•
•
• •
•
••• ••
•
•
•
••
•
•
•
•
•
•••
•
•
••
•• •
•••
•
•
•
•
•
•
-8
•
••
•
-10
•
0
50
100
150
200
250
Time
5-5
Simulated AR(1) Data with φ1 = −0.9, σa = 1
b Z limits
Showing ±2 × σ
plot(arima.sim(n=250, model=list(ar=-0.90)),
ylab=””)
•
•
-4
-2
0
2
4
•
•
••
•
••• •
•
•
•
•
•
•
•
•
•
•
•
••
•
•
•
••
••
•
•
•
••
•
•
•
•
•
•
•
••
•
•
• •
•• •
• •
•
•
• • ••
•
•
• • ••
•
•
•
• •
•
• •
• •
••
• •
•
•
•
• •
•
•
•
• •
••
•
•
•
•
••• •
•
•
•
•
•
•• ••
•
•
• • • ••
•
•
• • ••
•
•
•
•
• •
•• •
• • • • •• • • •• •
• •
•
•
•
•
• • •
• •
•
•
•
• ••
•
• • •
•• •••
• •
•
•
•
•
•• • • •
• • • •
•
•
••
•• •
•
•
•
•
•
•
•
• •
•
•
• • ••
•
•
•
••
•
•• •
••
•
••
•
•
•
0
•
•
50
100
150
200
250
Time
5-6
Autocovariance and Autocorrelation Functions
for the AR(1) Model
Autocovariance: γk ≡ Cov(Zt, Zt+k ) ≡ E(ŻtŻt+k )
γ1 ≡ E(ŻtŻt+1) = E[Żt(φ1Żt + at+1)]
= φ1E(Żt2) + E(Żtat+1)
= φ1γ0
+ 0
= φ1γ0
Thus ρ1 = γγ1 = φ1.
0
γ2 ≡ E(ŻtŻt+2) = E[Żt(φ1Żt+1 + at+2)]
= φ1E(ŻtŻt+1) + E(Żtat+2)
= φ1γ1
+ 0
= φ1(φ1γ0) = φ2
1γ0
Thus ρ2 = γγ2 = φ2
1.
0
γ
In general, for the AR(1) model, ρk = γk = φk1 for −1 < φ1 <
0
1.
5-7
True ACF and PACF for AR(1) Model with φ1 = 0.95
True ACF
0.0
-1.0
True ACF
1.0
ar: 0.95
0
5
10
15
20
15
20
Lag
True PACF
0.0
-1.0
True PACF
1.0
ar: 0.95
0
5
10
Lag
5-8
True ACF and PACF for AR(1) Model with φ1 = −0.95
True ACF
0.0
-1.0
True ACF
1.0
ar: -0.95
0
5
10
15
20
15
20
Lag
True PACF
0.0
-1.0
True PACF
1.0
ar: -0.95
0
5
10
Lag
5-9
Simulated Realization (AR(1), φ1 = 0.95, n = 75)
Graphical Output from Function iden
Simulated data
0
-6
-4
-2
w
2
4
6
w= Data
0
10
20
30
40
50
60
70
time
ACF
5
0.0
-1.0
6
ACF
0.5
7
1.0
Range-Mean Plot
5
10
15
20
15
20
4
Lag
0.5
0.0
-1.0
2
Partial ACF
3
1.0
PACF
1
Range
0
-4
-2
0
Mean
2
4
0
5
10
Lag
5 - 10
Simulated Realization (AR(1), φ1 = 0.95, n = 300)
Graphical Output from Function iden
Simulated data
-6
-4
-2
w
0
2
4
6
w= Data
0
50
100
150
200
250
300
time
ACF
0.0
5
-1.0
6
ACF
0.5
1.0
Range-Mean Plot
0
5
10
15
20
15
20
Range
Lag
2
0.5
0.0
-1.0
3
Partial ACF
1.0
4
PACF
-4
-2
0
Mean
2
4
0
5
10
Lag
5 - 11
Simulated Realization (AR(1), φ1 = −0.95, n = 75)
Graphical Output from Function iden
Simulated data
0
-5
w
5
w= Data
0
10
20
30
40
50
60
70
time
ACF
0.0
0
5
10
15
20
15
20
10
Lag
0.5
-1.0
0.0
6
Partial ACF
8
1.0
PACF
4
Range
12
-1.0
14
ACF
16
0.5
1.0
Range-Mean Plot
-1.5
-1.0
-0.5
Mean
0.0
0
5
10
Lag
5 - 12
Simulated Realization (AR(1), φ1 = −0.95, n = 300)
Graphical Output from Function iden
Simulated data
w
-5
0
5
w= Data
0
50
100
150
200
250
300
time
ACF
0.0
-1.0
12
ACF
14
0.5
1.0
Range-Mean Plot
5
10
15
20
15
20
10
0
Range
Lag
0.5
0.0
-1.0
4
Partial ACF
6
1.0
8
PACF
-0.4
-0.2
0.0
Mean
0.2
0.4
0
5
10
Lag
5 - 13
Module 5
Segment 2
AR(1) Stationarity Conditions
5 - 14
Using the Geometric Power Series to Re-express the
AR(1) Model as an Infinite MA
(1 − φ1B)Żt = at
Żt = (1 − φ1B)−1 at
2
= (1 + φ1B + φ2
B
+ · · · )at
1
= φ1at−1 + φ2
1at−2 + · · · + at
Thus the AR(1) can be expressed as an infinite MA model.
If −1 < φ1 < 1, then the weight on the old residuals is decreasing with age. This is the condition of “stationarity” for
an AR(1) model.
5 - 15
Using the Back-substitution to Re-express the
AR(1) Model as an Infinite MA
Żt = φ1Żt−1 + at
Żt−1 = φ1Żt−2 + at−1
Żt−2 = φ1Żt−3 + at−2
Żt−3 = φ1Żt−4 + at−3
Substituting, successively, Żt−1, Żt−2, Żt−3 . . ., shows that
3
Żt = φ1at−1 + φ2
1 at−2 + φ1 at−3 + · · · + at
This methods works, more generally, for higher-order AR(p)
models, but the algebra is tedious.
5 - 16
Notes on the AR(1) model
Zt = φ1Zt−1 + at,
at ∼ nid(0, σa2)
• Because ρ1 = φ1, we can estimate φ1 by φb1 = ρb1.
• The root of (1 − φ1B) = 0 is B = 1/φ1 and thus if −1 <
φ1 < 1, then the root is outside [−1, 1] so that AR(1) will
be stationary.
• φ1 = 1 implies Zt = Zt−1 + at, the “random walk” model.
• If φ1 > 1, then Zt is explosive.
• If φ1 < −1, then Zt is oscillating explosive.
5 - 17
True ACF and PACF for AR(1) Model with
φ1 = 0.99999
True ACF
0.0
-1.0
True ACF
1.0
ar: 0.99999
0
5
10
15
20
15
20
Lag
True PACF
0.0
-1.0
True PACF
1.0
ar: 0.99999
0
5
10
Lag
5 - 18
Simulated Realization (AR(1), φ1 = 0.99999, n = 75)
Graphical Output from Function iden
Simulated data
-140
-136
w
-132
-128
w= Data
0
10
20
30
40
50
60
70
time
ACF
0.0
-1.0
4
ACF
0.5
1.0
Range-Mean Plot
5
10
15
20
15
20
3
Lag
0.5
-1.0
0.0
Partial ACF
1.0
PACF
2
Range
0
-138
-136
-134
-132
Mean
-130
-128
0
5
10
Lag
5 - 19
Simulated Realization (AR(1), φ1 = 0.99999, n = 300)
Graphical Output from Function iden
Simulated data
-170
-160
w
-150
-140
w= Data
0
50
100
150
200
250
300
time
ACF
6
0.0
-1.0
ACF
8
0.5
1.0
Range-Mean Plot
0
5
10
15
20
15
20
Range
Lag
0.5
0.0
-1.0
2
Partial ACF
1.0
4
PACF
-170
-165
-160
-155
Mean
-150
-145
-140
0
5
10
Lag
5 - 20
Re-expressing the AR(p) Model as an Infinite MA
More generally, any AR(p) model can be expressed as
φp(B)Żt = at
1
Żt =
at = ψ(B)at
φp(B)
= ψ1at−1 + ψ2at−2 + · · · + at =
∞
X
ψk at−k + at
k=1
Values of ψ1, ψ2, . . . depend on φ1, . . . , φp. For the AR model
to be stationary, the ψj values should not remain large as j
gets large. Formally, the stationarity condition is met if
∞
X
j=1
|ψj | < ∞
Stationarity of an AR(p) model can be checked by finding the
roots of the polynomial φp(B) ≡ (1 − φ1B − φ2B2 − · · · − φpBp).
All p roots must lie outside of the “unit-circle.”
5 - 21
Module 5
Segment 3
AR(2) Stationarity and Model Properties
5 - 22
Checking the Stationarity of an AR(2) Model
Stationarity of an AR(2) model can be checked by finding the
roots of the polynomial (1 − φ1B − φ2B2) = 0. Both roots
must lie outside of the “unit-circle.”
B=
−φ1 ±
q
φ2
1 + 4φ2
2φ2
Roots have the form
z = x + iy
where i =
√
−1. A root is “outside of the unit circle” if
|z| =
q
x2 + y 2 > 1
This method works for any p, but for p > 2 it is best to use
numerical methods to find the p roots.
5 - 23
Checking the Stationarity of an AR(2) Model
Simple Rule
Both roots lying outside of the “unit-circle” implies
φ2 + φ1 < 1
→
φ2 < 1 − φ1
φ2 − φ1 < 1
→
φ2 < 1 + φ1
−1 < φ2 < 1
−1 < φ2 < 1
and defines the AR(2) triangle.
5 - 24
Autocovariance and Autocorrelation Functions
for the AR(2) Model
γ1 ≡ E(ŻtŻt+1) = E[Żt(φ1Żt
+ φ2Żt−1
+ at+1)]
= φ1E(Żt2)
+ φ2E(ŻtŻt−1) + E(Żtat+1)
= φ1γ0
+ φ2γ1
γ2 ≡ E(ŻtŻt+2) = E[Żt(φ1Żt+1 + φ2Żt
= φ1E(ŻtŻt+1) + φ2E(Żt2)
= φ1γ1
+ φ2γ0
+ 0
+ at+2)]
+ E(Żtat+2)
+ 0
Then using ρk = γk /γ0 and ρ0 = 1, gives the AR(2) ACF
ρ1 = φ1
+ φ2ρ1
ρ2 = φ1ρ1
+ φ2
ρ3 = φ1ρ2
+ φ2ρ1
..
..
..
ρk = φ1ρk−1 + φ2ρk−2
5 - 25
True ACF and PACF for AR(2) Model with
φ1 = 0.78, φ2 = 0.2
True ACF
0.0
-1.0
True ACF
1.0
ar: 0.78, 0.2
0
5
10
15
20
15
20
Lag
True PACF
0.0
-1.0
True PACF
1.0
ar: 0.78, 0.2
0
5
10
Lag
5 - 26
True ACF and PACF for AR(2) Model with
φ1 = 1, φ2 = −0.95
True ACF
0.0
-1.0
True ACF
1.0
ar: 1, -0.95
0
5
10
15
20
15
20
Lag
True PACF
0.0
-1.0
True PACF
1.0
ar: 1, -0.95
0
5
10
Lag
5 - 27
Simulated Realization
(AR(2), φ1 = 0.78, φ2 = 0.2, n = 75)
Graphical Output from Function iden
Simulated data
-10
-8
-6
w
-4
-2
w= Data
0
10
20
30
40
50
60
70
time
ACF
4
0.0
-1.0
5
ACF
0.5
6
1.0
Range-Mean Plot
5
10
15
20
15
20
3
Lag
0.5
-1.0
0.0
Partial ACF
2
1.0
PACF
1
Range
0
-8
-6
-4
Mean
-2
0
5
10
Lag
5 - 28
Simulated Realization
(AR(2), φ1 = 0.78, φ2 = 0.2, n = 300)
Graphical Output from Function iden
Simulated data
4
-2
0
2
w
6
8
10
w= Data
0
50
100
150
200
250
300
time
ACF
0.0
-1.0
6
ACF
0.5
7
1.0
Range-Mean Plot
5
10
15
20
15
20
5
0
Range
Lag
0.5
0.0
-1.0
2
3
Partial ACF
1.0
4
PACF
0
2
4
Mean
6
8
0
5
10
Lag
5 - 29
Simulated Realization (AR(2), φ1 = 1, φ2 = −0.95, n = 75)
Graphical Output from Function iden
Simulated data
0
-10
-5
w
5
10
w= Data
0
10
20
30
40
50
60
70
time
ACF
15
0.0
-1.0
ACF
0.5
20
1.0
Range-Mean Plot
0
5
10
15
20
15
20
Range
Lag
0.5
0.0
-1.0
5
Partial ACF
1.0
10
PACF
-0.4
-0.2
0.0
0.2
Mean
0.4
0.6
0
5
10
Lag
5 - 30
Simulated Realization
(AR(2), φ1 = 1, φ2 = −0.95, n = 300)
Graphical Output from Function iden
Simulated data
-6
-4
-2
0
w
2
4
6
8
w= Data
0
50
100
150
200
250
300
time
ACF
0.0
10
-1.0
12
ACF
0.5
1.0
Range-Mean Plot
0
5
10
15
20
15
20
Range
Lag
4
0.5
0.0
-1.0
6
Partial ACF
1.0
8
PACF
-0.5
0.0
0.5
Mean
1.0
0
5
10
Lag
5 - 31
AR(2) Triangle Relating ρ1 and ρ2 to φ1 and φ2
5 - 32
AR(2) Triangle Showing ACF and PACF Functions
5 - 33
Module 5
Segment 4
Yule-Walker Equations and Their Applications and
AR(p) Variance
5 - 34
Yule-Walker Equations (Correlation Form)
AR(1) Yule-Walker Equation
ρ1 = φ1
AR(2) Yule-Walker Equations
ρ1 =
φ1 + ρ1φ2
ρ2 = ρ1φ1 +
φ2
AR(3) Yule-Walker Equations
ρ1 =
φ1 + ρ1φ2 + ρ2φ3
ρ2 = ρ1φ1 +
φ2 + ρ1φ3
ρ3 = ρ2φ1 + ρ1φ2 +
φ3
5 - 35
Yule-Walker Equations (Correlation Form)
AR(4) Yule-Walker Equations
ρ1 =
φ1 + ρ1φ2 + ρ2φ3 + ρ3φ4
ρ2 = ρ1φ1 +
φ2 + ρ1φ3 + ρ2φ4
ρ3 = ρ2φ1 + ρ1φ2 +
φ3 + ρ1φ4
ρ4 = ρ3φ1 + ρ2φ2 + ρ1φ3 +
φ4
For given ρ1, . . . , ρ4, this is four linear equations with four
unknowns.
5 - 36
AR(p) Yule-Walker Equations (Correlation Form)
ρ1 = φ1 + ρ1φ2 + ρ2φ3 + · · · + ρp−1φp
ρ2 = ρ1φ1 + φ2 + ρ1φ3 + · · · + ρp−2φp
..
ρp = ρp−1φ1 + ρp−2φ2 + ρp−3φ3 + · · · + φp
For given ρ1, . . . , ρp, this is p linear equations with p unknowns.
Solving the Yule Walker equations for an AR(p) for φp gives
the PACF value φpp.
5 - 37
Applications of the Yule-Walker Equations
• Provides the true ACF of an AR(p) model (given φ1, φ2, . . . , φp,
compute ρ1, ρ2, . . . ρp recursively)
• Provides estimate of φ1, φ2, . . . , φp of an AR(p) model (given
sample ACF values ρb1, ρb2, . . . , ρbp, substitute in Y-W equations and solve the set of simultaneous equations for φb1, φb2, . . . , φbp )
• Provides the sample PACF φb1,1, φb2,2, . . . , φbkk for any realization.
(substitute ρb1, ρb2, . . . , ρbk into the AR(k) Y-W equations and
solve for φbk ). Same as Durbin’s formula.
• Provides the true PACF φ1,1, φ2,2, . . . , φkk for any ARMA model.
(substitute ρ1, ρ2, . . . , ρk into the AR(k) Y-W equations and
solve for φk ). Same as Durbin’s formula.
5 - 38
Variance of the AR(p) model
Żt = φ1Żt−1
+ φ2Żt−2
+ · · · + φpŻt−p
+ at
Multiplying through by Żt gives
Żt2 = φ1ŻtŻt−1 + φ2ŻtŻt−2 + · · · + φpŻtŻt−p + Żtat
Taking expectations and noting that E(Żtat ) = σa2 gives
γ0 = Var(Zt ) = E(Żt2) = φ1γ1 + φ2γ2 + · · · + φpγp + σa2
Then dividing through by γ0 and solving for γ0 gives
γ0 =
σa2
1−φ1 ρ1 −φ2 ρ2···−φp ρp
Note:
2
)
=
σ
E(Żtat) = E(φ1Żt−1at + φ2Żt−2at + · · · + φpŻt−pat + a2
t
a
5 - 39
Module 5
Segment 5
ARMA(1,1) Model Properties
5 - 40
Properties of the ARMA(1, 1) Model
φ1(B)Zt = θ0 + θ1(B)at
Zt = θ0 + φ1Zt−1 − θ1at−1 + at
Noting that µZ = E(Zt) = θ0/(1 − φ1)
φ1(B)Żt = θ1(B)at
Żt = φ1Żt−1 − θ1at−1 + at
Omitting derivations,
2
γ0 = Var(Zt ) = [(1 − 2φ1θ1 + θ12)/(1 − φ2
)]σ
a
1
ρ1 = (1 − φ1θ1)(φ1 − θ1)/(1 + θ12 − 2φ1θ1)
ρ2 = φ1ρ1
ρk = φ1ρk−1
5 - 41
True ACF and PACF for ARMA(1, 1) Model with
φ1 = 0.95, θ1 = −0.95
True ACF
0.0
-1.0
True ACF
1.0
ma: -0.95 ar: 0.95
0
5
10
15
20
15
20
Lag
True PACF
0.0
-1.0
True PACF
1.0
ma: -0.95 ar: 0.95
0
5
10
Lag
5 - 42
Simulated Realization
(ARMA(1, 1), φ1 = 0.95, θ1 = −0.95, n = 75)
Graphical Output from Function iden
Simulated data
-10
-5
w
0
5
w= Data
0
10
20
30
40
50
60
70
time
ACF
0.0
0
5
10
15
20
15
20
8
Lag
0.5
-1.0
0.0
Partial ACF
6
1.0
PACF
4
Range
10
-1.0
12
ACF
0.5
14
1.0
Range-Mean Plot
-10
-5
0
Mean
0
5
10
Lag
5 - 43
True ACF and PACF for ARMA(1, 1) Model with
φ1 = −0.95, θ1 = 0.95
True ACF
0.0
-1.0
True ACF
1.0
ma: 0.95 ar: -0.95
0
5
10
15
20
15
20
Lag
True PACF
0.0
-1.0
True PACF
1.0
ma: 0.95 ar: -0.95
0
5
10
Lag
5 - 44
Simulated Realization
(ARMA(1, 1), φ1 = −0.95, θ1 = 0.95, n = 75)
Graphical Output from Function iden
Simulated data
0
-10
-5
w
5
10
w= Data
0
10
20
30
40
50
60
70
time
ACF
0.0
-1.0
20
ACF
0.5
1.0
Range-Mean Plot
15
5
10
15
20
15
20
Lag
0.5
0.0
Partial ACF
10
1.0
PACF
-1.0
5
Range
0
-0.4
-0.2
0.0
Mean
0.2
0.4
0
5
10
Lag
5 - 45
True ACF and PACF for ARMA(1, 1) Model with
φ1 = 0.9, θ1 = 0.5
True ACF
0.0
-1.0
True ACF
1.0
ma: 0.5 ar: 0.9
0
5
10
15
20
15
20
Lag
True PACF
0.0
-1.0
True PACF
1.0
ma: 0.5 ar: 0.9
0
5
10
Lag
5 - 46
Simulated Realization
(ARMA(1, 1), φ1 = 0.9, θ1 = 0.5, n = 75)
Graphical Output from Function iden
Simulated data
w
-4
-3
-2
-1
0
1
2
w= Data
0
10
20
30
40
50
60
70
time
ACF
0.0
-1.0
2.8
ACF
3.0
0.5
1.0
Range-Mean Plot
5
10
15
20
15
20
2.6
Lag
0.5
-1.0
0.0
Partial ACF
2.4
1.0
PACF
2.2
Range
0
-2.0
-1.5
-1.0
-0.5
Mean
0.0
0.5
0
5
10
Lag
5 - 47
True ACF and PACF for ARMA(1, 1) Model with
φ1 = −0.9, θ1 = −0.5
True ACF
0.0
-1.0
True ACF
1.0
ma: -0.5 ar: -0.9
0
5
10
15
20
15
20
Lag
True PACF
0.0
-1.0
True PACF
1.0
ma: -0.5 ar: -0.9
0
5
10
Lag
5 - 48
Simulated Realization
(ARMA(1, 1), φ1 = −0.9, θ1 = −0.5, n = 75)
Graphical Output from Function iden
Simulated data
-2
0
w
2
4
w= Data
0
10
20
30
40
50
60
70
time
ACF
0.0
5
-1.0
6
ACF
0.5
7
1.0
Range-Mean Plot
0
5
10
15
20
15
20
Range
Lag
0.5
0.0
-1.0
2
Partial ACF
3
1.0
4
PACF
0.0
0.2
0.4
Mean
0.6
0
5
10
Lag
5 - 49
ARMA(1,1) Square Relating ρ1 and ρ2 to φ1 and θ1
5 - 50
ARMA(1,1) Square Showing
ACF and PACF Functions
5 - 51
Module 5
Segment 6
Stationarity and Invertibility of the ARMA(1,1) Model
5 - 52
Stationarity of an ARMA(1, 1) Model
(1 − φ1B)Żt = (1 − θ1B)at
Żt =
(1 − θ1B)
at
(1 − φ1B)
= (1 − θ1B)(1 − φ1B)−1 at
2
= (1 − θ1B)(1 + φ1B + φ2
B
+ · · · )at
1
= (φ1 − θ1)at−1 + φ1(φ1 − θ1)at−2 + φ2
1 (φ1 − θ1)at−3 + · · · + at
∞
X
j−1
ψj at−j
φ1 (φ1 − θ1)at−j + at =
=
j=0
j=1
∞
X
j−1
where ψj = φ1
(φ1 − θ1) for j > 0, ψ0 = 1, and φ0
1 ≡ 1.
ARMA(1,1) model is stationary if −1 < φ1 < 1.
ARMA(1,1) model is white noise (trivial model) if φ1 = θ1.
5 - 53
Invertibility of an ARMA(1, 1) Model
(1 − φ1B)Żt = (1 − θ1B)at
(1 − φ1B)
at =
Żt
(1 − θ1B)
= (1 − φ1B)(1 − θ1B)−1 Żt
= (1 − φ1B)(1 + θ1B + θ12B2 + · · · )Żt
= Żt + (θ1 − φ1)Żt−1 + θ1(θ1 − φ1)Żt−2 + θ12(θ1 − φ1)Żt−3 + · · ·
Żt = (φ1 − θ1)Żt−1 + θ1(φ1 − θ1)Żt−2 + θ12(φ1 − θ1)Żt−3 + · · · + at
∞
X
j−1
πj Żt−j + at
θ1 (φ1 − θ1)Żt−j + at =
=
j=1
j=1
∞
X
j−1
where πj = θ1
(φ1 − θ1), j > 0 and π0 = 1.
ARMA(1,1) model is invertible if −1 < θ1 < 1.
5 - 54
True ACF and PACF for ARMA(1, 1) Model with
φ1 = 0.9, θ1 = 0.9
True ACF
0.0
-1.0
True ACF
1.0
ma: 0.9 ar: 0.9
0
5
10
15
20
15
20
Lag
True PACF
0.0
-1.0
True PACF
1.0
ma: 0.9 ar: 0.9
0
5
10
Lag
5 - 55
Simulated Realization
(ARMA(1, 1), φ1 = 0.9, θ1 = 0.9, n = 75)
Graphical Output from Function iden
Simulated data
-3
-2
-1
w
0
1
2
w= Data
0
10
20
30
40
50
60
70
time
ACF
0.0
-1.0
4
ACF
0.5
5
1.0
Range-Mean Plot
5
10
15
20
15
20
3
Lag
0.5
-1.0
0.0
Partial ACF
2
1.0
PACF
1
Range
0
-0.1
0.0
0.1
Mean
0.2
0.3
0
5
10
Lag
5 - 56
Properties of ARMA(p, q) model
φp(B)Zt = θ0 + θq (B)at
φp(B)Żt = θq (B)at
(1 − φ1B − φ2B2 − · · · − φp Bp)Żt = (1 − θ1B − θ2B2 − · · · − θq Bq )at
0
• µZ ≡ E(Zt ) = 1−φ θ−···−φ
p
1
• An ARMA(p, q) model is stationary if all of the roots of
φp(B) lie outside of the unit circle.
• An ARMA(p, q) model is invertible if all of the roots of θq (B)
lie outside of the unit circle.
5 - 57
Expressing an ARMA(p, q) Model as an Infinite MA
φp(B)Żt = θq (B)at
Żt = [φp (B)]−1 θq (B)at = ψ(B)at
φp(B)ψ(B) = θq (B)
• Given φp(B) and θq (B), solve for ψ(B), giving ψ1, ψ2, . . . .
• Similar method for expressing an ARMA(p, q) model as an
infinite AR
φp(B) = π(B)θq (B)
Given φp(B) and θq (B), solve for π(B), giving π1, π2, . . . .
5 - 58
Expressing and ARMA(p, q) Model as an Infinite MA
Using ARMA(2,2) as a particular example
φ2(B)ψ(B) = θ2(B)
(1 − φ1B − φ2B2)(1 + ψ1B + ψ2B2 + ψ3B3 + · · · ) = (1 − θ1B − θ2B2)
Multiplying out gives
1 + ψ1 B
+ψ2B2
+ψ3B3 + · · ·
−φ1B −ψ1φ1B2 −ψ2φ1B3 − · · ·
−φ2B2 −ψ1φ2B3 − · · · = 1 − θ1B − θ2B2
Equating terms with the same power of B gives
B:
B2 :
ψ1 − φ1 = −θ1 → ψ1 = −θ1 + φ1
ψ2 − ψ1φ1 − φ2 = −θ2 → ψ2 = −θ2 + ψ1φ1 + φ2
B3 :
ψ3 − ψ2φ1 − ψ1φ2 =
..
..
Bk : ψk − ψk−1φ1 − ψk−2φ2 =
0 → ψ3 = ψ2φ1 + ψ1φ2
..
0 → ψk = ψk−1φ1 + ψk−2φ2
5 - 59
Module 5
Segment 7
ARMA Identification Exercises
5 - 60
Simulated Time Series #5
Graphical Output from Function iden(simsta5.d)
Simulated Time Series #5
-10
0
w
10
20
w= Sales
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
time
ACF
0.0
-1.0
ACF
25
0.5
1.0
Range-Mean Plot
20
0
10
20
30
Range
Lag
10
0.5
0.0
-1.0
Partial ACF
15
1.0
PACF
0
2
Mean
4
6
0
10
20
30
Lag
5 - 61
Simulated Time Series #6
Graphical Output from Function iden
Simulated Time Series #6
w
-40
-30
-20
-10
0
10
20
w= Sales
20
30
40
50
60
time
ACF
0.0
-1.0
30
ACF
35
0.5
1.0
Range-Mean Plot
10
20
30
25
Lag
0.5
0.0
-1.0
15
Partial ACF
1.0
20
PACF
10
Range
0
-20
-10
0
Mean
10
0
10
20
30
Lag
5 - 62
Simulated Time Series #7
Graphical Output from Function iden
Simulated Time Series #7
-40
-20
0
w
20
40
60
w= Deviation
20
30
40
50
time
ACF
0.0
-1.0
80
ACF
0.5
1.0
100
Range-Mean Plot
10
20
30
60
Lag
0.5
-1.0
0.0
Partial ACF
40
1.0
PACF
20
Range
0
-20
-10
0
10
Mean
20
30
40
0
10
20
30
Lag
5 - 63
Simulated Time Series #8
Graphical Output from Function iden
Simulated Time Series #8
w
-6
-4
-2
0
2
4
w= decibles
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
time
ACF
0.0
-1.0
5
ACF
0.5
6
1.0
Range-Mean Plot
10
20
30
4
Lag
0.5
0.0
Partial ACF
3
1.0
PACF
-1.0
2
Range
0
-4
-3
-2
-1
Mean
0
1
0
10
20
30
Lag
5 - 64
Simulated Time Series #11
Graphical Output from Function iden
Simulated Time Series #11
-6
-4
-2
w
0
2
4
w= Deviation
10
20
time
ACF
0.0
-1.0
10
ACF
12
0.5
1.0
Range-Mean Plot
10
20
30
8
Lag
0.5
0.0
Partial ACF
6
1.0
PACF
-1.0
4
Range
0
-0.3
-0.2
-0.1
0.0
Mean
0.1
0.2
0
10
20
30
Lag
5 - 65
Simulated Time Series #13
Graphical Output from Function iden
Simulated Time Series #13
-2
0
2
w
4
6
w= Pressure
10
20
time
ACF
0.0
4
-1.0
5
ACF
0.5
1.0
Range-Mean Plot
0
10
20
30
Range
Lag
0.5
0.0
-1.0
2
Partial ACF
1.0
3
PACF
0
2
4
Mean
6
0
10
20
30
Lag
5 - 66
Simulated Time Series #15
Graphical Output from Function iden
Simulated Time Series #15
w
-2
-1
0
1
2
3
w= Pressure
10
20
time
ACF
3.5
0.0
-1.0
ACF
4.0
0.5
1.0
Range-Mean Plot
10
20
30
3.0
Lag
0.5
0.0
Partial ACF
2.5
1.0
PACF
-1.0
2.0
Range
0
-1.0
-0.5
0.0
0.5
Mean
1.0
1.5
0
10
20
30
Lag
5 - 67