You are interested in µ , the mean of a distribution which is difficult to work with.
You have generated a random sample, x1 ,..., x100 , from that distribution. The sample
2
standard deviation of x1 ,..., x100 is 0.27418, and the average of x12 ,..., x100
is 0.21479.
Calculate an unbiased point estimate and 95% confidence interval for µ .
(a)
[10 marks]
(b)
If you wish to construct a 95% confidence interval for µ with a width of only
0.05, how many more values will you need to sample from the distribution?
[5 marks]
(a)
Observe that s 2 =
1  J 2
2
 ∑ x j − Jx  where J = 100.
J − 1  j =1

Thus, by the method of Monte Carlo, an unbiased estimate of µ is
x=
1 J 2  J −1  2
2
 99 
xj − 
∑
 s = 0.21479 − 
 ( 0.27418) = 0.37466 .
J j =1
 J 
 100 
So an approximate 95% confidence interval for µ is
( x ± 1.96 s / J ) = (0.37466 ± 1.96 × 0.27418 / 100) = (0.3209, 0.4284) .
(b)
To obtain a 95% CI which has a width of only 0.05, we need a total sample
2
size of J = (1.96 )
2
( 0.27418 ) = 462.0656,
or 463 after rounding up.
(0.05 / 2) 2
So, in addition to the 100 values already in hand, 363 more values will be needed.
Consider the time series process xt = α xt −3 + et + β et − 2 , where et ~ iid N (0, σ 2 ) .
R Code for Problem 6 (not part of the solution, just for interest)
You may assume that this time series process is stationary, causal and invertible.
(a)
Forecast xn +1 using the following data and estimates obtained from a
realisation of the time series process:
( xn −5 , xn − 4 , ..., xn ) = (−0.8, 0.3, 1.2, − 0.4, 0.2, 1.5)
αˆ = −0.68 , βˆ = 0.31 , eˆn −3 = 0.19 .
x1
s12
s22
x2
x3
s32
/
Êm
x
28 /15
46 / 45
23 / 30
ˆ ˆ m Zx
ˆ
3
[10 marks]
s2
∞
(b)
Find the coefficients ψ j in the causal form of the process, i.e. xt = ψ j et − j ,
j =0
and then use this causal form to derive a formula for Vxt (the variance of xt ).
Then calculate Vxt if α = 1/2, β = −1/ 3 and σ 2 = 10.
(c)
[10 marks]
Without using results in (b), derive the autocovariance and autocorrelation
functions γ k = Cov( xt + k , xt ) and ρ k = Corr ( xt + k , xt ) for all lags k = 0,1,2,....
Then apply your results to estimate α , β and σ 2 using the following
statistics from a realisation of the time series process of length n = 100:
x=
1 n
 xt = −0.108 ,
n t =1
r1 =
tn= 2 ( xt − x )( xt −1 − x )
tn=3 ( xt − x )( xt − 2 − x )
0.12
=
−
r
=
= 0.46 .
,
2
tn=1 ( xt − x ) 2
tn=1 ( xt − x ) 2
s2 =
1 n
2
( xt − x ) = 6.304

n − 1 t =1
[20 marks]
(Hint: The formulae that you derive for Vxt in (b) and γ 0 in (c) should be the same,
but the mathematical arguments used to derive those formulae should be different.)
Problem 9
[10 marks]
[To be attempted only by graduate students]
Refer to Problem 8. Find the coefficients φ j in the invertible form of the process,
∞
i.e. et =  φ j xt − j , and then use this invertible form to forecast xn +1 using the
j =0
following data and estimates obtained from a realisation of the time series process:
( xn − 4 , ..., xn ) = (−0.5, 1.6, 0.9, − 1.2, 0.3) , αˆ = 0.3 , βˆ = 0.5 .
For the purposes of this exercise, you may suppose that xt = 0 for all t < n − 4 .
END OF EXAMINATION
(a)
Observe that xn −1 = α xn − 4 + en −1 + β en −3 . Thus we may estimate en −1 by
eˆn −1 = xn −1 − αˆ xn − 4 − βˆ eˆn −3 = 0.2 − (−0.68) × 0.3 − 0.31× 0.19 = 0.3451.
Hence we may predict xn +1 = α xn − 2 + en +1 + β en −1 by
xˆn +1 = αˆ xn − 2 + eˆn +1 + βˆ eˆn −1 = −0.68 × (−0.4) + 0 + 0.31× 0.3451 = 0.3790 .
(b)
Observe that
xt = et + β et − 2 + α xt −3
= et + β et − 2 + α (et −3 + β et −5 + α xt − 6 )
= et + β et − 2 + α (et −3 + β et −5 + α [et − 6 + β et −8 + α xt −9 ])
= ( et + β et − 2 ) + α ( et −3 + β et −5 ) + α 2 ( et −6 + β et −8 ) + α 3 xt −9
= .....................................................................................
= ( et + α et −3 + α 2 et − 6 + ...) + β ( et − 2 + α et −5 + α 2 et −8 + ...) .
Equating this with xt = ψ 0 et + ψ 1et −1 + ψ 2 et − 2 + ... we see that:
ψ 0 = 1 , ψ 3 = α , ψ 6 = α 2 , ψ 9 = α 3 , ...
ψ 1 = ψ 4 = ψ 7 = ψ 10 = ... = 0
ψ 2 = β , ψ 5 = βα , ψ 8 = βα 2 , ψ 11 = βα 3 , ...
Hence also Vxt = (σ 2 + α 2σ 2 + α 4σ 2 + ...) + β 2 (σ 2 + α 2σ 2 + α 4σ 2 + ...)
= σ 2 (1 + α 2 + α 4 + ...) + β 2σ 2 (1 + α 2 + α 4 + ...)
2
2  1+ β 
1 
 1 
2 2
=
Vx
σ
;
that
is,
=σ2 
+
β
σ

t
2  .

2 
2 
 1−α 
 1−α 
 1−α 
For the case where α = 1/2, β = −1/ 3 and σ 2 = 10:
 1 + (−1/ 3)2  400
=
Vxt = 10 
= 14.815.
2 
 1 − (1/ 2)  27
(c)
Observe that
γ 0 = C ( xt , xt ) = C ( xt , α xt −3 + et + β et − 2 )
= α C ( xt , xt −3 ) + C ( xt , et ) + β C ( xt , et − 2 )
= αγ 3 + C ( xt , et ) + β C ( xt , et − 2 ) ,
where: C ( xt , et ) = C (et , α xt −3 + et + β et − 2 )
= α C (et , xt −3 ) + C (et , et ) + β C (et , et − 2 ) = 0 + σ 2 + 0
C ( xt , et − 2 ) = C (et − 2 , α xt −3 + et + β et − 2 )
= α C (et − 2 , xt −3 ) + C (et − 2 , et ) + β C (et − 2 , et − 2 ) = 0 + 0 + βσ 2
Also: γ 1 = C ( xt , xt +1 ) = C ( xt , α xt − 2 + et +1 + β et −1 )
= α C ( xt , xt − 2 ) + C ( xt , et +1 ) + β C ( xt , et −1 )
= αγ 2 + 0 + β C ( xt , et −1 ) ,
where C ( xt , et −1 ) = C (et −1 , α xt −3 + et + β et − 2 )
= α C (et −1 , xt −3 ) + C (et −1 , et ) + β C (et −1 , et − 2 ) = 0 + 0 + 0 = 0.
Furthermore: γ 2 = C ( xt , xt + 2 ) = C ( xt , α xt −1 + et + 2 + β et ) = αγ 1 + 0 + σ 2
γ 3 = C ( xt , xt +3 ) = C ( xt , α xt + et +3 + β et +1 ) = αγ 0 + 0 + 0 .
In summary so far:
γ 0 = αγ 3 + σ 2 + β 2σ 2 ,
γ 1 = αγ 2 ,
γ 2 = αγ 1 + σ 2 ,
γ 3 = αγ 0 .
These four equations in four unknowns can easily be solved to get:
 1+ β 2 
(in agreement with Vxt as obtained in (a))
2 
 1−α 
γ0 =σ 2 
 αβ 
,
2 
 1−α 
γ1 = σ 2 
 β 
,
2 
 1−α 
γ2 =σ 2 
 α (1 + β 2 ) 
.
2
 1−α 
γ3 =σ 2 
Now observe that γ 3 = αγ 0 , and also:
γ 4 = C ( xt , xt + 4 ) = C ( xt , α xt +1 + et + 4 + β et + 2 ) = αγ 1 + 0 + 0
γ 5 = C ( xt , xt +5 ) = C ( xt , α xt + 2 + et + 5 + β et +3 ) = αγ 2 + 0 + 0
γ 6 = C ( xt , xt + 6 ) = C ( xt , α xt +3 + et + 6 + β et + 4 ) = αγ 3 + 0 + 0 , etc.
We see that γ k = αγ k −3 for all k = 3,4,5,6,7,...
Thus the autocovariance and autocorrelation functions are given by:
2

2  1+ β 
,
σ


2 
1
α
−




 αβ 
,
 σ2
2 
γk = 
 1−α 

 σ 2  β  ,
2

 1−α 

αγ k −3 ,


1,




αβ


,
k =1
γ k  1+ β 2

=
 and ρ k =
β
γ
0


,

 1+ β 2
k =2


 αρ k −3 ,

k = 3, 4,5,...
We may now equate ρ1 =
r
r2
αˆ = 1 =
k =0



k =1

.

k =2


k = 3, 4,5,...
k =0
αβ
β
with r1 and ρ 2 =
with r2 to get:
2
1+ β
1+ β 2
−0.12
= −0.2609 and
0.46
r2 β 2 − β + r2 = 0 ⇒ β =
1 ± 12 − 4r22
= 0.66096, 1.51295 ⇒ β̂ = 0.66096 .
2r2
(Note: For β ≠ 0 , the MA polynomial β (λ ) = 1 + βλ 2 has roots λ = ±i /
β , for
which λ = 1 β . Thus the invertibility condition λ > 1 is equivalent to β < 1 .)
 1 − αˆ 2 
1 − αˆ 2
= 0.6485971
Finally, we may estimate σ by σˆ = γˆ0 
, where
2 
1 + βˆ 2
 1 + βˆ 
2
2
and γˆ0 is some estimate of γ 0 . Three reasonable choices for this estimate are:
γˆ0 =
1 n
2
( xt − x ) = s 2 = 6.304
∑
n − 1 t =1
γˆ0 =
1 n
n −1  2
2
( xt − x ) = 
∑
 s = 6.24096
n t =1
 n 
γˆ0 =
1 n
1
n −1  2
2
2
( xt − 0 ) = ( (n − 1) s 2 + nx 2 ) = 
∑
 s + x = 6.252624.
n t =1
n
n


The corresponding estimates of σ 2 are:
σˆ 2 = 6.304 × 0.6485971 = 4.089
σˆ 2 = 6.24096 × 0.6485971 = 4.048
σˆ 2 = 6.252624 × 0.6485971 = 4.055 .
1 n
1 n
1 n
 1 n
(Note: E  ∑ xt2  = ∑ Ext2 = ∑ Vxt ∑ γ 0 = γ 0 . So the third estimate of γ 0 ,
n t =1
n t =1
 n t =1  n t =1
γˆ0 = 6.252624, is unbiased. All 3 approaches here will be marked as fully correct.)
R Code for Problem 8 (not part of the solution, just for interest)
Solution to Problem 9
Observe that et = xt − α xt −3 − β et − 2
= xt − α xt −3 − β ( xt − 2 − α xt −5 − β et − 4 )
= xt − α xt −3 − β ( xt − 2 − α xt −5 − β {xt − 4 − α xt −7 − β et −6 })
= xt − α xt −3 − β ( xt − 2 − α xt −5 − β {xt − 4 − α xt − 7 − β [ xt − 6 − α xt −9 − β et −8 ]})
= .........................................................................................................
= xt + 0 xt −1 − β xt − 2 − α xt −3 + β 2 xt − 4 + αβ xt −5 − β 3 xt − 6 − αβ 2 xt − 7 + ...
We see that:
φ0 = 1 , φ1 = 0
φ2 = − β , φ4 = β 2 , φ6 = − β 3 , ...
φ3 = −α , φ5 = αβ , φ7 = −αβ 2 , ...
⎧⎪ form
− with t≤= n + 1, we obtain
Rearranging the invertible
= ⎪⎨
⎪
ˆ 2 x − αβ
ˆ ˆ xn − 4 + βˆ 3 xn −5 + αβ
ˆ ˆ 2 xn − 6 − ...
xˆn +1 = eˆn +1 + βˆ⎪⎩xn −1 −
+ αˆ xn − 2 − β>
n −3
= 0 + 0.5(−1.2) + 0.3(0.9) − 0.52 (1.6) − 0.3(0.5)(−0.5) + 0 + 0 + ...
= −0.655 .
Consider a time series x = ( x1 ,..., xn ) = (1.6, −0.6,1.8,0.2) which follows an AR(1)
process of the form xt = α + θ xt−1 + wt , where wt ~ iid N (0, σ 2 ) and | θ |< 1 .
(a)
Express the time series as an infinite moving average process. Hence,
or otherwise, derive the mean, variance, autocovariance at lag 1, and
autocorrelation at lag 1, of the process. Calculate these four quantities
if it can be assumed that α = 0.3 , θ = −0.5 and σ 2 = 4.
[10 marks]
(b)
Using formulae derived in (a) and the available data, apply the method of
moments so as to estimate the parameters α , θ and σ 2 .
(Proceed mathematically as if n were 'large'.)
[10 marks]
(c)
Using the parameter values assumed in (a), forecast the time series forward
one-step-ahead and ten-steps-ahead. Then recalculate these two forecasts
using the parameter estimates in (b).
[10 marks]
(a)
xt = α + θ xt−1 + wt
= α + θ (α + θ xt−2 + wt−1 ) + wt
= (1 + θ )α + θ 2 xt−2 + wt + θ wt−1
= (1 + θ )α + θ 2 (α + θ xt−3 + wt−2 ) + wt + θ wt−1
= (1 + θ + θ 2 )α + θ 3 xt−3 + wt + θ wt−1 + θ 2 wt−2
...............................................................................................
= (1 + θ + θ 2 + ...)α + θ ∞ xt−∞ + ( wt + θ wt−1 + θ 2 wt−2 + ...) .
α
+ wt + θ wt−1 + θ 2 wt−2 + ... .
Thus the required MA(∞) form is xt =
1− θ
α
0.3
So:
=
= 1/5 = 0.2 (mean of the process)
µ = Ext =
1 − θ 1 − (−0.5)
Vxt = σ 2 + θ 2σ 2 + θ 4σ 2 + ... =
σ2
4
16
=
=
= 5.3333
2
2
1− θ
1 − (−0.5)
3
(variance of the process)
C ( xt+1 , xt ) = C ( wt+1 + θ wt + θ wt−1 + ..., wt + θ wt−1 + θ 2 wt−2 + ...)
2
= θC ( wt , wt ) + θ 3C ( wt−1 , wt−1 ) + ...
= θσ 2 (1 + θ 2 + θ 4 + ...)
θσ 2
−0.5× 4
8
=
=
= − = –2.6667.
2
2
1− θ
1 − (−0.5)
3
(Alternatively, Corr ( xt+1 , xt ) = θ , from basic properties of the AR(1) process.
Therefore, C ( xt+1 , xt ) = Corr ( xt+1 , xt ) ×Vxt = −0.5(16 / 2) = −8 / 3 .)
Finally, Corr ( xt+1 , xt ) = θ = –0.5.
1 n
∑ ( xt − x )2 = 0.9875,
n t=1
1 n
g
g1 = ∑ ( xt − x )( xt−1 − x ) = -0.785625 and r1 = 1 = -0.7955696.
g0
n t =2
So the MOM estimate of θ is θˆ = r = –0.7956.
(b)
We find that: x = 0.75, g0 =
1
Also the MOME of µ is µˆ = x = 0.75, and so the MOME of α is
αˆ = µˆ (1− θˆ) = 1.3467. Finally, the MOME of σ 2 is σˆ 2 = g (1 − θˆ 2 ) = 0.3625.
0
(c)
Using the given parameter values:
xˆ5 = α + θ x4 = 0.3 + (−0.5)0.2 = 0.2
xˆ6 = α + θ xˆ5 = 0.3 + (−0.5)0.2 = 0.2 (the same)
..............................................................................
x̂14 = = 0.2 (also).
Using the estimates derived in (b):
xˆ5 = αˆ + θˆ x4 = 1.3467 + (−0.7956)0.2 = 1.1876.
xˆ = αˆ + θˆ x = αˆ + θˆ(αˆ + θˆ x ) = αˆ (1 + θˆ) + θˆ 2 x
6
5
4
4
................................................................................
xˆ14 = αˆ (1 + θˆ + θˆ 2 + .. + θˆ9 ) + θˆ10 x4
⎛1 − θˆ10 ⎞⎟ 10
⎟ + θˆ x4 = 0.6941.
= αˆ ⎜⎜⎜
⎜⎝ 1 − θˆ ⎠⎟⎟
(Note: The m-step-ahead forecast converges to x = 0.75 as m tends to infinity.)
Consider a time series x = ( x1 ,..., xn ) = (0.6, −1.0, −0.8, 0.4, −0.5,1.1) which follows
an MA(1) process of the form xt = wt + φ wt−1 , where wt ~ iid N (0, σ 2 ) and | φ | < 1 .
(a)
Apply the method of moments so as to estimate the parameters φ and σ .
Obtain the estimates by equating the autocovariance with the sample
autocovariance at lags 0 and 1, respectively. Use the definition of the
sample autocovariance as given in the formula sheet.
(b)
[10 marks]
Suppose that φ = 0.1 and σ = 0.5. Predict xn+1 and provide
an approximate 95% prediction interval for xn+1 .
[10 marks]
(a) Observe that:
γ 0 = Vxt = (1 + φ 2 )σ 2
γ 1 = C ( xt +1 , xt ) = C ( wt +1 + φ wt , wt + φ wt −1 ) = φσ 2
γ
φ
xij
ρ1 = 1 =
.
2
γ 0 1+φ
Also:
x = -0.033333
1 n
c0 = ∑ ( xt − x ) 2 = 0.60222
=
µ
σ
−
n t =1
µ
= +
=
σ = −
=
1 n=
c1 = ∑ ( xt − x )( xt −1 − x ) = -0.15574
n t =2
⎛ ⎞
λ = ⎜⎜ ⎟⎟⎟ =
=
c1
⎜⎝ ⎠⎟
r1 = = -0.25861.
c0
⎛σ
σ ⎞⎟
⎟
= λ ⎜⎜ +
⎜⎝ µ
µ ⎠⎟⎟
1 ± 1 − 4r12
Equating ρ1 = r1=implies r1φ 2 − φ + r1 = 0 and φ =
= -0.27870, -3.58813.
2r1
So φˆ = −0.2787. Then γ 0 = c0 yields σˆ 2 = c1 / φˆ = 0.5588 and so σˆ = 0.74754.
=
=
>
(b) wt = −φwt−1 + xt = −φ(−φ wt−2 + xt−1 ) + xt = −φ (−φ(−φ wt−3 + xt−2 ) + xt−1 ) + xt
= ... = xt − φ xt−1 + φ 2 xt−2 − φ 3 xt−3 + ...
So xt = wt + φ xt−1 − φ 2 xt−2 + φ 3 xt −3 − ...
So xn+1 = wn+1 + φ xn − φ 2 xn−1 + φ 3 xn−2 − ...
≈ wn+1 + φ xn − φ 2 xn−1 + φ 3 xn−2 − ... + (−1) n+1 φ n x1
since φ is 'small'.
Thus: E ( xn+1 | x) ≈ 0 + φ xn − φ 2 xn−1 + φ 3 xn−2 − ... + (−1) n+1 φ n x1
V ( xn+1 | x) ≈ Vwn+1 = σ 2 .
So we may predict xn+1 = x7 by
xˆ7 = 0.1x6 − 0.01x5 + 0.001x4 − ... − 0.000001x1
= 0.1(1.1) − 0.01(−0.5) + 0.001(0.4) − ... − 0.000001(0.6) = 0.1155.
Also, a 95% prediction interval for x7 is
( xˆ7 ± 1.96σ ) = (−0.865,1.096).
Consider the stationary AR(2) time series process
Note that another way to show the working for x̂7 is to write
xt = β xt−2 + wt ,
ŵ6
2
where w
ˆ
0.1wˆ 5
ŵt7 ~ iid N (0, σ ) .
ˆ
0.1wˆ 4
ˆ
0.1wˆβ3 . Justify your answer using an argument
(a) Find the range of possible values for
ˆ 0.1wˆ 2
which involves the unit circle in the complex plane and roots of a polynomial.
ˆ
1
0.1wˆ1
[10 marks]
ˆ 0.1wˆ
0
ˆ 6 the last
0.1three
wˆ 5 values of a time series which follows the above AR(2)
(b) Suppose wthat
)
0.01wˆ 4 that the sample
) =wˆ 4(2.4,
3.8, 2.7). Also suppose
process are given by ( x , x , x 0.1
48
49
50
0.1wˆ 3 )
0.001wˆ 3
autocorrelations at lags 1 and 2 are r1 = 0.04 and r2 = 0.22, respectively. Calculate a
0.1wˆ 2 )
0.0001wˆ 2
0.1wˆof
00001
wˆ1 using an
simple forecast of x51 based on the method
answer
1 ) moments. Justify your
0.1wˆ )
000001wˆ
argument which includes a derivation of the0 autocorrelation function for the 0process.
154694 0
[10 marks]
0.1155
(a) The time series process is β ( B) xt = wt , where β ( B) = 1− β B 2 is the AR operator
and B is the backward shift operator. So the AR polynomial is β (λ ) = 1− βλ 2 .
Now, if β = 0 then xt = wt , and so the process is automatically stationary.
Otherwise, we need to find the roots of β (λ ) , i.e. the solutions in λ of 1− βλ 2 = 0 .
Case 1: Suppose that β > 0 . Then the roots are λ = ±1/ β , each with modulus
| λ | = 1/ β . Thus | λ | > 1 ⇔ β < 1 . That is, all roots of the AR polynomial lie outside
the unit circle (and so equivalently the process is stationary) if and only if β < 1 .
Case 2: Suppose that β < 0 . Then the roots are λ = ±i / −β (where i = −1 ), each
with modulus | λ | = 1/ −β . Thus | λ | > 1 ⇔ −β < 1 ⇔ β > −1 . That is, all roots of
the AR polynomial lie outside the unit circle if and only if β > −1 .
In summary, the process is stationary if and only if β is in the interval (-1,1) .
Note: It can be shown that the AR(2) process xt = β1 xt−1 + β2 xt−2 + wt is stationary iff
(β1 , β2 ) lies strictly inside the triangle formed by the points (-2,-1), (2,-1) and (0,1).
This result implies that, for β1 fixed at 0, the condition for stationarity is −1 < β2 < 1 .
(b) Assuming that −1 < β < 1 , we have that
xt = β (β xt−4 + wt−2 ) + wt = β (β ( xt−6 + wt −4 ) + wt −2 ) + wt = ...
= wt + β wt−2 + β 2 wt−4 + ...
So:
γ 0 = Vxt = σ 2 (1 + β 2 + β 4 + ...) = σ 2 / (1− β 2 )
γ1 = C ( xt +1 , xt ) = C ( wt +1 + β wt−1 + β 2 wt−3 + ..., wt + β wt−2 + β 2 wt−4 + ...) = 0
γ 2 = C ( xt +2 , xt ) = C ( wt +2 + β wt + β 2 wt−2 + ..., wt + β wt−2 + β 2 wt−4 + ...)
= σ 2 (β + β 3 + β 5 + ...) = βσ 2 / (1− β 2 )
γ 3 = C ( xt +3 , xt ) = C ( wt +3 + β wt +1 + β 2 wt−1 + ..., wt + β wt−2 + β 2 wt−4 + ...) = 0
γ 4 = C ( xt +4 , xt ) = C ( wt +4 + β wt +2 + β 2 wt + ..., wt + β wt −2 + β 2 wt−4 + ...)
= σ 2 (β 2 + β 4 + β 6 + ...) = β 2σ 2 / (1− β 2 ) , etc.
Thus, ρ1 = γ1 / γ 0 = 0 , ρ2 = γ 2 / γ 0 = β , ρ3 = γ3 / γ 0 = 0 , ρ4 = γ 4 / γ 0 = β 2 , etc.
Equating ρ = r we get βˆ = r = 0.22 . So we forecast x by
2
2
2
xˆ51 = βˆ x49 + wˆ 51 = 0.22×3.8 + 0 = 0.836 .
51