Exponential smoothing and non-negative data
Exponential smoothing
and non-negative data
Muhammad Akram
Rob J Hyndman
J Keith Ord
Business & Economic Forecasting Unit
1
Exponential smoothing and non-negative data
Outline
1
Exponential smoothing models
2
Problems with some of the models
3
A new model for positive data
4
Conclusions
2
Exponential smoothing and non-negative data
Exponential smoothing models
Outline
1
Exponential smoothing models
2
Problems with some of the models
3
A new model for positive data
4
Conclusions
3
Exponential smoothing and non-negative data
Exponential smoothing models
Problem
Most forecasting methods in business are
based on exponential smoothing.
4
Exponential smoothing and non-negative data
Exponential smoothing models
Problem
Most forecasting methods in business are
based on exponential smoothing.
Most time series in business are inherently
non-negative.
4
Exponential smoothing and non-negative data
Exponential smoothing models
Problem
Most forecasting methods in business are
based on exponential smoothing.
Most time series in business are inherently
non-negative.
But exponential smoothing
models for non-negative
data don’t always work!
4
Exponential smoothing and non-negative data
Exponential smoothing models
4
Problem
Most forecasting methods in business are
based on exponential smoothing.
Most time series in business are inherently
non-negative.
But exponential smoothing
models for non-negative
data don’t always work!
They can produce
negative forecasts
0
1
10
20
30
Forecasts from ETS(A,A,N)
0
5
10
15
20
Time
25
30
35
Exponential smoothing and non-negative data
Exponential smoothing models
4
Problem
Most forecasting methods in business are
based on exponential smoothing.
Most time series in business are inherently
non-negative.
But exponential smoothing
models for non-negative
data don’t always work!
0
2
They can produce
negative forecasts
They can produce infinite
forecast variance
−5
1
5
10
Forecasts from ETS(A,M,N)
0
5
10
15
Time
20
25
30
Exponential smoothing and non-negative data
Exponential smoothing models
4
Problem
3
y
20
10
2
They can produce
negative forecasts
They can produce infinite
forecast variance
They can converge almost
surely to zero.
0
1
30
40
Most forecasting methods in business are
based on exponential smoothing.
Most time series in business are inherently
non-negative.
But exponential smoothing
models for non-negative
Simulation from ETS(M,N,N)
data don’t always work!
0
500
1000
Time
1500
2000
Exponential smoothing and non-negative data
Exponential smoothing models
Taxonomy of models
Trend
Component
Seasonal Component
N
A
M
(None) (Additive) (Multiplicative)
N
(None)
N,N
N,A
N,M
A
(Additive)
A,N
A,A
A,M
Ad
(Additive damped)
Ad ,N
M
(Multiplicative)
M,N
Ad ,A
M,A
Ad ,M
M,M
Md
(Multiplicative damped)
Md ,N
Md ,A
Md ,M
5
Exponential smoothing and non-negative data
Exponential smoothing models
Taxonomy of models
Trend
Component
Seasonal Component
N
A
M
(None) (Additive) (Multiplicative)
N
(None)
N,N
N,A
N,M
A
(Additive)
A,N
A,A
A,M
Ad
(Additive damped)
Ad ,N
M
(Multiplicative)
M,N
Ad ,A
M,A
Ad ,M
M,M
Md
(Multiplicative damped)
Md ,N
Md ,A
Md ,M
General notation ETS(Error,Trend,Seasonal)
5
Exponential smoothing and non-negative data
Exponential smoothing models
Taxonomy of models
Trend
Component
Seasonal Component
N
A
M
(None) (Additive) (Multiplicative)
N
(None)
N,N
N,A
N,M
A
(Additive)
A,N
A,A
A,M
Ad
(Additive damped)
Ad ,N
M
(Multiplicative)
M,N
Ad ,A
M,A
Ad ,M
M,M
Md
(Multiplicative damped)
Md ,N
Md ,A
Md ,M
General notation ETS(Error,Trend,Seasonal)
ExponenTial Smoothing
5
Exponential smoothing and non-negative data
Exponential smoothing models
Taxonomy of models
Trend
Component
Seasonal Component
N
A
M
(None) (Additive) (Multiplicative)
N
(None)
N,N
N,A
N,M
A
(Additive)
A,N
A,A
A,M
Ad
(Additive damped)
Ad ,N
M
(Multiplicative)
M,N
Ad ,A
M,A
Ad ,M
M,M
Md
(Multiplicative damped)
Md ,N
Md ,A
Md ,M
General notation ETS(Error,Trend,Seasonal)
ExponenTial Smoothing
ETS(A,N,N): Simple exponential smoothing with
additive errors
5
Exponential smoothing and non-negative data
Exponential smoothing models
Taxonomy of models
Trend
Component
Seasonal Component
N
A
M
(None) (Additive) (Multiplicative)
N
(None)
N,N
N,A
N,M
A
(Additive)
A,N
A,A
A,M
Ad
(Additive damped)
Ad ,N
M
(Multiplicative)
M,N
Ad ,A
M,A
Ad ,M
M,M
Md
(Multiplicative damped)
Md ,N
Md ,A
Md ,M
General notation ETS(Error,Trend,Seasonal)
ExponenTial Smoothing
ETS(A,A,N): Holt’s linear method with additive
errors
5
Exponential smoothing and non-negative data
Exponential smoothing models
Taxonomy of models
Trend
Component
Seasonal Component
N
A
M
(None) (Additive) (Multiplicative)
N
(None)
N,N
N,A
N,M
A
(Additive)
A,N
A,A
A,M
Ad
(Additive damped)
Ad ,N
M
(Multiplicative)
M,N
Ad ,A
M,A
Ad ,M
M,M
Md
(Multiplicative damped)
Md ,N
Md ,A
Md ,M
General notation ETS(Error,Trend,Seasonal)
ExponenTial Smoothing
ETS(A,A,A): Additive Holt-Winters’ method with
additive errors
5
Exponential smoothing and non-negative data
Exponential smoothing models
Taxonomy of models
Trend
Component
Seasonal Component
N
A
M
(None) (Additive) (Multiplicative)
N
(None)
N,N
N,A
N,M
A
(Additive)
A,N
A,A
A,M
Ad
(Additive damped)
Ad ,N
M
(Multiplicative)
M,N
Ad ,A
M,A
Ad ,M
M,M
Md
(Multiplicative damped)
Md ,N
Md ,A
Md ,M
General notation ETS(Error,Trend,Seasonal)
ExponenTial Smoothing
ETS(M,A,M): Multiplicative Holt-Winters’ method
with multiplicative errors
5
Exponential smoothing and non-negative data
Exponential smoothing models
Taxonomy of models
Trend
Component
Seasonal Component
N
A
M
(None) (Additive) (Multiplicative)
N
(None)
N,N
N,A
N,M
A
(Additive)
A,N
A,A
A,M
Ad
(Additive damped)
Ad ,N
Ad ,A
M
(Multiplicative)
M,N
M,A
Ad ,M
M,M
Md
(Multiplicative damped)
Md ,N
Md ,A
Md ,M
General notation ETS(Error,Trend,Seasonal)
ExponenTial Smoothing
ETS(A,Ad ,N): Damped trend method with additive errors
5
Exponential smoothing and non-negative data
Exponential smoothing models
Taxonomy of models
Trend
Component
Seasonal Component
N
A
M
(None) (Additive) (Multiplicative)
N
(None)
N,N
N,A
N,M
A
(Additive)
A,N
A,A
A,M
Ad
(Additive damped)
Ad ,N
M
(Multiplicative)
M,N
Ad ,A
M,A
Ad ,M
M,M
Md
(Multiplicative damped)
Md ,N
Md ,A
Md ,M
General notation ETS(Error,Trend,Seasonal)
ExponenTial Smoothing
There are 30 separate models in the ETS
framework
5
Exponential smoothing and non-negative data
Exponential smoothing models
Innovations state space model
No trend or seasonality
and multiplicative errors
Example: ETS(M,N,N)
yt = `t−1 (1 + εt )
`t = αyt + (1 − α)`t−1
0≤α≤1
εt is white noise with mean zero.
6
Exponential smoothing and non-negative data
Exponential smoothing models
Innovations state space model
No trend or seasonality
and multiplicative errors
Example: ETS(M,N,N)
yt = `t−1 (1 + εt )
`t = `t−1 (1 + αεt )
0≤α≤1
εt is white noise with mean zero.
6
Exponential smoothing and non-negative data
Exponential smoothing models
Innovations state space model
No trend or seasonality
and multiplicative errors
Example: ETS(M,N,N)
yt = `t−1 (1 + εt )
`t = `t−1 (1 + αεt )
0≤α≤1
εt is white noise with mean zero.
All exponential smoothing models can be
written using analogous state space
equations.
6
Exponential smoothing and non-negative data
Exponential smoothing models
New book!
Springer Series in Statistics
Rob J. Hyndman · Anne B. Koehler
J. Keith Ord · Ralph D. Snyder
Forecasting
with Exponential
Smoothing
The State Space Approach
13
State space modeling
framework
Prediction intervals
Model selection
Maximum likelihood
estimation
All the important research
results in one place with
consistent notation
Many new results
375 pages but only
US$39.95.
7
Exponential smoothing and non-negative data
Exponential smoothing models
New book!
State space modeling
framework
Prediction intervals
Rob J. Hyndman · Anne B. Koehler
Model selection
J. Keith Ord · Ralph D. Snyder
Maximum likelihood
Forecasting
estimation
with Exponential
All the important research
Smoothing
results in one place with
consistent notation
The State Space Approach
Many new results
375 pages but only
13
US$39.95.
www.exponentialsmoothing.net
Springer Series in Statistics
7
Exponential smoothing and non-negative data
Problems with some of the models
Outline
1
Exponential smoothing models
2
Problems with some of the models
3
A new model for positive data
4
Conclusions
8
Exponential smoothing and non-negative data
Problems with some of the models
Negative forecasts
0
10
20
30
Forecasts from ETS(A,A,N)
0
5
10
15
20
25
30
35
Time
Could solve by taking logs or some other Box-Cox
transformation. However, this limits models to be
additive in the transformed space.
9
Exponential smoothing and non-negative data
Problems with some of the models
Negative forecasts
0
10
20
30
Forecasts from ETS(A,A,N)
0
5
10
15
20
25
30
35
Time
Could solve by taking logs or some other Box-Cox
transformation. However, this limits models to be
additive in the transformed space.
Could solve by only using multiplicative models. But
these can have other problems.
9
Exponential smoothing and non-negative data
Problems with some of the models
Infinite forecast variances
ETS(A,M,N) model
yt = `t−1 bt−1 + εt
`t = `t−1 bt−1 + αεt
bt = bt−1 + βεt /`t−1 .
0
−5
`0 = 0.1
b0 = 1
α = 0.1
β = 0.05
σ=1
5
10
Forecasts from ETS(A,M,N)
0
5
10
15
20
25
30
10
Exponential smoothing and non-negative data
Problems with some of the models
Infinite forecast variances
200
y
0
2
4
6
8
6
8
6
8
ell
−1.0
−0.5
0.0
0.5
1.0
Time
0
2
4
b
−5
0
Time
−10
`0 = 0.1
b0 = 1
α = 0.1
β = 0.05
σ=1
0
−400
`t = `t−1 bt−1 + αεt
bt = bt−1 + βεt /`t−1 .
−15
yt = `t−1 bt−1 + εt
400
ETS(A,M,N) model
0
2
4
Time
10
Exponential smoothing and non-negative data
Problems with some of the models
Infinite forecast variances
Suppose εt has positive density at 0
For ETS models (A,M,N), (A,M,A), (A,Md ,N),
(A,Md ,A), (A,M,M), (A,Md ,M), (M,M,A) and (M,Md ,A):
V(yn+h | xn ) = ∞ for h ≥ 3;
E(yn+h | xn ) is undefined for h ≥ 3.
11
Exponential smoothing and non-negative data
Problems with some of the models
Infinite forecast variances
Suppose εt has positive density at 0
For ETS models (A,M,N), (A,M,A), (A,Md ,N),
(A,Md ,A), (A,M,M), (A,Md ,M), (M,M,A) and (M,Md ,A):
V(yn+h | xn ) = ∞ for h ≥ 3;
E(yn+h | xn ) is undefined for h ≥ 3.
For ETS models (A,N,M), (A,A,M) and (A,Ad ,M):
V(yn+h | xn ) = ∞ for h ≥ m + 2;
E(yn+h | xn ) is undefined for h ≥ m + 2.
11
Exponential smoothing and non-negative data
Problems with some of the models
Infinite forecast variances
Suppose εt has positive density at 0
For ETS models (A,M,N), (A,M,A), (A,Md ,N),
(A,Md ,A), (A,M,M), (A,Md ,M), (M,M,A) and (M,Md ,A):
V(yn+h | xn ) = ∞ for h ≥ 3;
E(yn+h | xn ) is undefined for h ≥ 3.
For ETS models (A,N,M), (A,A,M) and (A,Ad ,M):
V(yn+h | xn ) = ∞ for h ≥ m + 2;
E(yn+h | xn ) is undefined for h ≥ m + 2.
å These problems occur regardless of
the sample space of {yt }.
11
Exponential smoothing and non-negative data
Problems with some of the models
40
Convergence to zero
20
y
30
ETS(M,N,N) model
0
10
yt = `t−1 (1 + εt )
0
`t = `t−1 (1 + αεt )
500
1000
1500
2000
1500
2000
1500
2000
y
5
0
500
1000
10
y
15
20
Time
5
truncated
Gaussian errors
0
0
`0 = 10
α = 0.3
σ = 0.3 with
10
15
Time
0
500
1000
Time
12
Exponential smoothing and non-negative data
Problems with some of the models
Convergence to zero
ETS(M,N,N) model
yt = `t−1 (1 + εt )
`t = `t−1 (1 + αεt ),
0<α≤1
13
Exponential smoothing and non-negative data
Problems with some of the models
Convergence to zero
ETS(M,N,N) model
yt = `t−1 (1 + εt )
`t = `t−1 (1 + αεt ),
0<α≤1
δt = 1 + εt has mean 1 and variance σ 2 .
13
Exponential smoothing and non-negative data
Problems with some of the models
Convergence to zero
ETS(M,N,N) model
yt = `t−1 (1 + εt )
`t = `t−1 (1 + αεt ),
0<α≤1
δt = 1 + εt has mean 1 and variance σ 2 .
δt are iid with positive distribution such as
truncated normal, lognormal, gamma, etc.
13
Exponential smoothing and non-negative data
Problems with some of the models
Convergence to zero
ETS(M,N,N) model
yt = `t−1 (1 + εt )
`t = `t−1 (1 + αεt ),
0<α≤1
δt = 1 + εt has mean 1 and variance σ 2 .
δt are iid with positive distribution such as
truncated normal, lognormal, gamma, etc.
13
Exponential smoothing and non-negative data
Problems with some of the models
Convergence to zero
ETS(M,N,N) model
yt = `t−1 (1 + εt )
`t = `t−1 (1 + αεt ),
0<α≤1
δt = 1 + εt has mean 1 and variance σ 2 .
δt are iid with positive distribution such as
truncated normal, lognormal, gamma, etc.
`t = `0 (1 + αε1 )(1 + αε2 ) · · · (1 + αεt ) = `0 Ut ,
where Ut = Ut−1 (1 + αεt ) and U0 = 1. Thus Ut is
a non-negative product martingale.
13
Exponential smoothing and non-negative data
Problems with some of the models
Convergence to zero
Kakutani’s Theorem says that `t will
converge to 0 almost surely if εt has mean
zero and is non-degenerate.
14
Exponential smoothing and non-negative data
Problems with some of the models
Convergence to zero
Kakutani’s Theorem says that `t will
converge to 0 almost surely if εt has mean
zero and is non-degenerate.
Consequently, all sample paths for yt
converge to 0 almost surely.
14
Exponential smoothing and non-negative data
Problems with some of the models
Convergence to zero
Kakutani’s Theorem says that `t will
converge to 0 almost surely if εt has mean
zero and is non-degenerate.
Consequently, all sample paths for yt
converge to 0 almost surely.
Similar results follow for all purely
multiplicative models: (M,N,N), (M,N,M),
(M,M,N), (M,M,M), (M,Md ,N) and (M,Md ,M).
14
Exponential smoothing and non-negative data
Problems with some of the models
Four model classes
Class M: Purely multiplicative models: (M,N,N),
(M,N,M), (M,M,N), (M,M,M), (M,Md ,N) and
(M,Md ,M).
Class A: Purely additive models:
(A,N,N), (A,N,A), (A,A,N), (A,A,A), (A,Ad ,N)
and (A,Ad ,A).
Class X: Mixed models: (A,M,∗), (A,Md ,∗), (A,∗,M),
(M,M,A), (M,Md ,A). (11 models)
Class Y: Mixed models: (M,A,∗), (M,Ad ,∗) or (M,N,A).
(7 models)
15
Exponential smoothing and non-negative data
Problems with some of the models
Four model classes
Class M: Purely multiplicative models: (M,N,N),
(M,N,M), (M,M,N), (M,M,M), (M,Md ,N) and
(M,Md ,M).
Class A: Purely additive models:
(A,N,N), (A,N,A), (A,A,N), (A,A,A), (A,Ad ,N)
and (A,Ad ,A).
Class X: Mixed models: (A,M,∗), (A,Md ,∗), (A,∗,M),
(M,M,A), (M,Md ,A). (11 models)
Class Y: Mixed models: (M,A,∗), (M,Ad ,∗) or (M,N,A).
(7 models)
Only Class M can guarantee a sample space
restricted to the positive half-line.
15
Exponential smoothing and non-negative data
Problems with some of the models
Four model classes
Class M: Purely multiplicative models: (M,N,N),
(M,N,M), (M,M,N), (M,M,M), (M,Md ,N) and
(M,Md ,M).
Class A: Purely additive models:
(A,N,N), (A,N,A), (A,A,N), (A,A,A), (A,Ad ,N)
and (A,Ad ,A).
Class X: Mixed models: (A,M,∗), (A,Md ,∗), (A,∗,M),
(M,M,A), (M,Md ,A). (11 models)
Class Y: Mixed models: (M,A,∗), (M,Ad ,∗) or (M,N,A).
(7 models)
Only Class M can guarantee a sample space
restricted to the positive half-line.
All Class M models converge to 0 if E(ε) = 0
15
Exponential smoothing and non-negative data
Problems with some of the models
Four model classes
Class M: Purely multiplicative models: (M,N,N),
(M,N,M), (M,M,N), (M,M,M), (M,Md ,N) and
(M,Md ,M).
Class A: Purely additive models:
(A,N,N), (A,N,A), (A,A,N), (A,A,A), (A,Ad ,N)
and (A,Ad ,A).
Class X: Mixed models: (A,M,∗), (A,Md ,∗), (A,∗,M),
(M,M,A), (M,Md ,A). (11 models)
Class Y: Mixed models: (M,A,∗), (M,Ad ,∗) or (M,N,A).
(7 models)
Only Class M can guarantee a sample space
restricted to the positive half-line.
All Class M models converge to 0 if E(ε) = 0
All Class X models have infinite forecast variance
for h ≥ m + 2 where m is the seasonal period.
15
Exponential smoothing and non-negative data
A new model for positive data
Outline
1
Exponential smoothing models
2
Problems with some of the models
3
A new model for positive data
4
Conclusions
16
Exponential smoothing and non-negative data
A new model for positive data
New models
Let δt = (1 + εt ) be a positive random variable.
METS(M,N,N) model
yt = `t−1 δt
`t = `t−1 δtα
17
Exponential smoothing and non-negative data
A new model for positive data
New models
Let δt = (1 + εt ) be a positive random variable.
METS(M,N,N) model
yt = `t−1 δt
`t = `t−1 δtα
log(yt ) = log(`t−1 ) + log(δt )
log(`t ) = log(`t−1 ) + α log(δt )
17
Exponential smoothing and non-negative data
A new model for positive data
New models
Let δt = (1 + εt ) be a positive random variable.
METS(M,N,N) model
yt = `t−1 δt
`t = `t−1 δtα
log(yt ) = log(`t−1 ) + log(δt )
log(`t ) = log(`t−1 ) + α log(δt )
Thus the log-transformed model is identical to
Gaussian ETS(A,N,N) model if δt is logNormal
with median 1.
17
Exponential smoothing and non-negative data
A new model for positive data
Long term forecast behaviour
METS(M,N,N; LN) model
yt = `t−1 δt
`t = `t−1 δtα ,
δt ∼ logN(µ, ω)
18
Exponential smoothing and non-negative data
A new model for positive data
Long term forecast behaviour
METS(M,N,N; LN) model
δt ∼ logN(µ, ω)
yt = `t−1 δt
`t = `t−1 δtα ,
Range
µ + αω < 0
µ + αω = 0
−αω < µ < −αω/2
µ + αω/2 = 0
−αω/2 < µ < −αω/4
µ + αω/4 = 0
µ + αω/4 > 0
E(δtα )
<1
<1
<1
=1
>1
>1
>1
α/2
E(δt
<1
<1
<1
<1
<1
=1
>1
)
E(yh )
Decreasing
Decreasing
Decreasing
Finite
Increasing
Increasing
Increasing
V(yh )
Decreasing
Finite
Increasing
Increasing
Increasing
Increasing
Increasing
18
Exponential smoothing and non-negative data
A new model for positive data
METS(M,N,N;LN)
δt ∼ logN(µ, ω):
0.9
(b)
8
0.5
2
4
0.7
6
(a)
10
12
1.1
14
Long term forecast behaviour
0
500
1000
1500
2000
0
500
1500
2000
1500
2000
1500
2000
t
(d)
0.4
0.6
0.0
0.2
0.2
0.4
(c)
0.6
0.8
0.8
1.0
1.0
t
1000
0
500
1000
1500
2000
0
500
t
0.8
1.0
0.6
0.8
0.4
(f)
0.6
0.2
0.4
0.0
0.2
0.0
(e)
1000
1.0
t
0
500
1000
t
1500
2000
0
500
1000
t
(a) µ = αω/4
(b) µ = 0
(c) µ = −αω/4
(d) µ = −3αω/8
(e) µ = −αω/2
(f) µ = −3αω/4
19
Exponential smoothing and non-negative data
Outline
1
Exponential smoothing models
2
Problems with some of the models
3
A new model for positive data
4
Conclusions
Conclusions
20
Exponential smoothing and non-negative data
Conclusions
Conclusions
1
The standard exponential smoothing state
space models all have theoretical
problems when constrained to the positive
half line.
21
Exponential smoothing and non-negative data
Conclusions
Conclusions
1
2
The standard exponential smoothing state
space models all have theoretical
problems when constrained to the positive
half line.
This is not a concern for most short-term
forecasting.
21
Exponential smoothing and non-negative data
Conclusions
Conclusions
1
2
3
The standard exponential smoothing state
space models all have theoretical
problems when constrained to the positive
half line.
This is not a concern for most short-term
forecasting.
An alternative formulation has been
proposed that avoids these problems.
21
Exponential smoothing and non-negative data
Conclusions
21
Conclusions
1
2
3
4
The standard exponential smoothing state
space models all have theoretical
problems when constrained to the positive
half line.
This is not a concern for most short-term
forecasting.
An alternative formulation has been
proposed that avoids these problems.
The ergodic behaviour of the alternative
formulation requires careful parameter choice.
Exponential smoothing and non-negative data
Conclusions
21
Conclusions
1
2
3
4
The standard exponential smoothing state
space models all have theoretical
problems when constrained to the positive
half line.
This is not a concern for most short-term
forecasting.
An alternative formulation has been
proposed that avoids these problems.
The ergodic behaviour of the alternative
formulation requires careful parameter choice.
Exponential smoothing and non-negative data
Conclusions
21
Conclusions
1
2
3
4
The standard exponential smoothing state
space models all have theoretical
problems when constrained to the positive
half line.
This is not a concern for most short-term
forecasting.
An alternative formulation has been
proposed that avoids these problems.
The ergodic behaviour of the alternative
formulation requires careful parameter choice.
Paper: www.robhyndman.info
Exponential smoothing and non-negative data
Conclusions
21
Conclusions
1
2
3
4
The standard exponential smoothing state
space models all have theoretical
problems when constrained to the positive
half line.
This is not a concern for most short-term
forecasting.
An alternative formulation has been
proposed that avoids these problems.
The ergodic behaviour of the alternative
formulation requires careful parameter choice.
Paper: www.robhyndman.info
Book: www.exponentialsmoothing.net