Real Time Trend Extraction and
Seasonal Adjustment: a Generalized
Direct Filter Approach
ISF 2011, Prague
Marc Wildi
Zurich University of Applied Sciences
Marc.wildi@zhaw.ch
Signalextraction vs. Forecasting
Signal
X t  N oisy D ata
Filter: a set of w eights  k such that

Yt 

 k X tk
k  
is `fr ee of noise'
Yt is the S igna l
T rend, S easonally A djusted C om ponent, C yc l e
Filters:  k
• Ad hoc designs: no explicit modelling of the
data
– HP-Filter, CF-Filter, BK-Filter, Henderson Filter, …
• Model-based designs
– TRAMO/SEATS, X-12-ARIMA, Stamp
• Non-parametric filters (Loess)
• Very general setting!
Real-Time Signalextraction
Time Domain

YT 

 k X T  k  `senses' the future (X T  1 , X T  2 , ...)
k  
R eal-T im e Finite S am ple
YˆT 
T 1
 ˆ
k
X T k
k 0
M odel-B a sed A pproaches (M B A ):



k
Xˆ T  k 
k T
T 1

1
k
X T k 
k 0
T 1


k 0
X T k 
 k Xˆ T  k
k  
1
k


 k Xˆ T  k
k  
O ne- a nd m ulti-s tep ahead fore casts
Example
A R (1)  P ro cess : X t  a X t  1   t
T 1
F ilter: sym . ex p o n en tial w eig h tin g Yt  c


|k |
X tk
k   ( T  1)
T 1
YˆT  c  
1
|k |
k 0
X T k  c
Xˆ T  k 


|k |
a
|k |
XT 
k   ( T  1)
T 1
c 
|k |
1
|k |
k 0
|k |

k   ( T  1)
T 1
c 

X T k  c
X T k
k 1
0


|k |
  c  ( a )  X T 
 k   ( T  1)

T 1
c 
k 1
|k |
X T k 
c
1  a
XT
V ery cu m b erso m e w ay to d efin e a o n e-sid e d filt er!
Forecasting

YT 

 k X T k
k  
   1  1,
Forecasting: 
  k  0 for k  -1
T his is a very particular (asym m etric) `S ignal' D efinition
M odel-B ased O ne-step ahead Forecast!
Frequency Domain
Real-Time Signalextraction
Frequency Domain

T arget: YT 

 k X T k
k  
R eal-T im e E stim ate: YˆT 
T 1
 ˆ
k
X T k
k 0
T ransferfunctions

 (  ):=

 k exp(  ik  ) ( 
if sym m etric)
k  
T 1
ˆ (  ):=  ˆ k exp(  ik  )
k 0
Example: European IPI
TRAMO/SEATS (Airline-Model in red)
Forecasting

 (  ):=

 k exp(  ik  )
k  
   1  1,
Forecasting: 
  k  0 for k  -1
 ( )  1 * exp( i )
 ( ) is a very particular (allpass) Filter/T ransferfunction
R eplicates T raditional M odel-B ased O ne-s tep ahead Forecast in F-D !
Optimization Criterion: Mean-Square
Filter error: rt  Yt  Yˆt
C riterion: E[ rt ]  m in FILT E R W E IG H T S
2


2
|  ( )  ˆ ( ) | d S ( )  m in FILT E R W E IG H T S

R eal-W orld:

k
2
ˆ  )  m in
 ( k )  ˆ ( k ) S(
k
ˆ
Choice of Spectral Estimate Sˆ ( )
• Model-based:
– TRAMO (airline-model), X-12-ARIMA, state-space
• Ad-hoc:
– implicit model (HP, CF, BK, Henderson,…)
• Non Parametric
– Periodogram
• This choice is to some extent arbitrary: it
depends on the
preference/experience/expertise of the user.
• Very general setting!
Generalized DFA: Very General Setting!
• Arbitrary signals
– Including as a special case traditional one-step ahead
forecasting
• Arbitrary finite sample Spectral Estimate
– ad hoc, model-based, non-parametric
• Generalizes
–
–
–
–
Ad hoc filters
Model-based filters
DFA (based on the periodogram)
Traditional (one-step ahead) ARIMA-modelling, statespace modelling
– Extends to multivariate filtering!
Frequency-Domain:
Timeliness-Reliability Dilemma
Control of Timeliness/Speed:
2
Cosine Law applied to  ( )  ˆ ( )
ˆ ( )
 ( )  ˆ ( )
ˆ ( )
 ( )  ˆ ( )
 ( )  ˆ ( )
2
2
 ( )


 2  ( ) ˆ ( ) 1  cos( ˆ ( ))

Timeliness-Criterion
T /2

2
 ( k )  ˆ ( k ) Sˆ ( k ) 
k 1
T /2

2
A ( k )  Aˆ ( k ) Sˆ ( k ) 
k 1
T /2


  2A ( k )Aˆ ( k ) 1  cos( ˆ ( k )) Sˆ (  k )
k 1
M ean-S quare:   1
Faster Filter :  > 1
S low er Filter:  < 1
Emphasize Noise Rejection in Stop Band
(Reliability/Smoothness)
T /2

2
A ( k )  Aˆ ( k ) W ( k )Sˆ ( k ) 
k 1
T /2


  2A ( k )Aˆ ( k ) 1  cos( ˆ ( k )) Sˆ (  k )
k 1
W ( k ) assigns m ore w eight to am plitude in stop band
 assigns m ore w eigh t to tim e-shift in pas s band
Essence of Generalized DFA
• The new optimization criterion IS the timelinessreliability-dilemma and conversely
• `Philosophy’ may be contrasted with
– Maximum likelihood (particular parametric setting
lambda/expweight)
– Maximum entropy
• Contrast:
– Manipulate Real-Time filter characteristics explicitly
on the edge of the fundamental dilemma
– User relevant priorities (risk-aversion)
Effect of `Expweight’
Effect of Lambda
Example : European IPI
Replicate TRAMO RT-Performance:
TRAMO (red) vs. Gen. DFA (blue)
New Target: Customized Design
• Instead of optimal mean-square estimate the
user could specify a `faster’ and/or `smoother’
real-time estimate
• The new estimate is still purely model-based!
– It IS TRAMO (it could be X-12, Stamp,…)
– But it becomes faster/smoother (timelinessreliability dilemma)
Mean-Square vs. Enhanced TRAMO
• Typically, TRAMO-filter (blue) is noisy (poor noise
suppression in stop-band)
• The `customized’ filter (green) barely loses in terms of
time-shift in the pass-band. It clearly wins in terms of
noise suppression in the stop-band: better compromise
TRAMO (red) vs. Enhanced (green)
Conclusion
• As expected, the `customized’ real-time filter
(green) is as `fast’ as the MS-filter by TRAMO
(red) and it is much smoother (better noise
suppression)
SA vs. Customized RT-Trend
• Real-time customized trend filter is as fast as
traditional SA-filter and much (much)
smoother.
Conclusion
Philosophy Generalized DFA
The new criterion IS the
timeliness-reliability dilemma
Consequences
• Generalizes classical filter approaches (ad hoc,
model-based)
• Emphasizes user relevant priorities explicitly
Practicality
• Numerically (very) fast
– Closed-from approximation (I-DFA/open source)
– Fast exact optimization (Eurostat/proprietary)
• Short piece of (R-) code
– Could easily dock to any existent software/tool
Web:
•
•
•
•
SEFblog: http://blog.zhaw.ch/idp/sefblog
USRI: http://www.idp.zhaw.ch/usri
MDFA-XT: http://www.idp.zhaw.ch/MDFA-XT
SEF-page: http://www.idp.zhaw.ch/sef
Selected SEFBlog-Entries
• Forecasting the EURO-BUND-Future (6 months,
one Year)
– http://blog.zhaw.ch/idp/sefblog/index.php?/archives/
186-Forecasting-the-EURO-Bund-Future-6-monthsand-One-Year-Ahead-FirstPreliminary-Draft.html
• OECD-CLI: leading indicator for the US
– http://blog.zhaw.ch/idp/sefblog/index.php?/archives/
173-Tutorial-I-MDFA-Part-II-The-OECD-CLI-for-theUS.html
– http://blog.zhaw.ch/idp/sefblog/index.php?/archives/
175-Injecting-the-ZPC-Gene-into-I-MDFA-anApplication-to-the-OECD-CLI-for-the-US.html
SEFBlog-Entries
• Algorithmic Trading:
– http://blog.zhaw.ch/idp/sefblog/index.php?/archives/
157-A-Generalization-of-the-GARCH-in-Mean-ModelVola-in-I-MDFA-filter.html
• Tutorials Univariate Filter:
– http://blog.zhaw.ch/idp/sefblog/index.php?/archives/
159-I-DFA-Exercises-Part-I-Mean-SquareCriterion.html
– http://blog.zhaw.ch/idp/sefblog/index.php?/archives/
160-I-DFA-Exercises-Part-II-CustomizationSpeedReliability.html
SEFBlog-Entries
• Tutorials Multivariate Filter:
– http://blog.zhaw.ch/idp/sefblog/index.php?/archi
ves/172-Tutorial-I-MDFA-Part-I-Simulated-TimeSeries.html
– http://blog.zhaw.ch/idp/sefblog/index.php?/archi
ves/173-Tutorial-I-MDFA-Part-II-The-OECD-CLI-forthe-US.html