Introduction to Ecotrim and DEMETRA+ Lunch time presentation 14th of April 2011 Dario BUONO and Enrico INFANTE Unit B2 Methodology and Research 14/04/2011 Objectives Inform the production units about the Eurostat IT tools available for Temporal Disaggregation and Seasonal Adjustment Identify audience for a possible internal courses (including demo session) at ESTAT on the subject No methodological issues will be addressed No software demo, only screenshots 14/04/2011 Introduction Contents Summary information on ECOTRIM (30 mins) Summary information on DEMETRA+ with briefing on the ESS guidelines on Seasonal Adjustment (30 mins) User support provided by Unit B2 Methodology and Research (10 mins) Q & A (20 mins) 14/04/2011 Introduction Why Ecotrim and DEMETRA+ together? Target users are the same ECOTRIM TOOL FOR TEMPORAL DISAGGREGATION OF TIME SERIES Dario BUONO 14/04/2011 Topics Ecotrim in the framework of data production Techniques available in Ecotrim 14/04/2011 ECOTRIM Forewords Ecotrim is a program written for Windows and has been developed by Eurostat There is a project within Eurostat (UNIT C2) for the development/maintenance of the program The main aim of this presentation is to demonstrate the potential of the application User manual is available in EN and FR http://circa.europa.eu/Public/irc/dsis/ecotrim/library 14/04/2011 ECOTRIM Why Eurostat developed Ecotrim? ESA 95 paragraph 12.04 “The statistical methods used for compiling quarterly accounts ……..can be classified in two major categories: direct procedure and indirect procedure” ... indirect procedures are based on Temporal Disaggregation of the annual accounts data in accordance with mathematical and statistical methods using reference indicators that permit the extrapolation for the current year 14/04/2011 ECOTRIM Definitions Temporal Disaggregation – Process of deriving high frequency data from low frequency data and, if available, related high frequency information ECOTRIM – Supplies a set of mathematical and statistical techniques to carry out Temporal Disaggregation 14/04/2011 ECOTRIM Temporal Disaggregation techniques are useful in compiling short-term statistics: Quarterly National Accounts (QNA) Give a quarterly breakdown of the figures in the annual accounts Flash estimates Use the available information in the best possible way including, in the framework of a statistical model, the short-term available information and the low frequency data in a coherent way Monthly indicators of GDP The monthly estimates are derived from the available information respecting the coherence with quarterly data 14/04/2011 ECOTRIM Other short-term statistics Short-term industrial statistics Employment Money and banking statistics 14/04/2011 ECOTRIM Techniques available in Ecotrim 14/04/2011 ECOTRIM Basic principles Distribution – When annual data are either sums or averages of quarterly data (e.g., GDP, consumption, indexes and in general all flow variables and all average stock variables) Interpolation – When annual value equals by definition that of the fourth (or first) quarter (e.g., population at the end of the year, money stock, and all stock variables) Extrapolation – When estimates of quarterly data are made when the relevant annual data are not yet available 14/04/2011 ECOTRIM Techniques available in Ecotrim Temporal Disaggregation of univariate time series by using or not related series and fulfilling temporal aggregation constraints – Mathematical approach and the optimal, in the least squares sense, approach Temporal Disaggregation of multivariate time series with respect of both temporal and contemporaneous aggregation constraints Flash estimates of current year observations by using or not available information on related series 14/04/2011 ECOTRIM Temporal and Accounting constraint Statistical offices are often faced with the estimation of a set of quarterly series linked by some accounting relationship Temporal Disaggregation methods can also be used in such cases, to give a solution consistent with both temporal and accounting aggregation constraints 14/04/2011 ECOTRIM Estimates have to be consistent and coherent Temporal consistency – Quarterly values have to match annual values (for example the sum of quarterly values of the GDP must be equal to the annual value) Accounting coherence – Quarterly components of an account should respect the accounting constraints (for example, the sum of quarterly values of the GDP expenditure side components should be equal to the corresponding quarterly value of GDP) 14/04/2011 ECOTRIM Temporal Disaggregation methods Univariate Approach – – – – Smoothing methods Two steps adjustment methods Time series methods Regression based methods • Static models • Dynamic models Multivariate Approach – Two steps adjustment methods – Regression based methods 14/04/2011 ECOTRIM The classical univariate Temporal Disaggregation Given the annual observations for an account aggregate and the quarterly values for a related series, the unknown quarterly/monthly values of the aggregate are estimated Chow and Lin (1971) worked out a least-squares optimal solution on the basis that a linear regression model involving the quarterly aggregate series and the related quarterly series will hold 14/04/2011 ECOTRIM Statistics Rhô R-squared ( next to 1) Durbin-Watson ( about 2 ) Probability of F T-stat Reliability indicators (lower value for the range between Min and Max) 14/04/2011 ECOTRIM Reminders It must be reminded that the statistics obtained within Ecotrim outputs are based on the annual data It is advisable to use a unique model all along the series Structural breaks must be taken in consideration It is advisable to have observations covering at least 20 years when disaggregating annual data It is advisable to have observations covering at least 15 years when disaggregating quarterly data 14/04/2011 ECOTRIM ECOTRIM ECOTRIM is a program that supplies a set of mathematical and statistical techniques to carry out Temporal Disaggregation Structured for Windows 95/98 and Windows NT Visual Basic and C++ User friendly It can be used according to two different modes: Interactive mode Batch mode 14/04/2011 ECOTRIM Interactive mode Input session Univariate methods Multivariate methods • Boot, Feibes and Lismann • White noise • Denton • Random walk • AR(1) • Rossi • Fernàndez • Denton • Litterman Output session Graphs and display 14/04/2011 ECOTRIM Batch mode ECOTRIM performs Temporal Disaggregation of several jobs starting from a batch command file Batch mode is very useful when handling many series Batch command file 14/04/2011 ECOTRIM Batch session • univariate • multivariate Output Work with ECOTRIM Input Data Aggregated Series Choose the Aggregation Order 14/04/2011 ECOTRIM Work with ECOTRIM Once you have imported the Aggregated Series, you will able to import the Related Series Use the STATUS bar to check the currently loaded files 14/04/2011 ECOTRIM Work with ECOTRIM Univariate Methods Univariate Methods Left click on the series Choose the Type of Aggregation 14/04/2011 ECOTRIM Work with ECOTRIM Choose the Method… …and the Related Series 14/04/2011 ECOTRIM Work with ECOTRIM Compute!!! …and getting results 14/04/2011 ECOTRIM Work with ECOTRIM Display the data 14/04/2011 ECOTRIM Future of Ecotrim Updating the Visual Basic/C++ version and refurbish the interface of ECOTRIM with the purpose to match production and users’ requirements Design of the application as a Building Block, CVD compliant Make system Windows 7 compatible, compilation as a 64-bit application (former was 16-bit) Inclusion of more recent methods like: – Univariate Regression based Temporal Disaggregation – Reconciliation by Two-Step procedures (Quenneville & Rancourt 2005 – Di Fonzo & Marini 2009) 14/04/2011 ECOTRIM DEMETRA+ TOOL FOR SEASONAL ADJUSTMENT Enrico INFANTE 14/04/2011 What is DEMETRA+? DEMETRA+ is a tool for Seasonal and Calendar Adjustment developed by National Bank of Belgium for EUROSTAT Trend-Cycle Component Identify more components Outliers Irregular Component 14/04/2011 DEMETRA+ What are Seasonal and Calendar Adjustment? SEASONAL ADJUSTMENT Seasonality: Fluctuations observed during the year (each month, each quarter) and which appear to repeat themselves on a more or less regular basis from one year to the other Remove Seasonality CALENDAR ADJUSTMENT Remove Calendar Effects 14/04/2011 DEMETRA+ Calendar Effects: Any economic effect which appears to be related to the calendar (one more Sunday in the month can affect production) Why DEMETRA+? ESS Guidelines User friendly!!! It will be Open-Source (java) Based on X12 ARIMA / Tramo-Seats 14/04/2011 DEMETRA+ Documents about DEMETRA+ You can download the latest version of DEMETRA+ at the following link: http://circa.europa.eu/irc/dsis/eurosam/info/data/demetra.htm At the same link you can find: ESS guidelines on Seasonal Adjustment DEMETRA+ User Manual and other documents Help-Desk e-mail: estat-methodology@ec.europa.eu Information about the next ESTP course (at EUROSTAT, 3-5 May 2011) and the EUROSTAT Training Course 14/04/2011 DEMETRA+ How to work with DEMETRA+ First overview Browser panel 14/04/2011 DEMETRA+ presents the available time series How to work with DEMETRA+ First overview Workspace panel 14/04/2011 DEMETRA+ shows information used or generated by the software How to work with DEMETRA+ First overview Central empty zone will contain the actual analysis 14/04/2011 DEMETRA+ How to work with DEMETRA+ First overview TSProperties panel 14/04/2011 DEMETRA+ contains the current time series How to work with DEMETRA+ First overview Logs panel 14/04/2011 DEMETRA+ contains logging information How to work with DEMETRA+ Importing data from Excel Browser panel Excel tab Add command • Empty top-left cell • True dates in the first column • Title of the series in the first row • Empty cells for missing values 14/04/2011 DEMETRA+ Open the excel workbook How to work with DEMETRA+ Displaying the data Open one of the containers DRAG & DROP 14/04/2011 DEMETRA+ How to work with DEMETRA+ Single Seasonal Adjustment Choose and activate a specification Getting results Seasonality tests 14/04/2011 DEMETRA+ What happens after SA? Original, Calendar Adjusted and Seasonally Adjusted series Original Series 14/04/2011 DEMETRA+ What happens after SA? Original, Calendar Adjusted and Seasonally Adjusted series Calendar Adjusted Series 14/04/2011 DEMETRA+ What happens after SA? Original, Calendar Adjusted and Seasonally Adjusted series Seasonally Adjusted series The series has been cleaned!! 14/04/2011 DEMETRA+ What happens after SA? Growth Rates Original Series 14/04/2011 DEMETRA+ Xt Gt 1 X t 1 What happens after SA? Growth Rates Calendar Adjusted Series 14/04/2011 DEMETRA+ Xt Gt 1 X t 1 What happens after SA? Growth Rates Seasonally Adjusted series 14/04/2011 DEMETRA+ Xt Gt 1 X t 1 ESS Guidelines on SA http://epp.eurostat.ec.europa.eu/cache/ITY_OFF PUB/KS-RA-09-006/EN/KS-RA-09-006-EN.PDF Introduced in 2009 Chapters subdivided into specific items describing different steps of the SA process Items presented in a standard structure providing: 1. Description of the issue 2. List of options which could be followed to perform the step 3. Prioritized list of three alternatives from most recommended one to the one to avoid (A, B and C) 4. Concise list of main references Added value: 1. Conceptual framework and practical implementation steps 2. Both for experienced users and beginners 14/04/2011 DEMETRA+ ESS Guidelines on SA (example 1) Calendar Adjustment Calendar Effects typically include: Different number of working days in a specific period Composition of working days Leap year effect Moving holidays (Easter, Ramadan, etc.) A) RegARIMA approach, fully checks the significance and plausibility of the effects B) Regression approach based on the irregular component C) Proportional Adjustment; other adjustments; no adjustment A) Best alternative B) Acceptable C) To be avoided 14/04/2011 DEMETRA+ ESS Guidelines on SA (example 2) Validation of Seasonal Adjustment SA is a complex data treatment which needs accurate monitoring before the results are acceptable Descriptive Graphical Parametric and non- parametric tests A) Use a detailed set of criteria B) Use only default criteria already defined within the different tools C) Use only the basic set of criteria or do not validate A) Best alternative B) Acceptable C) To be avoided 14/04/2011 DEMETRA+ USER SUPPORT PROVIDED BY UNIT B2 Dario BUONO 14/04/2011 EUROSTAT UNIT B2 Methodology and Research Define, position and co-ordinate all statistical actions in the field of research Provide the EU with a service on the statistical methodologies in order to improve the data quality and reliability for the use of the EU policies Achieve a higher degree of harmonization of the EUROSTAT and ESS practices on SA: Definition of detailed guidelines on SA Maintenance and development of reliable tools for SA at ESS level (DEMETRA+) Fostering the exchange of practices among NSIs Organisation of training on SA 14/04/2011 EUROSTAT UNIT B2 Methodology and Research Define, position and co-ordinate all statistical actions in the field of research Provide the EU with a service on the statistical methodologies in order to improve the data quality and reliability for the use of the EU policies Consultative role in the ECOTRIM project Partial user support also on methodological aspects by EUROSTAT methodology helpdesk 14/04/2011 Questions? 14/04/2011