HANDBOOK ON
ECONOMIC TENDENCY
SURVEYS
Presentation by Luciana Crosilla and Mauro Politi
(ISTAT – National Statistical Institute, Italy)
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
Outline
1. Motivation (introductory notes; why an ETS handbook?)
2. The process (preparation step; milestones; method of work;
arranging the drafting)
3. The handbook (purposes; intended readership; about the
handbook; an overview)
4. The gobal consultation: preliminary results (summary of
comments)
5. Concluding remarks
1
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
Motivation
Introductory notes
 UN Statistical Commission considered an international programme of
work on short-term economic statistics as part of a coordinated
statistical response to the economic and financial crisis ;
 ETS are a theme comprised in the international programme because:
o ETS data have been recognized as tools to improve the
estimation of other economic indicators and as components of
composite indicators;
o Confidence indicators are useful in providing an early warning of
changes in economic activity (UN, EUROSTAT);
2
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
Motivation
Why an ETS handbook?
 UN encourages countries to conduct ETS (if not doing so already)
in respect the role of the surveys data in the economic research;
 UN has identified a handbook as a tool to assist member States in
compiling and reporting internationally comparable tendency
survey data;
 the shortcoming of a unique guide for “economic” tendency
surveys (the current handbooks deal with the consumer surveys
or the business “traditional” surveys, excluding e.g. agriculture
sector) comprising:
• best practices
• harmonized principles
• Guidance;
3
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The process
Preparation step: the team
International working group
 The Italian National Statistical Institute (ISTAT) – leader – Italy
 Statistics Netherlands (CBS) – Netherlands
 The Swiss Economic Institute (KOF – ETH) – Switzerland
 Philippine Statistical Authority (PSA former NSCB) – Philippines
 The Organisation for Economic Co-operation and Development
(OECD)
4
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The process
Preparation step: the Team
External revisers
 The European Commission (DG Ecfin)

French National Institute of Statistics and Economic Studies
(INSEE)
Contributors
 Instituto Brasileiro de Economia, Fundação Getulio Vargas, IBREFGV
5
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The process
Milestones

February 2012: decision of the Statistical Commission of UN to
produce a handbook;
 October 2012: constitution of the Working Group;
 Time span for the realization of a first draft of the handbook: January
2013 - October 2013;
 November 2013: the first draft was submitted to the revisers;
 Realization of a second draft (including further comments of the
revisers): January-may 2014;
 UN Global consultation: 28 July – 29 August 2014;
 The finalization of the handbook is expected by beginning of 2015.
6
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The process
Method of work
• No meetings (pros and cons);
• No multi-conference call (time gap between countries and language
difficulties);
• Bilateral conference call;
• Contacts only by e-mail;
• Standardization of the template to write the chapters;
• always in touch with UNSD;
• Hard centralized work of harmonization and revision (different
experiences, languages…) to avoid repetitions, non homogeneous
7
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The Process
Arranging the drafting
The drafting of the handbook was arranged for chapters: each chapter
deals with a (main) topic.
 For each chapter:
 three/four working group members worked
 One role was assigned to each member, among:
• Leader (responsible of the chapter: it coordinated the drafting
of the chapter and, in some cases, it has contributed in the
drafting)
• Contributor
• Reviser
 For some chapters (it depends on the topic), an external reviser
(EC, INSEE) was assigned.
8
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The process
Arranging the drafting
 The deadline for the finalization of the first draft and of the revised
draft was assigned;
 To each contributor was assigned
paragraphs expected in the chapter.
the drafting of one/two sub-
 EXAMPLE : Chapter 4
4.
SURVEY FRAME
SAMPLE DESIGN
CBS
ISTAT
NSCB
KOF
OECD
AND
Role
Leader/Contributor
Contributor
Contributor
Reviser
Reviser
Draft to be
submitted
Revised
draft
31/05/2013
30/06/2013
4.1 Business surveys
(assigned to CBS)
4.2 The consumers survey sample (assigned to ISTAT)
4.3 Frequency and timeline (assigned to NSCB)
9
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
Purposes
 to constitute internationally accepted set of guidelines applicable to all
countries and multiple economic activities;
 to provide guidance on the process of setting up tendency survey
operations, data processing and analysis;
 to make users aware of the statistical methods and techniques that
are employed in tendency surveys (associated with composite
tendency indicators).
10
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
Purposes
More specifically, the handbook should provide best practices and
harmonized principles on:

tendency survey sample selection
 questionnaire design
 survey questions
 survey execution
 data processing
 dissemination of results
 use of composite tendency indicators
11
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
Intended readership
 National statistical organizations that already carry out or intend to
carry out tendency surveys in future;
 Users of the survey data as researchers, analysts, compilers of
composite indicators etc.;
 Policymakers and anybody related to monitoring economic conditions
and business cycles.
12
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
About the handbook
 The handbook is about:
• methods that are currently used (or could be used) to produce
tendency survey data;
• some related aspects (e.g., process design, quality)
 The handbook is not about:
• laws, regulations, etc.
• official definitions of concepts
13
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
About the handbook
 In preparing the handbook, the working group has taken into
account the existing guidelines on the subject (European
Commission and OECD);
 The new handbook expands the scope of the existing guidelines to
include more economic sectors ;
 it focuses explicitly on the needs of developing countries in
administering tendency surveys.
14
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
About the handbook
The handbook is made up of:
 10 chapters
 9 topics: each chapter deals with one main topic
• Examples of topic: survey frame and sample design
the questionnaire design
 List of abbreviations and acronyms
15
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
An overview
CONTENTS
Chapter 1
2
3
4
5
6
7
8
9
10
16
Introduction
Constructing ETS: scope
The questionnaire design
Survey frame and sample design
Estimation procedures and accuracy
Data collection
Managing sources of non-sampling errors
Processing tendency survey data
Data dissemination and publication
Use of tendency survey results
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
An overview
Chapter 1 - Introduction
Define the scope of the handbook and introduces the main
characteristics of the ETS.
 Main goals of the handbook
 Qualitative data: characteristics and place in the system of
economic statistics
 On the use of ETS
 Survey process
 Outline of the handbook
17
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
An overview
Chapter 2 - Constructing ETS: scope
Describes the goal of ETS and contains the main definitions.
 The scope of BTS
• Sector coverage
• Regional Coverage
• Survey units
• Basic principles of questions
• Survey frequency

The scope of CTS
• Target population, sampling and reporting unit
• Principle of the questions
• Survey mode and frequency
18
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
An overview
Chapter 3 – The questionnaire design
Principles guiding the construction and selection of questions are
discussed as well as the choice of response scales.


19
Introduction
General considerations
•
Business and consumer tendency surveys: main
differences
•
Formulation of questions
•
Choosing subjects and sectors
•
Measurement scale
•
Reference period
•
Seasonality
•
Structural Information on respondents
•
Pre-testing of the questionnaire
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
An overview
Chapter 3 – The questionnaire design (continued)

An example of harmonized questionnaire
• BTS at a monthly frequency
• BTS at a quarterly frequency
• BTS at a (semi-) annual frequency
• CTS at monthly and quarterly frequencies
20
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
An overview
Chapter 4 – Survey frame and sample design
It focuses on important steps in the entire process of ETS: from where
and how to select a subset of units (i.e. the sample). Moreover, the
selection methods of units from survey frame, the allocation criteria
and the sample size are discussed.
 Introduction
 Survey frame
• Business tendency surveys
• Consumer tendency surveys
21
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
An overview
Chapter 4 – Survey frame and sample design (continued)
 Sample design
• Selecting units: probabilistic and deterministic samples
• Sample design features
22
•
Panel sample
•
BTS: specific features of sample design
•
CTS: specific features of sample design
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
An overview
Chapter 5 – Estimation procedure and accuracy
In ETS estimation processes, weighting schemes are usually used: the
goal of the weighting procedures and the choice of specific weights are
described in detail. Sampling error and its calculation are illustrated.
 Introduction
 Target variables and size weights
• Why use size weights in BTS?
• Choice of size weights in BTS
 Estimators and sample weights
• Horvitz-Thompson and ratio estimators
• Inclusion probabilities
23
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
An overview
Chapter 5 – Estimation procedure and accuracy (continued)
 Post-stratification
 Accuracy and sample size
• General theory
• Example: Simple Random Sampling
• Design effect
24
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
An overview
Chapter 6 – Data collection
Survey modes, reminder policy and training of interviewers are topics
discussed within this chapter.
 Data collection: designing a strategy
• Primary data collection
• Data collection strategy: survey techniques and
communication with respondents
• Determinants in designing the process of data collection
 Designing the process of data collection
• Pre survey: preparations and communication
• Field work: collecting the data
• Response enhancing measures
 Organizing the data collection
• Training of enumerators
• Technical facilities and documents
25
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
An overview
Chapter 7 – Managing sources of non-sampling errors
In chapter 7 non-sampling errors and the possible approaches to
reduce errors are analysed.
 Non-sampling errors
• Coverage errors
• Measurement and processing errors
• Non-responses
 Treatment of non-responses
• Total non-responses
• Partial non-responses
26
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
An overview
Chapter 8 – Processing tendency survey data
Qualitative responses have to be quantified to make them easier to
interpret and to obtain time series from different survey waves. In
addition, various standard methods for seasonal de-composition of
time series and widely adopted solutions for ETS time series are
discussed here.
 Conversion of multiple choice questions into a time series
• The balance statistic
• Diffusion indexes
• The probabilistic approach and other methods
• Latest proposals for the quantification of qualitative information
• The “disaggregate” approach
27
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
An overview
Chapter 8 – Processing tendency survey data (continued)
 The seasonal adjustment
• Seasonal patterns in opinion survey data
• General principles of seasonal adjustment of BCS data
• Widely adopted solutions for seasonal adjustment
28
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
An overview
Chapter 9 – Data dissemination and publication
Recommendations regarding metadata and output tables are made.
Strategies of publication are also discussed.
 Metadata
• Contact data
• The methodology
• Data and dissemination
 Publication procedures
• Variety of users
• Quality criteria
• Release calendar
• Forms of dissemination
• Confidentiality
• Revisions
29
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The handbook
An overview
Chapter 10 – Use of tendency survey results
The most common business cycle composite indicators are presented;
examples of forecasting applications, turning point detection and
micro-econometric approaches are given.
 Users of survey data
 The use of survey data
• Business cycle composite indicators
• Visualizing tools
• Forecasting using survey data
• Consumer confidence, macroeconomic analysis and business
cycle
• Micro-econometric methods
30
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The Global consultation: preliminary results
 A Global Consultation on the handbook was carried out during the
period from 28 July 2014 to 29 August 2014 by UN. To facilitate the
consultation a comment form was prepared.
 A total of 62 responses were received from countries (57) and
international/regional organizations (5).
 Overall there was a strong support for the Handbook: the responses
were extremely positive and a number of useful comments were
received.
31
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The Global consultation: preliminary results
Summary of the responses that were received to the questions
(as of 19 september)
Yes
Per cent Number
Question 1: Do you consider that the Handbook
provides useful recommendations on Economic
Tendency Surveys?
100
50
0
0
Question 2: Do you agree with the
recommendations provided in the handbook?
100
50
0
0
Question 3: Overall do you consider the
structure of the handbook clear and coherent?
100
50
0
0
98
48
2
1
Question 6: Do you consider the Handbook
ready for publication, subject to the
incorporation of the comments made during this
global review?
32
No
Per cent Number
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The Global consultation: preliminary results
Summary of comments
 Positive comments. The respondents noted:
 the comprehensive coverage of the topics (all the phases of the
statistical production processes);
 the usefulness of the recommendations provided in the handbook;
 the current structure of the handbook: clear, coherent and easy to
follow;
 98 per cent of the respondents considered the Handbook ready for
publication subject to the incorporation of the comments received
through the global consultation.
33
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
The Global consultation: preliminary results
Summary of comments
 Main suggestions of the respondents:
 To include references or examples from non EU countries (not only
experience from EU countries);
 To elaborate the text more on issues relating to countries with poor
statistical systems and poor data quality;
 chapter-specific technical and editorial comments and/or proposals for
revision;
 relevant reference should be made to ISIC classification;
 the inclusion of additional text on the calculation and interpretation of
composite indicators.
34
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
Concluding remarks
 The finalization of the handbook is expected by 2015 after:
 Incorporating all the comments from the Global Consultation and
alignment the text with the UN style for publications: by end
December 2014;
 final review of the working group and steering group: by end
January 2015;
 submit final draft (subject to language editing) to the 46th session
of the UN Statistical Commission.
 Final assessment of the Working Group – UNSD on the
work/experience
35
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014
THANK YOU FOR YOUR
ATTENTION !!!
lcrosilla@istat.it
politi@istat.it
36
Handbook on Economic Tendency Surveys - UN Workshop on short-term economic
indicators, Hangzhou, 8 October 2014