Implementation of Quality Assurance
Activities in Eurostat
Third CCSA Data Quality Conference
Rome 7-8 July 2008
A. Baigorri, M. Hahn, H. Linden
Eurostat
1
Structure of the presentation
 Quality assessments in Eurostat: methodology and
implementation
 Redesigned Quality Barometer: short term and long
term perspectives
 Plans to enhance quality assurance in the European
Statistical System
2
Quality assessments methodology
Each process is matched with an assessment category
SelfAssessment
Supported
Self-Assessment
Prod. unit: fills in Checklist,
produces outputs
DDG02: provides support
on demand
Peer review
Rolling review
A comprehensive review
Same
as
based on the checklist
Prod. unit: in
Supported Self-Assessment
implemented by ext.contractor,
control of assessment,
participates in all steps with one or more peers (int/ext) involving partner & user surveys
taking part in all steps
and cost-benefit analysis
DDG02: pre-fills the Checklist,
produces the outputs
Process 1
Process 2
Process 3
Process 4
Process 5
Characteristics:
- Periodicity
- Legal Basis
- Output
- ESTAT intervention
-availability of quality report
3
Quality Assessments Implementation Plan:
Principles
 Minimize burden for units
–
Test of the approach in advance.
–
Support from unit DDG-02
–
Flexibility
 Build on existing information
–
Quality reports
–
Process analysis
 Integrate QA with other horizontal activities
4
Integrating Quality Assessments
User Requirement
Standards
Labelling
Evaluation function
III. CONFORMITY
Quality
assessments
Rolling
reviews
Cost/benefit 5YP
Best practices
ASSESSMENT CHECKLIST
CVD steps
| | | | | |
II. EVALUATION
Quality diagram
Improvement actions
Quality
Barometer
AMP
I. DOCUMENTATION
MEASUREMENT
Process
description
Production
processess
Quality
reports
SDMX
Statistical products
User satisfaction survey
User perception
Eurostat Institutional/Legal environment
5
Quality Assessments Implementation Plan:
Support
 Training
- in-house courses on data quality reporting
- lunchtime presentations on the ESPAC
- peer review results
- seminars on quality assessments
 Communication
- intranet site on quality
- regular information on Cybernews
- Eurostat infos
- dissemination of good practices best practices
6
Quality Assessments Implementation Plan:
Agenda 2008
Sector
Quarterly non-financial accounts for general government
Quarterly Institutional Sector Accounts
Prevention of illegal migration and stay statistics
Asylum statistics
Waste and Recycling Statistics
International Trade in Services
M-3 and M-1 questionnaires
Harmonised unemployment rates
Household Budget Survey
Quarterly financial accounts for general government
ICT usage in enterprises
Structural Business Statistics-A. 1 to 4
Annual Questionnaires Quantities
Farm structure survey
Electricity and gas prices
Forestry statistics
Meat and livestock statistics - livestock
Meat and livestock statistics - slaughterings
Meat and livestock statistics - production forecasts
Meat and livestock statistics - supply balance sheets
PRODCOM - Industry Production
LUCAS
Catch Statistics
E-communication. telecommunication
Landing Statistics
Aquaculture Statistics
Fishing Fleet Register (from DG FISH data base)
Demographic data
Food safety: Statistics on Control and monitoring Activities
Food safety: Organic farming statistics
System of health accounts (SHA)
Id
C5-4
C2-2
F1-5
F1-3
E3-1
C4-4
G4-3
F2-3
F3-4
C5-3
F6-2
G1-1
G4-1
E1-3
G4-4
E3-13
E2-4
E2-5
E2-6
E2-7
G1-6
E1-2
E2-16
F6-3
E2-17
E2-18
E2-19
F1-1
F5-2
F5-3
F5-4
QR
Y
P
Y
Y
Y
P
P
Y
Y-5 years
Y
Y-annual
P
L
Y
Y
Y
P/F
P/F
P/F
P
Y
Y
P
N
P
N
P
N
N
N
N
A Cat
SSA
SSA
SA
SA
SSA
SSA
SA
SA
SSA
SSA
SSA
SSA
SSA
SSA
SA
SA
SA
SSA
SA
SA
SSA
PR
SSA
SSA
SA
SA
SA
SA
SA
SA
SA
Estimated start
03/03/2008
23/03/2008
01/04/2008
03/04/2008
08/04/2008
14/04/2008
21/04/2008
01/05/2008
15/05/2008
19/05/2008
01/06/2008
01/06/2008
01/06/2008
01/07/2008
01/07/2008
04/08/2008
01/09/2008
01/09/2008
01/09/2008
01/09/2008
01/10/2008
13/10/2008
01/11/2008
01/11/2008
01/12/2008
Sector
Causes of death statistics (COD)
ESAW: Accidents at work statistics
Railway transport
Air transport
Id
F5-7
F5-8
G5-2
G5-3
QR
N
N
F
Y
A Cat
RR
RR
RR
RR
Timing
February - October 2008
February - October 2008
April 2008 - March 2009
April 2008 - March 2009
7
Quality assessments in Eurostat:
next steps
 Evaluation of the 2008 program of assessments
-
Need for adjusting the instruments used
-
Indentify recurrent weaknesses in statistical processes to feed
improvement actions into the AMP
-
Promote initiatives to be implemented by horizontal units to facilitate
quality improvements by production units.
 Elaboration of the 2009 plan
- focussed on more objective assessments as peer and rolling
reviews.
8
Redefined Quality Barometer:
characteristics
Purposes:
- to measure the quality level at both EU level and country level;
- to monitor change;
- for comparisons across statistical domains and across countries.
Users:
- production side: top-management, departments/units, survey managers.
- user’s side: international organisations, policy makers, researchers, etc.
Scope:
- cover all types of statistical processes and their statistical outputs;
- the quality of key data (or main time-series);
- monitor quality on an annual basis.
9
Redefined Quality Barometer:
short term perspective
 Restricted QB based on quality indicators that are:
- applicable to most statistical domains;
- covering the main quality dimensions;
- well defined and easy to provide; and
- easy to interpret and understand.
 The restricted set of Quality Indicators prioritised for
the QB:
- Rate of available statistics (R2)
- Coefficient of variations (A1)
- Unit response rates (A2)
- Average size of revisions (A7)
- Time lag reference period and date of first results (T2)
- Number of accesses to databases (AC2)
- Rate of completeness of metadata (AC3)
- Length of comparable time series (C1)
10
Compilation of a restricted
Quality Barometer
 Including many statistical processes but only covering
the restricted set of quality indicators
 The Euro- SDMX Metadata Structure should be the
main source of information
 Continue the MS’s Expert Group and Eurostat network
for methodology for indicators and analysis of results
 First RQB planed for the end of 2008
11
Redefined Quality Barometer:
long term perspective
I. INTRODUCTION
IIa. Social statistics
Labour Force Statistics
Relevance
Accuracy
Timeliness
Comparability
Coherence
Summary for Labour Force Statistics
IIb. Business statistics
III. OVERALL SUMMARY
12
Enhancing quality assurance in the ESS

Regulation on European Statistics and Article 10 “Statistical quality”:
- Quality of the MS’s and Eurostat data shall be assessed according to the
ESS quality dimensions.
- Specific quality requirements, such as target values and minimum
standards for the statistical production, in sectoral legislation.

Euro-SDMX Metadata Structure (ESMS)
- Harmonizes quality reporting integrating the ESS quality dimensions
- improves metadata sharing and information to users on quality of
statistics

Development and sharing of “best practices” for statistics production
13
Enhancing quality assurance in the ESS
 Revision of the Eurostat/ESS quality reporting guidelines:
- Standard Quality Report
- Handbook: How to make a Quality Report
 Facilitate the implementation of CoP improvement actions
plans by NSIs
- Eurostat 2009 grants on the implementation of ESS standards,
handbooks, guidelines and application of tools
- ESS Quality Coaching focusing on the implementation of
quality management and quality assurance
14
Thank you for your attention!
http://ec.europa.eu/eurostat/quality
15