International Seminar on Modernizing Official Statistics:
Meeting Productivity and New Data Challenges
Tianjin, People’s Republic of China
24 - 26 October 2013
Meeting Challenges of Unstructured Data:
European Statistical System experience
Mariana Kotzeva, Advisor Hors Classe, Eurostat
ESTAT
Eurostat
This presentation:
• describes the current state and next
steps with regard to big data in the
European statistical system;
• Outlines 3 concrete projects related to
big data launched by Eurostat in 2013;
• Outlines challenges ahead putting a
focus on the issues that represent
unique opportunity for co-operation at
supranational and global level.
Mariana Kotzeva, Eurostat
ESTAT
Eurostat
Unstructured data – what does it mean?
• Data that are not arranged in columns and rows, not properly titled
or identified;
• Data for which it is necessary to apply tools to categorise, tag, pool,
identify key elements and make correlations in order to gain insight
from them;
• Content Unstructured data: refers to emails, documents and other
objects that are made up of free-following text;
Mariana Kotzeva, Eurostat
ESTAT
Eurostat
Big data?
Volume
Velocity
Mariana Kotzeva, Eurostat
ESTAT
Eurostat
Variety
Is unstructured data the same as big data?
• Unstructured data is just one piece of the puzzle;
• Unstructured data is responsible for the Variety characteristic of the
big data;
• Making sense of unstructured data isn’t about technology, it is a
business intelligence;
•
To transfer into actionable intelligence is the value added that
unstructured data bring to big data.
Mariana Kotzeva, Eurostat
ESTAT
Eurostat
Big data is high on the agenda of the European
Statistical System:
• Eurostat participates in the UNECE initiatives under the High
Level Group on Modernisation of Official Statistics;
• Memorandum on Big Data and Official statistics of the ESS,
adopted at 99th DGINs conference in September 2013;
• Eurostat created an internal structure that will deal only with
big data since 1st of Jan 2014;
• A Task Force on big data involving Member states is in process
of establishment for developing an ESS action plan and
roadmap on big data;
Mariana Kotzeva, Eurostat
ESTAT
Eurostat
European projects on big data launched in 2013
••Internet as a data Source for Information Society
statistics;
••Use of mobile positioning data for tourism
statistics;
••Collecting prices via the Internet for price
statistics;
Mariana Kotzeva, Eurostat
ESTAT
Eurostat
Internet as a data source for Information Society statistics
Current situation in Information society statistics:
Based on two traditional surveys - on households and on enterprises;
Purpose of the feasibility study:
Which items could be collected over the Internet?
Results:
•Items such as e-skills, e-commerce, use of e-government, use of
internet is feasible to be measured over Internet;
•Items measuring subjective opinions and “offline” items CANNOT be
measured using digital footprints;
•For enterprises, part of the ICT indictors would be available on the
enterprise’ websites but some of them will be available only via the
servers of the enterprises;
•Survey suggests a number of new indicators that can be derived form
Internet
Mariana Kotzeva, Eurostat
ESTAT
Eurostat
Use of mobile positioning data for tourism statistics
• Active mobile positioning;
• Passive mobile positioning.
Mariana Kotzeva, Eurostat
ESTAT
Eurostat
Issues identified so far:
• Reluctance of data providers to give access to data;
• Privacy concerns;
• Technological barriers;
• Diversity in data protection legislation;
• Diversity in the data sources that introduces even higher
complexity in programming generic software tools;
• Methodological and quality issues- i.e comparability versus
relevance.
Mariana Kotzeva, Eurostat
ESTAT
Eurostat
The way forward:
Official statistics community – now on the move!
Collaboration:
• •Big Data goes beyond statistics authorities
national/”government” strategies needed
• •Public-Private Partnerships (could help build-up)
• •Multidisciplinary teams – including legal expertise
Action:
• •Demystify “data science” and build up skills
• •Applications-driven approach and learning by doing .
Mariana Kotzeva, Eurostat
ESTAT
Eurostat
Thank you for your attention!
 mariana.kotzeva@ec.europa.eu
 http://ec.europa.eu/eurostat
ESTAT
Eurostat