2010 Analysis of existing metadata
case studies
Alice Born (Statistics Canada),
Joza Klep (Statistical Office of the Republic of
Slovenia)
METIS Work Session March, 2010
Existing case studies
17 existing cases studies from :
Albania (ALB), Australia (AUS), Austria (AUT), Canada
(CAN), Croatia (HRV), Czech Republic (CZE), Finland
(FIN), Germany (DEU), Latvia (LVA), Netherlands (NLD),
New Zealand (NZL), Norway (NOR), Portugal (PRT),
Slovenia (SVN), South Africa (ZAF), Sweden (SWE) and
the United Nations Industrial Development Organization
(UNIDO)
Available from the METIS-wiki
http://www1.unece.org/stat/platform/display/metis/
Scope of analysis
• In response to participants’ requests,
2010 analysis focuses on:
Section 1 and Section 2.4
Strategies for implementation
Section 4
System and design issues
Section 5
Organizational and workplace culture issues
Section 6
Lessons learned
Metadata strategies
• Template: Explanation of the overall strategy for
managing metadata across the organization.
• For example, the mandate/programme providing
framework for the project, and basic metadata
management principles used.
1. implicit metadata strategies
2. explicit metadata strategies
Implicit metadata strategies (part of wider
context)
•
•
•
•
•
•
New business architecture
Architecture for datawarehouse
SOA funcitionality
Process-oriented production
Finding and interpreting statistical data
Formal process for standardization
Explicit metadata strategies - examples
• Strategy for End-to-End Management of ABS Metadata - Australia
• Strategically, our metadata management system forms part of a
larger system of applications called the End-to-end Statistical Data
Management Facility (ESDMF). Metadata driven system is inevitable
because metadata is used and generated at every stage of the
statistical production process - South Africa
• The Business Model Transformation Strategy (BmTS) is designing a
metadata management strategy that ensures metadata - New
Zealand
• One of the primary objectives of the Integrated Metadatabase
(IMDB) is to inform users on concepts, methodologies and data
accuracy. The IMDB provides the metadata to support the statistical
products released by Statistics Canada's Dissemination Division,
and relates to the interpretability in the Agency's quality assurance
framework - Canada
Explicit metadata strategies - examples
• To standardize definitions across all statistical activities; to move the
production of statistics closer to the subject-matter experts in order
to speed up the statistical survey life cycle; to present statistics on
internet along with its context in order to make statistics
understandable and available to users of all types, i.e. to extend the
use of statistics beyond the usual statistical publications - Croatia
• Systematic use of metainformation inside and outside the SIS as a
tool for internal and external integration. SMS is focused on the
SPP. The model used for definition of a statistical variable ensures
its standard description from the beginning up to the end of LCST Czech Republic
Explicit metadata strategies - examples
• there are several projects - independently planned and implemented
- that involve a centralised metadata management. Taken together,
these projects form the work plan for the next 2-3 years. The task is
to combine the projects in a way that at least the outline of a
common metadata strategy starts to emerge - Germany
• The strategy focuses on establishing a conceptual framework, clear
roles and responsibilities, and a stepwise development involving
integration and linkage of systems - Norway
• In 2002 after thoughtful analysis of data and metadata flows,
Integrated Metadata Driven Statistical Data Management System
(further IMD SDMS) was created - Latvia
Implementation Strategy (2.4)
• Most countries reported a “step-wise”
implementation
• Also reported:
– swinging of pendulum
– integration of previously existing legacy
systems into an Integrated Metadata System
– »big-bang« with a subsystems
implementation strategy
Implementation Strategies – Stepwise
approached
• from a set of static web pages to definitions of concepts
variables and classifications
• Five surveys as pilots
• new metadata in parallel with general modificitaion of
statistical data system
• systems by systems, first classifications
• new development projects should act according to the
new business architecture
Implementation Strategies –
“swinging of the pendulum”
• from »big bang« (cathedral projects) – to »opportunistic
("piggybacking" on other projects ) and incremental with
a broad »master plan«
• development of a "2020 Vision" encapsulating longer
term ABS aspirations. Having clearly defined the state
the ABS aspires to reach longer term, the next step
would be to determine the most appropriate strategy for
moving forward.)
• these two considerations informed the development of
the "SESAME Framework" (Standards Enabled Shared
Active Metadata Environment) in 2008
Australia
Implementation Strategies –
integration of previously existing legacy
systems into the IMS
• In order to minimize the complexity of the
complete system, the individual components
(subsystems) should be able to work
independently, communicating with each other
and the central "Registry" by means of a web
service and program interface layer
Austria
Implementation Strategies
»big-bang« with a subsystems implementation
strategy
Czech Republic
System and design issues
Section 4 of case study template
4.1 IT Architecture
4.2 Metadata Management Tools
4.3 Standards and formats
4.4 Version Control and Revisions
4.5 Outsourcing v.s. in-house development
4.6 Sharing software components and tools
Tools and standards (4.2 and 4.3)
ISO-IEC 11179
59%
SDMX
47% (in use, planning to use)
DDI
29%
Neuchâtel
35%
Oracle database
29%
.NET
53%
PC-Axis
18%
Metadata system components
GSPBM
Corporate metadata system
Collection management system
Data archiving
Survey metadata (passive)
Process metadata (active)
Dataset registry
29%
41%
12%
29%
59%
35%
29%
Metadata system components
•
•
•
•
•
•
Data element registry
Classification system
Classification coding system
Questionnaire development tool
Questions and response choices
Questionnaires
59%
76%
12%
24%
41%
65%
Architecture and development (4.1, 4.4
and 4.5)
• Service-oriented architecture (SOA) and
enterprise service bus – 35% +
• In-house development – 59% and
mixed 29%
• Sharing of software – being discussed by
NSOs (UN Statistical Commission, MSIS)
Organisational and workplace culture issues
Section 5 of case study template
5.1 Overview of roles and responsibilities
5.2 Training and knowledge management
5.3 Partnerships and cooperation
Roles (5.1)
Subject-matter expert
IT experts
Methodologist
Dissemination expert
Standards
Project manager
Business analyst
Terminologist
One organizational metadata unit
82%
76%
65%
47%
29%
24%
12%
9%
47%
Training (5.3)
• Methods
– Intranet
– Manuals
– Workshops
– New employees
35%
24%
47%
18%
Partnerships and cooperation (5.4)
• ALB – worked with Statistics Sweden
• AUS – global partnership in software development,
implementation of SDMX, DDI
• AUT – University of Vienna, participation in METIS
• CAN – participation in METIS
• HRV – worked with Statistics Sweden
• CZE – internal
• FIN - participation in METIS, INSPIRE (spatial metadata)
• DEU - participation in METIS, Statistics Norway and
Swiss Federal Statistical Office
Partnerships and cooperation (5.4)
• LVA – metadata workshops delivered by Sweden,
Norway and Finland
• NLD - participation in METIS
• NZL - participation in METIS
• NOR - participation in METIS, Neuchâtel, Scandinavian
collaboration
• PRT – visits to CAN, advice to SVN and African
countries, SDMX Eurostat Workforce
• SVN - participation in METIS, PC- Axis Reference Group
• ZAF – visited AUS, NZL, LAV, Ireland and advice from
CAN
• SWE – member of Neuchâtel, PC-Axis Nordic Cooperation
Lessons learned: Main themes
•
•
•
•
•
•
•
Top management involvement
Significant change/framework
Quality of metadata
Complexities of metadata
Find a common language
Project management
International cooperation
Top management involvement
• It is a challenge to formulate a convincing business
case for metadata
• Business issue rather than IT
• All high-level units given a role
• Metadata strategy – official mandate
• All steps of statistical data processing for different
surveys allow standardization, while each survey
may require complementary functionality
Significant change/framework
•
•
•
•
•
Recognize that this is a major change
Communication strategy
Allow business areas to influence implementation
Integrate with business processes
Statisticians love frameworks so having one makes life
a lot easier
• Reference to formal documents like the metadata- and
IT-strategy is important (approved by the board of
directors)
Quality of metadata
• Some metadata is better than no metadata - as
long as it is of good quality
• Depends on cooperation, motivation and
competencies of metadata authors
• More efficient to start documenting the metadata
right at the outset of any new survey design
• Authors need to know and understand the how
and why of metadata
• Releasing metadata on the internet improves
metadata quality
Complexities of metadata
• Not one ideal structure/format
• Communication of complex metadata principles is a
challenge
• Papers for the METIS sessions and the common
metadata framework (especially the case studies) have
proven very useful, as they provide arguments for
discussions with statisticians and top management
• Fundamental principles of metadata management, which
have been defined by experts during recent years (i.e.,
In part A of the common metadata framework) will
become more and more commonly accepted standards
and state of the art for the production and dissemination
of statistical information
• Other metadata standards provide opportunities
Find a common language
• Applying externally recognised and supported standards
has a lot of benefits - including as a means of building
upon a wealth of intellectual efforts and experiences
from others
• Use a metadata framework as common language
• Training of statisticians, not only in the use of
applications but also, and above all, about the concepts
underlying the system and workflow of procedures
• Harmonization between subject areas
Project management
 Make simple prototypes as early as possible to get input
from users, written requirements are to abstract
 The advantages and disadvantages of a metadata model
can often only be properly evaluated once an it-system is
in place
 »Translation« from »statistical language« to »IT
language« can cause misunderstandings
• Keep in control of outsourced development activities,
supplier may have difficult time understanding the
statistical business
 Usability testing
• Opportunity costs caused by the non-existence of
centralized end-to-end metadata systems are rarely
found in accounting systems
International cooperation
• Might help to understand the subject of metadata
management
• Similarity of the statistical process is important for IT
personnel
• Building on existing international knowledge minimises
risks and maximises return on investment
• SMS is increasingly expected to interoperate with
metadata management as practised in other
communities (eg geospatial, academic/research) and
sectors
Future work
• New case studies in METISwiki – looking
for countries to add their case studies
• Convergence on the use of key metadata
standards and metadata models
– Can we move to more harmonization in
implementation across countries?
– Further analysis on comparing architectures
• Suggestions for improving template
• Add tables to WP on METIS website