JOINING UP GOVERNMENTS
EUROPEAN
COMMISSION
Promoting Semantic Interoperability
in Europe
vassilios.peristeras@ec.europa.eu
21 Oct 2013
Content
Introduction and Background
Semantic Standards
Work so far
Next Steps
Conclusion
2
Content
Introduction and Background
Semantic Standards
Work so far
Next Steps
Conclusion
3
Introduction and Background
4
Introduction and Background
5
Introduction and Background
Political context
Cooperating partners with compatible visions,
aligned priorities, and focused objectives
Legal interoperability
• Aligned legislation so that exchanged data is
accorded proper legal weight
Organisational
Interoperability
• Coordinated processes in which different
organisations achieve a previously agreed and
mutually beneficial goal
Semantic
Interoperability
• Precise meaning of exchanged information
which is preserved and understood by all parties
Technical
Interoperability
• Planning of technical issues involved in linking
computer systems and services
6
6
6
Introduction and Background
How to improve semantic interoperability
for eGovernment systems?
How do we promote technical interoperability?
7
Introduction and Background
How do we promote
technical
interoperability?
8
Introduction and Background
How to promote semantic interoperability?
9
Content
Introduction and Background
Semantic Standards
Work so far
Next Steps
Conclusion
10
Content
Introduction and Background
Semantic Standards
Work so far
Next Steps
Conclusion
11
Semantic Standards
No standard definition for “semantic standards”
Semantic Standards = commonly agreed ways to
describe information
Can be of two types:
• Data models (e.g. XML schema)
• Reference data (e.g. codelists, taxonomies, controlled
value lists)
12
Semantic Standards
Sharing and Reuse of semantic standards contributes to:
a) Reduce development costs, not reinventing the wheel
b) Reduce integration, and sharing of information costs
c) Increase interoperability between systems
13
Semantic Standards
Download the white paper on Joinup
https://joinup.ec.europa.eu/elibrary/document/towards-open-government-metadata
14
14
Semantic Standards
Semantic Standards Management
Semantic standards is an important asset for eGovernment
systems development and as such should be professionally
managed
Semantic standards management refers to the good practice of:
adopting policies, processes, and systems to plan, perform,
evaluate, and improve dissemination and use
15
15
Semantic Standards
Semantic Standards Management
Semantic standards management includes the following actions:
• Public administrations should identify
semantic standards with reuse potential
for developing eGovernment systems
• Public administrations should
systematically document semantic
standards
• Public administrations should check their
semantic standards to identify
inconsistencies, overlaps, opportunities
for harmonization
• Public administrations should make their
semantic standards open for reuse
REUSE
Existing
specifications
SHARE
Metadata
IDENTIFY
Metadata
DOCUMENT
Metadata
CHECK
Metadata
16
16
Semantic Standards
Semantic Standards Maturity Model
Semantic Standards Ignorance
Semantic standards are not documented, mainly because administrations don’t
consider this exercise important. This results in serious semantic IOP problems within
each country as developers use ad hoc data models, metadata, code lists, taxonomies,
etc for developing eGov systems.
Scattered and/or Closed Semantic Standards
Semantic standards may be documented but a) not in a centralised and organized way
and/or b) they are not available and accessible as open standards for developers
Open Semantic Standards for Humans
Semantic standards are documented, and are made available as open standards but
are not systematically published in a reusable format (e.g. only available as pdf
documents).
Open Reusable Semantic Standards
Semantic standards are centrally documented, are published as open standards, in a
machine readable format and/or provide an API for computers to access, query and
reuse them. Semantic Standards Management Systems (SSMSs) are introduced (e.g.
the Joinup platform, Digitalisér.dk).
Linked Open Semantic Standards
Semantic standards are documented using linked data principles and are managed by
advanced SSMSs.
17
17
Semantic Standards
Technology intensive systems
Technical Standards
Information intensive systems
Semantic Standards
18
Semantic Standards
… by ignoring and not using …
… we suffer from
waste in information industry…
19
Content
Introduction and Background
Semantic Standards
Work so far
Next Steps
Conclusion
20
Content
Introduction and Background
Semantic Standards
Work so far
Next Steps
Conclusion
21
Work so far
22
Work so far - ADMS
ADMS & Catalogue of
semantic standards
23
Work so far - ADMS
• Using the same semantic standards promotes
interoperability
• By reusing what is available you save resources, you
reduce risks, and you become more interoperable with
others
• A large number of semantic standards already exists
24
Work so far - ADMS
• The existing solutions are scattered in numerous
places and are very difficult to find
–
Several national initiatives to create
repositories/libraries/catalogues of semantic standards (e.g.
Germany, Denmark, Finland, Estonia…)
–
Standardization bodies and third party initiatives generate
valuable and highly reusable specifications (e.g. OASIS, W3C,
UN/CEFACT…)
–
Independent projects make available semantic standards to
their own websites
25
Work so far - ADMS
How could we promote the visibility and reuse of
existing semantic standards at the European level?
26
Work so far - ADMS
How could we promote the visibility and reuse
of existing semantic standards at the
European level?
- Costs
- Scalability
- Maintenance
- Sustainability
27
Work so far - ADMS
How could we promote the reuse of existing
semantic standards at the European level?
… by agreeing on a common language
(template) to describe semantic standards
… creating a yellow page infrastructure
with standards descriptions and links to
the actual standards
28
Work so far - ADMS
How could we promote the reuse of existing
semantic standards at the European level?
… enabling a federation of existing semantic
standards repositories and catalogues
–
Cross-querying and discovery should be supported
–
Respect the autonomy of each repository/catalogue
29
Work so far - ADMS
Federation of semantic assets repositories
http://ec.europa.eu/isa/library/videos/isa_adms.mp4
30
Work so far - ADMS
Common template (metadata) for describing semantic
standards
Asset Description Metadata Schema (ADMS)
31
Work so far - ADMS
Multi-disciplinary
Statistics
working group
October 2011-April 2012
57
16
7
people
Member States representatives
295
> 2500
Wide range of backgrounds
Business
Government
Repository owners
15
232
Number of virtual meetings
of ADMS Working Group
Number of messages on
ADMS WG mailing list
Number of accesses to ADMS
versions on Joinup
Number of public
comments
Number of JIRA issues
tracked and resolved
Standardisation bodies
Academia
Libraries
32
Work so far - ADMS
• May 2012: ADMS endorsed by the EU member states
(ISA Coordination Group)
33
Work so far - ADMS
ADMS implementation
ADMS-based federation of
semantic standards repositories
Catalogue of semantic
standards
• Semantic standards are described using ADMS
• Features simple and advanced search of semantic standards
• 2000+ semantic standards from 25 repositories are
currently searchable through Joinup (Sep. 2013)
Since January 2013
34
European Federated Interoperability Repository
European Federated Interoperability Repository
(EFIR)
ADMS-based federation of
interoperability solutions
Catalogue of interoperability
solutions
• Interoperability solutions include semantic standards, OSS
solutions, common services and tools, architectures
• All interoperability solutions will be described by ADMS
• The European Interoperability Architecture will be used to
index/classify the available solutions
35
Work so far - ADMS
ADMS as W3C Note
36
ADMS & Catalogue of
semantic standards
Core Vocabularies
37
Core Vocabularies
“Now! That should clear up a few things around here!”
38
Core Vocabularies
"…What has been discovered over the years is that there
are a number of (information) structures that are
universal and applicable to all kinds of organizations, both
private and public. There are four fundamental
categories: People and Organizations, Geography, Physical
Resources and Activities and Events"
David Hay, Describing the World: Data Patterns
39
Core Vocabularies
Core vocabularies
Simplified, re-usable, generic and
extensible data models that capture
the fundamental characteristics of a
data entity in a context-neutral
fashion.
CORE
PUBLIC
SERVICE
VOCABULARY
40
Core Vocabularies
Usage
- New systems: As default starting points for data
modeling
- Existing systems:
- As reference data models for integration and
information sharing
- As export specifications for publishing open data
CORE
PUBLIC
SERVICE
VOCABULARY
41
Core Vocabularies
Core Vocabularies Working Group
Multi disciplinary
working group
67
21
people
Core Location TF: chaired by EC Joint
Research Centre/H6 (INSPIRE Directive)
Member States and the
US, South Africa and
Croatia
EU institutions
+
W3C methodology
Core Business TF: chaired by DG MARKT
(European Business Registry project)
Standardization bodies
External experts/academia
Core Person TF: chaired by EUROJUST
42
42
Core Vocabularies
Core Vocabularies are available through the Join.up
platform
Core Vocabularies have been endorsed by the MSs in
the context of the ISA Coordination Group, May 2012
ISA Open Metadata License v1.1
https://joinup.ec.europa.eu/category/licence/isa-open-metadata-licence-v11
CORE
PUBLIC
SERVICE
VOCABULARY
43
Core Vocabularies
Core Business Vocabulary
W3C Note
44
DCAT – AP
DCAT
APPLICATION
PROFILE FOR
EUROPEAN
DATA PORTALS
Different
metadata
vocabularies
The DCAT Application profile for data portals in Europe
(DCAT-AP) is a specification based on the Data
Catalogue vocabulary (DCAT) for describing public
sector datasets and data catalogs in Europe
Limited
accessibility
and lack of
awareness
Limited
reuse of
open
datasets
45
ADMS & Catalogue of
semantic standards
Core Vocabularies
Pilots
using Linked Data technologies
46
Linked data pilots
1) Interconnecting Belgian address data using the
Core Location Vocabulary
– In collaboration with AGIV, CIRB, bpost, FEDICT, SPW, NGI /IGN
and EC INSPIRE team.
– Pilot available at: http://location.testproject.eu/BEL/
2) Linking national plant protection products
registers
In collaboration with DG SANCO and agencies from 8
Member States, i.e. Austria, Belgium, Germany, Greece,
Hungary, Poland, The Netherlands and Sweden.
•
•
Pilot available at: http://health.tesproject.eu/PPP/
using the Registered Organization Vocabulary
47
Linked data pilots
3) Publishing organisational data as Linked Open
Government Data
– In collaboration with the Greek Ministry of Administrative Reform
and e-Government and Agenzia per l'Italia Digitale.
– Pilot available at: http://org.testproject.eu/MAREG/
4) Reusable public service descriptions using the
Core Public Service Vocabulary
–
–
In collaboration with SPOCS and e-CODEX large scale pilots,
Flemish Intergovernmental Product and Service Catalogue and DERI,
National University of Ireland, Galway.
Pilot available at: http://cpsv.testproject.eu/CPSV/
48
Linked data pilots
5) Interconnecting maritime surveillance data using
the Core Location and the Registered Organization
Vocabularies
In collaboration with DG MARE, EMSA and the Spanish
Armada.
–
Pilot available at: http://maritime.testproject.eu/CISE/ (under
development)
49
Core Vocabularies uptake
6) The Flemish Local Authority Association has used all
Core Vocabularies and has created an extension to
cover local authorities needs
7) eSENS LSP will use ADMS to document interoperability
solutions across Europe
50
ADMS & Catalogue of
semantic standards
Core Vocabularies
Pilots
using Linked Data technologies
Guidelines & good
practices
51
Guidelines & Good Practices
rules
10
URIs
for persistent
Follow the pattern
e.g. http://{domain}/{type}/{concept}/{reference}
Re-use existing identifiers
e.g. http://education.data.gov.uk/id/school/123456
Link multiple representations
e.g. http://data.example.org/doc/foo/bar.html
e.g. http://data.example.org/doc/foo/bar.rdf
Implement 303 redirects for
real-world objects
Avoid stating ownership
e.g. http://education.data.gov.uk/ministryofeducation/id/school/123456
Avoid version numbers
e.g. http://education.data.gov.uk/doc/school/v01/123456
Avoid using auto-increment
e.g. http://education.data.gov.uk/id/school1/123456
e.g. http://education.data.gov.uk/id/school1/123457
Avoid query strings
e.g. http://education.data.gov.uk/doc/school?id=123456
Use a dedicated service
i.e. independent of the data originator
e.g. data.gov.uk and publications.europa.eu are decoupled from
specific government department and could readily be transferred and
run by someone else if necessary.
Avoid file extensions
http://education.data.gov.uk/doc/schools/123456.csv
https://joinup.ec.europa.eu/node/53858
52
Guidelines & Good Practices
More good practices & case studies
How Linked Data is transforming e-Government
https://joinup.ec.europa.eu/community/semic/document/case-study-how-linked-data-transforming-egovernment
Cookbook for translating relational data models to RDFs
Re-use
Do not reinvent
https://joinup.ec.europa.eu/community/semic/document/cookbook-translating-data-models-rdf-schemas
Core Vocabularies Handbook (in progress)
Business and cost models for Linked Government Data (in progress)
53
Content
Introduction and Background
Semantic Standards
Work so far
Next Steps
Conclusion
54
Content
Introduction and Background
Semantic Standards
Work so far
Next Steps
Conclusion
55
Next Steps I
• To further raise awareness on semantic standards, the
importance of semantic standards management, and
promote the use of ADMS to describe and publish
existing standards on the web
• To promote the use of Core Vocabularies
• To identify data of high value and make it available as
open and linked data (EU institutions and MSs)
• To work towards metadata alignment in MSs and all EU
institutions together with the EU Publication Office
56
Next Steps II
• To support DGs and MSs in Semantic Standards
Management
• To develop operational interoperability pilots using the
core vocabularies and linked data technologies
• To develop more core vocabularies in areas where
common specifications are missing and data sharing is
important
57
Challenges
Open Challenges
• Which process for creating semantic standards: Roles
for standardization bodies, governments, markets,
communities.
• How to align and promote the (re)use of semantic
standards: Alignment, internationalization
• Freedom versus regulation: Over-specification versus
lean start-up and minimum viable product approaches.
58
Content
Introduction and Background
Semantic Standards
Work so far
Next Steps
Conclusion
59
Content
Introduction and Background
Semantic Standards
Work so far
Next Steps
Conclusion
60
Conclusions
• Technical standards have contributed to mature
solutions at the technical interoperability layer for
decades
• Agreements, promotion and use of semantic standards
is needed to address problems at the semantic
interoperability layer
• The EC ISA Programme has initiated work in this area to
promote semantic standardization
61
Conclusions
How to prevent waste in information industry…
• Raise awareness on the importance of semantic standards
in your country
- White Paper: “Towards Open Government Metadata”
• Promote the Core Vocabularies in your country
- e.g. metadata standards, national interoperability frameworks
• Promote the use of ADMS to describe standards
- ADMS can be used to describe standards
Happy to support your projects…
62
To learn more…
•
ISA Programme, Action 1.1 on semantic interoperability:
http://ec.europa.eu/isa/actions/01-trusted-information-exchange/1-1action_en.htm
•
Joinup platform: https://joinup.ec.europa.eu/
•
Open Government Metadata: https://joinup.ec.europa.eu/elibrary/document/towardsopen-government-metadata
•
Core Vocabularies:
–
http://joinup.ec.europa.eu/asset/core_business/description
–
https://joinup.ec.europa.eu/asset/core_location/description
–
http://joinup.ec.europa.eu/asset/core_person/description
–
http://joinup.ec.europa.eu/asset/core_public_service/description
•
DCAT-AP: http://joinup.ec.europa.eu/asset/dcat_application_profile/description
•
ADMS: http://joinup.ec.europa.eu/asset/adms/description
•
Federation of semantic assets repositories:
http://joinup.ec.europa.eu/catalogue/all?current_checkbox=1
•
ADMS promotion video: http://ec.europa.eu/isa/library/videos/isa_adms.mp4
63
Contact us
vassilios.peristeras@ec.europa.eu
Visit our initiatives
Get involved
SOFTWARE
FORGES
COMMUNITY
ADMS.
SW
Follow @SEMICeu on Twitter
Join the SEMIC group on LinkedIn
CORE
PUBLIC
SERVICE
VOCABULARY
Join the SEMIC community on Joinup
64