XIIth Annual International Conference for
Professionals in Cultural Heritage
LoCloud Micro Services and
the Digitisation Workflow
Walter Koch (kochw@ait.co.at)
AIT-Angewandte Informationstechnik
Forschungs GmbH, Graz - Austria
LoCloud is funded by the
European Commission's ICT Policy Support Programme
Content
•
•
•
•
•
LoCloud Project (relevant activities)
Vocabulary Microservice
Microservice used in the OpenUp! Project
Digitisation Workflow (Books)
Digital Curation (Long Term Preservation)
2015-11-08
Walter Koch (AIT)
2
LoCloud Project
(relevant activities)
2015-11-08
Walter Koch (AIT)
3
locoud.eu
About:
– LoCloud is a Best Practice Network co-funded under the
CIP ICT-PSP programme of the European Commission.
– LoCloud is supporting small and medium-sized
institutions in making their content and
metadata available to Europeana.
– A cloud-based technology infrastructure will enable the
aggregation of local content, and
– a number of micro-services will help to reduce technical,
semantic and skills barriers and to render the content
more discoverable and interoperable.
2015-11-08
Walter Koch (AIT)
4
Microservices
Six development teams:
AIT, Athena RC, AVINET, IPCHS, UPV/EHU, VUKFC
All the services are implemented on virtual machines in a
cloud testlab (using the OpenNebula cloud computing
platform)
API documentation is available for each service
End-user interfaces are provided for three microservices
Each service has been integrated in MORe
2015-11-08
Walter Koch (AIT)
5
LoCloud Microservices
Cloud-based micro-services for metadata enrichment and
capture:
• Geo-location API
• Vocabulary service
• Historic Place Names service
• Geo-coding application
• Vocabulary matching service
• Background linking service
• Wikimedia application
http://support.locloud.eu/
2015-11-08
Walter Koch (AIT)
6
LoGeo API
• Uses NER-Name Entity Recognition
• Recognizes and returns one or more place names
candidates accompanied by geographical coordinates.
GPL
IPCHS
2015-11-08
Walter Koch (AIT)
7
Vocabulary service
• Based on “TemaTres”
• Incorporates 29 standard
thesauri
• allows users to add and
create vocabularies, and
add terms or translations
• add thesauri terms to all
items of a metadata
package
• Integrated in MORE
• Exploited by the LoCloud
vocabulary matching
microservice via its API
GPL
AIT
2015-11-08
Walter Koch (AIT)
8
Historic Place Names service
• A semi-automatic historical geoinformation management function
and web service.
• Enables local cultural institutions
to collaborate in developing the
Thesaurus of HPN and enriching
ingested metadata.
• Enables export of selected
LoCloud HPN Thesaurus data sets.
GPL
VUKF
2015-11-08
Walter Koch (AIT)
9
Geo-coding application
• End-user oriented application
• allows institutions to add geographical locations to existing content in
a controlled crowd-sourcing environment and then
• load the augmented data back into their original production
databases.
GPL
AVINET
2015-11-08
Walter Koch (AIT)
10
Vocabulary matching service
• Builds on work in the PATHS project (FP7)
• Automatically assigns relevant concepts and terms to item records
• Retrieves terms from the Vocabulary service
Apache License 2.0
UPV/EHU
2015-11-08
Walter Koch (AIT)
11
Background linking service
• Based on work in PATHS
• Follows the Named Entity
Disambiguation approach
• Enriches items with links to
external contextual
information in DBpedia and
Wikipedia
Apache License 2.0
UPV/EHU
2015-11-08
Walter Koch (AIT)
12
Wikimedia application
• A web service that uses a REST interface to communicate with
Wikimedia commons
• The main functionalities are harvesting content form
Wikimedia, parsing the harvested content, and identifying
useful entities that can be mapped to the ESE or EDM
metadata schemas.
• The mapped ESE / EDM records are then sent to MORe to be
delivered to Europeana.
Wikimedia Commons Licensing
Athena RC
2015-11-08
Walter Koch (AIT)
13
The LoCloud
Aggregation Structure
Enrichment
Services
Europeana
Cloud
Testlab
2015-11-08
Walter Koch (AIT)
14
About: Microservice
Wikipedia: https://en.wikipedia.org/wiki/Microservices
(2015-09-15)
[...] microservices is a
• software architecture style in which
• complex applications are composed
of small, independent processes
• communicating with each other using languageagnostic APIs.
• These services are small, highly decoupled and
focus on doing a small task,facilitating a modular
approach to system-building.
2015-11-08
Walter Koch (AIT)
15
Evolution of Architectures
pwc: Technology Forecast: Rethinking Integration, Issue 1, 2014, page 3
2015-11-08
Walter Koch (AIT)
16
Vocabulary Microservice
2015-11-08
Walter Koch (AIT)
17
http://vocabulary.locloud.eu/
• Vocabulary Microservices
• This is the entry page for the
LoCloud vocabulary application.
• LoCloud uses a variety of standard vocabularies
during the metadata enrichment process.
These vocabularies have been established with the
TemaTres tool and are available in the LoCloud cloud
testlab.
• In addition, it is possible to feed the tool with your
own vocabularies, establish new vocabularies or
add languages to existing vocabularies.
2015-11-08
Walter Koch (AIT)
18
Vocabulary WebService
20+ Functions, API
Parameters
Example
2015-11-08
Walter Koch (AIT)
19
Example: Search
UNESCO-Thesaurus
http://test113.ait.co.at/tematres/unesco/services.php?task=search&arg=peace
2015-11-08
Walter Koch (AIT)
20
SPARQL-Example
DISMARC-Genres
• Query: "Find all Broader Term for 'Acid Jazz‘“
• SPARQL End Point: http://test113.ait.co.at/tematres/vocab/sparql.php
Query
Result
Thesaurus
Entry
2015-11-08
Walter Koch (AIT)
21
Service Call („fetchTopTerms“)
embedded in a test application
MicroService
Vocabulary Web
Service
2015-11-08
Walter Koch (AIT)
22
Test Application Development
Designer View
Development of a
Drop Down List
2015-11-08
Walter Koch (AIT)
23
The Test Application:
Part of Data Entry
Path to the
Thesaurus Item
„Candidate Term“ 
Crowdsourcing activity 
Next Slide
(Vocabulary Management)
2015-11-08
Walter Koch (AIT)
24
Microservice used in the
OpenUp! Project (open-up.eu)
„Common Name Service“
2015-11-08
Walter Koch (AIT)
25
OpenUp! - Natural History
Aggregator for Europeana
Micro Service:
Common Name
Service
2015-11-08
Walter Koch (AIT)
26
OpenUp!
Transformation Workflow
Invoke Micro
Service:
Common Name
Service
Next Slide
2015-11-08
Walter Koch (AIT)
27
OpenUp!
Transformation Parameters
Micro Service:
Common Name Service
Service End Point (Cache)
2015-11-08
Walter Koch (AIT)
28
Result: OpenUp!
Enriched Meta Data Set
Latin Name
2015-11-08
Walter Koch (AIT)
Common
Names
29
Digitisation Workflow
(Books)
2015-11-08
Walter Koch (AIT)
30
Bibliotheca Alexandrina
http://www.bibalex.org
Points for MicroService Integration
2015-11-08
Walter Koch (AIT)
31
Digital Curation
(Long Term Preservation)
2015-11-08
Walter Koch (AIT)
32
Digital Curation
State-of-the-Art Examples (2010)
• Archivematica
– Linux VM based toolset allows producers and archivists
to implement a small scale workflow
• UC3 / CDL Merritt
– Micro-Services specifications
– implement building blocks for the Digital Curation
Lifecycle
2015-11-08
Walter Koch (AIT)
33
Archivmatica
Micro Services
2015-11-08
Walter Koch (AIT)
34
California Digital Library
UC3 Merritt
• California Digital Library UC Curation Center
• Independant, Interoperable Micro-Services
• Complex curation function through combination
of atomistic services
Knowledge
Enhancement
Annotation
2015-11-08
Walter Koch (AIT)
35
Micro-Services
(Merritt vs. Archivematica)
CDL/UC3-MERRITT
2015-11-08
ARCHIVEMATICA
Walter Koch (AIT)
36
High-Level Workflow
• OAIS activities provide
a sound basis
• Archivematica
Use Cases
• Merritt specifications
• Micro-Services are the
building blocks
2015-11-08
Walter Koch (AIT)
37
CDL
Curation Micro Services
1)
Annotation service 2)
The Annotation service provides
a mechanism by which the
consumers of managed digital
content can enrich that content
through additional description,
association, or related content.
The service supports the
following methods:
• Annotate
• Submit.
1) „An Emergent Micro-Services Approach to Digital Curation Infrastructure“, IPRES 2009
https://escholarship.org/uc/item/5313h6k9
2) UC Curation Center / California Digital Library, merritt: An Emergent Micro-services Approach
to Digital Curation Infrastructure; Rev. 0.6, 2010-03-25
2015-11-08
Walter Koch (AIT)
38
Contact
Walter Koch, kochw@ait.co.at
http://www.ait.co.at
2015-11-08
Walter Koch (AIT)
39