Metadata Interoperability Framework
(MIF)
ELAG 2014
Naeem Muhammad
Sam Alloing
Agenda
•
•
•
•
•
Who are we?
Europeana inside
Mapping and transformation service
Demo
Future
LIBIS
• Service provider for libraries, museums and archives
• Part of university of Leuven (Belgium), but also
external partners
• 19 team members
Europeana Inside
• Goal is to facilitate the content delivery to Europeana
through CMS (=ECK)
– Europeana is a portal for aggregation of cultural heritage objects
• 26 partners (Cultural institutions, software providers (
Adlib, Zetcom,… ))
• 30 months (April 2012 – September 2014)
• Deliver 960000 records to Europeana
• Special topic content re-ingestion
– Europeana enrichments send back to the CMS
Europeana Inside
Europeana Connection Kit (ECK)
ECK local
ECK
Core ECK services
•
•
•
•
•
•
•
•
Metadata Definition
Set Manager
Statistics
PID Generation
Preview
Validation
Data push (Sword)/ data pull (OAI-PMH)
Mapping and transformation
Europeana Connection Kit (ECK)
ECK local
ECK
LibisCoDe – example of ECK local
• Our version of the ECK local
• A plugin for CollectiveAccess
(http://collectiveaccess.org/)
– CollectiveAccess is an open source CMS
– LibisCode is also open source
• Supports a LIDO, MARC and (future?) EAD installation
profile.
LibisCoDe – How to use it
CollectiveAccess or
other CMS
CollectiveAccess
with LIBIScoDe
Your CollectiveAccess
with LIBIScoDe
Convert your
format to
MARC/LIDO
Convert EDM
to your
format
ECK core
ECK core
Europeana
Europeana
Mapping and transformation service
• Transforms one metadata format in another
– You can use a default mapping, this is only the transformation
service
– Or create a mapping file with a custom mapping (mapping and
transformation service)
– Implemented are MARC to EDM, LIDO to EDM
• LIDO = XML format used by museum
• EDM = RDF format from Europeana
• Why we created the service?
– Service-oriented architecture
– Different metadata formats needed to be supported
Input
• MARC XML or LIDO
• Mapping rule file
Output
• Only EDM is supported
– RDF based format from Europeana
– Core classes are supported (ProvidedCHO, Aggregation,
WebResource)
– We will be adding Contextual classes shortly (Place, Agent,
Concept,…)
Mapping rules
• Example MARC
– [command],[marc tag + subfield],[edm field]
– Doesn’t use indicators at this moment, could change
Mapping rules
• Example MARC (2)
– You only need to know the EDM field not the format
• For example edm:rights can be in multiple classes, but you just put
edm:rights, the EDM class is derived
ProvidedCHO
WebResource
Mapping rules
• Example LIDO
– No concise way to select the correct LIDO field, so we use
XPATH-like expression
– [command],[LIDO path],[edm field]
• Attributes are supported
Supported commands (1/2)
Action
COPY
Definition
Example
Copies the value from a source field to a target field. COPY,/lidoRecID,dc:identifier
If different source fields are defined to be copied to
the same target field a new occurrence of the target
field will be created
APPEND Append a source field to a target field. The specified APPEND,/category/term, some text ,dc:title
source field will be appended to the target field. If
or
there is already metadata in a field. The information
will be added to that field and not replaced. At the APPEND,/category/term, ,dc:title
time of append extra information can be added to
the value of the source field. In case no extra
information is needed an empty space can be left.
SPLIT
The source field will be split on the defined character SPLIT,/descriptiveMetadata/eventWrap/eve
and put in the target field. (In the example it will be ntSet/event/eventMaterialsTech/displayMat
split on space)
erialsTech, ,dcterms:medium
Supported commands (2/2)
Action
COMBINE
Definition
Example
Multiple source fields can be combined in one
COMBINE, /category/term;/lidoRecID,
target field. The fields are combined with a space dc:title
LIMIT
Limit the number of characters in a source field
LIMIT,/descriptiveMetadata/objectIdentificat
ionWrap/inscriptionsWrap/inscriptions/inscri
ptionTranscription,9,edm:description
PUT
Add a value to a target field. The value can contain PUT,"Koninklijke Musea voor Kunst en
commas, but no 2 pipes (||), because this is used Geschiedenis, Brussel",edm:dataProvider
as replacement character
REPLACE
Replaces a value in the source field with a replace REPLACE,/administrativeMetadata/rightsWor
string in the target Field.
kWrap/rightsWorkSet/rightsHolder/legalBod
yName/appellationValue,d'Histoire,History,e
In the example d'Histoire will be replaced by
dm:provider
History in the edm:provider element
CONDITION With condition you can combine different actions CONDITION,{
and use a conditional flow.
IF[/descriptiveMetadata/eventWrap/eventSe
IF can be used to define conditional action. With
t/event/eventType/term,EQUALS,Production
nested conditions also ELSE IF can be used.
,DO(COMBINE, /category/term;/lidoRecID,
The CONDITION actions starts and ends with curly
dc:title)]
brackets ‘{ ’. A new line is also mandatory.
}
Web service
• No user interface
• Meant to be integrated in CMS or use a REST client
• dmt.php/DataMapping/<provider>/<batch>/Transform
– POST request
– Provider: This is the name of the organization making the
request.
– Batch: This is the name of the batch submitted for
transformation.
Webservice - Parameters
• records or record
– Zip file with multiple XML files or one XML with multiple records
are supported (the mimetype needs to be correct)
• mappingRulesFile
– Needs to have mimetype text/csv
– Without this parameter you use the Transformation service
• sourceFormat
– Needs to be LIDO or MARC
• targetFormat
– Needs to be EDM
Web service - Response
• Returns a request_id
• Use it to download finished record(s)
• Use status:
– 0: no request exist with this id
– 1: not yet mapped/transformed
– 2: ready to be fetched
Web service – Other (GET) requests
• Status
• Fetch
– Download records
– Parameter is request_id
• List
– List of supported metadata formats
Demo
• Three possibilities
– Rest client tool (e.g. restclient)
– Libiscode with input records generation + mapping
– Libiscode with external records + mapping
Future?
• Add input formats (CSV, Filemaker XML, some custom
XMLs…)
• Update/add output formats (add EDM contextual
classes, add other formats)
• Extend/add/update actions
• Add queuing (near future)
• Add mapping interface (or integrate with MINT?)
Thank you !
Questions ?
LIBIS
into-info@libis.be – www.libis.be
+32 (0) 16 32 22 66
De Croylaan 54 – PB 5592 – B-3001 Heverlee
Information.
Innovation.
Inspiration.