Klaus Lepsky – Thomas Müller – Jens Wille
Cologne University of Applied Sciences & prometheus
Klaus Lepsky – Thomas Müller – Jens Wille
Metadata improvement
for
image information retrieval
Metadata improvement for image information retrieval – ISKO 2010
1
Rogier van der Weyden. St. Columba Altarpiece.
Adoration of the Magi. Central part. c.1455. Oil on
panel. Alte Pinakothek, Munich, Germany
Klaus Lepsky – Thomas Müller – Jens Wille
Cologne University of Applied Sciences & prometheus
Problem I
Triptych with the Adoration of the Magi, so-called
Columba Altarpiece, Munich, Alte Pinakothek, inv. nr.
WAF 1189-1191. Center panel: ca. 140 x 153, Wings
each: ca. 140 x 73 cm.
Title: St Columba Altarpiece (central panel)
Author: ROGIER VAN DER WEYDEN
Date: c. 1455
Technique: Oil on oak panel, 138 x 153 cm
Type: religious
Form: painting
Location: Alte Pinakothek, Munich
Metadata improvement for image information retrieval – ISKO 2010
2
Rogier van der Weyden
Weyden, Rogier /van
Rogier <van der Weyden>
The prometheus database with
Klaus Lepsky – Thomas Müller – Jens Wille
Cologne University of Applied Sciences & prometheus
Problem II
Pasture, Rogier /de le
Rogier <de LePasture>
Adoration
the magi
• more thanof400.000
pictures from 50 different collections
Roger <de LePasture>
St. Columba Altarpiece
• descriptions without any standardization
<de le Pasture>
Triptych with the Adoration of the Rogier
Magi
Roger <de le Pasture>
St• weak
Columba
Altarpiece
subject
indexing (central panel)
Regelet
<de le
Pasture>
St. Columba Altarpiece. Adoration of
the Magi.
Central
part
• resulting in low recall !
Rogelet <de le Pasture>
Triptych with the Adoration of the Pasture,
Magi, so-called
Rogier /de
Columba Altarpiece
Weyden, Roger /van der
Columba Altarpiece Adoration of the Magi. Central part
Roger <von Brügge>
Roger <de la Pasture>
Metadata improvement for image information retrieval – ISKO 2010
3
Art historical texts are texts about works of art and therefore
may be used as ‘metadata’ for these works.
Klaus Lepsky – Thomas Müller – Jens Wille
Cologne University of Applied Sciences & prometheus
Idea I
The central Adoration of the Magi, the only depiction of this kind in Rogier's extant
work, was a particularly popular subject in Cologne, for since the 12th century the city
had kept relics of the Magi, its most treasured possessions, in the cathedral. The
Netherlandish artist not only created this work for a patron in Cologne, but also drew
inspiration from an outstanding painting done in Cologne at this period, Stefan
Lochner's Altarpiece of the Patron Saints of Cologne, sometimes known as the
Cathedral
Altarpiece. information
This large retable,retrieval
painted in the
for the
chapel of by
Improving
of1440s
digital
images
Cologne town council, unites the patron saints of the city in a magnificent,
automatically connecting them with text based description
monumental work from which any trace of historical narrative has been eliminated.
Rogier, perhaps at the special wish of his patron, adopted the almost central position
of the Virgin Mary in the depiction of the Adoration; this was an unusual situation for
her in Netherlandish art. He also echoed other motifs, including the figure of a woman
seen in profile, dressed in green and with a plait of hair, who appeared on the left
wing of Lochner's altarpiece and again in Rogier's Presentation; details like the sword
and fluttering hatband of the youngest king; and, behind the Magi, the figure of a
bearded man holding a hat in front of his chest. Traces of the late Romanesque
church of St. Gereon in Cologne seem to have entered into the architecture of the
temple on the right wing - the polygonal rotunda, the gallery, and the small buttresses
on the outside.
(http://www.wga.hu/frames-e.html?/html/w/weyden/rogier/11columb/0columb.html)
Metadata improvement for image information retrieval – ISKO 2010
4
article “Drei Könige” (three kings)
homepage of rdk-web
Klaus Lepsky – Thomas Müller – Jens Wille
Cologne University of Applied Sciences & prometheus
Idea II
Metadata improvement for image information retrieval – ISKO 2010
5
identifiying names and bringing together
synonymous forms
Klaus Lepsky – Thomas Müller – Jens Wille
Cologne University of Applied Sciences & prometheus
Idea III
matching text and descriptions of pictures
• identifying corresponding text by
• finding names of artists in text
• finding names of works of art in text
• bringing together pictures and corresponding text
developing a system for an automatic standardization of
metadata and a matching algorithm for metadata and
text to improve poorly standardized metadata
Metadata improvement for image information retrieval – ISKO 2010
6
wordsearcher:
linguistic analysis, grammatical variants are derived to one basic form
kings > king #n
religious > religious #a, religion #n
Klaus Lepsky – Thomas Müller – Jens Wille
Cologne University of Applied Sciences & prometheus
lingo - linguistic approach to automatic text indexing
multiworder:
linguistic/lexical analysis, dictionary-based identification of phrases
„rogier van der weyden“
„adoration of the magi“
sequencer:
algorithmic identification of phrases, based on sequences of the word classes
holy kings > #a #n > „holy kings“
synonymer:
identifying and generating of synonyms
Pasture, Rogier /de le > „rogier van der weyden“
adoration of kings > „adoration of the magi“
Metadata improvement for image information retrieval – ISKO 2010
7
synonyms of names of pictures
Adoration of Kings
The Adoration of Kings
Klaus Lepsky – Thomas Müller – Jens Wille
Cologne University of Applied Sciences & prometheus
importance of synonyms
The Adoration of the three holy Kings
Adoration of the magi
synonyms of names of artists
Rogier van der Weyden
Weyden, Rogier /van
Rogier <van der Weyden>
Pasture, Rogier /de le
Rogier <de LePasture>
Roger <de LePasture>
Metadata improvement for image information retrieval – ISKO 2010
8
matching algorithm with levenshtein distance
using word distance!
Klaus Lepsky – Thomas Müller – Jens Wille
Cologne University of Applied Sciences & prometheus
perseusMatch
(1) grammatical variants
editing distance for characters('königs', 'könige') = 1
(editing distance for characters('kings', 'king') = 1)
editing distance for words('königs', 'könige') = 0
(editing distance for words('kings', 'king') = 0)
because
basic form('königs') = basic form('könige')
(basic form('kings') = basic form('king'))
(2) synonyms
editing distance for characters('könige','weise') = 6
(editing distance for characters('kings','magi') = 5)
editing distance for words(‘könige’,’weise’)= 0
(editing distance for words(‘kings’,’magi’)= 0)
because
synonyms('könige') contains 'weise'
(synonyms('kings')
contains 'magi')
Metadata improvement for image information retrieval – ISKO 2010
9
Klaus Lepsky – Thomas Müller – Jens Wille
Cologne University of Applied Sciences & prometheus
connecting text & pictures
"[...] Nic. Lancret stellte in der vor 1732
für den Marquis de Béringhen gemalten
Elementenfolge das Wasser durch eine
ländliche Ff.szene dar, die auch Allegorie
des Liebeswerbens ist (s. Sp. 256; Abb. 52;
Georges Wildenstein, L., Paris 1924, S. 70
Kat.nr. 2, s. auch Nr. 3, Abb. 5). [...]
LunaDiana als Herrin über Fischerei und
Schiffahrt zeigt ein Stich von Jan Saenredam
nach Hendrick Goltzius (um 1559: Abb. 22)."
lingo & perseus match
KünstlerIn: Hendrik Goltzius
Titel: Allegorie
Standort: Basel, Öffentliche
Kunstsammlung
Gattung: Malerei
Datierung: 1611
Metadata improvement for image information retrieval – ISKO 2010
10
Thank You for listening!
Metadata improvement for image information retrieval – ISKO 2010
11
Klaus Lepsky – Thomas Müller – Jens Wille
Cologne University of Applied Sciences & prometheus