Project CLiMB
Computational Linguistics for
Metadata Building
Columbia University
Funded by the Andrew W. Mellon Foundation
2002-2004
Using Computational Linguistic Techniques
to Harvest Image Descriptors
CLiMB - Columbia University
1
Photograph courtesy of the Council of Industrial Design's Design Archive.
CLiMB - Columbia University
2
Increase access to digital images
CLiMB - Columbia University
3
CLiMB:
Interdisciplinary Research at
Columbia University
Funded by the Andrew W. Mellon Foundation
2002-2004
•
•
•
Libraries
Computer Science Department
Center for Research on Information Access
(CRIA)
CLiMB - Columbia University
4
CLiMB Project Members
Judith Klavans, PI
Stephen Davis
Angela Giral
Patricia Renfro
Bob Wolven
Roberta Blitz
Rebecca Passonneau
Veronika Horvath
David Elson
CLiMB - Columbia University
5
Problems in Image Access
Traditional approach:
labor intensive
expensive
CLiMB - Columbia University
6
Project CLiMB
Help image catalogers
provide subject access?
Harvest image descriptors
from existing literature?
CLiMB - Columbia University
7
Can we harvest image descriptors?
CLiMB - Columbia University
8
CLiMB Technical Contribution
CLiMB will identify and extract
• proper nouns
• terms and phrases
from text related to an image:
By September 14, 1908, the basis of the Greenes' final design had
been worked out. It featured a radically informal, V-shaped plan
(that maintained the original angled porch) and interior volumes of
various heights, all under a constantly changing roofline that
echoed the rise and fall of the mountains behind it. The chimneys
and foundation would be constructed of the sandstone boulders
that comprised the local geology, and the exterior of the house
would be sheathed in stained split-redwood shakes.
— Edward R. Bosley. Greene & Greene. London: Phaidon, 2000. p.127.
CLiMB - Columbia University
9
CLiMB Overall Goals
The essence of CLiMB:
• Use scholars themselves as “catalogers” by
employing scholarly publications
• Enhance existing descriptive metadata
The CLiMB project:
• Research: Development of richer retrieval
through increased numbers of descriptors
• Practice: Development of CLiMB Toolkit
CLiMB - Columbia University
10
Metadata Squeeze
Basic Elements
• Digital images
• Associated text
• Target object identification (TOI)
• CLiMB Toolkit
• Evaluation
CLiMB - Columbia University
11
Greene & Greene Architectural Records
and Papers Collection
Drawings and Archives
Avery Architectural and Fine Arts Library
Columbia University Libraries
CLiMB - Columbia University
12
NYDA.1960.001.00023
All Saints Episcopal Church (Pasadena, Calif.). Alterations
1902-1903
CLiMB - Columbia University
13
Greene & Greene Catalog Record
Author:
Title:
Published:
Greene & Greene.
[Mrs. Dudley P. Allen house, 1188 Hillcrest Avenue (Pasadena, Calif.).
Alterations.]
Residence of Mrs. Dudley P. Allen, 1188 Hillcrest Ave., Pasadena, Cal.
[graphic] : Alteration / Greene & Greene, Architects.
[1917]
Physical Details:
Location:
4 sheets : various media ; 87.8 x 57.3 cm. (34 5/8 x 22 5/8 in.)
Columbia University, Avery Architectural Drawings
Other Authors:
Greene, Charles Sumner, 1868-1957.
Greene, Henry Mather, 1870-1954.
Houses
Alterations
Architecture--Designs and plans--United States.
Mrs. Dudley P. Allen house, 1188 Hillcrest Avenue (Pasadena, Calif.)
Subjects:
Component Item: [1] Item no. NYDA.1960.001.03224. [AVERYimage]. Electric lighting -floor plan, part plan of basement : Sheet no.
Component Item: [2] Item no. NYDA.1960.001.00073. [AVERYimage]. [Electric lighting]
floor plan, part plan of basement.
CLiMB - Columbia University
14
Greene & Greene Bibliography
(associated texts)
• Bosley, Edward R. Greene & Greene. London : Phaidon, 2000.
• Current, William R. Greene & Greene: architects in the residential
style. Fort Worth [Tex.] : Amon Carter Museum of Western Art, [1974]
• Makinson, Randell L. Greene & Greene: architecture as fine art. Salt
Lake City : Peregrine Smith, c1977.
• Makinson, Randell L. Greene & Greene: the passion and the legacy.
Salt Lake City : Gibbs and Smith, c1998.
• Smith, Bruce. Greene & Greene masterworks. San Francisco :
Chronicle Books, c1998.
• Strand, Janann. A Greene & Greene guide [Pasadena, Calif. : G.
Dahlstrom, 1974]
CLiMB - Columbia University
15
CLiMB - Columbia University
16
Metadata Squeeze
Basic Elements
• Digital images
• Associated text
• Target object identification (TOI)
• CLiMB Toolkit
• Evaluation
CLiMB - Columbia University
17
Target Object Identification (TOI)
• New methodologies and software =
new concepts and terms
• “Authority” list
• Varies from collection to collection
 Greene & Greene – Project Names
 North Carolina Museum – Creator/Title
CLiMB - Columbia University
18
Greene & Greene Catalog Record
Author: Greene & Greene.
Title:
[Mrs. Dudley P. Allen house, 1188 Hillcrest Avenue (Pasadena, Calif.)]
Residence of Mrs. Dudley P. Allen, 1188 Hillcrest Ave., Pasadena, Cal. [graphic] :
Alteration / Greene & Greene, Architects.
Published:
[1917]
Physical Details: 4 sheets : various media ; 87.8 x 57.3 cm. (34 5/8 x 22 5/8 in.) or smaller.
Location:
Columbia University, Avery Architectural Drawings
Other Authors: Greene, Charles Sumner, 1868-1957.
Greene, Henry Mather, 1870-1954.
Subjects:Houses
Alterations
Architecture--Designs and plans--United States.
Pasadena (Ca.)
Mrs. Dudley P. Allen house, 1188 Hillcrest Avenue (Pasadena, Calif.)
Genre Or Form: Architectural drawings--American.
Floor plans.
Component Item: [1] Item no. NYDA.1960.001.03224. [AVERYimage]. Electric lighting -floor plan, part plan of basement : Sheet no.
CLiMB - Columbia University
19
TOI list for NCMA
57.21.1
Lodovico Carracci
The Assumption of the Virgin
60.17.51
Domenico Zampieri, called Domenichino
The Madonna of Loreto Appearing to St.
John the Baptist, St. Eligius, and St. Anthony
Abbot
52.9.168
Bernardo Strozzi
St. Lawrence Distributing the Treasures of the
Church
CLiMB - Columbia University
20
Deriving TOIs: Image Vendor Data
CLiMB - Columbia University
21
CLiMB - Columbia University
22
North Carolina Museum of Art
North Carolina Museum of Art: Handbook of the Collections.
Rebecca Martin Nagy, editor. Raleigh: The Museum of Art. 1998
CLiMB - Columbia University
23
North Carolina Museum of Art
CLiMB - Columbia University
CLiMB - Columbia University
25
CLiMB - Columbia University
26
VRA Core 3.0 Record
•
•
•
•
•
•
•
•
Record Type = work
Type = painting
Title = Banquet Piece
Measurements. Dimensions = 79.7 x 94 cm
Material = oil on panel
Creator = Uyl, Jan Jansz. den
Date. Creation = ca. 1635
Location. Current Repository = Raleigh (NC,
USA) North Carolina Museum of Art
• ID Number. Current Repository = 52.9.43
• Style/Period = Dutch 17th century
CLiMB - Columbia University
27
Metadata Squeeze
Basic Elements
• Digital images
• Associated text
• Target object identification (TOI)
• CLiMB Toolkit
• Evaluation
CLiMB - Columbia University
28
CLiMB TOOLKIT:
Process Flow
5. Review
4. Select Subject
Access Terms
3. Analyze Text
2. Load TOI List
1. Load Text
CLiMB - Columbia University
29
CLiMB Toolkit
http://www1.cs.columbia.edu/~delson/CLiMB/gui/dev2
CLiMB - Columbia University
30
VRA Core 3.0 Record with CLiMB Subject Terms
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Record Type = work
Type = painting
Title = Banquet Piece
Measurements. Dimensions = 79.7 x 94 cm
Material = oil on panel
Creator = Uyl, Jan Jansz. den
Date. Creation = ca. 1635
Location. Current Repository = Raleigh (NC, USA),
North Carolina Museum of Art
ID Number. Current Repository = 52.9.43
Style/Period = Dutch 17th century
CLiMB Subject = Dutch still life painting
CLiMB Subject = vanitas
CLiMB Subject = burned down candle
CLiMB Subject = glass
CLiMB Subject = pewter
CLiMB - Columbia University
31
CLiMB Metadata and
Cataloging Schemas
• MARC 21
Field for CLiMB metadata:
653 - INDEX TERM--UNCONTROLLED (R)
• VRA Core Categories, Version 3.0
Field for CLiMB metadata: SUBJECT
• Dublin Core Metadata Element Set, Version 1.1
Field for CLiMB metadata: SUBJECT AND
KEYWORDS
CLiMB - Columbia University
32
Metadata Squeeze
Basic Elements
• Digital images
• Associated text
• Target object identification (TOI)
• CLiMB Toolkit
• Evaluation
CLiMB - Columbia University
33
CLiMB
Toolkit
CLiMB Toolkit Evaluation
CLiMB
University
14- Columbia
April 2004
34
CLiMB Toolkit Evaluation
• Evaluation with expert users
 Visual resources professionals
 Art librarians
• Overall satisfaction with interface design
• Overall satisfaction with functionality
• Identified areas for further development
CLiMB - Columbia University
35
Thank you!
Any further questions?
www.columbia.edu/cu/cria/climb
CLiMB - Columbia University
36
CLiMB - Columbia University
37
CLiMB - Columbia University
38
CLiMB - Columbia University
39
CLiMB - Columbia University
40
CLiMB - Columbia University
41
CLiMB - Columbia University
42
CLiMB - Columbia University
43
CLiMB - Columbia University
44