Project CLiMB
Computational Linguistics for
Metadata Building
Columbia University
Funded by the Andrew W. Mellon Foundation
2002-2004
Using Computational Linguistic Techniques
to Harvest Image Descriptors
1
Session Outline
Subject Access in a Digital Age:
A Sisyphean Task
Angela Giral - Columbia University
CLiMBing the Mountain: Automating
Subject Access for Image Professionals
Roberta Blitz - Columbia University
A New Vista?
Preview and Feedback for the CLiMB ToolKit
Rebecca Passonneau - Columbia University
2
Session Outline
Subject Access in a Digital Age:
A Sisyphean Task
Angela Giral - Columbia University
CLiMBing the Mountain: Automating
Subject Access for Image Professionals
Roberta Blitz - Columbia University
A New Vista?
Preview and Feedback for the CLiMB ToolKit
Rebecca Passonneau - Columbia University
3
Subject Access
in a Digital Age:
A Sisyphean Task
Angela Giral – Columbia University
4
Thank you!
Any further questions?
www.columbia.edu/cu/cria/climb
5
Session Outline
Subject Access in a Digital Age:
A Sisyphean Task
Angela Giral - Columbia University
CLiMBing the Mountain: Automating
Subject Access for Image Professionals
Roberta Blitz - Columbia University
A New Vista?
Preview and Feedback for the CLiMB ToolKit
Rebecca Passonneau - Columbia University
6
CLiMBing the
Mountain:
Automating Subject Access
for Image Professionals
Roberta Blitz – Columbia University
7
Presentation Outline
1. Introduction to CLiMB
2. Two examples with images from the
North Carolina Museum of Art
3. CLiMB Subject Access Terms
8
CLiMB:
Interdisciplinary Research at
Columbia University
Funded by the Andrew W. Mellon Foundation
2002-2004
•
•
•
CRIA (Center for Research on Information Access)
Libraries
Computer Science Department
9
CLiMB Project Members
Judith Klavans, PI
Stephen Davis
Angela Giral
Patricia Renfro
Bob Wolven
Roberta Blitz
Rebecca Passonneau
Veronika Horvath
David Elson
10
Problems in Image Access

Cataloging digital images

Traditional approach:
manual expertise



labor intensive
expensive
Can automated techniques help?
11
Can we harvest image descriptors?
12
CLiMB Technical Contribution
CLiMB will identify and extract
• proper nouns
• terms and phrases
from text related to an image:
By September 14, 1908, the basis of the Greenes' final design had
been worked out. It featured a radically informal, V-shaped plan
(that maintained the original angled porch) and interior volumes of
various heights, all under a constantly changing roofline that
echoed the rise and fall of the mountains behind it. The chimneys
and foundation would be constructed of the sandstone boulders
that comprised the local geology, and the exterior of the house
would be sheathed in stained split-redwood shakes.
— Edward R. Bosley. Greene & Greene. London: Phaidon, 2000. p.127.
13
CLiMB Overall Goals
The essence of CLiMB:
• Use scholars themselves as “catalogers” by
employing scholarly publications
• Enhance existing descriptive metadata
The CLiMB project:
• Research: Development of richer retrieval
through increased numbers of descriptors
• Practice: Development of CLiMB ToolKit
14
Squeezing Metadata out of
Scholarly Texts
• Image collection
• Associated text
• Target object identification (TOI)
• CLiMB ToolKit
• Evaluation
15
CLiMB Collections
• Greene & Greene Architectural Records,
Avery Architectural and Fine Arts Library
Columbia University
• Chinese Paper Gods Collection,
C.V. Starr East Asian Library
Columbia University
• Digital Images from the
North Carolina Museum of Art
16
Greene & Greene Architectural
Records and Papers Collection
Drawings and Archives
Avery Architectural and Fine Arts Library
Columbia University Libraries
17
NYDA.1960.001.00023
All Saints Episcopal Church (Pasadena, Calif.). Alterations
1902-1903
18
Greene & Greene Catalog Record
Author:
Title:
Published:
Greene & Greene.
[Mrs. Dudley P. Allen house, 1188 Hillcrest Avenue (Pasadena, Calif.).
Alterations.]
Residence of Mrs. Dudley P. Allen, 1188 Hillcrest Ave., Pasadena, Cal.
[graphic] : Alteration / Greene & Greene, Architects.
[1917]
Physical Details:
Location:
4 sheets : various media ; 87.8 x 57.3 cm. (34 5/8 x 22 5/8 in.)
Columbia University, Avery Architectural Drawings
Other Authors:
Greene, Charles Sumner, 1868-1957.
Greene, Henry Mather, 1870-1954.
Houses
Alterations
Architecture--Designs and plans--United States.
Mrs. Dudley P. Allen house, 1188 Hillcrest Avenue (Pasadena, Calif.)
Subjects:
Component Item: [1] Item no. NYDA.1960.001.03224. [AVERYimage]. Electric lighting -floor plan, part plan of basement : Sheet no.
Component Item: [2] Item no. NYDA.1960.001.00073. [AVERYimage]. [Electric lighting]
floor plan, part plan of basement.
19
Greene & Greene Bibliography
(associated texts)
• Bosley, Edward R. Greene & Greene. London : Phaidon, 2000.
• Current, William R. Greene & Greene: architects in the residential
style. Fort Worth [Tex.] : Amon Carter Museum of Western Art, [1974]
• Makinson, Randell L. Greene & Greene: architecture as fine art. Salt
Lake City : Peregrine Smith, c1977.
• Makinson, Randell L. Greene & Greene: the passion and the legacy.
Salt Lake City : Gibbs and Smith, c1998.
• Smith, Bruce. Greene & Greene masterworks. San Francisco :
Chronicle Books, c1998.
• Strand, Janann. A Greene & Greene guide [Pasadena, Calif. : G.
Dahlstrom, 1974]
20
21
Chinese Paper Gods
Anne S. Goodrich Collection
C.V. Starr
East Asian Library,
Columbia University
22
Pan-hu chih-shen
God of tigers
24
Chinese Paper Gods Catalog Record
Title: Chuang gong chuang mu [graphic].
Published: [193-]
Physical Details: 1 print : wood-engraving, color ; 34 x 30 cm.
In: Anne S. Goodrich Collection.
Location: Columbia University, C.V. Starr East Asian Library (CJK)
EAX GAC 1 no. 16
Subjects: Gods, Chinese, in art.
Folk art--China.
Genre Or Form: Woodcuts--Chinese.
Notes: Date according to time period Anne S. Goodrich collected prints in
Beijing.
Record ID: NYCP02-F20
25
Chinese Paper Gods Bibliography
(associated texts)
• Day, Clarence Burton. Chinese peasant cults : being
a study of Chinese paper gods. Taipei : Ch'eng Wen
Pub. Co., 1974.
• Goodrich, Anne Swann. Peking paper gods : a look at
home worship. Nettetal : Steyler Verlag, 1991.
• Laing, Ellen Johnston. Art and aesthetics in Chinese
popular prints: selections from the Muban Foundation
collection. Ann Arbor, MI : Center for Chinese
Studies, University of Michigan, c2002.
26
Chinese gods:
selection from LC Authority File
HEADING:
Used For/See From:
Nezha (Chinese deity)
Daluoxian (Chinese deity)
Jinhuan Yuanshuai (Chinese deity)
Jinkang Yuanshuai (Chinese deity)
Li Nezha (Chinese deity)
Luoche Taizi (Chinese deity)
Ne Zha (Chinese deity)
Nezhataizi (Chinese deity)
No-cha (Chinese deity)
Nuozha (Chinese deity)
Tailuoxian (Chinese deity)
Taizi Yuanshuai (Chinese deity)
Taiziyeh (Chinese deity)
Yühuang Taizi (Chinese deity)
Zhongtan Yuanshuai (Chinese deity)
Search Also Under: Gods, Chinese
27
Squeezing Metadata out of
Scholarly Texts
• Image collection
• Associated text
• Target object identification (TOI)
• CLiMB ToolKit
• Evaluation
28
Target Object Identification (TOI)
• Define based on institutional needs
• Varies from collection to collection
– Greene & Greene – Project Names
– Chinese Paper Gods – God Names
– North Carolina Museum – Artist / Work Names
• Compile authority list
29
Deriving TOIs from creator & title lists
30
North Carolina Museum of Art
Museum Catalog
(Associated Text)
Images
(Catalog Records)
North Carolina Museum of Art: Handbook of the Collections. Ed. Rebecca Martin Nagy.
Raleigh, NC: North Carolina Museum of Art, Hudson Hills Press, 1998.
31
North Carolina Museum of Art
Museum Catalog
(Associated Text)
Images
(Catalog Records)
• village of Cebolla
• adobe Church of Santo Niño
• rusted tin roof
• window
North Carolina Museum of Art: Handbook of the Collections. Ed. Rebecca Martin Nagy.
Raleigh, NC: North Carolina Museum of Art, Hudson Hills Press, 1998.
32
Georgia O'Keeffe (American, 1887-1986)
Cebolla Church, 1945
Oil on canvas, 20 1/16 x 36 1/4 in. (51.1 x 92.0 cm.)
Purchased with funds from the North Carolina Art Society (Robert F.
Phifer Bequest), in honor of Joseph C. Sloane, 72.18.1
North Carolina Museum of Art
<http://ncartmuseum.org/collections/highlights/20thcentury/20th/1910-1950/038_lrg.shtml>
33
MARC format
100
245
260
300
500
500
500
535
650
650
650
O’Keeffe, Georgia, ≠d 1887 -1986.
Cebolla church ≠ h [slide] / ≠ c Georgia O’Keeffe.
≠c2003
1 slide : ≠ b col.
Object date: 1945.
Oil on canvas.
20 x 36 in.
North Carolina Museum of Art ≠ b Raleigh, N.C.
Painting, American ≠ y 20th century.
Women artist ≠ z United States
Church buildings in art.
34
Cebolla Church, 1945
Oil on canvas, 20 1/16 x 36 1/4 in. (51.1 x 92.0 cm.)
Purchased with funds from the North Carolina Art Society (Robert F.
Phifer Bequest), in honor of Joseph C. Sloane, 72.18.1
Driving through the New Mexican highlands near her home, Georgia
O'Keeffe would often pass through the village of Cebolla with its rude
adobe Church of Santo Niño. The artist was moved by the poignancy of
the little building: its sagging, sun-bleached walls and rusted tin roof
seemed so typical of the difficult life of the people.
When O'Keeffe came to paint the church she addressed it directly,
emphasizing its isolation and stark simplicity. Literally formed out of the
earth, the building affirms the permanence and the hard, defiant patience
of the people. For O’Keeffe, it symbolized human endurance and
aspiration. "I have always thought it one of my very good pictures", she
wrote, "though its message is not as pleasant as many others".
And the question remains: What is that in the window?
35
MARC format with CLiMB subject terms
100
245
260
300
500
500
500
535
650
650
650
O’Keeffe, Georgia, ≠d 1887 -1986.
Cebolla church ≠ h [slide] / ≠ c Georgia O’Keeffe.
≠c2003
1 slide : ≠ b col.
Object date: 1945.
Oil on canvas.
20 x 36 in.
North Carolina Museum of Art ≠ b Raleigh, N.C.
Painting, American ≠ y 20th century.
Women artist ≠ z United States
Church buildings in art.
CLiMB
CLiMB
CLiMB
CLiMB
CLiMB
CLiMB
CLiMB
CLiMB
New Mexican highlands
village of Cebolla
adobe Church of Santo Niño
sagging, sun-bleached walls
rusted tin roof
isolation
human endurance
window
36
Joseph Cornell (American, 1903-1972)
Suzy's Sun (For Judy Tyler), 1957
Mixed media construction, 10 3/4 x 15 x 4 in. (27.3 x 38.1 x 10.2 cm.)
Purchased with funds from the State of North Carolina, 78.1.1
North Carolina Museum of Art
<http://ncartmuseum.org/collections/highlights/20thcentury/20th/1950-2000/030_lrg.shtml>
37
VRA Core 3.0
Record Type=work
Type=shadow box
Title=Suzy's Sun (For Judy Tyler)
Measurements.Dimensions=10 3/4 x 15 x 4 in. (27.3 x
38.1 x 10.2 cm.)
Material.Medium=mixed media
Creator.Personal name=Cornell, Joseph
Creator.Role=artist
Date.Creation=1957
Location.Current Repository=Raleigh (NC, USA),
North Carolina Museum of Art
ID Number.Current Accession=78.1.1
Subject=assemblages (sculpture)
38
Suzy's Sun (For Judy Tyler), 1957
Mixed media construction, 10 3/4 x 15 x 4 in. (27.3 x 38.1 x 10.2 cm.)
Purchased with funds from the State of North Carolina, 78.1.1
Joseph Cornell fabricated shadow boxes and filled them with objects collected
both by chance and choice. With invention and insight, he teased themes from
these unlikely groupings, most often revolving around time and memory.
In Suzy’s Sun (for Judy Tyler), the sun (a cutout from an antipasto tin) and the sea
(an implied presence) speak eloquently of life cycles and passing time. Equally
potent symbols, driftwood and the infinitely spiraling seashell readily bring to mind
the tides on which they ride, summoning a universal metaphor for the ebb and flow
of life itself. In small details-a postage stamp showing a multi-masted schooner,
the collaged word "hotel"-Cornell uses the romantic notion of travel to far-off lands
as additional commentary on one’s passage through life.
Cornell dedicated this box to an actress. Judy Tyler had just achieved a certain
celebrity when she was killed in an automobile accident. "Suzy" probably refers to
the artist’s assistant, Suzanne Miller. The sun, designated as Suzy’s, presides
over the box, as a life-sustaining force counteracting the finality of death.
39
VRA Core 3.0 with CLiMB subject terms
Title=Suzy's Sun (For Judy Tyler)
Creator.Personal name=Cornell, Joseph
Subject=assemblages (sculpture)
CLiMB=shadow boxes
CLiMB=sun
CLiMB=time
CLiMB=memory
CLiMB=cutout
CLiMB=antipasto tin
CLiMB=sea
CLiMB=life cycles
CLiMB=passing time
CLiMB=driftwood
CLiMB=spiraling seashell
CLiMB=tides
CLiMB=metaphor
CLiMB=ebb and flow of life
CLiMB=postage stamp
CLiMB=multi-masted schooner
CLiMB=collaged word hotel
CLiMB=travel
CLiMB=actress
CLiMB=Judy Tyler
CLiMB=Suzanne Miller
40
CLiMB Metadata Can Be Employed
in Standard Catalog Formats
USMARC: MARC 21 Concise Format for Bibliographic Data
(http://lcweb.loc.gov/marc/bibliographic/ecbdhome.html)
Fields for CLiMB metadata:
650 - SUBJECT ADDED ENTRY--TOPICAL TERM (R)
653 - INDEX TERM--UNCONTROLLED (R)
VRA Core Categories, Version 3.0: A project of the Visual Resources
Association Data Standards Committee
(http://www.vraweb.org/vracore3.htm)
Field for CLiMB metadata: SUBJECT
Dublin Core Metadata Element Set, Version 1.1: Reference Description
(http://dublincore.org/documents/dces/)
Field for CLiMB metadata: SUBJECT AND KEYWORDS
41
Possible controlled vocabularies
• *AAT
Getty Art & Architecture Thesaurus
• TGN
Getty Thesaurus of Geographic Names
• ULAN
Union List of Artist Names
• LC NAF
Library of Congress Name Authority File
• LCSH
Library of Congress Subject Headings
• TGM I
Thesaurus for Graphic Materials: Subject Terms
• TGM II
Thesaurus for Graphic Materials:
Genre and Physical Characteristic Terms
Getty
LC
42
Controlled vocabulary matches
CLiMB terms
Controlled vocabulary
New Mexican highlands
uplands (LCSH)
used for highlands
New Mexico (LCSH / TGN)
villages (AAT / LCSH / TGM I)
Cebolla (TGN)
adobe (AAT)
churches (AAT)
adobe churches (LCSH)
Santo Niño (TGN)
walls (AAT / LCSH / TGM I)
village of Cebolla
adobe Church of Santo Niño
sagging, sun-bleached walls
rusted tin roof
human endurance
roofs (AAT / LCSH / TGM I)
rust (AAT)
tin (AAT)
solitude (TGM I)
used for isolation
[no matches]
window
windows (AAT / LCSH / TGM I)
isolation
43
Thank you!
Any further questions?
www.columbia.edu/cu/cria/climb
44
Session Outline
Subject Access in a Digital Age:
A Sisyphean Task
Angela Giral - Columbia University
CLiMBing the Mountain: Automating
Subject Access for Image Professionals
Roberta Blitz - Columbia University
A New Vista?
Preview and Feedback for the CLiMB ToolKit
Rebecca Passonneau - Columbia University
45
A New Vista?
Preview and Feedback
for the CLiMB ToolKit
Rebecca Passonneau – Columbia University
46
The CLiMB ToolKit
• Software prototype
• For large image collections
• Semi-automated metadata
– Subject access terms
– Human intervention at all steps
• Iterative development cycle
47
Presentation Outline
1. Process flow in ToolKit
2. Prerequisites to use ToolKit
3. Demonstration
48
Squeezing Metadata out of
Scholarly Texts
• Image collection
• Associated text
• Target object identification (TOI)
• CLiMB ToolKit
• Evaluation
49
CLiMB TOOLKIT:
Process Flow
5. Review
4. Select Subject
Access Terms
3. Analyze Text
2. Load TOI List
1. Load Text
50
Prerequisites and Resources
1. Image collection
2. Minimal-level catalog records
3. Texts about the images (associated texts)
•
•
Scholarly Monographs
Museum Catalog
4. TOI list of images
•
•
NOW: CLiMB format
FUTURE: Authoritative list of images in your format
5. [ Controlled Vocabularies ]
51
The Visible CLiMB ToolKit
A Graphical User Interface (GUI):
•
Web Browser
•
Help Menus
•
Projects
52
The Invisible CLiMB ToolKit
Tools and Resources:
• TOI List
• Texts
• Text analysis tools to
find terms for TOIs
• Controlled Vocabularies
• Rulesets
53
Squeezing Metadata out of
Scholarly Texts
• Image collection
• Associated text
• Target object identification (TOI)
• CLiMB ToolKit
• Evaluation
54
CLiMB and the Image Professional:
Iterative Development and Evaluation
 Do you like what you see?
•
Overall look and feel
•
Help and Documentation
•
Text Loader
•
Other functions
 Do you like what you don’t see?
•
Automatic Proposals for Subject Access Terms
55
CLiMB ToolKit Demo
http://climb.cs.columbia.edu/demo/
56
Thank you!
Any further questions?
www.columbia.edu/cu/cria/climb
57