Flamenco Image Browser:
Using Metadata to Improve Image Search During
Architectural Design
Ame Elliott
Group for User Interface Research (GUIR)
& Dept. of Architecture
University of California, Berkeley
Problem Statement
Architects begin new projects by looking at
images of prior projects


How are elementary schools put together?
What’s new in patio materials?
Current image search interfaces are not
helpful for image browsing in the early phase
of design


Rely on well-formulated textual queries
Ignore context of architecture
Flamenco Image Browser
2
Scope of the Project
The Flamenco Image Browser is one customization
of a toolkit for domain-specific search (the Flamenco
projects)
The Flamenco Image Browser is one component of
my dissertation
Understanding Architects’ Work Practices
(ethnography, task analysis, interviews)
Prototyping Tools Hardware:
.
Digital Desk
Evaluation
Flamenco Image Browser
Software:
Image Search UI
Quantitative Qualitative
3
Scope of the Project
The Flamenco Image Browser is one customization
of a toolkit for domain-specific search (the Flamenco
projects)
The Flamenco Image Browser is one component of
my dissertation
Understanding Architects’ Work Practices
(ethnography, task analysis, interviews)
Prototyping Tools Hardware:
.
Digital Desk
Evaluation
Flamenco Image Browser
Software:
Image Search UI
Quantitative Qualitative
4
Flamenco Image Browser
Interface to a collection of architectural
images using metadata about the
images to support search
Metadata about the images provides
Query previews giving hints about possible
next actions
 Ways of broadening or refining search
terms

Flamenco Image Browser
5
Why Use Metadata?
Open questions about its utility

How to use metadata not well understood
Not an obvious thing to do for images


Contrasts with automated content analysis
Defies conventional wisdom



If a picture is worth 1,000 words, let the pictures speak
for themselves
Professional architects have different needs than
curators
The thumbnail is the whole document, not like text
search using a title to stand in for a long article
Flamenco Image Browser
6
Faceted Metadata
Image:
Planalto Palace
Parti Communiste Francais
Pantheon
Architect:
Oscar Neimeyer
Oscar Neimeyer
Jacques-Gabriel Soufflot
Period:
20th Century
20th Century
17th & 18th C.
Location:
Brasilia
Paris
Paris
Concept:
Stone
Curvilinear
Stone
Flamenco Image Browser
7
Faceted Metadata
Image:
Planalto Palace
Parti Communiste Francais
Pantheon
Architect:
Oscar Neimeyer
Oscar Neimeyer
Jaques-Gabriel Soufflot
Period:
20th Century
20th Century
17th & 18th C.
Location:
Brasilia
Paris
Paris
Concept:
Stone
Curvilinear
Stone
Flamenco Image Browser
8
Evaluation Methodology
Solicit feedback from architects to determine if
faceted metadata is helpful and how to present it

Not evaluating if the current metadata in the system is the
right metadata
Lo-Fi evaluation of paper prototype


1 hour one-on-one with 3 professional architects
Walk-through interactions on a paper computer, users thinkaloud (audio recorded), questions about the experience
Informal study of live version

1 hour one-on-one with 9 architects /grad students, 2 tasks
(audio recorded) and a survey
Flamenco Image Browser
9
734 images
of France in
the entire
collection
100 images of
France in the
1,000 results
for stone
Results of Lo-Fi Study
The difference between searching within results and
changing to a new search confused users


The “breadcrumb” did not help users orient themselves when
browsing results
Home >> Gehry, Frank >> Berlin
is different than
Home >> Berlin

Users were surprised at the outcome and could not
anticipate which they were getting (refine or expand).
Image search is exploratory, not reductive

Specific textual queries not important


“If I knew what I wanted to type in the box I’d just go get it
myself.”
Expanding search more important than refining
Flamenco Image Browser
12
Goals for Study of Live Web Site
Make iterative changes to the alternatives to
identify useful aspects of the UI – not
statistical analysis
Learn to what extent the metadata is useful
for searching


How much text is too much?
What kinds of queries will the users do?
Explore how to clarify searching within results
vs. starting a new search
Flamenco Image Browser
13
Informal Study of Live Version
Comparison- a form-based UI to the same collection of images
Task 1 – Free form search



Participants told they are helping to test an image search engine
Asked to talk about a project they’re working on and something
they’d like to find
Let them go at it and try to find images they’re interested in – they
can ask any questions they want but no formal instructions given
Task 2 – Treasure hunt

Participants given paper copies of 3 images (but no metadata) and
told to find them (range from easy to hard)
Exit survey
Flamenco Image Browser
14
About the System
Web browser used to view Cold Fusion generated pages,
queries made to a mySQL database
36,000 images from the UC Berkeley Architecture Slide library
5 facets of metadata about each image



Image title, architect, period, location, and concept
Shallow hierarchy for results in architect, period, and location
Concept facet is from the Getty Art and Architecture thesaurus - a
40,000 term hierarchical thesaurus of descriptors of the built
environment – this hierarchy not tested in the live version
Masonry
Stonework
Building Materials
Finishing Materials
Brickwork
Flamenco Image Browser
Shingles
17
Results: Metadata is Helpful
Very positive feedback about Flamenco

All 9 participants named the metadata in
the search results area as their favorite
aspect of Flamenco
Metadata was successful at giving hints
about where to go next

Perceived as useful “These are places I
can go from here.”
Flamenco Image Browser
18
Results: More Metadata Please
Participants asked for more metadata

Although there were complaints about the contents of the
metadata, users still wanted more


Longer lists of options (more hints)
Users wanted more control to make very specific searches
Half the participants requested the ability to control
order of results with metadata

Juxtapose visible images 2 different ways:


Overview (one image from each project) vs. like together ( all
images of a project next to each other)
Different than ranking for text retrieval (precision, recall), but
ordering does matter
Flamenco Image Browser
19
Results: Complaints
The UI was not successful at clarifying
searching within results vs. starting a new
search

Only 2 of the 9 participants understood the
distinction without discussion – but they want to do
both
The 1/3 of the participants who couldn’t find a
treasure hunt image felt that Flamenco was
slow

Corroborates findings that perceived system
speed is about finding what you want (Spool ‘00)
Flamenco Image Browser
20
Enthusiasm for Metadata
Contradicted my suspicions that a
sledgehammer and pick- through-the-rubble
approach would be preferable
No-one thought there was too much text

Adding more text preferable to adding another row
of images
Flamenco Image Browser
21
Conclusions
Metadata is useful for exploratory tasks

Good at giving hints about where to go next
Architects want to use metadata to get more
control


Of results display
Build complex queries
Sometimes the right word is worth more than
many pictures
Flamenco Image Browser
22
Contributions
Increase understanding of what makes
a good search interface
Provide insights about how image
search differs from text search
Flamenco Image Browser
23
Acknowledgements
Thanks to






Marti Hearst my advisor
Andy Edmonds for excellent coding
Andy Chou for help with the database
the Berkeley Museum Informatics Project for
access to the image collection
the Flamenco project team for search ideas
and the members of GUIR for helpful feedback
Flamenco Image Browser
24
Thanks!
Flamenco Image Browser
26
Related Work
“Berrypicking” model of information
seeking (Bates)
Information scent (Pirolli)
Query previews (Plaisant &
Shneiderman et al)
QBIC & Blobworld (content analysis)
Flamenco Image Browser
29
Architects’ Image Use Practices
Architects rely on personal image
collections and libraries of print media
Images emerge as relevant due to their
juxtaposition with other images
Flamenco Image Browser
30