INTRODUCTION
This paper hopes to address the two main problems in the field of art image retrieval. The first is
that access to art images is inherently more complicated than access to textual documents or access to
non-art images, which has resulted in the incoherent, underdeveloped state of the field . The second is
that the discipline is further hampered by its lack of information about its users and their informationseeking behavior. The implications of these shortcoming are dire but this paper attempts to provide a
clarification of the difficulties within the field as well as a starting point for research on the informationseeking behavior of the two presumed primary user groups of art image retrieval systems: artists and art
historians.
CURRENT RESEARCH ON THE TOPIC AND APPLICATION ISSUES
…..George’s Stuff goes here….
Image Indexing Approaches and their use by Image Retrieval Systems
There are essentially two approaches to image subject indexing. One is concept-based , and the
other is content-based. Concept-based indexing focuses on ideas represented by the image and is quite
subjective. Content-based indexing focuses on the color, texture, shapes and compositions of an image,
that is the image’s physical attributes.
Both indexing methods are important to art images in particular because as far as we can tell,
the users of art image retrieval systems have varied needs. These needs call for creative combinations of
concept-based and content-based access. These two forms of access are usually combined in art image
retrieval systems, the engines that help users to navigate indexed art image collections. There are more
systems than there are retrieval methods but all art image retrieval systems fall under one of these two
types: content-based image retrieval (CBIR) or text-based image retrieval (TBIR). In recent years, hybrids
have been developed but they merely combine the principles of these two system types.
CBIR is not to be confused with content-based indexing. CBIR systems may use content-based
indexing in order to operate but they may also use concept-based indexing. CBIR systems analyze input
(usually digitally) to retrieve images based on their content. Image stores are often indexed according to
a digital analysis of their content. TBIR systems analyze input to retrieve images based on the textual
access points assigned to them during the indexing process. These images may be indexed using the
concept-based or content-based method.
Each system type has its merits but the effectiveness of the system is greatly determined by the
needs of the user and how well the system meets those needs. The desires of the user(s) should
determine the design of any retrieval system, and documents must be indexed according to the function
of the retrieval system of which they are a part.
This logic has been applied to non-image retrieval systems for decades, and the field of non-art
image retrieval is also beginning to focus on the information-seeking behavior of its users. The domain
of art image retrieval, however, is not quite so user-oriented. This is most likely due to a poor
understanding of users in the field. Even though there are a plethora of debates about all aspects of art
image retrieval, there are very few user studies focusing primarily on the retrieval of artistic images.
IMPLICATIONS TO INDEXING
…George’s stuff goes here…
Prospective Users and the Effect their Needs have on the Appropriateness of an Art Image Retrieval
System
Artists
Presumed to be one of the primary user groups associated with art image access, artists are
nonetheless the least-studied of all of the user groups. This is perhaps due to the astonishing number of
reasons for their wanting to acces artistic images. Two hypothetical scenarios will be discussed below.
Each discusseds the hypthetical needs of artist-users but in each case, we draw the opposite conclusion
regarding what kind of art image retrieval system is most effective to meet the users’ queries.
One possible reason that artists may want intellectual access to
art images is for the purpose of provoking creative inspiration. This
concept is in its infancy and is discussed more by experts in the field of
artificial creativity (a sub-field within aritificial intelligence) than by
experts in information science. A retrieval system designed for this
purpose, an inspiration engine if you will, is inherently different from
most of the systems that librarians are used to dealing with. They tend
to require little bibliographic control, yet an inordinate number of
content-based and concept-based access points.
In theory, an inspiration engine takes user input and uses it to
retrieve a parade of artistic images that will (hopefully) connect with
the user in such a way as to spark creativity. It is thought that creativity
sprouts from the serendiptous stimulation of emotion when the user is
in a certain frame of mind. For example, if the user is devastated by a
recent loss and wishes to create art, the emotion in the forefront of his
mind will be most meaningful to him, and therefore most likely to
spark artistic creativity.
Let us imagine that such an engine exists and that we are charged with
indexing Masaccio’s Expulsion from the Garden of Eden (on the left) for
use in the system. Concept-based indexing will be the most appropriate
indexing approach in this case. It is, of course, possible that an image’s
content may be associated with a certain emotion (i.e. the excessive use of the color red indicating
anger) but it is even more likely that the image is conceptually associated with a certain emotion. In this
scenario, concept-based indexing within a TBIR system may be most successful.
The collective indexing of emotion in images is a method that would work well for this type of
system. This method involves the mass-tagging of images with emotive nouns followed by statistical
analysis of the tags. The emotive nouns are then weighted according to the strength of viewers’
responses and assigned to the images respectively. It is not difficult to imagine that Expulsion from the
Garden of Eden might be collectively tagged with emotive words like devasation, sadness, or
embarassment.
Remember our imaginary artist-user has recently suffered a loss, which triggered feelings like
devastration and he would like to express this creatively. It follows logically that seeing examples of the
concept of devastation in art could spark inspiration. Let us pretend that his input is the word
“devastation.” The inspiration engine will be charged with retrieving artistic images that inspired
“devastation” in the tagging viewers. In most cases, Expulsion from the Garden of Eden will be received
by our artist-user in a similar way as it was received by the tagging viewers, as an instance of devastation.
Inspiration engines are multi-disciplinary in nature, but luckily, their users are well-defined and
they require input from information professionals on the indexing of art images.
*****
The artist user group could also seek the aid of an art image retrieval system while searching for
material for use in artistic compilations. These queries would be different from the one above because
the user is more likely to require specific shapes, colors or objects within the retrieved images. The
image’s bibliographic and conceptual information is likely to be less useful to this user than information
about the content of the image. For this artist-user, a CBIR system is more appropriate.
Let us image that this user is looking to create a born-digital collage and is in need of
some red, O-shaped objects to add to the composition. Concept-based textual indexing, so appropriate
for our previous artist-user, would put the onus on this user, who would be required to imagine all the
kinds of things that can be O-shaped and red, and then use those terms to search for images that match
his query. CBIR takes the burden off the user in this case. He would need only to specify his query using
pictorial input, perhaps by using a simple drawing function to draw a red O which will serve as his query.
The system would then digitally assess the user’s input, come to the conclusion that it is an Oshape and it is the color red. It would then search its database of images for instances of red O-shaped
objects, with no attention paid to the identity of the object or the concepts it represents. There is no
need to limit this type of retrieval system to artistic images only, but considering that no information
system is limited to one type of user with one type of query, it is possible that this hypothetical system
would be limited to artistic images for different reasons altogether. Let us imagine that it retrieves the
following objects:
In order to achieve the same results with concept-based textual indexing, the user would have had to
conceptualize his query on his own, searching with terms such as “red lips” or “Spanish sculpture” or
“lifesaver.” He may not have generated any of those terms on his own. The next best thing for this user,
in the absence of CBIR, would be to search the color red and browse through thousands of objects
looking for the O-shape that he needs. This would be extraordinarily inconvenient.
Art Historians
Art history is a discipline that deals with the deeper, conceptual interpretations of art. It is therefore not
surprising that concept-based textual indexing would be a useful tool for such a user. But it is not the
only method of art image retrieval that is appropriate for an art-history-user. Let us imagine that an art
historian wishes to write an article about representations of war in 20th century art.
ASSESSMENT
Recall that artist-user A had needs that required concept-based textual indexing, more
specifically collective indexing of emotions. Also recall that artist-user B had needs that required
content-based indexing and an automated content-based retrieval system. Thus, it is completely
possible that CBIR would be appropriate for an artist-user in one case, when concept-based textual
indexing was appropriate for our first artist-user. Both users are artists, and both are, for the most part,
seeking artistic images to use in their artistic endeavors. But their circumstances are drastically different.
This is why user studies are so important to the development of art image retrieval systems. It is almost
impossible to argue the significance of one system over another without having a comprehensive
understanding of users, their information needs, and their information-seeking behaviors.