Evaluation and Comparison of Art Digital Libraries
Ann de Witt and Lauren Rodriguez
553: Digital Libraries
Abstract
Digital libraries are becoming more popular than ever and provide users with a collection of
information at their fingertips. In order to establish what a successful digital library should look
like and contain, we must find a common set of criteria to evaluate digital libraries. This study
intends to evaluate and recommend a way to improve the quality of digital libraries by evaluating
interface and content of four art digital libraries.
Introduction
With the emergence of technology over the years, digital libraries have grown
significantly in order to meet the information needs of the 21st century user. As digital libraries
continue to evolve, the importance of evaluation becomes necessary to provide the most useful
and informative collections to libraries’ targeted user communities. Technology has given art
museums and libraries the opportunity to virtually share their collections with the public through
digital collections online. The success of art digital libraries, in particular, relies largely on the
interface design, usability of the site, and content to meet users’ information needs. Evaluation of
these digital libraries is a crucial element of their continuing improvement and success.
This study focuses on the critical evaluation and comparison of art digital libraries. Two
evaluation classes will be used to evaluate and compare four digital libraries, including content
and interface. The purpose of this study is to evaluate the usability and effectiveness of art digital
libraries and suggest ways for libraries to address user centered design issues and generate a
greater user experience. Fuhr et al. (2007) define usability in terms of a system that is “easy to
learn, flexible and adapts to user preferences and skills” (p. 26). They also characterize usability
as the measure of interaction between user and system, comprising one side of the interaction
triptych model, which describes relationships among three main digital library components: user,
system, and content (p. 25). Fuhr et al. (2007) note users’ growing expectations and demand for
service and functionality from digital libraries, placing emphasis on the quality of content and
services they deliver (p. 22). Xie (2008) also indicates the value of DL content to users,
assigning the highest value to collections that demonstrate accuracy and authority (p. 1359).
Mobile Interface and Content
Because of the increasingly important role of Internet-enabled mobile devices in
providing access to digital library resources, this digital library evaluation will include an
assessment of how each organization is delivering mobile versions of websites, with a focus on
mobile-friendly interface design and content. Studies have shown that library users are willing to
use cell phones to do research in digital libraries despite the significantly smaller screen size
(Boyhun, 2013, p. 15). Many libraries are now creating mobile-optimized websites to offer
patrons usable interfaces to search digital collections and deliver content and library services.
Users are now demanding more content and services on mobile library sites, and Boyhun (2013)
notes that users aren’t satisfied with simple mobile websites that present a fraction of the
information and features available on the main sites (p. 16). The author also notes that recent
trends in library mobile website production demonstrate that institutions are placing more
priority on research information, including the placement of a search box on mobile home pages
(p. 22). Included in their recommendations for designing mobile library websites, Boyhun
emphasizes the importance of a search box on home pages (rather than just a search link) and
offering more mobile-friendly content (p. 25).
Selection of Digital Libraries
For this evaluation, a diverse sample of art-related digital libraries was selected to
represent the diversity of DL formats and approaches as well as the variety of organizations
creating them, including museums, public libraries, and commercial companies. For the museum
DL category, the Louvre’s (http://www.louvre.fr/en) website will serve as an example. The
public library DL representative is the New York Public Library (http://www.nypl.org), which
features a beta version of its new digital collections site. The commercial DL to be evaluated is
the Bridgeman Art Library (http://www.bridgemanart.com/en-US). And for a subject-specific
digital library, Monet (http://www.monet2010.com/en), produced by the Galleries Nationales,
offers a different and creative approach to art digital library systems.
Criteria
Although there isn’t a consensus on specific criteria used to evaluate digital libraries, or a
standard framework, a number of recent papers have suggested criteria and measurement
methods to assess the usability of digital library components. Saracevic (2000) recommends six
classes of DL evaluation criteria derived from the literature, including interface, technology,
content, processes and services, user, and context. This paper will focus on the usability of art
digital libraries based on two of these evaluation classes, content and interface, because of their
important roles in delivering digital collections to users.
Interface navigation provides structure and organization so that users can effectively
search and use the content of the site while interface design creates an aesthetic appearance and
enhances site functionality. Hariri and Norouzi (2011) also point out that interface is one of the
most important aspects of usability because it is the medium through which users communicate
and interact with the system (p. 699). The interface determines the user’s experience with the
digital library and may affect the user’s decision to return to it in the future. Content is an
especially important component of a digital library to evaluate because of its central role in
collections access. Fuhr et al. (2007) define a central use of digital libraries as information
access, placing emphasis on the importance of content as a motivation for digital library usage
(p. 22). The content characterizes the digital library and provides information to meet the needs
of a community (p. 26).
A number of papers recommending specific criteria and measurement methods used to
assess the usability of digital library interface and quality of content are described below.
Interface
Hariri and Norouzi’s (2011) literature review of studies on evaluation criteria with
respect to DL interface suggests a framework for the evaluation of digital library user interface.
The authors find that navigation, searching, design, guidance, error management, presentation,
learnability, user control, consistency, and language are the top ten user interface evaluation
criteria cited in the literature (p. 715). These categories were used to structure our evaluation of
interface. These criteria are also largely consistent with Nielsen’s (1995) ten user interface
heuristics, with special attention to user control and freedom, consistency and standards, error
prevention, flexibility and efficiency of use, help users recognize, diagnose, and recover from
errors, and help and documentation. Xie (2008) also defines the following user-defined criteria
for evaluating interface usability: interface usability in general, search and browse functions,
navigation, help features, view and output options, and accessibility (p. 1357).
Regarding measurements in usability studies, Buchanan and Salako (2009) suggest a key
challenge of evaluating DLs is identifying what to measure and how (p. 638). They point out that
usability of a digital library is largely related to the interface component of human computer
interaction (p. 639). Based on previous research, they define the following attributes in guiding
measurement: effectiveness (with respect to completing tasks), efficiency (measured in terms of
time required to complete tasks), aesthetic appearance (with special attention to appropriateness
of system interface design, including layout, color, fonts, and graphic properties), terminology
(ability to comprehend terms and phrases used to describe content and functions; consistency of
terms used throughout site), navigation (measuring ease of navigating site using available tools
such as bars, menus, color, and typography; as well as location awareness), and learnability
(measuring how quickly new users learn the system and functions) (pp. 639-41). These attributes
are useful in guiding our measurement of interface usability.
Content
The content areas used to guide this DL evaluation include completeness, scope, size,
quality of digital images, accuracy, diversity, informativeness, authority, currency, and mobile
content. Xie’s (2008) study of user-defined evaluation criteria indicates that collection quality
can be assessed in terms of scope, authority, accuracy, completeness, and currency (p. 1357).
The author finds that users assigned the highest value to collections that demonstrate accuracy
and authority (p. 1359). Fuhr et al. (2007) suggest the relationship between the user and the
content depends on the information needs of the user. If the user feels the content is relevant to
their information needs they will evaluate the content highly (p. 26).
The following criteria is used to structure our assessment of interface performance:
Construct
Evaluation of interface usability
Objective
Evaluate (and compare) the usability of interfaces
used in four art digital libraries. A diverse sample
of art DLs, including a museum, public library,
commercial organization, and subject-specific DL
will allow for a rich comparison and evaluation of
DL attributes as they relate to visual art
collections.
Criteria
The following components or features will be
assessed: navigation, searching and browsing
functions, design, guidance, error management,
learnability, user control, consistency, language,
and mobile interface.
Measurement
The following attributes will be used to guide
measurement of interface usability: effectiveness
(completing tasks), efficiency (time it takes to
complete tasks), aesthetic appearance
(appropriateness of system interface design,
including layout, color, fonts, and graphic
properties), terminology (use of terms and phrases
as appropriate for target audience), navigation
(time and steps required to navigate site using
available tools such as bars, menus, color, and
typography; as well as location awareness), and
learnability (how quickly new users learn the
system and functions).
Measuring Instrument
The following attributes will be used to guide
measurement of interface usability: effectiveness
(completing tasks), efficiency (time it takes to
complete tasks), aesthetic appearance
(appropriateness of system interface design,
including layout, color, fonts, and graphic
properties), terminology (use of terms and phrases
as appropriate for target audience), navigation
(time and steps required to navigate site using
available tools such as bars, menus, color, and
typography; as well as location awareness), and
learnability (how quickly new users learn the
system and functions).
Method
Observation; through critical analysis and
usability assessment
Evaluation Tables
The evaluations are based on a scale of 1-5.
1=Poor; 2=Fair; 3=Good; 4=Very Good; 5=Excellent; N/A=not applicable
Table 1. Evaluation of Interface
Criteria
The
New York Public
Louvre
Library
Navigation
5
4
Searching and
3
4
browsing
functions
Design
5
5
Guidance
3
3
Error
N/A
N/A
Bridgeman Art
Library
5
5
5
4
N/A
Monet, Galleries
Nationales
5
1
5
4
1
Management
Learnability
User Control
Consistency
Language
Mobile Interface
4
4
5
5
3
Table 2. Evaluation of Content
Criteria
The
Louvre
Completeness
5
Scope
5
Size
5
Quality of digital images
3
Accuracy
5
Diversity
4
Informativeness
5
Authority
5
Currency
5
4
4
5
4
5
New York
Public Library
4
5
5
5
5
4
4
5
4
4
4
5
3
2
Bridgeman Art
Library
3
5
5
4
5
5
3
4
4
4
3
5
5
1
Monet, Galleries
Nationales
5
5
5
5
5
4
5
5
1
Discussion
A summary of the evaluations of interface and content for each of the digital libraries
discussed emphasizes significant areas that perform well or need refinement. A full evaluation,
with notations for each criterion, is included in the appendices.
Interface
Bridgeman Art Library’s interface is user friendly and well designed, making it easy and
fun to explore their vast collection of art. Searching and browsing functions are easy to find and
use and enhance the user’s interaction with digital collections. The search box is placed
prominently on every page, including the home page, and prompts users to enter a search term. A
number of browsing options are made available through the categories section (under
“Collections”), including all artists, subjects, and cultures. The navigation of the site is easy to
use, but it doesn’t offer a help feature (other than contact information), slightly downgrading the
guidance and learnability performance criteria. Although the lightbox feature is a nice addition to
the site and is made accessible on every page of the website, it may be an advanced feature out of
reach of novice users. Notably, Bridgeman doesn’t offer a mobile version of their website, but
the interface and content are searchable and usable on a smartphone. Because the interface isn’t
optimized for display on cell phones, certain features are blocked or not functional.
The Louvre's digital library/website interface is a great representation of the museum
with virtual tours and visitor trails, which allow the user to choose a thematic trail designed to
give the user an overview of a selection of works that typify a period, an artistic movement, or a
theme. Visitors can plan their visit, buy tickets in advance, view visitor tips, learn how to use the
Louvre, view floor plans, check out the cafes and restaurants, and view a calendar of events.
Navigating the site is simple, and the site is organized into different sections, making it easy for
the user to find what they are looking for or simply browse through the site. The search feature is
included in the quick select bar on the right of the page and a novice user may find it difficult to
use however clicking on the magnifying glass will bring the user to a search page that is easier to
use. When searching, the results are organized into categories using tabs, which may be
confusing to the user and may cause them to miss important information if they are not familiar
with the site. A secondary search function is included under “Collection & Louvre Palace,”
where the user has the option to conduct a simple search by keyword or accession number or
search the databases. The search section also includes a searchable collection of six databases. It
is easy to navigate back to homepage by clicking on the pyramid icon in the tab section at top of
a page or by clicking on the "Louvre" icon, which appears at the top left of all subsequent pages.
The Louvre does not offer a mobile version, but the interface and content are fairly easy to use
on a mobile device.
The Monet site is the most interactive of all the digital libraries evaluated in this study. It
was created to reflect the exhibit that took place at the Galleries Nationales of the Grand Palais in
Paris from September 2010 to January 2011. The interface is highly interactive and grabs the
user’s attention upon first entering the site and hearing the delicate music that plays softly in the
background of every page. There does not appear to be a search function on the main page,
however, there is one included in the gallery section, but it does not appear to work when a
search is conducted. The user has the option of exploring the gallery of Monet’s paintings by
using the timeline feature. An interactive journey is also offered, which incorporates the user's
webcam and microphone to engage the user through the virtual tour. The Journey is educational
and fun and great for users of all ages. The interface is very visually appealing including Monet’s
paintings in the background of the web page. Unfortunately, the Monet site does not offer a
mobile version for iPhone or Android because Adobe Flash Player 10 is needed to run the site.
The New York Public Library’s Digital Collection (Beta) website offers an overall userfriendly interface with which users can search and interact with their vast collection of digital
images. The searching and browsing features enhance the user’s exploration of content, with
prominent placement of a search box on every page, including the home page, and a number of
options for narrowing and filtering results. Filtering criteria make it easy to refine a search
without having to perform an advanced search (which doesn’t appear to be an option), and
provide access to content indexed by location, name, collection, and subject, for example. The
clean, uncluttered design and graphic qualities optimize the user experience. The navigation of
the main library website is accessible at the top of every page of the digital collections site,
which is potentially confusing for users that navigate away from the feature site or assume the
main navigation pertains to the digital collections site.
Content
With more than a million images of art available, the size and scope of Bridgeman Art
Library’s digital library make it an important online resource for art historians, art enthusiasts,
publishers, and the general public. Overall, the quality of digital reproductions is high, but the
digitization is contingent on the partner organization’s methods. While the size of image display
allows for users to view and select images for publication or other uses, the website doesn’t
allow for images to be magnified. A “zoomify”-type feature would enhance the study of image
details, but perhaps this is not a priority for the organization, since it is primarily an image
licensing business. Because Bridgeman sources its images and information from partner
organizations all over the world, the images and accompanying metadata are accurate and
credible, lending authority to the online resource. In addition, according to the website,
Bridgeman regularly adds new collections and artists--including contemporary artists--to its
website, lending value and currency (as well as expanding the scope and size) to the digital
library.
The Louvre’s digital collection is made up of Western art from the medieval period to
1848, formative works from the civilizations of the ancient world, and works of Islamic art. It is
a large and complete collection, offering a closer look at what collections the museum has to
offer. The searchable databases offer a look at hundreds of thousands of works in the Louvre’s
collection. The digital images are high quality, and full screen mode is offered; however, the site
lacks the option to zoom in on the work. Digital collections are very detailed, and the museum’s
virtual tours and visitor trails provide detailed information about the museum and its collection.
There are several categories and subjects to choose from to meet the user’s individual needs.
Selected works of art are accompanied by a full description of the work and its influence on the
time period. The Louvre’s digital collection is very credible, and the user has access to all the
information they would find in the museum itself, if not more. The Louvre’s website is updated
on a regular basis to reflect the current happenings of the museum, including current exhibitions
and activities. The digital collections are maintained to highlight the museum’s collection and are
updated regularly as needed.
The Monet site offers a complete collection of the “Claude Monet 1840-1926” exhibit
that took place in Paris from September 2010 to January 2011. The digital collection includes
153 of Monet’s paintings and 19 videos on Monet’s life and paintings as well as the official
opening of the exhibition. The quality of the digital images is superb and there is a zoom feature
available, which allows the user to zoom in on any part of the painting. By zooming in, the user
can see the individual brush strokes, cracks, and the finest details of the painting. Information on
paintings is provided by the Galleries Nationales of the Grand Palais, Rmn-Grand Palais and the
Musee d’Orsay in Paris. Paintings are accompanied by detailed information as well as events that
took place in Monet’s life at the time. The digital collection is very informative and would be
useful to anyone seeking information on the life of Claude Monet. Detailed descriptions of
Monet’s works are provided, and the videos included in the Web TV section feature information
about the artist’s life. One of the biggest drawbacks of the site is that it is not updated, which
prohibits the collection’s ability to grow and adapt to users’ needs.
The highlight of New York Public Library’s Digital Collection (Beta) site is its content.
With over 800,000 digital images available to users, the library has created a rich visual resource
for their online community. The scope of collections is characterized by the indexes, available as
results refining options on the search results page. Browsing the available subjects and
collections, for example, highlight the scope and diversity of content. The high quality of digital
reproductions, size of image display, and zoom feature enhance the viewing of collections
online. Accuracy and authority are evident by viewing metadata included for items as well as the
credibility of the NYPL website in general. In addition, NYPL makes its digital collection site
(as well as its main website) available in a mobile friendly version, with the search box placed
prominently on the home page.
The New York Public Library is the only DL in our sample that provides a website
specifically designed for mobile use. Bridgeman Art Library’s website is accessible on small
smartphone screens, but it’s not considered a mobile website. The Monet digital library does not
offer a mobile website, and Flash Player 10 is required to run it on a mobile device. The Louvre
also does not offer a mobile site, but the webpage is fully accessible through the mobile browser
and works the same as it would on a desktop computer.
The importance of relevance is an important consideration in evaluating content, and it’s
also highly subjective to the user. Fuhr et al. (2007) stress that the relationship between user and
content depends on the information needs of the user, therefore if the user feels the content is
relevant to their information needs, they will evaluate the content highly (p. 26). Because of the
subjective nature of relevance, it’s not included in this evaluation. A usability study focusing on
a group of users with specific information needs might incorporate relevance as a criteria
category.
Comparisons
Interface
The ability to easily search a digital collection is very important when it comes to
evaluating a digital library. Not all the digital libraries evaluated in this study featured functional
search bars making it difficult to quickly and easily find the information the user is seeking. The
Louvre’s search feature was a bit hard to navigate. It would be better if a search bar displayed on
the main page rather than the search icon located on the quick select toolbar, which is difficult to
control. The Monet site did not include a search function on its main page, and the one featured
in the gallery section did not appear to work properly.
Content
One notable comparison is the availability of mobile websites for users to search digital
libraries on the go. Despite the growing demand for mobile-friendly library websites that offer
research information and services, including a search function, only one of the digital libraries
included in this evaluation provides a mobile-optimized website, the NYPL. The New York
Public Library (2014) also offers a free mobile app for iPhones and an Android mobile app
through Google Play. The Bridgeman Art Library site doesn’t offer a fully mobile version of
their website, but the interface and content are searchable and usable on a smartphone. Because
the interface isn’t optimized for display on cell phones, certain features such as columns of
information in the search results page overlap causing text and images to be partially blocked.
Functionality is also an issue with the lightbox on mobile phones, because the drag-and-drop
image function is disabled. Similarly, the Louvre’s website does not offer a mobile version
making the site slightly harder to use in a mobile browser. The search function disappears when
zooming in on the web page making it difficult to access the search icon unless the page is fully
zoomed out. The Monet digital collection does not offer a mobile version and is unable to
operate on Safari mobile browser using an iPhone because Adobe Flash Player 10 is required in
order to operate.
Recommendations
Based on these evaluations of digital library interface and content, a number of
recommendations can be made. Regarding interface, a user-friendly navigation system is an
important structural component that facilitates an effective and efficient interaction with the site.
Libraries and museums should evaluate how navigation menus for the main website intermingle
and affect the digital library navigation menu. In addition to providing quality digital
reproductions of images, zoom features enhance the use of digital image collections, making it
possible to inspect details and learn more about an object.
With respect to content, digital libraries should be regularly maintained and updated in
order to stay current and cater to users’ needs. User feedback is an important step in evaluating
and improving the quality of a digital library. For example, digital libraries could offer a
suggestions area on their sites to solicit the opinions of users, as the New York Public Library
digital collections site does. In addition, the use of social media can help to promote a digital
library and generate user feedback. By interacting with users via Facebook or Twitter,
organizations can gauge patrons’ interests in specific collections, contributing to content
decisions and providing an enhanced user experience. Libraries should also create mobilefriendly websites or apps to facilitate the use of digital collections and meet the growing
demands of mobile users.
Conclusion
Digital library evaluation is still a fairly new area of study, and a framework of standards
must be developed further in order to contribute to an effective evaluation system. By conducting
this study, we were able to evaluate a number of core aspects of digital library interface and
content in order to determine how they affect a user’s experience. We found that interface was
extremely important when it came to user interaction and ease of navigation. In order for a
digital library to be successful, it must be user friendly and well designed to reflect the needs of
the user community.
Due to the large number and diversity of digital libraries available today and the lack of
consensus in the literature, it is difficult to narrow down a specific set of evaluation guidelines.
Focusing on interface and content criteria has provided useful information in assessing the
effectiveness and performance of digital libraries. Our findings suggest easy-to-use search and
browse features and navigation menus are especially important components that allow users to
interact with a digital library. The attractiveness of the digital library may also affect a user’s
perception of the system and probability of revisiting the site. Another important finding pertains
to the presence of mobile-friendly websites. In order to reach a larger community of users,
organizations should develop mobile versions of their digital libraries so people can search their
collections on the go.
References
Allen, N., & Bishoff, L. (2002). Collaborative digitization: libraries and museums working
together. Advances In Librarianship, V26, 43-81.
Bohyun, K. (2013). The Present and Future of the Library Mobile Experience. Library
Technology Reports, 49(6), 15-28.
Buchanan, S., Salako, A. (2009). Evaluating the usability and usefulness of a digital library.
Library Review, 58(9), 638-651.
De Rijcke, S., & Beaulieu, A. (2011). Image as Interface: Consequences for Users of Museum
Knowledge. Library Trends, 59(4), 663-685.
Fuhr, N., Tsakonas, G., Aalberg, T., Agosti, M., Hansen, P., Kapidakis, S., … Sølvberg, I.
(2007). Evaluation of digital libraries. International Journal on Digital Libraries, 8, 2138.
Given, L., & McTavish, L. (2010). What’s Old Is New Again: The Reconvergence of Libraries,
Archives, and Museums in the Digital Age. Library Quarterly, 80(1), 7-32.
Hariri, N., Norouzi, Y. (2011). Determining evaluation criteria for digital libraries' user interface:
A review. Electronic Library, 29(5), 698-722.
Kimmel, S.C. (2013). Taking a Page From the Art Museum Curation as Mediation. Library
Media Connection, 31(5), 16-17.
New York Public Library. (2014). NYPL Mobile Apps for iPhone, Android and Mobile Web.
Retrieved from http://www.nypl.org/mobile-help
Nielsen, J. (1995). 10 Usability Heuristics for User Interface Design. Retrieved from
http://www.nngroup.com/articles/ten-usability-heuristics/
Saracevic, T. (2000). Digital Library Evaluation: Toward an Evolution of Concepts. Library
Trends, 49(2), 350.
Xie, H. (2008). Users’ evaluation of digital libraries (DLs): Their uses, their criteria, and their
assessment. Information Processing & Management, 44(3), 1346-1373.
doi:10.1016/j.ipm.2007.10.003
Appendices (evaluations)
Commercial Library: Bridgeman Art Library
[http://www.bridgemanart.com]
I. Interface
Navigation
(measurement: time and steps required to navigate site using available tools such as bars, menus, color,
and typography; as well as location awareness)
 Login window appears when entering website home page; user must agree to either login or
register for a new account before being able to begin navigation of website
 After entering login credentials, user is directed to the home page where navigation bar and
search bar are prominently located on page
 Each main section of the navigation bar produces a drop-down menu of secondary navigation
choices upon rollover, directing user to all of the prominent sections of the website
 Search box in prominently located on home page and all other pages making it easy to start a new
search
 Secondary features, such as “explore themes,” “new collections,” and “latest news,” are located at
the bottom of the home page, accompanied by graphics and explanatory text; additional
navigation elements are listed under buttons to direct user to subtopics (such as “new
collections,” “browse collections,” and “browse artists” under “new collections”)
 Zero steps required to arrive at page with search option, making it immediately available to
search the digital library
 In footer section, navigation choices are repeated making it easy to enter different sections of site
from bottom of page
 Exploring different sections of the website through the main navigation bar takes user to different
search/browse options. For example, clicking on “All artists” under “Collections” offers a
directory of all artists represented in their database, in which user can browse alphabetically
 Lightbox menu is displayed on right-hand panel of search results page allowing user to easily
access saved selections/collections
Searching and browsing functions
Using main search function:
 Collections search box is placed prominently on home page, making it easy to find; text
populated in search field reads “Search,” directing user to enter text in search field
 Search results are displayed in a grid of thumbnail images with a summary of information about
the artwork; user can click on individual thumbnails to access page with details about the piece,
or they can simply scroll over thumbnail to see a larger image
 Search options on left-hand panel of search results page display a number of options to narrow
results, including image size, color, and format
 User can also customize search results page by selecting a small, medium, large, or no image to
display
 User can easily drag thumbnail images to the Lightbox panel (on right) in order to save selections
for later; registering for an account enables the user to save selected images to a lightbox
 User can search in five different categories, which display after clicking on “All content” inside
of the search box; specifically, users can narrow search to copyrighted items, out of copyright, or
all content, for example
 Additional indexing terms are useful in mining collections—from individual record pages,
categories, keywords, artists, and locations offer additional choices for browsing
Using browse features as Search:
 Includes browse collections feature, accessible from navigation panel; “new collections” option
offers a number of different artist categories for user to browse
 Exploring different sections of the website through the main navigation bar takes user to different
search/browse options. For example, clicking on “All artists” under “Collections” offers a
directory of all artists represented in their database, in which user can browse alphabetically. This
feature helps to convey the scope of content included in their database.
Design
(Measurement of aesthetic appearance: appropriateness of system interface design, including layout,
color, fonts, and graphic properties)
 Layout: Modern, simple, and user-friendly; blocks of color and graphics lend organization to the
layout without cluttering design
 Color: Simple white, black, and gray background and text allow for images to be the centerpiece
of the design; Signature color of Bridgeman logo is carried throughout website, applied to the
search icon, text boxes, and header text in the footer section; used sparingly the hot pink color is
effective and compliments the design of the website
 Fonts: sans-sarif fonts used throughout are clean, modern, and easy to read online
 Graphic properties: Featured images are used as centerpiece, such as on the home page, large
graphic image serves as background of the page; Image collections also provide a backdrop to the
search box at the top of the page; Images rotate out as user clicks on new page
Guidance
(Measurement from Neilsen’s Usability 10 Heuristics: “Help and documentation: Even though it is better
if the system can be used without documentation, it may be necessary to provide help and documentation.
Any such information should be easy to search, focused on the user's task, list concrete steps to be carried
out, and not be too large.”)
 Guidance is optimized on this website in several ways; customer service is an obvious priority for
this organization (i.e., commercial digital library)
 Instructions for login and registration for site are available; confirmation email contains brief but
useful information regarding activating new account; Login and register links are placed
prominently in top right corner of the page
 Use of lightbox is supported by instructions (“Drag thumbnails here to add to lightbox”),
especially useful for novice users
 Information about site features and collections is available from the main navigation menu, as is
contact information, but there isn’t a help option placed prominently
Error Management
(Measurement from Neilsen’s Usability 10 Heuristics: “Even better than good error messages is a
careful design which prevents a problem from occurring in the first place. Either eliminate error-prone
conditions or check for them and present users with a confirmation option before they commit to the
action.” Also: “Help users recognize, diagnose, and recover from errors. Error messages should be
expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a
solution.”)
 I didn’t encounter any errors
Learnability
(Measurement: how quickly new users learn the system and functions)
If we break Learnability down, we could include 3 sections:
 Novice user: Instructions regarding login and registering for a new user account are available but
brief, and does not inform user as to what benefits them as account holders; instructions are


available to help users add images to lightbox; links to view lightbox are easily accessible, but
this feature may be new to novice users (i.e., may need explanation)
Intermediate user: should be able to register for a new user account with ease; process requires
entry of contact information and email address to activate account; should be able to use
searching, narrowing search results, and adding collections to lightbox without having to seek
assistance
Advanced user: should be able to register for a new user account with ease, as well as using
search and browse features; may have prior experience with saving collections to a personal
account
User Control and Freedom
(Measurement from Neilsen’s Usability 10 Heuristics: “Users often choose system functions by mistake
and will need a clearly marked "emergency exit" to leave the unwanted state without having to go
through an extended dialogue. Support undo and redo.”)
 User is able to exit out of login/registration windows by clicking on clearly marked “X” box in
upper right-hand corner of page in order to continue to previous web page
 Search box is placed prominently on every page making it easy to start a new search if
dissatisfied or finished with previous search results
 No “back” button or “view search results” is available, making it necessary for user to use the
browser back button
 Main navigation menu is displayed on all pages of the website, providing user with ability to
navigate back to a previous section or explore different sections of the site
Consistency
 Website design–layout, colors, fonts—is consistent throughout, tying in the different pages users
encounter, including the main page, results page, and individual item page
Language
(Measurement of terminology: use of terms and phrases as appropriate for target audience)
 Main search box appears in main website navigation and visible on all pages of the site
 No explanation of terms and features such as “Lightbox,” “low-res,” view selected assets,” which
may be confusing for novice users
 Many of the keywords (metadata) applied to artwork is useful in support of searching images in
different ways, including names of colors, subjects, locations, and other information that relates
to an image
Mobile Interface
(using Chrome on a Google LG smartphone)
 if the DL offers a mobile version - yes
 how/if the interface is easy to use in the small format - interface is slightly altered to display on a
small screen; navigation links extend to two lines instead of one, and navigation drop-down
menus are activated by clicking on individual navigation categories; graphic boxes are smaller
and text appears in a list format; clicking on Bridgeman logo links to the home page, but it
appears to display the desktop version of the site, not mobile-friendly
 search and browse functions - search box is placed prominently on mobile home page; search
results page displays in an awkward to use format. The three page columns overlap on the screen,
making it difficult to see all text and images; search results appear in a very long vertical column,
causing the viewer to do a considerable amount of scrolling; Lightbox column follows the vertical
scrolling, allowing user to add thumbnails to their lightbox account, however the drag option
doesn’t work on the mobile platform, so this becomes a confusing feature; Images don’t display
in a size that is appropriate for the smartphone screen size, causing user to scroll left and right

other significant observations - overall, the Bridgeman website can be accessed on a smartphone,
but it isn’t optimized for display on the small screen size.
II. Content
Completeness
 It’s difficult to determine the completeness of Bridgeman’s digital art collections because the site
may only include a small sample of a partner organization’s digitized collections.
Scope

The range of art subjects, periods, and artists, including contemporary, is vast. In addition, the
website indicates that new artists and collections are regularly added to the site, increasing the
scope and size of the available works.
Size

According to the website, Bridgeman provides access to more than a million images through its
digital library
Quality of digital images
 Overall, the quality of digital reproductions is high, but the digitization is contingent on the
partner organization’s methods. While the size of image display allows for users to view and
select images for publication or other uses, the website doesn’t allow for images to be magnified.
A zoomify feature would enhance the study of image details, but perhaps this is not a priority for
the organization, since it is primarily an image licensing business.
Accuracy
 Information about images appears to be very accurate, as does the digital reproduction of art
objects. (Bridgeman works with partner libraries and museums to digitize images, helping to
ensure the accuracy of metadata and digital images that are close replicas of the originals).
Diversity
 Evident from Bridgeman’s browse categories, the collections included represent a diversity of
subject matter, periods of art, cultures, and art media, adding to the educational value of the
resource.
Informativeness
 It depends on the item; some item records contain descriptive information such as captions
explaining the art object and others may only contain identification information such as title, date,
artist, medium, dimensions; Website doesn’t contain educational programs, such as online
exhibitions to help interpret art for online visitors.
Authority
 The website content is very credible as the digital images and accompanying metadata are derived
from the partner organizations that own the artworks. It’s just as credible as visiting the actual
museum’s website.
Currency
 New collections and contemporary artists are regularly added to the library, helping to refresh the
content and make it up to date
Mobile Content
(using Chrome on a Google LG smartphone)

what content is available, if the content is different from the mother site, how it's different and if
it's still effective in a shortened version - Bridgeman’s mobile home page contains the same
information, links, and content as the desktop website home page; the information is not
abbreviated for the mobile version.
Museum Digital Library: The Louvre
(http://www.louvre.fr/en)
Interface
The Louvre's digital library/website is a great representation of the museum with virtual tours and visitor
trails which allows the user to choose a thematic trail designed to give the user an overview of a selection
of works that typify a period, an artistic movement, or a theme. You can plan your visit, buy tickets in
advance, view visitor tips, how to use the Louvre, view floor plans, check out the cafes and restaurants
and view calendar of events.
Navigation
(measurement: time and steps required to navigate site using available tools such as bars, menus, color,
and typography; as well as location awareness)

Main page has a display of rotating pictures that highlight the museum's events, exhibitions,
multimedia, corporate giving and visitor trails. It is a nice way to visually see what is going on at
the museum and is frequently updated to reflect the museum's current offerings.
 Navigation bar easily located at the top of the web page with options such as plan your visit,
activities and tours, collection & Louvre Palace, and learning about art. When a link is selected a
drop down menu is displayed complete with pictures and other options to explore the selected
option.
 On far right of page a quick select toolbar is available in which the user can easily search, view
calendar, watch videos, purchase tickets and shop.
 Towards the bottom of the page you can find latest news, The Global Louvre and links to
Missions & Organizations, Louvre lens, Louvre Abu Dhabi, Databases, Publishing and
Audiovisual Productions, Press and Online Media.
Searching and browsing functions
 Search bar is included in the quick select toolbar and is a bit tricky to use. When you click on the
magnifying glass to search a small box is displayed which you can search in but it is tricky to
keep it open or if you click on the magnifying glass it will bring you to a search page which is
easier to use. I searched for Leonardo Da Vinci and the results returned were a list of the artists
paintings which you could click on and learn more about.
 Search results page displays results in categories using tabs (may be confusing to user) such as
Activities & Tours, Exhibitions & Events, Collection & Lourvre Palace etc. You have to click on
the tabs to see results in that category, may be overlooked by user.
 User must use back button on browser to return to results page.
 A secondary search function is included under Collection & Louvre Palace where the user has the
option to conduct a simple search by keyword or accession number or search the databases. I did
a simple search using the keyword "renaissance" and 179 works were returned which I was able
to browse through using an arrow or clicking on the displayed results below. When I clicked on
databases I was given a selection of 6 databases to search. I chose to search the first database
which was called Atlas database of exhibits and I searched "renaissance" again and the search
returned 36 works which were displayed with a picture of the work, the artist and the title of the
work.
Design
(Measurement of aesthetic appearance: appropriateness of system interface design, including layout,
color, fonts, and graphic properties)
 Layout: Layout is complex yet approachable and eye-catching. Use of visual imagery captivates
the user and shows off the museum's collection. Links are laid out nicely and easy to navigate.



Color: Black, white and grey are the colors used for font and links giving the site a very clean and
natural feel and not taking away from the colors displayed in the photographs of the artwork.
Once you get into other pages of the site purples and oranges are used but consistent theme of
black, white and grey is used throughout the site. The use of color in the photographs of the
museum's collection is excellent and very eye-catching.
Fonts: Fonts used are a mix of Georgia and Arial throughout the site providing easy to read
captions and headers. Navigation toolbar font size is small and completely uppercase, may be
difficult to read to some users. Important highlights such as exhibitions and events are displayed
in a larger font to capture the user's attention.
Graphic Properties: The main background of the page is a look through the museum's iconic
pyramid structure and is displayed in black and white as to not distract from the main content of
the page. As you go deeper into the site the background changes to a solitary color but the use of
imagery throughout the site is consistent and well done.
Guidance
(Measurement from Neilsen’s Usability 10 Heuristics: “Help and documentation: Even though it is better
if the system can be used without documentation, it may be necessary to provide help and documentation.
Any such information should be easy to search, focused on the user's task, list concrete steps to be carried
out, and not be too large.”)
 No option for "help" found on page however there is a "Contact Us" link where telephone
numbers, adresses and emails are provided. For questions about the website an address and email
are provided.

Search box includes most viewed pages and site search help is provided. A Site Search help box
pops up with tips and suggestions.
Error Management
(Measurement from Neilsen’s Usability 10 Heuristics: “Even better than good error messages is a
careful design which prevents a problem from occurring in the first place. Either eliminate error-prone
conditions or check for them and present users with a confirmation option before they commit to the
action.” Also: “Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no codes), precisely indicate the problem, and
constructively suggest a solution.”)
 I didn’t encounter any errors
Learnability
(Measurement: how quickly new users learn the system and functions)
 The site is easy to use and pictures in navigation toolbar make clicking on links easy and
attractive. Revolving images on main page highlight what is currently going on at the museum
and are easy for users to click on and learn more about.
 Search bar is located within the quick select toolbar may not be obviously identified by new users
who are accustomed to a search tool bar located at the top corner of the webpage.
 Easy to navigate from different sections of the webpage and explore what it has to offer.
 Novice: User has option of creating their own personal account which allows
them to bookmark favorite pages and leave comments on various sections of
the website. It is easy to setup a MyLouvre personal space account by clicking
create your account and selecting a profile picture (upload your own or chose
an avatar), enter your email, last name, first name and chose a username and
password. A confirmation email is provided and the account is set up. May have
difficulty learning how to bookmark and comment throughout site as there are no
instructions provided. Site is easy to navigate and use of easy access toolbar will be helpful for a
novice user.


Intermediate: Should have no problem setting up user account and navigating
through the site. Should be able to find bookmark button in purple bar at top of page
marked with a star.
Advanced: Should be able to set up user account, use bookmarks, and add comments easily.
Navigation should not be a problem to an advanced user.
User Control and Freedom
(Measurement from Neilsen’s Usability 10 Heuristics: “Users often choose system functions by mistake
and will need a clearly marked "emergency exit" to leave the unwanted state without having to go
through an extended dialogue. Support undo and redo.”)
 If user enters a section of a site and wants to go back there are tabs at the top which
chronilogically display where the user had previously been. Easy to navigate back to homepage
by clicking on pyramid icon in the tab section at top of page or by clicking on the "Louvre" icon
which appears at the top left of all subsequent pages.
 Quick select toolbar appears on all pages on the right of the page and is easily accessible for users
to use the search function or any of the other functions available.

Consistency
 Website design, layout, colors and fonts are consistent throughout making it easy to
the site and become familiar with the website.
navigate
Language
(Measurement of terminology: use of terms and phrases as appropriate for target audience)
 Phrases and terms used in navigation are helpful in navigating the site.
 Language is simple and concise.
 Select language option is provided on top right of page where you can change the language
according to preference (French, English, Japanese, Chinese etc.).
Mobile Interface
(using Safari on an iPhone)
 The Louvre does not provide access to a mobile website but the site loads nicely on the browser
and is easy to work with despite its small size.
 how/if the interface is easy to use in the small format- the interface is easy to use in the smaller
format and the use of zoom is necessary to navigate successfully. All features work the same as
they would on a computer.
 search and browse functions- the search icon is located on the right side of the webpage in the
quick select toolbar. When zooming in on the web page the quick select toolbar disappears, this is
problematic because it is difficult to click on the small icon when the page is not zoomed in.
Search function does work when fully zoomed out but may be difficult for some users to
navigate.
 other significant observations- Overall the site is functional on a mobile browser however it is a
bit more difficult to navigate and is best utilized on a computer.
II. Content
Completeness
 The Louvre’s digital collection is made up of Western art from the medieval period to 1848,
formative works from the civilizations of the ancient world, and works of Islamic art. It is a large
and complete collection offering a closer look at what collections the museum has to offer.
Scope

The digital collection is grouped into eight departments: Paintings, Egyptian Antiquities, Greek,
Etruscan, and Roman Antiquities, Near Eastern Antiquities, Sculptures, Decorative Arts, Islamic
Art, and Prints and Drawings.
Size




Atlas: Database of exhibits offers the direct online consultation of 35,000 works of art exhibited
in the Louvre.
Inventory of the Department of Prints and Drawings is a catalogue of the museum’s 140,000
works on paper by over 4,500 artists.
La Fayette: Databse of American Art is a bilingual catalogue which presents more than 1,700
works produced by United States artists that entered the national collections of France before
1940.
Jaconde: Database of the collections of the state of the museums of France incorporates 120,000
descriptions of drawings, prints, and paintings from the 7th century to the present day, from the
collections of over sixty French museums.
Quality of digital images
 The digital images are of high quality and full screen mode is offered however the site lacks the
option to zoom in on the work.
Accuracy
 Detailed information about paintings and other works appear to be very accurate.
Diversity
 The Louvre’s digital collection is very diverse including works from many time periods and
countries.
Informativeness
 Digital collections are very detailed and the museums virtual tours and visitor trails provide
detailed information about the museum and its collection. There are several categories and
subjects to choose from the meet the user’s individual needs. Selected works of art are
accompanied by a full description of the work and it’s influence on the time period.
Authority
 The Louvre’s digital collection is very credible and the user has access to all the information they
would find in the museum itself, if not more. The databases provide a vast amount of information
and resources about the museum’s collection.
Currency
 The Louvre’s website is updated on a regular basis to reflect the current happenings of the
museum including current exhibitions and activities. The digital collections are maintained to
highlight the museum’s collection and are updated regularly as needed.
Monet, Galleries Nationales
(http://www.monet2010.com/en)
I. Interface
Navigation
(measurement: time and steps required to navigate site using available tools such as bars, menus, color,
and typography; as well as location awareness)
 When you first enter the site there is one of Monet's beautiful paintings in the background and 4
interactive bubbles where you can go to the gallery and discover Monet's paintings, take a digital
journey through Monet's work, discover practical informations or explore web TV and twitter.
 Delicate music is played throughout the site as the user navigates through the pages.
 Clicking on the digital journey bubble brings the user to an interactive digital experience of
Monet's life and paintings. The interactive journey incorporates the user's webcam and
microphone to engage the user through the virtual tour. Some of the things that it has you do
include: clap your hands to make the magpie fly, blow into microphone to make the windmill
turn, clicking on the picture to change it from winter to spring, blow into microphone to make
poppies dance, make the water ripple by moving cursor over pond, change camera angle by
moving from bottom to top, remove fog by waving hands in front of webcam, and ring church
bells using keyboard. The Journey is educational and fun and great for users of all ages. I can see
this being a great website for kids being that it is so interactive.
 Clicking on the Gallery bubble brings you to the a timeline of Monet's work from 1864-1926
which you can conveniently scroll through using the bar at the bottom of the page. The timeline is
broken down into sections highlighting art movements such as Modernism and Impressionism as
a group, Rejecting modernism and solitary Impressionism, The series, and Water Lilies and the
Grand Decorations. You can scroll through the timeline and click on the paintings along the way
to learn more about Monet's work, which you can then click through in a slideshow manner.
There is a tab labeled Exhibition Tour on the left hand side of the page in which Monet's works
are categorized into different sections the user can explore by making a selection. Below that is a
button that is labeled All the Paintings which shows a listing of all Monet's paintings with dates.

Clicking on the Practical Informations bubble brings you to a informative couple of paragraphs
about the "Claude Monet 1840-1926" exhibit that took place at the Galeries Nationales of the
Grand Palais between 2010-2011.

If you click on the Web TV & Twitter bubble you are brought to a screen with three new bubbles
labeled News, Web TV, and Twitter Report. The News tab is not active and reads "The
exhibition-event is over. Thanks to everybody for coming!". The Web TV bubble brings the user
to a selection of videos about Monet's life however they are all narrated in French and there are
no subtitles. The twitter bubble brings you to the twitter account of the exhibition which is also in
French.
Searching and browsing functions
 There does not appear to be a search bar on the main homepage.
 Below All paintings in the Gallery section of the site the user will find a search box in which to
search Monet's work. When I went to conduct a search for "water lilies" nothing was returned,
next I searched "modernism" and still nothing was returned. There may be an error with the
search feature.
Design
(Measurement of aesthetic appearance: appropriateness of system interface design, including layout,
color, fonts, and graphic properties)




Layout: Layout is very detailed and interactive, option of clicking on bubbles or tabs on top of
page to navigate through the site. Back buttons located throughout all the pages and access to tabs
throughout the pages.
Color: Main colors used are black and white, with Monet's paintings scattered throughout the site.
Painting line the background of pages adding beautiful colors and warmth to the site.
Fonts: Fonts used are easy to read and give the site a clean look. Use of all capitals in many of the
headings and links, making it easy to read. Font used in descriptions and Practical Informations
section are very small and may be difficult for some users to read.
Graphic Properties: The graphic properties of this site are amazing and very eye catching. The
site is very interactive as well as the presentation. Loading time is affected by use of digital
effects and may be slow on some systems. Background pictures change throughout the pages to
highlight Monet's paintings.
Guidance
(Measurement from Neilsen’s Usability 10 Heuristics: “Help and documentation: Even though it is better
if the system can be used without documentation, it may be necessary to provide help and documentation.
Any such information should be easy to search, focused on the user's task, list concrete steps to be carried
out, and not be too large.”)
 No option for help found on main page.
 On Gallery page there is a small question mark in the top right corner that provides tips on
navigating the gallery and the overall view/zooming in and how to use the Exhibition Tour tab.
 The Journey section does not provide a help function but it walks you through the Journey very
thoroughly explaining what to do in each step.
Error Management
(Measurement from Neilsen’s Usability 10 Heuristics: “Even better than good error messages is a
careful design which prevents a problem from occurring in the first place. Either eliminate error-prone
conditions or check for them and present users with a confirmation option before they commit to the
action.” Also: “Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no codes), precisely indicate the problem, and
constructively suggest a solution.”)
 I encountered a problem with the search feature located on the Gallery page. I was able to type in
my search terms however no results were found and no actions took place. There is no option for
help.
Learnability
(Measurement: how quickly new users learn the system and functions)
 The site is of easy/moderate difficulty and may be difficult for some users to navigate. The
bubbles on the main page provide a quick and easy way to navigate the site and the back buttons
provided on each page make it simple to return to where the user left off.
 There is no scroll bar located on main page.
 Search function does not work properly.
 Novice user: On first entering the site the overall moving graphics and display may be
intimidating to user. Clicking through the sites bubbles will get user acquainted with site. The
Journey section will take the novice user through the digital experience step by step. The help
button located in the gallery section will provide the user tips on how to navigate through the
gallery page.
 Intermediate user: Intermediate user will navigate through the site easily, by using the menu bar
or clicking on the bubbles. The more experience with the site will make coming back to it later
more accessible and familiar to navigate.

Advanced user: The advanced user should not have trouble navigating through the site. The
journey section should provide a great interactive experience engaging the user in many ways.
User Control and Freedom
(Measurement from Neilsen’s Usability 10 Heuristics: “Users often choose system functions by mistake
and will need a clearly marked "emergency exit" to leave the unwanted state without having to go
through an extended dialogue. Support undo and redo.”)
 User control is satisfactory but missing a few key features. Menu tab appears on top of every page
and back buttons are included on all pages.
 Ability to search each page is not available.
 Help is not available on each page.
 The site is slower than most due to interactive graphics.
Consistency
 Website design, layout, colors and fonts are consistent throughout and transition nicely
throughout the site.
Language
(Measurement of terminology: use of terms and phrases as appropriate for target audience)
 Phrases and terms used in navigation are helpful in navigating the site.
 Language is simple and concise.
 Web TV section includes French only videos.
Upon first entering the site the user is prompted to select their language of choice including: French,
English, Spanish, Japanese or Chinese.
Mobile Interface
(using Safari on an iPhone)
 If the DL offers a mobile version - No, the website could not be opened on my phone's browser
because it needs Flash Player 10 in order to operate.
II. Content
Completeness
 The digital collection gallery includes 153 of Monet’s paintings throughout his lifetime. The Web
TV section of the site offers 19 videos on Monet’s life and paintings as well as the official
opening of the exhibition.
Scope
 The gallery includes a timeline of Monet’s life and includes the following sections: Modernism
and Impressionism as a group, Rejecting modernism and solitary Impressionism, The series, and
Water Lilies and the Grand Decorations.
Size
 The collection of paintings includes 153 of Monet’s paintings and 19 videos about his life.
Quality of digital images
 The quality of the digital images is superb and there is a zoom feature available which allows the
user to zoom in on any part of the painting. By zooming in the user can see the individual brush
strokes, cracks and the finest details of the painting.
Accuracy
 Information on paintings is provided by the Galeries Nationales of the Grand Palais, Rmn-Grand
Palais and the Musee d’Orsay in Paris. Paintings are accompanied by detailed information and the
events taking place in Monet’s life at the time.
Diversity
 The collection is diverse in the way it is depicted on the timeline of Monet’s life. The journey
section offers a unique interactive journey through Monet’s work.
Informativeness
 The digital collection is very informative and would be useful to anyone seeking information on
the life of Claude Monet. Detailed descriptions of Monet’s works are provided and the videos
included in the Web TV section include information about the artist’s life.
Authority
 The website was developed by Rmn-Grand Palais and the Musee d’Orsay in Paris and the
exhibition took place from September 2010-January 2011.
Currency
 Unfortunately the website is not kept updated and is a reflection of the exhibition that took place
from September 2010-January 2011.
Public Library: New York Public Library
NYPL Digital Collections (Beta)
[http://digitalcollections.nypl.org]
I. Interface
Navigation
(measurement: time and steps required to navigate site using available tools such as bars, menus, color,
and typography; as well as location awareness)
 Main collections search box is placed prominently on main/home page, making it easy to find;
text populated in search field reads “Search keywords,” giving user help in using the feature
 Navigation bar for main library website appears at top of page; this is a useful feature for
navigating the main website, but is it confusing for user to differentiate the digital collections site
from the larger website?
 on search results page, the main navigation appears horizontally at top of page, and the digital
collections site features a vertical menu on the left-hand column, including a number of results
narrowing criteria (such as topics, location, etc.) – is this confusing for user?
 Clicking on one of the indexed options on the left narrows search results; Filtering information
then appears at top of page, allowing user to navigate back to cancel the filtering criteria to return
to original search
 When clicking on individual item, it opens a new page containing the image and metadata; but
there doesn’t appear to be an option to navigate back to the search results page; user must use the
back button on their web browser
 Search box appears at top right of page, making it easy to start a new search
 “NYPL Digital Collections” title appears on all pages, making it easy to navigate back to the
digital collections site home page
 Footer section of pages allow user to navigate to other parts of the NYPL website as well as
Access the site in different languages
 Steps required to search and retrieve an item: 2
 Steps required to search and retrieve an item, starting from home page of NYPL website: 5 (4
when Beta site launches)
Searching and browsing functions
Using main search function:
 Main collections search box is placed prominently on main/home page, making it easy to find;
text populated in search field reads “Search keywords,” giving user help in using the feature
 See above – searching site is directly related to the navigation of site
 Main website search icon is available in main website navigation bar at top of page; user must
differentiate between the two, although the digital collections search box is placed prominently
enough that it’s visually implied for user to use that search box; Also, when clicking on main
search box, a menu appears making it easy to see that the search box is specifically for searching
either the “Catalog” or “NYPL.org” (Terminology note: user must know the difference between
“catalog” and “digital collection”)
Using browse features:
 Includes browse collections feature, accessible from home page; Approachable terms like
“browse” and “explore” are featured on main page, especially welcoming for new users
 “See More” options at bottom right section of individual item page place emphasis on browsing
of the collection, prompting user to explore related collections and images
Design
(Measurement of aesthetic appearance: appropriateness of system interface design, including layout,
color, fonts, and graphic properties)
 Layout: Overall layout is clean and simple; main digital collections page places visual emphasis
on search box and recently digitized collections feature; Results page layout optimizes the ability
to browse through available image thumbnails with easy access to filtering criteria options on the
left-hand side of page as well as a sort feature near top of page; Individual item page places large
image at top of page, making it easy to immediately view the image; user must scroll down to
access metadata and indexed options; additional icons are located at top of page, allowing user to
perform different functions, including printing, sharing, or manipulating the position of the
image; “See More” options at bottom right section of page again place emphasis on browsing of
the collection, prompting user to explore related collections and images
 Color: Muted and simple black, white, and gray color palette helps to highlight collection images,
not detract from them. The color of the main digital collections page varies, as the page features a
new collection image each time a user visits the page; the image populates the entire background
of the page, creating a dramatic and graphic introduction to the site and collection. One note
about readability of White text on black background—this contrast may be difficult for some
users to read
 Fonts: Use of sans-serif fonts gives the site a modern feel and is easy to read; size of fonts are
presented in a hierarchical manner, placing more emphasis on titles, navigation labels, and main
title of site; Font sizes of sorting criteria text are smaller
 Graphic properties: The graphic property of the main digital collections page varies, as the page
features a new collection image each time a user visits the page; the image populates the entire
background of the page, creating a dramatic and graphic introduction to the site and collection
Guidance
(Measurement from Neilsen’s Usability 10 Heuristics: “Help and documentation: Even though it is better
if the system can be used without documentation, it may be necessary to provide help and documentation.
Any such information should be easy to search, focused on the user's task, list concrete steps to be carried
out, and not be too large.”)
 Help option in main navigation bar retrieves contact information for the library as well as an Ask
NYPL e-chat link
 Search box on main page offers user a hint at what (and where) to enter into search box in order
to user the site; However, no information on page directs user to specific help pertaining to the
digital collections site
 Icons (and rollover text) appear on image thumbnails in results page indicating that the item has
multiple pages
Error Management
(Measurement from Neilsen’s Usability 10 Heuristics: “Even better than good error messages is a
careful design which prevents a problem from occurring in the first place. Either eliminate error-prone
conditions or check for them and present users with a confirmation option before they commit to the
action.” Also: “Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no codes), precisely indicate the problem, and
constructively suggest a solution.”)
 I didn’t encounter any errors
Learnability
(Measurement: how quickly new users learn the system and functions)
 Novice user: It was easy to use the system on the first try; browse features prompt new users to
explore site without entering any search criteria (keywords); Search box prompts user to enter


keyword, making the search feature easy to understand and approachable (as opposed to an
advanced search feature which might be intimidating for new users); using filtering options and
results narrowing features might be more advanced features for novice users
Intermediate user: make use of filtering options and results narrowing features; features on item
record pages, such as share, zoom, buy, print links
Advanced user: use sorting links to narrow and filter search results; make use of filtering options
and results narrowing features; features on item record pages, such as share, zoom, buy, print
links
User Control and Freedom
(Measurement from Neilsen’s Usability 10 Heuristics: “Users often choose system functions by mistake
and will need a clearly marked "emergency exit" to leave the unwanted state without having to go
through an extended dialogue. Support undo and redo.”)
 User control is overall good; title of site appears on every page, allowing the user to easily
navigate back to the home page if they aren’t satisfied with a search
 Similarly, the search box appears at the top of the search results page, allowing the user to start a
new search if they aren’t satisfied with the current search
 Also, the system allows for user to easily cancel or undo any filters or narrowing criteria they
apply to their search results
Consistency
 Website design–layout, colors, fonts—is consistent throughout, tying in the different pages users
encounter, including the main page, results page, and individual item page
Language
(Measurement of terminology: use of terms and phrases as appropriate for target audience)
 Main search box appears in main website navigation, visible on all digital collections site pages:
user must know the difference between “catalog” and “digital collection”
 “Enter keyword” text populated in search box is user friendly search language
 Approachable terms like “browse” and “explore” are featured on main page, especially
welcoming for new users
 Some of the terminology used in results narrowing menu, such as “collections” and “topics” and
“resource types” might require an explanation for some new users; a rollover text might help
assist in understanding such features
 “Share” feature on individual item page contains technical information that all users may not
understand; for example, “Copy the URL to your clipboard using CTRL+C (Win) or ⌘+C (Mac)”
may be too technical for some users
 Social networking icons featured on individual item page are supported by rollover text, assisting
novice users in determining the functions of the links
Mobile Interface
(using Chrome on a Google LG smartphone)
 if the DL offers a mobile version - NYPL offers a truly mobile-optimized website
 how/if the interface is easy to use in the small format - interface design is tailored to fit on the
screen without any necessary scrolling; navigation menu is activated by clicking a three-lined bar
at the top right corner of page, producing a drop-down menu of the main sections of the NYPL
website; it took two attempts to locate the digital collections feature, which is accessible through
the research section of the main navigation; Using the “Digital Gallery” link, a non-mobile page
appears displaying the digital gallery search and browse features; Clicking on the NYPL Digital
Collections Beta link leads to a mobile-friendly version of the digital collections home page; as
with the desktop website, the main navigation is accessible from the digital collections pages,


making it potentially confusing to re-find the digital collections section of the website; NYPL
Digital Collections title appears on all pages, making it easy to navigate back to the main digital
collections page.
search and browse functions - search link is located prominently at the top of the mobile home
page, a search box appears when clicking on the search icon, allowing for a search of the catalog
or the website; Search box is located on the digital collections home page, as well as links to use
browse features; search results page also displays in mobile-friendly fashion, with individual
image thumbnails in a single column, causing user to scroll up and down to see entire list;
Individual item page displays image so that entire picture can be viewed without scrolling, and
image details appear below it in a list format, but easy to read; search box remains on all pages,
making it easy to start a new search
other significant observations we notice
II. Content
Completeness
 The site appears to provide access to all of the library’s available digital images; Regarding the
percentage of their entire holdings represented in digital format, the website indicates, “At over
half a million images, the collections in NYPL Digital Gallery represent a small fraction of the
New York Public Library's overall visual holdings.”
Scope

Using the “browse all” feature on the home page, it helps to determine what content is included.
The results narrowing options on the left hand panel also allow users to browse through all
available categories, such as names, places, collections, and topics. The scope of collections,
media, subject matter, locations is vast.
Size

According to the website, the NYPL digital collections site contains 804,243 items (as of April
27, 2014)
Quality of digital images
 The high quality of digital reproductions and size of image display enhance the viewing of
collections online. Zoom feature allows for a closer inspection of image details, but it doesn’t
magnify images to a degree that fine details can be inspected.
Accuracy
 The information contained in item records appears to be accurate and complete.
Diversity
 diversity of subject matter is apparent using the browse all feature
Informativeness
 Not all item records contain descriptive caption information
Authority
 The information contained in item records appears to be credible as it was created by the library
staff.
Currency
 According to the website, “NYPL Digital Gallery will continue evolving as more visual
collections are added.”
Mobile Content
(using Chrome on a Google LG smartphone)
 what content is available, if the content is different from the mother site, how it's different and if
it's still effective in a shortened version