Christopher Yaple
553
Term Project
The Digital Library User: A Review and Evaluation of Public Library's Digital Collections from
the User's Perspective
1. Introduction
Digital libraries (DL) are often evaluated by their designers, librarians, or other
information professionals with a focus set on aspects such as the library's system performance
and infrastructure, the quality and completeness of its collection, or the economic factors
surrounding it, such as the funding and other resources needed to maintain their digital
collection. While these evaluations are of importance and a necessity for the ever growing field
of DLs, there is another group with their own interests and criteria to consider – the users. DLs
are not simply storage units for information or archives that benefit only a select number of
information professionals. So why is it that the users, the individuals driving the creation of these
DLs, are often disregarded in the research surrounding digital libraries (Bollen & Luce, 2002;
Fuhr et al., 2007; Monopoli, Nicholas, Georgiou & Korfiati, 2002; Xie, 2008; Yan, Zha, Zhang
& Hou, 2013)? Another trend in DL research, as well as in the research of their physical
counterparts, is to give focus and attention to academic and other large organizational
collections, while public libraries, institutions that serve and benefit users of all ages, economical
backgrounds, cultures, situations, etc., are ignored.
Because of these trends this paper plans to evaluate a number of public libraries' digital
collections from a user's perspective, focusing on the usability and user satisfaction associated
with each. It is this author's hope, by exploring these criteria and the users' perspective in
evaluation and then applying it to actual collections, that future designers, librarians, and other
information professional will better understand what users want, expect, and need from their
DLs. To do this we will first exploring the criteria, looking at other library and information
science literature in hopes to construct suitable definitions and guidelines that can then be used in
our evaluation of The Mercer County Public Library, The Hamilton Free Public Library, The
Trenton Free Public Library, and The Princeton Public Library.
2. The Criteria: Literature Review & Definitions
2.1 Usability
There is no doubt that the term usability is broad and can conjure an assortment of
definitions or ideas. Numerous LIS publications (Chen, Germain, & Rorissa, 2011; FriasMartinez, Chen, & Liu, 2008; Fry & Rich, 2011; Fuhr et al., 2007; Xie, 2008; Yan et al., 2013)
have discussed usability as criteria and a means of evaluating DLs, all of which understand that
usability is “essential for providing high quality services to a broad and diverse population of
users” (Bertot, Snead, Jaeger & McClure, 2006, p. 18). Chen et al. (2011) states, “a clearly
articulated definition of usability is crucial for stakeholders of a system to gain a consistent
understanding of its construct” and that “this, in turn, could serve as a building block for
establishing good usability policies, standards, and guidelines” (p. 599).
A DL's usability is judged using a number of aspects, one of which is often the first
encountered by users – the interface. Nadjla Hariri and Yaghoub Norouzi's (2011) Determining
Evaluation Criteria for Digital Libraries' User Interface states that “a DL is only as good as the
interface it provides to its users” (p. 699). Fuhr et al. (2007) also express the importance of the
interface as a gateway, describing the interface “as a bridge between the user and the systemenvironment so that the user can interact with the DL system” (p. 27). While the interface is the
entryway to and means of use for the DL, it also affects the user before they “step into” the
digital realm. Hariri and Norouzi (2011) state that the “interface characteristics...have a strong
impact on perceived ease of use and its subsequent effect on the intention to use an online
library” (p. 703). If the user interface is such an important component of the DL and a critical
aspect of usability as well, what makes a good interface?
In their list of the eleven attributes of usability, authors Chen, Germain, and Rorissa
(2011) describe the interface and its design as “the technical and visual design concerns of the
system or website interface, including its design elements...consistency, navigation, information
architecture, and task flow” (p. 606). Successful interfaces offer attractive design elements, while
at the same time providing simple and easy to understand menus, tools, and information. In the
case study Tweaks We've Made to the Digital Branch the author explains how their branch's
digital library received complaints about having “too much content on the main page” and how
“all the content seemed to run together” (2012, p. 8). A successful interface avoids clutter and
confusion. Fuhr et al. (2007) also stresses the importance of integration when considering the
different components and functionalities of the DL interface. Finally, Xie (2008) reminds us that
a successful interface “delivers results in a readable format” (p. 1358). Font, color, and layout
choice can either facilitate or hinder usability.
The view and output options are important for a DL's usability. When discussing the view
and output of a DL, phrases such as “visually clear” (Fuhr et al., 2007, p. 27) were used to
describe users' preferences, along with the desire “for high resolution items” (Xie, 2008, p.1364)
Other common options include zoom features or multiple layouts for when looking at multiple
documents at once. One major problem found in the view and output of DLs was small fonts
used when displaying text (Xie, 2008, p. 1364).
A third aspect brought up in the discussion of usability in DLs is that of control and
flexibility, a characteristic that fits into the idea of a DL's design elements as well. A DL needs to
provide its users with flexibility and control over the provided tools and options during their use
(Bertot et al., 2006; Chen, et al., 2011; Fuhr et al., 2007; Hariri & Norouzi, 2011). Control and
flexibility can be seen in a DL's ability to allow users to "manipulate, adapt, customize,
personalize, and access, using various devices and means," as well as being "compatible with
varying applications" (Chen et al., 2011, p.606). Flexibility is provided through customization
tools such the ability to group result, limiting the results through controls such as library or type
of document, search in either a basic of advance setting, as well as by subject, author, title,
keyword, etc., and refining a search after the initial results have been received (Bertot et al.,
2006, p. 21).
Flexibility can also be seen as the way in which a DL adapts to a user's preference and
skill (Fuhr et al., 2007). Authors Frias-Martinez, Che, and Liu (2008) explain that because of the
wide range of possible users DLs can reach, there is "a greater variability, in terms of their
background, knowledge and skills" among them (p. 48) and usually a preference for either field
dependence or fiend independence when performing cognitive tasks (p. 49).
The previously mentioned aspects of usability have focused on the physical components,
the design and layout of the DL, but there is another side of usability to consider. A DL's
learnability and memorability are a crucial component of its usability (Bertot et al., 2006; Chen
et al., 2011; Fuhr et al., 2007; Hariri & Norouzi, 2011; Yan et al., 2014). When discussing
learnability and memorability Chen et al. (2011) state that “the system should be efficient to use
so that once the user has learned the system, a high level of productivity is possible” and “the
system should be easy to remember so that the casual user is able to return to the
system...without having to learn everything all over again” (p. 606). These definitions are echoed
almost verbatim by Hariri & Norouzi (2011) and expanded by Bertot et al. (2006) who state that
a system should allow “users to intuitively use a digital library's various features” (p. 19). In their
study of digital libraries and virtual communities Yan et al. (2013) discuss system quality, an
aspect of DLs, and its usability, which relies on easy to learn software, features, and functions.
Although a DL's learnability and memorability are important to the overall usability of it,
they also rely on the inclusion of features that can help users throughout their time and use (Fry
& Rich, 2011; Hariri & Norouzi, 2011; Monopoli et al., 2002; Xie, 2008; Xie & Cool, 2006).
Simply put, help features “provide instruction on how to use a service” (Monopoli et al., 2002,
p.111). According to Xie & Cool (2006) users look to help features when they find themselves in
“help-seeking situations,” situations when they are unable to move forward in their search
process or use of information retrieval system. Help features can appear in a number of forms,
such as frequently asked questions (FAQs) or chat options, and can also be divided into general
or specific help (Xie, 2007). A DL's help features are especially important to its overall usability
because it has a direct relation to a user's view of the entire system. Xie & Cool's (2006) study
shows “that users' evaluation of help mechanisms of an IR system is related to the evaluation of
the IR system” as a whole. Much like how a DL's interface can effect a users presumed interest
or use, the DL's available (or lack of) help features will drive a user's preference in a positive or
negative manner.
Interestingly, a majority of publications that express the importance of help features in
DL also share that many users do not use these functions (Fry & Rich, 2011; Monopoli et al.,
2002; Xie, 2007; Xie & Cool, 2006). In their study, Monopoli et al. (2002) published that less
than a quarter of their respondents used the provide help features, while the users in Fry & Rich's
(2011) study “expect[ed] online systems to be easy to use and self-explanatory,” but
simultaneously did not use the help tutorials provided (p. 393). This disinterest and disuse of
help features may be due to the lack of knowledge about where to start looking for help, the need
for directions, help features that are not specific or personal enough, and help that is difficult to
understand (Xie & Cool, 2006) or the help features may simply not be helpful to users or provide
contradictory information that cannot be implemented in the actual DL (Xie, 2007). Despite the
contradictions present in the discussion of help features, the fact that it is so prevalent leads the
author to believe that it should be included when discussing usability in DL.
After collecting and examining these previously explored publications this paper has
defined usability as follows: Usability focuses on the DL's ability to allow users to achieve their
goals. This ability includes a number of components that can be separated into two groups. The
first group focuses on the design element of the DL interface, which also includes the control and
flexibility of the system for its users, as well as the inclusion of view and output options. The
second group of usability criteria includes the system's learnability and memorability, as well as
the inclusion of help features. This definition of usability will be used from here on out in this
paper.
2.2 User Satisfaction
In practically all systems a satisfied user is one of the preferred outcomes. This is no
different for DLs. One challenge DLs face when working to leave their users satisfied is the fact
that they are working with a full spectrum of users, each with different skills, levels of
knowledge, and other abilities or disabilities. Frias-Martinez et al. (2008) explains that “users of
DLs may have a greater variability, in terms of their background, knowledge, and skills” (p. 48).
Because of this “DLs must reach out to users from all walks of life, serving information needs at
all levels” (Hariri & Norouzi, 2011, p. 708). Another variability to consider among DL users is
the possibility of disabilities at all levels. DLs must provide their users with “appropriate
functionalities” (Chen et al., 2011, p. 602) no matter the disability, be it “visual, auditory,
mobility, cognitive, learning [or] others” (Bertot et al., 2006, p. 24).
Chen et al. (2011) sum this up in their discussion on user satisfaction in which they state
that “the system addresses users' cognition, information, processing, mental model, level of
knowledge/skill, and demographic characteristics” (p. 606). All of this is important because, as
Fuhr et al. (2007) put it, “the final aim of a DL system should be that of enabling people to
access human knowledge any time and anywhere, in a friendly multi-modal way” (p. 26).
Another way DLs allow for user satisfaction is through the provision of communication
and feed back channels for their users (Tweaks We've Made, 2012; Xie, 2008; Xie & Cool, 2006).
Xie (2008) states “interaction between users and digital libraries is...an important component for
evaluation” (p. 1350) and being that user feed back can drive user satisfaction, “a communication
channel is needed between developers of digital libraries and their targeted audience” (p. 1360).
In the case study Tweaks We've Made to the Digital Branch the author shares improvements that
were made after receiving feedback from users and the subsequent increase in user satisfaction,
stating that DL should “listen to complains when they come in” (2012, p. 9). But for comments
or complaints to come in, avenues must be laid out in the DL. Communication and feed back
channels can consist of standardized forms for users to fill out or listed email addresses, but it is
note by Xie & Cool (2006) that “chatting with a librarian is a preferred...feature for them.”
Unlike the communication discussed when looking at help features in a DL, the communication
and feed back here revolves around users being able to share their feelings about a DL, be they
positive or negative.
While it may seem like a given, the final component of user satisfaction in DL is a
satisfied user. DLs should strive to have their users leave satisfied and with their original goals
met (Fuhr et al., 2007; Hariri & Norouzi, 2011; Xie, 2008). Hariri & Norouzi (2011) stress the
importance of users' satisfaction, explaining that “DLs as a whole will thrive or wither only as
they serve or fail to serve their user communities” (p. 708). This is echoed by Xie (2008) who
simply states “user satisfaction is essential for the success of a digital library” (p. 1367).
Satisfaction is a hard aspect to pin down, especially for the user who cannot see the DL's system
performance or know what other aspects are going on behind the scenes, but is often associated
with “a pleasant and rewarding experience” (Hariri & Norouzi, 2011, p. 705) that is also
“efficient and enjoyable” (Fuhr et al., 2007, p. 26.)
Through the process of examining these included publications this paper has defined user
satisfaction as follows: User satisfaction evaluates the subject's ability to provide for users of
different levels of knowledge, skill, and ability, as well as providing for communication and
feedback with its users, and overall leaving its users satisfied with their experience. This
definition of usability will be used from here on out in this paper.
3. The User: Literature Review
When evaluating any DL it is important to consider its usability and the user satisfaction
it provide, but there is still another aspect that must be explored – the user. DL users view and
experience DLs differently than those who design and build them, as well as information
professionals such as librarians who also use them. Xie (2008) explains that “while users focus
more on the usefulness of digital libraries from their own perspective instead of from researchers'
and professionals' perspectives, they care less about cost, treatment, preservation, social impact
of digital libraries, and so on” (p. 1371). This difference in views is directly affected by what
users can and cannot see or experience. Fuhr et al. (2007) note that a DL's performance, often a
standard point of evaluation, “is an aspect of the system that the users cannot see or evaluate
directly” (p. 26) and that the same goes for other “practical issues” such as cost, effectiveness,
time, personnel, and infrastructure (p. 31).
Despite this, information professionals have not given much attention to the users and
their needs in their publications (Bollen & Luce, 2002; Chen et al, 2011; Fuhr et al., 2007; Xie,
2008). Bollen & Luce (2002) state “the study of user needs and preferences has largely eluded
quantitative analysis,” which has resulted in a lack of focus on the “actual effectiveness” of DL.
Chen et al. (2011) share their findings that library practitioners “seemed less attentive to users'
effective concerns” (p. 617) when using DL, and did not consider the users' environment, be it
“cultural, social, economic, political, or organizational” (p. 620). Xie (2008) described users as
“passive subjects” in most DL research and evaluation (p. 1351). Additionally, Fuhr et al. (2007)
explain that “relevance, as measured by information retrieval evaluation cycles, does not take
into account user satisfaction or pleasure” (p. 23).
Some authors have attempted to provide reasoning for this though. Fuhr et al. (2007)
simultaneously express the user's importance and complexity, stating that “users is the first
component of any interaction process and his characteristics are complex and constantly
evolving,” which could be a reason for the lack of published research on DL users (p. 25). Bollen
& Luce (2002) express a similar sentiment in their article, explaining that “user search focus can
shift from one scientific domain to another between, or even within, retrieval sessions” and
because of this researchers do not always have “stable characteristics” to focus on. Xie (2008)
shares that users' “evaluation of DLs is affected by their preferences, experiences and knowledge
structure,” and that “different users have different preferences” (p. 1370). It is not that researches
are not interested in the DL users though. Fuhr et al. (2007) show this awareness when they
explain that “digital libraries are destined to serve user” and “if unused these systems fall into
oblivion and terminate their operations” (p. 31). Xie (2008) agrees, stating that “only the targeted
users of a digital library can determine the usefulness of a collection” (p.1359).
Nevertheless, research has provided us with some important aspects of DL users to
consider. Hariri and Norouzi (2011) found that “regardless of users' information technology
backgrounds, their expectations of DLs functionality are the same,” and that “users' previous
experiences with DLs” affect their current DL usage (p. 710). Xie (2008) echoes this, stating that
“users' past experience of other IR systems influences their use of digital libraries” (p. 1370).
Xie's use of the term IR system and its interchangeability with the term DL touches on another
user-focused findings. Early on Xie (2008) explains that “to end users, digital libraries are
similar to the world wide web with improvements in performance, organization, functionality,
and usability” (p. 1346) and that they “consider digital libraries more as IR systems [and] less as
libraries” (p. 1359).
Another interesting aspect of users and their view of DLs is found in their criteria and
evaluation. Xie (2008) found that “there are discrepancies between the perceived importance of
DL evaluation criteria and the actual evaluations of digital libraries” and that users “have higher
standards in discussing the DL evaluation...[but] actually lower their standards in evaluating
digital libraries” (pp. 1368-1369). This can be seen in users' inclusion of the perceived preference
for DL help features and access for users with disabilities, but its absence when actual
evaluations took place (Xie, 2008). As Xie (2008) eloquently put it, “while users' proposed DL
evaluation criteria represent their desired images of digital libraries, their actual evaluation
reflect their expectation of digital libraries” (p. 1371).
4. Public Libraries' Digital Library Evaluation
4.1 Method for Evaluations
For this project the author will explore, attempt to use, and then evaluate the following
digital libraries as a user (i.e. someone who has no connection with the design process,
construction, or maintenance of the digital collection) with the criteria of usability and user
satisfaction. As previously stated, for this paper usability is defined as the DL's ability to allow
users to achieve their goals. This ability includes a number of components that can be separated
into two groups. The first group focuses on the design element of the DL interface, which also
includes the control and flexibility of the system for its users, as well as the inclusion of view
and output options. The second group of usability criteria includes the system's learnability and
memorability, as well as the inclusion of help features. User satisfaction is defined as the DL's
ability to provide for users of different levels of knowledge, skill, and ability, as well as
providing for communication and feedback with its users, and overall leaving its users satisfied
with their experience.
4.2 Mercer County Public Library
The Mercer County Public Library's “Virtual Branch” claims to provide “access to digital
movies, music, magazines, books, articles, reference sources, business directories and more twenty-four hours a day, seven days a week!” The MCL's DL provides users access to Freegal
Movies and Music, Flipster, hoopla, eLibraryNJ, and a number of databases such as EBSCOhost,
Gale Virtual Reference Library, and ERIC.
Design Elements. Overall, the interface design of the MCL DL is fine with its simple
layout and choice of font. Large images advertising each collection are placed side by side with
their title and description. Navigation can become confusing due to some repeating links, such as
a blog post linked twice within the same sentence, and links that bring users to their physical
library's collection and catalog. Similarly, all links leading users to other digital collections do
not open in a new page. This can lead to time spent backtracking to reach the original MCL page
the user began on.
Control and Flexibility. The fact that the MCL DL brings users to outside digital
collections exclusively leads to its evaluation of control and flexibility. Being that each collection
is different, they each have their own way of handling their customization and adjustment of
search features. While almost all allow you to search using specifics such as title or author,
others allow for users to create lists of their favorite items or narrow searches down by subtopics
such as genre or publication date.
Learnability and Memorability. This is similar to how users could evaluate the
learnability and memorability of this DL. Being that these collections are not MCL's per se, they
each have their own work flow and steps that need to be taken to meet a user's needs. The MCL
DL does provide instructions for each digital collection and type of tool used to access it (PC,
tablet, phone, etc.), as well as blog posts with instructions. If a user found this DL intuitive, they
would probably have no issue with remember the steps needed to take when accessing it again in
the future.
Help. The MCL DL lacks help features. The help features provided focus more on the
physical library and therefor would not help a user who ran into a problem in their digital
collection.
User Ability. While the MCL DL does not have any features for adapting to particular
user needs, it is relatively straightforward and simple to use. This could work for users of
differing technical skill and education levels, but would not benefit DL users with other
disabilities.
Communication and Feedback. The MCL DL provides users with a large icon and link
for contacting them at the bottom of their digital collection homepage. Unfortunately this is
simply a form in which users provide their name, contact information, and message. There are no
names or email addresses for the users and therefore no accountability on the library's end and no
way of knowing if the user's message reached anyone.
4.3 Hamilton Free Public Library
The Hamilton Free Public Library's digital library is unique in that it strictly separates
itself from the digital collections that are “provided as a convenience to our patrons and are for
resources that are not controlled by the Hamilton Free Public Library staff” and therefore can be
accessed without any library's credentials. The HFPL's “Online Resources” include links to the
New Jersey State Library, Ancestry.com, ERIC, the Mayo Clinic, and a number of other
resources.
Design Elements. The HFPL uses a plain and easy to read font, as well as a color scheme
that is not distracting to the user. Unfortunately navigating the digital library is not complete
intuitive. Among the links provided at the top of the page the one labeled “Search The Catalog”
provides users access to free eBooks while another link, “Online Resources,” provides links to a
number of sites, including those previously mentioned. While there should be no problem
reading these links, there is the issue that the eBooks are not clearly labeled (and mixed together
with the physical collection) and all of the DL components are not placed in one area.
Additionally, the “Online Resources” page is too long, forcing users to continuously scroll down
to see all that is provided.
Control and Flexibility. The HFPL does not offer its users any control of flexibility when
using their digital library. All of the resources are simply links to outside sources that are not
affiliated with the library. Each resource has its own design, layout, and work flow and therefore
its own options for customization and user flexibility.
Learnability and Memorability. Besides the possible confusion caused by the HFPL's
layout and link placement, there are no descriptions or instructions provided. This does not make
for an intuitive DL and its separation of digital content does not help. Each digital collection has
its own process and work flow, and therefore must be learned separately. Once one does learn
how to properly use this DL it is possible that they will remember how to use it simply because
of their initial confusion.
Help. The HFPL does provide two help features, a “How Do I...” section and their “Site
Map” in the their row of links provided at the top of the page. The issue with these help features
is that they provide help for the DL and its physical counterpart, with more emphasis on the
physical. When searching through the “How Do I...” help section I was lead to a page on how to
download eBooks. Oddly, this page was not presented on the other pages that had eBook links or
under the “Online Resources” page.
User Ability. This DL does not provide for users of different skill levels or abilities.
Instead it could be considered a “one-size fits all” type of website. Matched with the previous
comments about its unintuitive design, this website may be confusing and unforgiving for a
number of users.
Communication and Feedback. The HFPL does provide a “Contact Us” link that is easy
to locate at the center of their page's banner. The contact page includes a list of email addresses
and phone numbers for specific needs, such as reference questions, general questions for
children, and local history, as well for specific library administration.
4.4 Trenton Free Public Library
The Trenton Free Public Library's DL is separated into three sections: eBooks, language
learning tools, and research. The DL's eBooks and audiobooks come from eLibraryNJ, its
language tools from Rocket Languages, and its research section contains a number of databases
from EBSCO Host, Explora, and Jersey Clicks, to name a few.
Design Elements. The TFPL uses a static page layout from section to section that,
combined with its use of a plain text and large, clear images, makes for easy navigation. As
previously mentioned, the DL is split between three sections, each found under three different
drop down menus, which can make locating your desired resource complicating at first.
Additionally, on the database page the TFPL provides a list of database names, link, and icons,
but no description of what they contain. This could lead to wasted time for users who do not
already know what database will suit their needs.
Control and Flexibility. Again, TFPL's DL outsources their collection, providing users
with access to a number of DL, but none that are truly their own. Therefor TFPL does not
provide its users with control or flexibility.
Learnability and Memorability. Once a user finds their way to each digital collection
there should be no confusion in the future because links are properly labeled. Before leaving the
TFPL's page and entering eLibraryNJ users are presented with instructions on how to access the
collection and are made aware of the guided tour provided, as well as other helpful instructions.
Other than that, there is not much instruction for users so it is up to them to learn on their own.
Help. While TFPL does provide a FAQs section, it is semi-hidden in a drop down menu
of links labeled “About Us”. These FAQs cover the entire library, digital, physical, and even their
special collections. There is the “Ask a Librarian” feature, but this would be considered a tool for
research, not help, which is used for overcoming roadblocks in using the DL. Besides the
descriptions of certain pages and log in instructions there is no other help provided.
User Ability. The TFPL does not work to meet the needs of users' possible varying levels
of skill and ability. This is interesting because the TFPL notes that they are “serving as a bridge
across the digital divide for Trenton’s citizens” in their “About Us” section.
Feedback and Communication. The TFPL does provide a “Contact Us” page that can be
accessed by a link located at the bottom of each page, a common location point for this resource.
The contact page lists the names and positions of various administration, service, and IT staff,
along with email addresses and phone numbers to reach them at.
4.5 Princeton Public Library
The Princeton Public Library's eLibrary contains eBooks, audiobooks, magazines, music,
film & TV, language programs and other online courses. Their eBooks come from Princeton
OverDrive, a collection that “is hand-selected and built from the ground up,” as well as
TumbleBookLibrary and EBSCO eBooks. Some other services include hoopla for audiobooks,
music, and movies, Mango and Muzzy Language Learning for learning a new language, and
Zinio for magazines.
Design Elements. PPL's layout provides an easy to read list of services and other digital
offerings which additionally uses a row of anchored links that will move the user to a particular
section of the list without leaving the page. The links that do lead to outside pages open in a new
window. This is a good feature for users, allowing them to technically stay in the PPL's DL while
accessing other digital content. The PPL does separate their databases from the previously
mentioned digital content, providing an alphabetical list as well as one that groups the databases
by topic.
Control and Flexibility. While many of the DLs provided by PPL are outside sources, and
therefor the PPL has no say over the control and flexibility provided, they do provide users with
options for customization through their eBook collection in Overdrive. Not only does this DL
provide users with the ability to create lists of favorites or search preset lists of titles (non-fiction,
fiction, specific genres, etc.), as well as other options, it also is said to have its contents handselected by the PPL.
Learnability and Memorability. Besides the separation of digital content, the PPL's DL is
relatively intuitive thanks to its straightforward link labels and additional descriptions. Most
descriptions include simple written instructions on how to access each digital collection as well
as a link to its own user guide.
Help. As previously mentioned, most descriptions include a link to a user guide which
can help both first-time and returning users. There is a “Ask Us” link, which provides users with
email addresses, phone numbers, and a chat option that is available during their regular business
hours, but this seems separate from the PPL's DL.
User Ability. This DL does strive to span all age and interest range. It also provides a link
at the top of each page “En Espanol”. One might imagine that this link would switch the text on
the page from English to Spanish, but instead it brings users to another single page entirely in
Spanish that attempts to combine all of the library's information into one area. The PPL's
Overdrive DL does have language options, allowing users who are not as proficient with English
the same access as those who are.
Communication and Feedback. The PPL does provide their “Ask Us” section which
includes email addresses, phone numbers, and a chat option. Another “Contact Us” page that
strictly contains phone numbers provides users with contact information for particular
departments of the PPL.
4.6 Concluding Notes
The evaluation of The Mercer County Public Library's, The Hamilton Free Public
Library's, The Trenton Free Public Library's, and The Princeton Public Library's digital libraries,
with its focus on usability and user satisfaction, has resulted in the following points:

Interface design is not an issue for digital libraries. Oftentimes simple, straightforward
fonts and color schemes are used and images that are presented are large enough for users
to understand their purpose. Overall, the layouts of pages do not lead to any additional
confusion for users.

Digital libraries often split their collections between what could be considered
entertainment and research. Digital libraries will group their eBooks, audiobooks, music,
videos, and other similar digital collections into one page or list, while placing their
databases in another.

A digital library's collection is mainly a grouping of access points to other digital libraries
or outside resources. These individual digital libraries and resources can be drastically
different from one another in their content, services, and work flow. This has a direct
effect on all of the criteria used for evaluation, especially when looking at control and
flexibility, learnability and memorability, the users' different skill and ability levels, and
their overall satisfaction.

Digital libraries either assume to be or depend on being intuitive to their users. Often
there are little to no descriptions or help features provided for users.

When help is provided it provides information for both the physical and digital library,
usually with its emphasis on the physical library.

Digital libraries do provide contact information but it is often grouped together with the
help provided and seems to be more for the physical library than its digital counterpart.
Additionally, although many libraries provide email addresses and telephone numbers,
others only provide preset forms to send in. This offers no accountability on the library's
end.

Digital libraries tend to fit into a “one size fits all” mentality when dealing with their
users' variety. Besides the PPL's attempt at providing information in Spanish, none of the
libraries allowed for any customization or other options that could benefit users of
different technical skill and educational level or users with disabilities.
5. Implications for Future Digital Libraries
This study's research, its collection of previously published literature and findings from
its evaluation of the four DLs, can benefit both current and future DLs, especially those that
focus on meeting the needs of users who are not specifically focused on research and academics,
and others who work with them or in similar information science fields.
DLs should consider their place within the library as a whole and work to function as an
independent unit. Too often DLs are simply one link or page in a public library's website. DLs
should be created and maintained to stand on their own as a library, not a collection or smaller
part of one. This can be done by providing contact information and help features that are
exclusively used for the DL. If a DL only provides one point of contact or a couple of FAQs on
downloading an eBook, they should at least separate them from contact information and help that
almost solely focuses on their physical counterpart.
Similarly, they should provide a unified experience for users by combining the resources
they provide access to. At a broad level this can be done by looking at the types of resources the
DL provides. While it may seem that separating the digital collection into entertainment
resources and research resources will be helpful for users, if this is done by placing access points
throughout a web page it will only make navigation and learnability harder for them. Although
watching a movie and performing research in a scientific journal can be very different activities,
they are both a part of the digital library. At the same time DLs should look into ways to unify
these resources at a more micro-level. It is not expected of DLs, especially public ones, to always
build their own collection and then host it online; DLs often work to provide access for these
already created and maintained collections. But consideration should be given into combining
these multiple resources in a way that allows for a static search system, allowing for more
control and flexibility among users.
Contact information and help features were previously mentioned, but should also be
considered due to their large absence in many DLs. Often a DL's contact information is used
interchangeability with its help feature and does not work as a means of communication for
users. A DL's contact information and system should also work as a place for suggestions,
criticism, or a means to share information. Help features should be more prevalent in DLs and
provided in multiple formats. Most often FAQs are used as the sole means of help in DLs. Other
features such as chat functions or video tutorials could help numerous users. A DL could have a
separate section for these help functions, as to not clutter the other parts of the collection. DLs
should not be considered intuitive, mainly because users are diverse and ever-changing.
Finally, DLs should work to further consider the user and their satisfaction. Users are
more than their interests or information needs; they come with their own history and
expectations, skills and weaknesses, abilities and disabilities. One of the most common
differences, ones native language, does not seem to be a consideration for many DLs. A DL relies
on users and earns them through their satisfaction. Users cannot be satisfied with a system they
cannot use.
6. Limitations of the Study
It should be noted that this study was performed by a single user who has his own history
and expectations, skills and weaknesses, and abilities and disabilities. Not only did the user enter
this evaluation with working experience in public libraries, as well as education in the library and
information science field, they began their evaluation after reading, and therefore being
influenced by, a number of publications on the topic.
7. Conclusion
Digital libraries are amazing sources of information and entertainment for their vast
number of users. Public digital libraries serve an even more complex group of users who often
have different interests, information needs, skill levels, and other background features that will
affect their use of the digital library. These libraries depend on their users' continued use and
satisfaction or they will become defunct. Despite these facts, past publications on the topic of
digital libraries have often ignored the users and their perspective when evaluating digital
libraries. Through its review and subsequent evaluation, this paper has shown that there is a need
to focus on the user's perspective during the evaluations of digital libraries and that
improvements can me made to better satisfy the user's time spent employing our digital libraries.
Works Cited
Bollen, J. & Luce, R. (2002). Evaluation of digital library impact and user communities by
analysis of usage patterns. D-Lib Magazine, 8(6), 15p.
Chen, Y., Germain, C. A., & Rorissa, A. (2011). Defining usability: How library practice
differs from published research. Portal: Libraries And The Academy, (2), 599-628.
Frias-Martinez, E., Chen, S. Y., & Liu, X. (2009). Evaluation of a personalized digital library
based on cognitive styles: Adaptivity vs. adaptability. International Journal Of
Information Management, (29), 48-56.
Fry, A., & Rich, L. (2011). Usability testing for e-resource discovery: How students find and
choose e-resources using library web sites. The Journal Of Academic Librarianship,
(37), 386-401.
Fuhr, N., Klas, C., Aalberg, T., Sølvberg, I., Hansen, P., Kovács, L., & ... Landoni, M. (2007).
Evaluation of digital libraries. International Journal On Digital Libraries, 8(1), 21-38.
Hariri, N., & Norouzi, Y. (2011). Determining evaluation criteria for digital libraries' user
interface: A review. Electronic Library, 29(5), 698-722.
John Carlo, B., John T., S., Paul T., J., & Charles R., M. (2006). Functionality, usability, and
accessibility: Iterative user-centered evaluation strategies for digital libraries.
Performance Measurement & Metrics, 7(1), 17.
Monopoli, M., Nicholas, D., Georgiou, P., & Korfiati, M. (2002). A user-oriented evaluation of
digital libraries: case study of the 'electronic journals' service of the library and
information service of the University of Patras, Greece. Aslib Proceedings, 54(2), 103117.
Tweaks we've made to the digital branch. (2012). Library Technology Reports, 48(6), 7-11.
Xie, H. I. (2008). Users’ evaluation of digital libraries (DLs): Their uses, their criteria, and their
assessment. Information Processing And Management, (44), 1346-1373.
Xie, H. & Cool, C. (2006). Toward a better understanding of help seeking behavior: An
evaluation of help mechanisms in two IR systems. Proceedings Of The ASIST Annual
Meeting, 43 (ASIS and T '06 Information Realities: Shaping the Digital Future for All),
12p.
Yan, Y., Zha, X., Zhang, J., & Hou, X. (2014). Comparing digital libraries with virtual
communities from the perspective of e-quality. Library Hi Tech, 32(1), 173-189.