RBMS Metrics and Assessment Task Force
Subgroup 4: Use & Users, Web Stats, Circulation, Gate Counts, and Community Impact
Report Section Draft – Mark Greenberg (01/21/13)
WEBSITE USAGE
Web analytics data for special collections websites, including “home pages” describing
department and collections/services overall, virtual exhibits, collection databases (finding aids,
image databases, etc.); statistics from social media applications; user surveys).
Measuring Operational Activities, Efficiencies, etc.:
1. Describe the activity or process to be measured.
The activity measured consists of patron access to web-based special and digital collections
content, including department and collection information, services, digital exhibitions, policies
and procedures, and blogs on departmental websites, in institutional repositories, in commercial
repositories, and/or on social media sites. Digital objects created, converted, or born-digital may
include text (including ETDs), images, audio/video, moving image materials, and EADs.
2. Identify existing metrics (i.e., measures or counts), noting any references to
international, national, or community standards, guidelines, or definitions.
Existing metrics for web page use offer a wide range of information, including patron
demographics, the total number of distinct web pages and digital objects publicly available to
patrons on a special and/or digital collections website; the total number of times patrons view
and/or provide feedback for each page; the time spent on each page; and the number of times
patrons view, provide feedback, and/or download digital objects. Facebook has its own metrics
and definitions, including total likes, friends or fans, posts, people talking about this, check-ins,
and weekly total reach.1 YouTube metrics bear a close resemblance to Google Analytics, with
appropriate modifications for video content.2
1
2
See: http://ads.ak.facebook.com/ads/creative/insights/page-insights-guide.pdf
See: http://support.google.com/youtube/bin/answer.py?hl=en&answer=1714323
RBMS Metrics Task Force, Subgroup 4
Website Usage / 2 of 5
Prior to recent changes, ARL Supplementary Statistics counted the following relevant statistics:
number of search queries in databases or services, number of virtual visits to the library website
and catalog, number (objects or unique files) and size (in gigabytes) of digital collections,
number of times digital items were accessed, and number of queries conducted.3 ARL defined
virtual visits to the library website and catalog as “a user’s request of the library website or
catalog from outside the library building excluding the number of pages or gratuitous elements
… viewed” and excluded robot or spider crawls and page reloads. For digital objects and files,
ARL also excluded backup copies or mirror sites in order to prevent counting the same digital
content more than once. The 2011-12 ARL Statistical Survey removes data for virtual website
visits but in the section “Titles and Volumes,” it states “If your library digitizes content from its
own collection and the content is accessible under current copyright law you can report it. Do not
count HathiTrust, CRL, Internet Archive, etc. unless your library owns the digitized item and it
is accessible under current copyright law.”4 Further, for 2011-12 statistics, ARL has asked
library members to report separately for Special Collections units, total expenditures (excluding
fringe benefits), total materials expenditures, other operating expenditures, and total staff FTE.
ARL offers extensive instructions for reporting expenditures salaries and wages, and staffing.
ACRL also collects statistics. Its 2010-2011 worksheet requests information on the number of
virtual visits to the library website and catalog and uses the ARL data collection instructions.
ACRL also asks for the number of digital collections and items, their total size in gigabytes, the
number of times items were accessed, the total number of queries conducted, and the direct cost
in personnel, equipment, software, and contracted services to construct and manage digital
collections.5
Additional information on existing metrics is provided by a variety of digital content
repositories. The Florida Virtual Campus counts a view every time a user opens an object. A
search query that produces results does not count as a view. Accessing the bibliographic record
for an object also does not count as a view. Anytime the user views an item in the Image Viewer,
PDF viewer, etc., it counts as one view. For digital books, serials, etc., that display in the Image
Viewer, each time the user “turns the page,” it counts as a view. Multi-page texts packaged as a
PDF count as one view no matter the number of pages turned.
Publicly accessible website metrics or statistics for special and digital collections sites
specifically proved difficult to find and may not be reported separately at most institutions, but
3
See, for example, the 2008-09 ARL Supplementary Statistics Worksheet, available from:
http://www.libqual.org/documents/admin/09ssurvey.pdf
4
See the instructions for completing the 2011-12 ARL Statistical Survey, available from:
http://www.arlstatistics.org/About/Mailings/stats_2011-12
5
See: http://acrl.countingopinions.com/docs/acrl/ACRL11Survey.pdf
RBMS Metrics Task Force, Subgroup 4
Website Usage / 3 of 5
library-wide statistics are available on the Internet for several institutions. Metrics included in
their online reports vary. For example, Georgetown University Library for FY 2012 counted
unique visitors, web pages viewed, home page viewed, and top visited web pages.6 The
University of Georgia Libraries reports total page views and total visitors.7 Other institutions,
such as the Augustus C. Long Health Sciences Library at Columbia University, reported total
page views, visitors, access paths, and browser and mobile device use.8 Duke University
Libraries reports visits and page views, campus vs. off campus access, how patrons found the
libraries’ websites, and browser and mobile device use.9
3. Comment on:
a. the adequacy and the inadequacy of existing metrics, especially those that reference
international, national, or community standards, guidelines, or definitions;
A modest body of scholarship within the last decade analyzes best practices in library website
metrics and analysis.10 Several institutional websites offer additional insight. Iowa Library
Services, State Library of Iowa that the number of visits and pages visited are the most valuable
web metrics, and recommends “pay little or no attention to the other statistics.” Iowa Library
Services defines a visit as “every time a browser open your Website.” A unique visitor is counted
as one computer or IP address visiting the website,11 but ARL noted the misleading nature of
some user’s IP address due to ISPs, firewalls, and proxy servers. According to Thomas Crichlow,
Duke University Libraries’ varied collections, multiple website domains, and diverse patron
usage patterns across domains, make accurate, broad generations difficult. Changes to Duke’s
digital asset management system saw total page views fall, but 87 percent of the decrease was
due to changes to how search results display, multi-item view sizes, and paginated items.12
Less clear from the metrics is how best to interpret patron behavior in the context of the content
accessed. For example, does “average time on page” have the same meaning for a home page
and digital exhibition page? Time spent carefully reading content within a digital exhibition
likely is far more meaningful to research and learning than patrons who struggle to find
appropriate links on a home page and thus linger. Similar issues exist for a variety of other
metrics, and thus content context is important to assessment.
6
See: http://www.library.georgetown.edu/about-us/numbers/web
See: http://www.libs.uga.edu/statistics/
8
See: http://library.cumc.columbia.edu/health-sciences-library-website-statistics
9
See: http://library.duke.edu/about/assessment/web/documents/webstats-fall-2011.pdf
10
See in particular, Elizabeth L. Black, “Web Analytics: A Picture of the Academic Library User,” Journal of Web
Librarianship, vol. 3, no. 1, pp. 3-14; Laura B. Cohen, “A Two-Tiered Model for Analyzing Library Website Usage
Statistics, Part 1: Web Server Logs,” Portal: Libraries and the Academy, vol. 3, no. 2, pp. 315-26, and “Part 2: Log
File Analysis,” Portal: Libraries and the Academy, vol. 3, no. 3, pp. 517-26.
11
See: http://www.statelibraryofiowa.org/ld/q-s/silo/web/web-stats/web-stats-intro
12
See: http://library.duke.edu/about/assessment/web/documents/webstats-fall-2011.pdf
7
RBMS Metrics Task Force, Subgroup 4
Website Usage / 4 of 5
b. methods commonly used to collect data required by the metric;
Google Analytics (GA) offers an effective page tagging tool for keeping statistics on website
usage, including for tools like LibGuides, though server log file products such as AWStats and
Webalizer are also useful. GA collects a wide range of information, including information on
user demographics, behavior, technology, mobile devices, visitor flow, and content. Metrics
include page views, unique page views, pages viewed per visit, average time on page, bounce
and exit rates. YouTube and Facebook have their own internal data collection methods. The
plethora of possible reports produced by products such GA, AWStats, Webalizer, YouTube, and
Facebook makes focusing on a select group of metrics important. For digital collections use,
most digital asset management systems have their own method to collect data. More important
than the method for digital collection use is a common understanding of the metrics’ definitions.
c. the priority that RBMS should place on developing a better metric and related
definitions and guidelines for use.
Google Analytics, and its associated metrics and definitions, offers a suitable tool for tracking
website use, though further comparison between GA and server log file products like AWStats
and Webalizer may be appropriate. RBMS could consider requesting that special and digital
collections units use a common tool in the event that differences between products make crossinstitutional comparison difficult. Regardless of the tool, RBMS should also consider how many
and which metrics are appropriate for assessment purposes. Given the extensive number of
“things” that can be counted in Google Analytics, the relative usefulness of each, and problems
associated with some measures, RBMS should decide what counts are most accurate,
informative, and important for assessment purposes. RBMS should also consider how much and
which data it wishes to report on digital collections use. Trends within ARL to decrease reporting
requirements on digital collections and ACRL’s continued use of more robust reporting should
be reconciled. In its efforts to improve assessment, RBMS should give priority to locating or
developing methodologies for documenting the impact and value of web page and digital
collection use.
Assessing Value and Impact for Users
Thomas Crichlow of Duke University Libraries offers several useful values, impacts, and
possible outcomes. The number of patron visits and page views in relative terms can help
institutions adjust how they design homepages and promote and support various resources.
Comparing on-campus and off-campus access to web pages or digital objects sheds light on
which audiences are served by the content. A better understanding of traffic sources can focus
RBMS Metrics Task Force, Subgroup 4
Website Usage / 5 of 5
greater institutional attention on access methods such as portals and search engine optimization.
Awareness of which browser platforms and devices patrons most often use can focus efforts on
developing effective web technologies and interfaces. Overall, determining patron demographics
and behavior for special and digital collections website and digital object use can inform the way
an institution marshals its resources and expends its energies. More difficult to ascertain is the
value these efforts have on patrons. Use does not equate with impact or value.
Documenting the impact and value of website and digital object use on patrons is far more
difficult than the statistical analysis that comes from products like Google Analytics. I am not
aware of any current national or international standards for determining impact, though I suspect
some institutions have done qualitative studies with their patrons and have sought to track the
use of digital objects in publications, etc.
A plethora of well-established metrics and tools exist for gathering quantitative measures to
assess website and digital collection use. These metrics are most suited for determining patron
demographics and behavior but not impact.
Based upon patron demographics and behavior, special and digital collections departments could
refocus financial and human resources away from content little used, build content to meet
proven demand, increase marketing efforts to grow demand, redesign web pages, and/or improve
tagging to aid in navigation and searchability.