DOWNLOADS OR OUTCOMES? :
MEASURING AND COMMUNICATING
THE CONTRIBUTION OF LIBRARY RESOURCES
FACULTY AND STUDENT SUCCESS
Rachel A. Fleming-May, Ph.D., M.L.I.S.
Assistant Professor,
School of Information Sciences
The University of Tennessee-Knoxville
rf-m@utk.edu
TO
• Overview of current
concepts and practice
related to e-Resource
usage measurement

Discussion
• Alternative and emerging
techniques being
investigated by the IMLS
Lib-Value Project
AGENDA:
MY INTEREST
IN THIS ISSUE:
• Former Practitioner at an ARL Library
• Dissertation: “Use” in the Literature of LIS—Concept Analysis

“What is Library Use? Facets of Concept and a Typology of
its Application in the Literature of Library and Information
Science” (in an upcoming, but still unidentified, issue of The
Library Quarterly)
• Use…USAGE—measurement of e-Resource usage

With Jill Grogg: The Concept of Electronic Resource Usage
and Libraries (Library Technology Reports, Aug./Sept. 2010)1
• IMLS-funded Lib-Value Grant Management Team Member
AN EVENT?
SOMETHING THAT CAN BE MEASURED?
…WITH NUMBERS?
Use is often treated as a
PRIMITIVE CONCEPT in
Library and Information
Science:
an idea so fundamental to
the theoretical framework
as to be indefinable, even
when presented as a
phenomenon to be
measured and quantified.
USE IS FREQUENTLY ASSESSED IN ORDER TO
GENERATE “OBJECTIVE” DATA FOR DECISION MAKING.
TO MEASURE USE
We focus on Inputs
Number of patrons
who enter the building
…and Outputs
…such as the number
of book circulations.
WHAT
ABOUT ELECTRONIC RESOURCES?
• Many instances of use are removed from
the library, thus unobservable
• Multiple points of access (such as Google
Scholar) further confuse the issue: patrons
are less aware that they’re using library
resources
“some of the basic ‘natural laws of library
and information science’ may not apply as
well or as consistently in the realm of
electronic information discovery and use”4
“Among other changes, the
Complete College Tennessee Act:
• Funds higher education based
in part on success and
outcomes, including higher
rates of degree completion.”
“Questions such as, ‘Who uses these
resources?’ or ‘Are these huge outlays
of funds justified in terms of use, or
value derived from use?’ or ‘What
difference do all of these resources
make to students and faculty in
universities?’ must be answered if
university administrators, trustees,
students, and faculty are expected to
support ever-increasing levels of
funding for the acquisition and
development of these resources and
services.”5
Understanding
USE Matters.
SO,
IS
USE
A
PRIMITIVE CONCEPT?
No. Use does not, in fact, have a singular
conceptual meaning in the LIS domain and
can signify many actions, processes, and
events.
THE USE TYPOLOGY: DIMENSIONS
I.
OF
USE
Use as an Abstraction
Ia.
Use as a Facilitator
II. Use as an Implement
III. Use as a Process
IV. Use as a Transaction
IVa. Use as a Connector
• A GENERAL TERM FOR ALL TYPES OF
LIBRARY/INFORMATION USE
• DISASSOCIATED FROM ANY SPECIFIC
INSTANCE OF THE PHENOMENON
“Of the 57,148 households [surveyed], 27,511
(48.1%) had a household member who used the
public library in the past year. ”6
USE
AS A
PROCESS
Application of library/information
resources, materials, and/or services…

To complete a complex or multi-stage task

To the solution of a problem
“This study reveals that undergraduate students
experience information use in a complex, multitiered way that needs to be addressed by higher
educators when creating information literacy
pedagogy.”7
USE
AS A
TRANSACTION
• Isolated
instances of
library or
information use
• Can be recorded
and quantified
• Removed from
the user
“statistics provided by electronic book
vendors…show that [our] community uses ebooks quite heavily. The data do not show,
however, how books are used. For instance,
the available statistics show that a book
has been accessed but do not differentiate
between a one-second click on a title and
a five-hour immersion in a book…the data
also do not tell us why an electronic
version of a book was used instead of the
paper version”8
TRANSACTIONAL MODEL OF USE=
OVER-RELIANCE ON STATIC ASSESSMENTS
ELECTRONIC RESOURCE USAGE, SUCH AS…
OF
• Vendor-supplied data (COUNTER compliant or
otherwise)
• Transaction log analysis


Including page view time measurement (are they really
reading?)
Log-ons—what about database timeouts?
 Not exclusively
statistical
Visit to the
Reference
Desk
Db A: Log
on
Article
Download
UNDERSTANDING OF
USE AS PROCESS
 Requires multiple data
collection methods
 Requires “bipartisan”
support, i.e., working
with public services to
gain a fuller
understanding of how and
why patrons use the
resources they do.
HOW,
SPECIFICALLY?
• Observation
• Focus groups
• Interviews
• Surveys
• Inter-institution
information sharing
• Usability testing
• Triangulation.
THE LIB-VALUE PROJECT:
• Grant funded by IMLS, December 2009-2012
• Principal Investigators:

Carol Tenopir, University of Tennessee

Martha Kyrillidou, Association of Research Libraries

Paula Kaufman, University of Illinois

…other participants include librarians, academic faculty, and other researchers
from multiple institutions and disciplinary backgrounds
• Purpose: “…to study the value of academic libraries to students, faculty,
policymakers, funders…” and Return on Investment (ROI) in academic
libraries
• Follows two previous projects:


Phase I: Return on Investment in Academic Library (UTK and U of IL) as measured
by successfully funded grants9
Phase II: Expanded to 8 international universities10
IS
LIB-VALUE DIFFERENT?
• It’s a Comprehensive
Effort to analyze Value
and ROI using models
that


incorporate all inputs in
the library system
(faculty, staff, students,
library resources) and
determine how each
influences the system
articulate all values of
the library and areas of
investment and return
Teaching/
Learning
Scholarly Endeavors
HOW
Research
Social/
Professional
e-Science
Collaborative Scholarship
Institutional Repositories
Functional Areas
Slide adapted from Carol Tenopir’s presentation, “ForumValue, Outcomes,
and Return on Investment of Academic Libraries (Lib-Value) (funded by
IMLS)” at the January, 2010 ARL Assessment Forum, Boston, MA.
KEY QUESTIONS:
• Does the reputation of a
university’s library influence



Enrollment?
Recruitment of faculty and
students?
Material or financial
donations?
• Do library resources and/or
services play a role in

Student success?

Retention?
WHAT CAN
PROJECT?
PRACTITIONERS EXPECT FROM THE
• A fact-based articulation of the value and ROI of the
university library resources and services within the
wider mission of university administration.
• Develop a model for ROI and tools that implement
this model which can be used by other academic
libraries.

Modular: libraries will be able to choose from a “menu” of
approaches depending on the particular services and/or
resources they wish to assess
Slide adapted from Carol Tenopir’s presentation, “ForumValue, Outcomes,
and Return on Investment of Academic Libraries (Lib-Value) (funded by
IMLS)” at the January, 2010 ARL Assessment Forum, Boston, MA.
QUESTIONS?
Thank you for your time!
1.
Fleming-May, Rachel A., and Jill E. Grogg. 2010. The concept of electronic resource usage and libraries.
Vol. 46, Library Technology Reports.
2.
Swigger, Keith, and Adeline Wilkes. 1991. The use of citation data to evaluate serials subscriptions in an
academic library. Serials Review 17 (2):41-46; 52.
3.
Miller, Rush, and Sherrie Schmidt. 2002. E-Metrics: Measures for Electronic Resources. Serials: The
Journal for the Serials Community 15 (1):19-25.
4.
Peters, Thomas A. 2002. What's the use? the value of e-resource usage statistics. New Library World 103
(1172/3):39-47.
5.
Ibid.
6.
Sin, Sei-Ching Joanna, and Kyung-Sun Kim. 2008. Use and non-use of public libraries in the information
age: A logistic regression analysis of household characteristics and library services variables. Library &
Information Science Research 30 (3):207-215.
7.
Maybee, C. (2006). Undergraduate Perceptions of Information Use: The Basis for Creating UserCentered Student Information Literacy Instruction. The Journal of Academic Librarianship, 32(1), 79-85.
8.
Levine-Clark, Michael. 2006. Electronic Book Usage: A Survey at the University of Denver. portal:
Libraries and the Academy 6 (3):285-299.
9.
Luther, Judy. 2008. University investment in the library: What's the return? In Library Connect White
Papers.
10. Tenopir, Carol. 2010. University Investment in the Library, Phase II: An International Study of the
Library's Value to the Grants Process. In Library Connect White Papers.