Measurement
and
Assessment
of Library Programs
and Services
Eileen E. Hitchingham Dean, University Libraries
Virginia Tech (hitch@vt.edu)
The Changing Face of Libraries SOLINET 2002
Atlanta, GA
May 2, 2002
Q: How many statisticians does it take
to change a light bulb?
Statistics means never having to say
A: One -- plus or minus three
you're certain.
In God we trust... All others must bring data
Did you hear about the statistician
who had his head in an oven and his
feet in a bucket of ice?
When asked how he felt, he replied,
"On the average I feel just fine."
Sources:GARY C. RAMSEYER'S FIRST INTERNET GALLERY OF STATISTICS JOKES,
http://www.ilstu.edu/~gcramsey/Gallery.html, Statistics?!, Don’t Make Me Laugh!
http://www.keypress.com/fathom/jokes.html
Library Statistics...
600,000
500,000
To Show
or
700,000
Library
Visits
400,000
To Know?
300,000
(Jul -Nov)
200,000
100,000
40,000
Info/dir
Ref
0
98/99
30,000
20,000
10,000
0
97/98
98/99
99/00
2000/01 2001/02
99/00
00/01
Ptolemaic Information System
The University
Library
UGrad
Faculty
Grad
Sack, J. Open Systems for Open Minds: Building the Library Without Walls (College & Research Libraries 47(6): 535 -44, 1986
Copernican Information System
The Web
VT Community
of Users
University
Libraries
Other
Information
Sources
Assessment at VT Libraries
•
This year every position description in the library notes that the staff
member is responsible for collecting and using data to support continuous
process improvement
•
Assessment goals at the library:
– to better understand the wants, needs, and information seeking
behaviors of our user community so that we can provide them with
information collections and services that support their research,
teaching, learning and outreach goals
– to benchmark our processes and services against other libraries so that
we can consider emulating operations that appear likely to contribute to
making us more effective or efficient, and
– to continuously improve the processes and services associated with
acquiring, organizing, providing access to, preserving, and assisting
users in working with, information resources.
Examples of What We
Collect
• OPAC searches
• Circulations
• In-house use of
materials
• Library Visits
• Reference
interactions
• Storage items used
• Interloan data from
ILLiad
• Web hits, pages, etc.
• Peer group data
• Instruction
Evaluations
• LibQUAL+TM
OPAC searches
Author
Subject
Title
Call Number
Boolean
Keyword
Total searches
600,000
500,000
400,000
300,000
200,000
100,000
0
1997
1998
(Nov - Feb, kw and boolean counted as of Nov 99)
1999
2000
2001
2002
Circulations
180,000
120,000
60,000
0
1997
(Nov - Feb, initial and renews)
1998
1999
2000
2001
2002
OPAC Searches and Circulation
600,000
OPAC searches
External Circulations
500,000
400,000
300,000
200,000
100,000
0
1997
(Nov-Feb)
1998
1999
2000
2001
2002
Circulation Rates at Service Locations
97
98
99
2000
2001
30,000
25,000
20,000
15,000
10,000
5,000
0
A&Arch
(Jul - Feb, initial checkouts and renews)
GeoSci
Media Ctr
Storage
Vet Med
NVC
Circulation at Main Desk
97
250,000
200,000
150,000
100,000
50,000
0
(Jul - Feb, initial checkouts and renews)
Main Desk
98
99
2000
2001
Internal Use of Materials
Internal bks and bnd jrs
Microforms
Newspapers
Spec Collections
Current Sci sers
Government Docs
Ref sci
Current other sers
Ref not sci
Maps
90,000
80,000
70,000
79,075
77,710
83,639
60,000
50,000
40,000
30,000
20,000
10,000
0
98/99
99/2000
2000/01
Library Visits and Uses
800,000
700,000
600,000
500,000
400,000
300,000
200,000
100,000
0
98/99
99/00
Physical Materials Use
Jul-Nov
00/01
Gate
Web Pages Delivered From
Library Web - October
600,000
500,000
400,000
300,000
200,000
100,000
0
96
(October data, excludes Digital
Library and ILLiad)
97
98
99
2000
2001
Storage: Items Requested and
Items in Storage
Items in Storage
Items Requested
700,000
600,000
524,900
500,000
400,000
560,917
568,601
572,263
98/99
99/00
00/01
450,654
330,605
300,000
200,000
100,000
0
95/96
96/97
97/98
Storage: Types of Items Requested
Bks requested
Articles requested
Total items requested
24,000
18,000
12,000
6,000
0
95/96
96/97
97/98
98/99
99/00
00/01
ILLiad Everywhere Evaluation
(Fall 2001 Customers)
Interloan
Doc Deliv from VT campus
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Very bad
Bad
Adequate
Good
Very Good
Illiad: Number of Journals Titles Needed to
Deliver a Percentage of 32,664 Articles
1,930
2,000
1,800
Journal Titles
1,600
1,486
1,400
1,122
1,200
1,000
831
800
603
600
419
400
200
0
30
82
10
Pct
20
Pct
163
30
Pct
275
40
Pct
50
Pct
60
Pct
70
Pct
80
Pct
90
Pct
100
Pct
Rethinking Operations and Staff
Assignments
Info/dir
Ref
40,000
35,000
30,000
25,000
20,000
15,000
10,000
5,000
0
97/98
First Qtr
98/99
99/00
2000/01 2001/02
Staff Resources by Service Area
0%
TS
Ref
Instruc
Coll Mgmt CL Shelv
Systems
Branches
Dig Lib
Admin
ILL
Circ/Res
Bus Services
5%
10%
15%
20%
25%
Instruction and Outreach Data
Projected Total Library
Expenditures
$30,000,000
$25,000,000
$20,000,000
$15,000,000
$10,000,000
$5,000,000
$0
1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999
VT
Top 21 -30
LibQUAL+TM 2001
• 2002 is third year of LibQUAL+TM
• Participants include ARL, ACRL, medical libraries,
community colleges
• Measures user perceptions of library services
• 3 questions overall satisfaction, satisfaction way treated,
satisfaction with access to collections
• 25 questions address service issues of library as place,
affect, personal control, access to collections
• Measures minimum acceptable service, perceived level
of service, desired level of service
General Satisfaction Questions -2001
9
8.5
8
7.5
7
6.5
6
Overall quality of service Satisfied with way treated Satisfied with support for
in libraries
learning, research,
teaching
VT UGRAD
VT GRAD
VT FAC
All LibQUAL
Gap score Affect: Desired minus Perceived
LibQUAL+TM 2001
1.5
VT Ugrad
VT Grad
VT Fac
1
0.5
0
employees
instill
confidence in
users
giving users emplyees deal employees
individual
with users in
who are
attention
caring fashion consistently
courteous
willingness to
help users
employees dependability
who have
in handling
knowledge to users' service
answer user
problems
questions
employees
understand
needs of
users
ready to
respond
questions
Gap score Personal Control: Desired minus
Perceived
1.5
VT Ugrad
VT Grad
VT Fac
1
0.5
0
making
library website
making
electronic enabling me to information
resouces
locate
easily
accessible information on accessible for
from home or
my own
independent
office
user
modern eqpt
that lets me
easily access
the
information I
need
convenient
access to
library
collections
easy-to-use
tools allow me
to find things
on my own
Gap score Library as Place: Desired
minus Perceived
1.5
VT Ugrad
VT Grad
VT Fac
1
0.5
0
haven for quiet
and solitude
contemplative
environment
place for
reflection and
creativity
comfortable and
inviting location
space that
facilitates quiet
study
Gap scores Access Collections: Desired
minus Perceived
VT LibQUAL+TM 2001
3
2.5
2
VT Ugrad
VT Grad
VT Fac
1.5
1
0.5
0
convenient
business hours
timely document
delivery/ill
comprehensive
print collections
interdisciplinary
library needs
addressed
complete runs of jr
titles
Gap scores Access Collections: Desired
minus Perceived
VT and All LibQUAL+TM 2001
3
2.5
VT Ugrad
All Ugrad
VT Grad
All Grad
VT Fac
All Fac
2
1.5
1
0.5
0
convenient
business hours
timely document
delivery/ill
comprehensive
print collections
interdisciplinary
library needs
addressed
complete runs of jr
titles
THANKS!