Listening to Students:
Innovative Responses
Betsy Wilson
University of Washington
July 10, 2008
7th International JISC/CNI Conference
The Role of Library Assessment
The library community needs to invest more in data
collection and analysis and to take its examples
from commercial leaders that have a much more
detailed and insightful understanding of their
customer base and preferences. In particular,
there is a need for ongoing longitudinal data and
intelligence functions to provide a vital early radar
warning of oncoming change.
No library we are aware of has a department
devoted to the evaluation of the user, how can
that be?
(Information Behaviour of the Researcher of the Future, CIBER, January 2008)
North American University Libraries with
Notable Assessment Programs
• Alberta “Ongoing student
surveys”
• Cornell “In-house
research capability”
• Pennsylvania “Data
Farm”
• Rochester “Ethnographic
approaches”
• Virginia “Balanced
Scorecard”
• Washington “User needs”
Building a Community of Practice:
Library Assessment Conference
Co-sponsored by ARL,
Virginia and Washington
– 2006 Charlottesville,
Virginia (220 registrants)
– 2008 Seattle, Washington
(August 4-7, 380
registrants)
– 2010 Washington D.C.
area
• Focused on practitioners
– Papers, panels, posters,
informal contact,
workshops
ARL Sponsored Assessment
•
•
•
Tools
– ARL Statistics
– LibQUAL+®
– MINES for Libraries
Building a Community of Practice
– Library Assessment
Conferences
– Service Quality Evaluation
Academy
– Library Assessment blog
– Workshops
Individual Library Consultation
– Dynamic Duo: Jim Self (Virginia)
and Steve Hiller (Washington
– Making Library Assessment
Work (24 libraries)
– Effective, Sustainable, Practical
Library Assessment (6 libraries)
UW Libraries Assessment Priorities
• Understanding how
faculty and students work
• Information seeking
behavior and use
• Patterns of library use
• Value of library
• User needs
• Library contribution to
customer success
• User satisfaction with
services, collections,
overall
University of Washington
• Established in
1861
• Campuses in
Seattle, Tacoma,
and Bothell
• Public research
university
Students and Learning
• Diverse students
• 30,000
undergraduates
• 9,300 graduate
students
• 1,700 professional
students
Research and Discovery
• Collaborative and
interdisciplinary
• $1 billion US in grants
and contracts for
research
• Engineering,
technology, forestry,
aerospace, marine
sciences, health,
biosciences,
computer sciences
Seattle and Innovation
•
•
•
•
•
•
•
•
Microsoft
Amazon
RealNetworks
Serials Solutions
Starbucks
Boeing
Biotechnology firms
Gates Foundation
University of Washington Libraries
• Large, public research
library
• Over 7,000,000 volumes
• 50,000 serial titles
• Digital collections
• Excellent service
• Expert staff
• Innovation
• Global collections
• Assessment and planning
since 1992
Award Winning Library
Culture of Assessment
• …is an environment in
which decisions are based
on facts, research and
analysis, and where
services are planned and
delivered in ways which
maximize positive outcomes
and impacts for library
clients. A culture of
assessment is an integral
part of the process of
change and the creation of
the 21st century research.
All assessment is local
Creating a culture of
assessment
Methodological Diversity
• Qualitative and
quantitative methods
• Surveys (since 1992)
• Focus groups
• Observation and
interviews
• Usability
• LibQual
• Data mining
• And more…
We learned that…
• Remote use preferred
• Self-reliance and
unmediated
• Online equals
productivity
• Desktop delivery is
top priority
• Undergraduates value
place
Biosciences at the UW
• Internationally ranked
graduate and
research programs
• Largest segment of
University research
community
• Receive 80% of
externally funded
research monies
Key Questions
• Who are the bioscience
students and researchers?
• What do they do?
• Where are they physically,
administratively, and
intellectually?
• How do they find and use
information?
• What are the barriers?
• How can the Libraries help?
• How will we know if we
have made any difference?
Bioscience Findings
• Print is dead, really dead
• e-journal provider
• Faculty don’t come to the
physical library
• Students come to the
physical library
• Databases underused
• Personal information
management
• Buy from Amazon
More Findings
• Transaction cost from
discovery to delivery
• Fragmented systems
and processes
• Multi-disciplinary
collaborators
• Everywhere, in
scattered locations
• Independent and selfsufficient researchers
Biosciences Recommendations
• Integrate search/discovery tools into users workflow
• Expand/improve information/service delivery options
• Make physical libraries more inviting and easier to use
– Consolidate libraries, collections and service points
– Reduce print holdings
– Focus on services
•
•
•
•
Use an integrated approach to collection allocations
Get librarians to work outside library space
Lead/partner in scholarly communications and E-science
Provide more targeted communication and marketing
Biosciences Actions 2007-08
•
•
Appointed a Director,
Cyberinfrastructure Initiatives,
Biosciences, and eScience
Strategic priorities:
– Improve discovery to delivery
(WorldCat Local, etc.)
– Reshape physical facilities as
discovery and learning centers
– More rapid delivery services
– Enhance the Libraries support
for UW’s scientific research
infrastructure (DataNet, etc.)
– Do market research
Can We Generalize?
Did themes raised in the interviews/focus groups
reflect the bioscience population? The campus
community?
The 2007 Triennial Survey as a corroborating source:
• Mode of access
• Resource type importance
• Sources consulted for research
• Primary reasons for using Libraries web sites
• Libraries contribution to work and academic
success
• Useful library services (new and/or expanded)
Reasons for In-Person Library Visits
2001
Faculty and Undergrads Visiting Weekly or More Often
100%
Faculty
90%
80%
Undergrads
70%
60%
50%
40%
30%
20%
10%
0%
Use Collections
Use space or Only Collections Only space or
services
services
Library as Place
Change In Frequency of In-Person Visits 1998-2007 (weekly+)
80%
Grad
70%
Undergrad
Undergrad
60%
50%
Grad
Faculty
40%
30%
Faculty
20%
1998
2001
2004
2007
Undergrad Activities During
Library Visits
(% using at least weekly in 2004/2007)
70%
60%
50%
40%
30%
20%
10%
0%
Get
assistance
Look for
material
Use lib
computers
2004
2007
Individual
Work
Group Work
Off-Campus Remote Use 1998-2007
(Percentage using library services/collections at least 2x week)
70%
70%
Grad
60%
60%
50%
50%
Faculty
40%
40%
30%
30%
20%
Undergrad
20%
10%
10%
0%
0%
1998
2001
2004
2007
Sources Consulted for Information
on Research Topics
(Scale of 1 “Not at All” to 5 “Usually”)
Undergrad
Grad
Faculty
Open Internet
Search
Open Internet Ref
Source
Library Catalog
Bibliographic
Databases
2.5
2.75
3
3.25
3.5
3.75
4
4.25
4.5
Sources Consulted for Information
on Research Topics (Scale of 1 “never” to 5 “usually”)
4.5
4.25
4
3.75
3.5
3.25
3
Health Sci
BioSci
Bib Databases
Phy Sci-Engin
Open Internet
Hum-Soc Sci
Library Catalog
I wish the interface between scholar and UW libraries was better. I want to search with
Scholar, but use UW's credentials to access full-length articles. Right now, there are a
lot of intermediary pages I need to visit. Bioengineering grad student
Libraries Contribution to…
(Scale of 1 “Minor” to 5 “Major”)
Being a more
productive researcher
Keeping current in
your field
Finding info in new or
related areas
Efficient use of time
Grad
Faculty
Academic success
3
3.25
3.5
3.75
4
4.25
4.5
4.75
Takeaways after 16 Years
• Undergraduates
– Library as Place
– Work, meet, learn, live
• Graduate/professional
students
– Access to information and
services
– How can the library save
me time?
• Faculty
– Collections (physical and
virtual)
Innovative Responses
•
•
•
•
•
•
•
•
•
Extend hours in Undergraduate Library (24/7)
Create more diversified student learning spaces
Privilege electronic journals
Enhance usability of discovery tools and
website
Provide standardized service training for all staff
Stop activities that do not add value
Consolidate and merge branch libraries
Change/reallocate collections budget and
staffing
Improve librarian liaison program to academic
areas
WorldCat Local
• Discovery to delivery challenge
• One box, one stop shopping
– Formats integrated
– UW holdings, regional consortium, WorldCat, 50 million articles
– User doesn’t need to know how we do things internally
• Components
– Content
– Shared platform
– Interoperability with local services
• Circulation, ILL, OpenURL
– Common marketing
Results to Date
• 20% increase in use of
local collections
• 60% increase in
borrowing from regional
consortia
• 114% increase in worldwide interlibrary loan
• 200-275,000 searches
per month
• WCL is our #2 ranked
OpenURL origin and
growing
Innovation
UWILL
• University of
Washington
Information Literacy
Learning
lib.washington.edu/uwill/
UWired
• Established in 1994
• Multiple partners
• Teaching, learning,
and technology
• Information smart
global citizens
Successes
•
•
•
•
•
•
•
•
Collaboratories
Center for Teaching
Learning Commons
Technology-enhanced
spaces
Course tools
Workshops
MyUW
ONTECHNews
The Challenge: Maintain Relevancy
and Centrality
4.6
4.6
Overall Satisfaction by Group 1995-2007
4.5
4.33
4.33
4.34
Faculty 4.25
4.26
4.2
4.1
4.5
4.44
4.4
4.3
Faculty 4.56
Grad 4.18
4.11
4.32
4.4
Grad 4.36
UW Seattle UG 4.36 4.3
4.2
4.22
4.1
4
3.9
4
Undergrad 3.97
3.99
3.9
3.8
3.8
1995
1998
2001
2004
2007
Working without a Net
Suggestion Box in Hell
Listening and Responding
For more information:
http://www.lib.washington.edu/assessment/
Thank You