Informing Reference Services
@ Columbia University Libraries
Jennifer Rutner, Assessment & Planning Librarian
Columbia University Libraries
Using Today’s Numbers to Plan Tomorrow’s Services
RUSA Program, ALA Annual Conference
4:00PM, June 26,2011
Reference Assessment @ Columbia
Virtual Reference Assessment
Digital Centers
User Needs Assessment
Service Evaluation
Assessment Program
Mission Statement
Serve library users and
staff through the
gathering, analysis, and
application of highquality, actionable
information to guide
library decision making.
Project Teams and Decision Making
Appoint project
team
Conduct
assessment
Make
recommendations
Implementation
Defining Information Needs
Identify
existing data
Brainstorm
questions
Information
needs “IWTK”
Assign
methodology
Determine
audience
Prioritize
information
needs
Develop
protocol
Library vision
User needs
Assessing IM Reference
IM at Columbia Libraries
2001
• CUL launches Chat Reference
2006
• Evaluation of Virtual Reference transactions
2007
• Switch to Meebo
• Departmental library accounts
2009
• User Assessment
2010
• Switch to LibH3lp
• Expand staffing and hours, consolidate services
Motivation
• Understand how IM services are being used.
• Evaluate quality of service provided via IM
reference services.
• Identify user-needs for reference support.
Clients
• Reference Coordinating Committee
• IM Reference Coordinator
Methodology
Survey
• Service
evaluation
survey sent
to IM users
Focus groups
Data Analysis
• Students
who had
never used
IM
reference
• Libraries
staff
• Compilation
of IM
statistics
from 20072009
Findings: User Survey
•
•
•
•
•
46% were graduate students
41% were between 23-29 years old
61% were 1st time library IMers
79% discovered IM through the website
51% were at home when they IM’d
Findings: User Survey
• 74% were very satisfied with the information
they received when using the IM service
• 69% strongly agreed that the resources
suggested by the librarian were useful
• 85% strongly agreed that using IM to contact a
librarian was very easy
• 80% agreed that the hours were satisfactory
Findings: Focus Groups
• Hours: “It’s not 24 hours? That sucks.”
• Communication: “It is a little bit awkward, but it’s
good that librarians are taking this to the instant
messaging level. It’s just better service.”
• Marketing: “Very visible… if you’re trying to eventually
expand the hours and make it more visible for
students…”
• Staffing: “If they could answer the question, whoever
can answer the question.”
• Convenience: “I live off campus so, my Columbia
experience is minimal. So, this IM thing would probably
be helpful for someone like me, who doesn’t chill in the
library.”
Findings: Transaction Analysis
• 60% were policy/procedural, research or
holdings questions
• 8% were e-resource problems
• 90% come through the widget
• 57% required no follow-up
• 85% are <10 minutes
1702 transactions logged from 2007-2008.
Recommendations
An ideal IM service for our users would be a
24/7 service; staffed by knowledgeable,
friendly people; easy to access from the
Libraries’ web pages that they use most;
where they can get quick efficient service.
Impact: Service
Staffing
Hours
Platform
• Library-wide participation
• MLS interns
• 15 additional staff
• 36 additional hours/week
• Original: Live Assistance
• Interim: Meebo
• Today: Library H3lp
Impact: Usage
Year
2007
2008
2009
2010
2011 (through May)
# of IM Transaction
468
383
549
2293
1722
Digital Centers:
User Needs Assessment
Locations
Digital Social Science Center @ Lehman Social
Science Library
Digital Humanities Center @ Butler History &
Humanities Library
Digital Science Center @ Science & Engineering
Library
The Burke Library @ Union Theological Seminary
Avery Architectural & Fine Arts Library
Motivation
How is the changing information environment
impacting library use, and student needs for
technology and research support?
Clients
• Departmental libraries
• Libraries IT Office
• Library Leadership
Methodology
Social Sciences
Humanities
Sciences
2007
2009
2010
• Online
questionnaire
• In-library paper
questionnaire
• Focus groups,
TBD
• Online
questionnaire
• In-library paper
questionnaire
• In-library flip
charts
• Online
questionnaire
• Student focus
groups (2)
• Faculty
interviews
• Observation
studies
Results
Social Sciences
• Survey
n = 125
Humanities
• Survey
n = 940
Sciences
• Survey
n = 611
Impact: Digital Social Science Center
Mode
Location
Outreach
Study space
Technology
• From on-call to desk hours
• Increased visibility
• Centralized service point
• Promote consultation services
• “Info Expo” event
• Group study
• Presentation practice room
• Presentation practice room
• Quantitative support, GIS, bibliographic software
Impact: Digital Humanities Center
Staffing
• Dedicated 2 reference staff to the
DHC
Location
• Transitioning reference support
from the reading room to the DHC
Technology
Hours
• SCANNERS!
• Extended hours in the DHC
• Late night vs. overnight
Impact: Digital Science Center
Location
Staffing
• Increased reference visibility
• Emerging Technology Librarian hired
Study space
• Group study rooms and booths
Technology
• Quantitative software
• Visualization software
Instruction
• More workshops
Digital Centers: Evaluation
DSSC Evaluation
Motivation
Understand awareness of and
satisfaction with new services
at the DSSC.
Methodology
• Online survey
• In-library paper survey
Response
• 416 respondents
– 47% Graduate Students
– 80% from the social sciences
Findings: Technology
• 66% “never heard of” data services, Bloomberg
stations
• 47% “heard of/never used” GIS
• Overall satisfaction with technology services
Findings: Research Support
•
•
•
•
•
34% “never heard of” the reference desk
42% “never heard of” emailing a librarian
50% “never heard of” of research consultations
50% “never heard of” library workshops
66% “never heard of” IMing a librarian
Findings: Space
• 75% use the library for individual study
• 68% use the library for group study
• 67% are satisfied with individual study
• 67% are satisfied with group study
• 61% “never heard of” practice presentation room
• 73% “never heard of” conference calling room
Impact
Facilities
Study space
• Continue renovations
• Improve maintenance
• Continue to expand group study
Outreach
• Promote IM
• Promote GIS/Data services
Instruction
• Offer more workshops
• New orientation approach
Thank you.
jenrutner@columbia.edu
Special thanks to: Amanda Bielskas, Rob Cartolano, Kathleen Dreyer,
Mary Giunta, Jean LaPonce, Bob Scott, Jane Winland