Making Library Assessment Work:
Practical Approaches for Developing and
Sustaining Effective Assessment
Phase I Update
ARL VISITING PROGRAM OFFICERS 2004-06
Steve Hiller
Library Assessment Coordinator, University of Washington
Jim Self
Director, Management Information Services, University of Virginia
&
Martha Kyrillidou
ARL, Director of Statistics and Measurement Program
ARL New Measures 1999Key Areas
• User Satisfaction/Service Quality
– LibQUAL+™
– Measuring Library Service Quality Lyceum
• Market Penetration
• Ease and Breadth of Access
• Library Impact on Teaching and Learning
– Project SAILS, Learning Outcomes
•
•
•
•
Library Impact on Research
Cost Effectiveness
Library Facilities/Space
E-Metrics
– MINES for libraries
– ARL supplementary statistics
Barriers to Management Information and
Using Data Effectively in Libraries
•
•
•
•
•
•
•
Organizational culture/leadership support
Time/Staff/Resources
Data issues – Too much, compatibility, validity
Establishing priorities
Knowing what to measure and methods to use
Inexperience, perceived lack of skills and expertise
Understanding, presenting and knowing what to do
with the results
Hiller, S. and Self, J. (2004). From Measurement to Management: Using Data
Wisely for Planning and Decision-Making. Library Trends.
Making Library Assessment Work:
Practical Approaches for Developing and
Sustaining Effective Assessment
• 2 year ARL project to assist libraries with moving
assessment efforts forward
• Led by Steve Hiller (UW) and Jim Self (UVa)
• Recognition that libraries understand value of assessment
but have trouble sustaining efforts and using results
• Site visits conducted to evaluate assessment and develop
practical approaches to effective local practices
• Identify common barriers and facilitators to assessment
• 7 libraries participating in Phase I Spring 2005; 12-14
during 2005-06 academic year; final report in 2006
Phase I Participants
•
•
•
•
•
•
•
University of Arizona
Arizona State University
University of Connecticut
University of Illinois at UC
New York University
University of Notre Dame
University of Oregon
February 2005
February 2005
March 2005
May 2005
March 2005
June 2005
May 2005
Phase I Process
• Each library chooses a contact person
• Survey sent to contact person 2 months before visit
– Assessment activities, motivators, organizational structure
– What’s worked/hasn’t, specific areas to address, expectations
• Schedule of meetings established
• Site visit
• Short report completed within 30 days
– Specific recommendations made on moving forward
• Library evaluates process after final report received
Sample Site Visit Schedule
• Meet with University Librarian/Contact person
• Presentation on effective assessment
– 90 minutes to 2 hours with Q&A and discussion
• Group Meetings
–
–
–
–
with management/administrative group
assessment-related group (if formed
different departments
functional areas (e.g. info literacy)
• Wrap-up
ARL Project
Preliminary Observations after Phase I Visits
•
•
•
•
•
•
•
•
•
•
No “cookie-cutter” libraries; diverse organizational cultures
Sustainability seen as resource issue
Management information system seen as crucial underpinning
Integrating data use effectively into library management
Many internal statistics (non-ARL related) kept
Few libraries allocate sufficient resources (staff and funding) for
assessment or management information
Facilities renovation as catalyst to good assessment practices
Good assessment work being done but not published/
disseminated or coordinated
Need for basic/practical training on assessment methods/analysis
Little understanding of what’s happening outside the library in
data warehousing/management
Phase I Feedback from Participants
•
•
•
•
•
One day is too short
More resource materials would be helpful
More examples presented
Follow-up activity would maintain momentum
Establishing an assessment-related “community” of
practitioners would assist ongoing communication
and development
Phase II Changes
• Site visit time increased from 1 day to 1 1/2 days
• Appropriate resource materials provided
• Consulting on a follow-up activity
– Assessment plan implementation
– Specific assessment effort
• Individual meeting at a professional conference
• Group meeting at a professional conference
• Stronger project Web presence
Phase II Participants
• Confirmed
–
–
–
–
–
–
–
–
–
Cornell
Emory
Kansas
Louisville
Pennsylvania State
Southern California
Texas Tech
Wayne State
Western Ontario
• Willing to commit (not confirmed)
–
–
–
–
Boston Public Library
Colorado State
Nebraska
Purdue
• Interest pending (maybe)
– Alabama
– Alberta
– Marquette
Future Reporting
• Upcoming Presentations
– Northumbria International Conference on Performance
Measurement, Durham, England, August 22-25
– 3rd International Evidence-Based Librarianship Conference,
Brisbane, Australia, October 16-19
– ARL Membership Meeting, Washington, D.C., October 25-28
or in May 2006
• Making Library Assessment Work Web Site
– http://www.arl.org/stats/newmeas/VPO_Hiller_Self.html
• Final Report late summer 2006
• ARL Newsletter
• Conference in Fall 2006?