Usage Data
Practical evaluation
Katarina Standár
LM Information Delivery
Boatshow May 2013
FACT 1. Challenges for libraries increase every year
•
•
•
•
•
•
Less resources in general (at least not more)
Budget cut downs -> value for money
Retired staff positions not filled
More workload for everybody
Constant reorganization to find ways of working efficiently
More and more electronic media taking over from print
-> challenge for librarians to convince funders what actually costs in
the library
• More complicated system infrastructure
-> more difficult to know the content and data in different system
Knowledgebases, A-Z-lists, Databases etc
-> more specialised staff needed to work in the new environments
Reality for library users
COMPUTER
IPHONE
IPAD
W
W
W
Fulltext
articles
LIBRARY??
Reality for librarians
FACT 2. More and more numbers
• Management want more and more numbers and quantity
metrics from the library to get some proof of the use of
costly e-resources.
• They want decision making based on use and it is very
difficult to compare between systems or the numbers
from different types of e-material, for example e-books.
FACT 3. Transition from P to E still goes on
• Not all libraries have been able to change staff focus from print to
electronic
• If they have changed focus there is the problem of training staff who
are not always up to new work tasks or they think it´s very difficult
• Large libraries have to be in two worlds at the same time – the
printed and the electronic – different needs - that costs!
• Electronic books need to be handled as detailed as printed books –
but are they?
• Electronic material is much more moody and instable than printed
stuff
What can libraries do?
1. Collection of data sources – a package, an
aggregated database, an A-Z-list
2. Decide on a statistic analytics tool or several
models
3. Analyze and interpret the data and compare
with other data
Evaluation
• Libraries can create an Evaluation strategy
• Libraries need to regularly evaluate databases,
journals and e-books. Perhaps with different
methods for each type or a combination
• There are many different evaluation models and
tools
Tools and models
• COUNTER (Counting Online Usage of Networked Electronic
Resources)
Measure downloads of articles, books, chapters
HELP!!
• SUSHI (Standardized Usage Harvesting Initiative)
• TRANSFER code of practice
• KBART - a standardized format to exchange and update product
availability details across the supply chain
Bibliometric analysis tools – Mathematics for
librarians
• ISI Impact factors –measure content relevance, citation
• Eigenfactor score - mapping the structure of academic research
within networks
• PageRank algorithm - a numerical weighting to each element of a
hyperlinked set of documents
• Journal Usage Factor (JUF) - will provide information about the
average use of the items in an online journal
• Altmetrics - new and fast ways of measuring research, outside the
scope of traditional filters. Based on social media. What can libraries
do?
How?
• Reviewing journal packages and products Ongoing
evaluation process in a whole year
Extremely time consuming for the library..
• Using statistics tools on the market or models for
electronic resources
• Asking your users about what they think about the eresources
TERMS – Techniques for ER Management
Projects done
”Analysstöd för statistik över användandet av elektroniska
resurser”
Project in Sweden funded by the Royal library in Sweden
2012:
Analysis tool created
KPI indicators: Templates were created:
E-books, E-journals, Databases
to help find indicators to evaluate e-resources in a more wide
way than just looking at cost per use
Projects done
“Measures of health sciences journal use: a comparison of
vendor, link resolver and local citation statistics”
Journal of the Medical Libraries Association April 2013.
The study was done at University of Illinois at Chicago (UIC).
They use Serials Solutions 360 Link and A-Z-list and MARC
records.
Challenges gathering data
•
•
•
•
Counter reports – difficult to collect, not enough standards, E-books
Impact factor - highly discipline-dependent
Many journals that are used are not cited at all!
Archives – often bought separate from packages and single
subscriptions
• More and more hybrids of the material
• Numbers: Resources that has a low use but are critical to just a few
researchers? How do you evaluate them?
Challenges gathering data
• A library may have access to many different platforms for a single
journal
• Problems with the library´s vendor accounts on the platform
• Problems with the KnowledgeBase setup
• Many different systems in the library with same titles but structured
in different ways
• Subscriptions set up under multiple accounts can result in multiple
reports from the same platform
• Vendor packages change every year: cancelled titles, changed titles,
new titles, transferred titles
More challenges..
• Human errors
Not all e-resources deliver any statistics at all..
Extremely TIME CONSUMING
What can vendors do?
• Deliver standardized data to the systems
• Listen to libraries when developing statistic tools
• Try to understand better what libraries really
need the data for
My experience and thoughts
1. Why is the use of e-resources so much more discussed
than print material ever was? Who question an expensive
reference book on the shelf and the quantity cost per use?
Why not?
2. System Up to date information = higher usage!
3. E-resources Accessibility = higher usage!
My experience and thoughts
4. E-resources evaluation should be a part of the whole
library evaluation, not a separate part.
Why do we buy these resources and systems and how can
we prove it helps users?
5. Today there is a difficult landscape how to find accurate
data to evaluate! Who is doing it?
Thoughts or expericences?
• How do you work with evaluation in your library?
• How would you like vendors to help?
Thank you!