Going beyond citations –
SERUM: a new tool provided by
a network of libraries
Juan Gorraiz
Christian Gumpenberger
LIBER 2010 Aarhus, 30.06.2010
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
1
Schedule:
•
introduction to citation & usage metrics
•
SERUM – necessity, scope & requirements
•
context to other initiatives
•
suggested approach
•
conclusions
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
2
Some history
1927: Gross and Gross introduced total number of citations
excluding journal self-citations / identifying the most
relevant journals in a research area; instrument for
librarians improving acquisition practices
1972: Garfield published his article “Citation analysis as a tool in
journal evaluation” in Science
1975: Garfield launched the 1st ed. of JCR (Journal Citation
Reports) – initially called “a bibliometric analysis of
science journals”
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
3
JCR‘s first citation metrics
•
Total cites: as measure for the citation quantity
•
Impact Factor: as measure for the average citation frequency
•
Immediacy Index: as measure for the citation speed
•
Cited/Citing half-life: as measure for the ageing characteristics
of a subject field
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
4
Analytical citation tools
JCR (JIF), SCImago (SJR), CWTS (SNIP)
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
5
The target groups of journal citation metrics
•
Librarians: collection management
•
Publishers: monitor competitors, management of current
publications; identify new publishing opportunities
•
Editors: assessment of policies and objectives
•
Authors: identify relevant journals to read and to publish in
•
Science Policy Makers: track bibliometric trends, analyse
citation patterns within or between disciplines
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
6
The shortcomings of citations
•
highly controversial: different reasons why to cite whom
•
only indirect measure of impact: not for quality!
•
always delayed: significance varies according to discipline
•
insufficient: in some fields; surprisingly low mean citation rates
for vast majority of published scientific literature
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
7
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
8
Usage metrics „reloaded“
advent of the internet
growing availability of e-journals
rapid change in user preference
usage metrics
• alternative to citation metrics
• allow viewing scholarly communication from the user‘s
perspective
SERUM aims to provide access to global usage metrics on the
basis of download data.
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
9
The advantages of downloads
•
generally accepted as proxy for usage
•
direct measure; can be recorded automatically
•
can be accessed without delay
•
more suitable than citations for less publication-intensive
fields
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
10
The necessity of SERUM
•
initiative in response to the lack of globally available and
“usable” usage metrics for e-journals
•
usage metrics would definitely add new dimension to existent
quality criteria of scientific journals
•
downloads are not only relevant and necessary at article level,
as journals need to be analysed and evaluated as entities
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
11
Major aspects of SERUM
•
•
•
•
•
providing access to consolidated global usage data
of e-journals
implementing an international network of libraries that relates
locally tracked usage trends to global usage statistics provided
by publishers (authenticity check)
introducing new aspects in the evaluation of journals for all
mentioned target groups (librarians, publishers, editors, authors,
science policy makers) by offering new usage metrics and
journal rankings
separating usage from citations
focusing on simplicity and usability
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
12
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
13
Required data from publishers (1)
•
•
•
•
total number of downloadable items per journal
number of downloadable items disaggregated (by document
types) per journal
total download counts (full-text article requests – FTAs) for the
current year (independent from the publication years of the
journal)
download counts (full-text article requests – FTAs) for the
current year with explicit listing of downloads per publication
year of the journal for the last 5 years
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
14
Required data from publishers (2)
•
•
•
download counts at journal level disaggregated by following
document types:
Articles, Review Articles, Proceedings Papers, Letters, Notes
percentage of “non-downloaded” items
distinction of downloads according to their origin (grouping by
identical IP addresses without disclosing them)
total view and
geographical grouping (according to geographical distribution of
SERUM library network)
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
15
Tasks of SERUM library network
•
•
•
•
•
•
•
implementation of the selection procedure for considered
journal titles
categorization of journals
management and regular update of global usage data either
harvested from or delivered by publishers
introduction and promotion of usable metrics
provision of journal rankings
tracking of local usage data
authenticity check of global usage data based on local trends
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
16
The metrics in SERUM (1)
•
•
•
•
•
•
only downloads (full-text article requests – FTAs) considered
operation at journal level; designed as an instrument to depict the
usage characteristics of a journal as an entity
4 basic indicators are: DT (total number of downloads), JUF (journal
usage factor), DI (download immediacy) and DHL (download half-life)
disaggregation on the basis of document type (Articles, Review Articles,
Proceedings Papers, Letters, Notes and others)
calculation of DT,
JUF, DI & DHL for each document type
SERUM’s usage metrics will work at synchronic as well as at diachronic
level
all indicators calculated for the current and the previous two years as
weighted indicators on the basis of unique URLs, (the origin of
downloads) in order to detect data manipulation.
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
17
The metrics in SERUM (2)
•
•
•
•
•
•
time lines and graphs provided for the most relevant indicators
analysis of increase/decrease rates
according to the skewness distribution hypothesis of downloads at
article level in a given journal
interest in other distribution
parameters, like the percentage of “non-downloads”, quantiles or
quartiles
SERUM also aims to disaggregate on the basis of treatment or content
(e.g., theoretical, methodological and experimental) as well as in regard
to the merits of the downloads (academic vs. non-academic)
once operational for a while
corrections of metrics introduced to
reflect usage practices and traditions in the different fields and
disciplines
after establishing metrics for e-journals
corresponding metrics for
e-books as the next logical step
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
18
SERUM in the context of other initiatives
•
•
•
•
•
studies on correlation between citation and usage data
[e.g. Bollen et al., 2005; Moed, 2005; Brody et al., 2006;
Wan et al., 2008]
studies on usage metrics as emerging field in bibliometrics
[e.g. Duy & Vaughan, 2006, Bollen & Van de Sompel, 2008
and Schloegl & Gorraiz, 2009]
comparisons of locally collected usage data to vendor-provided
statistics [Duy & Vaughan, 2003 and Coombs, 2005]
Project COUNTER and SUSHI
MESUR
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
19
Suggested approach – SERUM pilot
•
•
•
win publishers
address OA publishers first, share
requirements and discuss feasibility
promote idea of SERUM amongst scientific libraries
win at least 2 further partners apart from Vienna University
Library for pilot
work out a more sophisticated requirements specification and
project plan with pilot partners
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
20
Conclusions
•
•
•
citations only show part of the whole picture
insufficient especially for the Humanities and Social Sciences
downloads offer a more feasible alternative
nevertheless
usable download metrics and services for a broad audience
are not yet available (MESUR has a different scope).
SERUM aims to provide these metrics and services with a
focus on simplicity, usability, comprehensibility and
fast & central availability
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
21
A blue sky project with high-flying ideas
http://www.flickr.com/photos/queenroly/129218137/
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
22
Thank you very much
for your attention!
Your questions?
Juan Gorraiz – Christian Gumpenberger
University of Vienna - Library and Archive Services
Bibliometrics Department
bibliometrie@univie.ac.at
http://bibliometrie.univie.ac.at
This work is licensed under
http://creativecommons.org/licenses/by-nc-sa/3.0/
23