Analyzing Usage Statistics of Electronic
Resources
Jagdish Arora
Director, INFLIBNET Centre
Not everything that counts can be measured.
Not everything that can be measured counts.
– Einstein
Why do we measure Usage?
Usage statistics provide essential evidence:
• for extent of usage of e-resources
• to showcase the trends in usage over a period of time,
Patterns of usage can be helpful guide for future collection
development decisions
• to take informed decisions regarding renewal / cancellation
of resources
• to demonstrate value for money / return on investment
Why Collect Usage Statistics?
• To make best and justified use of financial resources
• Calculating Return on Investment (RoI)
• Accountability
• To find out emerging subject disciplines
• Reporting usage to administration, funding agencies, etc.
• Strategic planning
• Comparison with other libraries / institutions
Why Collect Usage Statistics?
• Justifications for change in document format
–
–
–
–
From print to electronics
Lesser number of users visiting library physically
Lesser no. of books being issued / Less re-shelving
Increase in usage of e-resources
• Bench Marking
– Top-cited journals available in e-format as compare to other
libraries
– Results of usage of e-resources by existing libraries can serve as a
bench mark for other libraries
Why Collect Usage Statistics?:
Additional Information
• Journals that are used heavily
• Journals not being used at all
• Number of denials in case of limit of simultaneous usage
• Preference for formats: PDF, HTML
• Breach of license agreement: Heady downloads or
systematic downloads; How to handle it?
Acquiring Usage Statistics
• Content Provider (Publishers / Database Vendors)
– Some publisher do not provide usage data (eg.
MathSciNet, ISID, etc.)
– Data inadequate and inconsistent
– Data retained on the publisher’s web site only
– Inconsistency in usage not reflected
– Server caching not reflected
What do libraries want from usage data?
• Reliable usage Report in consistent format
• Usage at journals titles level
• Usage by subject area
• Analyse trends over time
• Ready access for reporting
• Evidence of value for money
• Benchmarking (comparative usage)
Adding More Value
• Cost-benefit analysis and RoI
• Impact of usage on research output
• Benchmarking
Why Evaluate at the Consortia Level?
Evaluation is Necessary
• Negotiation for renewal
• Cost / Benefit analysis
Evaluation is Possible
• Relativity
• Comparability
• Generalizibility
Why Evaluate at the Consortia Level ?
Review of current & prospective contracts
• Continuing price escalation not sustainable
• Evaluate prices to consortia and members
• Review contracts with additional criteria
• Promote models for quality not just quantity
• Plan for future
Problems with Manual Collection of Usage Statistics
• The usage statistics has to be gathered manually from different
publishers
• Each publisher has
– Different formats for data and delivery
– Different access methods
– Different availability dates
• Cost has to be calculated separately
• Data needs to be cleaned up and aggregated manually
• It is a labor-intensive and cumbersome process prone to data loss and
errors
Harvesting Usage Statistics using SUSHI
• Automated import of consortia stats
• Consortium can track statistics for each member
• Data can be retrieved across a series of dates, e.g. period
of months
• Member logins are pre-populated
• The library can access all COUNTER compliant usage stats
across their serials holdings
• The library can obtain a top level view of their most and
least-viewed publishers and titles
Negotiate More Effectively
• With COUNTER-compliant costs-per-view in hand,
negotiate with publishers to realize more realistic cost
models
• Uncover previously hidden cost information
• Utilize consortium-wide data to negotiate optimal
terms for the group as a whole
• Obtain a better understanding of our consortium
members’ usage patterns and collection needs
INFLIBNET Usage Portal
Benefits of Portal for Usage
• Usage statistics for every e-journal package for
every member institutions is automatically
collected
• Consortia-wide data readily available to the
whole group for analysis and reporting
• The usage data can be exposed completely or
partially to member institutions / consortium
Administrators
Consortium Usage Analysis
12765817
Increase in Number of Fulltext Downlaoads
14000000
6000000
4000000
2000000
4687776
7479060
8000000
6093511
10000000
3153628
No of Downloads
12000000
0
2007
2008
2009
Year
2010
2011
No of Downloads
14000
900
Thousands
Cost Incurred
Millions
Cost Incurred Vs Downloads
800
12000
10000
No of Downloads
600
8000
500
6000
400
300
4000
200
2000
100
0
0
2011
2010
2009
Year
2008
Expenditure on Subscription in Rs.
700
Cost Incurred vs Cost Recovered (2008 - 2011)
12000
Amount in Rs.
Millions
Cost Recovered
Cost Incurred
10000
8000
6000
4000
2000
0
2011
2010
2009
Year
2008
Average Cost per Download (Year wise)
2.5
2
Cost in USD
1.5
1
2.06
1.81
1.35
0.5
1.19
1.14
2010
2011
0
2007
2008
2009
Cost Incurred vs Cost Recovered in 2011
Increase in Usage of JSTOR for Core Members
2500000
140
Usage
No of Univ
116
2000000
120
105
100
1500000
80
64
64
60
1000000
40
34
500000
20
0
0
2007
2008
2009
2010
2011
Average Cost per Download for JSTOR & Consortium
2.50
J-Stor
Consortium
2.00
1.50
1.00
0.50
0.00
2006
2007
2008
2009
2010
2011
Increase in No of Members for JSTOR 2008 - 2012
250
Associate member
Core member
200
150
100
50
0
2008
2009
2010
2011
2012
Usage Trend Analysis for a Single
Publisher
(ACS)
Year-wise Download for ACS (2007 - 2012)
1600000
1400000
1000000
800000
600000
1196487
1398741
0
1031237
200000
594960
400000
560506
No of Downloads
1200000
2007
2008
2009
2010
2011
Year
Top Ten Journals of ACS in 2011
Sl. No.
Journal Name
Number of
Downloads
1
Journal of the American Chemical Society
194693
2
The Journal of Organic Chemistry
149561
3
Organic Letters
90272
4
Journal of Medicinal Chemistry
84866
5
Inorganic Chemistry
83202
6
Chemical Reviews
65255
7
Langmuir
58367
8
The Journal of Physical Chemistry B
57635
9
The Journal of Physical Chemistry C
56186
10
Journal of Agricultural and Food Chemistry
49706
MEASURING RESEARCH OUTPUT
AND
IMPACT OF E-RESOURCES
Measuring Research Output
The Science Citation Index (SCI), Social Science Citation Index
(SSCI) and Arts and Humanities Index (A&HI) are
internationally recognized database that works as a filtering
mechanism as it indexes qualitative research output from
selected journals.
The source articles appeared in three indices for 50 firstphase universities of the Consortium was searched in blocks
of five years from 1975 to 20010 with an aim to compare the
research output in the last block year, i.e. 2005 – 2009.
A un-precedental increase in research productivity in terms of
number of research articles is evident during 2005-2009 as
compared to previous block of five years, i.e. 1975-1979 to
2000-2004.
Increase in no. of articles in past 35 Years
(In block of Five Years)
45000
40000
Increase in No. of Articles
1975-1979 to 1980-84: 22.97%
35000
2000-2004 to 2005-2009: 76.59%
No. of Articles
30000
25000
AHCI
SSCI
20000
SCI
15000
10000
5000
0
Blocks of Five Years
Correlation Usage Vs. Publishing Output
5000
4500
4000
Number of Publications
3500
3000
2500
2000
1500
1000
500
0
0
500000
1000000
1500000
2000000
Number of Downloads
Pearson’s Ranks order Coefficient Correlation = 0.75
2500000
Usage analysis for A Single
Institution
Does this institution need Complete Collection or Selected Subject
Collections
Subjectwise Analysis of OUP for AMU
10000
70
66
Total Downloads
60
Average Downloads
No of Titles
1000
50
100
26
20
12
8
7
8
106
176
12
87
12
7
14
25
292
576
259
514
18
10
30
10
5
1
1
Biological
Science
Computer
Science &
Mathematics
Economics
Humanities
Law
Subject Category
Library and
Information
Science
0
Medical Science Social Science
No of Titles
7008
40
2571
Downloads
47
1
5
9
13
17
21
25
29
33
37
41
45
49
53
57
61
65
69
73
77
81
85
89
93
97
101
105
109
113
117
121
125
129
133
137
141
145
149
153
157
161
165
169
173
177
181
185
189
193
197
% of Downloads
No. of Titles Fulfilling the User needs of the Library
120
100
80
60
40
20
0
No of Titles
Title with Highest Download in Each Subject
1800
1665
1600
1400
1200
1000
899
800
600
400
204
200
83
20
34
8
Law
Library and
Information
Science
66
0
Biological Science Computer Science
& Mathematics
Economics
Humanities
Medical Science
Social Science
Correlation Usage Vs. Publishing Output
Banaras Hind University
1000
900
y = 0.0017x + 292.81
R² = 0.9596
Number of Publications
800
700
600
500
400
300
200
100
0
0
50000
100000
150000
200000
250000
300000
350000
Number of Downloads
Pearson’s Ranks order Coefficient Correlation = 0.98
400000
Banaras Hindu University
Annual Average Growth Rate
Engg & Technol
5.00%
4.00%
A&H
3.00%
Agr Sci
2.00%
1.00%
0.00%
Social Sci
Med & H Sci
-1.00%
-2.00%
Sc Mis & SI
Biolog Sci
Phy Sci
Chem Sci
Banaras Hindu University
Relative Specialization Index
Engg & Technol
-0.9975
A&H
-0.9980
Agr Sci
-0.9985
-0.9990
Social Sci
-0.9995
Med & H Sci
-1.0000
Sc Mis & SI
Biolog Sci
Phy Sci
Chem Sci
Banaras Hindu University
100000000
6000
10000000
5000
1000000
4000
100000
10000
3000
1000
2000
100
1000
10
1
0
World
India
BHU
Contribution of BHU as Compared to the World and India’s Total
Publications
Engg & Technol
A&H
Agr Sci
3557
1829
Social Sci
5286
57
Med & H Sci
BHU
416
958
1278
Sc Mis & SI
Biolog Sci
4422
Phy Sci
3337
Chem Sci
Publications Output of BHU
14.00
12.00
Engg & Technol
Real Average Citation
10.00
Agr Sci
Med & H Sci
8.00
Biolog Sci
Chem Sci
6.00
Phy Sci
4.00
Sc Mis & SI
Social Sci
2.00
-20.00%
A&H
0.00
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
Cited Rate
BHU’s Citation Impact in Nine Subject Areas
0
200
400
600
800
Gastroenterology Hepatology
Public Environmental Occupational Health
Agriculture
Business Economics
Cell Biology
Computer Science
Surgery
Environmental Sciences Ecology
Oncology
Pharmacology Pharmacy
Biochemistry Molecular Biology
Engineering
Chemistry
1
India
World
10
100
1000
10000
100000 1000000
Banaras Hindu University
Contribution of BHU to the World’s Most Productive Areas of
Research