E-Use Measurement: A Detour Around the Publishers
ALCTS | Summer 2005
Joe Zucca
Assessment, Planning and Publications Librarian
University of Pennsylvania Library
ALCTS Summer 2005
Student Use of E-Resources: Three Profiles by Residence
4.0
Harnwell N=757
3.5
N=345
Harnwell Stouffer
N=757
Kings
Court N=358
Kings Court
N=358
3.0
Stouffer N=345
2.5
2.0
1.5
1.0
0.5
id
ni
gh
1 t
am
2
am
3
am
4
am
5
am
6
am
7
am
8
am
9
a
10 m
a
11 m
12 am
no
on
1
pm
2
pm
3
pm
4
pm
5
pm
6
pm
7
pm
8
pm
9
p
10 m
p
11 m
pm
0.0
M
Avr. Login per Resident
logins per capita, by hour. Feb ‘03-May ’05 [49,857 logins total]
Hour of Day
ALCTS Summer 2005
The Top 15 Titles by Use, Comparative Ranks: 3 Houses and All Users
3 HOUSES as a
GROUP
ALL
DOMAINS
LexisNexis Academic
1
8
Factiva
2
4
Onefile
3
11
MEDLINE
4
1
JSTOR
5
12
ScienceDirect Journals
6
2
Oxford English Dictionary
7
27
PubMed Plus
8
3
MLA International Bibliography
9
25
PsycINFO
10
10
CINAHL/ Nursing
11
17
ABI/Inform - Proquest
12
18
ISI Citation Indexes
13
5
Encyclopaedia Britannica
14
37
Oxford Reference
15
28
TITLES
ALCTS Summer 2005
Key Attributes of Metrics and Measurement Processes
Metrics
Comprehensive [Licensed Stuff, Local, Public]
Consistent [Uniform Across Resources, Replicable]
Context-Sensitive
[Location, Time, Demographic, Cost, Resource Type & Class, Need]
Processes-Data
Granular, Normalized, Integrative, Anonymous
Processes-Architecture
Scaleable, Efficient (high output for low effort), Sustainable, Distributed
ALCTS Summer 2005
Schematic View of the Data Logging Process
1. User initiates a session using
pages spawned by a database
4. Pass to
vendor site
2. Db swaps handle for
authentic url
3. Proxy scan
3.1 Authentication
challenge
5. Db logs event
5.1 Proxy server
logs the event
ERMS
6. Data Farm
ALCTS Summer 2005
Data Farm Project: Events, Data, Plumbing, Information
Service Events
Borrow a Video
Inform
Customers
Access an E-journal
Staff
Catalog a Book
Staff
Visit a Library
Copy or Print
Data Streams
People and
Network Data
Clean | Anonymize | Integrate
Data Farm Environment
ALCTS Summer 2005
Data Farm Oracle Space:
14+ gb, in 75 tables, tracking events and their properties
E-Resource Use
Circulation
Reference Contact
Web Analytics
Acquisitiions Funds
Resolve Resource
Holdings
Resolve People
Tech Processing
Workflow
Resolve Places
Image Collection Use
Digital Library (erms)
Print Library (Voyager)
Building Use
Copier | Printer Use
Staff Census
Reference|Instruction (dynamic)
Metadata
Gate Swipes
Administration
LDAP
ALCTS Summer 2005
So, what do we do with the data?
Provide a distributed and multi-tiered environment for
interacting with raw data and generating quantitative information
Dashboard Reports (static)
Report Builders (dynamic)
Data Bureau (ad hoc reports and fishing expeditions)
Lower barriers between management information and people
with management responsibility (or any other staff)
Help to scale and institutionalize assessment
ALCTS Summer 2005
INTERACTIONS with DATA
Dashboards:
Vital signs at a glance
ALCTS Summer 2005
INTERACTIONS with DATA
Report Builders
ALCTS Summer 2005
Attempts at
ranking for
selection
based on use
and other
factors
ALCTS Summer 2005
A Hypothetical Scheme for Journal Selection Ranking
Qualitative Scores
Quantitative Scores
Use Metric
Rank
Librarian rank
Impact
Title 1
1
2.60
1
3
4
-1.54
1
Title 2
1
3.87
6
4
54
-1.72
2
Title 3
3
22.4
9
9
8
-1.89
3
Title n
Penn Artcls.
Factor
Cost Effc
ALCTS Summer 2005
INTERACTIONS with DATA
Distribution of E-resource Use Across the Libraries FY04
Ad Hoc | Mediated
Logins
Pct.
Cml Pct.
174,584
55.2%
55.2%
Biomedical (public areas)
41,577
13.2%
68.4%
Staff (Van Pelt-Dietrich and Biomedical)
41,356
13.1%
81.5%
Electronic Classroom
14,535
04.6%
86.1%
Fine Arts
13,671
04.3%
90.4%
Lippincott
10,293
03.3%
93.6%
Veterinary
5,708
01.8%
95.4%
Dental
4,304
01.4%
96.8%
Museum
3,598
01.1%
97.9%
Chemistry
1,950
00.6%
98.6%
Math
1,901
00.6%
99.2%
Undergraduate Study Ctr
1,502
00.5%
99.6%
Rare Book and Manuscript
954
00.3%
99.9%
High Density Storage
209
00.1%
100.%
Van Pelt (public areas)
Total Libraries
316,140
100%
ALCTS | Summer 2005
Ad Hoc Interactions with the Database
Items Charged
and and
E-resource
Log-ins
Fall 19-Dec
2004 5 2004
Items Charged
E-Resource
Logins. Sept
100000
COUNT Circ
10000
1000
100
10
Thanksgiving
Fall Break
Log of Logins & Charge Counts
COUNT Login
1
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
Sun
ALCTS | Summer 2005
Relationship Between E-Resource Use and Item Circulation, Fall 2004
72Correlat
observations
ion of E-Resource Use and I t em Circulat ion, Fall 2004
14000
2
R = 0.8924
12000
I tems Circulated
10000
8000
6000
4000
2000
0
0
500
1000
1500
2000
Logins
2500
3000
3500
ALCTS Summer 2005
Practical Applications of Data Farm Resources
Budget defense (slow but steady rates of increase, curtailing the need
to make major cuts in materials)
Print collection management
E-resource selection (driving trade-offs more typically than cuts)
Staffing | Facilities Planning
Developing and testing web tools
Feeding the data-mavens and spreading best practices
ALCTS Summer 2005
Key Attributes of Metrics and Measurement Processes [REPRISED]
Metrics
Comprehensive [Licensed Stuff, Local, Public]
Consistent [Uniform Across Resources, Replicable]
Context-Sensitive
[Location, Time, Demographic, Cost, Resource Type & Class, Need]
Processes-Data
Granular, Normalized, Integrative, Anonymous
Processes-Architecture
Scaleable, Efficient (high output for low effort), Sustainable, Distributed
E-Use Measurement: A Detour Around the Publishers
Joe Zucca
zucca@pobox.upenn.edu
University of Pennsylvania Library
ALCTS | Summer 2005
ScienceDirect Journals: Regression of View Counts (Elsevier) on Penn Login
Counts. Feb-Nov 2003. [1,270 cases]
4
Log of Science Direct View Count
2
R = 0.64
3.5
3
2.5
2
1.5
1
0.5
0
0
0.5
1
1.5
2
2.5
Log of Data Farm Login Count
3
3.5
4
ALCTS | Summer 2005
OCLC Databases: Regression of Vendor Login Counts on Penn Login Counts,
jl2001-je2002. [48 cases]
1800
2
R = 0.9814
1600
OCLC Login Count
1400
1200
1000
800
600
400
200
0
0
500
1000
1500
Penn Login Count [count-use]
2000
ALCTS | Summer 2005
Three E-Use Profiles by Residence: Top Ten Titles Used
Logins per Capita, Feb ‘03-May ‘05
Harnwell
Kings Court
Stouffer
N=757
N=358
N=345
LexisNexis Academic
2.7
1.7
0.6
Factiva
2.2
1.3
0.5
Onefile
1.4
1.4
0.4
JSTOR
1.2
0.7
0.5
MEDLINE (complete file)
1.1
0.8
0.5
Oxford English Dictionary
0.7
0.8
0.4
PubMed Plus
0.8
0.7
0.2
MLA International
Bibliography
0.7
0.8
0.3
PsycINFO
1.0
0.3
0.2
CINAHL
0.7
0.4
0.5
ALCTS | Summer 2005
Three E-Use Profiles by Residence: Top 15 Titles Used (by login count)
Harnwell
Kings Court
Stouffer
Total
Logins
LexisNexis Academic
2011
597
201
2809
Factiva
1672
454
164
2290
Onefile
1381
611
188
2180
MEDLINE
916
297
200
1413
JSTOR
887
261
158
1306
ScienceDirect Journals
809
226
190
1225
Oxford English Dictionary
565
286
126
977
PubMed Plus
617
250
82
949
MLA Intrn’l Bibliography
511
301
104
916
PsycINFO
731
102
53
886
CINAHL/ Nursing
563
134
171
868
ABI/Inform
689
121
43
853
ISI Citation Indexes
392
204
110
706
Encyclopaedia Britannica
359
235
45
639
Oxford Reference
354
215
64
633
TITLES