Implementation & Usage
of bXtm at the
University of Minnesota
Janet Arth
University of Minnesota Twin Cities
&
Sunshine Carter
University of Minnesota Duluth
U of M Duluth (UMD) and U of M Twin Cities (UMTC)
Enrollment, Fall 2011
Enrollment
Duluth
Twin Cities
Undergraduate Students
9,782
30,610
Graduate Students
765
13,562
Professional Students
355
3,625
Nondegree Students
904
4,760
11,806
52,557
Totals
bX
•
•
•
•
About bX
UMTC went live 7/15/2010
UMD went into production 9/20/2010
Both sites use the SFX “simplified menu”
Selected bX Configuration Options
• Show only recommendations with full text or all
recommendations?
• Display 1-5 initial articles in the bX service window
• Display 4-40 articles in the extended
recommendation list
• Articles with full-text can be linked or un-linked
• SFX button can be placed next to article
recommendations
• Full text icons can be displayed or removed
• Minimal recommendation level can range from 0100%. This relates to the percentage of relevancy to
an article
bX @
University of Minnesota Duluth
UMD Configurations
• After 1 ½ months of use, UMD Reference opted to
change four features
– Recommendation display: changed from 3 to 5
– FindIt Logo: changed from the ‘SFX swish’ to our
UMD FindIt logo
– Timeout: changed the bX timeout from 5 to 10
seconds
– Minimal Recommendation Level: changed from 0%
to 40%
bX@UMD
Query 2: Number of SFX requests and
clickthroughs per source
Requests: The number of times users clicked the SFX
button from a particular source (e.g., from with a
databases or from a bX recommendation on the SFX
Menu) resulting in getting the SFX Menu
Clickthroughs: The number of times a user clicks on
one of the options on that SFX Menu (there can be
multiple clickthroughs on a single menu)
Note: In the case of bX, the number of requests is
the real bX clickthrough count (until we activate bX
in Primo or start using APIs in other places).
UMD SFX Trend (Q2)
Query 4: Number of SFX requests and
clickthroughs per service type
• Service types include:
• getFullTxt
• getRecommendation: bX
• getReference: capture citation or push to
RefWorks
• getHoldings: MNCAT/Aleph catalog look-up
• getDocDel: push to ILLiad
• The number of SFX Menu requests that contained a
bX recommendation and number of clicks on the bX
recommendation
bX Usage at UMD -- Overview
AY2010/11 - AY2011/12
Top Sources (requests
and clickthroughs
Top Services (requests
and clickthroughs
•SFX A-Z List
• getFullTxt
•Academic OneFile
• getRecommendation
•PsycINFO
• getHolding
•bX Menu
•Google
• getDocumentDelivery
•Academic Search Premier
*Query 4 in SFX
•Thomson Gale (?)
Top Journals Recommended by bX
•Catalog
• Sex roles
• Science
•PubMed
• Pediatrics
•RefWorks
• Personality and Individual Differences
•Web of Science
• Medicine and science in sports and exercise
•Gale (?)
• Nature
•Primo (journals)
• Journal of personality and social psychology
•Web of Knowledge • Educational leadership
*Query 2 in SFX
•
•
Psychological bulletin
Journal of autism and developmental disorders
*Query 11 in SFX
bX Usage at UMD -- Detailed
AY2010/11 - AY2011/12
• bX recommendations were provided on 72,916 &
71,785 SFX service windows (Q4R)
• End users clicked on at least one recommendation
11,002 & 10,612 times (Q4CT)
• 17,189 & 17,007 bX recommendations were clicked
on by end-users in SFX (Q2R)
• After clicking on a bX recommendation, end users
selected 15,293 & 15,260 services. (Q2CT)
Looking at Interlibrary Loan … Why?
There was an Library Journal - Ex Libris Webinar on the
bX Scholarly Article Recommender Service, with
David Walker (California State University) and Nettie
Lagace (then at Ex Libris). David Walker noted that
some of the CalState sites experienced a notable
increase in ILLs after implementing bX.
Originally broadcast February 17, 2010.
Webinar is still available from this url:
www.exlibrisgroup.com/category/LJ-ELbXWebinar
Document Delivery (ILL) & SFX @ UMD
700
From SFX, patrons
follow through with
their ILL requests
35.7% of the time.
600
500
400
Any SFX Menu clickthrough
to ILLiad (Q4)
ILLiad Requests Created
from Any SFX Menu (ILLiad
reporting)
bx SFX Menu clickthrough to
ILLiad (Q13)
300
200
ILLiad Requests Created
from a bX SFX Menu (ILLiad
reporting)
Linear (Any SFX Menu
clickthrough to ILLiad (Q4))
100
May…
Apr 12
Mar 12
Feb 12
Jan 12
Dec…
Nov…
Oct 11
Sep…
Aug…
Jul 11
Jun 11
May…
Apr 11
Mar 11
Feb 11
Jan 11
Dec…
Nov…
Oct 10
Sep…
Aug…
Jul 10
Jun 10
Apr 10
Mar 10
Feb 10
Jan 10
0
May…
Linear (ILLiad Requests
Created from Any SFX Menu
(ILLiad reporting))
Document Delivery (ILL) & bX @ UMD
Conclusions about bX @ UMD
• Downward trend at UMD for clickthroughs and
requests, in general
• bX is appearing on a lot of service windows, but
recommendations are generally life and health
sciences related
• ILL requests from SFX menus is on the decline
• BUT, more requests to ILL are being made through
bX generated SFX menus
• Higher percentage of follow-through for requests to
ILL via bX generated SFX menus
Why?
• Are users getting better at determining relevancy of
an article?
• Is the decline of ILL requests from SFX due to our
ever increasing access to content. Will the end of
the “big deals” increase these requests.
• Is bX recommending a higher percentage of articles
we don’t have access to? UMD does not subscribe
to most health sciences literature.
• Is bX recommending highly relevant articles so as to
entice patrons to actually request them through ILL
(if we don’t have access)?
• Your thoughts and comments?
bX @
University of Minnesota Twin Cities
U of M Twin Cities: Process
Evaluation Phase
• A subcommittee of the Current Awareness /
Personal Information Management (CAPIM) group
led the initial evaluation phrase, during a 30 day free
trial
• Trial period was May 15-June 15, 2010
• bX was installed on the test server for the trial
– is a bit cumbersome to change database-generated
OpenURLs so the link goes to the test server
– did not want to potentially offer it for only a month to the
public
U of M Twin Cities: Process
• CAPIM recommended purchase of bX
• bX was installed on the production server
• The bX portion on the SFX Menu was customized
based on input from the CAPIM group and the TC’s
SFX Advisory Council
• UMTC went live with bX on July 15 with no fanfare
• Have not yet implemented bX in Primo because of
the lack of configurability (don’t want thumbs
up/thumbs down – SI 16384-221290 )
UMTC – bX on the SFX Menu
http://tc.liblink.umn.edu/sfx_local?sid=Entrez:PubMed&id=p
mid:18212285
Key changes:
• Replaced SFX red graphic with our SFX graphic
• Removed the “document” image that was to
the left of the SFX graphic
• Modified the “Users interested in …” wording
• Added bX information to SFX FAQ
• Moved the bX recommendations down the
menu, but later moved them back up
UMTC SFX Menu (before August 2012)
UMTC SFX Menu (since August 2012)
re
z
Pu
SF
X
bM
ed
G
o
A
o
-Z
gl
SF
Li
M
e
X
st
AR
Sc
C
it
h
C
at
o
It
la
io
EB
re
r
n
SC
co
Li
O
r
nk
W
ds
Ac
eb
er
i
ad
n
o
Al
f
em
Kn
ep
ic
o
h
Se
w
ar
le
dg
ch
e
P
re
O
VI
m
D
ie
r
M
O
E
D
VI
LI
D
M
N
Ps
AR
E
yc
C
SF
IN
It
X
FO
re
b
co
X
M
rd
en
s
Sc
u
in
iF
Pr
in
de
im
r
o
CA
P
EB
Re LUS
SC
fw
O
EI
CI
Co o r
N
ks
EB
m
H
pe
SC
AH
W
nd
O
il
L
M
ey
(
ex
B
N
et
us
O
ur
aL
in
nl
si
ib
es
in
n
:E
s
g)
e
So
BS
Li
br
ur
CO
ar
ce
Ac
y
Pr
El
P
ad
re
im
se
m
em
vi
o
ie
:E
er
ic
r
BS
Sc
Se
CO
a
ie
rc
Ac n c e
h
ad
D
i
em
re
ct
ic
Se
ar
ch
En
t
UMTC Query 2: Top Twenty Sources of Menu Requests
600000
50 0 0 0 0
40 0 0 0 0
30 0 0 0 0
20 0 0 0 0
10 0 0 0 0
0
2010-11 Sc h o o l Yea r
2011-12 Sc h o o l Yea r
UMTC Query 2: Overall SFX Menu Usage
Requests & Clickthroughs
30 0 0 0 0
250 0 0 0
20 0 0 0 0
150 0 0 0
10 0 0 0 0
50 0 0 0
ay
M
Ap
ril
ar
ch
M
ry
ru
a
Fe
b
ua
Ja
n
be
em
ec
20 10 -11 Req u ests
20 10 -11 Clickth ro u g h s
D
ov
ry
r
r
be
em
ob
O
ct
N
Se
pt
em
be
r
er
0
20 11-12 Req u ests
20 11-12 Clickth ro u g h s
UMTC Query 2: bX Menu
Requests & Clickthroughs
160 0 0
140 0 0
120 0 0
10 0 0 0
80 0 0
60 0 0
40 0 0
20 0 0
y
Ma
Ap
ril
rc h
Ma
ry
ru
a
ry
ua
Jan
Fe
b
De
ce
mb
er
er
mb
ve
er
to b
Oc
No
Se
p te
mb
er
0
20 10 -11 Req u ests
20 10 -11 Clickth ro u g h s
20 11-12 Req u ests
20 11-12 Clickth ro u g h s
UMTC Query 4: Options displayed on SFX Menu
250 0 0 0 0
20 0 0 0 0 0
150 0 0 0 0
10 0 0 0 0 0
50 0 0 0 0
20 11-20 12 Re q u e sts
20 11-20 12 Clickth ro u g h s
eN
ag
ge
tM
e ss
t(
ac
st r
t Ab
tD
ge
oF
u ll
Tx
t
ssA
bs
fo r
en
um
mm
co
Di
e li
tD
da
en
t Fu
)
ve
ry
n
t io
l lT
xt
in g
ge
tH
o ld
i ce
eb
tW
ge
t Re
oc
ge
20 10 -20 11 Clickth ro u g h s
ge
20 10 -20 11 Re q u e sts
ge
ge
t Re
fe r
Se
rv
en
To
ta
l:
ce
0
UMTC ILLiad requests from SFX Menu
from Query 13 and ILLiad Statistics
about 60% are actually requested
3500
3000
20 10 -11 g etDo cDel
clicks
2500
20 10 -11 Co m p leted
in ILLiad
2000
20 10 -11 fro m b X
g en erated m en u
20 11-12 g etDo cDel
clicks
1500
20 11-12 Co m p leted
in ILLiad
1000
20 11-12 fro m b X
g en erated m en u
500
y
Ma
Ap
ril
rch
Ma
ry
Fe b
ru a
ry
ua
be
ce m
De
Ja n
r
r
be
vem
No
tob
Oc
Se
p
te m
be
r
er
0
UMTC ILLiad requests from SFX
Query 13: School Year 2011-2012
Month
bX SFX Menu to
ILLiad clickthrough
Any SFX Menu to
ILLiad clickthrough
SFX to ILLiad clickthrough
to ILL completed in ILLiad
September
20
2720
1657
October
15
3136
1871
November
18
2792
1590
December
6
1902
1167
January
3
2398
1511
February
37
3446
2048
March
29
3102
1886
April
22
2783
1810
May
8
2008
1470
158
24287
15010
Year Totals
UMTC Query 11: bX-generated SFX Menu
Most Popular Journals 2011-2012 (requests/clickthroughs)
Note the absence of the “hard” sciences
JAMA (560/509)
New England Journal of Medicine
(513/523)
Journal of Adolescent Health (289/322)
Medicine and science in sports and exercise
(287/284)
Science (450/463)
Journal of Personality and Social
Psychology (410/401)
Nature (279/281)
Pediatric (408/593)
Journal of Advanced Nursing (270/290)
Sex Roles (377/385)
Journal of Marriage and Family
(365/400)
American Psychologist (267/286)
Journal of the American Academy of Child &
Adolescent Psychiatry (260/284)
Child Development (358/425)
The American Journal of Psychiatry (252/252)
Developmental Psychology (275/300)
Social Science & Medicine (320/302) Harvard Business Review (247/230)
Lancet (313/305)
Personality and Individual Differences (245/238)
UMTC: bX-generated SFX Menus
Top Targets 2011-2012 Clickthroughs (mysql query)
bX Recommendations (16,625)
All [28] Wiley Online targets (5637)
EBSCO Academic Search Premier (3895) All [28] EBSCOhost targets (9177)
Elsevier Science Direct Elsevier (3306) All [44] Elsevier Science Direct targets (8666)
Miscellaneous Free (2738)
All [28] JSTOR collections (3630)
Journals at OVID (2562)
All [7] Sage targets (3765)
RefWorks (2347)
All [92] Highwire targets (3154)
DOAJ (1124)
All [20] Springer targets (2420)
Aleph Catalog getHoldings (881)
All [2] APA targets (1947)
Capture Citation (785)
All [2] Taylor & Francis targets (1481)
Cambridge University (405)
All [10] Gale targets (1097)
MDConsult (404)
All [2] PubMed Central targets (586)
Project Muse (392)
All [4] Wilson targets (574)
Absence of Hard Sciences??
Likely due in part to the lack of recommendations
when the OpenURL is DOI driven; as it is for many
sites in the areas of physics, engineering and
computer sciences. See Incident #:16384-216084
and the following example.
Compare
• PMID-driven OpenURL gets recommendations:
http://tc.liblink.umn.edu/sfx_local?sid=Entrez&id=pmid:20161330
• DOI-driven OpenURL for same article has no recommendations:
http://tc.liblink.umn.edu/sfx_local?ctx_enc=info%3Aofi%2Fenc%3A
UTF-8&ctx_id=10_1&ctx_tim=2010-1007T10%3A22%3A2CDT&ctx_ver=Z39.882004&rfr_id=info%3Asid%2Fsfxit.com%3Acitation&rft.genre=artic
le&rft_id=info%3Adoi%2F10.1016%2Fj.foodchem.2009.01.087
PMID-driven OpenURL
DOI-driven OpenURL
bX Menus from other sites
• Kansas State University
http://getit.lib.ksu.edu/sfxlcl3/?sid=Entrez:PubMed
&id=pmid:18212285
Thanks to Jamene Brooks-Kieffer at KSU for the mysql
queries (from her ELUNA presentation in 2011)
• University of Iowa
http://infolink.lib.uiowa.edu/sfx_local/?sid=Entrez:P
ubMed&id=pmid:18212285
Thank you!
• Questions?
• Janet Arth: arth@umn.edu
• Sunshine Carter: scarter@d.umn.edu
A few additional slides
UMTC Query 2: Requests 2010-2011 (Sept-May)
Note: database/source counts are platform specific, e.g., there are also SFX menu
requests that come from ASP via MetaLib & Primo that are not included here, the ASP
count here is just for menu requests coming from the native EBSCO interface
Query 2
Menu Requests
Percent of Total
2,153,530
100.00 %
SFX A-Z List
603,883
28.04 %
Entrez PubMed
363,667
16.89 %
Google Scholar
203,528
9.45 %
SFX Citation Linker
144,945
6.73 %
MARC It records in Aleph
144,397
6.71 %
Web of Knowledge
125,896
5.84 %
EBSCO Academic Search Premier (ASP)
121,602
5.65 %
OVID MEDLINE
107,031
4.97 %
SFX bX Menu
84,936
3.94 %
OVID PsycINFO
73,350
3.41 %
MARC It records in Primo
63,368
2.94 %
SciFinder CAPLUS
61,513
2.86 %
Total
UMTC Query 2: Requests 2011-2012 (Sept-May)
Same note applies to this table
Query 2
Menu Requests
Percent of Total
2,089,070
100.00 %
Entrez PubMed
408,130
19.54 %
SFX A-Z List
354,123
16.95 %
Google Scholar
250,838
12.01 %
SFX Citation Linker
153,189
7.33 %
MARC It records in Aleph
124,921
5.98 %
WoK Web of Science
95,767
4.58 %
EBSCO Academic Search Premier (ASP)
92,773
4.44 %
OVID MEDLINE
83,909
4.02 %
OVID PsycINFO
78,956
3.78 %
74,790
3.58 %
MARC It records in Primo
69,002
3.30 %
SciFinder CAPLUS
46,016
2.20 %
Total
SFX bX Menu (bX not activated in Primo)
UMTC Query 11: bX-generated SFX Menu
Most Popular Journals 2010-2011 (requests/clickthroughs)
Again note the absence of the “hard” sciences
New England Journal of Medicine
Sex Roles
JAMA
Child Development
Journal of Marriage and Family
Journal of Youth and Adolescence
Science
American Psychologist
Medicine and science in sports and
exercise
Journal of the American Academy of
Child & Adolescent Psychiatry
Journal of Advanced Nursing
Pediatrics
Journal of Personality and Social
Psychology
Harvard Business Review
Nature
The American Journal of Psychiatry
Lancet
Journal of Communication
Journal of Adolescent Health
Social Science & Medicine
University of Minnesota Twin Cities
Weisman Art Museum