What is the impact of the publications
read by the different Mendeley users?
Could they help to identify alternative
types of impact?
Zohreh Zahedi, Rodrigo Costas & Paul Wouters
Centre for Science and Technology Studies (CWTS)
ALM Workshop, San Francisco, CA, USA
October 10-12, 2013
Outline
• Introduction
• Objectives
• Research Questions
• Methodology
• Findings
• Conclusions, Discussions & Limitations
1
Introduction
Altmetrics Tools:
Mendeley , Impact Story , altmetric.com,
PLOS ALM, F1000, Plum Analytics, ….
2
Previous research
Altmetrics & Citation Correlation:
Henning (2010);Priem, Piwowar & Hemminger (2012);
Bar-Ilan et. al. (2012a & 2012b); Li, Thelwall & Giustini
(2012); Li & Thelwall (2012); Zahedi, Costas &
Wouters (2013); Mohammadi & Thelwall (2013);
Schlögl et. al. (2013); Haustein et.al. (2013a & 2013b)
Altmetrics & Citation as predictors:
Wardle, 2010
Eysenbach, 2011
Waltman & Costas, 2013
3
Objectives & Research Questions:
To distinguish patterns in terms of impact depending
on the types of Mendeley users
Q1. What do the different Mendeley users read in
terms of document types and Subject fields?
Q2. To what extent do the readerships of the
different users in Mendeley correlate with citation
indicators?
Q3. What is the impact of publications read by
different users in Mendeley?
4
Methodology (1)
Random Samples:
1. 20,000 WOS publications from all disciplines
between 2005-2011
2. 200,000 WOS publications from all disciplines
between 2011-2012
Metrics: Mendeley & Impact Story APIs
5
Methodology (2)
Collecting altmetrics on the basis of DOIs of the publications
Using Mendeley & Impact Story APIs
Linking and matching with WOS
Adding bibliometric indicators
Analyzing the data
6
Types of Mendeley users
• Professors (Associate, Assistant)
• Lecturers (Senior)
• Postdocs
• Researchers (Academics/non-Academics)
• Students (Bachelor, Master, Postgraduate)
• PhD/Doctoral
• Librarian
• Other professionals
• Unknown
7
Distribution of readerships in the
samples by types of users
Sample 1
3% 1% 0.3%
Unknown
5%
Students
7%
33%
10%
Sample 2
PhD
PostDocs
Researchers
4%
3% 0.9%
0.4%
7%
34%
10%
Professors
13%
28%
Other
Professionals
Lecturer
17%
25%
librarians
8
Modeling impact by Mendeley users:
 Scientific: Professors, PhD, Postdocs, Academic
Researchers
 Educational: Lecturers, Bachelor, Master &
Postgraduate Students
 Professional: Librarians, Other Professionals,
non Academic Researchers
 Unknown: unidentified users
Sample 1
Sample 2
SCIENTIFIC
READERS
28%
EDUCATIONAL
READERS
53%
5%
14%
PROFESSIONAL
READERS
Unknown
25%
5%
52%
18%
9
What document type are more read by the
different users? (sample 1)
40%
PhD
35%
Unknown Professions
30%
Students
25%
PostDoc
20%
Researchers
Professors
15%
Other Professionals
10%
Lecturers
5%
Librarians
0%
Articles
Reviews
Non Citables
Letter
10
Which fields are more read by types
of users?(sample 1)
45%
40%
35%
PHD
Unknown
30%
25%
20%
15%
10%
Students
PostDocs
Researchers
5%
0%
Professors
Other
Professionals
Lecturers
librarians
11
Which fields are more cited/read per
publication?(sample 1)
45
40
35
30
25
20
15
10
5
Readers per Paper (RPP)
Citations per Paper (CPP)
0
12
Which fields are more cited/read per
publication? (sample 2)
8
7
6
5
4
3
2
1
Readers per Paper (RPP)
Citations per Paper (CPP)
0
13
To what extent do the different types
of users in Mendeley correlate with
citation indicators?
Correlation
Sample
1
Sample
2
Professors
Other
Professional Librarian
Readers
Unknown
PhDs
PostDocs
Students
Researchers
Lecturers
0,52
0,51
0,46
0,43
0,34
0,15
0,09
0,02
-0,01
-0,01
0,35
0,33
0,29
0,24
0,22
0,1
0,03
0,04
-0,01
-0,01
Citations
Correlation
Sample 1
Sample 2
Readers
Unknown
Scientific
Educational
Professional
0.52
0,51
0,48
0.34
0,05
0,35
0,33
0,30
0.21
0.07
Citations
14
What are the impact of publications
read by different types of readers?
25%
PP Top 10%
20%
15%
10%
Sample 1
sample2
5%
0%
15
Limitations
• Access only to the top 3 categories of readers in
Mendeley
• Data collection (time consuming)
• Speed of use the APIs (API limit)
• Scalability (limitations for the medium-large scale
analysis)
• Not perfect data matching with WOS (DOIs, ….)
16
Conclusions & Discussions
• Potential advantage of Mendeley over citations:
– for publications from social sciences and humanities
– for recent publications [!]
• Scientific users are more correlated with citations
than educational and professional users
• The other users could help to identify other types
of impact: educational, professional [?]
• Some users tend to read more highly cited papers
than others: Postdoc, PhD Students vs Professors
• Identifying the unknown users can shed some light
in detecting these other types of impact
• Further analysis needs to be done to dig into the
content of reading by different types of users
17
Thanks for your attention!
z.zahedi.2@cwts.leidenuniv.nl
18