Altmetrics - How do I rate thee?
Let me count the tweets!
Mike Taylor
Research Specialist
http://orcid.org/0000-0002-8534-5985
Mike Taylor of Elsevier Labs discusses altmetrics
and how it plays with open access, social
impact, Orcid and why it's got a long journey to
become truly significant
Elsevier Labs
• “Researchers” and “developers”, ie,
experimental architects
• Researchers’ specialities include text-mining,
NLP, semantics, ontologies, etc
Elsevier Labs (and me)
• Research Specialist
• My projects are: altmetrics, contributorship,
networks, identity / Orcid, author profiles
• Not an academic (although…)
• Not entirely commercially focused
In relation to altmetrics:
• Work with and support academic researchers
• Support movement towards integrated model
of reference / citation / mentions inc
bibliometrics
• Support and encourage innovation in this area
in Elsevier
• Publish data and findings
• Develop position as thought leader
Previous credits include:
• ORCID
• Collaborate on technical architecture
• Previously working on similar project with EU
university
• End of year 1 – great success
• www.orcid.org/statistics
• www.orcidlive.org
Why bother with Orcid?
• Disambiguation is a growing problem
• Importance of the personal
• Permanent labels are good things: DOIs,
ISSNs, ISBNs, Orcids
• 430,000 minted
• 102,000 contain ‘works’
The problem
• Disambiguation technologies are reliant on
good meta-data, and are focused towards
western/northern names
• Poor meta-data / Asian names are difficult
• Eg, 3 Korean family names cover >50%
population
• c/w US, several thousand, “Spangler” being
the tipping point
Elsevier infrastructure
• #1 to integrate with Scopus > Orcid (free to
use, no need to have Scopus account)
• Scopus display uses Orcid api
• Editorial submission system
• Integration into metadata hub
• Searchable field on Sciencedirect
Infrastructure / cultural issues
• Brazil, Korea, Denmark = excellent metadata
• China, Italy, India = dreadful metadata
Infrastructure / cultural issues
•
•
•
•
•
•
Brazil, Korea, Denmark = excellent metadata
China, Italy, India = dreadful metadata
Chinese attitude towards relative reward
Research tools are primarily English language
Poverty of infrastructure…
…which impacts on altmetrics
Altmetrics
•
•
•
•
The collection of social network data
Term coined in 2009 by Jason Priem on Twitter
http://altmetrics.org/manifesto/
Ambitions: filtering, hidden impact,
replacement for peer review
Altmetrics: a potentially brilliant
development with a terrible name
• “Alt” to what – not really “metrics” either
• The data in altmetrics is … from whatever is
available.
• Calling it “alt” potentially alienates “metrics”
people
Pragmatic and technocratic
• Eg, Mendeley is included, Zotero isn’t, Colwiz
isn’t.
• Big old pile of data: Twitter, Github, Dryad,
Facebook, blogs, usage data (sometimes),
re-use data (sometimes)
• Is mostly reliant on DOIs (caveats apply)
• Collect what you can, how you can
• (not the best basis for clear activity)
Research! Papers! Start-ups!
• Very exciting: Altmetric.com, Plum Analytic,
Grow Kudos, impactstory.org
• Lots of papers published (though not research
heavy, this is starting to happen)
• Several “special issues”
• Couple of PhDs in progress
• I’ve heard a book might get published
• NISO
Research findings
• Seems to be a correlation between Mendeley
adds and citation rates
• There are definitely patterns of things that
happen together (“impact flavors” – Piwowar,
Priem et al)
• There are definitely differences between
disciplines
• No OA advantage obvious (yet?)
• N is too small
Altmetric.com
•
•
•
•
Owned by Digital Science / Macmillan
The “donut”
Quick demo of Altmetric.com
Appears on all Scopus articles (with
Altmetric.com data) since June 2012
• Trialing on Sciencedirect
• Lots of criticism
Altmetrics data is variable
Which is why I favour small, low-judgment
buckets of data classes (what does it take to…):
• Social activity
• Component re-use
• Scholarly commentary
• Scholarly activity
• Mass media
Nuclear error editorial, Nature
4
3.5
3
log (Soc Act (t) )
Media (t)
2.5
Schol com (t)
Schol act (t)
2
log (Soc Act (d) )
Media (d)
1.5
Schol com (d)
Schol act (d)
1
0.5
0
9/3/09
9/10/09 9/17/09 9/24/09 10/1/09 10/8/09 10/15/09 10/22/09 10/29/09 11/5/09 11/12/09 11/19/09
Supervolcano, Nature Earth
45
40
35
30
log (Soc Act (t) )
Media (t)
25
Schol com (t)
20
Schol act (t)
log (Soc Act (d) )
15
10
5
0
Media (d)
Schol com (d)
Schol act (d)
Right handed vs left handed tail wagging
(Current Biology)
50
40
30
log (Soc Act (t) )
20
Media (t)
Schol com (t)
Schol act (t)
10
log (Soc Act (d) )
Media (d)
0
Schol com (d)
Schol act (d)
-10
-20
-30
Altmetrics – 7 use cases
1. Prediction of ultimate citation, identifying
potentially impactful authors
2. Measuring / recognizing component re-use /
preparatory work, reproducibility
3. Hidden impact (impact without citation)
4. Real-time filtering, real-time evaluation
5. Platform / publisher / institution comparison
6. Measuring social reach, estimating social impact
7. Altmetrics is of interest by itself
Data classes vs use cases
Prediction
Social
activity
Scholarly
activity
x
x
Scholarly
comment
Mass-media Re-use
Re-use
x
Hidden
impact
x
x
Real-time
filtering
x
x
Comparison
x
x
Social reach
x
x
x
x
x
x
Guesswork that needs verification and data!
Research project
• Are the classes internally viable?
• Do they survive disruption by uptake, new
contributors, how do we normalize?
• Are the classes distinct and discrete?
• How (and when and why) do they interact?
• (Questions I hope to address in articles over
next few years and in PhD)
Wider role – in community
• Support open standards
• Support researchers with data (etc)
• Form links between bibliometricians and
altmetricians
• Be a generator of ideas
• Support special editions, workshops etc
Wider role – in Elsevier
• Encourage support for my community work
• Champion open standards for metrics
(everyone is doing this)
• Support product development and outreach
• For example:
Things that we can do
• Real-time suggestions (allied to much
improved search)
• Hidden research
• Social impact statements for researchers
• Re-use indicators support open data
• ‘Evaluation / impact’ network with Fundref,
Orcid, DOIs, etc