OA Publishing Community Standards:
Article Level Metrics
A Funder’s perspective
6th Conference on Open Access Scholarly Publishing
September 17th – 19th, 2014
Kevin Dolby
Wellcome Trust
OutlineO

The Wellcome Trust 
How we use Article Level Metrics

Why they are important to us

Current sources of data (and the problems with them)

Initiatives and collaborations changing the landscape
The Wellcome Trust




Set up in 1936 under the will
of Sir Henry Wellcome.
Our vision is to achieve
extraordinary improvements
in human and animal health.
Our mission is to support the
brightest minds in biomedical
research and the medical
humanities.
We spend approximately
£650 million on research per
year.
Current grant portfolio
Monitoring progress: WT’s key indicators
Outcomes
Key indicators of progress
Discoveries
1.
2.
significant advances in the generation of new knowledge
contribute to discoveries with tangible impacts on health
Applications
3.
4.
contribute to the development of enabling technologies,
products and devices
uptake of research into policy and practice
Engagement
5.
6.
enhanced level of informed debate in biomedicine
significant engagement of key audiences & increased reach
Research leaders
7.
8.
develop a cadre of research leaders
evidence of significant career progression among those we
support
Research
environment
9.
10.
key contributions to the creation, development and maintenance
of major research resources
contributions to the growth of centres of excellence
11.
12.
significant impact on science funding & policy developments
significant impact on global research priorities and processes
Influence
Article level metrics vs. Journal level metrics
o Cited 2904 times;
o Normalised
Citation Impact =
327;
o Acta Crystal D
JIF = 7.232
Alternative metrics – beyond citations
Engagement and Influence
MEP
Centre for Bioethics
MEP
Professor of EBM
Journal editor
Health journalist
NGO
Health, Population &
Nutrition @ The World
Bank
Monitoring progress: WT’s key indicators
Outcomes
Key indicators of progress
Discoveries
1.
2.
significant advances in the generation of new knowledge
contribute to discoveries with tangible impacts on health
Applications
3.
4.
contribute to the development of enabling technologies,
products and devices
uptake of research into policy and practice
Engagement
5.
6.
enhanced level of informed debate in biomedicine
significant engagement of key audiences & increased reach
Research leaders
7.
8.
develop a cadre of research leaders
evidence of significant career progression among those we
support
Research
environment
9.
10.
key contributions to the creation, development and maintenance
of major research resources
contributions to the growth of centres of excellence
11.
12.
significant impact on science funding & policy developments
significant impact on global research priorities and processes
Influence
Sources of data

We need article-level data which is consistent in its
source and meaning in order to enable sensible
comparisons between outputs of different schemes

Differences between publisher-provided data mean
that, primarily, we use third-party data providers:
Thomson Reuters Web of Knowledge
 Altmetric

Citation data
Differences in citation data
Differences in citation data
Source
Scopus
Web of Science
Google Scholar
CrossRef
PMC
EuropePMC
Number of citations
76
64
103
56
44
66
For a sample of 358 Wellcome-associated papers, on average:
o WoS had 10 cites per paper
o Scopus 12 cites per paper
o Google Scholar 18 cites per paper
NISO Altmetrics project




Aim to “undertake a two-phase initiative to explore,
identify, and advance standards and/or best practices
related to a new suite of potential metrics in the
community”
First meeting in October 2013
The first phase identified areas for potential
standardization; a consultation period has sought to
prioritise these areas; phase two will advance and
develop these standards.
Final report due in November 2015
Shared sources – the ALM app
CrossRef DOI Event Tracker (DET) Pilot



Pilot group of publishers and potential users of this data
Aim is to test the feasibility of developing and running an
industry-scale infrastructure to track, store and
propagate DOI-related “events”.
These “events” may be come from a wide range of
sources including:





scholarly publications
professional or grey literature
scholarly tools
mainstream media
social media.
CrossRef DOI Event Tracker (DET) Pilot



Looking to establish that it’s possible (and important)
to separate out the infrastructure needed for tracking
common information about the events from the value–
added services (e.g. analysis and visualization)
So there will be the basic data available for all, with
potential “premium” services on top.
More to follow…
Key points

If ALM data is to be useful we have to understand where
it comes from and what it means

Consistency would be good, transparency even more
important

Availability is vital
thank you
k.dolby@wellcome.ac.uk
6th Conference on Open Access Scholarly Publishing
September 17th – 19th, 2014