Measuring your research impact
03 September, 2014
DIVISION OF LIBRARY SERVICES
Establishing a research impact strategy can help you gather the
evidence required to assess the impact of your research on people
and organisations. Verifying that your research is reaching the
individuals, communities and policy makers that will benefit from it.
DIVISION OF LIBRARY SERVICES
Your research impact strategy might include:
 increasing the visibility of your research via social media and
communication with your peers,
 capturing traditional and alternative [altmetric] impact data, and
 techniques for effective communication about your research impact in a
resumé or grant application.
This session will explore and address issues such as:

What does journal impact mean within a discipline?

What are the implications of research impact when crafting your ROPE
for grant applications?

How can we best present the data on research impact? (the 10 best
publications and the metrics that go with these within the ROPE)

What are some alternative metrics that can be used to measure
research impact?
DIVISION OF LIBRARY SERVICES
Journal impact factor (JIF)
‘The journal Impact Factor is the average number of times articles from the
journal published in the past two years have been cited in the JCR year.
The Impact Factor is calculated by dividing the number of citations in the JCR
year by the total number of articles published in the two previous years. An
Impact Factor of 1.0 means that, on average, the articles published one or two
year ago have been cited one time. An Impact Factor of 2.5 means that, on
average, the articles published one or two year ago have been cited two and a
half times. The citing works may be articles published in the same journal.
However, most citing works are from different journals, proceedings, or books
indexed by Web of Science.’
Thomson Reuters. (2012, May 22). Journal citation reports: Journal impact factor.
Retrieved from http://admin-apps.webofknowledge.com/JCR/help/h_impfact.htm impact_factor
DIVISION OF LIBRARY SERVICES
DIVISION OF LIBRARY SERVICES
The Journal Impact Factor (JIF) is used to measure and compare the influence of
journals. Impact factors can not be used to compare individual articles,
researchers or research programs. A high journal impact factor may indicate that
the journal is well regarded.
JIFs are published in the Journal Citation Reports (JCR) database. Impact factors
are only available for journals indexed in the Web of Science database*.
* WoS indexes about 12,000+ journals. Discipline coverage varies, with an
emphasis in the sciences – areas such as the humanities, social science,
education, business, information technology, law etc. are not covered
comprehensively.
DIVISION OF LIBRARY SERVICES
 The JIF is not a measure of the quality of individual articles.
 Only research articles, technical notes and reviews are included in the
calculation of impact factors.
 Only a small percentage of articles are highly cited and they are generally
found in a small subset of journals.
 Controversial papers not necessarily based on good research, may be
highly cited, distorting the impact factor of a journal.
 Journal impact factors may be influenced by self-citation; publication type,
i.e. review articles often attract more citations; publication language, i.e.
English vs non-English.
The ranking of a journal by impact factor within a discipline is more important
than the absolute value, i.e. Biological Conservation JIF 4.036 ranked 20/215 or
Q1 Subject category: Environmental Sciences.
There is a growing acceptance of the value article level metrics, but journal
based metrics continue to be used in many areas.
DIVISION OF LIBRARY SERVICES
Other journal indicators:
 SCImago Journal Rank (SJR)
 Source Normalised Impact per Paper (SNIP)
 Australian Business Deans Council (ABDC) Journal Quality List
Note: The use of ERA lists with journals assigned A*, A, B, C, D
ranks has been discontinued.
DIVISION OF LIBRARY SERVICES
Research Opportunity and Performance Evidence (ROPE)
ROPE is an Australian Research Council (ARC) term, but the concept can
applied in other contexts. In his 2014 CSU Research Office presentation about
preparing your ROPE for nationally competitive grants, Alan Johnson noted that
the NHMRC requirements are now similar to the ARC format.
Your 10 best research outputs are one of the main indicators that will be looked
at by assessors.
Have about 30 words explaining the impact of the publication, yours or third
party opinion, e.g. editors pick.
DIVISION OF LIBRARY SERVICES







Matrix of importance
Author order, you need to be early in the list of authors
Journal/ book importance how recent, relevant
Publication metrics proof
Regency
Relevance to discipline area
Availability
Use journal citation reports from WoS, Scopus or Google Scholar as
appropriate.
Citations and reader ratings relate to your research output, and therefore
more important than JIF. JIF should also be presented as a rank within a
subject category, i.e. 25/215 or Q1 [first quartile].
DIVISION OF LIBRARY SERVICES
Capturing research impact data for use in your ROPE
Thomson Reuters: ResearcherID
Researcher ID is a unique identifier for authors created by Thomson & Reuters owners of Web of
Science ISI database.
When you register for Researcher ID, you will be assigned an individual number which will stay with
you throughout your career, allowing you to easily track your publications and citation counts.
You can identify your publications in Web of Science and add them to ResearcherID, records can
also be imported from EndNote Web or using an RSI file. When publications are imported to your
Researcher ID profile, they will be automatically matched against against existing Web of Science
data and if available, any citation counts will be displayed with the record.
You can directly link your Researcher ID with your ORCID ID.
Find out more about adding records to your Researcher ID
Example: http://www.researcherid.com/rid/I-8088-2014
DIVISION OF LIBRARY SERVICES
Scopus: Author Id
The Scopus Author ID is generated when an author is added to the Scopus
database. It allows Scopus to group together all of the articles by that author. This
is especially useful for authors with common names, eg., Smith, Mary or Chan, Lee.
Many researchers will have a Scopus Author ID without even knowing about it.
It is a good idea to ensure that Scopus has linked your publications to the correct
Author ID, you can request changes directly to Scopus online.
You can directly link your Scopus Author ID with your ORCID.
Learn more about Scopus Author ID.
Example: http://www.scopus.com/authid/detail.url?authorId=7005764153
DIVISION OF LIBRARY SERVICES
Google Scholar
A Google Scholar profile is created by the author and allows you to establish a
profile that lists your publications and citations, and calculates your h-index and i10index. There may be some inflation in the citation figures due to duplication and
author misidentification.
You will need a Google account to create a Google Scholar Profile. Profiles can be
made private or public and therefore discoverable by other researchers and
prospective employers. Google Scholar automatically generates a persistent link for
public profiles, that can be used by authors to promote their research.
Learn more about creating a Google Scholar Profile
Example:
http://scholar.google.com.au/citations?user=rnk7T7wAAAAJ&hl=en&oi=ao
DIVISION OF LIBRARY SERVICES
ORCID
ORCID provides a persistent digital identity that allows researchers and authors to
identify their work from those of others with the same name. ORCID registration is
free and available online. Information from ORCID can be intergrated with Author ID
(Scopus) and Researcher ID (Web of Science) to provide a single aggregated enrty
point that lists all your research output.
ORCID does not provide citation metrics, but it can be used when you submit
publications, or apply for grants.
Discover more about ORCID.
Read more about why you should claim your ORCID
Examples:
http://orcid.org/0000-0001-7842-3722
http://orcid.org/0000-0002-7279-7851
+ alternative metrics …….
DIVISION OF LIBRARY SERVICES
ORCID
DIVISION OF LIBRARY SERVICES
DIVISION OF LIBRARY SERVICES
Presenting research impact data
From: Australian Research Council (ARC) application.
76. *McLeod, S. & McCormack, J. (2007). Application of the ICF and ICF-CY to children
with speech impairment. Seminars in Speech and Language, 28(4), 254-264. [SCOPUS
citations=14]
77. *McLeod, S., & McKinnon, D. H. (2007). The prevalence of communication disorders
compared with other learning needs in 14,500 primary and secondary school students.
International Journal of Language and Communication Disorders, 42(S1), 37-59.
[IF=1.433; 21/64 Rehab; ISI citations=15; SCOPUS citations=17]
78. *McKinnon, D. H. & McLeod, S. & Reilly, S. (2007). The prevalence of stuttering,
voice and speech sound disorders in primary school students in Australia. Language,
Speech, and Hearing Services in Schools, 38(1), 5-15. [IF=1.241; 26/64 Rehab; ISI
citations=18; SCOPUS citations=19]
DIVISION OF LIBRARY SERVICES
Sharynne McLeod
http://www.csu.edu.au/faculty/educat/teached/staff/profiles/professor/mcleod_sharynne
Google scholar Profile: http://scholar.google.com.au/citations?user=_-wFFqoAAAAJ
ORCID ID: http://orcid.org/0000-0002-7279-7851
Scopus Author Id: 7005764153:
http://www.scopus.com/authid/detail.url?authorId=7005764153
ResearcherID is: I-8088-2014:
http://www.researcherid.com/rid/I-8088-2014
h-index (Scopus) 17
h-index (WoS) 13
h-index (Google Scholar) 25
DIVISION OF LIBRARY SERVICES
McKinnon, D. H. & McLeod, S. & Reilly, S. (2007). The prevalence of stuttering, voice
and speech sound disorders in primary school students in Australia. Language, Speech,
and Hearing Services in Schools, 38(1), 5-15.
2013 JIF [Linguistics] 1.247 28/169 or you could use quartiles, making it easier to
read?
2013 JIF [Rehabilitation] 1.247 29/69
Scopus 41 citations
WoS 31 citations
Google Scholar 74 citations
2013 SJR SCImago Journal & Country Rank 1.08 [Language & Linguistics] Q1
51/579
2013 SJR SCImago Journal & Country Rank 1.08 [Speech & Hearing] Q1 10/62
2013 SNIP 1.195
Downloads from institutional repository (CRO) 14
DIVISION OF LIBRARY SERVICES
Crowe, K., & McLeod, S. (2014). A systematic review of cross-linguistic and multilingual speech and
language outcomes for children with hearing loss. International Journal of Bilingual Education and
Bilingualism, 17(3), 287-309.
Almetric data: Scopus
DIVISION OF LIBRARY SERVICES
Alternative metrics (altmetrics)
Alternative metrics or Altmetrics provide article level metrics for research
outputs, i.e. other impacts of a work, such as the number article views, numbers
of downloads, or mentions in social media and news media. Altmetrics can be
applied to a range of research outputs, including - articles, books, presentations,
videos, source code repositories, web pages, datasets etc.
Authors can establish a Researcher Profile with an aggregated altmetric service,
e.g. Impact Story, or refer to individual product specific altmetric providers, e.g.
Scopus or a publisher such as PLoS, in order to monitor and compile alternative
metric data related to their research.
DIVISION OF LIBRARY SERVICES
Altmetric data can supplement traditional scholarly citation counts and measures such as the hindex, and may be used to differentiate the impact of your research output from other works with
comparable traditional metrics. Altmetric data is available from available from:

CRO CSU Research Output (CRO) database records to number of times a copy of a
document has been downloaded.

Academia.edu Researchers can create a collection of their research output, with links to
copyright cleared content, e.g. CSU Research Output (CRO); details of which can be
discovered by other users. Analytics or metrics of items use are available.

Altmetric.com Discover metrics from social media sources, Google Scholar and online
reference managers such as Mendeley, CiteULike and others. Requires an institutional
subscription.
Provides a free Altmetric bookmarklet that can be used to obtain altmetrics on individual
papers [requires DOI, PMID, arXiv.org Article-id].
DIVISION OF LIBRARY SERVICES

Impact Story Can be used to compile an authors research, collecting data ORCID, Google
Scholar or other online profiles; provides details of views, downloads, discussions or
citations, demonstating the authors online impact. This online CV can be included as a link
in a signature blocks or embedded in a website. Requires a personal subscription.

Mendeley Reference manager and academic social network that can help you organise
your research, collaborate with others online, and discover the latest research.
PLoS Open Access journal publisher that provides metrics for PLoS articles, including the
number of times an article has been viewed, cited, saved and discussed.


Plum Analytics Tracks artifacts, including journal articles, books, videos, presentations,
conference proceedings, datasets, source code, cases. Reports impact metrics in 5 major
categories: usage, captures, mentions, social media, and citations. Requires an institutional
subscription.

ResearchGate Researchers can create a collection of their research output; with
links to copyright cleared content, e.g. CSU Research Output (CRO); details of
which can be discovered by other users.
DIVISION OF LIBRARY SERVICES
Aggregated services such as Impact Story, Altmetric.com and Plum Analytics require either a
personal or institutional subscription, but CSU researchers still have access to product specific
altmetric providers, e.g. Scopus, Almetric bookmarklet, or a publisher such as PLoS or
Elsevier, in order to monitor and compile alternative metric data related to their research. Article
download data [Number of Deliveries] is also available from CRO.
i.e.
DIVISION OF LIBRARY SERVICES
h-index attempts to measure both the productivity and impact of the
published work of a scientist or scholar. A scholar with an index of h has
published h papers each of which has been cited in other papers at least h
times [Wikipedia].
i10-index is the number of publications with at least 10 citations. The
second column has the "recent" version of this metric which is the number of
publications that have received at least 10 new citations in the last 5 years.
The h-index provided by Google Scholar is based
on different data to that used by WoS or Scopus.
DIVISION OF LIBRARY SERVICES
Scopus
DIVISION OF LIBRARY SERVICES
Publisher statistics
Elsevier author dashboard
DIVISION OF LIBRARY SERVICES
Impactstory
DIVISION OF LIBRARY SERVICES
Altmetric it!
DIVISION OF LIBRARY SERVICES
Altmetric it!
DIVISION OF LIBRARY SERVICES