Tell your impact story:
Beginners guide to quality & impact metrics
CURTIN UNIVERSITY LIBRARY
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
July 2014
This Session
Understand the when, how and why:
 Research impact and quality
 Four-sector assessment
 Scholarly
 Industry
 Government
 Societal
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Introduction
Evaluation of research quality and impact:
 Quality vs Impact
 Prestige
 Funding
 Promotion
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Scholarly / Journal metrics
Using journal metrics for quantitative
analysis:
 Journal Citation Reports - Web of Science
(Thomson product)
 SJR (SCImago Journal Rank) - Scopus
(Elsevier product)
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
JCR
JCR > Search for a specific journal > Journal title search > (Journal
Rank in Categories) Journal Ranking
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
SJR
SCImago website > Journal rankings> Subject area Business,
Management & Accounting/Accounting >
Journal Ranking
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Scholarly / Author metrics
Using the H- index to determine citation counts
of individual authors, groups of researchers etc:
• The h-index is determined by calculating where h
number of articles have been cited h or more times
• Found in citation databases e.g. Scopus or Web of
Science, Publish or Perish from Google Scholar
• Influencing factors/cautions
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
H-index Web of Science
H-index - Web of Science
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Scholarly / Article metrics
Average numbers of citations per discipline
Essential Science Indicators > Baselines > (By
Averages) View the average citation rates table
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Scholarly / Qualitative measures
Using qualitative measures to determine
impact:
• Who is citing you
• Measures of esteem
– Editor of a prestigious work of reference
– Fellowship of a learned academic society
– Recipient of an Australia Council grant
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Industry & Government
Demonstrating return on investment:
 Measures of impact beyond academic
 Locate evidence
 How research is being applied
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Industry & Government
Impact narrative:
 An effect on, change or benefit to
 Developing a framework
 Case study approach
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Industry & Government
Socio-economic Objectives (SEO):
http://www.abs.gov.au/Ausstats/abs@.nsf/Latestproducts/CF7ADB06FA2DFD69CA2574180004CB82?opendocument
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Industry & Government
Impact narrative:
 EIA case-study approach
 Link between research and impact
 Validation of impact
 Other examples
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Industry & Government
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Industry & Government
http://www.csiro.au/Portals/About-CSIRO/What-we-do/Impact-case-studies/BodyArmour.aspx
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Societal
Sources for detecting potential influence
Exploring the boundaries: How altmetrics can expand our vision of scholarly community and social impact
Mike Taylor
Information Standards Quarterly Summer 2013 Vol. 35 Iss. 2
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Societal
Measures of impact
 Article-level metrics & Altmetrics
Altmetrics in Evolution: Defining and redfiing the
ontology of article-level metrics
Jennifer Lin & Martin Fenner
Information Standards Quarterly Summer 2013 Vol. 35 Iss. 2
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Societal
Measures of impact
 Article-level metrics & Altmetrics
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Societal
Impact narrative:
 Altmetrics hinges on the very prevalence of its own
diversity. Its raison d’etre is to provide a more
expansive view of a research artefact's impact
Lin and Fenner (2013)
 Levels of engagement
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Societal
Levels of engagement:
Lin and Fenner (2013)
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Societal
Levels of engagement:
Lin and Fenner (2013)
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Societal
Controversies or problems
SCIENTISTS DON’T COUNT: WHY YOU SHOULD IGNORE
ALTMETRICS AND OTHER BIBLIOMETRIC NIGHTMARES
D Colquhoun & A Plested http://www.dcscience.net/?p=6369
 Impact vs popularity
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J
Societal
Future of altmetrics
Clearly now is the time to capitalise on the
interest and attention to finally bring assessment
of research out of the systems belonging to the
print era and into a more modern, multifaceted
system that takes advantage of the flexibility and
scale of the web
Social signals reflect academic impact: What it means when a scholar adds a paper to Mendeley
William Gunn Information Standards Quarterly Summer 2013 Vol. 35 Iss. 2
Curtin University is a trademark of Curtin University of Technology
CRICOS Provider code 00301J