Introduction to
Citation Metrics
Elizabeth Lee San Bao | elizabeth.lee@nus.edu.sg
5 March 2015
Session Outline
 What are Citation Metrics?
 Why do we use Citation Metrics?
 Types of Citation Metrics
 Limitations of Citation Metrics
 Managing your Research Publication History
What are Citation
Metrics?
What are Citation Metrics?
 Metrics used to count, measure, rank, analyse and compare a journal / article /
author / institution / organization’s research impact, based on their scholarly
published articles and their citations
 One of the most widely used methods in the field of bibliometrics – the
statistical analysis of written publications such as books and articles.
 Citation metrics base their data from citation indexes, which are produced by
different publishers, e.g. Thomson Reuters, Elsevier
 We will cover the following citation metrics today:
1. Journal Metrics
2. Author Metrics
Explaining Citation Count (Citation Analysis)
Paper A’s Cited References
Paper A’s Citing Articles
2005
1998
Paper A
2012
2013
2010
2007
2003
2013
Number of citing articles for Paper A = Citation Count for Paper A
“X papers have cited my Paper A as reference.”
Why do we use Citation
Metrics?
Why do we use Citation Metrics?
For researchers / institutions:
 Used to track impact of a researcher’s scholarly publications
 Record achievement for the purpose of academic promotion and tenure, research funding or
job applications
 Used as a way to compare quality of scholarly work with other researchers / institutions
 Used by University Faculty / Department / Research Institutes for reporting, benchmarking
and comparison of staff  annual staff performance assessment
 Build a research profile so others can find and follow one's work.
Assumption: Researchers with high citation counts are writing articles that describe
important and influential research
Why do we use Citation Metrics?
For graduate students:
 To be aware of which are the leading journals in specific subject disciplines
and to read up on them
 To be alert of hot research topics in their subject disciplines
 To examine which top peer-reviewed scholarly articles to read
 To trace historical development of certain research topics and prominent
scientists associated with them
 To identify any research gaps that may lead to potential dissertation topics
Types of Citation
Metrics
a) Journal Metrics
b) Author Metrics
Journal Metric 1: Journal Impact Factor
 Used in evaluating or comparing a journal's relative importance to others in the same
field.
 Represents the frequency with which an average article from a journal is cited in a
particular year.
 Calculated based on data from the Journal Citation Reports (JCR) by Thomson Reuters
– a quantitative tool for ranking, evaluating, categorizing and comparing journals.
* For more information, you can also visit the NUS LibGuide
on NUS Cited Reference Searching.
Journal Metric 1: Journal Impact Factor
 Journal Impact Factor is defined as the average number of times articles
from the journal published in the past two years have been cited in the
JCR year.
 For the journal “Reviews of Modern Physics”, the Journal Impact Factor is
shown below:
* You can refer to our LibFAQ on Journal Impact Factor here.
Journal Metric 1: Journal Impact Factor
Activity: Journal Impact Factor
Launch Journal Citation Reports here.
1
 Select “View a group of journals by:
Subject category”  Physics, Applied
 View journals by Journal Impact
Factor.
2
 Find the impact factor of the journal,
Acta Crystallographica Section A
 Look for the Impact Factor Trend.
 What do you notice?
 View journals from other subjects:
a) Mathematics
b) Engineering
c) Chemistry
 What do you notice?
Source: What does it mean to be #2 in Impact?
Journal Metric 2: 5-Year Journal Impact Factor
 Average number of times articles from the journal published in the past five years
have been cited in the JCR year.
 It is calculated by dividing the number of citations in the JCR year by the total
number of articles published in the five previous years.
 This metric can be used to better gauge the impact of journals in fields where the
influence of published research evolves over a longer period of time.
 For the journal “Reviews of Modern Physics”, the 5-Year Journal Impact Factor is
shown:
Journal Metric 3: Journal Immediacy Index
 Average number of times an article is cited in the year it is published.
 Indicates how quickly articles in a journal are cited  form of evaluation
to see which journals publish cutting-edge research.
 For the journal “Reviews of Modern Physics”, the Journal Immediacy
Index is shown:
Other Journal Metrics
Journal Metric
Description
Eigenfactor Score / Article Influence Score
Eigenfactor’s algorithm ranks journals similarly to how Google ranks websites.
It assigns a greater weight to those citations coming from influential journals
and does not count journal self-citations.
The Article Influence Score measures the relative importance of the journal
on a per-article basis. It is calculated by dividing the Eigenfactor score by the
number of articles published in the journal.
SCImago Journal Rank (SJR)
Uses algorithms similar to Google PageRank and is based on the prestige of
the citing journal. With SJR, the subject field, quality and reputation of the
journal has a direct effect on the value of a citation. Normalizes for
differences in citation behaviour between subject fields.
Source Normalised
Impact per Paper (SNIP)
Developed by Leiden University’s Centre for Science and Technology Studies
(CWTS), SNIP measures contextual citation impact by weighting citations
based on the total number of citations in a subject field. If there are fewer
total citations in a research field, then citations are worth more in that field,
and vice versa.
Google Scholar Metrics
Analyses articles published in the last 5 calendar years and its metrics are
based on citations from all articles that are indexed in Google Scholar. Uses
h5-index and h5-median as its journal metrics, where h is the largest number
of articles that have each been cited h times.
Author Metric 1: Number of Publications & Citation Count
 Measuring the total number of publications (usually, journal articles) during
author’s academic career
 Calculating the total number of citations to an author’s published articles (Done by
author search in Web of Science)
 In different subject disciplines, the rate of publishing books and journal articles will
vary.  do these numbers above reflect true quality of the research work done?
What about considering the following? :
 Being positively assessed during the peer-review process
 Measures of prestige such as invitations / honorary membership on editorial
boards, funding grants and industrial awards
Author Metric 1: Citation Count – Author Search in WOS
Example: Professor Andrew Wee
Name Variants:
a) Wee Thye Shen,
Andrew
b) Andrew Wee T S
c) Wee, Andrew Thye
Shen
d) Wee ATS
e) Wee A
f) Wee TSA
* You can view
Prof Wee’s ResearcherID
Profile Page for more
information.
Publication list updated as of 27 Nov 2014
Author Metric 1: Citation Count – Author Search in WOS
Author Metric 2: h-index
 Developed in 2005 by Jorge Eduardo (J.E.) Hirsch in his paper “An index to
quantify an individual's scientific research output”
 He states that:
“A scientist has index h if h of his/her Np papers have at least h citations each, and
the other (Np − h) papers have no more than h citations each.”
 In other words, an author with an index of h has published h papers each of
which has been cited in other papers at least h times.
Author Metric 2: h-index
“Obviously, a single number can never give more than a rough
approximation to an individual's multifaceted profile, and many
other factors should be considered in combination in evaluating an
individual.”
“There is considerable variation in the skewness of citation
distributions even within a given subfield, and for an author with a
Jorge E. Hirsch
relatively low h that has a few seminal papers with extraordinarily
high citation counts, the h index will not fully reflect that scientist's
accomplishments.”
“Conversely, a scientist with a high h achieved mostly through papers
with many coauthors would be treated overly kindly by his or her h.”
Hirsch, J.E. An index to quantify an individual's scientific research output, PNAS , vol. 102 no. 46 (2005)
Limitations of Citation
Metrics
Limitations of Citation Metrics
When conducting analysis with Citation Metrics, always consider the Citation
Sources. Citation Sources are not perfect.
 Coverage of indexed publications (Citation Sources)
a) Type (Journals, Reports, Books, Conference Proceedings, etc.)
b) Volume (Large vs small number of indexed sources)
c) Time Period (Established journals with longer publication history)
d) Geographical Range (International vs Regional coverage)
e) Language (i.e. bias towards English Language)
f)
Subject (Specific vs Multidisciplinary sources)
 Errors in Indexing
a) Typographical
b) Transcription
Limitations of Citation Count
Citation Count alone does not determine the:
 Full range of a Researcher’s activity and performance
 Whether citations are due to a longer publication history
 Researcher’s currency of research and recent performance
 Standing or prestige of the journals the articles are published in or cited by
 If citations are viewed positively or negatively
 Ranking of researchers from different disciplines
Making Research Count: Analyzing Canadian Academic Publishing Cultures, Paul Jarvey, Alex Usher,
Lori McElroy, Higher Education Strategy Associates, June 2012
Putting Citation Count in Perspective
As at 11 Feb 2015
The Most Highly Cited Paper in Publishing History: Protein Determination by Oliver H. Lowry,
http://www.jbc.org/content/280/28/e25.full
Putting Citation Count in Perspective
“…it is flattering to be the ‘most cited author’ but I am afraid
it does not signify great scientific accomplishment…
Although method development is usually a pretty pedestrian
Oliver Howe Lowry
affair, others doing more creative work have to use methods
and feel constrained to give credit for the same…”
Lowry OH, Rosebrough NJ, Farr AL & Randall RJ. Protein measurement with the Folin phenol reagent.
J. Biol. Chem. 193: 265-75, 1951. Classic Paper. Citation Classic Commentary
http://garfield.library.upenn.edu/classics1977/A1977DM02300001.pdf
Managing your
Research Publication
History
Managing your Research Publication History
 Be consistent in the use of your name syntax in all your publications, e.g.
Stephen Lee Jin Cheng
Lee, Stephen Jin Cheng
Lee, J.C. Stephen
Lee, SJC
 Avoid author disambiguation by choosing a distinct name that will ease
differentiation from other authors.
 List your institutional affiliations over the course of your career correctly,
including:
a)
b)
c)
d)
Name of Institution - National University of Singapore
Faculty - Faculty of Science instead of Science Faculty, Sci Fac, etc
Department - Department of Physics instead of Phy Dept
For Research Group - Include the Faculty (if applicable) and Department
Managing your Research Publication History
 Check that your publications are correctly listed and indexed in major
databases such as Web of Science and Scopus
 Promote your research online profile and research publications by
participating in researcher identification registries such as:
a) ORCID (Open Researcher and Contributor ID)
b) ResearcherID
c) Google Scholar Citations
Managing your Research Publication History
List of References
 Library Guides: http://libguides.nus.edu.sg/citedref | http://guides.library.jhu.edu/metrics
 Related Library FAQ: How do I get impact factors for journals? | LibFAQ on Citation Analysis
 Eigenfactor Score / Article Influence Score | SCImago Journal Rank (SJR) | Source Normalised
Impact per Paper (SNIP) | Google Scholar Metrics
 ORCID (Open Researcher and Contributor ID) | ResearcherID | Google Scholar Citations
 Professor Andrew Wee Thye Shen’s NUS Page:
http://www.physics.nus.edu.sg/~phyweets/pub.html
 “What does it mean to be #2 in Impact?”, Meredith McVeigh
 “An index to quantify an individual's scientific research output”, Jorge Eduardo (J.E.) Hirsch
 “Making Research Count: Analyzing Canadian Academic Publishing Cultures”, Paul Jarvey, Alex
Usher, Lori McElroy, Higher Education Strategy Associates
 “The Most Highly Cited Paper in Publishing History: Protein Determination by Oliver H. Lowry”,
Nicole Kresge, Robert D. Simoni and Robert L. Hill
 Protein measurement with the Folin phenol reagent. J. Biol. Chem. 193: 265-75, 1951. Classic
Paper. Citation Classic Commentary. Lowry OH, Rosebrough NJ, Farr AL & Randall RJ.
More Useful References
 NUS Library Books:
a)
Beyond bibliometrics : harnessing multidimensional indicators of scholarly impact
/ edited by Blaise Cronin and Cassidy R. Sugimoto
b) The publish or perish book : your guide to effective and responsible citation
analysis / Anne-Wil Harzing
c) Bibliometrics and citation analysis : from the Science citation index to
cybermetrics / Nicola De Bellis
 Article Level Metrics, PLOS
 SPARC Article-Level Metrics Primer, SPARC®, the Scholarly Publishing and
Academic Resources Coalition
 Open Access Explained! , Piled Higher and Deeper (PHD Comics)
 Standing up for Science 3: Peer Review - The nuts and bolts (A guide for early
career researchers), Sense About Science
 Science Metrics, Nature
Your (REAL) Impact Factor (from PHD Comics)
First published on 8 Dec 2008,
http://www.phdcomics.com/comics/archive.php?comicid=1108
Thank you for attending
today!
Please fill in the feedback
form:
http://bit.ly/17zYtII
Elizabeth Lee
elizabeth.lee@nus.edu.sg
Science Library Information Desk
sclib@nus.edu.sg | Tel: 6516 2454