WISER: Bibliometrics II
The Black Art of Citation Rankings
Angela Carritt
Juliet Ralph
November 2011
These slides are available on
http://www.bodleian.ox.ac.uk/services/training
/wiser/presentations
Overview of Session
• What are bibliometrics?
• Why bother?
• Problems
• Bibliometric measures for
•
•
•
•
…a researcher
…an institution
…a country
…a journal
• using Web of Science, Scopus, and other analytical tools
What are bibliometrics?
Citation analysis
Why bother
• Benchmarking of departments and research groups
• Grant applications
• Recruitment of individuals
• Where to publish
Bibliometrics & the REF
• Citation data will not be as important as initially anticipated
• Some panels will use citation data
•
•
•
Panel A – Life Sciences(1)
Some parts of Panel B - Earth and environmental sciences, chemistry, physics
and computer science(1)
Some parts of panel C – some areas of geography, environmental studies and
archaeology but not all, economics/econometrics(1)
• Expert review will be the primary way of measuring impact even where
citation data is used
• REF team will provide citation data (from Scopus)
• More information - HEFCE, Ref 02.2011 Assessment framework and
guidance on submissions
(www.hefce.ac.uk/research/ref/pubs/2011/02_11/)
(1) Information from Gibney, Elizabeth ‘REF panels to differ on impact
and citation use’ 29-07-2011 www.ResearchResearch.com
http://tinyurl.com/6bv69u2
http://www.slideshare.net/guest633b30/bibliometrics-and-scientometrics-1065282
The h-index: to quantify an
individual’s research output
Aims to measure
productivity and impact.
h-index = number of papers you have
published which have that number
of citations or more.
e.g. an h-index of 5 means you have 5
papers which have been cited at
least 5 times each.
Your h-index in Web of Science
Do an author search.
Get list of papers by them.
Sort by Times Cited.
Click on Create Citation Report - to
analyse batch of papers.
Calculation of h-index
Analysing
208 articles in
WoS.
H-index = 53
53 articles cited 53 times or more
Citation tracking & analysis in SCOPUS
• Scopus covers 18,000 journals & conference proceedings
• Science, Medicine, Social Sciences & Humanities
• Each record for a paper shows the number of times it has
been cited in Scopus since 1996
• Similar analytical tools to Web of Science
• www.scopus.com
Your h-index in Scopus
Do an author search.
Tick author name.
Click on View Citation
Overview.
h-index = 21.
Why?
Based on papers
in Scopus
published after
1995.
Lower if exclude self-citations
h-index
= 17
Your h-index and Google Scholar
www.harzing.com
Calculate it with Publish or Perish
Free download.
•Uses citation
data from
Google Scholar.
•
Bibliometrics for an institution
In Scopus, Affiliation search allows
search by institution name university of oxford retrieves same
results as oxford university
Overview of institution
Publications by a department
Search Address field.
Can search by postcode or
name of department/college.
NB. Some authors use dept
postcode, some use main
University one (OX1 2JD).
Re
Research Committee
Recommendation
(1) All authors should
use ‘Oxford
University’ or
‘University of Oxford’
in their publication
address, to ensure
that the publication
may be captured by
citation databases.
(2) [may also] cite
either dept or college.
Oxford & the REF
Symplectic Elements
• Oxford’s tool for gathering references and submission for
the REF.
• Automatic searching of databases such as Web of Science
& Scopus.
• Is your department using it?
• For more information go to
• www.admin.ox.ac.uk/pras/research/symplectic/
• Contact symplectic@admin.ox.ac.uk
Bibliometrics for journals
• Bibliometrics can be used to measure the influence of
academic journals
• Help you to decide where to publish
Bibliometrics & journals: Tools
• ISI Journal Citation Reports (JCR) (ISI Web of Science)
• SCImago Journal Ranking (SJR) and SNIP (Elsevier
Scopus)
• Eigenfactor (sponsored by the Bergstrom Lab in the
Department of Biology at the University of Washington
Journal Citation Reports (JCR)
• Based on citation data from Web of Science
• Covers
• > 5,900 journals in science and technology
• > 1,700 journals in the social sciences
JCR on Web of Knowledge
JCR on Web of Knowledge
JCR
Immediacy Index
Measures how quickly articles are cited.
Calculated:
no. of citations to articles published this year ÷ no. of
articles published this year.
Impact Factor - Number of times the
“average” article published in the previous 2
(or 5) years was cited this year.
Calculated:
no. of citations to articles published in the
last 2 (or 5) years ÷ no. of articles
published in same period.
Cited Half-Life - How many
years you have to go back to
account for 50% of citations
to the journal. e.g. 50% of
citations were to articles
published in the last 3.5
years. The rest cited earlier
articles.
Detailed view
Detailed view continued
Detailed view continued
Citations TO the journal by year of
cited article (e.g. 333 of this year’s
citations to Biological Review were
to articles published in in 2005 )
Detailed view continued
Citations from Biological
Review (to other journals
and self cites) by year of
cited article E.g. 334
citations from Biological
Reviews journal cited
articles published in 2007
Type of articles included
Eigenfactor Metrics
• http://www.eigenfactor.org/
• Take into account prestige of
citing sources
• Use “Google style” algorithms
• Attempts to measure how
often the average researcher
would encounter the journal
2 metrics
• Eigenfactor – increases with
the size of the journal
• Article Influence – Takes into
account number of articles
published. More comparable
to the JCR impact factor
Google’s PageRank from
http://en.wikipedia.org/wiki/PageRank
Eigenfactor
Eigenfactor
Eigenfactor
SJR – an alternative impact factor
• http://www.scimagojr.com/
• SCImago Journal Rank developed by Elsevier in
partnership with Spanish academics.
• Citation data from Scopus
• 50% more journals than ISI Web of Science.
• Weights citations according to the status of the citing
journal
• Updated every 2 months
• More info at http://www.info.sciverse.com/scopus/scopusin-detail/tools/journalanalyzer/
SCImago and the SJR
SCImago Journal Ranking
SCImago journal search
Orange line = average
number of citations (i.e.
same as JCR)
•Purple line = SJR
•
More
metrics
Comparing journals
Measuring impact for a country
SNIP – also by Elsevier
• Source Normalized Impact per Paper.
• Weights citation counts according to the total number of
citations in the subject area
• SNIP aims to account for differences in citation potential
and topicality across research fields.
• ‘Citation potential’ (citation frequency) higher in life
sciences than maths, engineering, social science.
• Higher in basic science than applied or clinical journals.
• Scopus is again the data source.
• http://www.journalmetrics.com/
SNIP (and SJR)
SJR
Choose
journals here
Use slider to
select different
time periods
Use tabs to see
different
metrics
SJR, Snip…
SJR – takes
into account
prestige of
citing journal
SNIP –
contextual
impact.
Weights
citations
according to
total number
of citations
in the
subject
Total number
of citations
Number of
articles
published
% of review
articles
% of articles
not cited at
all
Favourably reviewed in Nature
SJR & SNIP freely available – not
dependent on subscriptions to
Scopus.
Worthy challenger to ISI.
Journal Impact Factors: Problems
Use with caution…Results are skewed by many factors…
• Size
• Frequency / time of publication
• Type of content - review articles are more heavily cited than original
research…
• Journals that are not indexed by WOS / Scopus are disadvantaged
•
Non English Language journals disadvantaged
• Problems when journals change names
• Results are not comparable across discipline (some journals in the
wrong discipline)
• Journal impact factors should NEVER be used to assess impact of
researchers / groups etc
Other ranking tools
• ScienceWatch
• www.sciencewatch.com/
• Weekly tracking of highly-cited papers and topics
• Free Web resource for science metrics and
analysis.
• Includes interviews, essays, podcasts and profiles
from scientists, journals, institutions, and nations.
• Uses citation data from Thomson Reuters (owner
of ISI Web of Science).
Help & info
•See our Bibliometrics guide at
http://libguides.bodleian.ox.ac.uk/
bibliometrics
Brief bibliography
General
• Broadus, R. N., “Towards a definition of Bibliometrics” Scientometrics,
vol. 12, nos 5-6, (1987) 373-379 @
www.springerlink.com/content/v111750n14086384/fulltext.pdf
• HEFCE papers, reports, papers and pilots on the use of bibliometrics
in the REF @ www.hefce.ac.uk/research/ref/Biblio/
H-Index
• Hirsch, J. E. (15 November 2005). "An index to quantify an individual's
scientific research output". PNAS 102 (46): 16569–16572 @
www.pnas.org/content/102/46/16569.abstract
• Ball, P. “Index aims for fair ranking of scientists”, Nature 2005 Aug 18
436: 900
Brief bibliography
Journal Impact Factors and the JCR
• The Thomson Reuters Impact Factor (originally published in the Current
Contents print editions June 20, 1994) @
http://thomsonreuters.com/products_services/science/free/essays/impact_factor/
• Garfield, E. "The agony and the ecstasy: the history and meaning of the
Journal Impact Factor“ Paper at the International Congress on Peer Review
And Biomedical Publication, Chicago, September 16, 2005 @
http://garfield.library.upenn.edu/papers/jifchicago2005.pdf
Want more…
• Wikipedia entries on the following topics include useful bibliographies: the
h-index, journal impact factors and the Eigenfactor
• A Google Scholar (http://scholar.google.co.uk/) search will return many useful
articles including subject studies on the use of bibliometrics