v2 2015-06-05
Bibliometrics and Publishing
Peter Sjögårde, Bibliometric analyst
KTH Royal Institute of Technology, ECE School of Education and
Communication in Engineering Sciences (ECE), Unit for Publication
Infrastructure
Outline
•
•
•
•
KTH context – Aims at KTH
Introduction to bibliometrics
Bibliometric indicator for funding allocation at KTH
Annual Bibliometric Monitoring
KTH context – Aims at KTH
• Strengthen its position as one of Europe’s leading
technical universities
• Advance in relevant rankings
• Increase citation scores
What is bibliometrics?
Bibliometrics can be defined as the quantitative study of
publication collections
Statistics on:
• Publications, like journal articles (research output)
• Citations (research impact)
• Co-publishing (research collaboration)
Example of publication output
Publication frequencies
120
100
Number of publications
Report
Patent
Licentiate Thesis
Doctoral Thesis
Conference Proceedings (editor)
Conference Paper
Collection/Anthology (editor)
Chapter in book
Book Review
Book
Article, review/survey
Article in journal (other)
Article in journal (peer reviewed)
80
60
40
20
0
2004
2005
2006
2007
2008
2009
2010
2011
Field normalized citation rate (3 year
moving average), cf3
Examples of citation impact
Time series of field normalized citation
rate
1.40
1.20
1.00
0.80
0.60
0.40
0.20
0.00
2004
2005
2006
2007
Year
2008
2009
2010
Examples of a co-publishing network
Citation
Research
Publication A
Reader
Writer
Publication B
with Reference to
the publication A
Impact?
• On society
• On industry
• On research community
As measured by the use of bibliometrics (citations)
Differences between research fields
affecting bibliometric indicators
•
•
•
•
•
Publication pace (time for publishing and peer review process)
Length of publications
Number of references
Coverage of publications in citation database
Document type (e.g. reviews get more citations, proceedings
papers are often excluded)
• Size of research field
Field normalized citation rate
For a publication:
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑖𝑡𝑎𝑡𝑖𝑜𝑛𝑠
𝑎𝑣𝑒𝑟𝑎𝑔𝑒 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑖𝑡𝑎𝑡𝑖𝑜𝑛𝑠 𝑓𝑜𝑟 𝑠𝑖𝑚𝑖𝑙𝑎𝑟 𝑝𝑢𝑏𝑙𝑖𝑐𝑎𝑡𝑖𝑜𝑛𝑠
Same research field
same year
same document type
Average = 1
Citation indicators > Research Quality
•
•
•
Correlation between peer review assessments and
bibliometric indicators.
In general, high quality (as judged by peers) is needed to
receive high citation scores at aggregated levels.
• But not enough
There are research groups that get good judgments by
reviewers but has low citation scores.
1. Research with impact outside the research community
(e.g. applied research)
2. Research for which the bibliometric methods are
improper.
Lutz Bornmann and Hans-Dieter Daniel, “Selection of Research Fellowship Recipients by Committee
Peer Review. Reliability, Fairness and Predictive Validity of Board of Trustees’ Decisions,”
Scientometrics 63, no. 2 (April 1, 2005): 297–320, doi:10.1007/s11192-005-0214-2; A. F. J. van Raan,
“Advanced Bibliometric Methods as Quantitative Core of Peer Review Based Evaluation and Foresight
Exercises,” Scientometrics 36, no. 3 (July 1, 1996): 397–420, doi:10.1007/BF02129602; Charles
Oppenheim, “The Correlation between Citation Counts and the 1992 Research Assessment Exercise
Ratings for British Research in Genetics, Anatomy and Archaeology,” Journal of Documentation 53,
no. 5 (December 1, 1997): 477–87, doi:10.1108/EUM0000000007207; Fler studier är refererade i
Bornmann and Daniel, “What Do Citation Counts Measure?”; och i Blaise Cronin, The Hand of Science:
Academic Writing and Its Rewards (Lanham, Md: Scarecrow Press, 2005), 125–129
Application and use
Macro
Bibliometrics
Meso
Peer
review
Micro
Sybille Hinze. 2014. Nordic workshop on bibliometrics and research policy. Reykjavik, Iceland.
Bibliometrics at KTH
•
•
•
•
•
Research Assessment Exercise (research groups, UoA)
Funding allocation
Annual Bibliometric Monitoring
Reports and analyzes on demand from management and
schools
Consultancy work for other universities
Bibliometric indicator for funding allocation
at KTH
Mean field normalized citation rate for journals and serials
indexed by Web of Science
• Average of the citation rate of the journals in which KTH:s
researchers have published the last three years.
• Calculated for each department. Multiplied with the
number of faculty at each department.
• Aggregated to Schools and used for allocation to the
Schools.
Bibliometric indicator for funding allocation
at KTH
Example. Let A be a department. Assume that A has three publications, P1, P2, and P3,
published in three different journals. Further assumptions:
P1
P2
P3
Sum
A author fraction
(of publication)
1 (1/1)
0.25 (1/4)
0.5 (5/10)
1.75
Jcf (of journal for
A publication)
0.8
1
2
[A Author fraction] x
Jcf
0.8
0.25
1
2.05
Then the mean field normalized journal impact for A, mjcf(A), is equal to
1 × 0.8 + 0.25 × 1 + (0.5 × 2) 0.8 + 0.25 + 1 2.05
=
=
= 1.17
1 + 0.25 + 0.5
1.75
1.75
Let the number of faculty for A be 3. Then the value of the bibliometric indicator for funding
allocation at KTH is
3 × 1.17 = 3.51
Purpose of the bibliometric funding
allocation indicator
• Incentives to:
• publish in highly cited journals indexed by Web of
Science.
• publish few publications of high quality rather than a
high quantity of publications.
Annual Bibliometric Monitoring
https://intra.kth.se/bibliometri/public/start
Comments/questions to:
Per Ahlgren, Bibliometric Analyst
perahl@kth.se
Peter Sjögårde, Bibliometric Analyst
sjogarde@kth.se
Unit for Publication Infrastructure
pi@ece.kth.se
KTH Royal Institute of Technology
School of Education and Communication in Engineering Sciences (ECE)
Unit for Publication Infrastructure