Bibliometrics – A Tool in the Evaluation of Science
Håkan Carlsson
Gothenburg University Library
www.gu.s
e
Table of Contents
•
•
•
•
•
Introduction and definition
Different applications of bibliometrics
Bibliometric indicators
Financial allocaton models based on bibliometrics
Conclusions
www.gu.s
e
What is bibliometrics?
Definition:
“"Bibliometrics can be defined as the quantification of bibliographic
information for use in analysis “– Garfield
Statistical bibliography
www.gu.s
e
Metadata
• Authors
• Affiliation
• Source (journal)
• Title, Abstract, Keywords
• Year
www.gu.s
e
Metadata
• Authors
• Affiliation
• Source (journal)
• Title, Abstract, Keywords
• Year
• References
www.gu.s
e
Major areas of bibliometric research
• Fundamental statistical relationships
• Citation analysis
• Publication indicators for evaluation purposes
www.gu.s
e
Statistical Metods
Pioneer: Derek de Solla Price
1963: Little Science, Big Science
”We can say that 80 to 90 percent of all the
scientists that have ever lived are alive now”
www.gu.s
e
www.gu.s
e
Power-law
www.gu.s
e
Lotka’s Law
Antal författare med ett visst antal publikationer
16000
14000
Antal författare
12000
10000
8000
Serie1
6000
4000
2000
0
0
20
40
60
80
100
Antal publikationer
www.gu.s
e
www.gu.s
e
Rosvall M, CT, Maps of Information Flow Reveal Community Structure In Complex Networks
www.gu.s
e
Why Publish?
www.gu.s
e
Research evaluation indicators
• Production (number of publications, number of collaborators, …)
• Demand, Impact, Quality (number of citations, number of publications
in quality channels, …)
• Excellence (compared to the field, ratio top5% cited publications)
• Collaboration (network properties estimated by co-authorships)
• Internationalization (number of publications in international
journals, number of international collaborators)
• Interdisciplinarity (number of publications in channels in other
fields or with co-authors from other fields)
• …
Changes over time or in comparison with the field or specific
others
www.gu.s
e
Producation-Based Indicators
• Straight-forward and fast
• Time, organisational and document-type limitations
• Fractionalisation
www.gu.s
e
Publications (fractionalized, source: VR)
2000
Lunds univ.
Karolinska inst.
Antal publikationer
1500
Uppsala univ.
Göteborgs univ.
1000
Kungl. tekn. högsk.
Stockholms univ.
Linköpings univ.
Umeå univ.
SLU
Chalmers
Övriga
500
Luleå tekn. univ.
0
1985
1990
1995
2000
2005
www.gu.s
e
Fractionalized publications Gothenburg University
(research articles, chapters, books) GUP Februray 2010
1200,0
1000,0
2004
800,0
2005
2006
600,0
2007
2008
400,0
2009
200,0
0,0
si
Bu
,
ss
e
n
m
no
o
Ec
ics
d
an
w
La
ty
ul
c
Fa
ts
Ar
f
o
IT
Fi
ne
ty
ul
c
Fa
d
lie
p
Ap
d
an
r fo
Pe
rm
g
in
ts
Ar
ty
ul
c
Fa
of
e
nc
ic e
S
a
sk
n
re
lg
h
Sa
m
de
a
Ac
lty
cu
a
F
y
of
cia
So
lS
s
ce
n
cie
lty
cu
a
F
of
n
io
at
c
u
Ed
www.gu.s
e
How do we publish?
100%
90%
80%
70%
60%
Kapitel
50%
Bok
Artikel
40%
30%
20%
10%
0%
NE
Kultgeo
Jur
FEK
EHIST
GRI
www.gu.s
e
Citation-based indicators
Why cite?
Paying homage to pioneers.
Giving credit for related work (homage to peers).
Identifying methodology, equipment, etc.
Providing background reading.
Correcting one's own work.
Correcting the work of others.
Criticizing previous work.
Substantiating claims.
Alerting to forthcoming work.
Providing leads to poorly disseminated, poorly indexed, or uncited work.
Identifying original publications in which an idea or concept was discussed.
Identifying original publications or other work describing an eponymic concept or
term.
• Disclaiming work or ideas of others (negative claims).
• Disputing priority claims of others (negative homage).
•
•
•
•
•
•
•
•
•
•
•
•
www.gu.s
e
Average citation per publication type and year
(Source: WoS analysis by VR)
30
25
Average citations
20
Article
Letter
15
Meeting Abstract
Review
10
5
0
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
www.gu.s
e
Citation rate of different fields (source: VR)
Average citations to 5-year old original articles
Architecture
Art
Mathematics
Law
Information Science & Library Science
Nuclear Science & Technology
Computer Science, Information Systems
Mechanics
Physics, Nuclear
Physics, Multidisciplinary
Physics, Atomic, Molecular & Chemical
Electrochemistry
Chemistry, Multidisciplinary
Allergy
Microbiology
Neurosciences
Cell Biology
0.0
5.0
10.0
15.0
20.0
25.0
www.gu.s
e
Citation window
Antal citeringar (% av summa)
20
15
Engineering
Environmental studies
Psychology, biological
Sociology
10
5
0
0
5
10
15
20
25
År efter publicering
www.gu.s
e
Field normalized citation (”crown indicator”)
CPP (cites per publication)
_____________
FCS (Field Citation Score)
world average for a specific field,
year and article type
’
www.gu.s
e
Field normalized citation (”crown indicator”)
Endocrinology Letter 2003
CPP (cites per publication)
_____________
10
________
FCS (Field Citation Score)
world average for a specific field,
year and article type
CPP/FCS
10
1
www.gu.s
e
Field normalized citation (”crown indicator”)
Endocrinology Letter 2003
CPP (cites per publication)
_____________
FCS (Field Citation Score)
world average for a specific field,
year and article type
CPP/FCS
Oncology Review 2004
10
20
________
________
10
40
1
0,5
www.gu.s
e
Field normalized citation score, 1986-2006
Source: Swedish production of highly cited scientific publikactions (VR, 2010)
www.gu.s
e
Allocation Systems for University Funding
www.gu.s
e
”The Swedish Model”
• Publications in the database Web of Science (WoS) are aggregated for
each university
• Number of publications and number of citations to the publications are
normalized and multiplied together to form the publication indicator
www.gu.s
e
Coverage GUP in WoS, Scopus and Scholar
www.gu.s
e
”The Norweigan Model”
Publications (books, articles, book chapters) are registered in a national
database by the researcher
A quality indicator is added where extra credit is given to publications in
certain publication channels (i.e. journal or publisher determines the extra
credit)
Publication type
Article
Book chapter
Book
Level 1
Level 2
1
3
0,7
1
5
8
www.gu.s
e
Local Developments
• The university has decided to allocate part of its funding to the
faculties based on publications in GUP and external funds.
• Faculties have made similar decisions:
- Humanities
- Social sciences
- Medicine
- Science
www.gu.s
e
What can we learn?
• What have you learned from this session that you would bring back
and tell a colleague?
• How would you think strategically about your own publications?
• Journal choice (target audience, indexing, Web of Science)
• Visibility (access, promotion, personal webpage)
www.gu.s
e
www.gu.s
e
Håkan Carlsson
Hakan.Carlsson@ub.gu.se
www.ub.gu.se/bibliometri
www.gu.s
e