Measuring Science (II)
Morten Brendstrup-Hansen
• No science without scientific publications
• Scientific publications are direct and
tangible products of scientific activity
• Therefore, the idea of a measure of
scientific performance based on
publications is sound and straightforward
Peer review vs. bibliometric
analysis
• Peer review may be accurate, but is time
consuming and may easily be (suspect of
being) biased
• The accuracy of a scholarly founded
formal bibliometric analysis may be
examined and its validity may be
discussed
• A bibliometric analysis is based on publicly
available (and easily collected) data
In bibliometrics, what should we
count/calculate?
Where do we find the data for
bibliometric analyses?
Total number of papers
• Measures quantity, but does not take
quality into account; does not give due
weight to influence
Number of 'quality papers' e.g.
defined as papers in ISI-journals
• Relies on the inclusion in a particular
journal as a measure of quality instead of
trying to assess the actual quality of the
paper
Total number of citations
• Measures influence, but may be inflated
by a small number of unrepresentative big
hits
Number of citations per paper
• Punishes productivity
Number of papers with >x citations
• Combines publication data with citation
data
• Thus rewards quality as well as quantity if
a fair value of x is chosen,
• but different values of x need to be
decided upon for different fields of
research
h-index
• A scientist has the index h if h of his or her
papers have at least h citations each - Hirsch JE
(2005) PNAS 102(46): 16569-16572
Nc
h
h
Np
h-index
• A scientist has the index h if h of his or her
papers have at least h citations each - Hirsch JE
(2005) PNAS 102(46): 16569-16572
Nc
h
h
Np
g-index
• A set of papers has a g-index g if g is the
highest rank such that the top g papers
have, together, at least g2 citations Egghe L (2006) Scientometrics 69(1):131152
This is only the beginning
• More indices will probably be coined
• Indices should be validated e.g. by testing
their predicative power
Software link
• Publish or Perish is a piece of software
that calculates several bibliometric indices
from Google Scholar data. It is provided
free of charge at http://www.harzing.com/