Limited Access: Limited Research Impact
Impact cycle
begins:
12-18 Months
Research is
done
Researchers write
pre-refereeing
“Pre-Print”
Submitted to Journal
Pre-Print reviewed by
Peer Experts – “PeerReview”
Pre-Print revised by
article’s Authors
Refereed “Post-Print”
Accepted, Certified, Published
by Journal
Researchers can access the
Post-Print if their university
has a subscription to the
Journal
New impact cycles:
New research builds
on existing research
Stevan Harnad: Southampton and Montreal
Maximized Research Access and Impact Through Self-Archiving
12-18 Months
Impact cycle
begins:
Researchers write
pre-refereeing
Research is done
“Pre-Print”
Pre-Print is selfarchived in
University’s Eprint
Archive
Submitted to Journal
Pre-Print reviewed by Peer
Experts – “Peer-Review”
Pre-Print revised by
article’s Authors
Refereed “Post-Print” Accepted,
Certified, Published by Journal
Researchers can access the
Post-Print if their university
has a subscription to the
Journal
Post-Print is selfarchived in
University’s Eprint
Archive
New impact cycles:
Self-archived
research
impact is greater (and
faster) because
access is maximized
(and accelerated)
New impact cycles:
research builds on
Stevan Harnad: SouthamptonNew
and Montreal
existing research
“Online or Invisible?” (Lawrence 2001)
“average of 336% more citations to online articles
compared to offline articles published in the same venue”
Lawrence, S. (2001) Free online availability substantially increases
a paper's impact Nature 411 (6837): 521.
http://www.neci.nec.com/~lawrence/papers/online-nature01/
Stevan Harnad: Southampton and Montreal
Lawrence (2001) findings for computer science conference papers. More OA
every year for all citation levels; higher with higher citation levels
Stevan Harnad: Southampton and Montreal
Citation impact for articles in the same journal and year are consistently higher for articles that have been
self-archived by their authors. (Below is a comparison for Astronomy articles that are and are not in ArXiv.)
Stevan Harnad: Southampton and Montreal
Citation impact
Stevan Harnad: Southampton and Montreal
Astrophysics
HEP/Nuclear Physics
General Physics
Chemical Physics
Stevan Harnad: Southampton and Montreal
Changing citation behaviour
Time taken to be cited for articles in the arXiv database
10000
9000
1991
1993
1995
1997
1999
2001
2003
8000
6000
5000
4000
3000
2000
1000
Months from publication
Stevan Harnad: Southampton and Montreal
96
90
84
78
72
66
60
54
48
42
36
30
24
18
12
6
0
0
-6
Numbers
7000
Stevan Harnad: Southampton and Montreal
DATA: Michael Kurtz
DATA: Michael Kurtz
Stevan Harnad: Southampton and Montreal
Stevan Harnad: Southampton and Montreal
DATA: Michael Kurtz
Stevan Harnad: Southampton and Montreal
DATA: Michael Kurtz
The citation impact advantage is found in all fields
analyzed so far, including articles (self-archived
in any kind of open-access website or archive) in
social sciences (above right) biological sciences
(below right) and all fields of Physics (self-archived
in ArXiv, below). Note that the percentage of
published articles that have been self-archived
(green bars) varies from about 10-20%from field
to field and that the size of the open-access citation
impact advantage (red bars) varies from about
25% to over 300%, but it is always positive.
http://opcit.eprints.org/oacitation-biblio.html
Social Sciences
Signal detection analysis of the hit/miss rate of the
algorithm that searched for full-text OA papers on
the web:
d’ = 2.45 (sensitivity)
b = .52 (bias)
Stevan Harnad: Southampton and Montreal
Biological Sciences
By discipline: total articles (OA+NOA), gray curve; percentage OA: (OA/(OA+NOA))
articles, green bars; percentage OA citation advantage: ((OA-NOA)/NOA) citation, red
bars, averaged across 1992-2003 and ranked by total articles. All disciplines show an OA
citation advantage (Hajjem et al. IEEE DEB 2005)
By country: total articles (gray curve), percent OA articles (green bars), and
percent OA citation advantage (red bars); averaged across all disciplines and
years 1992-2003; ranked by total articles. (Hajjem et al. IEEE DEB 2005)
By year: total articles (gray curve), percent OA articles (green bars), and percent
OA citation advantage (red bars): 1992-2003, averaged across all disciplines.
No yearly trend is apparent in the size of the OA citation advantage, but %OA
is growing from year to year. (Hajjem et al. IEEE DEB 2005)
Figure 3a: The yearly percentage (OAc) of the articles with c
citations (c = 0, 1 2-3, 4-7, 8-15, 16+) that are OA (1992-2003).
This graph should really be read backwards, as citations increase
cumulatively as an article gets older (younger articles have fewer
citations). Reading backwards, for articles with no citations (c=0),
the percentage OAc decreases each year from 2003-1992, at first
rapidly, then more slowly. For articles with one and more citations
(c>0), OAc first increases rapidly from 2003 till about 1998, then
decreases slowly 1998-1992. Notice that the rank order becomes
inverted around midway (c. 1998), the percentages increasing
from c=0 to c=16+ for the oldest articles (1992) and the reverse
for the youngest articles (2003). The pattern is almost identical for
NOA articles too (see NOAc inset), so this is the relationship
between citation ranges and time for all articles, not a specific OA
effect. The OA effect only becomes apparent when we look at
OAc/NOAc (Figure 3b) (Hajjem et al. IEEE DEB 2005)
Figure 3b: The yearly ratio OAc/NOAc between the percentage of
articles with c citations (c = 0, 1 2-3, 4-7, 8-15, 16+) that are OA and
NOA (all disciplines). This ratio is increasing with time (as well as
with higher citation counts, c), showing that the effect first reported for
computer science conference papers by Lawrence (2001)
occurs for
all disciplines.
Stevan Harnad: Southampton and Montreal
OAc/NOAc ratio (across all disciplines and years increases as citation count (c)
increases (r = .98, N=6, p<.005). Percentage of articles is relatively higher among
NOA articles with Citations = 0; it becomes higher among OA articles with
citations = 1 or more. The more cited an article, the more likely that it is OA.
(Hajjem et al. IEEE DEB 2005)
Stevan Harnad: Southampton and Montreal
OA advantage
= EA + QA + UA + (CA) + (QB)
•
EA: Early Advantage: Permanent citation increment for preprint
•
QA: Quality Advantage: Self-archiving citations; higher-quality articles
benefit more
•
UA: Usage Advantage: Self-archiving increases downloads; higherquality articles benefit more
•
(CA: Competitive Advantage): OA/non-OA advantage (disappears at
100% OA)
•
(QB: Quality Bias): Higher-quality articles self-archive(d) more
(disappears at 100%OA)
Stevan Harnad: Southampton and Montreal
Self-Selected vs. Mandated Self-Archiving I: Non-Normalised Grand Aaverage Citation
Ratios. S= articles self-archived at institutions with (Sm, 237 articles) and without (Sn, 890) a selfarchiving mandate. N = citation counts for non-archived articles at institutions with (Nm, 16485)
and without (Nn, 89156) mandate (i.e., Nm = articles not yet compliant with mandate).There is no
indication that Sn ratios are greater than Sm ratios: rather the contrary. (NB: These averages are
across fields and based on very different samples sizes. Following figure (II) compares like with
like)
Self-Selected vs. Mandated Self-Archiving II: Within-Journal Citation Ratios. S =
citation counts for articles self-archived at institutions with (Sm) and without (Sn) a self-archiving
mandate. N = citation counts for non-archived articles at institutions with (Nm) and without (Nn)
mandate (i.e., Nm = articles not yet compliant with mandate). Grand average S/O (106203 articles;
279 journals) is the OA advantage (18%); this is about the same as Sn/Nn (27972 articles, 48
journals); ratio is larger for Sm/Nm (57%, 541 articles, 20 journals). Sn/Sm = -27%, so self-selected
self-archiving does not give more citations than mandated; rather the reverse.
The G-factor International
University Ranking
measures the importance
of universities as a
function of the number of
links to their websites
from the websites of other
leading international
universities.
Why is Southampton
ranked 3rd highest in
the UK and 25th in the
world, above Columbia
(27th) and Yale (51st)?
Copyright Peter Hirst, 2006.
Reasons for U. Southampton's High Webmetric Rank:
(1) U. Southampton's university-wide research performance
(2) U. Southampton's Electronics and Computer Science (ECS) Department's
involvement in many high-profile web projects and activities (among them the semantic
web work of the web's inventor, ECS Prof. Tim Berners-Lee, the Advanced Knowledge
Technologies (AKT) work of Prof. Nigel Shadbolt, and the pioneering web science
contributions of Prof. Wendy Hall)
(3) Since 2001 U. Southampton's ECS has had a mandate requiring that all of its
research output be made Open Access on the web by depositing it in the ECS
EPrints Repository, and that Southampton has a university-wide self-archiving policy
(soon to become a mandate) too
(4) Maximising access to research (by self-archiving it free for all on the web)
maximises research usage and impact (and hence web impact)
This all makes for an extremely strong Southampton web presence, as reflected in such metrics as
the "G factor", which places Southampton 3rd in the UK and 25th among the world's top 300
universities or Webometrics,which places Southampton 6th in UK, 9th in Europe, and 80th among
the top 3000 universities it indexes.
Research Assessment, Research
Funding, and Citation Impact
“Correlation between RAE ratings and mean
departmental citations +0.91 (1996) +0.86
(2001) (Psychology)”
“RAE and citation counting measure
broadly the same thing”
“Citation counting is both more cost-effective
and more transparent”
(Eysenck & Smith 2002)
http://psyserver.pc.rhbnc.ac.uk/citations.pdf
Stevan Harnad: Southampton and Montreal
Metrics
• Citations (C)
• CiteRank
• Co-citations
• Downloads (D)
• C/D Correlations
• Hub/Authority
• Chronometrics:
Latency/Longevity
• Endogamy/Exogamy
• Semiometrics
Stevan Harnad: Southampton and Montreal
Measure usage and impact
Stevan Harnad: Southampton and Montreal
Time-Course and cycle of Citations (red)
and Usage (hits, green)
Witten, Edward (1998) String Theory and Noncommutative Geometry Adv. Theor. Math. Phys. 2 : 253
1. Preprint or
Postprint appears.
2. It is downloaded
(and sometimes
read).
3. Next, citations
may follow (for
more important
papers)…
4. This generates
more downloads…
5. More citations...
Stevan Harnad: Southampton and Montreal
Usage Impact (downloads)
is correlated with Citation Impact
(Physics ArXiv: hep, astro, cond, quantum; math, comp)
http://citebase.eprints.org/analysis/correlation.php
downloads from first 6 months after publication
predict citations 2 years after publicattion
(Quartiles Q1 (lo) - Q4 (hi))
All
Most papers are not cited at all
r=.27, n=219328
Q1 (lo) r=.26, n=54832
Q2
r=.18, n=54832
Q3
r=.28, n=54832
Q4 (hi) r=.34, n=54832
hep
r=.33, n=74020
Q1 (lo)
Q2
Q3
Q4 (hi)
r=.23, n=18505
r=.23, n=18505
r=.30, n=18505
r=.50, n=18505
(correlation is highest for highcitation papers/authors)
Stevan
Harnad:
Average UK downloads per
paper:
10 Southampton and Montreal
(UK site only: 18 mirror sites in all)
Raw citation counts
Multiple Regression Analysis reveals
4 independent influences on citation
counts (overall, and in all subsets):
1. article age
2. journal impact factor
3. number of authors
4. open access
Log citation counts
BOAI Self-Archiving FAQ
http://www.eprints.org/self-faq/
"I-worry-about..." 32 FAQs (sub-grouped thematically)
I. 10. Copyright
32. Poisoned Apple
II. 7. Peer review
5. Certification
6. Evaluation
22. Tenure/Promotion
13. Censorship
III. 29. Sitting Pretty
4. Navigation (info-glut)
IV. 1. Preservation
2. Authentication
3. Corruption
23. Version control
25. Mark-up
26. Classification
16. Graphics
15. Readability
IV. 1. Preservation continued….
21. Serendipity
18. Libraries'/Librarians' future
V. 19. Learned Societies' future
VI. 17. Publishers' future
9. Downsizing
8. Paying the piper
14. Capitalism
24. Napster
31. Waiting for Gold
VII. 20. University conspiracy
30. Rechanneling toll-savings
28. Affordability
VIII. 12. Priority
27. Secrecy
IX. 11. Plagiarism
Stevan Harnad: Southampton and Montreal