How to Evaluate
Research Impact of Journals
1
Why Journal Impact?




Librarians - can support selection or removal of journals
from their collections, and determine how long to keep
each journal in the collection before archiving it.
Publishers and Editors - can determine journals’ influence
in the marketplace and review editorial functions.
Authors - can identify the most appropriate, influential
journals in which to publish, as well as confirm the status
of journals in which they have published.
Professors and Students - can discover where to find the
current reading list in their respective fields.
2
Journal Evaluation Tools


Ongoing quest for discipline-specific ranking for measuring
‘quality’.
Some journal evaluation tools:





Journal Citation Reports (Impact factor)
Web of Science (h-index)
Scopus (h-index)
SCImago Journal & Country Rank (SJR)
Other tools can be found at the Library website


http://www.lib.ied.edu.hk/research/journalmetrics/index.html
HKIEd Research Information Core Hub (RICH)

https://oraas0.ied.edu.hk/rich/web/search_publication.jsp
3
Journal Citation Reports (JCR)


Journal Citation Reports® offers a systematic, objective
means to critically evaluate the world's leading journals, with
quantifiable, statistical information based on citation data.
By compiling articles' cited references, JCR Web helps to
measure research influence and impact at the journal and
category levels, and shows the relationship between citing
and cited journals. Available in Science and Social Sciences
editions.
Available at Library website:

http://www.lib.ied.edu.hk/index_e-resources.php
4
JCR is used to …






Discover highest-impact journals
Compare a custom selection of journals
Analyze self-citations
Identify review journals
Identify impact factor trend
Develop and manage journal collections
5
Access JCR
6
Access JCR (Social Sciences Edition)
7
Access JCR (Social Sciences Edition)
8
Access JCR (Social Sciences Edition)
9
Access JCR (Social Sciences Edition)
Search Criteria
Metrics
Journal Title
10
Access JCR (Social Sciences Edition)
Click on the abbreviated journal title
to view the full journal record
11
Definition of the Metrics






JCR Year (year under review)
Total Cites
Impact Factor
Immediacy Index
Cited Half-Life
Citing Half-Life
12
JCR Year and Total Cites
1.
JCR Year: The year you select is the JCR year. All
of the data that you see for journals and subject
categories come from journal data published in
that year. e.g. the current JCR year is 2009
2.
Total Cites: The total number of times that a
journal has been cited by all journals included in
the database in the JCR year.
13
Total Cites Calculation
Journal: ACADEMY OF MANAGEMENT JOURNAL
Total Cites in 2009 of citations published in a particular year:
41+124+654+612+711 … + 8846 = 15082
14
Impact Factor
3.
Impact Factor: It is the average number of times
articles from the journal published in the past two
years have been cited in the JCR year.
(出版當年前2年該期刊所出版之文獻在今年被引用之平均次數)
e.g. The journal XYZ has an 2009 impact factor of 2.0
means that, on average, the articles published in
2007 and 2008 was quoted in 2009 an average of 2
times.
15
Impact Factor Calculation
Journal: ACADEMY OF MANAGEMENT JOURNAL
The impact factor is calculated by dividing the number of citations in the JCR
year by the total number of articles published in the two previous years.
Cites in 2009 to 2007 or 2008 papers
Impact Factor =
Papers published in 2007 or 2008
16
Implications of Impact Factor





Tell us how frequently has the average article in a
journal been cited in a particular year.
Tell us something about a journal as a whole
e.g. the extent to which its recently published papers
were cited in a given year.
Impact factor > 1 implied a journal is frequently cited
Higher citations rate means your article has higher
chances of getting cited or read by researchers.
Tells us NOTHING concrete about any specific
paper or specific author.
17
Immediacy Index
3.
Immediacy Index: Immediacy Index measures the
average number of times that an article, published in
a specific year within a specific journal, is cited over
the course of the same year.
(出版當年該期刊在當年被引用之平均次數)
e.g. The journal XYZ contained 100 articles in 2009.
These articles were in the same year quoted 1000
times, i.e. each article was quoted on average 10
times (Immediacy Index = 10).
18
Immediacy Index Calculation
Journal: ACADEMY OF MANAGEMENT JOURNAL
The immediacy index is calculated by dividing the number of citations to
articles published in a given year by the number of articles published in that
year.
19
Implications of Immediacy Index


Measures how quickly the average article in a
journal is cited
Tells you how often articles published in a journal
are cited during their year of publication
20
Cited Half-Life
5.
Cited Half-Life – It measures the number of years,
going back from the current year, that account for half
the total citations received by the cited journal in the
current year.
(該期刊在最新出版年當中50%被引用文獻都集中在最近回溯年份)
e.g. In JCR 2009 the journal XYZ has a cited half-life of 7.0.
This means that articles published in XYZ between
2003-2009 (inclusive) account for 50% of all citations
to articles from that journal in 2009.
* Only journals cited 100 or more times in the JCR year have a cited half-life.
21
Cited Half-Life Calculation
Journal: ACADEMY OF MANAGEMENT JOURNAL
Total cites in 2009: 41+124+654+612+ … +8846 = 15082
50% of the total cites in 2009 = 15082/2 = 7541
Total cites from 2009 to 2000 = 6236 (Cumulative % = 41.35%)
Conclusion: 50% of all citations from this journal cited in the
year 2009 were referred to articles published from over ten
years.
Cited Journal data show how many citations a journal received in the JCR year.
22
Implications of Cited Half-Life

You use the Journal Cited Half Life to see if articles from a journal that
were published a long time ago are still being cited. This shows you if the
journal has a good track record and was producing good articles in the
past. A primary research journal might have a longer cited half-life than a
journal that provides rapid communication of current information.

The Journal Cited Half Life may be useful to library staff in order to carry
out collection management. If a journal has a low Cited Half Life it
means that older papers are not being cited as much and it may be a
reason for not binding these into volumes and archiving them.

Dramatic changes in cited half-life over time may indicate a change in a
journals format. Studying the half-life data of the journals in a
comparative study may indicate differences in format and publication
history.
23
Citing Half-Life
5.
Citing Half-Life – identifies the number of years from the current
year that account for 50% of the cited references from articles
published by a journal in the current year.
(該期刊在最新出版年當中50%引用其他期刊文獻都集中在最近回溯年份)
e.g. In JCR 2009, the journal Food Biotechnology has a citing
half-life of 9.0. That means that 50% of all articles cited by
articles in Food Biotechnology in 2009 were published
between 2001 and 2009 (inclusive).
* Only journals that publish 100 or more cited references have a citing half-life.
24
Citing Half-Life Calculation
Journal: ACADEMY OF MANAGEMENT JOURNAL
Total citing in 2009: 40+106+227+247+ … +316 = 5194
50% of the total cites in 2009 = 5194/2 = 2597
Total cites from 2009 to 2000 = 2374 (Cumulative % = 45.71%)
Conclusion: 50% of all citations citing from this journal in the
year 2009 were referred to articles published from over ten
years.
Citing journal data show how many citations a journal made to other journals (including itself) in the JCR year.
25
Implications of Citing Half-Life

A publisher may use this number to adjust editorial
policies to compete in different market segments.
26
Eigenfactor Score & Article Influence Score

Eigenfactor Score


The Eigenfactor™ score of a journal is an estimate of the
percentage of time that library users spend with that journal.
The Eigenfactor™ Score calculation is based on the number of
times articles from the journal published in the past five years have
been cited in the JCR year, but it also considers which journals have
contributed these citations so that highly cited journals will
influence the network more than lesser cited journals. References
from one article in a journal to another article from the same
journal are removed, so that Eigenfactor Scores are not influenced
by journal self-citation.
http://www.eigenfactor.org/methods.htm
http://www.sciencenet.cn/m/user_content.aspx?id=217750
http://www.eigenfactor.org/methods.pdf
27
Eigenfactor Score & Article Influence Score

Article Influence Score
 The Article Influence determines the average influence of a journal's
articles over the first five years after publication. It is calculated by
dividing a journal’s Eigenfactor Score by the number of articles in the
journal, normalized as a fraction of all articles in all publications. This
measure is roughly analogous to the 5-Year Journal Impact Factor in that it
is a ratio of a journal’s citation influence to the size of the journal’s article
contribution over a period of five years.
 The mean Article Influence Score is 1.00. A score greater than 1.00
indicates that each article in the journal has above-average influence. A
score less than 1.00 indicates that each article in the journal has belowaverage influence.
28
Eigenfactor Score & Article Influence Score
AI:
• A score greater than
1.00 indicates that each
article in the journal has
above-average influence.
A score less than 1.00
indicates that each article
in the journal has belowaverage influence.
EF:
Cites in {2009} to items published in 20042008, also take into account:
• estimate of the percentage of time that
library users spend with that journal.
• eliminate journal self-citation.
• considers which journals have contributed
these citations so that highly cited journals
will influence the network more than lesser
cited journals.
• calculation included both Science and Social
Science journals.
ai is the number of articles published by journal i over the five-year
target window, divided by the total number of articles published by
all source journals over the same five-year window. Notice that a is
thus normalized to sum to 1 and it’s i-th entry specifies the fraction
29
of all published articles that come from journal i.
Mark and Export Records
30
Mark and Export Records
Select record and click “Update Marked List”
31
Mark and Export Records
Click “Marked List”, then
select “Save to File”
32
Journal Selection Process
1.
Basic Journal Publishing Standards : timeliness, editorial
conventions, English Language Bibliographic
Information, peer review
2.
Editorial Content : subject, specific category, compared
with other similar editorial content.
3.
International Diversity : reflect the global context of
scholarly research
4.
Citation Analysis : expert use of citation data help
identify influential useful publications
33
Using Journal Citation Reports Wisely

Should not depend solely on citation data in journal
evaluations.

Other conditions also can influence citation rates
such as language, journal history and format,
publication schedule, and subject specialty.

http://admin-apps.isiknowledge.com/JCR/help/h_using.htm
34
Thank You!
35