InCites TM
rachel.mangan@thomsonreuters.com
http://researchanalytics.thomsonreuters.com/incites/
InCites Overview
• Annual Subscription to bibliometrics reports based on Web of
Science
• 1 Research Performance Profile Report (RPP)
containing articles extracted from WoS with associated
metrics
– Thomson Reuters staff support building dataset to client specifications
– Dataset may be based on author, address, topic, or journal
• Entire Global Comparisons (GC) sets containing aggregated
comparative statistics for fields, countries, and institutions
– National
– Institutional by Region
– Multiple subject schema
2
InCites Overview
• Web based interface as well as data on demand
• Choices for delivery of a record set: FTP, MS Access
• Frequent Updates: Quarterly for RPP, Annually for GC (Spring)
• Access to TR Web Services: WS Premium, Researcher ID Services
• Data preparation support and training
• Ability to create custom reporting with sharing/saving data
• Links to WoS records: Limited View for non-subscriber, Full View for
subscribers
• Licensed right to export reports and statistics about the institution for
display on Institutional Web Site for promotional or information purposes
• Licensed Right to load data to Institutional Repository or local database for
institutional business purposes
3
CRITICAL ANSWERS TO PRACTICAL
QUESTIONS
 What is the published output of my institution in various disciplines of study
over the past 10 years?
 What impact did this research have, how frequently has it been cited and
where, by whom?
 With which researchers and institutions is our faculty collaborating?
 Which collaborations are most valuable, producing influential work – greatest
return on investment?
 How does my institution compare to our peer institutions in volume and
influence of published work in particular fields?
 Which programs within our institution perform best in terms of research
output, producing research that is the most influential compared to other
research in the world within those particular disciplines.
4
THE DATA
5
WEB OF SCIENCE
• Selectivity and control of content- high, consistent standards
• 11,000+ journals and 716 million+ cited references
• Multidisciplinary- Science, Social Science, Arts/Humanities
• Depth- 100+ years- including cited references
• Consistency and reliability- ideal for research evaluation e.g. field
averages
• Unmatched expertise- 40+ years of citation analysis and research
evaluation
• Conference Proceedings- 12,000 conferences annually
• Funding acknowledgments
• The gold standard-used by over 3,200 institutions in more than 90
countries
6
SOURCE OF DATA SET: WEB OF SCIENCE
RECORDS
7
THE METRICS
• Absolute Counts
• Normalised metrics (for journal, document type,
period, and category)
• Golden Rule: Compare like with like
• All document types included in RPP
8
NO ALL PURPOSE INDICATOR
This is a list of a number of
different purposes a university
might have for evaluating its
research performance.
Each purpose calls for
particular kinds of information.
Identify the question the results
will help to answer and collect
the data accordingly
9
IS THIS A HIGH CITATION COUNT?
10
CREATING A BENCHMARK-WHAT IS THE
EXPECTED CITATION RATE TO SIMILAR
PAPERS?
Articles published ‘Monthly Notices of the Royal Astronomical Society’
published in 2007 have been cited on average 13.87 times. This is the
expected count
We compare the total citations received to a paper to what is expected
215 (Journal Actual) / 13.87 (Journal Expected) = 15.5
The paper has been cited 15.5. times more than expected. We call this
Journal Actual/Journal Expected
11
PERCENTILE IN FIELD- HOW MANY
PAPERS IN THE TOP 1%, 5% OR 10% IN
THEIR RESPECTIVE FIELDS?
This is an example of the citation frequency distribution of a set of papers
(author, journal, institution or subject category) . The papers are ordered
none/least cited on the left, moving to the highest cited papers in the set on
the right.
We can assign each paper to a Percentile in the set.
In any given set, there
are always many low
cited/ none cited papers
(bottom 100%)
100%
In any given set, there
are always few highly
cited papers (top 1%)
50%
0%
Only article types article, note, and review are used to determine the
percentile distribution, and only those same article types receive a percentile
value
12
RESEARCH PERFORMANCE PROFILES
 Executive Summary
 Source Articles Listing
 Author Ranking
 Summary Metrics
 Author Ranking with Self Citation Analysis
 Collaborating Institutions
 Field Rankings
 Custom Reports – How to Generate an Author Ranking
for a Particular Field/Category
 Citing Dataset (view citing authors, institutions and journals)
13
SUMMARY METRICS
METRIC
MEASURE
IDENTIFY
% Cited to %Un Cited
% of papers in dataset that have received at least one
cite
Amount of research in dataset with no
impact
Mean Percentile
Average Percentile for set of papers in dataset.
Percentile is assigned to a paper within a set of
papers from same subject category/year/ document
type ordered most cited (0%) to least cited (100%)
Average ranking of papers in dataset.
How well the papers perform
compared to papers from same
category/year/document type
Average cites per
document
Efficiency (or average impact) of author papers
Authors with highest/lowest average
impact
Mean Journal
Actual/Expected Citations
Average ratio for papers in dataset. Ratio is
relationship between actual citations to each paper to
what is expected for papers in same journal/
publication year and document type
Average performance of papers in
dataset when compared to papers
from same journal/ publication year
and document type
Mean Category
Actual/Expected Citations
Average ratio for papers in the dataset. Ratio is
relationship between actual citations to each paper to
what is expected for papers in same category/
publication year and document type
Average performance of papers in
dataset when compared to papers
from same category/ document type
and publication year
Percentage articles
above/ below what is
expected
1% of papers are expected to be in top 1% percentile.
Green bar indicates by what percentage the papers
are performing better than expected. Red bar indicates
the percentage by which the papers are performing
lower that expected at a given percentile range
How well the papers in the dataset are
performing at the percentile level.
14
SOURCE ARTICLE LISTING METRICS
METRIC
MEASURE
IDENTIFY
Times Cited
Total cites to paper
Highest cited papers
Second Generation Cites
Total cites to a citing paper
(dataset)
Long term impact of a paper
Journal Expected Citations
Average Times Cited count to
papers from same journal,
publication year and document
type
Papers which perform better or
below what is expected compared
to similar papers from journal and
same period
Category Expected Citations
Average Times Cited count to
papers from same category,
publication year and document
type
Papers which perform better or
below what is expected compared
to similar papers in a subject
category from same period
Percentile in Subject Area
Percentile a paper is assigned to
with papers from same subject
category/year/ document type
ordered most cited (0%) to least
cited (100%)
Papers which rank the highest or
lowest in their field from same
period
Journal Impact Factor
Average cites in 2009 to papers
published in the previous 2 years
in a given journal
Journals which have high or low
impact in 2009
15
AUTHOR RANKING
METRIC
MEASURE
IDENTIFY
Times Cited
Total cites to an authors papers
Authors with highest /lowest total cites
to their papers
WOS documents
Total number of papers by an author in dataset
Authors with highest/ lowest number of
publications
Average cites per
document
Efficiency (or average impact) of author papers
Authors with highest/lowest average
impact
h-index
An authors research performance. Publications are
ranked in descending order by the times cited. The
value of h is equal to the number of papers (N) in the
list that have N or more citations
Authors with highest impact and
quantity of publications in a single
indicator
Journal Actual/Expected
Citations
Average ratio for authors papers. Ratio is relationship
between actual citations to each paper to what is
expected for papers in same journal/ publication year
and document type
Authors who’s papers perform above
or below what is expect in respective
journals. Compare authors in different
fields.
Category Actual/Expected
Citations
Average ratio for authors papers. Ratio is relationship
between actual citations to each paper to what is
expected for papers in same category/ publication year
and document type
Authors who’s papers perform above
or below what is expected in their
respective subject categories
Average percentile
Average Percentile for set of authors papers.
Percentile is assigned to a paper within a set of
papers from same subject category/year/ document
type ordered most cited (0%) to least cited (100%)
Authors who ‘s papers on average
rank high or low with regard to total
cites in the respective fields the
16
papers belong to
COLLABORATION AND RESEARCH
NETWORK
METRIC
MEASURE
IDENTIFY
Times Cited
Total cites to set of papers (collaboration)
Institutions/countries with which the research
has the most impact (cites)
WOS documents
Total number of papers published in collaboration with an
institution/country
Institution/ countries with which your
researcher collaborate the most
Average cites per
document
Efficiency (or average impact) of papers
Institution/ countries with which the research
has the highest/lowest average impact
h-index
Performance of a set of papers. Publications are ranked in
descending order by the times cited. The value of h is equal to
the number of papers (N) in the list that have N or more
citations
Institutions/ countries with which the
collaboration has the highest impact and
quantity of publications as measured in this
single number indicator
Journal Actual/Expected
Citations
Average ratio for collaboration papers. Ratio is relationship
between actual citations to each paper to what is expected for
papers in same journal/ publication year and document type
Collaboration with an institution or country
with which the papers perform above or
below what is expect when compared to
similar papers in their respective journals
Collaboration with best return on investment
Category
Actual/Expected
Citations
Average ratio for authors papers. Ratio is relationship between
actual citations to each paper to what is expected for papers in
same category/ publication year and document type
Collaboration with an institution or country
with which papers perform above or below
what is expected in their respective subject
categories
Average percentile
Average Percentile for set of collaboration papers. Percentile is
assigned to a paper from a set of papers from same subject
category/year/ document type ordered most cited (0%) to least
cited (100%)
Collaborations with which the papers on
average rank high (0%) or low (100%) with
regard to their total cites in the respective
17
fields the papers belong to
GLOBAL COMPARISIONS
• Web of Science document types included:
– Articles
– Reviews
18
Quick view of entire dataset
19
Article Level Metrics
20
Thomson Reuters value added metrics
Number of citations
Basic bibliographic information
about the article (including the field)
The Journal Impact Factor
from the latest edition of the
Journal Citation Reports
21
Thomson Reuters value added metrics
2nd generation citation
data, the articles that have
cited the citing articles
22
Thomson Reuters value added metrics
Expected performance metrics.
We calculate the number of citations a typical article would
expect to receive.
This is calculated for each Journal (JXC) and for each
Category (CXC) these metrics are also normalized for the
year and document type.
23
Thomson Reuters value added metrics
JXC Ratio (157 / 45.09) = 3.48
CXC Ratio (157 / 3.66) = 42.90
Although, it is not displayed on this screen, we also calculate the
ratio between the actual and expected performance.
This provides meaning and understanding of the citation counts
and is a normalized performance measure.
24
Thomson Reuters value added metrics
The percentile. As compared for the set of documents in the
same field and the same year.
This paper is in the top 0.2% of all papers in “General &
Internal Medicine” for the year 2007
The percentile is not calculated for all document types
25
Total citation
counts, mean and
median citations,
2nd generation total
citation counts and
mean 2nd
generation citation
counts
We generate summary metrics based on
totals and averages of all the articles in the
dataset
Mean Actual /
Expected
Citation Ratio.
Mean
Percentile
26
Author reports with self cites
removed
27
GLOBAL COMPARISONS
28
COMPARE COUNTRY / TERRITORIES
Data is available in tabular
format with all metrics in
one location
Graphical summaries make
the data easy to interpret
In this example we can see
the Citation Impact of
Sweden in selected fields
compared to the world
29
COMPARE COUNTRY / TERRITORIES
Normalized metrics are included for better understanding and
relevant comparisons.
In this example you can see the citation impact of selected
countries normalized to the world average
30
COMPARE COUNTRY / TERRITORIES
Various regional groupings, such as EU or Asia Pacific are included.
31
COMPARE COUNTRY / TERRITORIES
There are different category classification schemes available.
250+ narrow categories from the Web of Science, 22 broad categories and
56 (including 6 broad) OECD classifications
The inclusion of the OECD classification scheme makes for easy
integration of InCites data with OECD data, such as R&D spending
RAE 2008 Units of Assessment for RAE comparisions
32
HOW IS OUR RESEARCH COMPARED TO OUR PEERS?
Focus on Nanoscience & Nanotechnology:
% Articles in
Organization
Citation
Impact
33
HOW IS OUR RESEARCH COMPARED TO OUR PEERS?
Article count
Normalized for field
Make comparison’s with your peer institutions effectively and easily.
In this example: Normalized metrics demonstrate that Stanford Univ
and Univ Sydney have significant output in the field of Economics &
Business this information would ordinarily be difficult to identify
34
AGGREGATE PERFORMANCE INDICATOR
DIFFERENT RESEARCH FOCUS
• Harvard’s research is heavily
focused on the Life Sciences,
which are generally highly cited
fields
• Princeton’s research focus is
more biased towards less cited
subjects such as Space Science,
Physics, Mathematics and
Geosciences
• The Aggregate Performance
Indicator takes into account these
differences in the research focus
of the university and the different
characteristics of the subject
areas to generate a single metric 35
AGGREGATE PERFORMANCE INDICATOR
Here we can see various metrics including the
Aggregate Performance Indicator for selected
global institutions.
We have see that Harvard produces many
more Articles than Princeton and that the
average Citations per Article is also higher.
However, as discussed in the previous slide,
Harvard’s research focuses heavily on highly
cited fields such as the Life Sciences which
improves the gross citation statistics of
Harvard. A more granular approach is required
to understand the performance in difference
fields.
The Aggregate Performance Indicator
measure the relative performance and volume
of research in each field and generates a
single normalized metric which provides a
more balanced approach when trying to
compare the research performance of
36
institutions.