Measuring the Interdisciplinarity of a Federal R&D Program Using
Bibliometrics and Science Overlay Maps
Authors: Asha Balakrishnan, Vanessa Peña, Bhavya Lal
IDA Science and Technology Policy Institute, 1899 Pennsylvania Avenue NW, Suite 520,
Washington, D.C.
Email: abalakri@ida.org, vpena@ida.org, blal@ida.org
Abstract
A. Introduction
STPI conducted an assessment of a government R&D program aiming to fund higher risk
research that address a national need or grand challenge. To attain these goals, the program
requires interdisciplinary teams to conduct potentially transformative research. A program
funding potentially transformative research at a key R&D agency requested the Science and
Technology Policy Institute (STPI) to conduct an assessment of the program’s awards funded
between FY2007 to FY2009. We were particularly interested in measuring the interdisciplinarity
of the researchers as well as the interdisciplinarity of the proposals. We selected a comparison set
of awards to compare the test program to. Among several questions of interest regarding the
program’s funded researchers and awards was: How does the interdisciplinarity of the test
program’s funded proposals and principal investigators (PIs) compare with the proposals and PIs
funded through traditional mechanisms?
B. Methodology
To address this question, STPI identified a comparison group of awards that proxy for
awards from traditional programs. Comparing the test program awards to the more traditional
awards, STPI used emerging bibliometric techniques to measure interdisciplinarity and rendered
science overlay maps for PIs and awarded research proposals to visualize interdisciplinarity.
To assess the interdisciplinarity of the test awards, as well as the awards in the comparison
group, STPI used the concept of an “integration score” (or I-Score), an emerging bibliometric,
developed to measure the integration of knowledge across a body of research. I-Scores were
computed to assess the interdisciplinarity of both the PI and his or her awarded proposal, using
data from both thetest and the comparison group.

Using PI publication histories beginning in 1980 (obtained from ISI Thomson’s Web of
Science), we computed the integration scores of PIs from the test and comparison
group. Computed I-Scores are based on the dispersion of the subject categories of the
journals cited in a researcher’s publications. This is based on the assumption that the
greater the dispersion of subject categories cited, the greater the diversity of knowledge
drawn upon.1

Using publications listed as cited references in the awarded proposals, we also
computed the proposals’ integration scores. Here the assumption is that the cited
publications reflect an award’s disciplinary base including disciplinary contributions
provided by all awardees.
Maps of science have been developed by various researchers and used in science policy and
management since the late 1970’s. Most recently, maps of science have been created for the
entire scientific landscape that is based on publication data.2 STPI used a map of science that
integrates 221 SCs and 18 macro-disciplines (clusters of SCs) within the natural and social
sciences based on 2007 publication data. The maps for PIs and their proposals were compared,
providing an orthogonal perspective of interdisciplinarity to the I-Score.3
This presentation will focus on the approaches and methodologies used to assess the
interdisciplinarity of both researchers and their awarded proposals.
1
Porter, AL, DJ Roessner, and AE Heberger. 2008. How interdisciplinary is a given body of research? Research
Evaluation 17 (4):273-282.
2
Boyack, K. W., R. Klavans, and K. Borner. 2005. Mapping the backbone of science. Scientometrics 64 (3):351374.
3
For instance, if the I-Score of a PI and their awarded proposal are similar, this does not necessarily mean that the
disciplines are the same. Rather, there could be a difference between the range of disciplines for the PI and award
that would not be indicated by a change in the I-Score.