Measuring the Interdisciplinarity of a Federal R&D Program Using Bibliometrics and Science Overlay Maps Authors: Asha Balakrishnan, Vanessa Peña, Bhavya Lal IDA Science and Technology Policy Institute, 1899 Pennsylvania Avenue NW, Suite 520, Washington, D.C. Email: [email protected], [email protected], [email protected] Abstract A. Introduction STPI conducted an assessment of a government R&D program aiming to fund higher risk research that address a national need or grand challenge. To attain these goals, the program requires interdisciplinary teams to conduct potentially transformative research. A program funding potentially transformative research at a key R&D agency requested the Science and Technology Policy Institute (STPI) to conduct an assessment of the program’s awards funded between FY2007 to FY2009. We were particularly interested in measuring the interdisciplinarity of the researchers as well as the interdisciplinarity of the proposals. We selected a comparison set of awards to compare the test program to. Among several questions of interest regarding the program’s funded researchers and awards was: How does the interdisciplinarity of the test program’s funded proposals and principal investigators (PIs) compare with the proposals and PIs funded through traditional mechanisms? B. Methodology To address this question, STPI identified a comparison group of awards that proxy for awards from traditional programs. Comparing the test program awards to the more traditional awards, STPI used emerging bibliometric techniques to measure interdisciplinarity and rendered science overlay maps for PIs and awarded research proposals to visualize interdisciplinarity. To assess the interdisciplinarity of the test awards, as well as the awards in the comparison group, STPI used the concept of an “integration score” (or I-Score), an emerging bibliometric, developed to measure the integration of knowledge across a body of research. I-Scores were computed to assess the interdisciplinarity of both the PI and his or her awarded proposal, using data from both thetest and the comparison group. Using PI publication histories beginning in 1980 (obtained from ISI Thomson’s Web of Science), we computed the integration scores of PIs from the test and comparison group. Computed I-Scores are based on the dispersion of the subject categories of the journals cited in a researcher’s publications. This is based on the assumption that the greater the dispersion of subject categories cited, the greater the diversity of knowledge drawn upon.1 Using publications listed as cited references in the awarded proposals, we also computed the proposals’ integration scores. Here the assumption is that the cited publications reflect an award’s disciplinary base including disciplinary contributions provided by all awardees. Maps of science have been developed by various researchers and used in science policy and management since the late 1970’s. Most recently, maps of science have been created for the entire scientific landscape that is based on publication data.2 STPI used a map of science that integrates 221 SCs and 18 macro-disciplines (clusters of SCs) within the natural and social sciences based on 2007 publication data. The maps for PIs and their proposals were compared, providing an orthogonal perspective of interdisciplinarity to the I-Score.3 This presentation will focus on the approaches and methodologies used to assess the interdisciplinarity of both researchers and their awarded proposals. 1 Porter, AL, DJ Roessner, and AE Heberger. 2008. How interdisciplinary is a given body of research? Research Evaluation 17 (4):273-282. 2 Boyack, K. W., R. Klavans, and K. Borner. 2005. Mapping the backbone of science. Scientometrics 64 (3):351374. 3 For instance, if the I-Score of a PI and their awarded proposal are similar, this does not necessarily mean that the disciplines are the same. Rather, there could be a difference between the range of disciplines for the PI and award that would not be indicated by a change in the I-Score.