Towards a Science of Science (Policy)
Dr. Katy Börner
Cyberinfrastructure for Network Science Center, Director
Information Visualization Laboratory, Director
School of Library and Information Science
Indiana University, Bloomington, IN
katy@indiana.edu
NSF Workshop on “A Deeper Look at the Visualization of Scientific
Discoveryy in the Federal Context”, Arlington,
g
VA
Sept 12th, 2008
Overview
1
1.
F t
Features
off Science
S i
2.
3.
4.
5.
6.
Detailed Needs Analysis
Conceptualization of Science / Terms & Definitions
Data Quality, Coverage and Interlinkage
Algorithms
g
and Cyberinfrastructures
y
Case Studies and Evaluation
7
7.
St d d and
Standards
dP
Practices
ti
8.
Dissemination of Results
1. Features of Science
E. O. Wilson in Consilience: The Unity of Knowledge (1998) writes:
“F t
“Features
that
th t distinguish
di ti i h science
i
from
f
pseudoscience
p d i
are repeatability,
p t bilit economy, menuration,
ti
heuristics, and consilience.”
For a true science of science (policy) this can be detailed as:
 Repeatability refers to the fact that any science analysis, model, or map can be rerun or
regenerated by a scholar other than the author. This requires that datasets are accessible and
documented in a way that they can be recompiled, software is made available or is
doc mented in sufficient
documented
s fficient detail so that it can be reimplemented and run
r n with the exact same
parameter settings.
 Economy entails that results are presented in a form that is both simplest and most pleasing
not only for the expert but also for a general audience.
 Mensuration means properly measured, using universally accepted scales in an unambiguous
fashion.
 Heuristics – the best science stimulates further discovery.
 Consilience – those explanations and results that are consistent with each other are most likely
to survive.
2. Detailed Needs Analysis
As part of
 TLS: Towards a Macroscope for Science Policy Decision Making. NSF SBE0738111 award (Katy Börner, Weixia Huang, Kevin W. Boyack).
that aims to design
g qualitatively
q
new tools for science policy
p
makers, we are conductingg
interviews with science policy makers at different levels of the science enterprise to identify
what 'science of science' research results and tools might be most beneficial.
Each interview comprises a 40 min, audio
audio-taped,
taped, informal discussion on specific information
needs, datasets and tools currently used, and information on what a 'dream tool' might look
and feel like. There is also a pre-interview questionnaire to acquire demographics and a postinterview questionnaire to get your input on priorities.
Results of 30+ interviews should become available at the end of 2008.
But see also Paul Gemperline’s slides.
3. Conceptualization of Science / Terms & Definitions



Identify ‘basic
basic units’
units of
science, their
‘interlinkages’, and
major static and
d
dynamic
i properties
i off
interest.
Define (ideally
operationalize) major
terms such as impact,
interdisciplinarity, etc.
What conceptual
description might best
represent science? See
also
l SSpecial
i l Issue
I
off
Journal of Informetrics,
3(1), Jan 2009.
Conceptualizing Science
Hypothetical Model of the Evolution and Structure of Science, by Daniel Zeller
O di
On
display
l iin 33rdd iiteration
i off exhibit.
hibi
Authors are mortal
mortal. Papers are immortal
immortal.
Densely knit communities. The importance of weak links.
Cumulative structure of science. The unknown is rendered as monster shaped voids.
Impact of funding on science (yellow).
3. Conceptualization of Science / Terms & Definitions
Process of Analyzing and Mapping Knowledge Domains
Börner, Katy, Chen, Chaomei, and Boyack, Kevin. (2003) Visualizing Knowledge Domains. In Blaise Cronin (Ed.), Annual
Review of Information Science & Technology, Volume 37, Medford, NJ: Information Today, Inc./American Society for
Information Science and Technology, chapter 5, pp. 179-255.
4. Data Quality, Coverage and Interlinkage
“F
“From
Data
D t Silos
Sil to
t Wind
Wi d Chimes”
Chi
”
 Interlink creators, data, software/tools, publications, patents, IP, funding, etc.
 Create public databases that any scholar can use. Share the burden of data
cleaning
l
and
d federation.
f d
Scholarly Database: Web Interface
Search across publications, patents, grants.
Download records and/or
/ ((evolving)
g) co-author,, paper-citation
p p
networks.
Register for free access at https://sdb.slis.indiana.edu.
Scholarly Database: # Records & Years Covered
Datasets available via the Scholarly Database (* future feature)
D t t
Dataset
# Records
R
d
Y
Years
C
Covered
d
U d t d
Updated
Restricted
R
t i t d
Access
Medline
13,149,741
1965-2005
Ph R
PhysRev
398 005
398,005
1893 2006
1893-2006
Y
Yes
PNAS
16,167
1997-2002
Yes
JCR
59,078
1974, 1979, 1984,
1989 1994-2004
Yes
3,179,930
1976-2004
Yes*
NSF
174,835
1985-2003
Yes*
NIH
1,043,804
1972-2002
Yes*
USPTO
Yes
Total
18,021,560
1893-2006
4
Aim for comprehensive
p
time, ggeospatial,
p
and topic
p coverage.
g
3
Holdings of Scholarly Databases/Wikipedia, 1665 to 2006
#papers per publication year
5. Algorithms and Cyberinfrastructures

http://nwb.slis.indiana.edu/
http://epic.slis.indiana.edu/
See https://nwb.slis.indiana.edu/community
July 1st, 2008
6. Case Studies and Evaluation
Desirable Features
 Theoretically grounded
 Practically relevant
 Testable
T t bl hypotheses
h th
 Well documented results
 Repeatable
Types of Studies
 Individual, local, global level
 Temporal, geospatial, semantic, network analysis
Result Presentation
 Textual, tabular, charts, geo maps, science maps, other reference systems
Spatio-Temporal Information Production and Consumption of Major U.S.
Research Institutions
Börner, Katy, Penumarthy, Shashikant, Meiss, Mark and Ke, Weimao. (2006)
M i the
Mapping
h Diffusion
Diff i off Scholarly
S h l l Knowledge
K l d Among
A
Major
M j U.S.
U S Research
R
h
Institutions. Scientometrics. 68(3), pp. 415-426.
Research questions:
1 Does space still matter
1.
in the Internet age?
2. Does one still have to
j research
studyy and work at major
institutions in order to have access to
high quality data and expertise and to produce high
quality research?
3 Does
3.
D
the
h Internet
I
lead
l d to more global
l b l citation
i i
patterns, i.e., more citation links between papers
produced at geographically distant research
instructions?
Contributions:
 Answer to Qs 1 + 2 is YES.
 Answer to Qs 3 is NO.
 Novel
N l approach
h to analyzing
l i the
h dduall role
l off
institutions as information producers and
consumers and to study and visualize the diffusion
of information among them.
Mapping the Evolution of Co-Authorship Networks
Ke, Visvanath & Börner, (2004) Won 1st price at the IEEE InfoVis Contest.
16
17
Latest ‘Base Map’ of Science
Kevin W. Boyack, Katy Börner, & Richard Klavans (2007). Mapping the Structure and Evolution of
Ch i Research.
Chemistry
R
h 11th
11 h International
I
i l Conference
C f
on Scientometrics
Si
i andd Informetrics.
I f
i pp. 112-123.
112 123
 Uses combined SCI/SSCI
from 2002
• 1.07M papers, 24.5M
references, 7,300 journals
• Bibliographic coupling of
p p r aggregated
papers,
r t d tto
journals
 Initial ordination and clustering
of journals gave 671 clusters
 Coupling counts were
reaggregated at the journal
cluster level to calculate the
• (x,y) positions for each
journal cluster
• by association, (x,y)
positions
ii
for
f each
h jjournall
Math
Law
Policy
Computer Tech
Statistics
Economics
Education
CompSci
Vision
Phys-Chem
Chemistry
Physics
Psychology
Brain
Environment
Psychiatry
GeoScience
MRI
Biology
BioMaterials
BioChem
Microbiology
Pl t
Plant
Cancer
Animal
Disease &
Treatments
Virology
Infectious Diseases
Science map applications: Identifying core competency
Kevin W. Boyack, Katy Börner, & Richard Klavans (2007).
Funding patterns of the US Department of Energy (DOE)
Math
Law
Computer Tech
Policy
Statistics
Economics
CompSci
Vision
Education
Phys-Chem
Chemistry
Physics
Psychology
Brain
Environment
Psychiatry
GeoScience
MRI
Biology
GI
BioBi
Materials
BioChem
Microbiology
Plant
Cancer
Animal
Virology
Infectious Diseases
Science map applications: Identifying core competency
Kevin W. Boyack, Katy Börner, & Richard Klavans (2007).
Funding Patterns of the National Science Foundation (NSF)
Math
Law
Computer Tech
Policy
Statistics
Economics
CompSci
Vision
Education
Phys-Chem
Chemistry
Physics
Psychology
Brain
Environment
Psychiatry
GeoScience
MRI
Biology
GI
BioBi
Materials
BioChem
Microbiology
Plant
Cancer
Animal
Virology
Infectious Diseases
Science map applications: Identifying core competency
Kevin W. Boyack, Katy Börner, & Richard Klavans (2007).
Funding Patterns of the National Institutes of Health (NIH)
Math
Law
Computer Tech
Policy
Statistics
Economics
CompSci
Vision
Education
Phys-Chem
Chemistry
Physics
Psychology
Brain
Environment
Psychiatry
GeoScience
MRI
Biology
GI
BioBi
Materials
BioChem
Microbiology
Plant
Cancer
Animal
Virology
Infectious Diseases
7. Standards and Practices

Which ones exist?
Relevant Scholarly Reviews/Issues
 Börner,, Katy,
y, Chen,, Chaomei,, andd Boyack,
y c , Kevin.
v . (2003).
( 003). Visualizing
Vs
g Knowledge
w dg
Domains. In Blaise Cronin (Ed.), Annual Review of Information Science & Technology,
Medford, NJ: Information Today, Inc./American Society for Information Science and
Technology, Volume 37, Chapter 5, pp. 179-255.
http://ivl.slis.indiana.edu/km/pub/2003 borner arist.pdf
http://ivl.slis.indiana.edu/km/pub/2003-borner-arist.pdf
 Shiffrin, Richard M. and Börner, Katy (Eds.) (2004). Mapping Knowledge Domains.
Proceedings of the National Academy of Sciences of the United States of America,
101(Suppl_1). http://www.pnas.org/content/vol101/suppl_1/
 Börner,
Bö
Katy,
K
Sanyal,
S
l Soma
S
andd Vespignani,
V i
i Alessandro
Al
d (2007).
(2007) N
Networkk SScience.
i
IIn
Blaise Cronin (Ed.), Annual Review of Information Science & Technology, Information
Today, Inc./American Society for Information Science and Technology, Medford, NJ,
Volume 41, Chapter 12, pp. 537-607. http://ivl.slis.indiana.edu/km/pub/2007-bornerarist.pdf
i df
8. Dissemination of Results
Advantages for Funding Agencies
 Supports
pp
monitoringg of ((long-term)
g
) moneyy flow and research developments,
p
, evaluation of
funding strategies for different programs, decisions on project durations, funding patterns.
 Staff resources can be used for scientific program development, to identify areas for future
development, and the stimulation of new research areas.
g for Researchers
Advantages
 Easy access to research results, relevant funding programs and their success rates, potential
collaborators, competitors, related projects/publications (research push).
 More time for research and teaching.
Advantages ffor IIndustry
Ad
d
 Fast and easy access to major results, experts, etc.
 Can influence the direction of research by entering information on needed technologies
(industry-pull).
Advantages for Publishers
 Unique interface to their data.
 Publicly funded development of databases and their interlinkage.
For Society
 Dramatically improved access to scientific knowledge and expertise.
Places & Spaces: Mapping
Science
a science exhibit that introduces
people to maps of sciences,
their
h i makers
k and
d users.
http://scimaps.org.
Exhibit Curators: Dr. Katy
Börner & Elisha F. Hardy
Mapping Science Exhibit – 10 Iterations in 10 years
The Power of Maps (2005)
Science Maps for Economic Decision Makers (2008)
The Power of Reference Systems (2006)
Science Maps for Science Policy Makers (2009)
Science Maps for Scholars (2010)
S i n M
Science
Maps
p as Vi
Visuall Int
Interfaces
rf
to
t Digital
Di it l Libraries
Libr ri (2011)
Science Maps for Kids (2012)
Science Forecasts (2013)
How to Lie with Science Maps (2014)
The Power of Forecasts (2007)
So far, the exhibit has been shown in 49 venues on four continents. It is currently on display at
- National Science Foundation, 10th Floor, 4201 Wilson Boulevard, Arlington, VA, permanent display.
- National Research Council in Ottawa, Canada, April 3-Aug. 29, 2008.
- National Science Library of the Chinese Academy of Sciences, Beijing, China, May 17-Nov. 15, 2008.
Illuminated
Diagram Display
W. Bradford Paley,
Kevin W. Boyack,
Richard Kalvans, and
Katy Börner (2007)
Mapping,
Illuminating and
Illuminating,
Interacting with
Science.
SIGGRAPH 2007,
San Diego, CA.
All papers, maps, cyberinfrastructures, talks, press are linked
from http://cns.slis.indiana.edu