More than the Sum of the Parts:
Introduction to the Science of Team
Science
William M. Trochim, PhD
Professor of Policy Analysis & Management, Cornell University, Ithaca
Director of Evaluation, Weill Cornell CTSC
Wednesday, November 4, 2015
12:00-2:00pm
Griffis Faculty Club
1300 York Avenue
Overview of Talk
• Introduction to the Science of Team Science
– Mapping a Research Agenda for a Science of Team
Science
• Evaluating Team Science
– Measuring Collaboration and Transdisciplinary Integration
• Initiating Scientific Teams
– The CTSA Team Science Pilot Project
• Team Science at WCMC
– The Clinical and Translational Science Center
I. Introduction to the
Science of Team
Science
Mapping a Research Agenda
for a Science of Team Science
Project Goals
• Done as preparation for the First Annual Science of
Team Science Conference, 2010, Chicago, IL
• Develop a “roadmap” of a comprehensive research
agenda for the science of team science
– Help orient participants during the conference
– Provide a framework that can help inform the field
• Use a team science approach to mapping team
science
– Structured concept mapping
•
•
•
•
Integrative mixed methods approach
Integrates group process with statistical analysis
Utilizes web technology for distributed participation
Provides a rigorous visually interpretable result
The Setting
Define the Focus
Develop a focus
“One topic that should be part of a
comprehensive research agenda
for the science of team science
is…”
Identify Participants
Develop a focus
Identify the participants
All conference invitees (>
800) were asked to
Brainstorm & Rate
15 steering
- 63 Rated
committee
members sorted
“One topic that should be part of a comprehensive
research agenda for the science of team science is…”
Brainstorm Outcomes
Develop a focus
Identify the participants
Generate Ideas
89 using publication and bibliometric data
(e.g., citation rates, impact factors) to
assess team science
69 importance of developing multi-method
strategies to assess processes and
outcomes of team science
2 how to evaluate success of team
science-based research centers
65 measuring effectiveness of team
science on multiple levels: individual
team, impact of research, effectiveness
of team science funding programs, etc.
240
95
Organize Outcomes
Develop a focus
Identify the participants
Generate Ideas
Structure Ideas
Sort the ideas
Rate the ideas
Importance
Compute Maps
Develop a focus
Identify the participants
Generate Ideas
Structure Ideas
77
55
24
21
20
23
74
22
75 6
71
26
9
52
8
51
14
15
16
11
13
99
5
32
76
42 53
43
73
105
70
7
36
104
110108
79
8 109
37
106
33
101
54
27
107
34
45
25
88
1124748
46
94
39
49
3
78
38 115
28
95
12
50
111
40
10
103
116
84
18
100
64
2
93
82
61
62
98
60
59
1
86 41
58
87
67
68
65
91
1
63
19
92
81
2
35
89
66 29
31
72 83117
30
97
85
90
6
114
17
3
7
113
102 569
5796 4 44
56
80
4
Compute Maps
• Aggregate sorts
• Multidimensional
scaling
• Hierarchical cluster
analysis
Interpret Maps
Develop a focus
Identify the participants
Generate Ideas
21
20
2324
74
75 6 22
71
26
9
52
8
51
14
15
16
11
13
99 34
5
77
55 3276
424353
73
10537
70
27
7
33
107
25
88
36
104
110
10879
109
106
1124748
4694
101
39
49
54
45
8
3
78
95
12
50
10
38 115
28
111
40
68
3
90
7
6
114
91
17
1
113
93
81
64 2
1 63
4 102 569
2
82
86 41
19
61
57 4 44
62
65
58
5696
98
35
89
60
6629 80
87
67
31
72 83117
30
59
85 97
116
18
100
103
84
Structure Ideas
Compute Maps
92
Interpret Maps
Use Maps
Develop a focus
Identify the participants
Generate Ideas
Area 1
Area 2
4.22
Mission & Ideology
Mission & Ideology
Financing
Financing
Regionalization
Technology
Management
Regionalization
Information Services
Compute Maps
Information Services
Community & Consumer Views
Management
Technology
3.47
Structure Ideas
4.4
Community & Consumer Views
r = .51
3.56
Interpret Maps
Utilize Maps
Provide initial roadmap for
conference and the field
Who participated?
Gender
Education
Academic Discipline
Employment
Team Science Focus
Experience with Team Science
Professional Experience
How did we obtain the
results?
This initial map shows all the potential outcomes in relation
to one another
83
77
32
88
73
62
6989
65
17
46
2
58
50
79
4
72
8
29
15
45
64
13
18
63
52
3
22
10
33
67
36
59
38
75
53
14
9
70
51
40
74
86
34
24
78
90
5
81
80
31
25
49
35
93
61
1
47
94
48
76
84
21
43 27
71
68
95
23
66
16
37
42
41
55
92
56
7
54
85
12
57
11
82
19
28
30
91
6
39
26
60
44
87
20
Each point represents one of the brainstormed outcomes
83
77
32
88
73
62
6989
65
17
46
2
58
50
79
4
72
8
29
15
45
64
13
18
63
52
3
22
10
33
67
36
59
38
75
53
14
9
70
51
40
74
86
34
24
78
90
5
81
80
31
25
49
35
93
61
1
47
94
48
76
68
95
23
66
16
84
21
91
6
39
26
87
60
44
43 27
71
37
42
41
55
92
56
7
54
85
12
57
11
82
19
28
30
6 heterogeneity of team membership
20
Conceptually similar outcomes are in close proximity
83
77
32
88
73
62
6989
65
17
46
2
58
50
79
4
72
8
29
15
45
64
13
18
63
52
3
22
10
33
67
36
59
38
75
53
14
9
70
86
34
24
78
90
5
81
80
31
25
49
35
93
16
61
1
47
94
48
76
68
95
23
66
20
51
40
74
37
42
41
55
92
56
7
54
85
12
57
11
82
19
28
30
84
21
91
6
39
26
87
60
44
43 27
71
6 heterogeneity of team membership
91 status differences and power dynamics within the team
44 team member interchangeability
Conceptually different outcomes are further apart
83
77
32
88
73
62
6989
65
17
46
2
58
50
79
4
72
8
58 evaluating and learning
from successful teams
29
15
45
64
13
18
63
52
3
22
10
33
67
36
59
38
75
53
14
9
70
86
34
24
78
90
5
81
80
31
25
49
35
93
16
81 co-authorship and multi-PI authorship in
team science
90 use of collaborative computerized tools to
support and enhance team science
61
1
47
94
48
76
68
95
23
66
20
51
40
74
37
42
41
55
92
56
7
54
85
12
57
11
82
19
28
30
84
21
91
6
39
26
87
60
44
43 27
71
6 heterogeneity of team membership
91 status differences and power dynamics within the team
44 team member interchangeability
The outcomes are organized into clusters
83
77
32
88
73
62
6989
65
17
46
2
58
50
79
4
72
8
29
15
45
64
13
18
63
52
3
22
10
33
67
36
59
38
75
53
14
9
70
86
34
24
78
90
5
81
80
31
25
49
35
93
16
61
1
47
94
48
76
68
95
23
66
20
51
40
74
37
42
41
55
92
56
7
54
85
12
57
11
82
19
28
30
84
21
91
6
39
26
87
60
44
43 27
71
This map shows each of 95 statements grouped into seven clusters by hierarchical cluster analysis.
The Map Results
Measurement & Evaluation of Team Science
77
83
17
32
88
46
79
4
29
64
89
69
2
65
8
46
17
13
58
50
83
63
88
3
79
52
18
77
32
29
4
64
22
6989
65
8
2
58
50
13
18
63
3
52
22
using publication and bibliometric data (e.g., citation rates, impact factors) to assess team science
importance of developing multi-method strategies to assess processes and outcomes of team science
how to evaluate success of team science-based research centers
measuring effectiveness of team science on multiple levels: individual team, impact of research, effectiveness of team
science funding programs, etc.
measurement of key constructs (e.g., collaboration, disciplinarity, team effectiveness, personal/behavioral
characteristics, team processes, readiness; synergy, productivity, shared knowledge)
best approach(es) to assessing scientific teams within an institution
research on methodology and measurement of team science
evaluation of team science and its impacts
evaluating and learning from successful teams
how to measure an increase in team science activity and collaboration at an institution, in comparison with other
institutions
how to evaluate existing and new tools
how network information can provide insight into performance and evaluation of teams
key performance indicators to encourage team science evaluation into individual development and professional growth
comparing the effects of team science versus traditional science in advancing scientific knowledge
infrastructures to capture relevant data to better assess team science outcomes
to assess whether the findings produced by team science are more broadly disseminated, as compared to traditional
science
social network analysis of scientific teams
strengthening the research methods for studying scientific teams (e.g., using quasi-experimental methods)
how to use team science approaches and methods in the investigation of team science
economic value created by team science
how to demonstrate an effective team in a grant proposal
the availability of organizational structure data as a data source
approaches for capturing the expertise of team science leaders
Structure & Context for Teams
41
54
70
42
11
40
9
67
20
53
whether collaborative spaces for team science encourage collaboration
use and impact of community-based organizations and community clinical practices in teams
what types of team organizations are best at facilitating team science
the impact of team size on process and outcomes in team science
the effects of the type and complexity of research question on team science
how research networking tools can enhance team science
the relationship between productivity and the composition of teams
the effect of research centers in promoting a team science approach
the relationships among creativity, innovation and the composition of teams
contextual/situational factors that influence the effectiveness of team collaboration
28
30
67
57
56
7
11
54
53
14
41
70
40
30
56
7
28
14
37
57
37
42
keys for success in team science
effects of sustained, hard team work
status of the team as it appears to external individuals and groups
a study of team science outcomes with junior versus senior Pis
how the changing ecology and structure of teams influence future scientific collaborations
the network characteristics of productive science team members and subgroups
how team dynamics can impact science
9
20
Characteristics & Dynamics of Teams
91
44
21
26
60
94
23
84
68
87
27
47
24
6
39
48
43
49
34
95
51
status differences and power dynamics within the team
team member interchangeability
how roles in teams are defined and communicated, and by whom
the influence of research team morale
personal and behavioral factors in team science collaborations
different types of conflicts that occur in scientific teams and how to address these effectively
communication styles in teams
leadership characteristics that drive effective team science
optimal team composition (e.g., specialists, generalists, boundary spanners) to enable use of diverse expertise.
social skills and competencies required for successful team science
the psychological and personality factors associated with being an effective team scientist
team member physical proximity (co-location)
ideal composition of scientific teams
heterogeneity of team membership
how teams grow, shrink, expire over time
issues to consider when initiating or building a new team
collaborative readiness factors
finding potential/likely research collaborators
what factors contribute to the development of trust in different collaborations
why people join teams
gender differences in team contributions
34
24
49
68
95
23
47
94
48
51
84
21
43 27
91
6
39
26
60
44
87
Management & Organization for Teams
76
93
78
61
35
66
86
1
71
value of rotating team leadership
the management of scientific teams
how to sustain scientific teams
membership in multiple, potentially overlapping, potentially conflicting teams
organizational policies that foster team science
virtual organizations and team science
formal vs. informal organizational structures of institutions
types of organizational structures of team science
disciplinary language barriers in team science
86
78
35
66
93
61
1
71
76
Institutional Support &
Professional Development for Teams
31
74
25
80
85
55
92
5
75
81
12
90
16
incentives and incentive systems for team science
resources and infrastructure needed within and across institutions to promote collaboration and team science
how the university tenure and promotion system can be restructured to encourage team science
processes and methods that encourage and support teams (e.g., group activities, scientific conferences, grant
opportunity distribution, systems-based approaches
relationships between team science in the academy and industry
timing, with regards to investigator career stage, in team science
ethical issues in conducting team science (e.g. intellectual property ownership, defining collaborative relationships;
attributing credit for work)
training and education issues in team science
funding to support the science of team science, research on team science
co-authorship and multi-PI authorship in team science
the effects of team science on the scientist's work and career
use of collaborative computerized tools to support and enhance team science
individual benefit/risk analysis to engaging in team science
85
12
92
75
74
90
5
81
80
31
25
16
Disciplinary Dynamics & Team Science
82
38
59
36
33
19
10
variations in team science related to disciplinarity
how to overcome disciplinary traditions to move toward interdisciplinary traditions
using team science and interdisciplinary research to support emerging areas of science
applying what is known about teams in different disciplines (e.g., management) and contexts (e.g., international)
relationships and connections between multi-, inter- and transdisciplinary research efforts and team science
how best to disseminate findings and best practices from the science of team science
understanding differences between intra- vs. interinstitutional scientific teams
10
33
36
59
19
82
38
Definitions & Models of Team Science
62
73
72
15
45
72
45
73
62
15
theories and models of team science
best practices of team science
developing testable hypotheses about team science
the definitions of team, scientific team, and team science
definition of different types of disciplinarity (interdisciplinary; multidisciplinarity; transdisciplinarity)
Cluster Map
Definitions &
Models of Team
Science
Measurement & Evaluation
of Team Science
Disciplinary Dynamics
& Team Science
Structure & Context
for Teams
Institutional Support &
Professional Development for
Teams
Management &
Organization for
Teams
Characteristics &
Dynamics of Teams
Regional Interpretation
Meta-Issues
Measurement & Evaluation
of Team Science
Definitions &
Models of Team
Science
The Team
Disciplinary Dynamics
& Team Science
Structure & Context
for Teams
Institutional Support &
Professional Development for
Teams
Nuts & Bolts
Management &
Organization for
Teams
Support
Characteristics &
Dynamics of Teams
Ratings and Pattern Matches
Importance Point Rating Map
Point Legend
Layer
Value
1
2.58 to 2.95
2
2.95 to 3.33
3
3.33 to 3.70
4
3.70 to 4.07
5
4.07 to 4.44
Definitions & Models
of Team Science
83
77
17
32
88
73
62
8
46
2
Measurement & Evaluation
of Team Science
58
50
79
4
72
29
15
45
64
13
18
63
52
3
Structure & Context for Teams
22
10
Disciplinary Dynamics
& Team Science
33
67
59
38
53
14
9
70
40
74
34
24
78
90
5
81
25
Institutional Support & Professional
Development for Teams
31
49
35
23
66
93
16
61
1
47
94
48
76
20
51
86
80
37
42
41
55
92
56
7
54
85
12
57
11
82
75
28
30
36
19
6989
65
84
21
91
6
68
95
39
26
87
60
44
43 27
71
Management & Organization
for Teams
Characteristics & Dynamics of Teams
Importance Cluster Rating Map
Definitions &
Models of Team
Science
Measurement & Evaluation
of Team Science
Disciplinary Dynamics
& Team Science
Structure & Context
for Teams
Institutional Support &
Professional Development for
Teams
Cluster Legend
Layer
Value
1
3.43 to 3.49
2
3.49 to 3.55
3
3.55 to 3.61
4
3.61 to 3.68
5
3.68 to 3.74
Management &
Organization for
Teams
Characteristics &
Dynamics of Teams
Top Ten Statements By Average Importance
8
30
13
45
65
2
35
9
3
74
measurement of key constructs (e.g., collaboration, disciplinarity, team
effectiveness, personal/behavioral characteristics, team processes,
readiness; synergy, productivity, shared knowledge) (4.44)
keys for success in team science (4.23)
evaluation of team science and its impacts (4.22)
best practices of team science (4.16)
measuring effectiveness of team science on multiple levels: individual
team, impact of research, effectiveness of team science funding
programs, etc. (4.16)
how to evaluate success of team science-based research centers (4.14)
organizational policies that foster team science
(4.13)
the relationship between productivity and the composition of teams (4.11)
comparing the effects of team science versus traditional science in
advancing scientific knowledge
(4.08)
resources and infrastructure needed within and across institutions to
promote collaboration and team science (4.03)
Gender
Female
Male
3.8
3.66
Measurement & Evaluation of Team Science
Measurement & Evaluation of Team Science
Definitions & Models of Team Science
Institutional Support & Professional
Development for Teams
Definitions & Models of Team Science
Disciplinary Dynamics & Team Science
Disciplinary Dynamics & Team Science
Structure & Context for Teams
Structure & Context for Teams
Management & Organization for Teams
Management & Organization for Teams
Characteristics & Dynamics of Teams
Characteristics & Dynamics of Teams
Institutional Support & Professional
Development for Teams
3.46
3.35
r = .5
Education
Bachelors or Masters
Doctorate
3.7
3.75
Measurement & Evaluation of Team Science
Measurement & Evaluation of Team Science
Definitions & Models of Team Science
Institutional Support & Professional
Development for Teams
Disciplinary Dynamics & Team Science
Disciplinary Dynamics & Team Science
Structure & Context for Teams
Structure & Context for Teams
Management & Organization for Teams
Institutional Support & Professional
Development for Teams
Characteristics & Dynamics of Teams
Management & Organization for Teams
Definitions & Models of Team Science
Characteristics & Dynamics of Teams
3.18
3.47
r = .25
Professional Experience
Below Median
Above Median
3.85
3.61
Measurement & Evaluation of Team Science
Measurement & Evaluation of Team Science
Disciplinary Dynamics & Team Science
Definitions & Models of Team Science
Definitions & Models of Team Science
Structure & Context for Teams
Institutional Support & Professional
Development for Teams
Institutional Support & Professional
Development for Teams
Management & Organization for Teams
Disciplinary Dynamics & Team Science
Structure & Context for Teams
Management & Organization for Teams
Characteristics & Dynamics of Teams
Characteristics & Dynamics of Teams
3.5
3.33
r = .7
Employment Sector
Academics
Government
3.8
3.67
Measurement & Evaluation of Team Science
Measurement & Evaluation of Team Science
Definitions & Models of Team Science
Definitions & Models of Team Science
Structure & Context for Teams
Institutional Support & Professional
Development for Teams
Disciplinary Dynamics & Team Science
Disciplinary Dynamics & Team Science
Management & Organization for Teams
Institutional Support & Professional
Development for Teams
Structure & Context for Teams
Management & Organization for Teams
Characteristics & Dynamics of Teams
Characteristics & Dynamics of Teams
3.55
3.27
r = .69
Team Science Focus
Practice
Science
3.73
3.82
Measurement & Evaluation of Team Science
Definitions & Models of Team Science
Measurement & Evaluation of Team Science
Institutional Support & Professional
Development for Teams
Structure & Context for Teams
Disciplinary Dynamics & Team Science
Management & Organization for Teams
Structure & Context for Teams
Disciplinary Dynamics & Team Science
Management & Organization for Teams
Characteristics & Dynamics of Teams
Institutional Support & Professional
Development for Teams
Definitions & Models of Team Science
Characteristics & Dynamics of Teams
3.41
3.4
r = -.08
Experience with Teams
None
3.67
1 or 2
4
3-5
3.67
Characteristics &
Dynamics of Teams
More Than 5
3.77
Measurement &
Evaluation of Team
Science
Management &
Organization for Teams
Definitions & Models
of Team Science
Definitions & Models of
Team Science
Institutional Support &
Professional
Development for
Teams
Measurement & Evaluation
of Team Science
Structure &
Context for Teams
Structure & Context for
Teams
Disciplinary
Dynamics & Team
Science
Institutional Support &
Professional Development
for Teams
Management &
Organization for
Teams
Disciplinary Dynamics &
Team Science
3.45
Characteristics &
Dynamics of Teams
3.28
3.41
3.42
Academic Discipline
Biomedical
3.96
Social
3.77
Physical
3.67
Humanities
4.47
Engineering/Math
3.95
Measurement &
Evaluation of Team
Science
Business Administration
3.5
Measurement &
Evaluation of Team
Science
Institutional
Support &
Professional
Development for
Teams
Institutional
Support &
Professional
Development for
Teams
Definitions &
Models of Team
Science
Management &
Organization for
Teams
Structure &
Context for
Teams
Disciplinary
Dynamics & Team
Science
Characteristics &
Dynamics of
Teams
Structure &
Context for
Teams
Disciplinary
Dynamics &
Team Science
Characteristics &
Dynamics of
Teams
Definitions &
Models of Team
Science
Management &
Organization for
Teams
3.56
3.44
3.23
3.59
2.9
2.64
Using the Maps:
The Science of Team Science
Conference Program
Regional Interpretation
Meta-Issues
Measurement & Evaluation
of Team Science
Definitions &
Models of Team
Science
The Team
Disciplinary Dynamics
& Team Science
Structure & Context
for Teams
Institutional Support &
Professional Development for
Teams
Nuts & Bolts
Management &
Organization for
Teams
Support
Characteristics &
Dynamics of Teams
Regional Interpretation - Panel X1
Meta-Issues
Measurement & Evaluation
of Team Science
Definitions &
Models of Team
Science
The Team
Disciplinary Dynamics
& Team Science
Structure & Context
for Teams
Institutional Support &
Professional Development for
Teams
Nuts & Bolts
Management &
Organization for
Teams
Support
Characteristics &
Dynamics of Teams
Regional Interpretation – Panel X2
Meta-Issues
Measurement & Evaluation
of Team Science
Definitions &
Models of Team
Science
The Team
Disciplinary Dynamics
& Team Science
Structure & Context
for Teams
Institutional Support &
Professional Development for
Teams
Nuts & Bolts
Management &
Organization for
Teams
Support
Characteristics &
Dynamics of Teams
Regional Interpretation – Panel X3
Meta-Issues
Measurement & Evaluation
of Team Science
Definitions &
Models of Team
Science
The Team
Disciplinary Dynamics
& Team Science
Structure & Context
for Teams
Institutional Support &
Professional Development for
Teams
Nuts & Bolts
Management &
Organization for
Teams
Support
Characteristics &
Dynamics of Teams
Regional Interpretation - Panel X4
Meta-Issues
Measurement & Evaluation
of Team Science
Definitions &
Models of Team
Science
The Team
Disciplinary Dynamics
& Team Science
Structure & Context
for Teams
Institutional Support &
Professional Development for
Teams
Nuts & Bolts
Management &
Organization for
Teams
Support
Characteristics &
Dynamics of Teams
Regional Interpretation - Panel X5
Meta-Issues
Measurement & Evaluation
of Team Science
Definitions &
Models of Team
Science
The Team
Disciplinary Dynamics
& Team Science
Structure & Context
for Teams
Institutional Support &
Professional Development for
Teams
Nuts & Bolts
Management &
Organization for
Teams
Support
Characteristics &
Dynamics of Teams
Regional Interpretation - Panel X6
Meta-Issues
Measurement & Evaluation
of Team Science
Definitions &
Models of Team
Science
The Team
Disciplinary Dynamics
& Team Science
Structure & Context
for Teams
Institutional Support &
Professional Development for
Teams
Nuts & Bolts
Management &
Organization for
Teams
Support
Characteristics &
Dynamics of Teams
Regional Interpretation - Panel X7
Meta-Issues
Measurement & Evaluation
of Team Science
Definitions &
Models of Team
Science
The Team
Disciplinary Dynamics
& Team Science
Structure & Context
for Teams
Institutional Support &
Professional Development for
Teams
Nuts & Bolts
Management &
Organization for
Teams
Support
Characteristics &
Dynamics of Teams
Publication on Mapping
Publications on SciTS Conference
II. Evaluating Team
Science
Measuring Collaboration and
Transdisciplinary Integration
Transdisciplinary Tobacco Use Research Centers
• NIH-funded Center Grant Initiative (P50)
• Co-funded by NCI, NIDA, NIAAA
• Seven centers (in 2002)
–
–
–
–
–
–
–
University of Wisconsin
Roswell Park Cancer Institute Corporation
University of Minnesota
University of Southern California
The Miriam Hospital
University of Pennsylvania
Yale University
Evaluation of TTURCs
•
•
•
•
We needed to evaluate the initiative
Began with a concept map
Transformed that into a logic model
Identified that collaboration and transdisciplinary
integration were two of the immediate “markers” of
the effects of the initiative
• There were no good measures of those constructs
• As part of the evaluation, we developed a measure
Health Impacts
Communication
Science & Models
TD Research Institutionalization
Structure
Professional
Validation Recognition
Outcome
The following sequence of
slides shows the
development of the logic
model for the evaluation.
We begin with the concept
map derived from the
process involving
Health Outcomes
participants from the
Transdisciplinary Tobacco
Use Research Centers
Policy Implications
Improved Interventions
(TTURCs) and other
stakeholders (e.g.,
funders, researchers). The
Translation to Practice
13 clusters of outcomes
are shown here.Communications
Methods
Training
Collaboration
Transdisciplinary Integration
Scientific
Integration
Process
Collaboration
Publications
Professional
Validation
Collaboration
Communication
Recognition
Communications
Training
It is convenient to show
more immediate process
outputs on the left and
more long-term outcomes
on the right. To achieve
this, we rotated the original
map 90 degrees
clockwise. This sets us up
for a “logic model” flow as
depicted in the next slide.
Collaboration
TD Research
Institutionalization
Publications
Policy Implications
Health
Outcomes
Translation to Practice
Transdisciplinary
Integration
Methods
Scientific
Integration
Process
Improved Interventions
Science & Models
Health
Impacts
Structure
Outcome
TTURC
Logic
Model
Recognition
Communication
TD Research
Institutionalization
Professional
Validation
Communication
Collaboration
Training
Publications
Methods
Collaboration
Improved
Interventions
Science &
Models
Transdisciplinary
Integration
Scientific
Integration
Policy
Implications
Health
Impacts
Translation
To Practice
Health
Outcomes
Recognition
Communication
TD Research
Institutionalization
Professional
Validation
Communication
Collaboration
Training
Collaboration
Publications
We had no way to
measure these
Methods
immediate markers!Health
Impacts
Immediate Markers
Health
Outcomes
Improved
Interventions
Science &
Models
Transdisciplinary
Integration
Policy
Implications
Translation
To Practice
Scientific
Integration
Intermediate Markers
Long-Term Outcomes
Collaboration
90
71
77
53
96
42
The diversity of disciplines participating and collaborating in tobacco research is 0.49
increased.
Collaborative efforts occur that include non-TTURC scientists.
0.55
Links are established to research centers that are not funded through the
0.56
TTURC mechanism.
Sustainable transdisciplinary collaborations occur between and within existing 0.56
centers.
Scientists who have not previously studied nicotine addiction or tobacco
0.70
cessation begin to do so.
Ease of communication exists both across and between TTURCs
0.80
Average:
0.61
42
77
2
71
96
90
53
Collaboration Scales
Please evaluate the collaboration within your center by indicating if the collaboration is
(1) inadequate, (2) poor, (3) satisfactory, (4) good, (5) excellent
___ Acceptance of new ideas
___ Communication among collaborators
___ Ability to capitalize on the strengths of different researchers
___ Organization or structure of collaborative teams
___ Resolution of conflicts among collaborators
___ Ability to accommodate different working styles of collaborators
___ Involvement of collaborators from outside the center
___ Involvement of collaborators from diverse disciplines
___ Productivity of collaboration meetings
___ Productivity in developing new products (e.g., papers, proposals, courses)
___ Overall productivity of collaboration
Please rate your views about collaboration with respect to your center-related research by indicating if you (1) strongly disagree, (2)
somewhat agree, (3) not sure, (4) somewhat agree, or (5) strongly agree with the statement.
___ In general, collaboration has improved your research productivity.
___ In general, collaboration has improved the quality of your research.
___ Collaboration has posed a significant time burden in your research.
___ You are comfortable showing limits or gaps in your knowledge to those with whom you collaborate.
___ In general, you feel that you can trust the colleagues with whom you collaborate.
___ In general, you find that your collaborators are open to criticism.
___ In general, you respect your collaborators.
Analysis of Collaboration Scales
• Confirmatory factor
analysis using
LISREL, maximum
likelihood estimation
• Internal consistency
(Cronbach’s alpha)
• Results support a
three-factor model
Transdisciplinary Integration
85
93
13
84
7
29
20
49
28
72
8
Tobacco use and nicotine research scientists integrate research from fields
different from their own.
More scientific papers appear with transdisciplinary representation,
conceptualization and measurement.
Research and theory papers prepared by TTURC scientists integrate theories
from diverse scholarly domains.
Outcomes reflect synergistic potential of transdisciplinary research.
New transdisciplinary research proposals are derived from truly novel pilot work
from the centers.
Transdisciplinary research is continued in a subsequent grant application.
Tobacco control research and programs are informed by the role of genetics,
neuroscience and pharmacology in addiction.
Investigators from different disciplines collaborate on new RO1 proposals.
The TTURCs serve as a model to promote transdisciplinary research.
Some of the difficulties in multi-center, multi-disciplinary approaches are
identified and partial solutions developed.
Collaborative cross-center papers get published in peer-reviewed journals.
Average:
0.34
0.35
0.36
0.40
0.41
0.41
0.41
0.44
0.55
0.57
0.74
0.45
93
29
72
8
7
84
28 85
4
49
13
20
Transdisciplinary Integration Scale
For all items, respondents were asked to Please rate the following attitudes about
transdisciplinary research by indicating if you (1) strongly disagree, (2) somewhat agree, (3) not
sure, (4) somewhat agree, or (5) strongly agree with the statement.
___
___
___
___
___
___
___
___
___
___
___
___
___
___
___
I would describe myself as someone who strongly values transdisciplinary collaboration.
Transdisciplinary research interferes with my ability to maintain knowledge in my primary area.
I tend to be more productive working on my own rather than working as a member of a transdisciplinary research
team.
In a transdisciplinary research group, it takes more time to produce a research article.
Transdisciplinary research stimulates me to change my thinking.
I have changed the way I pursue a research idea because of my involvement in transdisciplinary research.
Transdisciplinary research has improved how I conduct research.
I am optimistic that transdisciplinary research among TTURC participants will lead to valuable scientific outcomes
that would not have occurred without that kind of collaboration.
Participating in a transdisciplinary team improves the interventions that are developed.
Because of my involvement in transdisciplinary research, I have an increased understanding of what my own
discipline brings to others.
My transdisciplinary collaborations are sustainable over the long haul.
Generally speaking, I believe that the benefits of transdisciplinary scientific research outweigh the inconveniences
and costs of such work.
I am comfortable working in a transdisciplinary environment.
Overall, I am pleased with the effort I have made to engage in transdisciplinary research.
TTURC members as a group are open-minded about considering research perspectives from fields other than their
own.
Analysis of Transdisciplinary Integration
• Confirmatory factor
analysis using LISREL,
maximum likelihood
estimation
• Internal consistency
(Cronbach’s alpha)
• Results support a onefactor model
Published Results
Mâsse, L. C., Moser, R. P., Stokols, D., Taylor, B. K., Marcus, S. E., Morgan, G. D., Hall, K.L.,
Croyle, R.T., Trochim, W. (2008). Measuring Collaboration and Transdisciplinary Integration in
Team Science. American Journal of Preventive Medicine, 35(2, Supplement 1), S151-S160
III. Initiating Scientific
Teams
The CTSA Team Science Pilot
Project
Overview of Pilot Study
• Emerged from work of the CTSA Strategic Goal 3
Committee on Collaboration
• The seven CTSA affiliated institutions participating
in this study are:
–
–
–
–
–
–
–
University of Texas Medical Branch at Galveston
Northwestern University
University of New Mexico
Medical University of South Carolina
Weill Cornell Medical College
University of North Carolina
Virginia Commonwealth University
Background of the Study
•
•
•
•
Many major biomedical science advances are often the result of multi-investigator
studies, and collaborative work has very high scientific impact and utility (Jones,
Wutchy, & Uzzi, 2003; Wutchy, Jones, & Uzzi, 2007).
Many of the CTSA affiliated institutions have attempted to educate researchers in
the use of team science – yet there has been relatively little coordinated effort to
provide an agreed upon curricula or uniform educational program
There is a profound need for translational scientists to develop team-related skills
(e.g., conflict resolution, team processes, etc.) – most of which are not addressed
in typical graduate school curricula nor available through continuing education for
scientists
Some research (Begg et al., 2013) indicates that only half of the CTSA affiliated
institutions actually offer some form of training in team science skills
Objectives of the Study
• The overall objectives of this pilot study involve:
1. Determination of the value and effectiveness of a team
science educational program for existing translational
science teams
2. Determination of how team science training can be
improved in both content and educational process for
future team science training interventions
Phases of the Study and Timeline
Potential Benefits of the Study
•
•
•
While direct benefit from receiving training by participation in the study cannot be guaranteed, the
potential benefits are largely centered around enhanced knowledge and enhanced team capacity.
No compensation will be provided as a benefit.
Specific knowledge and awareness is expected in the areas of:
–
–
–
–
–
–
–
–
–
–
•
•
Status of Team Science
Team Assembly
Team Management
Team Evaluation
Incentives and Challenges
Team Leadership
Team Building
Managing Conflict
Sustaining Team Engagement
Evaluating Team Success
Participants should obtain an overview of their current team dynamics and processes that facilitate and
inhibit the collaborative efforts of their team.
Participants will obtain needs assessment of team developmental challenges (diagnosed needs from
structured exercises) and a written action plan to implement to address team challenges.
Expected Time Requirements
•
•
•
•
•
•
•
Participation in a REDCap pre-training web survey (40-60 minutes)
Participation in reviewing a PowerPoint presentation and a YouTube presentation
providing an overview to the study (30 minutes)
Participation in an online training module involving team science principles,
including a pre- and post-assessment and knowledge check (120-240 minutes)
Participation in an online training module involving team dynamics in clinical
research, including a pre- and post-assessment and knowledge check (120-180
minutes)
Participation in a face-to-face experiential exercise involving team collaboration
and discussion of collation practices with your team (160-180 minutes)
Participation in a face-to-face structured exercise involving team diagnosis and
action plan to address developmental needs within your team (120-180 minutes)
Participation in a REDCap post-training survey (40-60 minutes)
Online and Face-to-Face Components
The Science of Team Science Module
• Team Science 101
• Assembling a Team
• Managing a Team
• Evaluating Team Performance
• Incentives and Challenges
Clinical Science Module
• Assembling the Team and Developing Team Leadership
• Team Building Activities
• Managing Conflict Upward and Downward
• Sustaining Team Engagement and Communication
http://www.teamscience.net/
IV. Team Science at
WCMC
The Clinical and Translational
Science Center
The Clinical and Translational Science Center
• Types of Services Provided
Category
Planning Your Study
Services
Biostatistical Services, Research Ancillary Support, Clinical
Research Study Budget Development, Regulatory Support,
Ethics Consultation, Research Participant Advocacy, Research
and Scholarly Communication Support, Collaborator Search
Tool, Investigational Products Core Facility, Translational
Research Support Team (TREST), Education/Training
Conducting Your
Study
Core Laboratory services, 3D Printing, Adult Inpatient/Outpatient
Unit, Pediatric Inpatient/Outpatient Unit, Research Nutrition
Services, Research Participant Advocacy, Research Match,
Translational Research Support Team (TREST), Outreach and
Recruitment with Community Partners, REDCap, Ohmage,
REDCap Plus (REDCap with Epic Feed), EMR Recruitment
Alerts, Cohort Discovery (i2b2), CTSC ARCH Workshops,
Computational Biology Resources, Education/Training
Closing Your Study
Biostatistical Services, Editorial Consulting Services, Publishing,
Education/Training
Current CTSA Consortium Efforts
• Newly formed workgroups on team science in the
CTSA Domain Task Forces
• X02/U01 Grant Proposals from the Team Science
Pilot Study Group
• Efforts at the WCMC CTSC
Resources in Team Science
• National Cancer Institute’s Team Science Toolkit
Great general resource center that includes tools, measures, bibliography, link to listserv, communications
materials and, of course, the Toolkit
https://www.teamsciencetoolkit.cancer.gov/Public/Home.aspx
• Fundamentals of Team Science Workshop (UCF/NSF, June 2015)
Includes Powerpoint slides from all sessions
http://www.fundamentalsteamscience.org/
• SciTS Resources Page
Good introduction to the field and the annual conference
http://www.scienceofteamscience.org/scits-a-team-science-resources
• NIH Collaboration & Team Science: A Field Guide
A somewhat dated and introductory guide for scientific teams
https://ccrod.cancer.gov/confluence/download/attachments/47284665/TeamScience_FieldGuide.pdf?versi
on=2&modificationDate=1285330231523&api=v2
• CTSA Online Assistance for Leveraging the Science of Collaborative
Effort (COALESCE) TeamScience Online Training
Training being pilot tested in cross-CTSA pilot study
http://www.teamscience.net/
For additional information on this talk, contact:
William Trochim
wmt1@cornell.edu
wit2005@med.cornell.edu
Description
In the past several decades, biomedical research has increasingly transitioned from a single investigator model to one that requires
collaboration of complex multidisciplinary teams. But scientific collaboration is more than a “sum of the parts” endeavor, and we are only
beginning to learn about the best ways for researchers to work effectively together. In order to better organize what we do know and
advance our knowledge in this area, a new field called the Science of Team Science (SciTS) was initiated in 2010. This field conducts and
summarizes research across the entire spectrum of relevant issues, including (but not limited to): the formation and management of
scientific teams; their structure and dynamics; institutional support and professional development; inter-, multi-, and trans-disciplinarity; and
the measurement and evaluation of team science. This introduction to team science describes the collaborative formation of this field in
2010 through the use of a unique team science method known as concept mapping, presents the development of a measure of
collaboration among scientists on a team, and discusses the current efforts of the NIH-funded Clinical and Translational Science Award
“hubs” – including the Weill Cornell Clinical and Translational Science Center (CTSC) – to provide training and support to emerging and
existing scientific teams. William Trochim is a Professor of Policy Analysis & Management at Cornell in Ithaca and the Director of Evaluation
for the Weill Cornell CTSC. He is the developer of the concept mapping methodology used to “map” the key research issues in the new field
of team science, is the co-developer of the measure of scientific collaboration and transdisciplinary integration presented here, and is a
member of the CTSA Team Science Affinity Workgroup that is managing the current cross-CTSA multi-site pilot study on team science. This
introduction to the field is designed both to introduce the research in this exciting new area and to provide resources for students and faculty
engaged in collaborative research efforts.
Suggested (and optional!) Background Readings:
•
•
•
•
Falk-Krzesinski, H.J., Contractor, N., Fiore, S.M., Hall, K.L., Kane, C., Keyton, J., Klein, J., Spring, B., Stokols, D., and Trochim, W.
(2011). Mapping a Research Agenda for the Science of Team Science. Research Evaluation 20, 143-156.
Falk-Krzesinski, H. J., Börner, K., Contractor, N., Cummings, J. N., Fiore, S. M., Hall, K. L., Keyton, J., Spring, B., Stokols, D.,
Trochim, W., and Uzzi, B. (2010). Advancing the Science of Team Science. Clinical and Translational Sciences, 3, 5, 263-266.
Börner, K., Contractor, N., Falk-Krzesinski, H., Fiore, S.M., Hall, K., Keyton, J., Spring, B., Stokols, D., Trochim, W., and Uzzi, B., A
Multi-Level Systems Perspective for the Science of Team Science, Sci Transl Med, 15 September 2010: Vol. 2, Issue 49, p. 49cm24.
Mâsse, L. C., Moser, R. P., Stokols, D., Taylor, B. K., Marcus, S. E., Morgan, G. D., Hall, K.L., Croyle, R.T., Trochim, W. (2008).
Measuring Collaboration and Transdisciplinary Integration in Team Science. American Journal of Preventive Medicine, 35(2,
Supplement 1), S151-S160.
Biographical Sketch
WILLIAM TROCHIM is Professor of Policy Analysis and Management at Cornell University and
Professor of Healthcare Policy and Research at the Weill Cornell Medical Center. He is the Director of
Evaluation for the Weill Cornell Clinical and Translational Science Center, and the Director of the
Cornell Office for Research on Evaluation. His research focuses on the development and assessment
of evaluation and research methods and their use for managing and enhancing science and
biomedical research in the twenty-first century. Dr. Trochim has developed quasi-experimental
alternatives to randomized experimental designs, including the regression discontinuity and regression
point displacement designs. He created a structured conceptual modeling approach that integrates
participatory group process with multivariate statistical methods to generate concept maps and models
useful for theory development, planning and evaluation. Dr. Trochim is currently conducting research
with the National Institutes of Health on the evaluation of biomedical clinical and translational research
and with the National Science Foundation on evaluating science, technology, engineering and
mathematics (STEM) education programs. In addition, in recent work sponsored by the National
Science Foundation, he created and tested an approach to developing any research or evaluation
project that incorporates evolutionary systems theory approaches and includes a step-by-step systems
evaluation protocol and a web-based cyberinfrastructure. He has published widely in the areas of
applied research methods and evaluation including the books: Research Design for Program
Evaluation: The Regression-Discontinuity Approach (1984), Concept Mapping for Planning and
Evaluation (2005), Research Methods: The Concise Knowledge Base (2005), and the Research
Methods Knowledge Base (2007). Dr. Trochim served for four years on the American Evaluation
Association’s Board of Directors, was the chair of several AEA committees (electronic
communications, public affairs), and was President of AEA (2008).