How the Science of Teams
Can Inform Team Science
Nancy J. Cooke
March 13, 2015
Team Science Retreat
Wake Forest School of Medicine of
Wake Forest Baptist Medical Center
Overview
•
•
•
•
Why Team Science?
Update on NRC Study
My research and experience
A Multi-Level Systems Perspective
Micro Level: Challenges and Support
Meso Level: Challenges and Support
Macro Level: Challenges and Support
• Conclusion
Why Team Science?
• Today’s problems require a team of multidisciplinary
individuals
• Team Science is impactful (highly cited; Wuchty, et
al., 2007; Uzzi, et al., 2013)
• Team Science is innovative (Uzzi, 2013)
• Team Science is productive (Hall, et al., 2012)
• Team Science has broad reach/uptake (Stipelman, et
al, 2014)
Why Team Science?
But…
•Not all science requires a team
•Team science is difficult
Enhancing
the Effectiveness
of
An Update
on the NRC
Team
Science:
Symposium
at
ICPS
Study of Team Science
March 13, 2015
Board on Behavioral, Cognitive, and Sensory Sciences
Division of Behavioral and Social Sciences and Education
National Research Council
Study Background
• Rationale: Clear need to provide research-based
guidance to improve the processes and outcomes of
team science
• Sponsors: NSF, Computer and Information Systems
and Engineering Directorate and Elsevier
• Goal: Enhance the effectiveness of collaborative
research in science teams, research centers, and
institutes.
• Audiences: NSF and other public and private research
funders; the scientific community; the SciTS
community; universities; research centers and
institutes.
6
Committee Charge
Conduct a consensus study on the science of team science to recommend
opportunities to enhance the effectiveness of collaborative research in science
teams, research centers, and institutes… Explore:
•How individual factors influence team dynamics, effectiveness and productivity
•Factors at the team, center, or institute level that influence effectiveness
•Different management approaches and leadership styles that influence
effectiveness
•How tenure and promotion policies acknowledge academic researchers who
join teams
•Organizational factors that influence the effectiveness of science teams (e.g.,
human resource policies, cyberinfrastructure)
•Organizational structures, policies and practices to promote effective teams
Committee
•
•
•
•
•
•
•
•
•
•
•
•
•
NANCY J. COOKE (Chair), Arizona State University
ROGER D. BLANDFORD (NAS), Stanford University
JONATHON N. CUMMINGS, Duke University
STEPHEN M. FIORE, University of Central Florida
KARA L. HALL, National Cancer Institute
JAMES S. JACKSON (IOM), University of Michigan
JOHN L. KING, University of Michigan
STEVEN W. J. KOZLOWSKI, Michigan State University
JUDITH S. OLSON, University of California, Irvine
JEREMY A. SABLOFF (NAS), Santa Fe Institute
DANIEL S. STOKOLS, University of California, Irvine
BRIAN UZZI, Northwestern University
HANNAH VALANTINE, National Institutes of Health
Study Status
Report expected in April
More information is available
at:
http://sites.nationalacademies.org/DBASS
E/BBCSS/CurrentProjects/DBASSE_080231
Research Base for Informing
Team Science
• SciTS – Science of Team Science (itself a
multidisciplinary approach)
• Social Science
• Complex Systems
• Communications
• Management
• Medicine
• Physical Sciences
The Foresight Initiative
• National Geospatial-Intelligence Agency (NGA)
has awarded Arizona State University a grant
of $20 million
• Five-year partnership known as the Foresight
Initiative will examine how climate change
affects resources and contributes to political
unrest, as well as articulate sustainability and
resilience strategies.
Foresight: A Science Team
Approximately 60 Investigators
•
•
•
•
•
15 ASU Faculty from 8 ASU units
Post docs, research faculty, graduate students
Three National Labs
National Geospatial Intelligence Agency
Expertise in visualization, modeling climate
change, cognitive science, social media,
human factors
Foresight: Team Science is
Challenging
•
•
•
•
•
•
•
•
Communicating across disciplines
Role confusion
Meetings
Remote participation
Goal conflicts
Sub-teams
Authorship
Resource Allocation
My Research and Experience Relevant
to Team Science
Team = Heterogeneous and
interdependent group of individuals
(human or synthetic) who plan, decide,
perceive, design, solve problems, and act
as an integrated system (vs. group)
Heterogeneous = differing backgrounds,
differing perspectives on situation
(surgery, basketball)
Cognitive activity at the team level=
Team Cognition
Improved team cognition  Improved
team/system effectiveness
Teams and Cognitive Tasks
I’ve Studied Team Cognition in These
Tasks
Uninhabited Aerial Vehicle Command and Control
Naval Mission Planning
Cyber Defense
Intelligence Analysis
Human-Underwater Robot Interaction
Medical Emergency Teams
Professional Cooking
Human-Robot Search and Rescue
Methods: Synthetic Task Environments
A compromise between field studies and laboratory
experiments
MacroCog
Underwater
Robots
CyberCog
16
What I’ve Learned
•
•
•
•
Teams Learn
Teams Forget
Membership Matters
Team Training Matters
Teams Learn
As teams acquire experience, performance improves, interactions improve,
but not individual or collective knowledge
600
Tm 1
Tm 2
500
Team Performance
Tm 3
Tm 4
400
Tm 5
Tm 6
300
Tm 7
Tm 8
200
Tm 9
Tm 10
100
40-min missions
Tm 11
0
1
2
3
4
5
6
Mission
• Individuals are trained to criterion prior to M1
7
8
9
10
Spring Break
• Asymptotic team performance after four 40-min missions (robust finding)
• Knowledge changes tend to occur in early learning (M1) and stabilize
• Process improves and communication becomes more standard over time
Teams Forget
8-10 week retention interval
Team forgetting is best predicted by interaction based measures, not
by individual forgetting (despite shared score components)
Regression model made up of individual decrements:
F (4, 20) = 2.018, MSe=5880.23, p>.10, R2 = .29
Introduction of coordination and team SA:
F (10, 14) = 2.71, MSe = 4011.88, p < .05, R2 = .66
Membership Matters
•
•
•
Retention Interval
3-5 weeks
10-13 weeks
10 Teams
9 Teams
10 Teams
10 Teams
Same
Mixed
•
117 males(92) & females(25) divided into 39
3-person (unfamiliar) Session 2 teams
Two between subjects conditions (retention
interval and familiarity) randomly assigned
with scheduling constraints
Participants randomly assigned to one of
three roles
Session 1: 5 40-min missions
Session 2: 3 40-min missions
Composition
•
Same Condition
Session 1
AVO
Mixed Condition
Session 2
PLO DEMPC
AVO
PLO DEMPC
Session 1
AVO
Session 2
PLO DEMPC
AVO
Retention
Retention
Interval
Interval
PLO DEMPC
3-5 OR 10-13 Weeks
Team Retention and Composition
All but Short-Intact teams suffer performance loss after the break
3-5 OR 10-13 Weeks
But a different story for Team Process (quality of
team interactions)…
Team Process improves for mixed, but not intact
teams after the break.
(There were no changes in knowledge after the break)
Team Training Matters
Cross training (aligned with shared cognition) vs.
procedural/rigid training vs. Perturbation training (focused
interactions)
Shared Mental Models
+
+
Team Cognition =
The collective knowledge of team members
•
•
•
•
•
Assumptions
Individual is the unit of analysis
Measure individuals and aggregate
Increasing similarity or convergence over time is
associated with better teamwork
Focus on knowledge, static cognition (team mental
model, shared mental model)
A collection of knowledge experts should be an expert
team
Interactive Team Cognition
Team interactions often in the form of
explicit communications are the
foundation of team cognition
ASSUMPTIONS
1) Team cognition is an activity; not a property or
product
2) Team cognition is inextricably tied to context
3) Team cognition is best measured and studied
when the team is the unit of analysis
US 2004 Olympic Basketball Team
"We
still have a couple of days,
but I don't know where we
are," replied USA head coach
Larry Brown … I've got a pretty
good understanding of who
needs to play. Now the job is to
get an understanding of how
we have to play."
A team of experts does
NOT make an expert
team
Collaborative skill is
not additive
US 1980 Olympic Ice Hockey Team
Herb Brooks and 20
young “no-names”
won the 1980
Olympic Gold Medal
in Ice Hockey
An expert team
made up of nonames…
A Multi-Level Systems
Perspective
• Micro -individual
• Meso – team, group
• Macro - organization, population
Borner, Contractor, FalkKrzesinski, et al., 2010
Micro Level: Challenges
• Who should engage in team science?
– Risks of early career tenure-track scientists
• Who should be on the team?
– Team composition
– Team assembly
• Faultlines and subgroups
Micro Level: Support
• Recommender systems
• Research networking systems
• Matching task to team assembly
Meso Level: Challenges
Input
Process
Output
IPO Model (Hackman, 1987)
Development
Conceptualization
Translation
Implementation
Four Phase Model of Transdisciplinary
Research (Hall, Vogel Stipelman, 2012)
Meso Level: Challenges
Team Process Behaviors
•Communication – shared mental models
•Coordination
•Conflict Resolution
•Back-up Behavior
•Situation Assessment
Meso Level: Support
•
•
•
•
Training
Leadership
Technology
Tools for Team Science
– NCI Team Science Toolkit
Macro Level: Challenges
•
•
•
•
•
Organizational rewards for team science
Disciplinary culture
Geographic dispersion
Complexity of multi-team systems
Mis-aligned goals
Macro Level: Support
•
•
•
•
•
Environment
Technology
Rewards
Collaboration Plans
Team Charters
Team Charters
• Communication plan between teams
(modes, media, who to whom)
• Plan for regular interactions
• Plan for leadership – shared
• Identify boundary spanning individuals
Asencio, Carter,
DeChurch, et al., 2012
Conclusion
• Team science is challenging
• Team research has implications for
making science teams more effective
• Challenges and support can be found at
the micro, meso, and macro levels