Turning a Team of Experts
Into an Expert Team:
Contributions From the
Science of Teamwork and
Simulation
Eduardo Salas, Ph.D.
Department of Psychology and
Institute for Simulation & Training
University of Central Florida
esalas@ist.ucf.edu
1
“…medicine is a team sport, with two
exceptions: people’s lives depend on it and
there are no coaches.”
-Atul Gawande, MD
In sports, when the team loses, the whole team
loses, but in medicine, only the patient loses.
2
Outline
What is teamwork?
II. How do you enhance teamwork?
III. What is the efficacy of team training?
I.
I.
Highlight some quantitative
findings…

II.
Highlight some qualitative findings…

IV.
Series of meta-analyses
Offer some observations
Offer some insights, principles…
3
Defining Characteristics of Teams





Two or more individuals
Multiple information
sources
Meaningful task
interdependencies
Coordination among
members
Common, valued goals





Specialized member
roles and
responsibilities
Task-relevant
knowledge
Intensive
communication
Adaptive
mechanisms
Hierarchically
organized
4
The Environment...








Complex, multi-component decisions
Rapidly evolving, ambiguous situations
Information overload
Severe time pressure
Severe consequences of error
Adverse physical conditions
Performance/command pressure
Distributed, multi-operator problems
5
The Science…
Can we turn teams of experts
into expert teams?
6
But…





What is teamwork?
How do we enhance it?
Does team training work in healthcare?
Can we derive some evidence based principles
of team training?
What do we know from 30 years of science?

Inform Healthcare
7
The Drivers of Team Performance
Team Performance
Cognitions
Behaviors
Attitudes
Knowledge
Skills
Affect
“Think”
“Do”
“Feel”
8
The Drivers of Team Performance
Knowledge Competencies:
•Cue/strategy associations—SMMs
Team Performance
•“Knowledge
of how to get knowledge”
•Teammate characteristics, familiarity
•Team mission, objectives, norms, resources
•Roles and responsibilities
proficiency, taskwork
Cognitions•Individual-task
Behaviors
Attitudes
Knowledge
Skills
Affect
“Think”
“Do”
“Feel”
9
The Drivers of Team Performance
Skill Competencies:
•Mutual performance monitoring & adaptability
Team Performance
•Supporting/back-up behavior
•Team leadership
•Task-related assertiveness
•Conflict resolution
•Closed-loop
Cognitions communication
Behaviors
Attitudes
Knowledge
Skills
Affect
“Think”
“Do”
“Feel”
10
The Drivers of Team Performance
Attitude Competencies:
•Collective efficacy
Team Performance
•Shared vision
•Team cohesion
•Mutual trust
•Collective/team orientation
•Value
of teamwork Behaviors
Cognitions
Attitudes
Knowledge
Skills
Affect
“Think”
“Do”
“Feel”
11
What effective teams do…
12
What effective teams do...

Hold shared mental models



…have members who anticipate each
others’ needs
…can coordinate without the need to
communicate overtly
Optimize resources




…self-correct
…compensate for each other
…reallocate functions
…can adapt their strategies under stress
13
What effective teams do...

Have clear roles and responsibilities




…manage expectations
…ensure team members’ roles are clear
but not overly rigid
… understand each others’ roles and
how they fit together
Share a clear and valued vision

…have a common purpose
14
What effective teams do...

Have strong team leadership




…are led by someone with good leadership
skills and not just technical competence
…have team members who believe the
leaders cares about them
…provide situation updates
…foster both task related behaviors and
teamwork, coordination and
cooperation
15
What effective teams do...

Engage in a cycle (a discipline) of prebrief
performance
debrief




…regularly provide feedback to one another,
both individually and as a team (“de-brief”)
…establish and revise team goals and plans
…differentiate between higher and lower
priorities
…periodically diagnose team "effectiveness,”
including processes, results, and team vitality
(morale, retention, energy)
16
What effective teams do...

Develop a strong sense of “collective”—
trust, teamness, confidence





…manage conflict well—team members
confront each other effectively
…have a strong sense of team orientation
…trust other team members’ “intentions”
…strongly believe in the team’s collective
ability to succeed
…develop collective efficacy
17
What effective teams do...
 Engage in “rhythms” of performance


…manage time
…change at the “midpoint”
18
How do you promote effective teamwork?

Before task…

During task…

After task…
19
Promoting Teamwork … BEFORE

Compose the team:
membership/selection
(Klimoski & Jones, 1995)

Select team members for
required competencies




Select team leaders


Personality tests
Simulations
Situational judgment tests
Simulations
Not much research!
20
Promoting Teamwork… BEFORE

Enhance individual
team member
capabilities






Individual taskwork
training
Goal setting/
performance
management
Individual feedback
On-the-job training
Coaching/mentoring
Team leader training
(Tannenbaum et al., 1998)
21
Promoting Teamwork… BEFORE

Improve Team
interactions/processes

Role clarification



Team building
Facilitated process analysis


Self correction
Team pre-brief/de-brief
interventions


Who does what with whom?
Set expectations
Some validity
22
Promoting Teamwork … BEFORE

Build team competencies
(Salas & Cannon-Bowers, 2001)


Cross-training (e.g.
positional rotation,
position clarification
training)
Team coordination
training/CRM
(Wiener, Helmeich & Kanki, 1993)

Meta-cognition training
(Cohen et al., 1998)
23
Promoting Teamwork … DURING

Maintaining Teamwork


Scenario-based training
Guided Team self-correction
(Smith-Jentsch et al., 1998)


Designed to teach
teams to correctly
identify errors and
adapt “in real time”
to compensate for
those errors
Focus on behaviors:





Mutual performance
monitoring tasks
Stress awareness
Compensatory
behavior
Taking initiative
Communication
24
Guided Team Self-Correction
Self-Correction Process
•Event Review
•Error Identification
•Feedback Exchange
•Planning for Future
Attitudes
• Collective Orientation
• Higher Cohesion
Enhanced Cognitions &
Shared Mental Models
•Improved:
•Expectations
•Explanations
•Task Understanding
Behaviors
• Enhanced Team
Processes:
-Coordination
-Communication
25
Team Training Defined

Team Training IS…






A family of learning strategies
The systematic acquisition of teamwork KSAs
Focus on cognitive, behavior, and/or affective teambased change
Planned activity
Based on pedagogical principles
Team Training is NOT...



A group of people in front of a mannequin
A place, program or workbook
Not just saying “do better” at M&M conferences
26
Structure of Team Training
Tools
•Team Task Analysis
•Task Simulation &
Exercises
•Feedback
•Principles
Strategies
•Cross-Training
Methods
•Information-Based
•Demonstration-Based
–Video
•Practice-Based
–Guided Practice
–Role Play
•Coordination Training
•Team Leader Training
•Others...
Team Training Objectives
Content
•Competencies
–Knowledge
–Skills
–Attitudes
27
Team Training Strategies










Cross-training
Event-based approach to training
Scenario-based training
Self-correction training
Guided self-correction training
Assertiveness training
Stress exposure training
Team adaptation training
Team leader training
CRM and derivatives
28
How we know team training
works…
29


Quantitative Perspective
Answers the question: Does team training
work?
Meta-Analysis…?



Technique which summarizes all previous
individual studies
Eliminates problems of using different
measures, samples, etc.
Uncovers ‘true’ relationship

Reported as:


“r”  Uncorrected
or “ρ” (pronounced “rho”) Corrected for
measurement and sampling errors
30
The Quantitative Literature

2 Meta-Analyses

Does Team Training Improve Team
Performance? A Meta-Analysis (Salas,
DiazGranados, et al., 2008)

Testing Three Team Training Strategies in
Intact Teams: A Meta-Analysis (Salas,
Nichols, & Driskell, 2007)
31
The Quantitative Literature


Comprehensive Search (1955 – 2008)
Over 300 empirical articles






245 journal articles
33 conference presentations
20 theses/dissertations
13 book chapters
13 technical reports
3 unpublished works
32
The Quantitative Literature

Over 10,000 teams represented







Aviation teams - 6,962
Lab teams - 1,845
Organizational teams - 1,527
Military teams - 464
Medical teams - 181
Education teams - 26
Sports teams - 12
33
Quantitative Results
Salas, DiazGranados, Klein, Burke, Stagl, Goodwin, & Halpin (2008)

Assessed relative effectiveness of these interventions on
team outcomes:

Cognitive



Affective




Communication
Coordination
Performance



Attitudes towards teamwork
Burnout
Process


Teamwork knowledge
Transactive memory
Simulation performance
Number of errors
45 studies; 93 effect sizes

4 healthcare studies
34
Quantitative Results
Salas, DiazGranados, Klein, Burke, Stagl, Goodwin, & Halpin
(2008)

Results



Overall, team training was shown to have a
moderate, positive effect on team functioning
(ρ = .34)
Moderator analysis investigating effect of team
training on distinct outcomes
Four outcomes investigated:




Cognitive outcomes (ρ = .42)
Affective outcomes (ρ = .35)
Process outcomes (ρ = .44)
Performance outcomes (ρ = .39)
35
What do these results mean?



Team training does work!
Training content, team membership stability
and team size impact the effectiveness of team
training interventions.
Team training can explain 12%-19% of the
variance of a team’s performance which
means:




Reducing medical errors
Saving lives
Saving aircrafts
Increasing the bottom line
36
The Quantitative Literature
TTMA

(Salas, Nichols, & Driskell, 2007)
Investigated relative contributions of three
different team training strategies

Cross-training (CT)



Team coordination and adaptation training (TCT)


Team members rotate positions during training
Develops shared understanding of KSAs and
responsibilities of team members
Teaches how to alter coordination strategy and to
reduce the amount of communication necessary for
successful task performance
Guided team self-correction training (GSC)

Use facilitated debriefs to learn how to diagnose the
team’s problems and to develop effective solutions 37
The Quantitative Literature
TTMA (Salas, Nichols, & Driskell, 2007)


7 studies; 28 effect sizes
Results

Overall, team training improved performance (r = .29)



Accounted for 8.4% of variance
Team training effectiveness by components

TCT (r = .61) – Accounted for 37.2% of variance

GSC (r = .45) – Accounted for 20.0% of variance

CT (r = - .09) – No effect
Independent contributions to team training effectiveness

TCT (r = .30) – Accounted for 9.0% of variance

Cross-training and guided team self-correction training did
not make significant independent contributions
38
What do these results mean?

TACT/CRM contributes the most to
improving team effectiveness


TACT/CRM can explain 37% of the
variance of a team’s performance
Outcomes that were examined in the
primary studies included:




Shared expectations
Implicit coordination
Overall teamwork
Communication
39
Why, how, and when
does team training work?:
A focus on healthcare
40
A Qualitative Review

Conducted a content analysis of the team
training literature, examining factors such
as…





How training was designed?
What content was included?
What instructional strategies were used?
How was it implemented?
How was it evaluated?
41
Description of Qualitative Database


Comprehensive Search (1955-2008)
Resulting in over 450 articles


Both empirical and theoretical
Representing team training in:





Aviation- 75 articles
Military- 150 articles
Sports- 11 articles
Organizations- 110 articles
Medicine- 120 articles


Of these 120, 37 were empirical studies of medical
team training
Coded to determine principles & best practices for
team training in healthcare
42
The Qualitative Method

Coding framework was established


50 pieces of information from each study (N=37)
Themes:





Content analysis conducted by analyzing text
data



Background information
Training design
Components and features of training
Training evaluation
Generate themes
Provide understanding of phenomenon
Articles coded by 2 raters

Graduate students skilled in qualitative data analysis
43
Medical Team Training Database
Specialty Targeted for
Team Training
• 16% of the articles explicitly targeted
Anesthesia
 14% explicitly targeted Emergency
Medicine
 5% explicitly targeted Pediatrics
 35% targeted some other specialty
area
 24% targeted mixed specialties
 5% did not identify a target specialty
Type of Providers
Targeted for Training
•



22% specified physicians and nurses
5% specified anesthesiologists
11% specified students/residents
49% identified multiple types of
providers
 14% specified other classifications or
did not identify a targeted population
44
Medical Team Training Database
Type of Team
Training
•




Training Focus:
Teamwork vs.
Taskwork
• 68% focused explicitly on
teamwork
 30% focused on a combination of
teamwork and taskwork
 3% did not specify the focus of the
training
62% of the articles used CRM
5% of the articles used team building
3% of the articles used cross training
3% of the articles used goal setting
30% of the articles used alternative
types of team training
45
Medical Team Training Database
Training
Participation
• 54% participated in teams
o 25% of these reported intact
teams
o 10% of these reported ad-hoc
teams
o 65% of these did not specify
• 20% participated as individuals
• 27% did not specify
46
Medical Team Training Database
Instructional
Methodology
• 86% used a mix of both traditional
informational methods of instruction (e.g.
lecture/classroom) plus modeling/demo, and
practice or simulation
 8% used practice and simulation only
 3% of studies relied on informational
methods only
 3% could not be determined
Opportunities
for Practice
• 86.5% explicitly reported providing
opportunities for practice
o 50% of these reported using high
fidelity simulation
o 25% of these reported using role play
o 6% of these reported using low fidelity
simulation
o 19% of these did not specify the type of
practice opportunity
47
Medical Team Training Database
Feedback
Training
Duration
• 54% explicitly reported that feedback
was given after practice scenarios
o100% of these reported that
feedback was given face to
face
oMajority was behavior based
feedback
• 51% reported keeping training to
1 day or less
oRange: 30 min – 8 hrs
 32% reported that training lasted
more than 1 day
oRange: 2 days – several months
48
Medical Team Training Database
Training
Evaluation
Results
(Level 1 & 2)
• 19% evaluated only trainee reactions (e.g.
useful, enjoyable, etc.)
 78% evaluated beyond training reactions
o 10 studies evaluated
knowledge/learning/affect
o 21 evaluated behavior/process
o 9 evaluated organizational outcomes
• 54% explicitly reported that participants
responded positively to training
• 30% explicitly reported improvement in
participant learning/knowledge/affect
after training
49
Medical Team Training Database
Results

(Level 3 & 4)

27% explicitly reported improvements in
participant behavior on the job (e.g.
communication, brief/debrief, preparedness)
24% reported the impact of training impact
organizational outcomes…
o Taylor et al (2008) : After CRM training
significant positive changes in microalbumin
and lower extremity amputation
prevention
o Created an ‘all outcome index’ (range
0% - 100%) and saw marginally
significant improvement after training
from 42.96% to 45.34% (p = .079)
o Dunn et al. (2007): Several campuses indicated
that adverse events such as wrong site
surgery and wrong procedure have
decreased (though no explicit empirical
50
evidence provided!)
The Science of Simulation-Based
Training….
What works!
51
Simulation-Based Training


Used to accelerate expertise…
Explicit links are developed and
maintained between training
requirements, exercise design,
observation, and feedback…
52
Needed Links in Simulation-Based Team Training…
Tasks & KSAs
Learning
Outcome
Performance
History
Scenarios &
Events
Measures
Feedback
53
The Science…
54
Myth
55
Reality



Simulation is just a tool!
A delivery mechanism
Need to embed instructional features!
56
SBT Works Best With…
 …Clear & Precise Learning Outcomes
 …Carefully Crafted Scenarios
 …Diagnostic Measures
 …Robust Observations Protocols
57
SBT Works Best With…



…Providing Developmental Feedback
…An Organized Pedagogical System
…Thinking About What Happens Before,
During, and After SBT
58
Some Principles to Optimize SBT for
Healthcare:
Beyond the bells and whistles
59
Clear & Precise Learning Outcomes
Tips:




Develop learning outcomes; serve as guide
Training teamwork competencies, not tasks
Define teamwork requirements
Conduct a cognitive task analysis; uncover
aspects of the job
60
Carefully Crafted Scenarios
Tips:



Create opportunities to demonstrate
competencies (“triggers”)
Embed metrics (and instructional features)
Link “triggers” to competencies and learning
outcomes
61
Diagnostic Measures
Tips:




Ensure metrics are diagnostic of clinical
performance for AAR
Reinforce desired clinical behaviors
Focus on process measures
Design event-based measurement protocols
62
Robust Observation Protocols
Tips:



Train the observers
Focus on both the individual and team level
Focus on 4-5 key performance dimensions
63
Feedback
Tips:




Make it timely
Provide diagnostic and developmental feedback
Focus on both: Individual and team level
feedback
Create discipline to debriefing
64
An Organized Pedagogical System
Tips:

Provide Information…


Demonstrate Behaviors…


Trainees learn ‘rough approximation’ of teamwork behaviors
through observational learning
Create Practice Opportunities…


Trainees build declarative knowledge of the KSAs involved in
teamwork
Trainees learn by doing…good performance is reinforced, poor
performance is corrected
Provide Feedback and remediation…

Trainees refine teamwork competencies (deficiencies in KSAs
are identified and addressed with corrective feedback and
additional targeted training)
65
Think About What Happens Before, During,
and After SBT
Tips:





Sound positive messages about SBT & safety
Make sure you involve top-level support
Make sure key players are on board
Prepare organizational climate
Create opportunities to practice (after SBT)
66
Continued…
Tips:





Show management support (after SBT)
Ensure trainee motivation
Promote self-efficiency
Reinforce desired behaviors (after SBT)
Provide incentives
67
Team training can only be successful if…
1. The design and delivery is scientifically
rooted
2. The focus is on team-based competition
3. Physicians take ownership
4. Teamwork is rewarded and reinforced by
both providers and administration
5. Practice opportunities are
embedded…simulation!
6. Mechanics to maintain shared mental
models are in place…briefs/debriefs
68
Thank You For Your Time!
Eduardo Salas, Ph.D.
Department of Psychology and
Institute for Simulation & Training
University of Central Florida
esalas@ist.ucf.edu
69