Care Teams Research:
The Importance of Theory, Fidelity,
Climate and Causality
1
DOUGLAS WHOLEY
H E A LT H P O L I C Y & M A N A G E M E N T
S C H O O L O F P U B L I C H E A LT H
U N I V E R S I T Y O F M I N N E S O TA
A C A D E M Y H E A LT H 2 0 0 9
M E T H O D S F O R S T U D Y I N G O R G A N I Z AT I O N A L
EFFECTS
© Doug Wholey, Health Policy & Management, University of Minnesota, Organizational Research
6/30/2009
Presentation Organization
2
 Methods & Theory
 INTACT Conceptual Model & Hypotheses
 Methods
 Sampling Strategies
 Measurement and Power
 Results
 Client and staff outcomes
 Team level mediated effects
 Acknowledgements
 Based on INTACT: Improving Networks & Teamwork in Assertive
Community Treatment
 National Science Foundation, David Knoke (Co-PI), Xi Zhu, Pri Shah,
Mary Zellmer-Bruhn, Ayse Gurses, Pinar Karaca-Mandir, Michelle Duffy,
Yuqing Ren, Katie White, David Knutson, Tom Witheridge
© Doug Wholey, Health Policy & Management, University of Minnesota, Organizational Research
6/30/2009
Theory and Science
3
“Science is based on counterfactuals and theoretical
models. Human knowledge is produced by
constructing counterfactuals and theories. Blind
empiricism unguided by a theoretical framework for
interpreting facts leads nowhere.”
Heckman, J. J. 2005. The scientific model of causality. In R. Stolzenberg
(Ed.), Sociological Methodology, Vol. 35, pp. 4–5. Quoted in Ward, A. &
Johnson, P. J., “Addressing confounding errors when using nonexperimental, observational data to make causal claims.” Synthese
163(2008): 419-432.
© Doug Wholey, Health Policy & Management, University of Minnesota, Organizational Research
6/30/2009
CMO
4
 The knowledge that transfers between studies is theories
rather than “lumps of data”

Naming & Shaming: Provider report cards, School rankings
 “Produce theories in the form of ‘generative causal
propositions’ which relate mechanisms, context and
outcomes. … simplify all processes down to an essential
core of attributes.”

Ray Pawson and Nick Tilley, Realistic Evaluation, Sage 1997, 120, 122.
Context
Mechanism
Outcome
Regularity
© Doug Wholey, Health Policy & Management, University of Minnesota, Organizational Research
6/30/2009
Methods | Theory
5
 Lay theories “as it is expressed in our everyday
language and experience … common-sense or naive
psychology”



Heider, The Psychology of Interpersonal Relations, 1958: 4-7
We can learn from common sense just as we can learn from
folk medicine but is it adequate as science?
Do we need to avoid pseudo-science that looks like science
because it uses sophisticated methods?
 Theory
 Organizational Behavior, Organizational Theory, Management
Science
© Doug Wholey, Health Policy & Management, University of Minnesota, Organizational Research
6/30/2009
INTACT: Improving Networks & Teamwork in
Assertive Community Treatment - Conceptual Model
6
Context
Policies, Funding, Institutional Supports,
Incentives, Staffing, Delegation,
Coaching, Training
Moderating Team Processes
Learn-How/Learn-What,
Constructive Controversy,
Helping
Fidelity
Team
Composition,
Meetings,
Treatment
Planning ,
Protocols
Meditating-Team
Processes
Lean Management
(low waste, obtain
information easily)
Moderating Emergent States
Psychological Safety, Social
Capital, Transactive Memory
Mediating Emergent
States
Encounter
Preparedness
Outcomes
Client
Outcomes
Staff
Outcomes
Teamwork
© Doug Wholey, Health Policy & Management, University of Minnesota, Organizational Research
6/30/2009
Teams: What is a team?
7
A team is“(a) two or more individuals who (b) socially
interact (face-to-face or, increasingly, virtually); (c) possess
one or more common goals; (d) are brought together to
perform organizationally relevant tasks; (e) exhibit
interdependencies with respect to workflow, goals, and
outcomes; (f) have different roles and responsibilities; and
(g) are together embedded in an encompassing
organizational system, with boundaries and linkages to the
broader system context and task environment.”
Kozlowski, Steve W. J. and Daniel R. Ilgen. 2006.
"Enhancing the Effectiveness of Work Groups and Teams."
Psychological Science in the Public Interest 7:77-124 (79).
© Doug Wholey, Health Policy & Management, University of Minnesota, Organizational Research
6/30/2009
Sampling Strategies
8
Examples
Sampling Strategies
 Convenience
 Generalizability
 Theory testing
 Maximize variability in key
independent variables
 Arthur Stinchcombe, The
Logic of Social Research.
Chicago: University of
Chicago Press. 2005
 Assertive Community
Treatment teams (NSF)

Full population in Minnesota
 Congestive Heart Failure
(RWJ/INQRI)

Full population in Medical
Centers in VA
 Primary Care Practices
(NPCRDC)

Phase: Qualitative study
comparing practices in
deprived and non-deprived
areas
© Doug Wholey, Health Policy & Management, University of Minnesota, Organizational Research
6/30/2009
Measurement: Fidelity & Climate in ACT
9
Fidelity: Team Design
(Necessary, not sufficient)
Teamwork & Climate
 The fit of the implementation
 Mediating Processes
with evidence standards
 Targeted population
 Structures


Team
Appropriate DOL (KSAs)
 Coordination

Daily team meetings
 Formalization

Treatment Plans linked to
encounters
 http://www.actassociation.org/
fidelity/

Lean management
 Mediating States

Preparedness
 Moderating Processes


Learn what / learn how
Constructive controversy
 Moderating States



Psychological safety
Social capital
Transactive memory systems
© Doug Wholey, Health Policy & Management, University of Minnesota, Organizational Research
6/30/2009
Measurement
10
 Customization: Concepts to Measures


Psychological safety in ACT
 I felt that it was easy to ask for a change in the time of a visit.
 I felt that it was easy to ask for a partner to accompany me on a visit.
Take-away
 Are off the shelf instruments adequate for precise measurement?
 Do off the shelf instruments reduce response rates?
 Instruments have to be translated to specific research context
 Measurement validation


Item level – construct validity - convergent and discriminant validity
Team level measures: Intraclass correlation – is there similarity
within teams and differences across teams?
© Doug Wholey, Health Policy & Management, University of Minnesota, Organizational Research
6/30/2009
Measurement Results (Sample)
11
Construct
Constructive
Controversy
Psychological
Safety
Learn-What
Definition
“The critical and open discussion of
divergent perspectives including task
related facts, data and opposing ideas.”
“A shared belief that the team is safe for
interpersonal risk taking.”
Involvement in activities that identify the
best practices that are currently available.
Alpha F (Prob)
0.80
0.86
0.79
3.62
(0.00)
2.69
(0.00)
1.22
(0.22)
ICC
0.18
0.13
0.02
Learn-How
Involvement in activities that operationalize
practices in a given setting and solve
0.90
problems by trials and errors.
1.99
(0.00)
0.08
Low Waste /
Lean*
Low waste in searching for information and
0.68
resources.
3.80
(0.00)
0.19
Encounter
Preparedness
The degree to which ACT team members
feel prepared to perform their tasks.
3.02
(0.00)
0.16
© Doug Wholey, Health Policy & Management, University of Minnesota, Organizational Research
0.79
6/30/2009
Encounter Preparedness: Box & Whiskers
12
1-Almost never, Occasionally, Often, Usually, 5-Almost always
© Doug Wholey, Health Policy & Management, University of Minnesota, Organizational Research
6/30/2009
Results: Team Level Models of Encounter
Preparedness (n=26, Mediation analysis)
13
Intercept
Fidelity
Team Size
Interdependence
Constructive
Controversy
Psych. Safety with Team
Lead
Psych.
Constructive Safety with
Controversy Team Lead
3.08
0.49
2.22
0.79
-0.16
0.06
-0.44
0.45
0.03
-0.02
0.74
-1.45
1.66
0.03
2.64
0.13
0.31
4.03
0.52
5.09
1.67
3.18
Psych. Safety with Team
Obtain Information
Easily
R-Squared
Obtain
Information
Encounter
Easily
Preparedness
-3.16
0.78
-1.43
0.41
-0.52
0.84
-1.14
2.16
0.01
0.00
0.17
-0.09
0.18
0.28
0.20
0.38
-0.17
0.02
-0.48
0.07
0.31
0.85
0.60
© Doug Wholey, Health Policy & Management, University of Minnesota, Organizational Research
-0.15
-0.28
0.61
3.31
0.56
6/30/2009
Effects of Preparedness on Staff and Clients
14
Staff Outcomes
 ↑ Preparedness
 ↓ Negative affect, ↑
Positive affect




The frustration effect!
↓ Burnout
↑ Satisfaction
↓ Turnover
Client Outcomes
 ↑ Preparedness
 ↓ Hospital days due to
mental illness
 ↑ Independent living
© Doug Wholey, Health Policy & Management, University of Minnesota, Organizational Research
6/30/2009
Next Steps
15
 Causal modeling
 Using contextual measures as instruments to do causal
modeling
 Change over time
 Will have three waves of data spaced at six month intervals
linked with quarterly evaluations of client outcomes
© Doug Wholey, Health Policy & Management, University of Minnesota, Organizational Research
6/30/2009
Conclusion
16
 Methods are conditional on theory


Methods cannot save inadequate conceptualization
With good theory, methods are straightforward (but can be difficult)
 Sampling for theory testing is different than sampling for
generalizability
 Instruments need to be customized to context for precise
measurement and good response rates
 Many phenomena are nested


Team members or clients are nested within teams which are nested
within sponsors
Concepts have to be validated at the appropriate theoretical level
(intraclass correlations)
© Doug Wholey, Health Policy & Management, University of Minnesota, Organizational Research
6/30/2009