Program will start
3:00PM Eastern
2:00PM Central
1:00PM Mountain
Noon Pacific
11:00AM Alaska
Conference Planning Committee
Jack Cummings
Peg Dawson
Susan Gorin
Patti Harrison
Ron Palomares
Rick Short
Participant Evaluation
www.indiana.edu/~futures/eval.html
RESEARCH TO PRACTICE: The
Role of Evidence-Based
Interventions in Practice
Thomas Kratochwill
University of Wisconsin-Madison
Purpose of Presentation


Provide an Overview of the
Evidence-Based Intervention (EBI)
Movement and Implications for
School Psychology
Review Implications of the EBI
Movement for Graduate Education
Programs
Purpose (Cont.)


Review Implications of the EBI
Movement for Practice
Implications for the Futures
Conference
Resources

Chambless & Ollendick (2001).
Empirically Supported
Psychological interventions:
Controversies and evidence.
Annual Review of Psychology, 52,
685-716
Resources (Cont.)

Kratochwill & Callan-Stoiber (in
press). Evidence-based
interventions in school psychology:
Conceptual foundations of the
Procedural and Coding Manual…
School Psychology Quarterly, 17
(4),
The Evidence-Based Intervention
Movement in Perspective





Evidence-Based Medicine
The Scientist-Practitioner Model
The Managed Care Movement
Reform in Special Education
Professional Organizations
Professional Organizations



Committee on Science and Practice,
Society for Clinical Psychology, Division
12 American Psychological Association
(APA)
Committee for Empirically –Supported
Practice, Division of Clinical Child
Psychology Division 53, APA
Interdisciplinary Committee on
Evidence-Based Youth Mental Health
Care
Professional Organizations

Task Force on Evidence-Based
Interventions in School Psychology
(Sponsored by APA Division 16 and
the Society of the Study of School
Psychology and Endorsed by the
National Association of School
Psychologists)
Purpose of the Task Force



To identify prevention and
intervention outcome studies
relevant to the practice of school
psychology
To code those studies according to
Task Force criteria
To determine to what degree the
interventions are evidence-based
Purpose (Cont.)


To offer the field of school
psychology and related fields some
guidelines for using and adopting
effective programs
To provide a template for
improving research in the field of
school psychology and education
Task Force Domains of
Interest



School-and community-based
intervention programs for social
and behavioral problems
Academic intervention programs
Family and parent intervention
programs
Domains (Cont.)


School-wide and classroom-based
programs
Comprehensive and coordinated
school health care programs
Methodological Focus
Group Research
 Single-Participant Research
 Qualitative Research Procedures
 Confirmatory Program Evaluation

The Review Process


Six components considered
Each component involves validity
issues that need to be addressed
during the review process
EXISTING RESEARCH REPORTS ARE
CONSIDERED ALONG VALIDITY LINES




Statistical Conclusion Validity
Internal Validity
Construct Validity
External Validity
ORGANIZATION OF
RESEARCH DOMAIN


The Problem Formulation
Challenge
Validity Issues in Problem
Formulation (e.g., operational
specification and operational
detail)
IDENTIFICATION OF
RESEARCH

Threats of Validity (e.g., fugitive
literature, file drawer problem,
publication bias)

Resources needed to search
REVIEW OF STUDIES




Unrepresentative Studies
Methods of Gathering Studies
Retrieval Bias
Sample Representiveness
EVALUATION AND ANALYSIS
OF RESEARCH SYNTHESIS


Narrative Reviews of Research
(experimental and
nonexperimental)
Quantitative Reviews of of
Research (meta-analysis and
generalized causal inference)
SUMMING UP: INTERPRETATION,
PRESENTATION, AND DISSEMINATION



Omission of Details
Omission of Evidence on
Moderators
Telling the Story to the Profession
RESEARCH TO PRACTICE: The
Role of Practitioners in
Evidence Based Interventions
(EBIs)
What are the Big Issues?
Research to Practice (cont.)


Issues surrounding the role of
practitioners in the development
and use of evidence-based
interventions (EBIs).
Issues related to the researchpractice gap.
Next Step Agendas


Review five assumptions that
guide the integration of EBIs
into practice.
Suggest potential solutions to
problems surrounding the
adoption of EBIs in practice.
Five Assumptions Guiding the
Integration of EBIs into Practice
Settings


The development and use of EBIs
in practice needs to be a shared
responsibility (agenda) among
researchers and practitioners.
Implementation of EBIs in
practice may require
development and use of both
practice guidelines and expert
consensus guidelines.
(Assumptions cont.)

Offering professionals a menu of
EBIs is insufficient to promote
application in practice. Manuals
and guidelines must be
accompanied by additional
strategies for effective
integration of EBIs into practice.
Assumptions (Cont.)

To close the research-practice gap
we must facilitate professional
development for faculty and
practitioners.
(Assumptions cont.)

To strengthen the connection
between research and practice
we must promote and use a
scientist-practitioner model in
graduate training and
professional work, especially
when evaluating intervention
outcomes.
Shared Responsibility for EBI
Development



Involvement of Practitioners on
the Task Force
Participation in Practice Research
Networks
Evaluation of EBIs in Practice With
Case Studies and
Ethnographic/Qualitative Methods
Development of Guidelines


Practice Guidelines
Expert Consensus Guidelines
AACAP Practice Parameters for the
Treatment of Children and Adolescents
with Conduct Disorders
A.
Treat comorbid disorders (e.g., ADHD,
specific developmental disabilities,
intermittent explosive disorder,
affective or bipolar disorder, anxiety
disorder, and substance use disorder).
(cont.)
Family interventions included parent
guidance, training, and family therapy.
B.
1.
2.
3.
Identify and work with parental strengths.
Train parents to establish consistent positive and
negative consequences and well-defined
expectations and rules. Work to eliminate harsh,
excessively permissive, and inconsistent
behavior management practices.
Arrange for treatment of parental
psychopathology (i.e., substance abuse).
(cont.)
C.
Individual and group psychotherapy with
adolescent or child. Technique of
intervention (supportive versus explorative;
cognitive versus behavioral) depends on
patient’s age, processing style, and ability to
engage in treatment. Usually a combination
of behavioral and explorative approaches is
indicated, especially when there are
internalizing and externalizing
comorbidities.
(cont.)
Psychosocial skill-building training should
supplement therapy.
Other psychosocial interventions should be
considered as indicated.
D.
E.
1.
2.
3.
Peer intervention to discourage deviant peer association
and promote a socially appropriate peer network.
School intervention for appropriate placement, to promote
an alliance between parents and school, and to promote
prosocial peer group contact. Vocational training may be
useful.
Juvenile justice system intervention, including court
supervision and limit-setting, and special programs when
available.
(cont.)
4.
5.
Social services referral, to help the family
access benefits and service providers,
(e.g., case managers).
Other community resources, such as Big
Brothers and Big Sisters programs,
Friends Outside, and Planned
Parenthood, as indicated.
(cont.)
6.
7.
Out-of-home placement (crisis shelters, group
homes, residential treatment)
Job and independent-living skills training (pp.
1335-1345).
Source: From Steiner, H. (1997). Practice
parameters for the assessment and treatment of
children and adolescents with conduct disorder.
American Academy of child and Adolescent
Psychiatry, 36(10 Suppl), 1225-1395.
Guidelines Must Be Enhanced



Use Enhanced Guidelines that
focus on context
Use Enhanced Guidelines that
focus on theoretical mechanism for
intervention
Use Enhanced Guidelines that
focus on theoretical mechanisms
for change (at the individual,
classroom, and system levels)
Facilitate Professional
Development




Focus on Graduate Programs and
Practitioners in Schools
Adopt Competency-Based Models
of Education
Increases Acceptability
Promotes Integrity of
Interventions
Promote Evaluation of
Intervention Outcomes


Consistent with the ScientistPractitioner Model
Facilitate the Intervention Utility
of Assessment
Specific Task Force Strategies and
Tactics to Guide The Use of EBIs in
Practice



Develop a Practice Research
Network in School Psychology
Promote Effectiveness Studies
Develop and use Consumer
Guidelines for practitioners when
implementing interventions
(Strategies and Tactics cont.)


Use Enhanced Guidelines for
Interventions
Dissemination and Education
Efforts
Practice Research Network


Practitioners engaged in the
evaluation of EBIs
Example: Hawaii Empirical Basis to
Services Task Force
[Clinical Psychology: Science and
Practice, 9 (2) 2002]
Promoting Effectiveness
Studies



Efficacy vs. Effectiveness
Effectiveness Studies Focus on
Generalization of Interventions
Moving Beyond Effectiveness to
“Real Life Study Design” (Hohmann &
Shear, 2002)
Practitioner Partnerships


Practice Evaluation of EBIs
Establishing the Data Base for
Practice
Example Dimensions to Consider in Use of
Evidence-Based Intervention in School
Psychology Practice
Dimension Considered in Practice
1.
Does your client appear similar to
those described in the EBI?
2.
Are you able to replicate the
intervention based on the description
provided in the manual and/or
procedures?
3.
Are the conditions of implementation
of the EBI similar to those of your
setting?
(cont.)
4.
5.
6.
Are the measures you used to assess
outcome identical to those used to
establish the EBI?
Were all the measures recommended
in the EBI used to evaluate the
intervention?
Was ongoing evaluation (repeated
assessment) of student progress
conducted?
(cont.)
7.
8.
9.
Have new outcome measures been
added to the intervention evaluation?
Did you use an intervention manual or
protocol that specified the
intervention?
Can individual characteristics of
students be identified that are related
to intervention outcomes?
(cont.)
10.
11.
12.
When group intervention data are
reported, is the percentage of
individuals showing the effect
reported?
When individual data are reported,
have the data been presented in
graphic form?
Have the EBI positive effects reported
in research been replicated with your
student(s)?
(cont.)
13.
14.
15.
16.
Have you replicated the EBI more than
once?
Have others in your school setting
replicated the EBI?
Would you rate the effects as strong
as the original EBI effects?
Would you rate the effects as clinically
meaningful?
(cont.)
17.
18.
19.
Did you and/or staff find the EBI
acceptable for use in your school?
Was the EBI cost-efficient for
implementation in your school?
Are there specific contextual factors in
your setting that could account for the
success or failure of the EBI?
(cont.)
20.
Do you plan to adopt the EBI for future
implementation in your setting?
Source: Adapted from: Hayes, S.C., Barlow,
D.H., & Nelson-Gray, R.O. (1999). The
scientist practitioner: Research and
accountability in the age of managed care.
Enhanced Guidelines for
Interventions





Understanding Basic Principles of
Change
Understanding Indications and
Contraindications of EBIs
Understanding Variability in
Intervention Implementation
Problem Solving Strategies
Evaluation of EBIs in Practice
Dissemination Efforts and
Education


Graduate Education
Practice
Implications for Graduate
Education



Focus on Prevention/Intervention
Research Courses
Focus on Prevention/Intervention
Content in the Curriculum
Expand Faculty Through
Technology (no more renaissance
faculty)
Implications for Practice



Implications for Assessment
Implications for Consultation
Implications for Intervention
Implications for Practice:
Assessment



Emphasis on Screening to Identify
Students At-risk
Emphasis on Assessment to Plan
Interventions
Emphasis on Assessment to
Monitor Intervention Outcomes
Implications for Practice:
Consultation




Emphasis on Problem Solving
Emphasis on Collaborative
Teaming
Emphasis on Organizational
Development Consultation
Emphasis on Professional
Development Activities
Implications for Practice:
Intervention




Emphasis on Prevention
Emphasis on Multi-Level Systems
of Prevention
Emphasis on Concomitant
Academic and Social/Emotional
Intervention Focus
Emphasis on Intervention
Responsiveness in DecisionMaking
Summary and Conclusions

This presentation provided an
overview of several guiding
assumptions embraced by the
Task Force on Evidence-Based
Interventions in School
Psychology.
Summary and Conclusions

The assumptions can guide specific
strategies the school psychology
community can use to move
forward the agenda of integrating
EBIs into graduate education and
practice.
Summary and Conclusions

The development of EBIs is a
shared scientific agenda among
researchers and practitioners.
For Further Information,
Contact:
Thomas R. Kratochwill, PhD
School Psychology Program
1025 West Johnson Street
University of Wisconsin-Madison
Madison, Wisconsin 53706
Phone: (608) 262-5912
E-Mail: tomkat@education.wisc.edu