Definitions and Myths
Elisabeth Cannata, Ph.D.
Vice President,
Community-Based Family Services and Practice Innovation
Wheeler Clinic
Michael Williams, LMFT
MST Program Manger,
Advanced Behavioral Health
A call to action across the
healthcare delivery system for:
 Accountability of healthcare providers
-- demonstrating that what we do works
 Improved client care
-- identifying what interventions are most effective for treating
particular conditions
-- increasing consumer confidence in treatment
 Consistency
-- helping consumers to know what to expect from treatment
-- shaping healthcare practice through well-defined guidelines
for clinicians to follow
Definitions of EBP
 The most widely cited definition of EvidenceBased Practice is that of the Institute of Medicine
(2001) adapted from Sackett et al., 2000:
“Evidence-based practice is the
integration of best research evidence
with clinical expertise and patient
values.”
Shared Conceptualization of Evidence-Based
Practice for Behavioral Health
BEST AVAILABLE
EVIDENCE
CLINICIAN
EXPERTISE
CLIENT
PREFERENCES
Adapted from the Council for Training in Evidence-Based Behavioral Practice
(2008)
Evidence Based Practice or Practice Based
Evidence?
 Without the collection of data, how do we know what
is working?
 What parts of a program are working, and which parts
are not?
 With evidence, comes the ability for standardized
approaches and consistent positive outcomes for
families.
 Understanding hierarchies of evidence (what kind of
research and data gives best prediction of positive
outcomes?)
Development and Dissemination of Evidence-Based
“Practices” or “Treatments” (the “gold standard” for
empirically supported approaches to intervention)
• Randomized control group studies demonstrating
effectiveness in comparison to alternative treatment
interventions
• Manualized, replicable practice parameters
• Well-defined target population
• Independent evaluation
System of Classification
for Evidence-Based Treatments
Best Support: Well-Established Treatments
 At least two good between group design experiments demonstrating efficacy in one
or more of the following ways:


Superior to pill placebo, psychological placebo or another treatment
Equivalent to an already established treatment in experiments with adequate statistical
power (about 30 per group)
OR
 A large series of single case design experiments (n > 9) demonstrating efficacy. These
experiments must have:


Used good experimental designs
Compared the intervention to another treatment.
AND
 Further Criteria:
 Experiments must be conducted with treatment manuals
 Characteristics of the client samples must be clearly specified
 Effects must have been demonstrated by at least two different investigators or teams
of investigators
(Adapted from Silverman & Hinshaw, 2008)
System of Classification for
Evidence-Based Treatments (continued)
Promising (Probably Efficacious) Treatments

Two experiments showing the treatment is (statistically significantly) superior to
a waiting-list control group
Manuals, specification of sample, and independent investigators are not
required.
OR

One between-group design experiment with clear specification of group, use of
manuals, and demonstrating efficacy by either:
Superior to pill placebo, psychological placebo or another treatment
 Equivalent to an already established treatment in experiments with adequate
statistical power (about 30 subjects per group)

OR

A small series of single case design experiments (n > 3) with clear specification of
group, use of manuals, good experimental designs, and compared the
intervention to pill or psychological placebo or to another treatment
(Adapted from Silverman & Hinshaw, 2008)
So,…Evidence-Based and Promising
PracticeTreatments…What Are They?
Treatment models that target clinically meaningful
populations, syndromes or situations
Treatment Models that have a demonstrated research
basis of support
Clinical interventions that have an underlying conceptual
framework and well defined treatment protocols (i.e.
manualized)
Parameters/control measures to ensure adherent
implementation/delivery of the treatment model
Implementation and On Going Quality
Assurance is what Sets These Models Apart
 Programs are created utilizing established
mechanisms for team structure, size, caseload etc. that
has been shown to produce the best results for clients
 Continuous assessment of program practices and
outcomes provide opportunities for continuous
improvement and strengthening of things are working.
 Oversight of program adherence prevents model drift
EXTENSIVE TRAINING AND SUPERVISION OF
THERAPISTS
Evidence Based and Promising Practice Models are
structured around significant training and supervision
expectations:
• Model specific training at the start and throughout
practice (ranging from a week to 6+ months of focused
training by model experts)
• Extensive weekly supervision by program supervisors
and often by model consultants
• Individual and team supervision
• 24 hour support for staff by supervisors trained in the
specific model
In CT, Partnership with Higher Education to Prepare the
Workforce for Evidence-Based and Promising Practices
 Development of a graduate level curriculum to expose
students of social work, marriage and family therapy,
psychology and counseling to the array of in-home
family treatment models available in CT
 Brings providers of the different treatment models in
to the classroom to share their experiences
 Brings families into the classroom as educators
Shared Characteristics of empirically supported
treatment programs that have been promoted for
children and adolescents in CT:
 Ecological and family-focused approaches to treatment
 Strengths-based
 Targeting intensive treatment needs
 Comprehensive, broad scope assessment and
intervention
 Extensive training for therapists
 Ongoing quality assurance
 Aimed at keeping youth in the community and with
their families
 Many are conducted in-home
 Typically, multiple sessions per week
What is important for CT decision makers to
know?
 Provider accountability
 On going monitoring of CT specific outcomes
 The collection of data aligns nicely with Results Based
Accountability (RBA)
 CT has emphasized the development of a treatment
system that is not one size fits all
 Ability to use research to match families to treatments
with the best chance for positive outcomes.
 CT is implementing these programs with a high
degree of success
Misconceptions about EBPs
Responses to Common
Criticisms of EBPs
“EBPs are too much of a ‘cookbook’ approach to
very complex issues”
While the structure of each of the models gives a
sequence of steps and/or tools to follow systematically in
delivering the services according to the model, the
actual “ingredients” (i.e. the goals, objectives and
specific interventions of the treatment plan) are based
on highly individualized assessment and treatment
planning which are facilitated by the clinical expertise of
the practitioner.
Responses to Common
Criticisms of EBPs
“EBPs ignore client values and preferences”
All of the models emphasize the fundamental principle
that families must be full partners in defining the
problem, determining the treatment goals and assessing
how the intervention is working
Responses to Common
Criticisms of EBPs
All of the emphasis on EBPs inhibits the
implementation of practices that may be effective,
but just not studied
With EBPs, there is a focus is on therapist/program
accountability. The clinician strives to find the best
supported intervention…However, where there is
not data, there should still be rigor in looking at
the specific outcomes for any given client with
whatever intervention is collaboratively chosen
Alliance to Support Evidence Based and
Promising Practices (ASEP)
 Collection of providers and stakeholders working
collaboratively on issues pertinent to CT families and
providers
 Currently has subgroups addressing the future of
EBPs, Workforce Development, and Utilizing
Outcomes
 Established in 2010
 Always looking for new members!
Mental Health, Juvenile Justice, Adolescent
Substance Abuse, & Child Welfare
Robert W. Plant, Ph.D
Director of Community Programs and Services
Setting the Stage for EBPs and Best Practices at
DCF
 Delivering and Financing Behavioral Health Services
in Connecticut (Child Health & Development
Institute, 2000)
 Blue Ribbon Commission (2000)
 Report of the President's New Freedom Commission
on Mental Health, 2003
Highlights of DCF EBP and Best Practice
Dissemination
1998-99
First MST Team
2002-03
MDFT as part of Hartford Youth
Project
2002-03
2005-06
Roll-out of KidCare
Community-Based Services
In-Home Rates
2006-09
TF-CBT
2008-11
EMPS
2011-
FST – MDFT Conversion
Policy/Practice to Support EBPs
 Community Based Focus (KidCare)
 Results Based Accountability
 Enhanced Care Clinic Standards
 DSS Rehabilitation Option and DCF
Certification Regulation
 Performance Improvement Center Concept
 Use of Federal Block Grant Dollars
Policy/Practice to Support EBPs
 Appreciation of Implementation Science and Sustainability
 Preference for EBPs in budget decisions and new program




development
Support of Workforce Development
Reimbursement that covers costs of delivering care
Economics of Improved Outcomes (focus on quality and net
outcome versus quantity)
Support of “Service to Science” and “Implementation Drivers”
approach
Other Considerations
 Many “EBPs” but most are not ready for prime time
 Major Gaps in EBP Array
 Need more EBPs in Congregate Care/Inpatient
 Need more robust models for Outpatient– ChildSteps
(John Weisz)
 How much QA/QI is optimal (cost-benefit)
Julie Revaz, MSW
Manager of Programs and Services
CSSD’s Investment in EBPs
 Creation of the CSSD Center for Best Practices
 CPEC study in 2002
Resulted in Investment in Blueprint Programs





Multisystemic Therapy (MST)
Brief Strategic Family Therapy (BSFT)
Functional Family Therapy (FFT)
Multidimensional Family Therapy (MDFT)
Multidimensional Treatment Foster Care (MTFC)
EB and Promising Practices
 Gender-responsive
 Culturally-competent
 Trauma-informed
 Strength-based
 Community-based
 Motivational interviewing
 Risk, need and responsivity principles
 Quality assurance
 Cognitive-behavioral interventions / approaches
Lessons Learned in Program Implementation










Establish new services / make paradigm changes slowly.
Listen carefully. Promise small and deliver big.
Begin early to track and monitor outcomes.
Focus on organizational readiness to deliver EBPs.
Train providers in juvenile justice issues.
Adopt a multisystemic approach to the implementation
and maintenance of the EBP.
Empower QA staff to speak candidly. Hold the QA
group accountable for outcomes, too.
Attend to workforce development issues.
Develop a plan for transfer of knowledge.
Be sure to celebrate and scream about successes along
the way.
Juvenile Court Intake
Court Intake by Fiscal Year
18,000
16,000
15,857
14,000
16 yr-olds enter
Juvenile System
January 1, 2010
12,920
12,000
Number
10,000
11,944
11,333
10,910
9,728
9,231
8,149
8,000
11,140
8,702
6,000
4,000
4,947
3,192
3,192
2,713
2,000
2,438
0
FY 2007
FY 2008
FY 2009
FY 2010
Fiscal Year
Total
Delinquent
Status Offense
FY 2011
DCF Commitments
Juveniles Commited to the Department of Children and Families
1999-2010 and 2011 (Projected)
800
700
687
680
625
600
591
589
16 yr-olds enter
Juvenile System
January 1, 2010
Number
500
401
400
385
338
300
281
270
263
253
206
200
100
0
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
Juvenile Justice System Improvement
Project (JJSIP)
Building on research evidence about effective practice, the
Juvenile Justice System Improvement Project (JJSIP) aims
to change juvenile justice systems and practice by:
(a) implementing the Comprehensive Strategy framework and improving
programs system-wide through the use of the SPEP, and through
(b) use of the Standardized Program Evaluation Tool (SPEP).
The SPEP tool facilitates evaluation of the expected effectiveness of available
services and guides their improvement.
Robert P. Franks, Ph.D.
Vice President & Director
Connecticut Center for Effective Practice
Child Health & Development Institute
Evidence-based
Practice
# of Teams
# Children &
Adolescents Served
Annually
MST
6 (DCF)
15 (CSSD)
206 (DCF)
564 (CSSD)
1 (DCF)
16 (DCF)
1 (DCF)
15 (DCF)
1 (DCF)
12 (DCF)
2 (DCF)
60 (DCF)
MDFT
9
4
(DCF)
(CSSD)
225 (DCF)
180 (CSSD)
MDFT - RESIDENTIAL
2
2
(DCF)
(CSSD)
36 (DCF)
60 (CSSD)
FFT
4 (DCF)
3 (CSSD)
350 (DCF)
110 (CSSD)
BSFT
120 slots (CSSD)
360 (CSSD)
MTFC
1 (DCF)
20 (CSSD)
10 (DCF)
20 (CSSD)
IICAPS
125 (DCF)
15 (CSSD)
2,600 (DCF)
312 (CSSD)
MST Specialty Teams
MST- Problem Sexual
Behavior
MST-Building Stronger
Families
MST – Transitional Age
Youth
MST-Family Integrated
Transitions
Totals by Agency - DCF
CSSD
3,599 (DCF)
1,592 (CSSD)
Total in Connecticut
5,191 total
Therapist-rated
MST Client Ultimate Outcomes
Ultimate Outcomes (Achieved) (Table 5)
% of Cases
Is the youth currently living at home?
74.1%
Is youth attending school, vocational training,
or in a paying job?
76.8%
Youth has not been arrested since beginning
MST for an offense during MST?
73.4%
N = 1,764
Changes in Recidivism Over Time:
Pre to Post MST Convictions
(Table 6)
Time in Months
Pre-MST
Post-MST
12 mo.
3 mo.
6 mo.
12 mo.
FWSN
19%
0.1%
2%
3%
Status/Violation
33%
8%
12%
18%
Misdemeanor
46%
7%
13%
22%
Misdemeanor or
Felony
53%
10%
17%
29%
Felony
14%
4%
6%
11%
Any Offense
78%
16%
26%
39%
Example: MST Growth in CT
30
25
20
PILOTS
CSSD
DCF
15
10
5
0
1999 2000 2001 2002 2003 2004 2005 2006
State-wide dissemination
PRACTICE Components
 P sychoeducation & P arenting skills
 R elaxation
 A ffective expression and regulation
 C ognitive coping
 T rauma narrative development and processing
 I n vivo gradual exposure
 C onjoint parent child sessions
 E nhancing safety and future development
Slide 39
Map of agencies
Outcomes
(August 2007 – June 2010)
Number of Children Served
Child Outcomes
 N = 190 TF-CBT completers w/ pre-post data
 PTSD symptoms
 Child report: 43% decrease
 Parent report: 29% decrease
 Depression symptoms
 Child report: 55% decrease
 Parent report: 50% decrease
 **Almost 80% remission of PTSD diagnosis**
 High caregiver satisfaction
Summary
 A range of evidence-based practices are available in Connecticut and have been
proven to work
 The Behavioral Health Partnership has resulted in a variety of positive changes and
continued opportunities for system enhancement
 There has been a growing emphasis on maintaining children in their homes and
communities
 Consumer demand is increasing for models that are demonstrated to work and that
engage families
 There has been an increased use of evidence-based practices, especially intensive inhome models, that can be alternatives to out-of-home placements, but access is
limited and needs to increase
 As resources diminish there will be even more emphasis on accountability and
services which can demonstrate positive outcomes and saving of taxpayer dollars