Identifying and Developing
Evidence Based Practices
Presentation to:
Implementing Evidence-Based Practices and Performance
Measures for Massachusetts Mental Health Services: An
Educational Forum
Brandeis University
September 28,2005
Steve Leff
The Evaluation Center@HSRI
Human Services Research Institute
Who Wants to Identify EBPs? Consumers
Evidence-based practices is a new metaphor to me. I
thought evidence was the source of mental health
practices. Are you saying it was not? May still not be?
On what are practices then based? At no time have I
ever heard that people in the mental health
profession based their acts upon anything other than
evidence. Was that a lie? Is this "new" metaphor also
a lie? And how would I tell the difference? (H. A.
Maio, personal communication, May 17, 2002)
Human Services Research Institute
Who Wants to Identify EBPs?
States
Oregon SB 267 requires that 25 percent of state fund be used to
treat people with mental illness who use or have a propensity to use
emergency mental health services. In 2007-09, the percentage of
funds to be spent on EBPs increases to 50 percent and in 2009-2011
to 75 percent.
Instructions for Filling out OMHAS Application for an EBP Review
The Office of Mental Health and Addiction Services will consider any
practice for inclusion on its list of approved evidence-based
practices. To have a practice considered for this list the attached
application must be filled out completely with all requested
information. The Office will review the information provided in
accordance to criteria outlined in the OMHAS Operational Definition
for Evidence-based Practices...Practices that do not meet those
criteria will be noted as reviewed. The office may request additional
information if needed.
Human Services Research Institute
Who Wants to Identify EBPs? Providers
ValueOptions
EVIDENCE-BASED PRACTICE GUIDELINES
Use of evidence-based practice guidelines is the clinical
foundation of disease management. One way in which
this is accomplished is through the employment of
consensus- and evidence-derived treatment algorithms,
which incorporate current science, including strategies,
and tactics for patient care. …In order to successfully
implement an algorithm-driven treatment approach,
ongoing training, technical assistance, and support for
clinicians in the use of algorithms is necessary.
Human Services Research Institute
Identifying EBPs in My Crystal Ball
• EBPs will be identified by registries and venues for the
•
•
•
•
•
•
reporting of meta-analysis
There will be explicit criteria for recognizing practices
Federal, state, and private payer listings of EBPs will
converge
Listings will become “Arbiters of value” for most, if not
all consumers, payers and policy makers
This will alter the nature of mental health services
Favor some practices and not others
Make favored practices more consistent and expensive
Human Services Research Institute
Registries
• Registries employ scientists to review evidence for individual
“brand name” interventions
• “Brand name” interventions package patented or copyrighted
materials and training
• To use you have to purchase these products and services
• 35% of programs listed on SAMHSA’s National Registry of
Evidence-based Programs and Practices were distributed by
for-profit companies
• National Registry of Evidence-based programs and
Practices (NREPP) - SAMHSA
http://modelprograms.samhsa.gov/textonly.cfm?page=nreppover
• What Works Clearinghouse - DOE http://www.whatworks.ed.gov/
Human Services Research Institute
Meta-analyses
• Meta-analyses review generic practices which are sets of
interventions with specified common characteristics
• They are usually not copyrighted or patented, often they
haven’t been “packaged”
• Prepared and updated by scientists working independently or
as members of organizations
• Published on the web, in journals, and in printed reports
• “Meta-analytically supported treatments” (MASTs) - Will Shadish
(University of Memphis)
• Cochrane Collaboration - http://www.thecochranelibrary.com
• Campbell Collaboration -
http://www.campbellcollaboration.org/index.html
• Evidence-based Mental Health -
http://ebmh.bmjjournals.com/
Human Services Research Institute
Example of a Brand Name EBP: Multisystemic
Therapy : http://www.mstservices.com
•
MST Services is at the center of efforts to disseminate Multisystemic
Therapy within the United States and internationally. MST Services
provides assistance with all aspects of program development and staff
training through licensing agreements with the Medical University of
South Carolina and the Family Services Research Center.
•
Core MST clinical training package and other components

Pre-training organizational assessment and assistance

Master's-level therapists or highly competent bachelor's-level
professionals

initial 5-day training,

Weekly on-site supervision from doctoral level mental health
professionals

quarterly booster training, and

monitoring of fidelity and adherence to the MST treatment model.

formal outcome tracking systems.

outcome-focused personnel evaluation methods.
Human Services Research Institute
Example of a Meta-analysis Site
The Cochrane Library (ISSN 1464-780X).
Published by John Wiley & Sons, Ltd.
Vocational rehabilitation for people with severe
mental illness
Crowther R, Marshall M, Bond G, Huxley P
Authors' conclusions
Pre-vocational Training places persons in competitive
jobs after a period of preparation. Supported
Employment place clients in competitive employment
immediately whilst providing on-the-job support.
Supported employment is more effective than Prevocational Training in helping severely mentally ill
people to obtain competitive employment. There is no
clear evidence that Pre-vocational Training is effective.
Human Services Research Institute
Feeding the Pipe Line
• Consumers will want new evidence based interventions
proven to address recovery and other unmet needs
• Diverse populations will want interventions tested with
their groups and in their geographic areas
• Providers and developers will want their interventions
recognized as evidence based
• Registries and meta-analysis sources will want to
identify additional evidence based practices
Human Services Research Institute
Coming Soon To A Computer Near You
• Becoming Evidence Based: An Interactive Guide to
Planning and Grading Effectiveness Studies
• The What’s Evidence Library: Selected Readings on the
Nature of Good Evidence
• Meta2base: Summaries of Scientific Reviews of Mental
Health Practices
Visit The Evaluation Center@HSRI at
http://www.tecathsri.org/
Human Services Research Institute