Identifying and Implementing
Evidence-Based Mental Health
and Substance Abuse Services
September 16, 2005
Kevin D. Hennessy, Ph.D.
Science to Service Coordinator
Substance Abuse & Mental Health Services Administration
U.S. Department of Health & Human Services
“A Life in the Community For Everyone”
“Building Resilience & Facilitating Recovery”
A Cross-Cutting Principle:
Science to Services/Evidence-Based Practices
How do we translate research into practice?
How do we connect services to science?
An old and well-defined problem
Few research innovations are implemented successfully in typical
health care settings
Many widely used interventions have little if any research support
Most in treatment don’t receive evidence-based interventions
Many reports exist documenting the problem and recommending
actions
Bridging the Gap Between Research and Practice: Forging Partnerships
with Community-Based Drug and Alcohol Treatment (Institute of Medicine
Report) – 1998
Mental Health: A Report of the Surgeon General – 2000
Achieving the Promise: Transforming Mental Health Care in America (The
President’s New Freedom Mental Health Commission) – 2003
Knowledge to practice gap remains at 15-20 years
Original research
18%
Negative
results
variable
Dickersin, 1987
Submission
46%
Koren, 1989
Negative
results
0.5 year
Kumar, 1992
0.6 year
Kumar, 1992
Acceptance
Publication
35%
Lack of
numbers
Balas, 1995
0.3 year
Poyer, 1982
Expert
opinion
Bibliographic databases
50%
Poynard, 1985
Inconsistent
indexing
17:14
6. 0 - 13.0 years Antman, 1992
Reviews, guidelines, textbook
9.3 years
Implementation
It takes 17 years to turn 14 per cent of original research
to the benefit of patient care
SAMHSA Activities for Reducing
the Knowledge – Practice Gap
Identify evidence-based practices (NREPP)
Develop guides, manuals and other materials
Evidence-Based Practice Implementation Resource Kits
Disseminate materials and provide trainings through
regional centers
Centers for the Application of Prevention Technology (CAPTs)
Addiction Technology Transfer Centers (ATTCs)
Center for Mental Health Services Technical Assistance Centers
Create interagency and private-public partnerships
National Registry of
Evidence-based Programs and
Practices (NREPP)
Purpose: A voluntary rating and classification system for
mental health and substance abuse prevention and treatment
interventions.
Goal: NREPP will become a leading national resource for
science-based information on substance abuse and mental
health prevention and treatment interventions.
NREPP Timeline
1998…Started with a focus on substance abuse prevention as
the National Registry of Effective Prevention Programs
1998-2003…reviewed and rated over 1,100 substance abuse
prevention programs
information on over 150 Model, Effective, and Promising
Programs on web site (www.modelprograms.samhsa.gov)
2004…initial expansion of system to include substance abuse
treatment, mental health promotion, and mental health
treatment programs
NREPP
Current Status
Formal public comment process by SAMHSA this summer
Notice in August 26th Federal Register
Available through SAMHSA web site – access through “Quick Picks” on
SAMHSA’s home page (www.samhsa.gov)
Written comments due no later than October 25th
U.S. Mail – SAMHSA, c/o NREPP Notice, 1 Choke Cherry Road, Rockville,
MD 20877
Electronic Mail – nreppcomments@samhsa.hhs.gov
Posting of recently completed reviews (using older NREPP system)
by Fall 2005
No new reviews until 2006
New NREPP Website will be launched in 2006:
www.nationalregistry.samhsa.gov
NREPP: Comprehensive & Transparent
Comprehensive
Scientists independently evaluate and classify outcomes
for programs/practices based on statistical significance,
replications and 16 evidence quality criteria
Program impact and practical significance will be
reported through effect sizes
For status programs, additional information on
implementation, training, quality tools, staffing, costs,
etc. will be provided
NREPP: Comprehensive and Transparent
Transparent
Review results will be posted on the new NREPP website
By 2006 Web site will feature:
 Outcomes searchable database
 Links to educational materials on review criteria
 Self-assessment tool for candidate programs
 Links to technical assistance centers and supports
NREPP: Draft Rating Criteria
Two Types of Criteria
Evidence Rating Criteria
“Utility Descriptors”
NREPP: Evidence Rating Criteria
16 evidence rating criteria applied to each
program outcome (0 to 4 scale).
Include:
Theory-driven selection of measures & analytic methods
Reliability
Validity
Intervention and comparison fidelity
Assurances to participants
Standardized Data Collection
Selection bias
Attrition and missing data
Analysis meets data assumptions
NREPP: “Utility Descriptors”
Look beyond scientific evidence and ensure that
programs are able to be implemented.
Assessments also made of the following:
Implementation
Quality Monitoring
Unintended or Adverse Events
Population Coverage
Cultural Relevance
Staffing
Cost
NREPP: Questions to the Public
From the August 26th Federal Register Notice (FRN)
Is proposed system objective, transparent, efficient, and scientifically
defensible?
How might SAMHSA engage stakeholders in determining priority
review areas?
How best to use statistical significance and measures of effect size in
NREPP?
Beneficial to use multiple categories of effectiveness?
Approach to assessing and presenting cultural relevance?
NREPP: Questions to the Public
From the August 26th Federal Register Notice (FRN) - continued
Approach to re-reviewing existing NREPP programs?
Types of technical assistance needed to promote adoption of NREPP
interventions?
How best to involve consumers, families, and other non-scientists in NREPP?
What, if any, guidance on use of NREPP interventions within block grants?
How best to promote other sources – e.g., clinical judgement, consumer
values, etc – in decisions regarding selection, delivery and financing of
services?
Panel to annually review NREPP operational and technical suggestions?
NREPP’s Reach
Influences SAMHSA discretionary and block
grant investments
Serves as a resource for states and communities
seeking to implement evidence-based Mental
Health & Substance Abuse prevention and
treatment services
Provides an important tool for both public and
private purchasers in selection of effective
services
Publications and Materials on
Best Practices in Mental Health and
Substance Abuse Prevention & Treatment
What type of Products?
•
•
•
•
•
•
Training Manuals
Evidence-based Implementation Resource Kits (CMHS)
Interactive Web-based Technical Support (Prevention Platform)
Treatment Improvement Protocols (TIPs)
Quick Reference Cards & Pocket-size Guides
Fact Sheets, Brochures, Reports, and Periodicals
Publications and Materials on
Best Practices in Mental Health and
Substance Abuse Prevention & Treatment
Who Uses the Products?
Program Administrators & Staff
Clinicians
Policymakers
Consumers
Where to Find the Products:
For SA Treatment go to www.kap.samhsa.gov
For SA Prevention go to www.prevention.samhsa.gov
For Mental Health go to www.mentalhealth.samhsa.gov
Evidence-Based Practice
Implementation Resource Kits
Focus on six practices
Assertive community treatment
Family psychoeducation
Supported employment
Integrated dual disorders treatment
Illness management and recovery skills
Standardized pharmacological treatment
Two Phases
I – Development (Fall 2000 – Summer 2002)
II – Pilot-Testing (Summer 2002 – Summer 2005)
More information is available through SAMHSA’s web site
(www.samhsa.gov) under “Mental Health System Transformation”
Evidence-Based Practice
Implementation Resource Kits
Kits have targeted and distinct components for:
Consumers
Family and Other Supports
Practitioners and Clinical Supervisors
Mental Health Program Leaders
Public Mental Health Authorities/Administrators
Kits include:
Research summaries
Information sheets for all stakeholders
Tip sheets for program leaders and administrators
Program manuals/workbooks for practitioners
Introductory and training/demonstration videos
Fidelity assessment tools and cultural competence statements
Outcome measures
Materials, Trainings & Technical Assistance
for Prevention
Centers for the Application of Prevention Technology
www.captus.org
Purpose: Assist States and Communities in the application
of evidence-based substance abuse prevention programs,
practices, and policies
Goal: To increase the impact of prevention
Materials, Trainings & Technical Assistance
for Treatment
SAMHSA’s Addiction Technology Transfer Centers
www.nattc.org
Purpose: To transmit the latest knowledge,
skills and attitudes of professional addiction
treatment practice
Goal: To enhance clinical practice
Center for Mental Health Services
Research, Training and TA Centers
Currently 55 Centers provide a range of services – many
are free but charges may apply for some:
Technical assistance
Information and referrals
On-site consultation
Training
Library services
Publications
Other resources
More information is available through SAMHSA’s web site:
www.mentalhealth.samhsa.gov/links/
Promoting Interagency Partnerships
Collaborations with National Institutes of Health to
jointly fund state planning grants facilitating
implementation of evidence-based practices
– RFA-MH-03-007 – State Implementation of EvidenceBased Practices: Bridging Science and Service (with
NIMH)
– RFA-MH-05-004 – State Implementation of EvidenceBased Practices II: Bridging Science and Service (with
NIMH)
– RFA-DA-05-002 – Enhancing State Capacity to Foster
Adoption of Science-Based Practices (with NIDA)
Creating Public – Private Partnerships
Partnership between:
SAMHSA’s Center for Substance Abuse Treatment (CSAT)
Robert Wood Johnson Foundation (RWJF)
Goals of improving organizational processes to facilitate client access
and retention in addictions treatment. Initial successes include:
Reduced Waiting Times by 31% (n=24)
Reduced No-Shows by 22% (n=13)
Increased Admissions by 25% (n=19)
Increased Treatment Continuation Rates by 33% (n=8)
More info available through web site – www.niatx.org
Advancing Science to Service Efforts and
Reducing the Knowledge – Practice Gap
Focus on elements of health care redesign
detailed in Crossing the Quality Chasm (Institute
of Medicine, 2001)
– Apply evidence to health care delivery
– Align payment policies with quality improvement
– Prepare the workforce
– Use information technology
In Conclusion
“The future is here.
It’s just not widely distributed yet.”