Evidence-based policy and practice….
The Movement Toward
Evidence-Based Policy and Practice
What’s
all the
fuss?
Michelle Hughes, Benchmarks
Brett Loftis, Council for Children’s Rights
Co-Chairs, Intentional Death Prevention Committee
October 24, 2011
The Changing Landscape…
• Increased focus on accountability in multiple
sectors.
• Increased knowledge about affecting change.
– More information about “what works” and what
doesn’t
– Increased focus on measuring community-level impact
(are we moving the “community needle”)
• Stewardship of public/philanthropic dollars:
limited resources need to be used strategically.
• Focus on EBPs seen across all fields, in public and
private sectors, at all levels of government
Definition of Evidence-Based Policy
and Practice
• Evidence-based policy has been
defined as an approach that “helps
people make well informed
decisions about policies,
programmes and projects by
putting the best available evidence
from research at the heart of
policy development and
implementation” (Davies, 1999a).
Continuum of Evidence
Some Examples of Why RCT Important
• Continuum of “evidence”
• From anecdotes, qualitative interviews, case
studies, surveys, pre-post assessments without
comparison/control groups to “stronger
evidence” that include results from quasiexperimental and experimental evaluations.
• The more rigorous the evaluation design – the
more confidence we have that the results are
produced by the intervention and not some other
contributing factor (e.g., group with intervention
more motivated, chance)
Coventional wisdom is often wrong.
Medicine:
•Hormone replacement therapy for post-menopausal
women (increases risk of stroke and heart disease for
many women)
•Dietary fiber to prevent colon cancer (shown ineffective)
•Stents to open clogged arteries (shown no better than
drugs for most heart patients).
Cited from Jon Baron, NC Family Impact Seminar, 2009.
More Examples…Social Interventions
BUT evidence alone is not enough….!!!
Prevention :
•DARE
– ENORMOUS support for DARE. At one point, DARE
taught in 80% of the school districts in the United
States, in 54 other countries around the world, and
was taught to 36,000,000 students each year (Hanson, D
J., cited from www.alcoholfacts.org)
– Pre-post assessments indicate increased student
knowledge of risks, and improved attitude changes
toward using alcohol & drug use, however, RCTs
revealed program has no effect in reducing
alcohol/drug use.
Cited from Jon Baron, NC Family Impact Seminar, 2009.
Implementation MATTERS!
• Successfully replicating evidence-based programs
requires….
Proven practice +
fidelity/quality implementation
= Better Outcomes
• Fidelity: adherence to core elements which
contribute to effectiveness
• A poorly implemented practice/program will
yield poor outcomes (and results in a poor
investment)
Improved outcomes for
children and families
Implementation, Implementation,
Implementation….and….
IMPLEMENTATION!!!
Insufficient Methods
• Implementation by laws/compliance by itself does not
work
• Implementation by “following the money” by itself
does not work
• Implementation without changing supporting roles and
functions does not work
• Diffusion/dissemination of information by itself does
not lead to successful implementation
• Training alone, no matter how well done, does not lead
to successful implementation
Fixsen, Naoom, Blase, Friedman, Wallace, 2005
Why the CFTF Has Been So Effective
Im
(fidelity measurement)
Training
Selection
Systems
Intervention
rs
i ve
Dr
on
ati
ni z
ga
Or
Co
mp
et
en
cy
Dr
i ve
rs
Coaching
Facilitative
Administration
Integrated &
Compensatory
Leadership
Technical
© Fixsen & Blase, 2008
n
tio
ta
e n rs
em ve
pl Dri
Performance Assessment
Adaptive
Decision Support
Data System
• Uses data to inform our understanding of
needs/problems
• Asks the question -- where is our final
destination (what is the outcome we want)
• Uses evidence in policy making
– What do we know is effective in addressing this
issue?
– How do we know? What is the evidence?
Future Directions
Resources
• With social interventions, need for increased
focus both on the “what” (effective
interventions) AND the “how” (what does it take
to implement effectively)
• How can we begin to help support effective
implementation of recommended interventions?
• Coalition for Evidence-Based Policy
http://coalition4evidence.org/wordpress/
• Duke University Center for Child and Family Policy,
Family Impact Seminar on Evidence-Based Policy
http://familyimpactseminars.org/index.asp?p=1&page
=seminar&seminarid=181&siteid=34
• National Implementation Research Network
http://www.fpg.unc.edu/~nirn/
• NIRN Financing Evidence-Based Program and Practices
Brief
http://www.fpg.unc.edu/~nirn/resources/publications
/financing_paper_final_2008.pdf
– EFFECTIVE direct service delivery needs a supportive
infrastructure (clinical supervision, pre-service and inservice training, coaching, & program evaluation)
– NIRN Brief on Financing of Evidence-Based Programs
and Practices