Supporting the High-Quality Implementation and Sustainability
Of Evidence-based Programs: The Role of Community Coalitions
Brittany L. Rhoades, Brian K. Bumbarger, Julia Moore, & Sebrina Doyle
The EPISCenter, Prevention Research Center, Penn State University
Introduction
Research on the CTC Model in Pennsylvania
Communities That Care: Creating Fertile Ground for Prevention
Over 10 years of research in Pennsylvania has demonstrated the importance of well-functioning coalitions for the high quality implementation
of Evidence-Based Programs (EBPs) and improved population-level youth outcomes.
CTC CHARACTERISTICS
Form local coalition
of key stakeholders
Re-assess risk
and protective
factors
Collect local data
on risk and
protective
factors
Leads to community
synergy and
focused resource allocation
Use data to
identify
priorities
Select and implement
evidence-based program
that targets those factors
THEORIZED ADVANTAGES OF COALITIONS
•
• Comprised of local key
stakeholders
• Choices are data-driven
• Programs implemented
are evidence-based
• Receive adequate
training and technical
assistance to support
fidelity and sustainability
• Receive regular
evaluation and feedback
Proxy prevention system,
increasing community readiness
for adoption and implementation of
evidence-based programs (EBPs).
•
Improves EBP implementation
quality and population-level public
health outcomes
•
Greater community connections
may also increase and diversify
program support and the likelihood
of sustainability.
Historical Context of CTC in Pennsylvania
A TEST-BED FOR TYPE 2 TRANSLATIONAL RESEARCH
•
The Pennsylvania Commission on Crime and Delinquency (PCCD) has promoted
and supported Communities that Care (CTC) coalitions since 1994.
•
Nearly 120 CTC coalitions were formed from 1994-2001; approximately 70
continue to function.
•
Through CTC and non-CTC coalitions, PCCD has funded nearly 200 replications
of a variety of evidence-based programs (EBPs).
• Since 1998, Penn State’s Prevention Research Center (PRC) has conducted a
series of cross-section and longitudinal studies to understand what factors
influence effective processes and outcomes for both CTC teams and for PA youth.
• Through annual web-based surveys to EBP grantees & CTC coalition members,
the Evidence-based Prevention and Intervention Support Center (EPISCenter)
and PRC collect data on program planning, connection to and functioning and
support of coalitions, program implementation, barriers & assets, and technical
assistance needs.
• The PA Youth Survey (PAYS), a statewide survey of 6th, 8th, 10th, and 12th grade
students, was used to study CTC’s effectiveness at reducing youth problem
behaviors and their associated risk and protective factors.
Training and Technical Assistance
•Technical assistance includes feedback reports based on responses of CTC
board members and staff to an annual online survey regarding functioning of the
coalition. The data is used to problem solve and create action plans to improve
CTC functioning (Feinberg, Puddy, Gomez, & Greenberg, 2008).
•Receiving sufficient training in the CTC model has lead to:
-Positive coalition member attitudes toward prevention
-Better CTC coalition functioning
•Technical assistance dosage is related to better board functioning in younger
CTC sites (Feinberg, Greenberg, Osgood, Anderson, & Babinski 2002; Feinberg,
Ridnour, & Greenberg, 2008)
CTC Coalition Functioning
In a recent study, 146 representatives from organizations implementing EBPs
and connected to some form of coalition completed surveys. 63% of respondents
reported being affiliated with a CTC coalition.
The data indicated
that CTC Coalitions
performed better
than non-CTCs in
areas related to
connection,
functioning and
support.
Coalition Effectiveness in CTC and Non-CTC Coalitions
Support for High Quality Implementation of EBPs
The following factors have been found to predict a CTC site’s ability to support
high quality implementation of EBPs (Brown, Feinberg, & Greenberg, 2010):
• Coalition age: older sites provide less support, perhaps as a result of ‘mission
drift’
• Community poverty: higher poverty sites provide less support
• Coalition functioning: sites with higher levels of funding, leadership strength,
board efficiency, strong internal/external relationships, fidelity to the CTC model
provide greater support
Sustainability of CTC Coalitions
• 66% of CTC sites were operating 4 years after termination of the 3-year state
seed grant.
• Factors that predict fundraising success and sustained CTC activity include:
-Board member knowledge of prevention
-Positive coalition functioning (leadership quality, member involvement,
communication,
social cohesion, task focus)
-Fidelity to CTC model
-Planning for sustainability
(Feinberg, Bontempo, & Greenberg, 2008; Gomez, Greenberg, & Fienberg, 2005)
Population Level Youth Outcomes
Research from cross-sectional and longitudinal studies comparing youth reports on
the PA Youth Survey in CTC communities vs. youth in similar non-CTC communities
showed differences in risk and protective factors and behavioral outcomes.
A cross-sectional study (Feinberg, et al., 2007) found:
• Less favorable attitudes toward antisocial behavior and ATOD use
• Greater school commitment
• Fewer friends who showed drug using or delinquent behavior
• Less alcohol/cigarette use and delinquent behavior
• These effects were strongest and most consistent for CTC sites implementing EBPs
that targeted the grade cohorts that were the focus of this study
In addition, a longitudinal study found similar results when examining change across time:
• CTC communities had 7X as many positive effects than would be expected by chance
• CTC communities that were implementing universal EBPs had 11X as many
Longitudinal Study of CTC in PA
We followed 419
age-grade cohorts
over a 5-year period
and found youth in
CTC communities
using EBPs had
significantly lower
rates of delinquency,
greater resistance to
negative peer
influence, stronger
school engagement
and better academic
achievement
35
33.2
30
25
20
15
10
16.4
10.8
10.8
5
0
Delinquency
< Negative Peer Influence
Academic Performance
School Engagement
Feinberg, Jones, Greenberg, Osgood, & Bontempo (2010)
Research Summary
1. CTC board functioning (communication, task-focus, leadership, social
cohesion) is a key predictor of local perceptions of CTC effectiveness, ability to
support the high quality implementation of EBPs and sustainability of CTCs
2. Regular evaluation and feedback to local sites assists the joint work of TA
providers and CTC boards in maintaining high-quality site functioning and
program fidelity
3. CTC board functioning is dependent on adequate training and ongoing
technical assistance (TA), and as a result PA has undergone continuous quality
improvement to increase the quality of technical assistance
4. With such support, CTC sites are capable of
achieving positive population level change
The EPISCenter is a project of the Prevention Research Center, College of Health and Human Development, Penn State University, and is funded by the Pennsylvania Commission on Crime
and Delinquency and the Pennsylvania Department of Public Welfare as a component of the Resource Center for Evidence-Based Prevention and Intervention Programs and Practices.
Please visit www.episcenter.psu.edu