Acknowledgements
y Collaborating Investigators:
{ Emilie
Smith (PI)
{ Daniel Perkins
{ Linda Caldwell
{ Wayne Osgood
{ Howard Rosen
y University and Community Partners
{
{
Hempfield Behavioral Health
Penn State Survey Research Center
y Funded by NIDA, W.T. Grant and Wallace
Foundations
Team Effort
y Penn State:
{ Sharon Childs (Project Coordinator),
{ Linda Halgunseth (Research Associate),
{ Jason Rose (Postdoctoral Fellow),
{ Kenyan Cattell (Administrative Assistant),
{ Alba Blandino (Staff Assistant),
{ Amanda Craig, Jill Schulte, Kaprea Johnson (Research
Assistants)
y Survey Research Center:
{ Lisa White, Julie Bressler, Michael Rausch, Sally Crandall,
Kurt Johnson, Director
y Hempfield Behavioral Health:
{ Alison Rosen (Supervisor),
{ Coaches - Tabatha Hahn, Linda Moulder, Jamie Shraudner
`
`
`
`
`
`
`
Nearly 7 million children attend afterschool programs (Capizanno, Tout &
Adams, 2000).
The U.S. Government spends 1 billion dollars in afterschool programming
Afterschool programs viewed as a “safe haven” from substance abuse,
delinquency, and teen pregnancy
Unsupervised and unstructured out-of -school time can be detrimental to
adolescent development (Mahoney, Stattin, & Lord, 2004)
Appropriately structure is beneficial (Durlak & Weissberg, 2007;
Gottfredson et al., 2004).
Interest in fostering positive youth development (PYD) (Eccles &
Gootman, 2002; Larson, 2000).
As such, afterschool is an important setting to examine role in children’s
prosocial adjustment and behavior.
Effects of Afterschool Programs on Children
y Large proportion of delinquency occurs between 3 and 6pm (Snyder &
Sickmund, 1999). Unsupervised out-of -school time can be detrimental
(Mahoney, Stattin, & Lord, 2004)
y Afterschool programs as a “safe haven” from substance abuse,
delinquency, and teen pregnancy
y Afterschool programs as a setting promoting positive youth
development (Larson, 2000).
y Benefits of afterschool programs are moderated by structure and
activities of the program (Durlak & Weissberg, 2007; Gottfredson et al.,
2004).
Developmental Characteristics of Afterschool
Settings (Eccles & Gootman, 2002)
y PASS Model
y P - Focus on prevention and promotion
y A – Agency, involvement, and leadership among
participating youth
y S – Structure that is developmentally appropriate
y S - Supportive relationships with adults and
potentially with peers?
The Good Behavior Game
y Long (40 year) history in school-based
intervention (Baer, Wolf et al., 1969)
y Focus of rigorous, randomized, longitudinal trials
(Kellam, Ialongo, Brown, and colleagues)
y Benefits for highest-risk aggressive boys in urban
schools (Baltimore, MD)
y Current “Science Migration” Study
Novel setting – Afterschool
{ Novel processes – “how” does GBG work?
{
Adult structure and support ?
Ù Child collective efficacy?
Ù Settings-level effects
Ù
What Is PAX-GBG ?
y Vision: “My Wonderful School”
y Posted behavioral guidelines
y Positive reinforcement and praise,
correction, redirection, (“tootle”
notes, spleems)
y Child involvement in cooperative
teams
y Training and consultation on
PAX-GBG from coaches
Theory of Change Logic Model: Fostering Appropriate Structure and Support
in Afterschool Settings
CONTEXTUAL
INFLUENCES
•Program
Mission
•Program
Infrastructure
ACTIVITIES
•Promote
structure and
support in
afterschool
•Staffing
•Training on
PAX-GBG
•Neighborhood/
Community
Factors
•Collaborative
visioning and
goal-setting
•Socio-Cultural
Context
•Weekly
consultation
and support
OUTPUTS
•Staff shared
norms
•Implementat
ion of PAXGBG
•Improved
Staff
structure and
support
•Staff shared
collective
efficacy
PROXIMAL
OUTCOMES
IMPACT: LONGTERM OUTCOMES
•Youth
shared
norms,
expectancies
•Reduced
chaos,
overcontrol)
•Supportive
adult and
youth
relationships
•Decreased
youth
problem
behavior and
adjustment
•Youth
Collective
efficacy
•Positive
Youth identity
and
acculturation
Sample of Afterschool Programs in
Southeast Pennsylvania
Study Design
Multi-level Data Collection
Time
Information
Source
Fall
Director Survey
Staff Survey
Student Survey (PDA)
Site Observations
Site director
Staff
Students
Project field staff
Winter
Site Observations
Project field staff
Spring
Director Survey
Staff Survey
Student Survey (PDA)
Site Observations
Site director
Staff
Students
Project field staff
Ongoing
Attendance
Implementation Data
School-Related Data
Program Files
Observational Interrater Reliability- 3 Waves
Interrater reliability : Average % agreement across waves
Measure
Promising Practices
Ratings Scale
Arnett Caregiver
Interaction Scale
Youth Program
Quality Assessment
Subscale
Fall 2009
Winter 2010
Spring 2010
Supp. Rel. w/ Adults
69.7
70.61
76.58
Supp. Rel. w/ Peers
65.21
71.48
70.7
Appropriate Struct.
68.33
66.52
71.13
67.8
56.25
66.04
Chaos
68.56
75
82.91
Harshness
80.43
88.46
87.85
Sensitivity
65.37
69.23
80
Detachment
67.44
74.28
78.31
Permissiveness
58.53
65.14
76.05
Active engagement
74.62
76.52
77.78
Belonging
80.98
88.07
83.97
Conflict resolution
75.58
75
80
Staff pos. engage.
76.63
90.91
82.69
Responsibility
71.74
72.09
77.63
Choice
82.61
100
80.77
Level of Engage.
Fostering Inter-rater Reliability
y Gold Standard Video
Process
{
{
{
{
Establish GSV by group of
scientific experts
Establish definitions and
GSV ratings via consensus
process
“Certify” observers with
GSV pre-deployment
Examine in-field interrater reliability
Using Technology in Data Collection
y Observers Use “Ruggid”
y
y
y
y
y
Laptops
Facilitate efficient
examination of inter-rater
reliability
Student Surveys conducted on
PDA
Infused with children’s
cartoon and jokes as breaks
(Remember the Roadrunner?)
Keeps children’s attention and
interest
Allows daily transmission of
survey data from the field
Child Survey Measures
Implementation
`
Questions
◦ Allows us to see ‘How much’ and for ‘How long’ GBG is
being used
◦ Eventually, will allow us to see if GBG implementation
relates to changes in settings and child behavior
`
Two Forms
◦ Staff Report – Calendar
◦ https://online.survey.psu.edu/weeklyobservation/
◦ Coaches Report – Observations
`
Bi-Weekly Feedback
◦ to staff
◦ to coaches
Test of Conceptual Model: Role of
Children’s Collective Efficacy
{
Derived from agentic theory
Ù
{
Distinguishes self-efficacy from collective efficacy
Ù
{
Based upon premises of community, agency and empowerment (Chavis
& Wandersman, 1990; Rappaport, 1981; Riger, 1993 Sarason, 1976;
Zimmerman, 1995)
Recent Research
Ù
{
CE is not simply the accumulation of individual self-efficacy, but rather
the group’s shared belief of achieving a common goal (Sampson, et al.,
1999).
Connected to Community Psychology
Ù
{
Views individuals as people who can “produce desired effects” when they
have “incentive to act.” (Bandura, 1997, 2000).
Interest has been rejuvenated by the work of Sampson, Raudenbush and
Earls (1999)
Definitions of CE
Ù
Defined as a sense of mutual trust and cohesion and willingness to
“intervene on the behalf of the common good.”
Research on Collective Efficacy
{
{
{
{
Moderator of crime and violence in poor, at-risk neighborhoods. (Sampson
et al., 1999).
Role of “intergenerational closure” (adults and children feeling linked to
each other)
Yet, intergenerational closure, exchange, and social control have been found
in neighborhoods with more concentrated affluence
Studies of rural African American parents demonstrate that increases in CE
were related to
Ù
Ù
{
Collective efficacy (CE) among public school teachers
Ù
{
{
more authoritative parenting and
less youth peer deviance and delinquency (Simons, Simons, Burt, Brody, & Cutrona, 2005)
Related to student’s academic achievement (Goddard, 2001; Goddard, Hoy &
Hoy, 2000, p.480).
However, CE was lower in schools with more socioeconomic disadvantage.
Very little research on children’s collective efficacy
Developmental Model of Children’s Collective Efficacy, Adjustment, and
Behavior
Wave 1 and Wave 4
Grade
-.02
e2
SDQ
Difficulties Scale
Gender
.21
-.27
.21
.34
COLLECTIVE EFFICACY
-.34
e4
like to try smoking someday?
Fit Index
x2
CFI
RMSEA
.76
.01
tried smoking already?
Collective Efficacy Model
7.92(7), p>.05
1.00
.00
e1
e5
What Are We Learning?
y Fit of our conceptual model
{ Supports role of children’s collective efficacy in afterschool
y Ways to foster quality measures of afterschool
settings
y Trends in implementation of various strategies in
afterschool
y Preliminary findings on staff appropriate structure
and support (e.g. decreased harshness and
detachment)
y Potential ways to help strengthen afterschool
programs
There is promise that we can
strengthen afterschool programs!
Questions and Comments?