Effects of a comprehensive character
education program on student smoking
– Findings from 4 studies
Brian R. Flay, D.Phil.
Professor, Public Health
Oregon State University
Corvallis, OR
Society for Research on Nicotine and Tobacco, Portland, OR, Feb 29 2008
Outline
I. The Positive Action program
philosophy and theory
II. The Positive Action program
components
III. Evaluations and results
IV. Conclusions
2
I. Overview of PA Program
• Positive Action (PA) is a comprehensive school-based program
designed to
– promote student character and positive behavior,
– prevent an array of student problem behaviors, and
– improve student achievement.
• PA is grounded in a broad theory of self-concept that posits
– (a) students’ self-concepts and characters are determined by how they
behave and how they feel about themselves when they do various
behaviors and
– (b) making positive and healthy behavioral choices results in feelings
of self-worth.
• Major features of the program include:
– Teacher delivered scripted PA curriculum lessons in classroom
– Teacher and school staff modeling/reinforcement of “PA
behaviors” inside and outside of the classroom
– School-wide activities (e.g., PA assemblies) led by principal and
PA Committee
3
Positive Action Overview
1. The core of Positive Action is contained in an underlying
philosophy.
2. The philosophy is described in the Thoughts-ActionsFeelings Circle.
3. Positive Action teaches the positive actions for the whole
self through six units that are contained in five program
components.
4. The five completely prepared components are:
1. K–12 curriculum,
2. Climate Development,
3. Counselors program,
4. Family program, and
5. Community program.
4
Basic Philosophy (Theory of Action)
of the Positive Action Program & Circle
You feel good about
yourself when you do
positive actions.
Ref. (Reference)
Cognitive Behavior Therapy
and Positive Psychology
5
Basic Philosophy (Theory of Action)
of the Positive Action Program & Circle
...and
You feel bad about
yourself when you do
negative actions.
C.F. (Reference) Depression
6
II. The POSITIVE ACTION Program
Components
K–12 classroom curriculum
over 1,200 lessons - using Teacher’s Kits (manuals and materials for each
grade), classroom teachers present 15–20-minute lessons
Principal’s Kits (Elementary and Secondary)
a school-climate program to promote the practice and reinforcement of
positive actions in the whole school population (students and staff)
Counselor’s Kit
used with selected individual students, small groups and families
Family Kit
contains prepared weekly home lessons paralleling the school program
along with school parent-involvement activities
Community Kit
manuals and materials that align and encourage collaboration of all the
environments (schools, families and community) involved in the program
7
Positive Action Focus Units
(Learning Goals)
• In the classroom curriculum and all other materials,
the Positive Action content is taught through six focus units.
Unit 1: Self-Concept: What It Is, How It’s Formed, and
Why It’s Important (Philosophy & Circle)
Unit 2: Physical and Intellectual Positive Actions for a
Healthy Body and Mind (includes motivation to learn)
Unit 3: Social/Emotional Positive Actions for Managing
Yourself Responsibly
Unit 4: Social/Emotional Positive Actions for Getting
Along with Others by Treating Them the Way
You Like to Be Treated (Social-Emotional Skills & Character)
Unit 5: Social/Emotional Positive Actions for Being
Honest with Yourself and Others (Mental Health)
Unit 6:
Social/Emotional Positive Actions for Improving
Yourself Continually (Setting & Achieving Goals)
8
Logic/Theoretic Model of the Expected
Effects of the Positive Action Program
Program Components
Climate Development,
Family Kit,
Teacher/Staff Training,
K–12 Instruction
Curriculum,
Drug Education
Supplements,
Community Kit,
Counseling Kit
Immediate Outcomes
* Improved relationships among school
administrators, teachers, parents & community.
* Improved classroom management.
* Increased involvement of school with
parents & community.
PA Unit
1.
2.
3.
4.
5.
6.
Improved character/self-concept
Learning/Study skills
Self-Management
Interpersonal/social skills
Self-honesty, responsibility
Goal setting, future orientation
Expected Effects
Attitudes
Toward
Behaviors,
Social
Normative
Beliefs,
SelfEfficacy
Improved
Social
and
Character
Development
Improved
Learning
Environment
Expected Impact
Fewer Disciplinary
Problems; Reduced
Substance Use;
Less Violence
Improved School
Attendance, Grades
and Test Scores
9
Elementary Curriculum
Kits - Grades K–6
•
K–6 Instructor’s Manuals
– 140 15-minute lessons
• Role-playing
• Plays
• Stories
• Questions
• Poetry
• Games
• Student activity sheets/booklets
and materials for 30 students
• Posters
• Music
• Reinforcement stickers
• Hands-on activities
–Puppets
–Flannel board characters
–Games
–Others
10
Drug Education Curriculum
Grade 5 and Middle School
• Grade 5 Drug Education
Supplement Instructor’s Manual
• Middle School Drug Education
Supplement Instructor’s Manual
–18 15- to 20-minute lessons
–30 15- to 20-minute lessons
• Stories
• Questions
–Student activity booklets and
materials for 30 students
–Posters
–Music
• Stories
• Questions
–Student activity booklets and
materials for 30 students
–Scripted play: “Escape from
the Shadows”
–Games
–Posters
–Music
• Lessons are taught at the end of
each unit of the regular PA
curriculum.
• The kits can also stand alone.
11
Elementary Climate Kit
•
•
•
•
Principal’s manual
Support Staff manual
Parents’ manual
Assemblies
– Certificates of Achievement
– Positive Action Balloons
•
•
•
•
•
Positive Action Calendar
Words-of-the-Week Cards
Reinforcement Stickers
ICU Boxes and Notes
Positive Action Tokens
•
•
•
•
Positive Notes
Notepads
Positive Action News
Music – 27 songs
– CDs
– Song books
• Posters
• Video
– Overview
• Implementation Plan
• Scope and Sequence
Booklet
12
Other Program Components
• Counseling Program
– Counselor Kit
– Lessons for individual students or small groups
• Family Program
– Classes
– Home Kit
• Community Program
– Community messages and activities
– Civic engagement
– Media messages
13
III. Four studies and results
• I. Intensive Case Study
• II. Matched control study using archival data
• III. RCT in Hawaii elementary schools
• IV. RCT in Chicago Public Schools
14
Study I: Intensive case study of program
implement in a small rural school
• First year of PA in a rural Title I school in
Northern Florida.
• Variation in level of implementation
– None or Some (7 classrooms)
– Almost All (7 classrooms)
– Every Component and Lesson (11 classrooms)
• Students, Teachers and Parents surveyed at
beginning and end of year.
15
Clear dose-response relationship between level of program
implementation and student smoking in grades 4 and 5 (Χ2 =
12.6, p < .01, with no diffs. at pretest)
Grade 4 and 5 Smoking by Level of PA
0=Never, 1=Once, 2=More than once
Ever smoked
pretest
2.00
Ever smoked
posttest
Mean
1.50
1.00
0.50
0.00
None
Medium
Level of PA Implementation
Total
16
Study II: Matched control study
using school-level archival data
•
School Report Card data to find matching control schools for the PA schools
– Poverty (as indicated by percentage of free/reduced lunches)
– Mobility rates
– Ethnic distribution
•
We had data on feeder patterns for each middle and high school.
•
Data on whether or not each elementary school had actively implemented
PA for 4 or more years.
– 55 PA schools, 29 non-PA schools
•
Calculated % of PA graduates in each middle and high school.
•
Archival data included reports of disciplinary actions for cigarette smoking
•
ANCOVA adjusting for school characteristics.
•
Flay & Allred, Prevention Science, 2003
17
Effects of Elementary School PA on
Middle School Cigarette Smoking
20
19
18
16
16
14
13
12
10
<65%
65-75%
>75%
Percent of students from PA schools
ANCOVA one-tailed p = .03
18
Effects of Elementary PA on HS Tobacco Use
85.00
Mean Number of tobacco incidents
80.00
75.00
70.00
80.25
72.20
65.00
64.21
60.00
<15%
ANCOVA one-tailed p = .04
15-27%
27-50%
% Elementary PA
19
Study III: Randomized Trial in
Hawaii Elementary Schools
• Elementary schools (N = 20) on 3 Islands
• 5yr Effectiveness Trial
• Random assignment
– PA or Control conditions
• Strata matched on multiple indicators of risk
• PA schools received:
– 4yrs (2002-03 through 2005-06)
– Teacher training
• Developer, Carol Allred 3-4hrs,initial yr; booster
sessions 1-2hrs
• Technical Support, Project Coordinator Howard
Humphries
• Proc Mixed ANOVA analyses accounting for
students clustered within schools
20
Baseline Equivalence: 2000-01
100
90
Program Schools
80
Control Schools
70
60
50
40
30
20
10
SAT Read: %
School Demographics
Disp. R's per
10 students
Discplunary
Referals
Absentees
%
Above
Average
Below
Average
Aabove
Average
Below
Average
LEP %
Sp Ed %
PerCap/1000
Budget /day
Lunch %
Stability %
Enrol/10
0
SAT Math: %
Achievement
Behavior
No differences are close to being statistically significant except % free/reduced lunch p = .099
21
Baseline Equivalence on Ethnic Distribution
40.00
Program Schools
35.00
Control Schools
All of Hawaii
30.00
Percent
25.00
20.00
15.00
10.00
5.00
Am
.I
nd
ia
In
n
do
-C
hi
ne
se
Ko
re
an
e
tu
ge
s
Po
r
an
ic
H
isp
Bl
ac
k
C
hi
n
es
e
oa
n
Sa
m
es
e
pa
n
te
W
hi
in
o
Fi
lip
O
th
er
Ja
H
aw
ai
ia
n
0.00
Ethnic Group
No differences are close to being statistically significant. Sample is fairly representative of all HI schools
22
HawaiiFig.
Trial:
13: % of 5th graders reporting substance
use by condition
16.00
14.00
C
PA
12.00
10.00
8.00
6.00
30%
reduction
p = .012 for scale of combine substance use,
accounting for nesting of students within schools
4.00
2.00
0.00
-2.00
E v e r t rie d
t o ba c c o
E v e r t rie d
a lc o ho l
E v e r be e n
drunk
E v e r t rie d
ille ga l drugs
23
Hawaii program effects on absenteeism
Figure 8: Average Daily Absences by Condition
CONTROL
PA
STATE Standard
11.5
11
School Year
10.5
10
9.5
9
8.5
8
2001-02
2002-03
2003-04
2005-05
24
Hawaii dose-response relationship
2.5
Figure 19: Te ache r ratings of
s tude nts "Ge ts Along With
Othe rs " by le ve l of
im ple m e ntation and control
2.45
2.4
2.35
Hi
Med
Low
Control
25
Study IV: Randomized Trial of PA
in Chicago Public Schools
• Elementary Schools (N = 14)
• 3 yr Effectiveness Trial
• Random Assignment
– PA or Control Conditions
• Strata matched on multiple indicators of risk
• PA Schools Received:
– 3 yrs (2003-04 through 2006-07)
– Teacher Training
• Developer: Dr. Carol Allred 3-4 hrs training, initial yr; booster sessions 2 hrs,
subsequent yrs
• Technical Support: Chicago Project Coordinator, Vanessa Brechling
• Data Collection
– Local (UIC)
– Multi-site (US Dept of Education)
26
Comparability of Matched Sets of Schools
Chicago Study (No significant differences)
100
ES (Program)
90
LS (Control)
80
60
50
40
30
20
10
es
C
rim
s
he
r
ip
ity
Q
ua
l
Pa
r
%
Te
ac
at
ic
l
tP
ee
n
En
ro
y
ilit
M
ob
%
er
ty
Po
v
y
%
an
c
Tr
u
%
At
tt
en
d
e
%
hi
ev
Ac
an
%
As
i
p
H
is
%
ck
Bl
a
%
W
hi
te
s
%
ne
s
ea
di
Pa
ir
0
R
Percent
70
27
Figure B: Effects of 3 years of PA on Behavior
% Improvement
Serious violence
12.5
Alcohol use
21.7
Smoking
28.2
0
5
10
15
20
25
30
% Improvement
28
% Ever Used Sustances and % Reduction by Condition:
Grade 5 Chicago Randomized Trial
60.0%
C
PA
% reduction
52.2%
50.0%
39.9%
Percent
40.0%
36.2%
32.1%
28.2%
30.0%
28.3%
21.7%
19.5%
20.0%
13.2%
10.0%
9.4%
10.7%
5.1%
0.0%
Ever smoked/usd Ever drank alcohol Ever gotten drunk
tobacco
Ever used
tobacco, alcohol
p = .023 for scale of combined substance use,
or drugs
accounting for clustering of students within schools
29
Effects on School-Level reports of
misconducts and suspensions (Chicago)
Average N of misconducts per 100 students by
year and condition (80% reduction at 2006-07)
45.00
40.00
PA
Average N of suspensions per 100 students by
year and condition (77% reduction at 2006-07)
35.00
C
PA
C
30.00
35.00
25.00
30.00
25.00
20.00
20.00
15.00
15.00
10.00
10.00
5.00
5.00
0.00
0.00
2003-04
2004-05
2005-06
Year
2006-07
2003-04
2004-05
2005-06
2006-07
Year
In ANCOVA models predicting year 4 differences from year 1 levels and condition,
differences at year 4 are marginally significant for misconducts (p = .054)
and significant for suspensions (p = .037) using one-tailed tests.
30
IV. Conclusions
• Implementing school-wide character education programs to address
a wide range of outcomes is challenging
– Limited resources of urban school systems
– NCLB
• Evaluation of school-based character/social development programs
is complicated by control schools implementing similar programs
under “business as usual” conditions
• Clear dose-response relationships were evident in quasiexperimental evaluations
• In RCTs program effects on smoking at 5th grade were obtained after
3 or 4 years of programming
• School-level reports of misconducts and suspensions strengthen the
robustness of the findings
• Time trends in outcomes and long-term follow-up studies suggest
increasing effects over time
• School-wide social and character development education can be
effective at:
– decreasing multiple negative behaviors, including smoking
31
Why does Positive Action have such
strong effects in multiple domains?
• Links all behaviors/actions to feelings, thoughts and
values
• Increases awareness of social influences and correction
of normative beliefs
• Emphasizes universal values and principles, and is
appropriate for all ethnic and SES groups
• Is consistent with multiple theories of education, learning
and behavior development and change
• Works on the multiple social ecologies in which youth
live and develop
– School, home, community
32
Future Research
• Investigate potential differential impacts of PA
based on student gender, child risk level, etc.
• Investigate whether schools with different levels
in the quality of implementation yield different
“impacts”
• Examine impact of PA as student cohort
progresses into upper elementary grades
(grades 6-8)
– Critical transitional period within emotional,
behavioral, and academic domains
33
Future Work/Needs –
The bigger picture
• Larger scale trials
– ICCs for attitudes (.03-.1) and behavior (.01-.05) are generally
smaller than for achievement (.15-.2)
– Still need Ns of 20 or more per condition rather than 7-10
• Improved measures of integrity and dosage delivered and
received
– Teacher, student and observer reports
– Contractual reporting systems?
• Longer term follow-ups
– Effects take several years to even start emerging
– Prior work suggests important long-term effects are possible
• Methods of analysis to accommodate differential
implementation
– Propensity scoring, CACE, instrumental variable
34
Acknowledgments
Studies I and II were unfunded, conducted by the author at the University of
Illinois at Chicago (UIC).
Study III was funded by grant #R01-DA13474 from NIH/NIDA to Brian Flay,
initially at UIC then at Oregon State University (OSU).
The findings reported from Study IV are based on research conducted as part
of the Social and Character Development (SACD) research program funded by
the Institute of Education Sciences (IES), U.S. Department of Education
through Grant # R215S020218 to UIC (2003-2005) and OSU (2005-2008). The
SACD Consortium consists of representatives from IES, the Centers for
Disease Control and Prevention (CDC), and the national evaluation contractor,
Mathematica Policy Research, Inc. (MPR), and each grantee site participating
in the evaluation. The content of this presentation does not necessarily reflect
the views or policies of the SACD Consortium members including IES, CDC,
and MPR, nor does mention of trade names, commercial products, or
organizations imply endorsement by the U.S. Government.
Correspondence concerning this presentation should be addressed to Brian R.
Flay, D.Phil., Principle Investigator, Department of Public Health, 254 Waldo
Hall, Oregon State University, Corvallis, OR 97330,
brian.flay@oregonstate.edu.
35
SACD disclaimer statement:
The Social and Character Development (SACD) research program funded by
the Institute of Education Sciences (IES), U.S. Department of Education
includes a national evaluation study conducted by Mathematica Policy
Research (MPR), and complementary research studies conducted by each
grantee. The findings reported here are based on the complementary research
activities carried out by Brian Flay, Oregon State University, and David L.
DuBois, University of Illinois at Chicago, under the SACD program. These
findings may differ from the results reported for the SACD national evaluation
study. The findings presented in this conference presentation are based on a
smaller sample size of children, classrooms, and teachers, utilized a different
set of outcome measures, and sought to answer complementary research
questions. The content of this presentation does not necessarily reflect the
views or policies of the SACD Consortium including IES, CDC, and MPR, nor
does mention of trade names, commercial products, or organizations imply
endorsement by the U.S. Department of Education.
36
Have a Positive Action Day!
37