The Effects of School-Wide Positive
Behavior Support on School Climate:
A Middle School Longitudinal Study
K. Richard Young, BYU
Ryan H. Shatzer, BYU
Richard P. West, USU
Ellie L. Young, BYU
Paul Caldarella, BYU
Funding
U.S. Department of Education
Field Initiated Research Projects
OSEP Grant # H324CO30124B
Written by Brigham Young University in
collaboration with participating school district
2003 – 2008
Contents
•
•
•
•
Introduction
Methods
Results and Discussion
Questions
Questions
• What strategies and resources are needed
to implement a successful SWPBS
program in middle schools?
• How can you measure the effects of
SWPBS in middle schools?
INTRODUCTION
Positive Behavior Support (PBS)
Build
Positive
Relationships
Data-Based
Decision
Making
PBS
Reinforce
Appropriate
Behavior
~5%
Establish
Clear
Expectations
~15%
Tertiary Prevention:
Students with
High-Risk Behavior
Secondary Prevention:
Students with
At-Risk Behavior
Primary
Prevention:
School-Wide
Teach
Appropriate
Skills
~80% of Students
School Climate
• “A set of internal characteristics that
distinguishes one school from another
and influences the behavior of its
members”
(Hoy and Hannum, 1997)
• Encompasses the dimensions of
(1) Values
(2) Educational practices
(3) Personal interactions
(National Research Council, 2004)
School Climate
Our definition includes
perceptions of ...
Physical Aspects of the School
Programs and Resources
Instructional Management
Relationships between Staff,
Students, and the Community
Parent and Teacher Support
Outcomes of Healthy School Climate
• Improved Student Achievement
–
–
–
–
Standardized test scores
Reading levels
GPA
Academic and school adjustment
(Brand et al., 2008; Esposito, 1999; Hoy & Hannum, 1997)
• Decreased Student Misbehavior
–
–
–
–
–
Student drug use
Aggression and school violence
Anti-social behavior
Absences and suspensions
Student delinquency
(Astor et al., 2002; Aveyard et al., 2004; Gottfredson et al., 2005; Sprott, 2004)
METHODS
Build
Positive
Relationships
Data-Based
Decision
Making
PBS
Reinforce
Appropriate
Behavior
Establish
Clear
Expectations
Teach
Appropriate
Skills
•
PBS Framework
Tertiary-Level Intervention
– One-on-one interventions
•
Secondary-Level Intervention (Achievement Plus Class)
–
–
–
–
–
–
•
Social Skills Instruction: Prevention Plus and peer praise notes
Organizational Skills: Planners and goal setting
Learning Strategies: Study skills and accessing supports
Emotional Resiliency: “Strong Kids” and journaling
Behavioral Self-Management: Conflict resolution techniques
Experiential Learning: Ropes course and service-learning
School-Wide Intervention (SWPBS)
–
–
–
–
–
–
–
–
Faculty training by PBS support staff
Proactive screening for students at-risk for ED
School-wide assemblies, activities, and celebrations
Posting school rules and common classroom rules
School-wide instruction of social skills
Teacher praise notes to students
Administrative Intervention
Data-based decision making: Indicators of School Quality (ISQ)
This PBS framework was implemented over 4 years at the treatment middle school (2004-2007)
Sample
Treatment School
n
%
Control School
n
Teachers
50
56
Students
1063
1331
%
Male
548
51.6%
693
52.1%
Female
515
48.4%
638
47.9%
Caucasian
934
87.9%
1210
90.9%
Other
129
12.1%
121
9.1%
Special Education
141
13.3%
148
11.1%
Reduced Lunch
402
37.8%
366
27.5%
Gender
Ethnicity
* The numbers reported are school averages over the four years of the intervention
Measures
• School Records
– Grade Point Average (GPA)
– Absences
– Office Discipline Referrals (ODRs)
• Indicators of School Quality (ISQ)
– Teacher ratings of school climate (7 categories)
– Student ratings of behavior support (5 categories)
• Supplemental Questionnaire
– Teacher ratings of school climate (3 factors)
Indicators of School Quality (ISQ)
•
•
•
•
•
•
•
Conducted over 9 Years
1,300 Schools
33 Utah School Districts
14 States + 2 International
70,000 Teachers
500,000 Families
700,000 Students
Indicators of School Quality (ISQ)
The Surveys …
• are short
• include all parents,
students, faculty, and
other school staff
• employ age and culturally
appropriate language
• evaluates several
environment domains
The Reports …
• are easy to read
• provide data to compare
each school to a normative
group of schools
• provide a progress
summary
• summarize the statistical
and practical adequacy of
the response to the
survey
Indicators of School Quality
Status Report
Parent Support
Parent Support
Parents volunteer often at the school
Parents know what happens at school
Enough parents participate at parent/teacher conferences
Parents support extracurricular activities
Teacher Excellence
Teachers are knowledgeable about the subjects they teach
Teachers care about students as individuals
Teachers maintain discipline in their classrooms
Teachers are well organized
Teachers enjoy teaching
Student Commitment
Students are well behaved
Enough students participate in extracurricular activities
Students are excited about learning
Students have pride in their school
Administration
Administration is accessible to parents, students, and staff
Administration is supportive of teachers
Administration is well organized
Administration applies appropriate discipline at the school
Administration has high expectations for all students
Instructional Quality
This school prepares students for future employment
This school provides a quality education
Instruction at this school is innovative
Instruction at this school challenges students
Resource Accessibility
Staff has access to enough ongoing training
Counselors are accessible to students
Students have adequate computer access
The school has enough quality textbooks
Students have enough extracurricular opportunities
Safety
Students and staff feel safe at school
Students feel safe going to and from school
The school is clean and in good repair
The school grounds and hallways are well supervised
Parent
Teacher Student
Staff
ISQ Predicts Academic Achievement
3rd Grade
Parent
Parent Support
Teacher
Excellence
Student
Commitment
School
Leadership
Instructional
Quality
Resource
Management
School Safety
Teacher
Student
5th Grade
Parent
Teacher
Student
8th Grade
Parent
Teacher
Student
11th Grade
Parent
Teacher
Student
Elements of Behavior Support
Elements
Items
Positive Relationships
Is there an adult at school whom you trust to ask for help?
Clear Expectations
Are you frequently confused about what is expected of you at school?
Building Skills
Social
Do you make friends easily?
Academic
Do you do well on tests and assignments?
Rewards & Recognition
Are you recognized enough for your efforts to succeed?
Build
Positive
Relationships
Data-Based
Decision
Making
PBS
Reinforce
Appropriate
Behavior
Establish
Clear
Expectations
Teach
Appropriate
Skills
RESULTS & DISCUSSION
Data Analysis
• Analysis of Variance (ANOVA)
– Linear Trend
• Were there significant improvements over the
four years in the treatment middle school?
– Interaction Effects
• Were there significant improvements in the
treatment school, relative to the control school?
School Record Data
2004-2007
School Records Summary
• Academic Achievement
– GPA
• Absences
– Tardies*
– Unexcused Absences*
• ODRs
– Disorderly Conduct ODRs*
*Significant interactions
GPA
3.40
3.35
3.30
3.25
3.20
Treatment
3.15
Control
3.10
3.05
3.00
2004
2005
2006
PBS School:
Significant increase in GPA, but no interaction effect
Linear trend: F (1, 4457) = 11.81, p < .001, d = .14
2007
Tardies
9.0
8.0
7.0
6.0
5.0
Treatment
4.0
Control
3.0
2.0
1.0
0.0
2004
2005
2006
2007
PBS School:
Significant interaction: F (3, 10758) = 77.51, p < .001
Decrease in tardies: Linear trend: F (1, 4822) = 70.08, p < .001, d = .32
Periods per student per year
Unexcused Absences
18
16
14
12
10
Treatment
Control
8
6
4
2
2004
2005
2006
2007
PBS School:
Significant interaction: F (3, 10758) = 12.04, p < .001
Decrease in absences: Linear trend: F (1, 4822) = 5.85, p < .05, d = .11
Periods per student per year
Disorderly Conduct ODRs
1.4
1.2
1.0
0.8
Treatment
0.6
Control
0.4
0.2
0.0
2004
2005
2006
2007
PBS School:
Significant interaction: F (3, 10758) = 14.01, p < .001
Decrease in ODRs: Linear trend: F (1, 4822) = 11.27, p < .001, d = .14
Per student per year
Practical Significance
• It was estimated that during the last year
of PBS the treatment school saved:
– 167 student and administrative hours due to
the reduced number of ODRs
– 643 days in the classroom due to the
reduced number of absences
– 213 hours of class time due to reduced
tardiness
*Based on 15 minutes spent per ODR and 5 minutes late per tardy
Indicators of School Quality (ISQ)
Teacher Version: 2004-2007
ISQ Results Summary
• Teacher Version
– Teacher Excellence*
– Instructional Quality*
– School Leadership*
– Student Commitment
– Parent Support
– Resource Management *
– School Safety
*Significant interactions
Teacher Excellence
4.7
4.6
4.5
Treatment
4.4
Control
4.3
4.2
2004
2005
2006
PBS School:
Significant Interaction: F (3, 306) = 2.75, p < .05
Significant Linear Trend: F (1, 145) = 4.71, p < .05, d = .48
2007
Instructional Quality
4.4
4.2
4.0
Treatment
Control
3.8
3.6
2004
2005
2006
PBS School:
Significant Interaction: F (3, 306) = 3.13, p < .05
Significant Linear Trend: F (1, 145) = 8.81, p < .01, d = .64
2007
School Leadership
4.8
4.6
4.4
Treatment
4.2
Control
4.0
3.8
2004
2005
2006
PBS School:
Significant Interaction: F (3, 306) = 6.17, p < .001
Significant Linear Trend: F (1, 145) = 19.69, p < .001, d = 1.15
2007
Student Commitment
4.2
4.0
3.8
3.6
Treatment
Control
3.4
3.2
3.0
2004
2005
2006
PBS School:
Non-significant Interaction: F (3, 307) = 1.29, p = .28
Significant Linear Trend: F (1, 145) = 11.56, p < .001, d = .74
2007
Parent Support
4.0
3.8
3.6
Treatment
3.4
Control
3.2
3.0
2004
2005
2006
PBS School:
Non-significant Interaction: F (3, 307) = 0.37, p =.77
Significant Linear Trend: F (1, 145) = 4.07, p < .05, d = .50
2007
Resource Management
4.2
4.0
3.8
Treatment
3.6
Control
3.4
3.2
2004
2005
2006
PBS School:
Significant Interaction: F (3, 306) = 3.09, p < 0.05
Significant Linear Trend: F (1, 144) = 11.67, p < .001, d = .77
2007
School Safety
4.8
4.6
4.4
4.2
Treatment
Control
4.0
3.8
3.6
2004
2005
2006
PBS School:
Non-significant Interaction: F (3, 306) = 2.11, p = .10
Non-significant Linear Trend: F (1, 144) = 0.44, p =.51
2007
Behavior Support Items
Student Survey: 2005-2007
Positive Relationships
100%
95%
90%
85%
80%
Treatment
75%
Control
70%
65%
60%
2005
2006
2007
Positive Relationships: Is there an adult at school whom you trust to ask for help?
Percentages are those students that marked “Yes”
Clear Expectations
100%
95%
90%
85%
80%
Treatment
75%
Control
70%
65%
60%
2005
2006
2007
Clear Expectations: Are you frequently confused about what is expected of you at school?
Percentages are those students that marked “No”
Social Skills
80%
75%
70%
65%
60%
Treatment
55%
Control
50%
45%
40%
2006
Building Social Skills: Do you make friends easily?
Percentages are those students that marked “Yes”
2007
Academic Skills
100%
95%
90%
85%
80%
Treatment
75%
Control
70%
65%
60%
2005
2006
2007
Building Academic Skills: Do you do well on tests and assignments?
Percentages are those students that marked “Yes”
Rewards and Recognition
80%
75%
70%
65%
60%
Treatment
55%
Control
50%
45%
40%
2005
2006
2007
Rewards and Recognition: Are you recognized enough for your efforts to succeed?
Percentages are those students that marked “Yes”
Supplemental Questionnaire
Teacher Survey: 2004-2007
Factor Analysis
Item
Factor 1
Students show respect for each other
.82
Students use appropriate social skills
.81
Students share responsibility for making school safe
.75
Students are motivated to use appropriate social skills
.66
Students learn important social skills
.64
Everyone at school treats others with positive regard
.62
Behavior problems dealt with appropriately
.57
Factor 2
Factor 3
Student Pro-social
Behavior
School develops links to community
.83
School involves families
.79
Adults invite discussions of safety
School
.51 Communication/
.51 Collaboration
School uses positive means for student cooperation
.51
Students have opportunities to express feelings
Student with special needs receive special help
.80
Educational
.76
Assistance
School identifies needs of high-risk students
School strives for academic success
Only factor loadings over .50 are shown
All eigenvalues > 1; Explains 59.38% of the variance
.65
α=
.90
.82
.76
Supplemental Questionnaire Summary
• Teacher Survey
– Student Pro-social Behavior*
– School Communication/Collaboration*
– Educational Assistance*
*Significant interactions
Student Pro-Social Behavior
4.2
4.0
3.8
Treatment
3.6
Control
3.4
3.2
2004
2005
2006
PBS School:
Significant Interaction: F (3, 399) = 13.84, p < .001
Significant Linear Trend: F (1, 185) = 38.63, p < .001, d = 1.20
2007
School Communication/Collaboration
4.4
4.2
4.0
Treatment
3.8
Control
3.6
3.4
2004
2005
2006
PBS School:
Significant Interaction: F (3, 400) = 5.55, p < .001
Significant Linear Trend: F (1, 185) = 15.14, p < .001, d = .78
2007
Educational Assistance
4.8
4.6
4.4
Treatment
4.2
Control
4.0
3.8
2004
2005
2006
PBS School:
Significant Interaction: F (3, 398) = 5.20, p < .05
Significant Linear Trend: F (1, 184) = 10.64, p < .001, d = .65
2007
Overview
• School-wide PBS showed moderate
improvements in school climate
– School records (GPA, attendance, ODRs)
– Indicators of School Quality (ISQ)
– Supplemental questionnaire
• Gradual, continuous improvement
• Limitations
– Measures of treatment fidelity
QUESTIONS
Questions
• What strategies and resources are needed
to implement a successful SWPBS
program in middle schools?
• How can you measure the effects of
SWPBS in middle schools?
References
Astor, R. A., Benbenishty, R., Zeira, A., & Vinokur, A. (2002). School climate, observed risky behaviors, and victimization
as predictors of high school students' fear and judgments of school violence as a problem. Health Education &
Behavior, 29(6), 716-736.
Aveyard, P., Markham, W. A., Lancashire, E., Bullock, A., Macarthur, C., Cheng, K. K., et al. (2004). The influence of
school culture on smoking among pupils. Social Science & Medicine, 58(9), 1767-1780.
Brand, S., Felner, R. D., Seitsinger, A., Burns, A., & Bolton, N. (2008). A large scale study of the assessment of the social
environment of middle and secondary schools. Journal of School Psychology, 46(5), 507-535.
Esposito, C. (1999). Learning in urban blight: School climate and its effect on the school performance of urban, minority,
low-income children. School Psychology Review, 28(3), 365-377.
Gottfredson, G. D., Gottfredson, D. C., Payne, A. A., & Gottfredson, N. C. (2005). School climate predictors of school
disorder. Journal of Research in Crime and Delinquency, 42(4), 412-444.
Horner, R. H., & Sugai, G. (2004). School-wide Positive Behavior Support: Building Sustainable Support. www.pbis.org.
Hoy, W. K., & Hannum, J. W. (1997). Middle school climate: An empirical assessment of organizational health and student
achievement. Educational Administration Quarterly, 33(3), 290-311.
Merrill, K (2003). Strong Teens: A curriculum for promoting social and emotional resiliency in students from grades 9-12.
The Oregon Resiliency Project. University of Oregon.
National Research Council (2004). Engaging Schools: Fostering High School Students' Motivation to Learn. Division of
Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
Sprott, J. B. (2004). The development of early delinquency: Can classroom and school climates make a difference?
Canadian Journal of Criminology and Criminal Justice, 46(5), 553-572.
Taylor, M., West, R., & Smith, T. (2006). The Indicators of School Quality (ISQ) Survey Manual. Logan, UT: Center for the
School of the Future.
Young, K., Marchant, M., & Wilder, L. K. (2003). School-based interventions for students with emotional/behavioral
disorders. In P. Allen-Mears and M. W. Fraser (Eds.) Interventions with children and adolescents: An
interdisciplinary perspective. (pp. 175-204). Boston: Allyn & Bacon.
Young, K., & Shatzer, R. (2008, July). School-wide Positive Behavior Support: Effects on Middle School Teachers’
Perception of School Quality. Poster presented at the Office of Special Education Programs (OSEP) Conference,
Washington D.C.
http://education.byu.edu/pbsi
richard_young@byu.edu
ryan_shatzer@byu.edu
rich.west@usu.edu
ellie_young@byu.edu
paul_caldarella@byu.edu
301 MCKB
Brigham Young University
Provo, UT 84602
(801) 422-3694
byu.pbsi@gmail.com