10-11 Evaluation Report
Winter 2012
Celebrating Statewide Achievements
About the Initiative
The North Carolina Positive Behavior
Intervention and Support Initiative is part
of the North Carolina State Improvement
Program funded through IDEA.
Vision :: All schools in North Carolina
will implement Positive Behavior
Intervention and Support as an
effective and proactive process for
improving social competence and
academic achievement for all students.
Mission :: Provide leadership, professional development, resources, and
on-going support in order for schools
to successfully implement Positive
Behavior Intervention and Support.
Action :: The North Carolina Positive
Behavior Intervention and Support
(PBIS) sites are working to integrate
their Safe Schools Plans, Character
Education efforts and strategies, and
discipline efforts in order to make
schools caring and safe communities
for learning.
Beginning with the 2007-08 school year, schools participating in the North
Carolina Positive Behavior Intervention and Support (PBIS) Initiative could
apply for recognition by documenting ongoing administrator participation,
an active PBIS team, and an in-school coach and by providing evidence of
implementation progress. Three levels of implementation are recognized:
PBIS Green Ribbon Schools :: Green Ribbon schools have
completed Module I team training and begun PBIS implementation.
They have attained at least a Level I on the Implementation Inventory
and 80% total on the Schoolwide Evaluation Tool (SET).
PBIS Model Schools :: Model schools have completed all of the
requirements for Green Ribbon Schools and have completed
Module 2 training. They have also achieved a Level 2 on the
Implementation Inventory and 90% total SET score.
PBIS Exemplar Schools :: Exemplar schools have completed
requirements for Green Ribbon and Model schools and have
completed all three team training Modules. They have scored a
Level 3 or higher on the Implementation Inventory and 95% total
on the SET. At least two consecutive years of required behavioral,
attendance, and academic data shows improvement, and they have
documented at least one additional data element that they are
tracking as a team (e.g. EBS Self-Assessment, Staff Retention Data,
Climate Surveys, Referral Information for Special Education, Direct
Behavior Rating, etc.).
PBIS Schools earning recognition
What’s Inside...
350
Celebrating Statewide Achievements. . 1
300
Context. . . . . . . . . . . . . . . . . . . . . . . . . 2
250
Fidelity . . . . . . . . . . . . . . . . . . . . . . . 3-4
200
Impact. . . . . . . . . . . . . . . . . . . . . . . . 5-6
150
School Spotlight . . . . . . . . . . . . . . . . . 7
100
Supporting Sustainability. . . . . . . . . . 8
50
PBIS High Schools. . . . . . . . . . . . . . . . 8
10-11 PBIS
0
2007-08
2008-09
2009-10
2010-11
Exemplar
14
38
56
68
Model
8
62
96
131
Green Ribbon
46
83
85
123
1
CONTEXT
Positive Behavior Intervention & Support in North Carolina
Alleghany
Northampton
Warren
Wilson
Lee
Lenoir
Hoke
Cu
mb
erl
an
d
Steady growth has
been evident in the
number of schools that
have implemented PBIS
and 40% of the schools
in NC are participating
in the Initiative.
Hyde
Craven
Pamlico
Sampson
Jones
Duplin
Onslow
Robeson
One hundred of the 115 school districts
in the state have at least one school
participating in the North Carolina Positive
Behavior Intervention and Support Initiative.
Greene
Dare
Wayne
Harnett
Moore
d
an
otl
Sc
Anson
nd
mo
ch
Ri
Union
Stanly
M
on
tg
om
er
y
Tyrrell
Bladen
Carteret
Pender
HanNew
ove
r
nd
Gaston
Cabarrus
urg
enb
ckl
Me
ela
Polk
Lincoln
Beaufort
Pitt
Johnston
Rutherford
n
gto
shin
Wa
ns
Wake
Chatham
n
wa
Cho
Randolph
Martin
C
md urrit
uc
en
k
ta
nk
uo
ma
ui
Rowan
Da
vid
so
n
Edgecombe
Ca
sq
rq
Iredell
Bertie
Nash
Pa
Pe
lle
Halifax
Franklin
m
rha
Du
Catawba
Guilford
Davie
Gates
Hertford
ce
Person
nge
Ora
r
de
an
ex
Al
ev
Transylvania
McDowell
Burke
Cl
Ja
ck
son
He
nde
rso
n
Forsyth
Caswell
n
Va
Wilkes
Caldwell
ey
nc
Ya
Clay
Macon
Buncombe
Yadkin
Rockingham
nce
ma
Ala
ll
he
itc
M
Graham
Cherokee
ood
yw
Ha
Swain
Stokes
nvi
Watauga
Avery
Madison
Surry
G ra
Ashe
Columbus
Brunswick
PBIS = Green
Schools Participating in PBIS Initiative
Implementation Year
NC PBIS Initiative critical features
Leadership Team :: Representative stakeholders meet regularly to support effort.
Coordination :: State and local coordinators manage day-to-day operations.
Funding :: Implementation is supported with targeted state-approved funds.
Visibility :: Ongoing local, state, and national dissemination efforts are evident.
Political Support :: Leadership Team reports annually on activities and outcomes.
Training Capacity :: Team of trainers is available to build and sustain practices.
Coaching Capacity :: Individuals are available to support emerging practices.
Demonstrations :: Large number of schools available for model demonstrations.
Evaluation :: Process is in place for collecting and reporting information on: implementation of
action plan, use of Statewide PBIS (SWPBIS), or impact on student outcomes.
2
Winter 2012
Fidelity
The North Carolina Positive Behavior Intervention and Support Initiative requests that participating schools
complete and submit implementation and evaluation tools:
• Implementation Inventory
• Schoolwide Evaluation Tool (SET) or Benchmarks of Quality (BoQ)
• NC PBIS Office Discipline Referral (ODR) Reporting Spreadsheet
The NC PBIS Initiative recommends that teams use other tools such as the EBS Survey, School Safety
Survey, and Team Implementation Checklist, which are online at www.pbisassessment.org.
In addition, the Initiative uses other data already collected by the Department of Public Instruction, such
as information on achievement, attendance, and suspensions. The NC PBIS Initiative provides additional
tools for school use around these data points as well as a Data Collection Manual on the website.
PBIS Schools regularly assess the extent to which key features of PBIS are being implemented and they
use this information to develop action plans for refining and sustaining the process.
10-11 Average SET SCOres N=453
100
90
80
70
60
Average SET subscale scores
varied across levels of schools
(N=453) and were generally
higher for elementary (n=318)
than middle (n=107) or high
(n=28) schools.
50
40
30
20
10
0
SET scores across school years
100
90
80
70
60
50
While the number of schools
completing the SET has more
than doubled over the past
four years, average scores
have remained above the
recommended standard of
80% across all subscales.
40
30
20
10
0
Levels of Implementation Across School Years
10-11 PBIS
3
Fidelity
The Implementation Inventory yields fidelity of implementation data across the full PBIS continuum of
behavior support.
Level 0 < 80% on all subscales
Level 1 ≥ 80% on Universal subscale
Level 2 ≥ 80% on Universal and Secondary subscales
Level 3 ≥ 80% on Universal, Secondary, and Tertiary subscales
Level 4 ≥ 80% on Universal, Secondary, Tertiary, Practices, Systems and Data subscales
While more schools continue
to submit Implementation
Inventory Online (IIO) scores,
fewer seem to be scoring at
level 2. Secondary level item
analysis reveals lower scores
in training and support for
the PBIS team and staff
implementing secondary
interventions, as well as
need to improve secondary
data collection practices.
Number of Schools at each IIO Level
Data also shows shifts in
the proportion of schools
scoring at each level. Over
the last two years, over
Proportion of schools at each IIO Level
50% of schools scored at or
above Level 2 compared to
less than 25% scoring at or
above Level 2 in 2007-08.
150
2007-08 (N = 161)
2008-09 (N = 300)
2009-10 (N = 386)
2010-11 (N = 490)
120
90
60
30
0
100
80
60
40
20
0
Level 4
Over time, state averages
have increased across all
six subscales of the IIO.
The largest increases have
occurred in the Secondary
& Data subscale averages.
Level 3
Level 2
Level 1
Level 0
Average IIO Scores
100
80
60
40
20
0
4
Winter 2012
Impact
The North Carolina Positive Behavior Intervention and Support Initiative is focused on improving social
competence and academic achievement for all students. Information from impact evaluation indicators
reflects the extent to which targeted outcomes are being and/or likely to be achieved.
10-11 ODR Rates per 100 students per school day
ODR/100 Students/Day
1.2
1.0
0.8
0.6
0.4
0.2
0.0
For the 10-11 school year, ODR data was
gathered through ODR spreadsheets.
While the NC High School ODR rates
appear to be above the national average
for PBIS schools, this is due to the data of
one school that reported ODR rates more
than 2 times higher than the next highest
reported HS rate, 7 times higher than
the national average, and almost 9 times
higher than the state average for all other
PBIS high schools. When this outlier is
removed, average reported ODR rates
are below the national average.
Out-of-school Suspension Rates in PBIS Schools Over Time
40
ODR/100 Students
35
Overall suspension rates
continue to decrease in
PBIS schools.
30
25
20
15
10
5
0
Type of School
Out-Of-School Suspensions in PBIS Schools
35
ODR/100 Students
30
25
Rates of suspension in
PBIS schools continue
to decrease, and are
lower in Exemplar
schools.
20
15
10
5
0
10-11 PBIS
5
IMPACT
More PBIS schools with
suspension rates below
the state average also
had above average
academic performance
composites (greater
than 74.75), and met
expected growth,
high growth, and AYP
targets.
PBIS School Performance and OSS Rates (State Average OSS
rate/100 students = 15.07 & Performance Composite = 74.75)
100
80
60
40
20
0
Average Performance Composites
78
76
74
Exemplar PBIS
schools had higher
average performance
composites, and more
Exemplar schools
met expected growth
targets.
72
70
68
Percentage of Schools Meeting Expected Growth
84.0
83.5
83.0
82.5
82.0
81.5
81.0
80.5
80.0
6
Winter 2012
school spotlight
Brunson Elementary School • Forsyth County Schools
2008-09, 2009-10, 2010-11 Exemplar School
Percentage of Students Passing Both Reading & Math EOGs
0.20
100
80
0.15
60
0.10
40
20
0.05
0
0.00
cedar creek middle School • franklin County Schools
2008-09, 2009-10, 2010-11 Exemplar School
Percentage of Students Passing Both Reading & Math EOGs
2.00
100
80
1.50
60
1.00
40
20
0.50
0
0.00
holly springs high School • wake County Schools
2010-11 Green Ribbon School
Percentage of Passing Scores on EOC Tests
100
80
40
30
60
40
20
20
10
0
0
10-11 PBIS
OSS/100 std
7
Supporting Sustainability
North Carolina’s PBIS schools contribute to research and support sustainability in several ways. The
data that schools contribute each year helps the state team support ongoing implementation. From
time to time, coaches and schools are asked to respond to surveys to collect additional data. This data
is used to adapt professional development and technical assistance to meet the changing needs of
schools and districts.
In addition, data collected from schools, districts, and coaches is routinely shared with the national
PBIS community at conferences. During the 2010-11 school year, information from a coaching survey
completed by school district coaches and coordinators was used to develop a poster for the 2010 Illinois
PBIS Implementers’ Forum and a presentation for the International Conference for Positive Behavior
Support. One of the findings of the project was that the presence of a district leadership team was
associated with higher school performance on the SET and having more schools earn recognition.
Additional information is available from your regional coordinator.
PBIS High Schools
With a growing number of high schools participating in PBIS, we began to ask questions about changes
in graduation rates at high schools implementing PBIS. Because graduation is the culmination of 4 years
of high school (and the outcome of the foundations laid in elementary and middle school), we compared
the state graduation rate with the
graduation rates of high schools who
Average Graduation Rates
had been implementing PBIS for at
least four years. Overall graduation
85
rates as well as graduation rates for
80
students with disabilities exceed the
state rate in schools implementing
75
PBIS for at least four years, and the
70
highest average rates exist among
high schools who earned model
65
recognition (no high schools earned
60
exemplar recognition in 2010-11).
55
50
Public Schools of North Carolina
state board of education | department of Public Instruction
exceptional Children division
The North Carolina Positive Behavior Intervention and
Support Initiative is part of the North Carolina State
Improvement Program funded through IDEA.
For more information, contact: Heather Reynolds
Behavioral Support and Special Programs
Exceptional Children Division
NC Department of Public Instruction
6356 Mail Service Center, Raleigh, NC 27699-6356
910-807-3313 (VOICE) • 910-807-3243 (FAX)
8
STAFF
Exceptional Children Division
Mary Watson, Director
Laura Snyder, Assistant Director
Behavior Support and Special Programs
Joe Simmons, Section Chief
Dywanda Pettaway, Program Assistant
State PBIS Consultant
Heather Reynolds
PBIS Technical Assistance Partner
Bob Algozzine, UNC-Charlotte
Regional PBIS Coordinators
Minnie Kidd, Region 1
Vacant, Region 2
Correy Watkins, Region 3
Richard Tedescucci, Region 4
Cayce McCamish, Region 5
Laura Winter, Region 6
Vacant, Region 7
Julie Weatherman, Region 8
Winter 2012