08-09 Evaluation Report
January 2010
Celebrating Statewide Achievements
About the Initiative
The North Carolina Positive Behavior
Intervention and Support Initiative
is part of the North Carolina State
Improvement Program funded through
IDEA.
Vision :: All schools in North Carolina
will implement Positive Behavior
Intervention and Support as an
effective and proactive process for
improving social competence and
academic achievement for all students.
Mission :: Provide leadership, professional development, resources and
on-going support in order for schools
to successfully implement Positive
Behavior Intervention and Support
Action :: The North Carolina Positive
Behavior Intervention and Support
(PBIS) sites are working to integrate
their Safe Schools Plans, Character
Education efforts and strategies, and
discipline efforts in order to make
schools caring and safe communities
for learning.
What’s Inside...
Beginning with the 2007-08 school year, schools participating in the North
Carolina Positive Behavior Intervention and Support (PBIS) Initiative could
apply for recognition by documenting ongoing administrator participation,
an active PBIS team, and an in-school coach and by providing evidence of
implementation progress. Three levels of implementation were recognized:
PBIS Green Ribbon Schools :: Green Ribbon schools have
completed Module I team training and begun PBIS implementation.
They have attained at least a Level I on the Implementation Inventory
and 80% total on the SET.
PBIS Banner Schools :: Banner schools have completed all of
the requirements for Green Ribbon Schools and have completed
Module 2 training. They have also achieved a Level 2 on the
Implementation Inventory and 90% total SET score.
PBIS Exemplar Schools :: Exemplar schools have completed
requirements for Green Ribbon and Banner schools and have
completed all three team training Modules. They have scored a
Level 3 or higher on the Implementation Inventory and 95% total
on the SET. At least two consecutive years of required behavioral,
attendance, and academic data shows improvement, and they have
documented at least one additional data element that they are
tracking as a team (e.g. EBS Self-Assessment, Staff Retention Data,
Climate Surveys, Referral Information for Special Education, Direct
Behavior Rating, etc.)
Recognition
07-08 Schools
08-09 Schools
PBIS Green Ribbon
45 (59%)
83 (45%)
PBIS Banner
16 (21%)
62 (34%)
PBIS Exemplar
15 (20%)
38 (21%)
Celebrating Statewide Achievements. . 1
Statewide Leadership
and Coordination. . . . . . . . . . . . . . . . . 1
Statewide Leadership and Coordination
Context. . . . . . . . . . . . . . . . . . . . . . . . . 2
Professionals coordinate training, support trainers/coaches/coordinators in
Local Education Agencies (LEAs), and facilitate the evaluation of the statewide
initiative. A leadership team meets regularly to identify and address visibility
and political support; training and coaching; and evaluation assessments.
Tools. . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Fidelity . . . . . . . . . . . . . . . . . . . . . . . . . 3
Impact. . . . . . . . . . . . . . . . . . . . . . . . 4-5
Promoting Sustainability . . . . . . . . . . 6
Partnership. . . . . . . . . . . . . . . . . . . . . . 7
PBIS Trainers . . . . . . . . . . . . . . . . . . . . 8
Contact Information. . . . . . . . . . . . . . . 8
08-09 Evaluation Report
Regional Coordinators (RCs) offer professional development and technical
assistance on programming, policy and behavior support for students with
emotional and behavioral needs and discipline issues to local education
agencies throughout the state. They also conduct a focused review of student
outcomes, programs, and services in participating school systems and
charter schools.
1
CONTEXT
Positive Behavior Intervention and Support in NC
Alleghany
Northampton
Warren
Stanly
Lenoir
Hoke
d
an
otl
Sc
Anson
Moore
nd
mo
ch
Ri
Union
Ninety-three of the 115 school districts in the state have
at least one school participating in the North Carolina
Positive Behavior Intervention and Support Initiative.
Steady growth has
been evident in the
number of schools
that have implemented
PBIS and current
estimates suggest
that about 85% are
still implementing.
Craven
Pamlico
Sampson
Jones
Duplin
Onslow
Robeson
Hyde
Greene
Wayne
Harnett
Cu
mb
erl
an
d
Dare
Bladen
Carteret
Pender
HanNew
ove
r
Gaston
ns
nd
Polk
Cabarrus
rg
nbu
ckle
Me
Rutherford
Lee
M
on
tg
om
er
y
Tyrrell
Beaufort
Pitt
Johnston
Lincoln
n
wa
Cho
Chatham
n
gto
shin
Wa
Martin
Wilson
C
md urrit
uc
en
k
ta
nk
uo
ma
ui
Wake
Edgecombe
Ca
sq
rq
Randolph
Bertie
Nash
Pa
Pe
e
Halifax
Franklin
m
rha
Du
Rowan
Da
vid
so
n
Gates
Hertford
ce
Person
nge
Ora
Guilford
Davie
Iredell
Catawba
ela
Transylvania
Burke
ev
Ja
ck
son
r
de
an
ex
Al
Cl
Macon
He
nde
rso
n
McDowell
Forsyth
Caswell
n
Va
Wilkes
Caldwell
ey
nc
Ya
Clay
Buncombe
Yadkin
Rockingham
nce
ma
Ala
ll
he
itc
M
Graham
Cherokee
ood
yw
Ha
Swain
Stokes
ll
nvi
Watauga
Avery
Madison
Surry
G ra
Ashe
Columbus
Brunswick
Schools Implementing PBIS in North Carolina
1000
790
800
691
600
548
400
296
200
0
147
1
5
00-01 01-02
27
9
02-03
03-04
04-05 05-06
06-07
07-08
08-09
Implementation Year
Tools
The North Carolina Positive Behavior Intervention and Support Initiative requests that participating schools
complete and submit implementation and evaluation tools:
• Team Implementation Inventory
• Schoolwide Evaluation Tool (SET) or Benchmarks of Quality (BoQ)
• NC PBIS Office Discipline Referral (ODR) Reporting Spreadsheet
The NC PBIS Initiative recommends that teams utilize other tools such as the EBS Survey, School Safety
Survey, and Team Implementation Checklist, which are online at www.PBISsurveys.org.
In addition, the Initiative uses other data already collected by the Department of Public Instruction, such
as information on achievement, attendance and suspensions. The NC PBIS Initiative provides additional
tools for school use around these data points as well as a Data Collection Manual on the website.
Schools implementing PBIS regularly assess the extent to which key features of PBIS are being
implemented and they use this information to develop action plans for refining and sustaining.
2
January 2010
Fidelity
35
EBS Survey
Schools report higher
percentages of schoolwide
elements being in place.
30
25
2007-08, n = 133
2008-09, n = 153
20
15
10
5
0
0-10
11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 91-100
Percent in Place Schoolwide
Average set scores
Average [08-09] scores on SET
subscales varied across levels
of schools (N=336) and were
generally higher for elementary
(n=226) than middle (n=68),
high (n=24), or combined (n=18)
enrollment schools.
100
90
80
70
60
50
40
30
20
10
0
t
m
ip
g/
ns
ns
or
tem
rin n
ste
tio ed
tio ht
rsh
pp
ys
Sy
ito atio
de
Su
cta efin
cta Taug rd S
n
a
n
t
e
e
u
o
p D
p
ic
Le
tio
M Eval
wa
Ex
Ex
str
ola
Re
Di
Vi
K-6 (67%)
6-9 (20%)
9-12 (7%)
K8-K12 (5%)
levels of implementation across school years
100
90
80
More schools reported SET
scores and they were generally
higher for 2008-2009 than for
previous school years.
70
60
50
40
30
20
10
0
/
t
m
ip
ns
ns
or
tem
ing
ste
tio ed
tio ht
rsh
pp
ys
or ion
Sy
de
Su
cta efin
cta Taug rd S
nit luat
a
n
t
e
e
o
e
o
c
p D
p
i
L
ti
M Eva
wa
Ex
Ex
str
ola
Re
Di
Vi
2006-07 (N=63)
08-09 Evaluation Report
2007-08 (N=200)
2008-09 (N=336)
3
Impact
The North Carolina Positive Behavior Intervention and Support Initiative is focused on improving social
competence and academic achievement for all students. Information from impact evaluation indicators
reflects the extent to which targeted outcomes are being and/or likely to be achieved.
ODR Rates by SET Scores
0.4
ODR/100/DAY
Rates of office
discipline referrals in
schools that met SET
expectations were
lower than in schools
that did not meet them.
Met
0.3
0.2
0.1
0.0
Levels of behavior risk
in schools implementing
PBIS were comparable
to widely-accepted
expectations and
better than in not
implementing PBIS
schools.
Not Met
2004-05
2005-06
2006-07
2007-08
2008-09
Elementary School Office Discipline Referral (ODR)
Risk (Majors and Minors) in North Carolina
100
90
80
70
60
50
40
30
20
10
0
4
2004-05
(N=21)
2005-06
(N=35)
2006-07
(N=66)
2007-08
(N=110)
2008-09 Comparison
(N=143) (N=5)
2004-05
(N=21)
2005-06
(N=35)
2006-07
(N=66)
2007-08
(N=110)
2008-09
(N=143)
Comparison
(N=5)
6+ ODR
5
3
4
4
3
10
2-5 ODR
12
9
11
11
9
23
0-1 ODR
82
88
85
85
88
67
January 2010
IMPACT
S
uspension rates in PBIS Schools
There has been a
consistent decrease in
reported suspensions
across schools
implementing PBIS
over the past five years.
80
Suspension/100 Students
70
60
2004-05
50
2006-07
2005-06
2007-08
2008-09
40
30
20
10
0
K-6 (N=20-143)
6-9 (N=8-56)
9-12 (N=4-18)
School Type
M
ean ODRs per 100 per School Day
1.2
1.05
1.0
0.92
0.84
0.8
1.00
0.85
0.62
0.6
0.4
0.34
Office discipline referral
data (majors) from
schools implementing
PBIS in North Carolina
[07-08] compare
favorably with national
averages.
0.29
0.2
0.0
K-6
6-9
National Average
9-12
K8-K12
NC Average
NC PBIS Initiative critical features
Leadership Team :: Representative stakeholders meet regularly to support effort.
Coordination :: State and local coordinators manage day-to-day operations.
Funding :: Implementation supported with targeted state-approved funds.
Visibility :: Ongoing local, state, and national dissemination efforts.
Political Support :: Leadership Team reports annually on activities and outcomes.
Training Capacity :: Team of trainers is available to build and sustain practices.
Coaching Capacity :: Individuals are available to support emerging practices.
Demonstrations :: Large number of schools available for model demonstrations.
Evaluation :: Process in place for collecting and reporting information on: implementation of
action plan; use of Statewide PBIS (SWPBIS); or impact on student outcomes.
08-09 Evaluation Report
5
Promoting Sustainability
Supporting Data Collection and Usage
In an effort to support PBIS schools in collecting and using data for decision-making, the PBIS Regional
Coordinators (RCs) have been participating in NCWISE training. Additional training has been requested,
so that RCs can provide additional technical assistance around data management and collection issues.
In addition, a revised Data Collection Manual will soon be available online at the NC PBIS website.
Watch for an email announcing that the revision has been posted.
Supporting Training and Coaching
Several advanced module trainings are currently in development. These advanced modules are geared
toward in-school coaches, and are designed to increase their knowledge base and skills so that they can
become the PBIS experts in their schools. School systems with multiple PBIS schools are encouraged
to provide networking opportunities for these in-school coaches through the planning of the LEA PBIS
Leadership Team. If you are an in-school coach looking for networking opportunities, contact your
PBIS Regional Coordinator.
In addition to the Trainer Boot Camp being provided to support PBIS trainers, the NC PBIS State
Leadership Team is developing Trainer and Coach Self-Evaluations. These documents are meant to help
current and future trainers assess their current skills and identify areas where more information and/or
experience is needed, so that regional and state activities can be planned to meet the needs of Trainers
and Coaches.
PBIS Highlights
South Toe Elementary (Yancey County)
2007-2008
2008-2009
74
46
60.3%
73%
2007-2008
2008-2009
1111
684
62.5%
73.3%
2005-2006
2008-2009
Enrollment
685
773
Office Discipline Referrals
1428
800
47.9%
66.1%
40%
12%
Office Discipline Referrals
Percentage of Students Passing BOTH Reading and Math EOG
Marion Intermediate School (Cleveland County)
Office Discipline Referrals
Percentage of Students Passing BOTH Reading and Math EOG
Cedar Creek Middle School (Franklin County)
Percentage of Students Passing BOTH Reading and Math EOG
Staff Turnover
6
January 2010
PARTNERSHIP
N
orth Carolina Responsiveness to Instruction and
Positive Behavior Intervention and Support Participation
Alleghany
Rtl
Moore
Lenoir
y
Cu
mb
Hoke
Hyde
an
Craven
Pamlico
Sampson
erl
Jones
Duplin
d
Onslow
Robeson
Both (PBIS and Rtl)
Greene
Dare
Wayne
Harnett
d
an
otl
Sc
PBIS
Anson
om
er
nd
mo
ch
Ri
Union
tg
Tyrrell
Beaufort
Pitt
Bladen
Carteret
Pender
HanNew
ove
r
nd
Stanly
s
ela
Gaston
Lee
M
on
n
wa
Cho
ev
Polk
Cabarrus
rg
nbu
ckle
Me
Cl
Lincoln
an
im
Wilson
Johnston
Rutherford
n
gto
shin
Wa
Martin
Ca Cu
md rrit
uc
en
k
uo
ta
nk
sq
qu
Wake
Chatham
Edgecombe
Pa
r
Pe
Randolph
e
nc
Va
Rowan
Catawba
Da
vid
so
n
Bertie
Nash
m
y
Transylvania
Iredell
Gates
Hertford
Halifax
rha
ce
Ja
ck
son
Davie
Northampton
Warren
Franklin
Du
n
Ya
ll
Macon
He
nde
rso
n
nce
he
Clay
Buncombe
Guilford
Person
nge
Ora
itc
Graham
Cherokee
ood
yw
Ha
Swain
McDowell
Burke
Forsyth
Caswell
ma
M
Madison
r
de
an
ex
Al
Caldwell
Yadkin
Rockingham
Ala
Wilkes
Stokes
le
nvil
Watauga
Avery
Surry
Gra
Ashe
Columbus
Brunswick
As schools seek effective strategies for school improvement, they often attempt multiple implementations.
In North Carolina efforts have focused not just on Positive Behavior Intervention and Support or
Responsiveness to Instruction (RtI) implementation, but on helping schools and districts understand how
PBIS and RtI fit together. The approach being utilized in North Carolina was the subject of a session at the
6th International Conference on Positive Behavior Intervention and Support in March 2009.
Sc
hool improvement: creating a continuum of support
Academic
Intensive, Individual Interventions
Tutoring
Academic Remediation Plans
behavior
Intensive, Individual Interventions
Functional Behavior Assessment and Behavior
Intervention Planning
Specially Designed Instruction
Targeted Group Interventions
Targeted Group Interventions
Small group instruction
Focused academic help sessions
Social Skills instruction
Reinforcement of specific skills
Group Behavioral Strategies
Classroom Coaching
Universal Interventions
Effective instructional practices
Recognition of academic achievement
Culturally responsive practices
Data-based decision-making
Parent and Community Partnerships
Universal Interventions
School-wide rules and procedures
Systematic reinforcement
Social Skills Instruction
Culturally responsive practices
Data-based decision-making
Parent and Community Partnership
08-09 Evaluation Report
7
PBIS Trainers
NC PBIS Trainer Boot Camp
During the summer of 2008, the first PBIS Trainer Boot Camp was held. The intensive two-day
experiential training was designed to develop the skills of individuals interested in being PBIS trainers.
The training included PBIS updates and skills specific to training and public speaking.
What set the training apart from most professional development experiences was the opportunity, and
expectation, that participants would practice these new skills immediately. Trainees were split into small
groups, “practiced” presenting for their peers, and then had an opportunity to reflect on the process
with their group members and PBIS staff. This training experience has been the most highly rated of
any PBIS training to date.
The process and outcomes of this training were featured in a breakout session at the 6th International
Conference on Positive Behavior Intervention and Support in March 2009 by state PBIS Consultant
Heather Reynolds. Trainer Boot Camp opportunities continue to be offered.
Boot Camp Evaluation
Percent Positive Ratings
100
80
60
40
20
0
tes
ed
s
me
Ac
v
hie
o
utc
O
eA
tiv
fec
Ef
Positive
Public Schools of North carolina
state board of education | department of Public Instruction
exceptional Children division
The North Carolina Positive Behavior Intervention and
Support Initiative is part of the North Carolina State
Improvement Program funded through IDEA.
For more information, contact: Heather Reynolds
Behavioral Support and Special Programs
Exceptional Children Division
NC Department of Public Instruction
6356 Mail Service Center, Raleigh, NC 27699-6356
910-807-4059 (VOICE) • 910-807-3243 (FAX)
8
vi
cti
ing
le
ab ce
u
l
Va ien
rd per
a
w Ex
Re
Neutral
k
or
w
t
e
N
Negative
STAFF
Exceptional Children Division
Mary Watson, Director
Laura Snyder, Assistant Director
Behavior Support and Special Programs
Joe Simmons, Section Chief
Pat Gillott, Program Assistant
State PBIS Consultant
Heather Reynolds
PBIS Technical Assistance Partner
Bob Algozzine, UNC-Charlotte
Regional PBIS Coordinators
Minnie Kidd, Region 1
Margaret Gannon, Region 2
Correy Watkins, Region 3
Richard Tedescucci, Region 4
Cayce McCamish, Region 5
Dana Rusher, Region 6
Dreama McCoy, Region 7
Jullie Weatherman, Region 8
January 2010