09-10 Evaluation Report
January 2011
Celebrating Statewide Achievements
About the Initiative
The North Carolina Positive Behavior
Intervention and Support Initiative is part
of the North Carolina State Improvement
Program funded through IDEA.
Vision :: All schools in North Carolina
will implement Positive Behavior
Intervention and Support as an
effective and proactive process for
improving social competence and
academic achievement for all students.
Mission :: Provide leadership, professional development, resources, and
on-going support in order for schools
to successfully implement Positive
Behavior Intervention and Support
Action :: The North Carolina Positive
Behavior Intervention and Support
(PBIS) sites are working to integrate
their Safe Schools Plans, Character
Education efforts and strategies, and
discipline efforts in order to make
schools caring and safe communities
for learning.
Beginning with the 2007-08 school year, schools participating in the North
Carolina Positive Behavior Intervention and Support (PBIS) Initiative could
apply for recognition by documenting ongoing administrator participation,
an active PBIS team, and an in-school coach and by providing evidence of
implementation progress. Three levels of implementation are recognized:
PBIS Green Ribbon Schools :: Green Ribbon schools have
completed Module I team training and begun PBIS implementation.
They have attained at least a Level I on the Implementation Inventory
and 80% total on the SET.
PBIS Banner Schools :: Banner schools have completed all of
the requirements for Green Ribbon Schools and have completed
Module 2 training. They have also achieved a Level 2 on the
Implementation Inventory and 90% total SET score.
PBIS Exemplar Schools :: Exemplar schools have completed
requirements for Green Ribbon and Banner schools and have
completed all three team training Modules. They have scored a
Level 3 or higher on the Implementation Inventory and 95% total
on the SET. At least two consecutive years of required behavioral,
attendance, and academic data shows improvement, and they have
documented at least one additional data element that they are
tracking as a team (e.g. EBS Self-Assessment, Staff Retention Data,
Climate Surveys, Referral Information for Special Education, Direct
Behavior Rating, etc.)
PBIS Schools earning recognition
What’s Inside...
Celebrating Statewide Achievements. . 1
Context. . . . . . . . . . . . . . . . . . . . . . . . . 2
Tools. . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Fidelity . . . . . . . . . . . . . . . . . . . . . . . . . 4
Impact. . . . . . . . . . . . . . . . . . . . . . . . 5-6
Supporting PBIS research . . . . . . . . . 7
Sustainability and Capacity . . . . . . 7-8
09-10 PBIS 250
200
150
100
50
0
2007-08
2008-09
2009-10
2007-08
2008-09
2009-10
PBIS Exemplar
14
38
56
PBIS Model
8
62
96
PBIS Green Ribbon
46
83
85
1
CONTEXT
Positive Behavior Intervention & Support in North Carolina
Alleghany
Northampton
Warren
nd
Gaston
Stanly
Anson
Lee
Lenoir
Hoke
Cu
mb
erl
an
d
One hundred of the 115 school districts
in the state have at least one school
participating in the North Carolina Positive
Behavior Intervention and Support Initiative.
Hyde
Craven
Pamlico
Sampson
Jones
Duplin
Onslow
Robeson
Steady growth has
been evident in the
number of schools that
have implemented
PBIS. Over the last
two years participation
has increased by about
13% each year.
Greene
Dare
Wayne
Harnett
Moore
d
an
otl
Sc
nd
mo
ch
Ri
Union
M
on
tg
om
er
y
Tyrrell
Bladen
Carteret
Pender
HanNew
ove
r
Cabarrus
urg
enb
ckl
Me
ela
Polk
Lincoln
n
gto
shin
Wa
Beaufort
Pitt
ns
Wilson
Johnston
Rutherford
n
wa
Cho
Wake
Chatham
Martin
C
md urrit
uc
en
k
ta
nk
uo
ma
ui
Catawba
Randolph
Edgecombe
Ca
sq
rq
Rowan
Da
vid
so
n
Bertie
Nash
Pa
Pe
lle
Halifax
Franklin
m
rha
Du
Iredell
Gates
Hertford
ce
Person
nge
Ora
Burke
Guilford
Davie
ev
Transylvania
McDowell
r
de
an
ex
Al
Cl
Ja
ck
son
He
nde
rso
n
Forsyth
Caswell
n
Va
Wilkes
Caldwell
ey
nc
Ya
Clay
Macon
Buncombe
Yadkin
Rockingham
nce
ma
Ala
ll
he
itc
M
Graham
Cherokee
ood
yw
Ha
Swain
Stokes
nvi
Watauga
Avery
Madison
Surry
G ra
Ashe
Columbus
PBIS = Green
Brunswick
09-10 Schools Participating in PBIS Initiative
1000
909
790
800
691
600
548
400
296
200
0
147
1
9
27
02-03
03-04
5
00-01 01-02
04-05 05-06
06-07
07-08
08-09
09-10
Implementation Year
NC PBIS Initiative critical features
Leadership Team :: Representative stakeholders meet regularly to support effort.
Coordination :: State and local coordinators manage day-to-day operations.
Funding :: Implementation is supported with targeted state-approved funds.
Visibility :: Ongoing local, state, and national dissemination efforts are evident.
Political Support :: Leadership Team reports annually on activities and outcomes.
Training Capacity :: Team of trainers is available to build and sustain practices.
Coaching Capacity :: Individuals are available to support emerging practices.
Demonstrations :: Large number of schools available for model demonstrations.
Evaluation :: Process is in place for collecting and reporting information on: implementation of
action plan, use of Statewide PBIS (SWPBIS), or impact on student outcomes.
2
January 2011
Tools
The North Carolina Positive Behavior Intervention and Support Initiative requests that participating schools
complete and submit implementation and evaluation tools:
• Implementation Inventory
• Schoolwide Evaluation Tool (SET) or Benchmarks of Quality (BoQ)
• NC PBIS Office Discipline Referral (ODR) Reporting Spreadsheet
The NC PBIS Initiative recommends that teams use other tools such as the EBS Survey, School Safety
Survey, and Team Implementation Checklist, which are online at www.PBISsurveys.org.
In addition, the Initiative uses other data already collected by the Department of Public Instruction, such
as information on achievement, attendance, and suspensions. The NC PBIS Initiative provides additional
tools for school use around these data points as well as a Data Collection Manual on the website.
PBIS Schools regularly assess the extent to which key features of PBIS are being implemented and they
use this information to develop action plans for refining and sustaining the process.
09-10 Average SET SCOres n=425
Average SET subscale scores
continued to vary across levels
of schools in 09-10. Elementary
and middle school scores
tended to be higher than scores
for combined enrollment and
high schools.
100
90
80
70
60
50
40
30
20
10
0
t
m
g/
ip
ns
ns
or
tem
rin n
ste
tio ed
tio ht
rsh
pp
ys
Sy
ito atio
de
Su
cta efin
cta Taug rd S
n
a
n
t
e
e
u
o
p D
p
ic
Le
tio
M Eval
wa
Ex
Ex
str
ola
Re
Di
Vi
K-6 (68%)
6-9 (18%)
9-12 (7%)
K8-K12 (7%)
SET scores across school years
In the 09-10 school year,
both the number of schools
reporting SET scores and the
scores reported were higher.
In recent years, average
SET scores for the state
were consistently above the
recommended standard of 80%.
100
90
80
70
60
50
40
30
20
10
0
/
s
s
rt
ip
tem
tem
ing n
po
ion d
ion t
sh
ys
ys
er
up
tor atio
tat fine
tat augh
S
S
i
d
S
c
c
n
a
n
o u
rd
pe De
pe T
ict
Le
tio
M Eval
wa
Ex
Ex
str
ola
Re
Di
Vi
2006-07
2007-08
2008-09
2009-10
Levels of Implementation Across School Years
09-10 PBIS
3
Fidelity
The Implementation Inventory yields fidelity of implementation data across the full PBIS continuum of
behavior support. With the introduction of the online version, more schools than ever are participating
in the Implementation Inventory Online (IIO).
Level 0 < 80% on all subscales
Level 1 ≥ 80% on Universal subscale
Level 2 ≥ 80% on Universal and Secondary subscales
Level 3 ≥ 80% on Universal, Secondary, and Tertiary subscales
Level 4 ≥ 80% on Universal, Secondary, Tertiary, Practices, Systems and Data subscales
Current 09-10 data
indicate a shift
towards more schools
implementing with
increased fidelity at the
upper range of
the continuum.
Implementation Inventory Levels
150
2007-08 (N = 161)
2008-09 (N = 300)
2009-10 (N = 386)
120
90
60
30
0
Between 07-08 and
09-10, PBIS schools
have shown growth
in implementation
across all six subscales.
Level 0
Level 1
Level 2
Level 3
Level 4
Implementation Inventory Levels
100
80
60
40
20
0
Universal Secondary
2007-08
A closer look at 09-10
Implementation Inventory subscale averages
reveal the need to develop additional support
for implementation of
secondary and tertiary
levels in schools.
These data are driving
plans for training and
technical assistance.
4
Tertiary
Practices
2008-09
Systems
Data
2009-10
2009-10 Implementation Inventory subscales
Universal
Secondary
Tertiary
Practices
Systems
Data
0
20
40
60
80
100
January 2011
Impact
The North Carolina Positive Behavior Intervention and Support Initiative is focused on improving social
competence and academic achievement for all students. Information from impact evaluation indicators
reflects the extent to which targeted outcomes are being and/or likely to be achieved.
Mean ODRs per 100 students per School Day
National Average
NC Average
K-6
6-9
Office discipline referral
data (majors) from
schools implementing
PBIS in North Carolina
compare favorably
with national averages.
9-12
K8-12
0.0
0.2
0.4
0.6
0.8
1.0
ODR/100 Students/Day
Suspension Rates 2009-10
Suspension rates
average 15% lower
in PBIS implementing
schools than the average
state suspension rate.
Suspension rates at
Exemplar schools were
half the state rate.
State
All PBIS
Exemplar
0
5
10
15
20
25
30
ODR/100 Students
Suspension/Enrollment/100 Students
Suspensions Across Type of School
09-10 PBIS
60
50
40
30
There has been a
consistent decrease in
reported suspensions
across schools
implementing PBIS
over the past five years.
2005-06
2006-07
2007-08
2008-09
2009-10
20
10
0
K-6
6-9
School Type
9-12
5
IMPACT
More PBIS schools
with suspension
rates below the state
average had above
average performance
composites (greater than
74), and met expected
growth, high growth,
and AYP targets.
09-10 PBIS School Performance and OSS Rates
(Average OSS Rate/100 Student/Day = .14)
100
OSS Rate Above Average
80
OSS Rate Below Average
60
40
20
0
Performance
Composite > 74
09-10 Average Performance Composites
80
70
60
50
40
30
20
10
0
State
All PBIS
Exemplar
09-10 Percent Meeting Expected Growth
Met Expected
Growth
Met High
Growth
Met AYP
While only Exemplar
PBIS schools average
performance composite
was above the state
average performance
composite, the
percentage of PBIS
schools meeting
expected growth
was the same as the
percentage of schools
statewide meeting
expected growth.
95% of PBIS exemplar
schools met expected
growth targets.
100
80
60
40
20
0
6
State
All PBIS
Exemplar
January 2011
sustainability and capacity
PBIS Highlights
Chowan Middle School
2008-09
2009-10
Students with 0 or 1 Office Referrals
80.32%
83.87%
Students with 2 or 5 Office Referrals
15.07%
13.85%
Students with 6 or more Office Referrals
4.61%
2.28%
EOG Performance Composite
65.7%
76.6%
Watauga High School
EOC Scores
Graduation Rate
100
100
90
90
80
80
70
70
60
WHS
State
2006-07
2007-08
2008-09
2009-10
60
2006-07
2007-08
2008-09
2009-10
Supporting PBIS research
During the 09-10 school year ,the state PBIS team presented
two posters at national conferences. Cayce McCamish and
Dana Rusher presented “Teaming Up: PBS, RtI, and Pre-referral
Team Development” at the national PBIS Implementers Forum
in the fall. The poster was the result of a survey project about
how schools are developing teams to assist with braiding
PBIS and RtI implementation. As schools in North Carolina
implement multiple problem-solving initiatives, the team
structures most common are RtI-Only teams and PBS-and-RtIOnly teams; the majority of reporting schools will absorb prereferral and PBS team responsibilities within the functioning of
the RtI team. The second most common structure is for schools
to maintain two separate teams with the RtI team absorbing
the pre-referral team and the PBS team continuing to operate
separately but in conjunction with the RtI team.
Supporting Sustainable Scale-Up with A Regional Coordinator Model
HeatHer reynolds, nortH Carolina department of publiC instruCtion, Hreynolds@dpi.state.nC.us
MONDAY
TUESDAY
WEDNESDAY
THURSDAY
FRIDAY
Levels of Implementation Across School Years
State Leadership Team
100
90
Off-SiteTAandAdmin
Prof Dev
Eval
70
On-Site TA
Holiday
80
State PBS Consultant
Training/Presentation/Prep
Meetings
60
Regional Coordinators
50
Other
Figure 1: How Regional Coordinators Spent an Average Month in 08-09
• AttendPBScoordinatormeetingsandtraining
30
Trainer
Team Facilitator
In-School Coach
20
10
regional responsibilities (figure 1):
• 80%ofthetime(about16days/month)workinginregion
andstate(Figure4)
• Hoststateandregionalmeetingsforimplementers
40
LEA Coordinator
External Coach
regional Coordinator Job desCription
School
Administrator
Recorder
Data Manager
Time-Keeper
Communication
Coordinator
2006-07 (N=63)
School Staff
• CoordinateregionalPBStraining
• PlanPBSsummerinstituteandconferencepresentations
• Conductsitevisits,aspossible
• ProvidePBStechnicalassistanceandsupportintheregion
• Coordinatedatacollectionfortheregion
• AssistlocalPBStrainerswithusingandunderstandingdata
• StayinformedaboutnationalPBSresearch
2007-08 (N=200)
2008-09 (N=336)
Figure 5: Average SET Scores
• ProvidePBSawarenesspresentationsintheregion
• WorkwithdistrictPBStrainerstocompletedatarequirements
forimplementingschools
0
rt
ip
tem
tem
ing/
po
ionsed
ionsht
sh
ys
up
Sys
itor tion ader
ctat in
ctat ug d S
tS
on ua
pe Def
pe Ta
ion
ar
Le
M Eval
tric
Ex
olat
Rew
Dis
Vi
Ex
Students
Community
Parents
support for sustainability & sCale-up
Figure 2: NC PBS Initiative Flowchart
nC pbs sCHools
790
800
691
700
600
548
500
RegionalCoordinators(RCs)havebeenanintegralpartofNC’sPBS
efforts.TheRCsprovideacriticallinkbetweenthestateleadership
teamandtheschoolsanddistrictsimplementingPBSacrossthe
state(Figure2).Theyfacilitatecommunicationandensurethat
trainingandtechnicalassistanceareavailableforPBSimplementers.
InthefouryearsthattheRCshavebeeninplace,thenumberof
schoolsimplementingPBShasgrown537%to790schoolstotal
(32%ofallschoolsinNC).About160schoolswereaddedeachyear
(Figure3).Moreschoolsaresubmittingdata,andthedatashow
increasesinthefidelityofimplementationacrossthestate(Figure5).
Inaddition,NCPBSschoolsarereportinglowerratesofoffice
disciplinereferralsthanthenationalaverage(Figure6).
In March, Heather Reynolds presented “Supporting
Sustainable Scale-up with a Regional Coordinator Model”
at the International Conference for Positive Behavior
Support. This poster examined the impact that the regional
coordinators had on the growth and development of the PBIS Initiative. Since the regional positions
were added, the number of schools implementing PBS has grown 537%, more schools are submitting
data, and the data show both increases in the fidelity of implementation and lower rates of office
discipline referrals than the national average for PBIS schools. The poster can be found at:
http://www.ncpublicschools.org/docs/positivebehavior/resources/workshop/supporting-scale-model.pdf
• CoordinatewithPBSStateConsultantandregional
BehaviorSupportConsultant
400
293
300
school district responsibilities:
• CoachparticipatingschoolsinLEA
• HelptrainnewschoolsinLEA
• Directdatamanagementandprogramevaluationefforts
• Chair/AssistSchoolSystemPBSLeadershipTeam
200
147
0
1
00-01
5
01-02
• Documentthatthedatarequirementsarecompleted
forimplementingschools
• Manageschoolsystemactionplan
9
02-03
27
03-04
DISTRICT 7
05-06
DISTRICT 5
06-07
07-08
08-09
1.0
0.92
0.84
1.00
0.85
0.8
DISTRICT 3
DISTRICT 1
0.62
0.6
0.4
DISTRICT 8
National Average
NC Average
0.34
0.29
0.2
DISTRICT 6
Figure 4: NC PBS Regions
09-10 PBIS
1.05
04-05
Figure 3: PBS Implementing Schools
• WorkwithPBStrainersandchairpersonsinLEA
• Linkbetweenschools,districtleadershipteam,and
leadershipofschoolsystem
Mean ODRs per 100 per School Day
1.2
100
0.0
DISTRICT 4
DISTRICT 2
K-6
6-9
9-12
K8-K12
Figure 6: NC Average ODR Rates Compared to the National Average
7
Sustainability and Capacity
The NC PBIS Initiative works to build capacity and support sustainability across the state. By creating
partnerships both within the education sector as well as with other agencies and organizations in
related child-serving fields, the initiative provides a wide base of support for PBIS and encourages
capacity building and continued development of education connections and community partnerships.
Capacity Building
• Trainer Boot Camp
• Links to System of Care
• Training Partnership with RtI
School Facilitator/Coach
• Professional Development based
on regional and state data
Schools
LEA Facilitator
Education Connections
• Intra-agency Workgroups
Regional Coordinators
• Conferences and Workshops
• Regional Trainings
State PBIS Consultant
• LEA Implementation Teams
PBIS State Leadership Team
Community Partnerships
• NC Collaborative for Children,
Youth, and Families
• Western NC After School
Care Coalition
• Smart Start Foundation
• 4-H Youth Development
Organization
• NC CSEFEL
PBIS Knowledge Base
ImplementaAon Science
Public Schools of North Carolina
state board of education | department of Public Instruction
exceptional Children division
The North Carolina Positive Behavior Intervention and
Support Initiative is part of the North Carolina State
Improvement Program funded through IDEA.
For more information, contact: Heather Reynolds
Behavioral Support and Special Programs
Exceptional Children Division
NC Department of Public Instruction
6356 Mail Service Center, Raleigh, NC 27699-6356
910-807-3313 (VOICE) • 910-807-3243 (FAX)
8
PBIS NaAonal TA Center
PBIS Blueprints
System of Care Principles
STAFF
Exceptional Children Division
Mary Watson, Director
Laura Snyder, Assistant Director
Behavior Support and Special Programs
Joe Simmons, Section Chief
Pat Gillott, Program Assistant
State PBIS Consultant
Heather Reynolds
PBIS Technical Assistance Partner
Bob Algozzine, UNC-Charlotte
Regional PBIS Coordinators
Minnie Kidd, Region 1
Margaret Gannon, Region 2
Correy Watkins, Region 3
Richard Tedescucci, Region 4
Cayce McCamish, Region 5
Dana Rusher, Region 6
Dreama McCoy, Region 7
Julie Weatherman, Region 8
January 2011