Rubric A
SPDG Evidence-based Professional Development Components Rubric
Professional
PD components
development (with required elements the
(PD) domains description should contain)
A(1)
Selection
Clear expectations are
provided for PD
participants and for
schools, districts, or other
agencies.
Sample descriptions corresponding to each of the four ratings
Exemplar description
(=4)
Good description
(=3)
Barely adequate description
(=2)
“Professional development participants “Professional development participants “Professional development participants
received a letter describing
received a letter describing
received a letter describing
expectations for participation. The
expectations for participation. A copy expectations for participation. A copy
letter states that they are required to of the letter is attached.
of the letter is attached.
attend 10 training sessions, each lasting
two hours; attend five follow-up
The local education agency sent a letter Principals from all district schools
Required elements:
coaching sessions; enter data in the
to all district principals explaining how signed a Memorandum of Agreement
management
system;
and
meet
with
they would be expected to support
(MOA). A copy of the agreement is
 Description of
the leadership team at least once a
participating teachers, and principals attached.”
expectations for PD
month. A copy of the letter is attached. were asked to sign a Memorandum of
participants (e.g.,
Agreement (MOA). The MOA states
attendance in training,
The local education agency sent a letter that they will allow teachers to
data reporting).1
to all district principals explaining how participate in all 10 trainings; they will
 Identification of what
allow teachers to participate in five
schools, districts, or other they would be expected to support
agencies agreed to provide participating teachers, and principals follow-up coaching sessions; and they
(e.g., necessary resources, were asked to sign a Memorandum of will facilitate monthly team meetings
Agreement (MOA). The MOA states
with participants. A copy of the
supports, facilitative
that principals will allow teachers to
agreement is attached.”
administration for the
2,3
participate
in
all
10
trainings;
they
will
participants).
allow teachers to participate in five
 Description of how
Elements missing from description
Elements missing from description
follow-up
coaching
sessions;
and
they

Participant
expectations
 Participant expectations
schools, districts, or other
 School/district/agency requirements
agencies were informed of will facilitate monthly team meetings
with participants. A copy of the
 Description of how schools/districts/
their responsibilities.2,3
agreement is attached.”
agencies were informed of their
responsibilities
Provide a brief description
of the form(s) used for these
agreements.
Inadequate description
(=1)
“The project team will develop a letter
that describes expectations for
professional development participants.
The project team will develop a
Memorandum of Agreement (MOA)
that principals from all district schools
will sign.”
Elements missing from description
 Description of completed activities
(only planned activities were
described)
 Participant expectations
 School/district/agency
requirements
 Description of how schools/
districts/agencies were
informed of their responsibilities
1
Rubric A
SPDG Evidence-based Professional Development Components Rubric
Professional
PD components
development (with required elements the
(PD) domains description should contain)
A(2)
Selection
Sample descriptions corresponding to each of the four ratings
Exemplar description
(=4)
“Our interviews with potential SPDG PBIS
trainers include a skills element, where the
applicant demonstrates some of the critical
skills required to successfully train
participants. SPDG trainers must have at least
10 years of experience implementing Positive
Required elements:
Behavioral Interventions and Supports (PBIS),
 Expectations for trainers’ as indicated on a resume they submitted.
They signed a contract agreeing to participate
qualifications and
experience and how these in two eight-hour workshops on their role and
responsibilities. The contract states that as
qualifications will be
trainers they are expected to conduct 10
ascertained.
trainings with school staff on PBIS
o Description of role and implementation, complete a post-event
evaluation after each PBIS training session,
responsibilities for
and provide post-training skill data to the
trainers (the people
district-level coaches.
who trained PD
Clear expectations are
provided for SPDG trainers
and SPDG coaches/
mentors.1
participants).
 Expectations for coaches’/
mentors’ qualifications
and experience and how
these qualifications will be
ascertained.
o Description of role or
responsibilities for
coaches or mentors
(the people who
provided follow-up to
training).
Good description
(=3)
“SPDG trainers must have at least 10
years of experience as a trainer of
Positive Behavioral Interventions and
Supports (PBIS), as indicated on a
resume they submitted. They signed a
contract agreeing to participate in two
eight-hour workshops on their role and
responsibilities. The contract states
that as trainers they are expected to
conduct 10 trainings with school staff
on PBIS implementation, complete a
post-event evaluation after each PBIS
training session, and provide posttraining skill data to the district-level
coaches.
Our interviews with potential SPDG PBIS
Expectations for coaches were
coaches include a skills element, where the
discussed during an introductory
applicant demonstrates some of the critical workshop.”
skills required to successfully train
participants. Coaches must have at least 15
years of experience as a coach or mentor, and
at least 10 years of experience implementing
Elements missing from description
PBIS. They also submitted resumes and signed  Coaches’ experience
a contract agreeing to observe training
 Coaches’ role and responsibilities
sessions weekly, provide observation data to
 How coaches’ qualifications will be
the project director monthly, and meet with
ascertained
trainers at least twice per month to strategize
about how to improve training techniques.
These expectations were also discussed
during an introductory workshop.”
Barely adequate description
(=2)
“SPDG trainers must have at least 10
years of experience as a trainer of
Positive Behavioral Interventions and
Supports (PBIS).
Inadequate description
(=1)
“See the attached contracts for trainers
and coaches.”
Coaches’ expectations were discussed
during an introductory workshop.”
Elements missing from description
 Trainers’ role and responsibilities
 How trainers’ qualifications will be
ascertained
 Coaches’ experience
 Coaches’ role and responsibilities
 How coaches’ qualifications will be
ascertained
Elements missing from description
 Trainers’ experience
 Trainers’ role and responsibilities
 How trainers’ qualifications will be
ascertained
 Coaches’ experience
 Coaches’ role and responsibilities
 How coaches’ qualifications will be
ascertained
2
Rubric A
SPDG Evidence-based Professional Development Components Rubric
Professional
PD components
development (with required elements the
(PD) domains description should contain)
B(1)
Training
Accountability for the
delivery and quality of
training.
Required elements:
 Identification of the lead
person(s) accountable for
training.
 Description of the role and
responsibilities of the lead
person(s) accountable for
training.
Sample descriptions corresponding to each of the four ratings
Exemplar description
(=4)
Good description
(=3)
Barely adequate description
(=2)
Inadequate description
(=1)
“The director of the Training Center,
“The director of the Training Center
“In November 2013, the SPDG hired the “The project team plans to hire
Nancy Love at the University of
oversees all training activities. She
lead person accountable for training. someone to oversee all training
Pittsburgh, oversees all training
devotes a substantial amount of time The director of the Training Center
activities.”
activities. She devotes a substantial
to designing a training plan, ensuring all oversees all training activities.”
amount of time (.75 full-time
trainers meet skill-level expectations,
equivalent) to the following tasks: (1) planning and monitoring training
designing a training plan, (2) ensuring events, and assessing the efficacy of
all trainers meet skill-level
the trainers and the overall training
expectations, (3) planning and
plan.”
monitoring training events, and (4)
assessing the efficacy of the trainers
and the overall training plan. She meets
with the training team once a month to
discuss progress and problem-solve
issues.”
Elements missing from description
 Detailed information about
accountability (e.g., frequency of
contact with training team)
Elements missing from description
 Description of lead person’s
responsibilities
 Detailed information about
accountability (e.g., frequency of
contact with training team)
Elements missing from description
 Description of completed activities
(only planned activities were
described)
 Identification of lead person
 Lead person’s responsibilities
 Detailed information about
accountability (e.g., frequency of
contact with training team)
3
Rubric A
SPDG Evidence-based Professional Development Components Rubric
Sample descriptions corresponding to each of the four ratings
Professional
PD components
development (with required elements the
(PD) domains description should contain)
B(2)
Training
Exemplar description
(=4)
Good description
(=3)
Effective research-based
“The project team implemented
“The project team implemented
adult learning strategies are training based on adult learning
training based on adult learning
4,5,6
used.
strategies described by Dunst &
strategies described by Dunst &
Trivette (2012).1 In our series of 10
Trivette (2012).1 In our series of 10
Required elements:
trainings, we used effective learning
trainings, we used effective learning
strategies from each of the following strategies from each of the following
 Identification of adult
categories: Introduction, Practicing,
learning strategies used, categories:
Evaluation, and Mastery.
including the source (e.g.,  Introduction: Trainers facilitated a
citation).
workshop for participants.
Once per quarter, the project team
 Description of how adult  Practicing: Participants applied the
conducts participant performance
learning strategies were
new skills through a role-play
reviews and facilitates a group
used.
exercise.
reflection to determine how well the
 Description of how data  Evaluation: The trainer assessed
are gathered to assess
participant strengths and weaknesses. strategies were implemented.
how well adult learning
 Mastery: Participants completed a
1
Dunst, C.J., & Trivette, C.M. (2012).
strategies were
self-assessment form on skills
Moderators of the effectiveness of
implemented.
attained.
adult learning method practices.
Journal of Social Sciences, 8, 143-148.”
Once per quarter, the project team
conducts participant performance
reviews and facilitates a group
reflection (a member of the research
Elements missing from description
 Description of how adult learning
team takes notes and categorizes the
strategies were used
data) to determine how well the
strategies were implemented.
1
Dunst, C.J., & Trivette, C.M. (2012).
Moderators of the effectiveness of
adult learning method practices.
Journal of Social Sciences, 8, 143-148.”
Barely adequate description
(=2)
“The project team implemented
training based on four adult learning
strategies described by Dunst &
Trivette (2012).1
Inadequate description
(=1)
“The project team implemented
training based on adult learning
strategies.”
1
Dunst, C.J., & Trivette, C.M. (2012).
Moderators of the effectiveness of
adult learning method practices.
Journal of Social Sciences, 8, 143-148.”
Elements missing from description
 Identification of adult learning
strategies used
 Description of how strategies were
used
 Description of how data were
gathered to assess how well
strategies were implemented
Elements missing from description
 Identification of adult learning
strategies used
 Source of research-based strategies
 Description of how strategies were
used
 Description of how data were
gathered to assess how well
strategies were implemented
4
Rubric A
SPDG Evidence-based Professional Development Components Rubric
Professional
PD components
development (with required elements the
(PD) domains description should contain)
B(3)
Training
Training is skill-based (e.g.,
participant behavior
rehearsals to criterion with
an expert observing).3,5
Required elements:
 Description of skills that
participants were
expected to acquire as a
result of the training.
 Description of activities
conducted to build skills.
 Description of how
participants’ use of new
skills was measured.
Sample descriptions corresponding to each of the four ratings
Exemplar description
(=4)
Good description
(=3)
“Participants were observed prior to
the start of training and immediately
following the training to assess their
skills in the following areas:
 Ability to use progress monitoring
measures correctly,
 Ability to analyze progress monitoring
data to group students according to
learning needs,
 Ability to identify needs and apply
appropriate intervention strategies to
meet student learning needs (i.e.,
fluency, comprehension, phonemic
awareness), and
 Ability to adjust instruction over time
in accordance with progress
monitoring data to improve student
learning outcomes.
“Participants were observed prior to
the start of training and immediately
following the training to assess their
skills in the following areas:
 Ability to use progress monitoring
measures correctly,
 Ability to analyze progress monitoring
data to group students according to
learning needs,
 Ability to identify needs and apply
appropriate intervention strategies to
meet student learning needs (i.e.,
fluency, comprehension, phonemic
awareness), and
 Ability to adjust instruction over time
in accordance with progress
monitoring data to improve student
learning outcomes.”
Participants also engaged in roleplaying exercises to practice new skills
learned. Follow-up observations were
conducted three months after training
ended to ensure participants continued
applying new skills and knowledge over
time to improve outcomes for students
with disabilities.”
Elements missing from description
 Activities to build skills
Barely adequate description
(=2)
Inadequate description
(=1)
“Participants were observed to assess “Plans are in place assess participants’
their skills. Participants also engaged in skills by conducting observations.”
role-playing exercises to practice new
skills learned.”
Elements missing from description
 Skills to be acquired
 Frequency of skill assessment
Elements missing from description
 Description of completed activities
(only planned activities were
described)
 Skills to be acquired
 Activities to build skills
 Method used to assess skill
acquisition
 Frequency of skill assessment
5
Rubric A
SPDG Evidence-based Professional Development Components Rubric
Professional
PD components
development (with required elements the
(PD) domains description should contain)
B(4)
Training
Sample descriptions corresponding to each of the four ratings
Exemplar description
(=4)
Good description
(=3)
Barely adequate description
(=2)
Inadequate description
(=1)
Training outcome data are
collected and analyzed to
assess participant
knowledge and skills.5
“The project team used Mathematical “The project team assessed participant “The project team assessed participant “The project team plans to assess
Knowledge for Teaching to assess
knowledge and skills of the
knowledge and skills of the
participant knowledge and skills by
participant knowledge and skills related mathematics professional development mathematics professional development administering a pretest and a posttest.”
to the mathematics professional
sessions using the Mathematical
sessions by administering a pretest and
development sessions. Teachers
Knowledge for Teaching measure.
a posttest using the Mathematical
Required elements:
completed a pretest prior to their
Teachers completed a pretest prior to Knowledge for Teaching measure.”
their participation in the professional
 Identification of training participation in the professional
development
and
completed
a
posttest
development, during an introductory
outcome measure(s).
 Description of procedures immediately following the last training participant meeting in August 2013.
Teachers completed a posttest
to collect pre- and post- session. Trainers met with each
training data or another participant individually to discuss their immediately following the last training
assessment results and areas that need session. Trainers met with each
kind of assessment of
improvement and shared a summary of participant individually to discuss their
knowledge and skills
aggregated results with the SPDG
assessment results and areas that need
gained from training.
implementation committee. Overall,
improvement and shared a summary of
 Description of how
scores
on
the
measure
improved
for
90
aggregated results with the SPDG
training outcome data
percent of participants. Trainers
implementation committee.”
were reported.
reviewed
the
data
and
determined
that
 Description of how
many participants did not show
training outcome data
improvement on questions related to
were used to make
the Middle School Content Knowledge
appropriate changes to
Elements missing from description
Elements missing from description
Elements missing from description
the training and to provide Areas section of the assessment
 Description of how data were used
 Pre- and posttest procedures
 Description of completed activities
further supports through measure. Coaches provided additional
(only planned activities were
 Description of how data were
support to participants in this area, and
coaching.
described)
reported
trainers used the data to revise the
 Outcome measure used
 Description of how data were used
training materials related to this
 Pre- and posttest procedures
 Description of how data were
content area.”
reported
 Description of how data were
used
6
Rubric A
SPDG Evidence-based Professional Development Components Rubric
Professional
PD components
development (with required elements the
(PD) domains description should contain)
B(5)
Training
Trainers (the people who
trained PD participants) are
trained, coached, and
observed.5,7
Sample descriptions corresponding to each of the four ratings
Exemplar description
(=4)
Good description
(=3)
“Each trainer participated in 40 hours “Each trainer participated in 40 hours
of training in Positive Behavioral
of training in Positive Behavioral
Interventions and Supports (PBIS)
Interventions and Supports (PBIS)
provided by a contracted consultant
provided by a contracted consultant
with over 30 years’ experience
with over 30 years’ experience
Required elements:
implementing PBIS. The consultant
implementing PBIS. The consultant
 Description of training
observed each trainer at least once per provided approximately two hours of
provided to trainers.
week and completed the PBIS Fidelity coaching per trainer per week when
 Description of coaching
Assessment to evaluate the trainer’s
necessary to address individual ‘just-inprovided to trainers.
adherence to the schedule, provision of time’ coaching needs. When several
 Description of procedures content, and use of appropriate
trainers had similar needs, the
for observing trainers.
processes. The training coordinator
consultant conducted group coaching
 Identification of training used that information to determine the of trainers to address targeted needs.
fidelity instrument used type of support each trainer needed. In addition, if measures of participant
(measures the extent to The consultant provided approximately skill are not showing the desired levels
which the training is
two hours of coaching per trainer per across multiple trainings, the
implemented as intended). week when necessary to address
consultant meets with the trainers and
 Description of procedures individual ‘just-in-time’ coaching needs. discusses how provision of content
to obtain participant
When several trainers had similar
might be improved.”
feedback.
needs, the consultant conducted group
Elements missing from description
 Description of how
coaching of trainers to address targeted  Observation procedures
observation and training needs. In addition, if measures of
 Identification of fidelity instrument
fidelity data were used
participant skill are not showing the
used
(e.g., to determine if
desired levels across multiple trainings,
changes should be made the consultant meets with the trainers
to the content or structure and discusses how provision of content
of trainings, such as
might be improved.”
schedule, processes; to
ensure that trainers are
qualified).
Barely adequate description
(=2)
“Each trainer participated in 40 hours
of training in Positive Behavioral
Interventions and Supports (PBIS)
provided by a contracted consultant
with over 30 years’ experience
implementing PBIS.”
Elements missing from description
 Coaching for trainers
 Observations of trainers
 Identification of fidelity instrument
used
 Description of procedures to obtain
participant feedback
 Description of how data were used
Inadequate description
(=1)
“Each trainer will be trained, coached,
and observed to ensure that she/he has
the necessary skills.”
Elements missing from description
 Description of completed activities
(only planned activities were
described)
 Training for trainers
 Coaching for trainers
 Observations of trainers
 Fidelity instrument used
 Procedures to obtain
feedback
 How data were used
7
Rubric A
SPDG Evidence-based Professional Development Components Rubric
Professional
PD components
development (with required elements the
(PD) domains description should contain)
C(1)
Coaching
Sample descriptions corresponding to each of the four ratings
Exemplar description
(=4)
Good description
(=3)
Accountability for the
development and
monitoring of the quality
and timeliness of SPDG
coaching services.8
“The project team hired a consultant, “The project team hired a consultant
Brad Greene from Delaware College,
who is responsible for all coaching
who is responsible for all coaching
services. He signed a contract agreeing
services. He signed a contract agreeing to commit 30 hours per week to
to commit 30 hours per week to
carrying out several responsibilities,
carrying out several responsibilities,
including the creation and oversight of
Required elements:
including the creation and oversight of the project’s coaching service plan. In
 Identification of the lead the project’s coaching service plan. In this role he is responsible for creating
person(s) responsible for this role, he is responsible for creating job descriptions, creating interview
job descriptions, creating interview
protocols, setting up training for the
coaching services.
coaches, and identifying additional
 Description of the role and protocols, setting up training for the
responsibilities of the lead coaches, overseeing coaching fidelity training needs for coaches. The
person(s) accountable for measures, using fidelity and outcome consultant also observes each coach
data to determine if additional training once per month.”
coaching services.
 Description of how data for coaches is necessary, and evaluating
the coaches’ progress annually. The
were used to provide
feedback to coaches and consultant meets with each coach once
per month to share the observational
improve coaching
data and outcomes data and strategize
strategies.
with coaches about how to address
their needs.”
Elements missing from description
 Description of how data were used
Barely adequate description
(=2)
“The project team hired a consultant
who is responsible for all coaching
services. The consultant meets with
coaches monthly to strategize about
how to address coaches’ needs.”
Elements missing from description
 Detailed information about the role
and responsibilities of the lead
person.
 Description of how data were used
to provide feedback
Inadequate description
(=1)
“The project team plans to hire a
consultant to oversee all coaching
services.”
Elements missing from description
 Description of completed activities
(only planned activities were
described)
 Identification of lead person
 Lead person’s responsibilities
 Description of how data were
used
8
Rubric A
SPDG Evidence-based Professional Development Components Rubric
Professional
PD components
development (with required elements the
(PD) domains description should contain)
C(2)
Coaching
SPDG coaches use multiple
sources of information in
order to provide assistive
feedback to those being
coached and also provide
appropriate instruction or
modeling.
Sample descriptions corresponding to each of the four ratings
Exemplar description
(=4)
“Coaches use behaviorally focused
feedback, help practitioners to
interpret and use this feedback, and
make a strong link between
performance improvement and desired
outcomes (see London & Smither,
2002).
Good description
(=3)
Barely adequate description
(=2)
Inadequate description
(=1)
“Coaches utilize a fidelity checklist,
email communications and a selfassessment to provide appropriate
instruction to PD participants.
“Coaches conduct monthly onsite visits “Currently there is no standardized
to observe instructional practice and
coaching strategy being implemented.
assess implementation progress. PD
See information provided in C(1).”
participants receive a monthly email
from their coach describing skills they
Coaches conduct monthly onsite visits should practice, based on the practices
to observe instructional practice.
observed.”
Coaches collect direct observational
Required elements:
Our coaches model the practices that data during onsite visits and provide
the participants are expected to use
feedback after each visit. During the
 Should describe the
during training sessions. During onsite feedback sessions, coaches model
coaching strategy used
and the appropriateness visits, coaches discuss challenges the instructional practices and provide
participant is facing in implementing
instruction with a focus on transfer of
for use with adults (i.e.,
the practices. PD participants are asked skills. Coaches review past
evidence provided for
to evaluate implementation of the
observational data before each visit to
coaching strategies).6
practices
monthly.
Coaches
use
the
determine what instruction needs to be
 Describe how SPDG
self-evaluation data and observations reinforced for each participant.”
coaches monitored
implementation progress. from onsite visits to evaluate
implementation progress. Coaches
 Describe how the data
from the monitoring are prepare a consultation report after
used to provide feedback each onsite visit that provides feedback
for participants. The consultation
to implementers.
Elements missing from description
Elements missing from description
Elements missing from description
reports provide coaches with a record
 Description of coaching strategy
 Description of coaching strategy
of implementation progress for each PD  Evidence to support coaching
strategy
(and evidence to support coaching
(and evidence to support coaching
participant that facilitates appropriate
strategy)
strategy)
instruction.”
 Multiple sources of information
used to provide feedback
 Multiple sources of information
used to provide feedback
 Description of how monitoring data
are used
9
Rubric A
SPDG Evidence-based Professional Development Components Rubric
Professional
PD components
development (with required elements the
(PD) domains description should contain)
D(1)
Accountability for fidelity
Performance measurement and
Assessment reporting system is clear
(e.g., lead person
(Data-based designated).10
Decisionmaking)
Required elements:
 Provide a description of
the role/responsibilities of
the lead person and who
this person is.
Sample descriptions corresponding to each of the four ratings
Exemplar description
(=4)
Good description
(=3)
“Each school is required to have a
“The SPDG steering committee has
designated implementation team
designated the SPDG evaluator as the
consisting of at least two staff members person responsible for fidelity
who are responsible for fidelity
measurement and reporting. The SPDG
measurement and reporting. The SPDG evaluator’s contract outlines the set of
evaluator, Tricia Lovett from the
short-term and long-term tasks that
University of Texas, is responsible for she must complete. She submitted an
oversight of fidelity measurement
application highlighting her
within schools. The SPDG evaluator: (1) qualifications and, after a thorough
collects fidelity data from school-level interview process, the SPDG steering
teams weekly (the implementation
committee hired her.”
team sends the information to the
evaluator electronically), (2) meets with
each implementation team monthly to
discuss implementation fidelity data,
and (3) evaluates progress. The
evaluator is also responsible for sharing
information about fidelity with the
state-level SPDG steering committee.”
Elements missing from description
 Responsibilities of the lead person
Barely adequate description
(=2)
Inadequate description
(=1)
“Districts designate a person or persons “Districts and schools will hire a person
to be responsible for SPDG initiative
who will be accountable for program
implementation and outcomes. This
implementation.”
person(s) collects data and reports
problems related to implementation to
central office personnel.”
Elements missing from description
 Lead person designated
 Detailed information about lead
person’s role/responsibilities
Elements missing from description
 Description of completed activities
(only a planned activity was
described)
 Lead person designated
 Descriptions of lead person’s
role/responsibilities
10
Rubric A
SPDG Evidence-based Professional Development Components Rubric
Professional
PD components
development (with required elements the
(PD) domains description should contain)
Sample descriptions corresponding to each of the four ratings
Exemplar description
(=4)
Good description
(=3)
Barely adequate description
(=2)
Inadequate description
(=1)
D(2)
Coherent data systems are
Performance used to make decisions at
Assessment all education levels (SEA,
regional, LEA, school).
“The SEA and regional SPDG teams,
“Districts in the state overseeing
“Districts in the state overseeing
“Data are used for decision-making at
along with districts and schools chosen implementation of PBIS in their schools implementation of PBIS in their schools all levels of implementation.”
to participate in the SPDG program,
have leadership teams composed of at enter data in the District
have implementation teams composed least five people. District leadership
Implementation Data System. Each
of at least three people who meet on a teams visit schools monthly to observe school in the district designates one
Required elements:
regular basis (at least quarterly for
PBIS implementation and complete the member of the implementation team
 Describe data systems that state and regional teams and at least Implementation Inventory. They enter to be responsible for entering data on
monthly for district and school teams). data in the District Implementation
outcomes for students in that school
are in place for various
District-level data are entered in the
Data System and meet monthly to
into the PBIS Implementation Data
education levels.
discuss results. They identify areas in System on a quarterly basis.”
 Describe how alignment or District PBIS Data System and then
imported into the Statewide PBIS Data which implementation teams may need
coherence is achieved
System monthly. The data are
support or assistance and discuss areas
between various data
accessible
to
key
staff
at
all
education
in which schools have made progress in
systems or sources of
levels and are discussed quarterly. The implementation. Each school in the
data.
state-level team used the data to
district designates one member of the
 Describe how multiple
identify
areas
in
which
it
could
implementation team to be responsible
sources of information are
standardize implementation
for entering data on outcomes for
used to guide
procedures across districts, and
students in that school into the PBIS
improvement and
10
districts
used
the
data
to
determine
if
Implementation Data System on a
demonstrate impact.
student outcomes are improving.
quarterly basis. Each school has its own
School and district teams also observe data reporting system.”
PBIS implementation and report
progress to the SEA and regional teams.
Elements missing from description
Elements missing from description
Elements missing from description
The SEA and regional teams provide a
 Description of data systems in place
 Description of how data systems are
 Description of how data systems are
at multiple education levels
aligned
aligned
monthly report to the State Board.”
 Description of how multiple sources
of information are used to guide
improvement/demonstrate impact
 Description of how data systems are
aligned
 Description of how multiple sources
of information are used to guide
improvement/demonstrate impact
11
Rubric A
SPDG Evidence-based Professional Development Components Rubric
Professional
PD components
development (with required elements the
(PD) domains description should contain)
Sample descriptions corresponding to each of the four ratings
Exemplar description
(=4)
Good description
(=3)
Barely adequate description
(=2)
D(3)
Implementation fidelity and “Districts involved with the SPDG
“Districts involved with the SPDG
“Districts involved with the SPDG
Performance student outcome data are project collected implementation
project collected implementation
project collected implementation
Assessment shared regularly with
fidelity data using the PBIS
fidelity data using the PBIS
fidelity data. Key district
stakeholders at multiple
Implementation Checklist (IC), and
Implementation Checklist (IC), and
representatives used the data to
levels (SEA, regional, local, schools collected student outcome data schools collected student outcome data identify schools that would benefit
individual, community,
using the Behavior Rating Scale (BRS). A using the Behavior Rating Scale (BRS). A from additional coaching.”
other agencies).10
designated school representative sends designated school representative sends
a data file to the district each month. a data file to the district each month.
Required elements:
District representatives send schoolDistrict representatives send schoollevel outcome and implementation
level outcome and implementation
 Describe the feedback
fidelity data to state representatives
loop for each level of the fidelity data to state representatives
quarterly. State and district
quarterly. State and district
system the SPDG works
representatives met in January 2015 to representatives met in January 2015 to
with.
discuss the data and identify schools
discuss the data and identify schools
o Describe how these
with low scores on the IC and BRS. They with low scores on the IC and BRS.”
data are used for
determined that most schools would
decisionmaking to
ensure improvements benefit from additional coaching.
District representatives met with the
are made in the
schools targeted to develop an action
targeted outcome
plan for improving implementation
areas.
Elements missing from description
Elements missing from description
 Describe how fidelity data fidelity and increasing coaching.
 Description of the feedback loop for
 Description of how fidelity data
Schools had 60 days to submit a report
inform modifications to
different education levels
informs
modifications
to
to state representatives describing
implementation drivers
implementation drivers
 Description of how fidelity data
(e.g., how can Selection, action plan progress.”
inform modifications to
Training, and Coaching
implementation drivers
better support high
fidelity).10
Inadequate description
(=1)
“Districts involved with the SPDG
project utilize implementation fidelity
data for decision-making.”
Elements missing from description
 Description of the feedback loop for
different education levels
 Description of how data are used to
make decisions
 Description of how fidelity data
inform modifications to
implementation drivers
12
Rubric A
SPDG Evidence-based Professional Development Components Rubric
Professional
PD components
development (with required elements the
(PD) domains description should contain)
Sample descriptions corresponding to each of the four ratings
Exemplar description
(=4)
“The project’s needs assessment indicated
a high number of disciplinary issues
occurring in schools. Therefore, one target
goal was for schools to show improvement
on the Benchmarks of Quality element
related to teaching expected behavior
routines in the classroom. The number of
students referred to school administrators
Required elements:
was expected to decrease within six
 Describe how benchmarks months of improved implementation of
classroom systems. The school district
are created and shared.
outlined these expectations during in Describe positive
recognition processes for service meetings in August 2013, which all
teachers were required to attend. Schools
achievements.
that met the targets for implementation
 Describe how data are
and student outcomes were acknowledged
used to “market” the
in a district newsletter that was distributed
initiative.
in September 2014 to all staff in the district
and sent to the state education agency.
Aggregated data were also posted on the
district’s website, with additional
information about PBIS. The website
contains a description of the program and
progress to date, with contact information
for additional information.
In addition, the SPDG creates a colorful and
graphic-filled annual report that goes out
D(4)
Goals are created with
Performance benchmarks for
Assessment implementation and
student outcome data, and
successes are shared and
celebrated.10
to stakeholders across the State and to
the appropriate officials in the Federal
Government. This report details
successes and areas targeted for
improvement.”
Good description
(=3)
Barely adequate description
(=2)
Inadequate description
(=1)
“The project’s needs assessment
indicated a high number of disciplinary
issues occurring in schools. Therefore,
one target goal was for schools to show
improvement on the Benchmarks of
Quality element related to teaching
expected behavior routines in the
classroom. The number of students
referred to school administrators was
expected to decrease within six months
of improved implementation of
classroom systems. The school district
outlined these expectations during inservice meetings in August 2013, which
all teachers were required to attend.
Schools that met the targets for
implementation and student outcomes
were acknowledged in a district
newsletter that was distributed in
September 2014 to all staff in the
district and sent to the state education
agency.”
“One target goal was for schools to
show improvement in teaching
expected behavior routines in the
classroom. The school district outlined
these expectations during in-service
meetings in August 2013, which all
teachers were required to attend.
Schools that met the targets for
implementation and student outcomes
were acknowledged in a district
newsletter that was distributed in
September 2014 to all staff in the
district and sent to the state education
agency.”
“Schools were expected to show
improvement in teaching expected
behavior routines in the classroom, and
student behavior was expected to
decrease.”
Elements missing from description
 Description of how data are used to
“market” the initiative
Elements missing from description
 Description of how benchmarks are
created
 Description of how data are used to
“market” the initiative
Elements missing from description
 Description of how benchmarks are
created and shared
 Description of positive recognition
processes for achievements
 Description of how data are used to
“market” the initiative
13
Rubric A
SPDG Evidence-based Professional Development Components Rubric
Professional
PD components
development (with required elements the
(PD) domains description should contain)
Sample descriptions corresponding to each of the four ratings
Exemplar description
(=4)
Good description
(=3)
D(5)
Participants are instructed “School personnel responsible for
“Individuals who will submit data
Performance in how to provide data to collecting and submitting data visit the attend a one-day workshop in which
Assessment the SPDG Project.
designated data submission website
they participate in hands-on exercises
and log in using their unique user ID
on how to submit the data. During the
Required elements:
and password. Then they enter data in workshop they learn how to access the
 Procedures described for several fields in the system and submit data submission website, how to enter
the data. Individuals who will submit
and submit the data, and they are
data submission.
data
attend
a
one-day
workshop
in
provided with the data submission
 Guidance provided to
which they participate in hands-on
schedule. The individuals responsible
schools/districts.
exercises on how to submit the data. for submitting the data are given a
During the workshop, they learn how to phone number and email address for
access the data submission website,
the state’s SPDG coordinator for posthow to enter and submit the data, and workshop assistance. The message
they are provided with the data
includes helpful tips and FAQs related
submission schedule. The individuals to data submission. The SPDG
responsible for submitting the data are coordinator sends guidance to school
given a phone number and email
administrators so that they may also
address for the state’s SPDG
support participants in submitting their
coordinator for post-workshop
data in a timely manner.”
assistance. The message includes
helpful tips and FAQs related to data
submission. The SPDG coordinator
sends guidance to school
Elements missing from description
administrators so that they can also
 Procedures for data submission
support participants in submitting their
described
data in a timely manner.”
Barely adequate description
(=2)
Inadequate description
(=1)
“Designated school staff collect and
“The LEA is responsible for providing
report data to the SPDG Project. The
data to the SPDG Project.”
SPDG coordinator sends guidance to
school administrators so that they may
also support participants in submitting
their data in a timely manner.”
Elements missing from description
 Procedures for data submission
described
 Information about instructions
provided to participants
Elements missing from description
 Procedures for data submission
described
 Information about instructions
provided to participants
 Guidance provided to schools/
districts
14
Rubric A
SPDG Evidence-based Professional Development Components Rubric
Professional
PD components
development (with required elements the
(PD) domains description should contain)
Sample descriptions corresponding to each of the four ratings
Exemplar description
(=4)
Good description
(=3)
E(1)
Administrators are trained “A four-hour workshop was provided “Principals identify and address
Facilitative
appropriately on the SPDG- for principals of schools participating in teachers’ needs (e.g., make
Administrative supported practices and
the SPDG. Principals learned how to
adjustments to teacher schedules) and
Support/
have knowledge of how to identify and address teachers’ needs
support teachers in making appropriate
Systems
support its implementation. (e.g., make adjustments to teacher
accommodations to meet student
Intervention
schedules) and support teachers in
learning needs. Principals also ensure
Required elements:
making appropriate accommodations that team members have adequate
to meet student learning needs.
planning time and made
 Role/job description of
administrators relative to Principals were also advised to ensure recommendations for additional
coaching. All principals regularly review
program implementation that team members have adequate
planning time, and they were trained in student outcome data to obtain
provided.
how to recognize that a PD participant feedback from teachers on
 Describe how the SPDG
may need additional coaching. All
implementation to assess progress
trains and supports
administrators so that they principals attended the workshop, and toward improving student outcomes.”
they meet monthly with a member of
may in turn support
the SPDG steering committee to discuss
implementers.
how the principals are supporting the
implementers and receive ideas for
strategies and support for their efforts.
Recommendations for improvement
are provided if necessary. Principals
regularly review student outcome data
Elements missing from description
and obtain feedback from teachers on
 Description of how the SPDG trains
implementation to assess progress
and supports administrators
toward improving student outcomes.”
Barely adequate description
(=2)
Inadequate description
(=1)
“Principals are encouraged to
“LEA and school administrators are
participate in training. The training
required to support implementation of
provided to administrators outlines the SPDG-supported practices.”
processes for supporting effective
implementation of SPDG-supported
practices.”
Elements missing from description
 Description of the role of
administrators
 Description of how the SPDG
ensures that administrators support
implementers
Elements missing from description
 Description of the role of
administrators
 Description of how the SPDG trains
administrators
 Description of how the SPDG
ensures that administrators support
implementers
15
Rubric A
SPDG Evidence-based Professional Development Components Rubric
Professional
PD components
development (with required elements the
(PD) domains description should contain)
E(2)
Leadership at various
Facilitative
education levels (SEA,
Administrative regional, LEA, school, as
Support /
appropriate) analyzes
Systems
feedback regarding barriers
Intervention and successes and makes
the necessary decisions and
changes, including revising
policies and procedures to
alleviate barriers and
facilitate implementation.
Sample descriptions corresponding to each of the four ratings
Exemplar description
(=4)
“Twice annually, teachers in each
district implementing RtI complete two
web-based surveys: one in which they
report on facilitators and barriers to
implementation and one in which they
report data on student reading fluency.
After the deadline for data submission,
the project’s lead evaluator logs into
the data system to export the data in
preparation for analysis. The project’s
evaluation team reviews survey data
from teachers and student outcome
Required elements:
data, and reports district- and statelevel results to the SPDG steering
 Describe processes for
collecting, analyzing, and committee. This committee is
composed of key representatives from
utilizing student and
teacher data to recognize the state education agency, two
representatives from each school
barriers to
implementation success. district implementing RtI, and each
school’s principal. The team strategizes
 Describe processes for
about how to address the barriers to
revising policies and
procedures to support a improve student outcomes. One
common facilitator to implementation
new way of work.
was having weekly meetings with other
teachers to discuss challenges. Based
on this information, the steering
committee created a formal meeting
structure for teachers implementing
RtI. The meetings must include at least
one school administrator.”
Good description
(=3)
“The project’s evaluation team reviews
survey data from teachers and student
outcome data and reports state-level
results to the SPDG steering
committee. This committee is
composed of key representatives from
the state education agency, two
representatives from each school
district implementing RtI, and each
school’s principal. The team strategizes
about how to address the barriers to
improve student outcomes. One
common facilitator to implementation
was having weekly meetings with other
teachers to discuss challenges. Based
on this information, the steering
committee created a formal meeting
structure for teachers implementing
RtI. The meetings must include at least
one school administrator.”
Elements missing from description
 Description of process for collecting
student and teacher data
Barely adequate description
(=2)
Inadequate description
(=1)
“Twice annually, teachers in each
“LEA administrators utilized data for
district implementing RtI complete a
making decisions related to improving
survey in which they report on
student outcomes.”
facilitators and barriers to
implementation. Data are sent to the
district, along with data on student
reading fluency. The project’s
evaluation team reviews survey data
from teachers and student outcome
data and reports state-level results to
the SPDG steering committee. The
team strategizes about how to address
the barriers to improve student
outcomes.”
Elements missing from description
 Description of process for revising
policies and procedures
Elements missing from description
 Description of process for collecting,
analyzing, and utilizing student and
teacher data
 Description of data use to address
implementation barriers
 Description of process for revising
policies and procedures
16
Rubric A
SPDG Evidence-based Professional Development Components Rubric
1
http://nirn.fpg.unc.edu/sites/nirn.fpg.unc.edu/files/resources/NIRN-MonographFull-01-2005.pdf (pp. 36-39).
2
http://learningforward.org/standards/resources#.U1Es3rHD888 .
3
Guskey, T.R. (2000). Evaluating professional development (pp. 79-81). Thousand Oaks, CA: Corwin Press.
4
Dunst, C.J., & Trivette, C.M. (2012). Moderators of the effectiveness of adult learning method practices. Journal of Social Sciences, 8, 143-148.
5
http://nirn.fpg.unc.edu/sites/nirn.fpg.unc.edu/files/resources/NIRN-MonographFull-01-2005.pdf (pp. 39-43).
6
http://learningforward.org/standards/learning-designs#.U1GVhbHD888 .
7
http://nirn.fpg.unc.edu/sites/nirn.fpg.unc.edu/files/resources/NIRN-MonographFull-01-2005.pdf (pp. 47-55).
8
http://nirn.fpg.unc.edu/sites/nirn.fpg.unc.edu/files/resources/NIRN-MonographFull-01-2005.pdf (pp. 44-47).
9
http://learningforward.org/standards/data#.U2FGp_ldWYk .
10
http://implementation.fpg.unc.edu/sites/implementation.fpg.unc.edu/files/resources/NIRN-ImplementationDriversAssessingBestPractices.pdf (pp. 15-16).
17