IMPROVING TEACHER FIDELITY WHEN
IMPLEMENTING PBS PROCEDURES:
COACHING TO BUILD SUSTAINABILITY
IN OUR SCHOOLS
KERI S. BETHUNE, PH.D., BCBA -D
JAMES MADISON UNIVERSITY
INTRODUCTION
• Research has shown that the prevalence of
challenging behaviors is higher among individuals
with disabilities compared to challenging behaviors
of their typically developing peers (Sigafoos, Arthur, &
O’Reilly, 2003).
• More specifically, individuals with autism spectrum
disorders (ASD) and intellectual disability (ID) are
more likely to exhibit self-injurious behavior and
stereotypic behaviors than individuals with ID alone
(Rojahn et al., 2010).
INTRODUCTION
• Some of the negatives effects of challenging behavior
for students include
(a)failure to reach their academic potential,
(b)increased school suspensions and increased school
absences,
(c)higher rates of peer rejection accompanied by a lack
of an appropriate social network,
(d)disruption of other students’ learning, and
(e)an increase in the time required for planning from
the entire educational team which negatively
impacts the time teachers can spend on academic
planning (Chandler & Dahlquist, 2006).
INTRODUCTION
• FBAs and resulting function-based interventions have
been more successful in reducing challenging
behaviors than non-function-based interventions
(Filter & Horner, 2009; Mustian, 2010; Umbreit et al., 2007).
• Although conducting FBAs and implementing
function-based interventions are effective in
reducing challenging behavior among students with
disabilities, and are federally mandated, many school
personnel fail to implement this process with the
fidelity needed to achieve successful outcomes (Blood
& Neel, 2007; Scott, Nelson, & McIntyre, 2005).
INTRODUCTION
• Coaching is when an initial training occurs and
afterwards an expert in the targeted area provides
individualized feedback and support to help
teachers implement and sustain the new teaching
behaviors (Kretlow & Bartholomew, 2010).
• The purpose is to increase teachers’ accuracy
when implementing evidence-based practices
(EBPs; Kretlow & Bartholomew, 2010).
• Throughout the coaching process, the expert
prompts the teacher to use the targeted teaching
behavior in an actual teaching situation and
provides immediate feedback on his or her
performance (Kretlow & Bartholomew, 2010).
INTRODUCTION
• Coaching can be divided into a number of types
including supervisory coaching, side-by-side
coaching, and bug-in-ear coaching.
• Side-by-side coaching is when an expert, but not a
direct supervisor, observes the teacher
implementing the target practice, provides
feedback in vivo, and may even model the
practice with the students while the teacher
observes (Kretlow & Bartholomew, 2010).
INTRODUCTION
• Coaching has been used to increase teachers’ accuracy
when implementing evidence based practices (EBPs) and to
have promising results towards improved student
achievement (Kretlow & Bartholomew, 2010).
• Coaching has been effective in increasing teacher accuracy
in implementation of
• classwide positive behavior supports (PBS; Filcheck, McNeil, Greco, &
Bernard, 2004),
• reading interventions (Jager, Reezigt, & Creemers, 2002; Kohler, Ezell, &
Paluselli, 1999; Linguaris-Kraft & Marchand-Martella, 1993; Morgan et al.,
1994),
• explicit math instruction (Kretlow, Cooke, & Wood, 2011; Kretlow, Wood, &
Cooke, 2011), and
• teacher praise in an RTI model (Myers, Simonsen, & Sugai, 2011).
COACHING TEACHERS ON
FUNCTION-BASED INTERVENTIONS
INTRODUCTION
Purpose of the Study
• To evaluate the effects of coaching on special
education teachers’ implementation of
function-based interventions with students
with severe disabilities
RESEARCH QUESTIONS
• What are the effects of coaching on special
education teachers’ accuracy of implementation of
function-based interventions?
• What is the effect of coaching on teachers’ ability to
generalize a function-based intervention (developed
using the same process) to a second identified
challenging behavior and/or setting with their
students?
RESEARCH QUESTIONS
• To what extent do students’ challenging behaviors
decrease and replacement behaviors increase as
teachers implement function-based interventions
with increasing fidelity?
• Do teachers and administrators find the use of
coaching appropriate and efficient?
METHOD
• Participants & Setting
• Elementary school in rural county in the
southeastern US
• 4 teacher/student pairs
• Student inclusion criteria
• received special education services
• displayed challenging behavior that interfered
with participation in school activities
METHOD
• Participants & Setting
• Teacher inclusion criteria
• Held a current North Carolina teaching license in
special education
• Taught at least one student receiving special
education services
• Agreed to attend a one day workshop on FBA
• Had not previously been involved in a
coaching/mentoring program
• Was not a Board Certified Behavior Analyst or a
Board Certified Associate Behavior Analyst
METHOD
Demographic Information
Teacher Student Age
Mrs.
Susan
4
Yacht
Mrs.
Karla
8
Williams
Mr.
Michael
5
Carlisle
Mrs.
Jack
10
Green
Disability
trisomy 9
mosaicism
Problem Behavior
taking glasses off
Down
syndrome
noncompliance
autism
tantrum
autism
off task behavior
METHOD
• Dependent Measures
• Teachers
• Percent accuracy of teacher implementation of the
function-based intervention as measured by the
procedural fidelity checklist
• Students
• Students’ problem behavior (frequency or percent
interval)
• Students’ replacement behavior (frequency or percent
interval)
• Social Validity Questionnaire
METHOD
Research Design
• Teachers - delayed multiple baseline across participants
• Delayed multiple was used because of practical difficulties
encountered after the beginning of student data collection
• Delayed multiple baselines use the same experimental logic as
standard multiple baselines with the exception that participants
enter the baseline phase in a staggered fashion (Cooper et al.,
2007)
• Note: Although the terms are sometimes used synonymously,
the delayed multiple baseline used in this study is not the same
as a non-concurrent multiple baseline as described by Watson
and Workman (1981) and Gast (2010)
METHOD
Research Design
•Students - multiple baseline across
participants’ design
•Phase changes were dependent on teacher
behavior
METHOD
General Procedures
• Initial observation
• General observations
• Collect ABC data
• Inservice Training (workshop)
• One-day, six hour inservice
• First half of the day provided information regarding
function-based interventions in a PowerPoint format
• Second half of the day moved to a workshop format where
the researcher worked with the participants to complete
an FBA and develop function-based interventions for their
paired students
METHOD
General Procedures
• Student baseline
• Data were taken on the frequency or percent intervals of
the student challenging and replacement behavior
• Teachers did not start function-based interventions
• 10 min sessions
• Functional Analysis
• Researcher conducted a brief alternating treatments design
to confirm the hypothesis regarding the function of each
student’s problem behavior
• 4 sessions, five min per session
METHOD
General Procedures
• Teacher baseline
• Teachers implemented function-based
interventions during the same activity
• Researcher provided no prompting or
support
• 10 min sessions
METHOD
General Procedures
• Coaching
• Researcher provided side-by-side coaching with
the teacher as they worked with the target student
during the identified task
• After coaching , the researcher observed the
teacher implementing the same task/activity and
scored the teacher accuracy of implementation
and student behaviors
• Once teachers had a minimum of 90% accuracy for
two consecutive days no more coaching sessions
were implemented.
METHOD
General Procedures
• Generalization
• Generalization was measured by observing the teacher
work with the same student during a different activity
• 10 min sessions
• Maintenance
• Maintenance data were collected once per week until all
participants had completed coaching, then again after two
and a half weeks
• Data Analysis
• Visual inspection of graphed data
METHOD
• Interobserver reliability was conducted on
32.2% of teacher sessions and 31.3% of
student sessions distributed evenly across
phases.
• Averaged 100% for teachers’ accuracy of
implementation of function-based interventions
• Averaged 99.7% (range 90% through 100%) for
student behavior
• Fidelity was conducted on 50% of coaching
sessions and averaged 100%.
METHOD
• Social validity questionnaire
• Teacher survey (nine questions) and psychologist
survey (eight questions) focused on
• If coaching was an effective method to improve teachers’
implementation of function-based intervention
• If coaching was a feasible method to improve teachers’
implementation of function-based intervention
• If coaching was a socially acceptable method for
improving teachers’ implementation of function-based
intervention
TEACHER
RESULTS
TEACHER RESULTS
STUDENT FA
RESULTS
STUDENT
INTERVENTION
RESULTS
TEACHER SURVEY RESULTS
TEACHER SURVEY RESULTS
PSYCHOLOGIST SURVEY RESULTS
PSYCHOLOGIST SURVEY RESULTS
DISCUSSION
• These results support the use coaching to improve
teachers implementation of function-based
interventions
• Functional relationship
• Teachers generalized skills across activities
• All problem behavior decreased during the study;
however, some decreased during teacher baseline
while others did not decrease until accurate
implementation
• A functional relationship exists between accurate
implementation of the function-based intervention
and increases in the primary replacement behaviors
DISCUSSION
• Limitations
• Experimenter was not removed from FBA
process
• Use of long intervals for data recording (30s
or 1 min) could lead to over/under
estimation of behavior
• Need for future replication
• Use of researcher as coach
DISCUSSION
• Implications for practice
• Workshop trainings alone are insufficient ,
coaching was a quick and easy way to improve
teacher implementation
• Schools might be able to use their high scoring
teachers as coaches
• Development of an ongoing FBA team within
schools may help teachers develop and review the
results of FBAs and function-based interventions
COACHING ON TIER ONE
IMPLEMENTATION OF SWPBS
SCHOOL WIDE POSITIVE BEHAVIOR SUPPORTS
School wide positive
behavior supports
(SWPBS) is a
proactive system of
behavior support
that is implemented
in a tiered model in
many elementary,
middle, and high
schools (Lewis &
Sugai, 1999).
INTRODUCTION
• Teachers are expected to implement the
basic components of the SWPBS plan,
including;
a.
b.
c.
d.
teaching students expected behavior,
appropriately reinforcing expected behavior,
discouraging problem behavior, and
accurately recording student data according to
the SWPBS system (Lewis & Sugai, 1999).
INTRODUCTION
• Benefits of SWPBS include reduced office
referrals, improved school attendance,
improved academic achievement, reduced
rates of dropouts, and reduced referrals for
special education services (Cregor, 2008).
• SWPBS plans have been shown to be more
effective that punitive and reactionary
approaches to student problem behavior
(Lewis & Sugai, 1999).
INTRODUCTION
• SWPBS plans are being implemented by an
increasing number of school districts, with the
majority of states having adopted SWPBS
models as part of their systems level school
improvement plans (George & Kincaid, 2008).
• A key component when implementing SWPBS
is the accuracy with which the plan is
implemented.
INTRODUCTION
• When examining training and implementation
protocols for SWPBS, the use of SWPBS coaches
is a core component to used to promote the
success of the SWPBS plan (Horner, Sugai,
Smolkowski, Eber, Nakasato, Todd, & Esperanza,
2009; McIntosh, Filter, Nennett, Ryan, & Sugai,
2009; & Stormont & Reinke, 2012).
• Some literature has made recommendations
towards coaching and SWPBS (Scott &
Martinek; 2006; Stormon & Reinke, 2012;
TACPBIS, 2010).
INTRODUCTION
• At this time, little research has examined the
training and effectiveness of the the coaches.
• Given the increase of SWPBS implementation
in schools, and the importance of coaching
to the success of the SWPBS plan, the
effectiveness of coaching needs to be
evaluated.
INTRODUCTION
• The purpose of this study was to determine the
effects of coaching on teacher’s accuracy of
SWPBS implementation.
• The research questions include:
a) What is the effect of coaching on teachers
accuracy of implementation of a SWPBS plan?;
and
b) Do teachers and coaches find the coaching
process effective, socially, acceptable, cost
effective, and manageable?
METHOD
Setting
• The study took place in elementary school in a
small city in the eastern US
• The school had recently undergone a change
to a more comprehensive SWPBS plan
• They already had a PBS Team that was
providing training to the school through the
use of faculty meetings and analyzing the
data on a monthly basis
• All teachers were implementing the SWPBS
plan when the study began
METHOD
Participants
•The coach was selected from the school’s
SWPBS Team
• The coach was also the instructional coach for the
school
•Four teacher participants were recruited by
volunteering after the entire staff was
presented the opportunity at a faculty meeting
• Teachers had to be licensed teachers who taught
at least one group instructional lesson per day
METHOD
Design
•Multiple baseline across participants design
Data Collection
•Teacher’s implementation of SWPBS by the
percentage of correctly implemented steps
using a SWPBS
•Fidelity data was collected on coaches ability
to accurately implement the coaching
procedure in the form of percentage of
correctly implemented steps using a coaching
fidelity checklist
SWPBS Checklist
IOA:
Y/N
Teacher:
Date:
(IOA Score = ___________________)
________________________
_________________________
Session:
________________________
Recorder: _______________________
Classroom Setup/Antecedent Variables
1. Appropriate Environmental Arrangement: seating is appropriate and allows students to interact with content, there is
visual corresponding with instruction, all students have access to materials, no student completely isolated, school rules are posted
2. Presents appropriate class-wide instruction: Whole group instruction occurring, age appropriate SOL based instruction
General Student Reinforcement
3. Verbally praises class (or individual) for demonstrating appropriate behavior: Specific praise is provided (not
just “good job,” but states specifically what the student(s) did correctly, offered minimally every time a student or group responds). If
teacher asks a series of students a string of the same question, they may specifically verbally praise at the end of the entire string
(counts as one).
4. Provides student specific verbal praise minimally 4 times for everyone 1 correction. Score here once
for every 4:1 ratio opportunity (not every time the teacher provides verbal praise).
Token Reinforcement: When providing a token… (Must do all three each time a Knight Buck is given)
5. Teacher states the specific skill the reward is for
6. Teacher states larger school rule the student was following
7. Teacher provides positive social acknowledgement
Providing Error Correction: When providing error correction in a neutral, “emotion-free” manner: It is the
teacher’s choice to provide an error correct (verses ignoring the behavior), this will be scored when a teacher initiates an error
Providing Error Correction: When providing error correction in a neutral, “emotion-free” manner: It is the
teacher’s choice to provide an error correct (verses ignoring the behavior), this will be scored when a teacher initiates an error
correction, UNLESS the student had a dangerous behavior, then the teacher is expected to provide an error correction
8. Teacher secures student’s attention and signals an error has been made.
9. Teacher re-teaches prerequisite skills or functional behavior alternatives: teacher can restate the rule, reteach
the rule, or teach/reteach an alternative replacement behavior
10. Teacher provides an opportunity for the student to practice the correct response.
11. Teacher tests the student for the correct response.
12. Teacher provides positive reinforcement when the student displays the appropriate behavior: Either
specific verbal praise, a Knight Buck, or a classroom specific reinforcer
If student behavior continues after being provided the error correction in a neutral, “emotion-free” manner
13. If student behavior continues, teacher makes an environmental modification: (Ex: change in seat/materials)
14. If student behavior continues, teacher presents the individual student with a loss of privileges: (e.g.,
recess without friends, loss of Knight Buck Store – Not Loss of Knight Bucks, Work Alone, Lunch without Friends, or Loss of a
Classroom Specific Privilege).
15. If student behavior continues, teacher makes a parent contact: Notification of a parent contact to be made later
is acceptable (either a note or phone call home or an in-person contact)
16. If the student behavior continues, teacher makes an office referral.
Total Correct Steps:
Percent Accurate:
+ = Step performed correctly
- = Step not performed or performed incorrectly
METHOD
Intervention
• The coach provided two side-by-side
coaching to teacher participants, including:
• A pre-coaching meeting
• An individual coaching session
• A feedback meeting
• Each participant got two coaching sessions
METHOD
General Procedures
• Coach training
• The coach was provided with the SWPBS plan and a
written description of the side-by-side coaching
process and the teacher data sheet
• The researcher and the coach met to verbally review
the information
• Then, the researcher and the coach watched videos
of group classroom instruction. Both scored the
teachers using the data sheet, and then compared
data (80% minimum was acceptable)
• Finally, the coach modeled how to coach the
teachers on the video
METHOD
General Procedures
• Baseline
• No training was provided to teachers
• Teachers were observed during a scheduled
10 min observation daily (observation times
were held constant from day to day)
METHOD
General Procedures
• Coaching
• The coaching intervention consisted of three
components
a) one pre-coaching meeting,
b) at least one side-by side coaching session,
and
c) at least one feedback meeting
METHOD
General Procedures
• Pre Coaching Meeting:
• Coach provided the teacher with specific
feedback
• The coach and the teacher planned the lesson for
the side-by-side coaching session
• The coach modeled selected strategies for
targeted skills (such as the steps for providing
reinforcement and error correction to students)
• The coach supported the teacher by asking guiding
questions and providing feedback
METHOD
General Procedures
• Side by side coaching:
a) Model appropriate reinforcement of student behavior
b) Model the consequences for rule-violating behavior,
including error correction
c) Immediately after modeling each skill, prompt the teacher
to demonstrate the same skill;
d) Model ways to engage students in active student
responding;
e) Give specific praise to the teacher at least once per skill;
f) Provide non-evaluative corrective feedback when needed
g) Provide another opportunity for the teacher to implement
the skill after error correction
METHOD
General Procedures
• Post Coaching Meeting:
• Reinforce steps correctly implemented
• Review steps where errors were made
METHOD
Inter-observer Agreement (IOA)
• A second observer simultaneously observed
at least 30% of sessions
• Agreement was calculated using interval by
interval method
Number of agreements
Number of agreements + disagreements
x 100
METHOD
Procedural Fidelity
• The researcher used a checklist to collect
data on the extent to which the coach
adhered to the prescribed steps in the
coaching process
• Procedural fidelity was calculated by dividing
the number of steps the researcher correctly
performs by total number of steps
METHOD
• IOA was conducted in 41.4% of sessions and
averaged 91.0% (range 80% - 100%)
• Fidelity was conducted for in 33.3% of sessions
and averaged 100%
METHOD
Social Validity
• Collected to assess the impact and feasibility
of the coaching
• Teachers were given a written questionnaire
which included open-ended and closed (i.e.,
Likert) items evaluating each individual
enhancement and training component
• Coaches were provided a questionnaire
regarding the impact and feasibility of
coaching
RESULTS
TEACHER SURVEY RESULTS
COACH SURVEY RESULTS
DISCUSSION
• There is a functional relationship between
coaching and teacher’s improved
performance on implementation checklists
• Data remained variable for most teachers
throughout the study
DISCUSSION
Limitations
• Overlapping data points between
baseline and intervention
• Variability throughout study
• Need for future replication
DISCUSSION
Implications for practice
• Inservice trainings continue to be insufficient
to train teachers to implement SWPBS
procedures
• Instructional coaches may be able to serve
as behavioral coaches to train teachers to
implement SWPBS plans
PRACTICAL CONCERNS
• How do districts identify candidates for
coaches?
• Is side-by-side coaching the most effective
coaching procedure?
• Would bug-in-ear or supervisory coaching be more
effective?
• We still need more data on coaching teachers
to implement tier 1 and tier 3 interventions, and
data is needed on coaching to increase
fidelity of implementation of tier 2 interventions.
Questions…
• Contact Information
• Keri Bethune
• bethunks@jmu.edu