Implementation Science
Lessons Learned…
“It's a wise man
who profits by his
own experience,
but it's a good
deal wiser one
who lets the
rattlesnake bite
the other fellow”
Josh Billings
Stages of Implementation
Exploration/Ad
option
Development
Commitment and
acquiring more
information about
the innovation
Installation
Creating the
systems and
materials to
prepare for
implementation
Initial
Implementation
Provide Significant
Support to
Implementers;
implementer is
focused on shortterm day to day use
of the innovation
with little time for
reflection
Full
Implementation
Use of the
innovation is
stabilized. Few if
any changes are
being made in ongoing use
Elaboration
(Innovation)
Embedding within
Standard Practice
and is able to find
ways to make the
innovation more
efficient
Continuous
Regeneration
(Sustainability)
Improvements:
Increase
Efficiency and
Effectiveness
Exploration/Ad
option
Development
Commitment and
acquiring more
information about
the innovation
Exploration/Adoption
• Assess capacity, resources, and buy-in
• Begin developing awareness and support for
adoption (parents, staff, central office)
• Explain the risks involved in adopting (“What
will be different for us as a result?”)
• Identify staff who will be comfortable
managing the risks (change in practice,
ambiguity) for leadership roles
Exploration/Ad
option
Development
Commitment and
acquiring more
information about
the innovation
Exploration/Adoption
• Assess whether you will have access to
experts (trainers, coaches, colleagues) who
know the innovation well
• Build consensus and commitment for
adoption
More about developing consensus is coming
up next!
Installation
Creating the
systems and
materials to
prepare for
implementation
Installation
• Create “space” for the work to happen
• Begin recruiting and hiring staff
• Conduct training
Installation
Creating the
systems and
materials to
prepare for
implementation
Installation
• Develop the infrastructures to support
implementation
– On-going training
– Coaching and supervision
– Staff and fidelity evaluations
– Outcome evaluations
– How will the work get done? Committees, regular
meeting times, how often, how long
Initial
Implementation
Provide Significant
Support to
Implementers;
implementer is
focused on shortterm day to day
use of the
innovation with
little time for
reflection
Initial Implementation
• “You cannot do something well without doing it
poorly first!”
• Survive the awkward stage
– Learn from mistakes-implementation will feel messy!
– Continue “buy-in” efforts-people will ask, “What the heck
did I agree to do?!”
– Manage expectations- keep people focused on what is
expected for implementation
Initial
Implementation
Provide Significant
Support to
Implementers;
implementer is
focused on shortterm day to day
use of the
innovation with
little time for
reflection
Initial Implementation
• Change practices
• Put the components you have been planning
(infrastructures to implement) in place
• Change organizational and community
structures and culture
• Manage the change process- “Nope, I change
my mind!”
• Overcome fear and inertia- “I feel like a failure”
Full
Implementation
Use of the
innovation is
stabilized. Few if
any changes are
being made in ongoing use
Full Implementation
• Implementers are in a routine while
implementing the innovation
– Comfortable with the materials and components
of the innovation
• Implementation is not awkward
• Few if any changes are made in the on-going
use of the innovation
Elaboration
(Innovation)
Embedding within
Standard Practice
and is able to find
ways to make the
innovation more
efficient
Elaboration (Innovation)
• Replicate school implementations within the
district based on initial implementation
– Select a model demonstration site(s) and then
learn from implementation efforts
– Let early adopters make the mistakes first
– Continue to refine systems that support how data
are analyzed and how decisions are made
Elaboration
(Innovation)
Embedding within
Standard Practice
and is able to find
ways to make the
innovation more
efficient
Elaboration (Innovation)
• Ensure feedback loops are in place between
the implementation support systems
– Building leadership team
Planning Team
Implementation
• All the components of the innovation are in
place and fully functioning
Continuous
Regeneration
(Sustainability)
Improvements:
Increase
Efficiency and
Effectiveness
Continuous Regeneration
(Sustainability)
• It is important to first, do it right (fidelity),
then do it better (innovate)
• Continuous Regeneration is doing it better
– More efficient with implementation
• It is important to have feedback loops in place
to review data to determine if alterations to
the innovation are still positively impacting
student outcomes
Continuous
Regeneration
(Sustainability)
Improvements:
Increase
Efficiency and
Effectiveness
Continuous Regeneration
(Sustainability)
• Consultation with:
– Experts, trainers, coaches, colleagues
• Systems adoption within the district
– New staff orientation
– Should get easier over time
These stages are recursive: Setbacks at one stage
moves us back to the previous stage
Ignore
File
Grievance
Snow Day!
Data
Vote
coach
off
Change
Practice
Violate
Norms
Have a
“AHA!”
Dominate
conversation
Go to Book
Study
Late for
meeting
Ignore emails
Go to a
PLC
Apply for
PBIS
Prep for
Meeting
New State
Initiative
“We
already do
that.”
New District
Initiative
Today is a
book
study?
Attend
District PD
Be on
time
Stages of Implementation
Establishing consensus is a
recursive process. Managing the
change process requires the
leader to monitor the pulse of
staff
Exploration/Ad
option
Development
Commitment and
acquiring more
information about
the innovation
Installation
Creating the
systems and
materials to
prepare for
implementation
Principals should
anticipate staff who will
perceive the change as
first or second order
before Exploration. When
Exploration begins, it will
become apparent
Initial
Implementation
Provide Significant
Support to
Implementers;
implementer is
focused on shortterm day to day use
of the innovation
with little time for
reflection
Full
Implementation
Use of the
innovation is
stabilized. Few if
any changes are
being made in ongoing use
Elaboration
Embedding within
Standard Practice
and is able to find
ways to make the
innovation more
efficient
Continuous
Regeneration
Improvements:
Increase
Efficiency and
Effectiveness
Understanding People’s Concerns
about Implementation
Leading Change is Hard Work!
Implementation Takes Time
•Exploration (Sustainability)
•Installation (Sustainability)
•Initial Implementation
•Full Implementation
(Effectiveness &
Sustainability)
2–4 Years
Fixsen, Naoom, Blase, Friedman, & Wallace, 2005
Change is Cyclical
and Iterative-Exploration
Installation
Implementation
7. Special Ed Referral and Evaluation
Report
6. Teaming/Data-Based Decision Making:
Individual Problem Solving
5. Teaming/Data-Based Decision Making:
Group Interventions
4. Progress Monitoring
3. Interventions with Fidelity
2. Core Instruction with Fidelity
1. Screening
Competency Drivers
Organization Drivers
Leadership
Implementation Science
Know Your Driver
Implementation Drivers
Sustainable Student Outcomes
Performance Assessment
(Fidelity)
Systems Intervention
Coaching
Training
Selection
Facilitative
Administration
Integrated &
Compensatory
Decision Support Data
System
Leadership
Adaptive
Technical
© Fixsen & Blase, 2008
Sustainable Student Outcomes
Effective Implementation of Evidence
Based Practice (e.g. RtI)
Selection
Integrated &
Compensatory
Leadership
© Fixsen & Blase, 2008
Sample Interview Questions for Elementary
Principal Position
• If you were to walk into a classroom with excellent
instruction, what would you expect to see?
• If you were to walk into a classroom with poor
instruction, what would you do about it?
• What are the common features of Positive Behavior
Intervention Support and Response to Intervention
systems? What is your experience coordinating,
leading, or performing within these systems?
• How would you teach reading to a 1st grader?
Interview Data Activity
Directions: Please review the data provided for the following six (6)
students. At the bottom of each page, please complete the instructional
recommendation section and include your rationale.
Selection gone wrong
Sustainable Student Outcomes
Effective Implementation of
Evidence Based Practice (e.g. RtI)
Training
Selection
Integrated and
Compensatory
Leadership
© Fixsen & Blase, 2008
Implementation Driver: Training
Purpose: Train staff in specific skills needed to implement the
EBP
• Created a PD plan that is developed collaboratively across
departments
• PD continually recycles key elements (i.e., core) to address drift
and train new staff
• Data from fidelity measures (SET, EBIS Group Planning Form,
etc) used to inform PD plan
• Data from outcome measures (DIBELS, Maze, OAKS) used to
inform PD plan
Implementation Driver: Training
• New staff receive an extra five days of training and
extra coaching throughout the year that focuses on
EBIS components
• All PBIS teams get a full day each year for training,
coaching, and time to plan
• All principals trained to conduct walkthroughs and
instructional rounds for coaching and instructional
leadership
Sustainable Student Outcomes
Effective Implementation of Evidence
Based Practice (e.g. RtI)
Coaching
Training
Selection
Integrated and
Compensatory
Leadership
© Fixsen & Blase, 2008
Implementation Driver: Coaching
Purpose: Ensure transfer from training to practice
• A “Coaching Delivery Plan” is developed
• Lead Person is designated and supported
• Coaches are fluent with RTI and PBIS
• Feedback from multiple sources is collected, provided
to practitioners and used to determine areas in need
of more coaching and training
Implementation Driver: Coaching
Joyce and
Showers, 2002
TRAINING
COMPONENTS
OUTCOMES
(% of Participants)
Demonstrate
Knowledge
Demonstrate
Skill
Use in the Classroom
Theory and
Discussion
10%
5%
0%
+ Demonstration in
Training
30%
20%
0%
+ Practice &
Feedback in
Training
60%
60%
5%
95%
95%
95%
+ Coaching in
Classroom
Implementation Driver: Coaching
District level coaching
– Attend grade level EBIS meetings
– Provide feedback using Coaching Rubric
– Facilitate monthly meetings with school
literacy coaches, counselors and principals
– Update EBIS Handbook annually
Implementation Driver: Coaching
School level coaching:
– Principal lead walkthroughs/Instructional rounds
– Literacy specialists coach teachers and
instructional assistants on instructional practices
and literacy across content areas
– Counselors and PBIS team members coach
teachers and instructional assistants on behavior
Sustainable Student Outcomes
Effective Implementation of Evidence
Based Practice (e.g. RtI)
Performance Assessment
(Fidelity)
Coaching
Training
Selection
Coaching
Training
Integrated and
Compensatory
Selection
Leadership
© Fixsen & Blase, 2008
Implementation Driver:
Performance Assessment
Purpose: Ensure EBP is Implemented with fidelity
• Performance of practices, teams and
coaches are assessed at least
annually. (Quarterly until fidelity is
reached.)
• Assessment measures include direct
observation, and review of fidelity and
outcome measures.
• Feedback is provided and successes
are celebrated
Implementation Driver:
Performance Assessment
Data
Sources:Improvement Plan goals
 Continuous
Elementary
School
Literacy Plans
• and
Fidelity
Assessment:
SET,
are
directly linked
to these
measures.
Benchmarks
of Quality,
EBIS
Group
Planning Form, Coaching Rubric, Walk
Through Forms
• Outcome measures: DIBELS, Maze,
OAKS
• Annual EBIS Systems Assessment:
DSSP
Sustainable Student Outcomes
Effective Implementation of Evidence Based Practice (e.g.
RtI)
Performance Assessment
(Fidelity)
Coaching
Training
Selection
Integrated &
Compensatory
Leadership
Decision Support Data
System
© Fixsen & Blase, 2008
Implementation Driver:
Decision Support Data System
Purpose: Employ data systems that guide MTI and RTI practices in order to
increase student achievement
Implementation Fidelity Assessments
•
•
•
•
Data from SET (School-wide Evaluation Tool)
Coaching Rubric, walk through form, etc.
Elementary EBIS Team Planning Survey
Annual EBIS Systems Assessment: DSSP
Implementation Driver:
Decision Support Data System
Student Performance Measures
– Outcomes measures: OAKS, ODRs, DIBELS
– Universal Screening: DIBELS, OAKS, ODRs, Maze
– Progress Monitoring: Curriculum Based Measures
(AIMSweb, DIBELS)
– Diagnostic Measures: Grade +, Phonics Screener
Sustainable Student Outcomes
Effective Implementation of
Evidence Based Practice (e.g. RTI)
Performance Assessment
(Fidelity)
Coaching
Facilitative (Barrier Busting)
Administration
Training
Selection
Integrated and
Compensatory
Leadership
Decision Support Data
System
© Fixsen & Blase, 2008
Implementation Driver:
Facilitative Administration
Purpose: To ensure that administrators provide staff with
support to implement EBP AND ensure that staff are
implementing with fidelity.
Support
•Schedule
•Resources
•Training
•Procedures
Monitoring
•Fidelity checks
•Principal Walk Throughs
•Staff Evaluation process
Sustainable Student Outcomes
Effective Implementation of Evidence Based
Practice (e.g. RtI)
Performance Assessment
(Fidelity)
Systems
Intervention
Coaching
Facilitative
Administration
Training
Selection
Integrated and
Compensatory
Leadership
Decision Support Data
System
© Fixsen & Blase, 2008
Implementation Driver: Systems
Intervention
– Engage and grow “champions”
– District leadership actively participates with state
education department agencies and legislature
– Multiple district and school level teams engage in
reciprocal communication (PEP – PIP cycles)
– Positive feedback provided to schools on achieving
CIP Goals
Sustainable Student Outcomes
Effective Implementation of Evidence Based
Practice (e.g. RtI)
Performance Assessment
(Fidelity)
Coaching
Systems
Intervention
Training
Selection
Facilitative
Administration
Integrated and
Compensatory
Leadership
Adaptive
Technical
Decision Support Data
System
© Fixsen & Blase, 2008
Adaptive vs Technical Leadership
VS
Lessons Re-Learned… OR
What SISEP Has Taught Us!
1.
2.
3.
4.
5.
6.
Pick the right people
Teach them what to do
Coach them
Make sure they’re doing it right
Determine if the system is working
Keep lines of communication open and support
your people
7. Get help from everyone you can
8. Celebrate success!
“About the most originality that any writer can hope to
achieve honestly is to steal with good judgment”.
Josh Billings