What Is “It” and How Do
We Make “It” Happen
SPDG Webinar
March 24, 2011
Karen Blase, PhD
Dean L. Fixsen, PhD
Melissa Van Dyke, LCSW
Michelle Duda, PhD
Frank Porter Graham Child Development Institute
University of North Carolina at Chapel Hill
1
Searching for “It”



Research findings, materials, manuals, and
journal articles do not necessarily provide
clarity around core intervention elements
Current and new evidence-based practices,
frameworks, programs will have a range of
operational specificity
Developing clarity around the “it” is critical
2
Innovation Fluency

Definition: Innovation Fluency refers
to the degree to which we know the
innovation with respect to:



Evidence
Program and Practice Features
Implementation Requirements
3
Innovation Fluency

Definition: Innovation Fluency refers to
the degree to which we know the
innovation with respect to:



Evidence
Program and Practice Features
Implementation Requirements
Implementation Pre-Requisites
4
Implementation Pre-Requisites

After you
 Have chosen based on student needs
 Looked for “best evidence” to address the
need
An Evidence-Based Practice or Program
An Evidence-Informed Initiative or Framework
Systems Change and Its Elements

Then it’s time to:
 Clearly identify and operationalize the
elements
5
What is “it”?

Operationalize
Part of Speech: verb Definition: to define a concept
or variable so that it can be measured or
expressed quantitatively
Webster's New Millennium™ Dictionary of English, Preview Edition (v 0.9.7)
Copyright © 2003-2008 Lexico Publishing Group, LLC


The “it” must be operationalized
whether it is:
»
»
»
An Evidence-Based Practice or Program
A Best Practice Initiative or New Framework
A Systems Change Initiative
Practice Profiles
»
Help Operationalize Practice, Program, and Systems
Features
6
Practice Profile




Defining “it” Through the Development and
Use of Practice Profiles
Guiding Principles identified
Critical Components articulated
For each critical component:

Identified gold standard

Identified acceptable variations in practice

Identified ineffective practices and
undesirable practices
Hall and Hord, 2010 Implementing Change: Patterns, Principles, and Potholes (3rd
Edition) and Adapted from work of the Iowa Area Education Agency
7
????

Have you ever developed or
helped to develop a Practice
Profile or Innovation
Configuration?
 Vote Now:
» Yes
» No
8
Practice Profiles:
Pay Now or Pay Later

Identifies Critical Components




For each Critical Component:



Identified “gold standard” activities
Identified acceptable variations in practice
Identified ineffective practices and undesirable practices
Your Implementation Support
Capacity
Building

Guiding Principles
Critical Components Match the Guiding Principles
Core Activities to Achieve the Critical Components
»
»
»
»
Identify and Support Implementation Team
Provide Conceptual Overview and Rationales
Provide Resources, Worksheets, Templates
Facilitate Consensus Building
9
But Where Do We Find
Resources for Building
Practice Profiles?
10
Resources for Building
Practice Profiles
•
•
•
•
•
National Centers
Experts in Your State
National Purveyors
Manuals and Materials
Implementing Districts and
Schools
• Other States
• Consensus Building in Your State
11
Example

Problem-Solving Practice Profiles in
an RtI Framework
RESOURCE - Professional Practices in Problem
Solving: Benchmarks and Innovation
Configurations
~ Iowa Area Education Agency Directors of Special
Education, 1994
12
Practice Profile




Defining “it” Through the Development and
Use of Practice Profiles
Guiding Principles identified
Critical Components articulated
For each critical component:

Identified gold standard

Identified acceptable variations in practice

Identified ineffective practices and
undesirable practices
Hall and Hord, 2010 Implementing Change: Patterns, Principles, and Potholes (3rd
Edition) and Adapted from work of the Iowa Area Education Agency
13
A Sampler of
Problem Solving Guiding Principles



Problems affecting student performance do not exist
exclusively within the makeup of learners but occur as a
result of interactions between learner characteristics and
the educational setting
The effectiveness of a solution cannot be determined
prior to its implementation. Therefore solutions must be
implemented, monitored, reviewed, and changed as
necessary
A problem is not defined as the difference between the
learner’s potential and achievement, but as the
discrepancy between the demands of the educational
setting and the learner’s performance in the setting.
Professional Practices in Problem Solving: Benchmarks and Innovation Configurations
Iowa Area Education Agency Directors of Special Education, 1994
14
Improved outcomes for
students
Performance Assessment
(fidelity measurement)
Systems
Intervention
Coaching
Training
Selection
Adaptive
Integrated &
Compensatory
Technical
Facilitative
Administration
Decision Support
Data Systems
Leadership
© Fixsen & Blase, 2008
15
Given These Guiding Principles….



Problems affecting student performance do not exist exclusively within
the makeup of learners but occur as a result of interactions between
learner characteristics and the educational setting
The effectiveness of a solution cannot be determined prior to its
implementation. Therefore solutions must be implemented, monitored,
reviewed, and changed as necessary
A problem is not defined as the difference between the learner’s
potential and achievement, but as the discrepancy between the
demands of the educational setting and the learner’s performance in the
setting.
What Attributes, Attitudes or Skills Might
Be Important to Select for as Teachers
and Building Staff are Recruited and
Hired?
Professional Practices in Problem Solving: Benchmarks and Innovation Configurations
Iowa Area Education Agency Directors of Special Education, 1994
16
Practice Profiles


Each Critical Component is a heading
Each level of implementation specifies
the activities necessary to
operationalize that Critical Component
Critical Component
Critical Component 1:
Description
Ideal
Implementation
Acceptable
Variation
Unacceptable
Variation
Drastic
Mutation
Unacceptable
Variation
Description of
implementer
behavior
Hall and Hord, 2010, Implementing Change: Patterns, Principles, and Potholes (3rd Edition) and
Adapted from work of the Iowa Area Education Agency
17
Professional Problem Solving
9 Critical Components
Critical Component
Ideal
Implementation
Critical Component 1:
Description
Description of
implementer
behavior





Parent Involvement
Problem Statement
Systematic Data
Collection
Problem Analysis
Goal Development
Acceptable
Variation




Unacceptable
Variation
Unacceptable
Variation
Intervention Plan
Development
Intervention Plan
Implementation
Progress Monitoring
Decision Making
Professional Practices in Problem Solving: Benchmarks and Innovation Configurations
Iowa Area Education Agency Directors of Special Education, 1994
18
Professional Problem Solving
9 Critical Components
Critical Component
Ideal
Implementation
Critical Component 1:
Description
Description of
implementer
behavior





Parent Involvement
Problem Statement
Systematic Data
Collection
Problem Analysis
Goal Development
Acceptable
Variation




Unacceptable
Variation
Unacceptable
Variation
Intervention Plan
Development
Intervention Plan
Implementation
Progress Monitoring
Decision Making
Professional Practices in Problem Solving: Benchmarks and Innovation Configurations
Iowa Area Education Agency Directors of Special Education, 1994
19
Professional Problem Solving
Parent Involvement as a Critical
Component
Critical Component
Critical Component 1:
Description of Parent
Involvement and
Rationales for its
importance
Ideal
Implementation
Parents are
informed at all
decision –
making points
and invited to
participate by
phone, letter, or
email. Parents
“choose” to
participate.
Acceptable
Variation
Unacceptable
Variation
Unacceptable
Variation
Parents are
informed at
all decision
–making
points and
invited to
participate
by phone,
letter, or
email.
Parents
“choose”
not to
participate.
Parents are
informed of
decisions at
all decision –
making
points. But
parents are
not invited to
participate.
Parents are
not informed
or invited to
participate at
decisionmaking
points.
Professional Practices in Problem Solving: Benchmarks and Innovation Configurations
Iowa Area Education Agency Directors of Special Education, 1994
20
Professional Problem Solving
Parent Involvement – Critical
Components
Critical Component
Critical Component:
Parent Involvement
Ideal and/or
Acceptable
Knowledge,
Skills, and
Abilities
Parents are
informed at all
decision –making
points and invited
to participate by
phone, letter, or
email. Parents
“choose” to
participate.
We know
what is
necessary
to put “it”
in place.
Driver
Analysis
Areas of
Impact
(outcomes)
This is how We can prove
that we’ve
we ensure
that “it” is in “got it”!
place!
21
Professional Problem Solving
Parent Involvement – Critical
Components
Critical Component
Ideal and/or
Acceptable
Critical Component: Parents are
Parent Involvement informed at all
decision –making
points and invited
to participate by
phone, letter, or
email. Parents
“choose” to
participate.
Knowledge,
Skills, and
Abilities
Driver
Analysis
Areas of
Impact
(outcomes)
- Knows schedule
of meetings
-How to welcome
parents
- How to solicit
parent input
-How to thank
parents for
attending
- How to ask
parents for
feedback on the
convenience of
times and location.
Ensuring
that “it” is
in place?
% of meetings
with Parent
attendance.
% of meetings
with Parent
participation
Parent
satisfaction
ratings
22
Professional Problem Solving
Parent Involvement – Critical
Components
Critical Component
Ideal and/or
Acceptable
Critical Component: Parents are
Parent Involvement informed at all
decision –making
points and invited
to participate by
phone, letter, or
email. Parents
“choose” to
participate.
Knowledge,
Skills, and
Abilities
Driver
Analysis
Areas of
Impact
(outcomes)
- Knows schedule
of meetings
-How to welcome
parents
- How to solicit
parent input
-How to thank
parents for
attending
- How to ask
parents for
feedback on the
convenience of
times and location.
Ensuring
that “it” is
in place?
% of meetings
with Parent
attendance.
% of meetings
with Parent
participation
Parent
satisfaction
ratings
23
Exploration and Periodic Assessment of Implementation Drivers:
Discussion Tool
Parent Involvement
Competency Drivers for Ideal/Acceptable: Parents are informed - Knows schedule of meetings
Teachers Engaged in
Problem Solving
-How to welcome parents
at all decision-making points and
invited to participate by phone, letter, or -How to solicit parent input
email. Parents “choose” to
-How to thank parents for attending
participate.
- How to ask parents for feedback on
the convenience of times and location.
Selection
Training
Coaching
Fidelity/Performance
Monitoring System
24
Exploration and Periodic Assessment of Implementation Drivers:
Discussion Tool
Parent Involvement
Competency Drivers for Ideal/Acceptable: Parents are informed - Knows schedule of meetings
Teachers Engaged in
Problem Solving
-How to welcome parents
at all decision-making points and
invited to participate by phone, letter, or -How to solicit parent input
email. Parents “choose” to
-How to thank parents for attending
participate.
- How to ask parents for feedback on
the convenience of times and location.
Selection
Training
Coaching
Fidelity/Performance
Monitoring System
25
General Assessment of Level of
“Intensity” of Use of the Competency
Drivers
Current Skills
New Skills
Low
Medium
Medium
High
Current Context
New Context
Ideal/Acceptable: Parents are informed at all decision –
making points and invited to participate by phone, letter, or
email. Parents “choose” to participate.
26
Professional Problem Solving
Parent Involvement as a Critical
Components
Critical Component
Critical Component:
Parent Involvement
Ideal and/or
Acceptable
Knowledge,
Skills, and
Abilities
Parents are
informed at all
decision –making
points and invited
to participate by
phone, letter, or
email. Parents
“choose” to
participate.
- Schedule of
meetings
-How to
welcome
parents
- How to solicit
parent input
-How to thank
parents for
attending
- How to ask
parents for
feedback on the
convenience of
times and
location.
Driver
Analysis
How to
ensure that
“it” is in
place?
….we’ve
“got it”!!
Areas of
Impact
(outcomes)
% of meetings
with Parent
attendance
% of meetings
with Parent
participation
Parent
satisfaction
ratings
27
General Assessment of Level of
“Intensity” of Use of the Competency
Drivers
A – High
Let’s Vote
B – Medium C – Low
Current Skills
New Skills
Low
Medium
Medium
High
Current Context
New Context
Ideal/Acceptable: Parents are informed at all decision –
making points and invited to participate by phone, letter, or
email. Parents “choose” to participate.
28
Exploration and Periodic Assessment of Implementation Drivers:
Discussion Tool
Critical Components of Problem-Solving –
Parent Involvement
Competency
Drivers for
Teachers
Engaged in
Problem Solving
Selection
Ideal/Acceptable: Parents are informed - Knows schedule of meetings
-How to welcome parents
at all decision-making points and
invited to participate by phone, letter,
-How to solicit parent input
or email. Parents “choose” to
-How to thank parents for attending
participate.
- How to ask parents for feedback on the
convenience of times and location.
What “unteachables” should be assessed? How?
What behavior rehearsals might be developed?
Training
What behavior rehearsals might be used?
Coaching
Who might provide feedback after decision-making meetings? How often?
Fidelity/
Performance
Monitoring
System
What might you measure to see if teachers are engaging in the skills and activities to
bring parents to the table in a meaningful way? Who? When? How often?
29
Exploration and Periodic Assessment of Implementation Drivers:
Discussion Tool
Parent Involvement
Competency Drivers for Ideal/Acceptable: Parents are informed - Knows schedule of meetings
Teachers Engaged in
Problem Solving
-How to welcome parents
at all decision-making points and
invited to participate by phone, letter, or -How to solicit parent input
email. Parents “choose” to
-How to thank parents for attending
participate.
- How to ask parents for feedback on
the convenience of times and location.
Selection
Practice parts of a decision-making meeting to assess the “how to” elements.
Provide a vignette of a parent who is challenging to work with and complains
that he/she is not heard. Be clear about the importance of parent involvement
Training
Practice hosting a meeting with parents present a) Receptive and engaged b) Less
receptive and engaged
Coaching
Assigned “coach” to sit in on at least 3 decision-making meetings a year for each
teacher with verbal and written feedback immediately after each meeting on
fidelity items below and for support and problem-solving
Fidelity/Performance
Monitoring System
Measures: % of parents notified two weeks in advance % of meetings for which
parents attended; % of meetings in which a) teachers welcomed parents, b)
solicited parent input at least twice per meeting, c) thanked parents for
attending, d) asked for feedback on time and location of the meeting.
30
Professional Problem Solving
9 Critical Components
Critical Component
Ideal/Acceptable
Critical Component 1:
Description
Description of
Implementer
Behavior





Parent Involvement
Problem Statement
Systematic Data
Collection
Problem Analysis
Goal Development
Knowledge,
Skills, and
Abilities




Driver
Analysis
Areas of
Impact
(outcomes)
Intervention Plan
Development
Intervention Plan
Implementation
Progress Monitoring
Decision Making
31
Exploration and Periodic Assessment of Implementation Drivers:
Discussion Tool
Critical Components of Problem-Solving
Competency Drivers
for Teachers
Engaged in Problem
Solving
Selection
Parent
Involvement
Problem
Statement
Systematic
Data
Collection
Problem
Analysis
Goal
Development
Training
Coaching
Fidelity/Performance
Monitoring System
32
Importance Ratings of Implementation Drivers:
1 = Not at All Important to 7 = Very Important
Critical Components of Problem-Solving
Competency Drivers
for Teachers
Engaged in Problem
Solving
Selection
Parent
Involvement
Problem
Statement
Systematic
Data
Collection
Problem
Analysis
Goal
Development
7
3
6
6
4
4
5
7
7
7
6
5
7
7
7
4
5
7
7
5
Training
Coaching
Fidelity/Performance
Monitoring System
33
Exploration and Periodic Assessment of Implementation Drivers:
Discussion Tool
Critical Components of Problem-Solving
Organization
Drivers
Parent
Involvement
Problem
Statement
Systematic
Data
Collection
Problem
Analysis
Goal
Development
Outcome Data
Facilitative
Admin at Local
School Level
Facilitative
Admin at District
Level
Systems Change
at State Level
2011,SISEP,
34
Exploration and Periodic Assessment of Implementation Drivers:
Discussion Tool
Critical Components of Problem-Solving
Leadership
Challenges
Parent
Involvement
Problem
Statement
Systematic
Data
Collection
Problem
Analysis
Goal
Development
What challenges do
you expect? And
how might they be
avoided or
minimized?
What challenges are
on the table now?
Who needs to be
part of the
solutions?
What strategies
seem appropriate
given the nature of
the challenge?
2011, SISEP
35
Improvement Over Time
Competency
Drivers
• Selection
• Training
• Coaching
• Fidelity
PDSA
Organization
Drivers
• Facilitative Admin
• Systems Intervention
• Decision-Support Data
Systems
PDSA
Practice Profile
Improvement
36
Things to Think About


Think about your SPDG effort and your
involvement and guidance at the State, District,
and School levels.
Currently, our SPDG work is well
operationalized ?

….At the Classroom level
»

…At the School Level
»

_Strongly Agree _Agree __Disagree __Strongly Disagree
_Strongly Agree _Agree __Disagree __Strongly Disagree
….At the District Level
»
_Strongly Agree _Agree __Disagree __Strongly Disagree
37
Things to Think About


Think about your SPDG effort and your
involvement and guidance at the State, District,
and School levels.
Would developing Practice Profiles be helpful?
Vote - __Yes __No



Benefits?
Challenges?
What might be some “next right steps” in
further operationalizing your SPDG program,
practice, innovation, or framework?
38
Summary




We are more likely to be successful if we “get it”
We “get it” when we have a Practice Profile
»
Guiding Principles
»
Critical Components
»
Activities – Ideal, Acceptable, Not acceptable
When we “get it” we can use the Implementation
Drivers and ensure that “it” shows up and is
effective.
By applying the Implementation Drivers to the
“it” we refine and improve the Practice Profile &
make strategic use of the Drivers
39