Lessons Learned from
OSEP’s Model Demos:
Creating Change to
Promote Children’s
Success
1
Lessons Learned
from Model Demonstration Projects
Mary Wagner, Ph.D., Principal Investigator
Phyl Levine, Ph.D., Director
SRI International
OSEP Project Directors’ Conference
July 19, 2010
Washington, DC
Model Demonstration Project Cohorts
• Cohort 1: Progress monitoring/RtI for struggling readers K-5
– Lehigh University, University of Pittsburgh
– University of of Minnesota, Minneapolis Public Schools
– University of Oregon
• Cohort 2: Tertiary behavior interventions K-8
– University of Kansas, Illinois PBIS Network
– University of Oregon
– University of Washington
• Cohort 3: Early childhood language development
– Orelena Hawks Puckett Institute
– University of Kansas
– Vanderbilt University, Florida State University
• Cohort 4: Writing proficiency in high school
– University of Kansas
– University of Hawaii-Manoa
The Model Demonstration Coordination
Center (MDCC) was launched in 2005 to:
• Identify characteristics of an effective
implementation/evaluation/refinement process
that moves a practice from early testing to
being ready for wider adoption
• Coordinate the evaluation of each cohort of
Model Demonstration Project’s (MDPs) and
synthesize and analyze their findings to
maximize the strength of evidence produced
MDCC activities:
• Facilitate a collaborative partnership with the MDPs
to create opportunities for learning
• Contribute and/or broker methodological expertise
• Develop a data system to assemble
MPD-provided data
• Conduct cross-MDP and cross-cohort
analyses
• Communicate implementation and
evaluation findings to promote a
deeper understanding of the model
demonstration process and its results
Framework
for
understanding
model
implementation
and outcomes
Characteristics of the “source” in progress
monitoring models*
Source
The Model
Core intervention components
Measurement model and procedures
How data are used to:
▪ Monitor student progress
▪ Adapt instruction for struggling
readers
▪ Inform eligibility determination for
special education and development
and monitoring of IE P goals
*Using cohort 1 as an example
Characteristics of progress monitoring
“purveyors”
Purveyor
The MDP Grantee
Core implementation components
▪ Introducing model to schools/teachers
▪ Formal professional development
▪ Ongoing coaching
▪ MDP staff selection and staffing
strategy
Characteristics of the progress monitoring
“destination” organizations
Destination
Participating Schools and Teachers
Characteristics of participating schools,
classrooms, and teachers
Implementation outcomes―changes in:
▪ Staff knowledge, attitudes, and actions
▪ Organizational structures, processes
and culture
▪ External relationships
Sustained implementation
Influences on destination organizations
and their implementation
• District-level (e.g., other initiatives,
superintendent turnover, history with grantee
university)
• State level (e.g., RtI initiatives, testing
requirements)
• Other factors (e.g., union power/influence)
Intervention outcomes of progress monitoring
models Student Outcomes
Increased:
Oral reading fluency
SAT-10 scores
Percentage reaching
benchmarks
Percentage proficient on
state tests
Systems Outcomes
Fewer students referred
to special education
for reading disabilities
Increased use of student
PM data in determining special education
eligibility and setting
and monitoring IEP
goals
Feedback on progress monitoring
model implementation and
effectiveness
• Reflect on lessons learned
within cohorts
• Fidelity data
• Social validity data
Analysis and reporting
• Describe variations in each component of the
conceptual framework for the three MDPs in a
cohort
• Generate hypotheses from implementation/
innovation research regarding how variations may
shape implementation experiences
• Hold hypotheses up to implementation experiences
and outcomes
– Across MDPs in a cohort
– Across cohorts
• Derive principles regarding an effective and efficient
model demonstration process
Cohort I:
The University of Minnesota: Teri Wallace
Minneapolis Public Schools: Douglas
Marston
Lehigh University: Edward Shapiro and
The University of Pittsburgh: Naomi
Zigmond
The University of Oregon: Gerald Tindal
14
Project MP3: Monitoring
Progress In Pennsylvania Pupils
Lehigh University
15
What Makes RTI(I) Work?
• Consensus
• Infrastructure
• Implementation
16
Lessons Learned: What Made It Work?
Consensus
• District committed to sustainability from
outset
• District support at highest level of
administration
• Willingness of schools to own the process
17
Lessons Learned: What Made It Work?
Infrastructure Support
• Professional Development Specialist in district
• District redirected resources to support implementation
• District partnered with Project to secure new instructional
resources
• Schools had similar context
• Each school had identified principal leadership, from the
district perspective
• Presence of well established core reading program
• Willingness to modify schedules
• Willingness to seek needed professional development
• Inclusion of parent advisory component
18
Lessons Learned: What Made It Work?
Implementation Support
• Universal screening measures already in place
for several years
• Use of data, use of progress monitoring, not
evident despite universal screening
• Schools modified professional development
schedule to meet project specifications
• Schedules included recognized time for core
and grade level meetings
19
Process Outcomes
• Implementation fidelity high for model in all schools
• Data use and data based decisions high in all schools
• District expanded the project to all elementary
buildings in district within 2 years of project ending
• Maintained presence of parent advisory group to
process
• Maintained process despite change in building
principal at one building
• Staff able to assume data management task after
support for a year
• Model fit within the statewide initiatives
20
Some Not So Good Outcomes
• Despite strong implementation in all buildings,
one building had much poorer student
outcomes
• Instructional leadership of building principal
level becomes crucial
21
DPM
University of Minnesota and
Minneapolis Public Schools
http://progressmonitoring.net/ - see link to video
22
Context
• District had long history of commitment to using
data for instructional decision-making though not
implemented fully in all schools
• Implementation school’s leadership was
committed to RTI, Reading Initiatives, Professional
Development, etc.
• Schools had numerous and varied initiatives
needing alignment
• Teachers’ knowledge of RTI and its various
components varied across the schools
23
Implementation - Tools
• System Supports
– Data Management System
– Time – Master Calendar
• Professional Development
• Leadership and Changing Roles
24
Process Outcomes
• Procedures were developed to assess teachers’
fidelity of implementation related to
interventions, progress monitoring procedures,
and data review meetings
• Professional development strategies were
targeted based on the needs shown in student
data. For example, PLCs focused on Tier 2 and 3
interventions needed to meet student needs.
Coaching responded to needs identified through
fidelity checks, and more…
25
Process Outcomes, continued
• Data meetings met to review progress of all
students and focused on students receiving Tier 2
and 3 interventions. A RTI coordinator was hired
to facilitate these meetings, organize data, etc.
Originally supported through grant dollars, this
position remains despite district cuts.
• Teacher attitudes were measured to determine
their view of the components of RTI. They were
supportive and positive.
26
Teacher Testimonial
I thought I was an “OK” teacher
and having been through the
whole process of learning more
about instruction, understanding
the developmental aspects of
reading, knowing what to do
because I have good assessment
data – I feel like a great teacher.
27
Tertiary
Intervention:
The K-I Center
Wayne Sailor
Principal Investigator
University of Kansas
Lessons Learned from
Model Demos:
Creating Change to
Promote Children’s
Success
Cohort II
Lucille Eber
Co-Principal Investigator
Illinois PBIS Network
OSEP Project Director’s Conference
July 19-21, 2010
28
Cohort II- cont.
Intensive Positive Behavior Support (IPBS)
The University of Oregon: Cynthia Anderson
Scaling the Pyramid: A Model of Tertiary
Intervention Services to Students with
Challenging Behavior
University of WA: Carol Davis
Ilene Schwartz
I
29
K-I Center Leadership
• Jamie Bezdek
• Amy McCart
• Holly Sweeney
•
•
•
•
•
Kimberli Breen
Kelly Hyde
Sheri Luecking
Diane McDonald
Jen Rose
30
Implementation Outcomes:
Change in Knowledge/Skills of Adults
• Role of Building Administrators with behavior
support
• Change in role of Special Education personnel
• Role of District Administrators in guiding the
systems change process
31
Example: Principal Role
(no longer a “Special Education issue”)
Administrators being taught the system features,
the data/tools, and the practices well enough to
guide/lead any “corrections” needed within
FBA/BIP and wraparound plans.
Administrator Training Skill set example:
If an individual behavior intervention plan
is not working, what should a principal look for or ask?
32
Administrator Role Example
School personnel should not be able to
choose NOT to provide students with
evidence-based interventions.
Academic ‘analogy’:
Personnel are expected to provide
evidence-based academic instruction;
same expectations need to be
established for behavior support.
33
Implementation Outcomes:
Change in Organizational Structure/Culture
• Systems need to establish competency and
confidence with fidelity of behavior
interventions within general education.
• All staff “owning” success of ALL students
• Shift in system habits: from “test/place” to
interventions and progress monitoring
34
Implementation Outcomes:
Changes in External Relationships
• District Leadership Teams
• District External Coaches
• Special Education Directors/Organizations
• Community Partners
• Families
35
Implementation Outcomes:
Sustainability of Model
36
Replication of Tertiary Demos Moving Faster…
Schools with Item in Place
IL Phases of Implementation: Secondary Phase I (n=8 Replication Schools)
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Team meets
Sec/Ter.
Students are
regularly
tracking tool
referred for
rate for simple
used
tier 2
tier 2
interv entions
interv entions
Fall 2008
DPR used
Spring 2009
70% success
Implementation Outcomes:
Sustainability of Model
• Policy changes are addressed
– transition support to ensure behavioral success
– Job Description and supervision changes
• Training and technical assistance for behavior skill
sets are routinely scheduled in districts
• Use of data by district teams (risk ratio, LRE, etc)
becomes ‘business as usual’.
38
Policy Examples
• Summer FTE of School Social Workers to
support some students w/Tier 3 plans
• Transition planning support expected as
students change grade levels
• Job Descriptions include expectations for
evidence-based behavior support
39
Training Examples
• Administrator role with behavior support at
all three tiers
• Changing role of special education
personnel in facilitating teams/plans
• Teacher expectations with behavior support
at all three tiers
• Using RtI ‘framework’ with family
engagement
40
Data Examples
Review data by ethnicity and disability
routinely at building and district levels
– Discipline data
– Academic data
– Restrictiveness of placement data
41
Students with IEPs Served
in Separate Placements
Ratio
8
6
4.91
4
2
130
9.01
122
7.48
120
4.58
110
102
0
100
90
FY07
FY08
Students w/IEPs in separate placements
District Ratio
State Target
# of Students
10
Lessons Learned from Model
Demos:
Creating Change to Promote
Children’s Success
Cohort 3
Early Childhood Language
Interventions
OSEP Project Director’s Conference
Washington DC
July 19-21, 2010
43
The Center on Everyday Child
Language Learning
Orelena Hawks Puckett Institute
Carl Dunst
Carol Trivette
University of Kansas
Dale Walker
Jane Atwater
Kathy Bigelow
Vanderbilt University
Florida State University
Ann Kaiser
Juliann Woods
44
Common Features of Cohort 3
Process
Models of naturalistic evidence-based language
interventions for children ages birth through 5
Implement the model in 3 sites representing typical
settings
Enroll children in Part C programs and follow them into
Part B programs
Provide professional development to providers
implementing the model
Assess impacts using some common measures
45
Coaches build the capacity of parents and early educators to
promote communication
Parents/teachers use evidence-based practices
Build on strengths and collaborate with parents about strategies
and routines
Embed intervention into home and school routines
Collaborate with Part C and Part B during transitions
46
Parents are their child’s first communication partners
Teachers and service providers support child communication
across settings
KTTP Communication teams include families, providers across
agencies, and communication coaches
Communication coaches facilitate parent and service providers
learning
Coordinated transition and continuity in communication
intervention between Parts C and B is lead by the parent in the
communication team
47
Identify children’s interests and everyday activities that are suited for
learning communication skills
Increase child participation in these interest-based everyday activities
Embed instructional practices for supporting and strengthening
communication in the contexts of activities
Part C providers facilitate parent learning and use of strategies
Approach fits within existing programs
Strengthen parents’ abilities to enhance the transition to Part B
48
Strategies to Change in External
Relationships
Use the team that includes parents to
collaborate with the Part B preschool services
during transition
Empower parents to lead the team to support
the transition to Part B preschool services
Empower parents to promote continuity for
children from Part C to Part B preschool
services
49
Process Reflections on Transitions
Projects differ in the people they are trying to include
in the transition from Part C to Part B
Projects all include parents in the transition but the
emphasis of the role of the parent varies
Projects differ in terms how direct their involvement is
in the transition process
These differences lead to different transition
challenges: joining an already formed team,
development of parent leadership, development of
transition skills in parents
50