The Path Towards Continuous
Improvement: Practices Necessary to
Support Data Based Decision Making
Margie McGlinchey; Ph.D.
Kim St. Martin; M.A.
June 15, 2010
Changing on Purpose
• New practices do not fare well in
existing organizational structures and
systems
Innovations and Systems
EXISTING SYSTEM
EFFECTIVE INNOVATIONS
ARE CHANGED TO
FIT THE SYSTEM
EXISTING SYSTEM IS
CHANGED TO SUPPORT
THE EFFECTIVENESS OF
THE INNOVATION
EFFECTIVE INNOVATION
Innovations and Systems
AN ETERNAL STRUGGLE
THE YIN & YANG OF CHANGE
Changing on Purpose
• People, organizations, and systems
– Cannot change everything at once (too big;
too complex; too many of them and too few
of us)
– Cannot stop and re-tool (have to create the
new in the midst of the existing)
– Cannot know what to do at every step (we
will know it when we get there)
– Many outcomes are not predictable (who
knew!?)
Improvement Cycles
• PDSA (plan, do, study, act) Cycle
• Rapid – Cycle
– Problem Solving
– Practice Improvement
• Transformation Zones
• PEP-PIP Cycle (policy enabled practice –
practice informed policy)
• Usability testing
PDSA Cycles: Trial & Learning
•
•
•
•
•
Plan – Decide what to do
Do – Do it (be sure)
Study – Look at the results
Act – Make adjustments
Cycle – Do over and over again until
the intended benefits are realized
Shewhart (1924); Deming & Juran (1948);
Six-Sigma (1990)
PDSA Cycles: Trial & Learning
David Thomas, 1985
Actual Program
Ideal Program
To Narrow the Variation of the Actual from the Ideal
PDSA Cycles: Trial & Learning
•
•
•
•
•
Plan – Innovation core components
Do – Selection, training, coaching
Study – Fidelity, outcomes
Act – Make adjustments
Cycle – Do over and over again until
fidelity is reached in a reasonable
period of time
PLAN
ACT
DO
Innovation
P
A
Training
S
P
D
STUDY
Administration
A
D
S
P
P
Coaching
A
S
D
A
Fidelity
S
D
PDSA Cycle - Eternal
• Plan – Coaching methods
• Do – Frequency, intensity, duration
• Study – Time to reach fidelity for the
past 20 teachers/ staff
• Act – Revise coaching methods to
shore up weak areas
• Cycle – do over and over again as
conditions and people change forever
more
Transformation Zones
• A “vertical slice” of an organization or
system
– Small enough to be manageable
– Large enough to include nearly all of the
relevant aspects of the current system
• Operations in the TZ allow strengths and
problems to “rise to the surface”
– Reality-based, in the moment
– Coalesce resources – human, financial and
technological – for mutual benefit
Transformation Zones
“If you wish to make an improved
product you must already be
engaged in making an inferior one.”
Jacobo A. Varella
American Psychologist,
1977
Transformation Zones
• Make use of an innovation in practice
(operationalize, fidelity, outcomes)
• Establish implementation supports (work on
practical Drivers and Stages)
• Engage leaders in the PEP-PIP improvement
cycle to align system components and functions
• Create capacity and momentum for larger scale
changes
PEP-PIP Cycle
• Policy enabled practice (PEP)
• Practice informed policy (PIP)
The PDSA cycle in slow motion
• Monthly instead of hourly, daily cycles
• Bigger issues where agreement (issues,
solutions) and certainty (if this, then that)
are not apparent
Implementation
Teams
Practice Informed
Policy (PIP)
“External” Leadership
Policy Enabled
Practice (PEP)
System
Change
Management
Team
Usability Testing
• Getting ready to scale up in a
Transformation Zone
• The use of larger samples under more
representative conditions
• Test the innovation with a more diverse
group
• Test the implementation methods with a
more diverse group
Usability Testing
• 4 or 5 “typical users” to try out the current
version of the implementation methods
and the innovation
• Fix the problems from the first “trial”
• Repeat with a new group of 4 or 5 typical
users
• Fix the problems form the second “trial”
• After 4 or 5 “trials” you should have most
of the problems solved
Usability Testing
• Each “trial” finds about 80% of the
problems with that version
• Fixing the problems is a problem – is it
the innovation or implementation?
– That is why fidelity data are so important
(the “do” part of the PDSA cycle)
– High fidelity and poor outcomes = an
innovation problem
– Low fidelity = an implementation problem
Activity
Worry
• Discuss with the person next to you
• What specific concerns you have regarding the
need for improved behavior and reading skills at
your school
• Write down two “worries” you discussed with your
partner relating to PDSA Cycles and your districts
plan for changing on purpose
• Share your concerns with the table
Wonder
• Will your data/evidence validate your concerns?
• How will PDSA Cycles work to assist you in
developing the next “right” steps
Back to the Building…
Let’s shift focus to systems we have in
place at the building to embed data based
decision making into our continuous
improvement cycles for reading.
Evaluation of
Core/Strategic/Intensive Reading Programs
Assuming we share a common goal of
teaching all students to read…
1. Are we developing reading support
systems for all students?
2. Are we improving reading outcomes at
each grade level and across time?
As the building leadership team, we
have three tasks:
1. Answer the question about how we are
doing across the whole school and across
time-at all tiers, for all students.
2. Identify improvements to celebrate with the
staff
3. Identify what systems and procedures need
to be in place to support each grade level in
the development of a collaborative plan to
address the needs of each and all students.
First: The Celebration!
Review student performance data at every grade
level.
– Is there an upward trend from fall to winter to
spring?
– Across years, is there an upward trend? Do
we have more successful students this year
than we did last year? (“goesupness”Roland Good)
A good goal for the Leadership team would be to
identify something positive to celebrate at each
grade level.
Activity
• With your partner discuss the following
questions:
– What information if any, is typically shared at
each grade level in celebration?
– What does the celebration consist of?
• Who?, What?, When?, Where?
– Is the information integrated with your School
Improvement Plan and evaluation of your
School Improvement Plan?
Setting Our Sights on Bigger Goals
Through the Problem Solving Process
✔ Problem Identification

 Problem analysis
 Plan Implementation
 Plan Evaluation
Problem Identification
– What are the standards? (goals,
benchmarks, etc.)
– What is the actual performance?
– Is the performance acceptable?
– Example:
• Histograms
MiBLSi Cohort 4:
Middle-of-Year Kdg Phoneme Segmentation Fluency*
*Missing Data
62% (n=908) Low Risk
23% (n=342) Some Risk
14% (n=207) At Risk
Questions:
1.What is the instructional priority?
Phonemicare
awareness
skills of 35 correct sounds
2.How
we doing?
per minute by Spring
Setting Our Sights on Bigger Goals
Through the Problem Solving Process
✔ Problem Identification

✔ Problem analysis

 Plan Implementation
 Plan Evaluation
Problem Analysis
• Are we confident that the results are accurate?
• Why is the problem occurring?
• What else do we need to know about the
discrepancy between the goal and the
outcome?
• What information will be used to answer the
questions?
• Summary of Effectiveness
• PET-R curriculum maps, goals, delivery of
instruction, management etc.
Setting Our Sights on Bigger Goals
Through the Problem Solving Process
✔ Problem Identification

✔ Problem Analysis

✔ Plan Implementation

 Plan Evaluation
Plan Implementation
• What needs to happen differently so that more
students are reaching our reading goals at each
grade level?
• What are the implications for resources,
professional development needs, instructional
priorities, intervention, communication?
• What information would be used to answer the
questions?
– PET-R
– Current hypotheses
– Summary of Effectiveness Student Data
– Your current action plan
Setting Our Sights on Bigger Goals
Through the Problem Solving Process
✔ Problem Identification

✔ Problem analysis

✔ Plan Implementation

✔ Plan Evaluation

Plan Evaluation
• How and when will implementation of the
plan and effect of the plan be evaluated?
• What information will the team use?
– Results and plans from grade level team
meetings
– Progress monitoring assignments and results
Progress Monitoring Assignments
and Progress Review
Making progress
Continue Instruction
Not making progress
Revise Instruction
PDSA Cycles: Trial & Learning
•
•
•
•
•
Plan – Decide what to do
Do – Do it (be sure)
Study – Look at the results
Act – Make adjustments
Cycle – Do over and over again until
the intended benefits are realized
Shewhart (1924); Deming & Juran (1948);
Six-Sigma (1990)
Activity
– What data will be reviewed by your school team
evaluate progress?
– What actions will the school leadership team
take to increase student achievement? (PD,
intervention)
– What support is needed at grade level meetings
to maximize their effectiveness and efficiency (
How often? Action plans?)
– How will this information be used to support,
and integrate with your School Improvement
Plan?
A Continuous Loop
Grade Level
Teams Inform
School
Leadership
Policy
School
Leadership
Grade Level
Team
School Leadership
Team Enables
Planning at Grade
Level
Effective Data Driven
Meetings (EDM’s):
Building Level
Why Data-Driven Meetings
• All staff have a stake in the relentless
focus of designing instructional and
intervention plans so all students can
effectively learn
• Use of a standard protocol for reviewing
data, establishing goals and objectives,
strategies, and actions is essential for
continuous improvement
Pairing Language and Terms
• Professional Learning Communities (PLCs) and
Effective Data Driven Meetings are one in the
same
• Both involve:
–
–
–
–
Collaboration
Data
Focus on critical skills
Systematic procedures for monitoring student
progress
– Action oriented
Problem Solving at Multiple Levels
MDE/MiBLSi
Leadership
Regional
Technical
Assistance
ISD
Leadership
Team
LEA District
Leadership
Team
Building
Leadership
Team
Building Staff
Students
Each level of the MiBLSi
system can use the
Continuous Improvement
Process to improve the
support structure below it.
“Begin with the End in Mind”
Covey, 1989
Activity
•While watching the EDM role play, please
look for:
–Pre-skills necessary that allowed the EDM to
be successful
–Focus
–Similarities and differences from what you
currently experience
Did Your List Look Like This?
•
•
•
•
•
Meeting mechanics
Norms
Clearly defined roles and responsibilities
Focused on data
Understanding of an analysis of data prior to
attending the meeting
• Understanding of critical skills for focus and
interventions to support skill development
• Action plan
Purpose of Collaboration
“Members of a professional learning
community recognize they cannot
accomplish their fundamental purpose
of high levels of learning for all students
unless they work together
collaboratively. The collaborative team
is the fundamental building block of a
PLC.”
DuFour, DuFour, Eaker, & Many, 2006
Climate of a Collaborative Team
•
•
•
•
•
•
Celebrative
Focused on mission
Action-oriented
Risk-taking
Accountable
Supportive (data is not used to punish)
Before Grade Level Meetings
• Share data with staff at least a week before
meeting so that grade level teachers have
an opportunity to think about and process
the data.
• Develop and distribute an Agenda
• Make sure all individuals related to
supporting grade level instruction are
invited.
Expectations of a Collaborative
Team
•
•
•
•
•
Agenda – roles and responsibilities
Norms – agreements and commitments
Goals/Objectives – be SMART
Data – use for decision-making
Action Plan – written, reviewed, and
revisited
Four Essential Questions
• What is it we want our students to learn?
• How will we know if each student has
learned it?
• What will we do when some students do
not learn it?
• How can we extend and enrich the
learning for students who have
demonstrated proficiency?
Agenda
• Always have an written agenda,
distributed well before the meeting.
• The agenda should contain roles &
responsibilities from action plan of
previous meeting.
Examples of EDM Agendas
Fall Benchmark EDM
• Establish/Review Team Norms
• The Core Program and the Big Ideas
– What skills are critical at this point in the
school year for your grade level?
– What does your data say (histograms)
– Are adjustments needed to our core
program?
– Are there any materials/training/support
necessary for the core program?
– Action Plan
Fall Benchmark EDM
• The At-Risk Learners (30 minutes)
– Who is at-risk (DIBELS Class List Reports to
identify students)
– How much growth is needed and by when?
– Identify at-risk learners whose difficulty in
reading causes an increase in misbehavior
– Action Plan for Support
• Students that have similar needs are placed in the
appropriate instructional group with the appropriate
intervention program
• Address behavioral challenges concurrently with
academics
Tips for Establishing Norms
•
•
•
•
•
•
•
Create own
State as commitments
Review frequently
Evaluate semi-annually
Focus on a few
Address violations
Establish a Parking Lot
Roles/Responsibilities: Critical
Piece in the Process
• Moderator – facilitates meeting content and
flow according to agenda
• Norms monitor – ensures adherence to the
agreed upon meeting commitments
• Time keeper – keeps meeting moving toward
action plan
• Data keeper – organized individual who makes
sure the appropriate data is available
• Scribe – takes notes during the meeting
especially regarding action plan
Activity
• After seeing example EDM agendas,
identify a grade level to begin this
process
• Develop an agenda that will be used
for the meeting
• Who will facilitate establishing norms
for your grade level meetings?
• Identify roles and responsibilities
Grade Level Supports on Website
Considerations
• Using data for decision making involves
complex skills and trusting relationshipsboth are equally important
• Make sure you have knowledgeable
individuals who can analyze data
• Make sure you create a safe environment
• The more you know, the more you can
support
• Remember the Implementation Drivers!
Activity
• How will the Building Leadership Team
communicate and support the planning
work that needs to happen at each grade
level?
• How will the results of each grade level
meeting be communicated to and
monitored by the Leadership team?
Next Steps
• It is helpful to have a guidance document
for getting started in the beginning of the
year
• Checklists help you remember important
things in the midst of multiple competing
activities, like launching a new school
year!
Activity
• Take a few minutes to review the checklist
for starting the year right…
• Would any of these steps on the checklist
help to address your “worries”
Thank You!
Margie McGlinchey, Co-director Michigan Integrated Behavior
Learning Support Initiative (MiBLSi)
margiemcglinchey@mac.com
Kim St. Martin, MiBLSi Lead Technical Assistance Provider
kimstmartin@me.com
MiBLSi Website:
http://www.cenmi.org/miblsi/Home.aspx