Early Warning
Indicator Systems
Convening
welcome
Welcome
Dr. Kelly Goodsell
Executive Director
Learning, Teaching and Family Support
PSESD
Special Thank You’s
District Leaders and Staff
The Bill and Melinda Gates Foundation
Dr. Robert Balfanz and Spokane District Team
Youth Ambassadors
PSESD Staff and all of our Partners
Opening Remarks
Dr. Nancy Coogan
Superintendent
Tukwila School District
To double the
number of students
in South King
County and South
Seattle who are on
track to graduate
from college or
earn a career
credential by 2020
Youth Perspective
Youth Ambassadors
Devan Rogers & Gio Winston
Seattle Public Schools
Project Overview
Linda Filley Bentler
Program Manager
Early Warning Indicator Systems
PSESD
Early Warning Indicator Systems
• Research shows that students at risk of dropping out give
warning signs years in advance.
• School districts can use an Early Warning Indicator System
(EWIS) to identify which students are off track and at risk of
leaving school before graduation (ABC’s—attendance,
behavior and course performance).
• When student data and accurate indictors are aligned with
appropriate interventions, students who are off-track can be
brought back on-track for graduation.
• Patterns in the information about students who are
off-track can lead to improvements in district and school
policies, structures and practices.
Road Map Graduation Statistics
Road Map Results Report 2013
• 74% of students graduated on time
and lower for students of color:
Hispanic (46 %), American Indian (50 %),
Multi-Racial (63%) and Black (64 %)
• 16% of Road Map students are ELLs and 50% of ELLs
graduated on time
• 46% of SPED students graduated on time
• 20% of students switched schools and of this group,
almost three quarters transferred between districts.
Phase One: 2012-2013
• Inventory of Road Map districts current EWIS status
• Literature Review of EWIS research with special focus
areas: ELL, special education, mobility, family
engagement
• Regional agreement on Collective Indicators for
ABC’s
• Framework for Interventions
• Convening to highlight EWIS best practices for
interventions
Collective Indicators
Collective Indicators
• 5 – 10 absences (excused or unexcused) and
one course failure in 8th or 9th grade
• Single suspension/expulsion in 8th or 9th grade
Repository
www.psesd.org/road-map-early-warning-system
NOTE: Excludes Seattle, for which data were not available.
SOURCE: OSPI student-level database
Early Warning Indicator #1
Road Map Results Report 2013
Early Warning Indicator #2
SOURCE: OSPI student-level database
Road Map Results Report 2013
Early Warning Indicator Framework for Intervention
Attendance
Behavior
Course
Performance
Reengagement
Intensive Intervention
Credit Retrieval
Intensive
Wraparound
1:1
Tutoring
Alternatives to Suspension/Expulsion
SPEDCoordination
Intervention
Targeted Academic
Intervention
Extended Learning Opportunities
ELL Coordination
Mentoring/Structured
Meetings
Coordinated Interventions
Change of School Transitions (6th/8th/9th)
EWISDe nitions/Thresholds
Prevention
Foundation
MOUMobility
Family Communication
Teacher/Student
Relationship TPEP
Universal Supports
Discipline Alignment
Grading Alignment
Attendance Alignment
Integrated Data System
Visual Reports & Planning
Training & Expectations for Use
Phase Two: 2013-2015
• Action Team Meetings for regional coordination
• Baseline Data on Collective Indicators
• EWIS Action Plans using National High School Center
model
• Coaching/PD offered individually and collectively
• Development in three areas: EWIS Design, Staff Training
or Intervention Design/Alignment
• Convening to learn best practices for implementations,
share with other PSESD districts and partners
Project Outcomes
We are working toward…
• District and building EWIS teams identified,
trained and meeting regularly
• User friendly ABC data (Attendance, Behavior,
Course performance) automatically generated
in real time and easily available
• Interventions tracked and documented to
identify most effective methods (district and
CBO’s)
• Progress evaluated for continuous improvement
Collaboration with Race to the Top Projects:
• Project 2: Regional Data Portal
Stay Strong
• Project 6: Integrated System of Middle and High
School Counseling and Advising
• Project 7: College and Career Readiness Pathway
• Project 8: College and Career Readiness
Investment Fund
• Commitment 5: High School and Beyond Plan
Early Warning System Implementation Steps
The National High School Center, Early Warning System Implementation Guide
Step 1
Step 7
Evaluate and refine
the EWIS process;
monitor systems issues
Establish roles and
responsibilities to
manage a EWIS
Step 2
Use a EWIS tool
analyze and display
data on indicators
Step 6
Step 3
Monitor students
and interventions for
progress
Review the EWIS
data for accuracy
and patterns
Step 5
Assign and provide
interventions to
students
Step 4
Interpret the EWIS data
to identify students and
systems issues
18
District Progress
Auburn School District
• Focusing on 8th graders, and 9th grade transition
• Reviewing ABC’s to identify students at risk in team
meetings and tracking interventions
• Building on partnerships with Communities in
School
• Exploring options for integrated data displays
District Progress
Federal Way School District
• Focusing on secondary schools initially and led by
Assistant Principals district team
• Launching Versifit data tool to integrate indicators
in real time data display to generate priority list
• Creating professional development train the
trainers for teams for spring/summer
• Planning regular building meetings to align
interventions with instructional coaches and
interventionists
District Progress
Highline School District
• “Freshman Tracker” program to focus on ABC’s in
critical 9th grade year
• Success Deans monitor data and interventions on
approximately every 15 day schedule
• Summer School program to help catch up 9th
graders before 10th grade
• Alternatives to out-of-school suspensions
• Updating automatic data display of ABC’s for
new SIS
District Progress
Kent School District
• Revise EWIS data display dashboard for ABC’s
• Collaborate with central office and school
building sites at secondary level initially to plan
building implementation
• Create PD training and support for data use and
intervention alignment
• Train school staff on system and begin
strategically implementing support interventions
District Progress
Renton School District
• Elementary – Middle – High School Focus
• Defining cross-sectional team to plan districtwide
implementation, establish roles and responsibilities of
team members
• Review preliminary results based on draft indicators,
adjust and finalize indicators
• Create dashboard(s) and alert systems using current
student information systems
• Create and train pilot school teams in particular feeder
path (elementary – middle – high ) to use the system,
adjust based on feedback
District Progress
Seattle School District
• Development and roll out of what the tools (ADW)
are and how to use (e.g., data protocols)
• Provide data coaching workshops/trainings as
needed to operationalize use of EWIS reports on
regular basis
• Gather feedback on existing tools and reports
and seek input on prioritization for next steps
District Progress
Tukwila School District
• District level review of policies and procedures for
overt and covert barriers. Examining disaggregate
discipline data to set baseline for goals and targets
• High School – Middle School – Elementary focus.
High School Team meets weekly to review and act on
data, middle school team is forming, then elementary
• Creating PD plan to address goals in discipline
data/referrals
• Exploring using Homeroom for main ABC data tool
Other PSESD Districts & Partners
Are there other PSESD Districts or Community
Partners present who want to share briefly
their plans for Early Warning Indicator Systems
or alignment with EWIS?
Resources
Puget Sound ESD - a best practice central repository
•
www.psesd.org/road-map-early-warning-system
Education Northwest/REL - Training and interactive modules to develop an early warning system
• http://mtaarelnwallianceews.educationnorthwest.org/
OSPI - Dropout Early Warning and Intervention System (DEWIS) supports
• https://eds.ospi.k12.wa.us/iGrants/docs/1314/FormPackages/CompState/BuildingBridges672/BBs_DPIGrantApplicationProcessInfo_SFv4.pdf
• https://k12.wa.us/GATE/BuildingBridges/pubdocs/DEWISGuide-Final.pdf
• http://www.k12.wa.us/legisgov/2013documents/DropoutPreventionInterventionAndReengagemen
tDec2013.pdf
• https://k12.wa.us/GATE/BuildingBridges/default.aspx
Everyone Graduates Center – at John Hopkins University
•
http://www.every1graduates.org/
National High School Center - Early Warning System Implementation Guide and Tools
• http://www.betterhighschools.org/ews.asp
Convening Objectives
1. Learn about best practices in Early Warning Indicator System
implementation
2. Gain insights and practical knowledge on how
implementation impacts their job role
3. Increase awareness of the intersection between race and
equity and EWIS work
4. Build momentum to fully implement EWIS in Road Map
districts
5. Sustain and strengthen regional collaboration in the
Road Map region and share information about EWIS with
other PSESD districts
Regional Action Team Members
Auburn:
Denise Daniels, Lenny Holloman, Gordon O’Dell
Federal Way: Eric Dickinson, Scott Haines, Ron Mayberry
Highline:
Rachel Klein, Dr. Alan Spicciati
Kent:
Brad Brown, Dr. Linda Del Giudice, Razak Garoui,
Stosh Morency, Susan White
Renton:
Dr. Pete Bylsma, Dr. Tammy Campbell, Michelle Hintz
Seattle:
Eric Anderson, Janet Blanford, Amy Klainer,
Sylvia Shiroyama
Tukwila:
Dr. JoAnne Fabian and team
CCER:
Kirsten Avery “Avery”, Nicole Yohalem
PSESD:
Linda Filley Bentler, Dr. Bruce Cunningham
Keynote Presentation
Dr. Robert Balfanz
Co-Director
Everyone Graduates Center
John Hopkins University
Why Early Warning
Systems Matter and Where
They Are Going Next
Robert Balfanz
Everyone Graduates Center
Johns Hopkins University
March 11, 2014
Our Nation Faces a Graduation Challenge
• There is little work for young adults without a high
school degree
• And no work to support a family without some postsecondary schooling or training
• As a result entire communities are being cut off from
participation in American society and a shot at the
American Dream
• This weakens the Nation
Dropping Out of High School Means You
Are Much More Likely
• To be out of the labor market as full time employed
• To be incarcerated
• To have health issues and at an earlier age
• To pass on these set of disadvantages to your
children
• As a result, dropping out has high individual and
community costs
This Presents our Schools and
Communities with a Big Challenge
• Every student regardless of needs, prior levels of
school success, and current motivations needs to
graduate from high school prepared to succeed in
post-secondary school and training
In Era When All Students Need to
Graduate Prepared for College and
Career
• The best teachers and the best curriculum are not
enough
• Students also need to attend school regularly, focus in
class, and complete their assignments
• Poverty complicates this and the challenges are
significant
• Schools can and need to be organized to help enable
students to attend, behave, and try
Early Warning Systems keep students
on the path to high school graduation
and improve school outcomes
• Signals when students are just beginning to fall off the
path to high school graduation
• Helps get the right intervention to the right student at the
right time
• Enables schools to maximize impact of critical
resources-time and funding
• Shows which students need stronger adult relationships
• Enables adults to pool their knowledge, talents and time
to change student behavior and solve problems
What Will the Next
Generation of Early
Warning Systems
Look Like?
Summary Findings from Early
Adopters of Early Warning
Systems Conference Held at Bush
Institute Nov. 2013
Good News: Schools and Districts Across
the Country are Adapting Early Warning
Systems to Work in Their Community
• By building a response system matched to the scale and scope
of their challenges
• In schools with 20 or fewer students with off-track indicators
individual counselors, social workers, or graduation coaches
have led the effort
• In schools with 20 and 50 students small dedicated teams of
staff members - e.g. student support teams have been
successful
• When more than 50 students are involved teacher teams will
need to play a critical role
• In all cases investments in mission building, professional
development, coaching and networking have been critical to
success
Good News: Dropout Prevention and College
and Career Readiness Can Be Propelled by a
Unified System of Predictive Indicators and
Tiered Evidence Based Interventions
• Consistent Finding that no matter how you look at it or
at what grade - the ABC’s - Attendance, Behavior, and
Course Performance - are predictive of student
success
• Students who attend school regularly, behave/try/selfmange, and do well in their courses graduate from high
school and succeed in college
• Students who do not, dropout or do not succeed in
college
Good News: The ABC’s are Actionable
• Attendance, behavior, and course performance can
be modified and improved through organized and
informed actions at the school, district, and
community level
• This provides a coherent message to schools and
students-drive up good attendance, behavior and
course performance, pay attention to and
prevent/intervene when low attendance, problematic
behavior, and poor course performance occur
Good News: Grades 5 to 14 Can be
Viewed as an Ecosystem
• Early adolescence to early adulthood is a distinct
phase of life
• To build pathways from poverty to adult success we
need to get all students through this stretch
• Pre-k to 4th grade “on track” metrics are important
but they may or may not be the same as–early
adolescence to early adulthood metrics–we do not
know yet–except that attendance always matters
Good News: We Know the Key
Inflection Points
• Students who are good at school by 9th grade by and
large succeed, those that struggle and do not earn
promotion to 10th grade by and large do not
• Most but not all students 9th grade trajectories are set
in the middle grades
• Chronic absenteeism in the early grades sets students
up to fall off-track in the middle grades
• To pivot from high school to adult success many
students need help navigating the grade 10 through
the initial years of college space
Challenges: Behavior Domain is
Complex
• We are just starting to identify the behavior signals
that indicate students are on and off track to adult
success
• Hence its an area both ripe for innovation and for
making mistakes
• But there does seem to be value added in bringing
in more expanded and closer to real time behavior
data
Two Sixth Graders
Challenges: Indicators Can Identify Too
Many Needy Kids in One Place
• Once a student has an off-track indicator reversing
course will require either changing student behavior
and/or solving a problem
• Both of these require an effective relationship between
an adult and the student
Solution
• Need to get to place where early warning indicators
are used to direct evidence based prevention
activities and resource allocation at the school,
district, and state level
Challenges: Need to Gain Broad
Acceptance of the Validity of Indicators
• Solution – Need to convince more people that
attendance and grades matter as much as test
scores for student success
In sum, what we face is a giant
engineering challenge of creating
schools designed to enable all
students to graduate prepared for
college and career and within them
getting the right support to the right
students at the right time at the scale
and intensity required.
The Good News is we are, as folks in
this room are showing, good at
engineering challenges
For more information
• Visit the Everyone Graduates Center
website at www.every1graduates.org
• E-mail Robert Balfanz at
rbalfanz@jhu.edu
Keynote Table
Reflections
Discuss Dr. Balfanz’s keynote presentation at
your table for about five minutes.
• Which findings surprised you?
• What information seems most relevant or
useful for your district or organization?
• What questions do you have?
Interactive Presentation
Dr. Robert Balfanz
with
Brad Brown, Facilitator
Director
Student and Family Services
Kent School District
Early Warning System Spokane’s Lessons Learned
Travis Schulhauser
Director of Assessment & Program Effectiveness
Corey Turner
Application Development Analyst
Steve Fisk
Principal, North Central High School
Dr. John Traynor Jr.
Associate Professor for Teacher Education at Gonzaga
University
Early Warning
System
Spokane’s Lessons
Learned
MARCH 11, 2014
Background

Mary Beth Celio Research Report

First Principal Training – summer of 2012

Principal Input

Expansion 2013/2014 school year
Lesson 1 – Data
Organization


History of Data in Spokane Schools

Large data binders with all levels of
data

Website with links to visualizations
Overwhelmed by data

Leaders need data access at the right
level and focus

Prioritizing Data

Efficiency

Common Language for Collaboration
Lesson 1 – Data
Organization (con’t)
Macro to Micro

Satellite view: aggregated measure of performance
- the “whole”
ex. district weekly attendance rate

Naked eye: school level aggregated measure of
performance
ex. school weekly attendance rate

Microscope view: nitty-gritty details that make up
the whole
ex. Johnny’s attendance data
Lesson 2 – Intentionality
Behind the Design
Five Undergirding Principles (demo)
1.
Data Interpretation (color and trend)
2.
Data Triangulation (multiple metrics getting at the same idea)
3.
Signaling (tells our users what is important)
4.
Integration (easy accessibility from the web)
5.
Data Consolidation and Organization (desperate data in one
place)
Lesson 3 – Growing leaders
through data

It’s not good enough to just identify

It’s not good enough to just intervene

We have to be proactive and responsive

You cannot assume leaders know how to lead
with data
•Early Warning
System
•Slipping Grades
•High Risk Report
Identify
Intervene
•Academic Mentors
•ICAN
•Community
Dashboard
•Mentor
Accountability
•ICAN Program
Evaluation
Monitor
Lesson 4 – It’s not just
about risk

Social Support and Academic Press

Key Performance Indicators across all
data aspects

Leaders have to focus on both areas
and our system guides them on key
indicators
Job-Alike Breakout Sessions
• Break Until 11 AM
• Breakout Sessions Until 12:15 PM
EWIS Data – Cedar Room
District Leadership – Nisqually/Duwamish Room
Building Leadership – Puyallup Room
Partnerships for Intervention Supports – Snoqualmie Room
• Lunch with District/Table teams to discuss 1-2 key
learnings from breakout sessions
Lunch – District / Table Debrief
Discuss with your District or Table Team 1-2 Key
Learnings from Convening and each
Breakout Session
If time allows, begin defining SMART goals
(Use Yellow Worksheet in Packet for notes if preferred)
Community Partnerships for Student Supports
Lori Guilfoyle
Impact Manager
United Way of King County
Report Backs: Key Take Aways
Each district or table/org:
1. What is going well with your EWIS
plans or alignment?
2. What is one key take-away from the
convening that resonates for your
team?
Convening Evaluation
Please fill out evaluation at back of
your packet and turn in before leaving.
Thank you for joining us!