Using Early Warning
Data to Improve Student
Outcomes
What are Early Warning Data and the Early
Warning Indicator System (EWIS)?
2
Massachusetts Department of Elementary & Secondary
Education
What are Early Warning Data?
 General term that refers to data used to predict,
diagnose, and monitor student risk for not meeting
academic milestones
 Early warning data includes the EWIS student risk
levels and other data found in the EWIS reports
 Also includes local data sources which include
monitoring indicators and diagnostic data
3
Massachusetts Department of Elementary & Secondary
Education
What is the EWIS?
 Massachusetts Early Warning Indicator System (EWIS)
 Created by the Massachusetts Department of
Elementary and Secondary Education
 Identifies students who are at risk of missing key
academic milestones in grades 1–12
 Provides student risk levels at the beginning of each
school year
 Student-level and aggregate EWIS information is
available through five reports in Edwin Analytics
Massachusetts Department of Elementary & Secondary
Education
4
EWIS Four Key Academic Milestones
Grade Level
Groupings
Grades 1-3
Academic Milestones
Reading by the end of third grade:
Reaching proficiency or higher on 3rd
grade ELA State Assessment
2014-15
Statewide
Results for
Milestone
57%
Grades 4-6
Middle school ready: Reaching
proficiency or higher on 6th grade ELA
and math State Assessments
55%
Grades 7-9
High school ready: Passing grades on all
9th grade courses
79%
Grades 10-12 High school graduation: Meeting all local
and state graduation requirements
87%
Massachusetts Department of Elementary & Secondary
Education
5
What are EWIS Student Risk Levels?
 Students who attended a Massachusetts public school
in the year prior is assigned a level of risk of low,
moderate, or high
 A student’s risk level indicates their likelihood of
missing the key academic milestone for their grade
level
 E.g., A 5th grade student that is high risk is considered to be
off track for meeting the academic milestone of proficient or
higher on the 6th grade ELA and math MCAS
 EWIS student risk levels are based on statistical
formulas that are applied to multiple sources of readily6
available data from the prior year
Massachusetts Department of Elementary & Secondary
Education
EWIS is / is not…
The Massachusetts EWIS is…
=
A tool to better target interventions
and supports at the individual, small
group, and whole school levels.
A systematic way to flag students that
need to be further examined to
determine what additional support are
necessary.
Related to measures that are included
in the state’s school and district
accountability system (PPI), but is not
itself an accountability measure.
Based on data and risk models that the
Department will reexamine each year
to continually improve the system.
The Massachusetts EWIS is not…
A “life sentence” or a reason to label
students.
A diagnostic tool that provides details
on the exact reason(s) why a student is
at risk for not meeting an expected
academic goal.
An accountability measure.
A stagnant system that will remain
exactly the same in future years.
Massachusetts Department of Elementary & Secondary
Education
7
Local Early Warning Data
 Monitoring indicators:
 Rely on data that are available during the school year –
such as attendance, academic performance, and
behavior
 Complement the use of EWIS student risk levels,
schools/districts can use monitoring indicators to flag
students who are at risk and track student progress at
multiple times throughout a single school year
 Diagnostic Data:
 Supplemental data sources that can help schools and
districts better understand underlying causes of student
risk, and track student progress in interventions
Massachusetts Department of Elementary & Secondary
Education
8
EWIS Report Options in Edwin Analytics
9
Massachusetts Department of Elementary & Secondary
Education
EWIS Reports in Edwin Analytics
 There are five EWIS reports in Edwin:
 EWIS
 EWIS
 EWIS
 EWIS
 EWIS
District View (EW301)
Student List (EW601)
Graphical View (EW302)
Subgroup Analysis (EW318)
Risk Level Indicator Analysis (EW317)
 All reports may be filtered numerous way (e.g., specific
schools, grade levels, student subgroups)
 EWIS Risk Level is also a filtering option in new
classroom (teacher) reports
10
Massachusetts Department of Elementary & Secondary
Education
Edwin Analytics: EWIS Reports
11
Massachusetts Department of Elementary & Secondary
Education
EWIS Report: Sample Prompt Page
12
12
Massachusetts Department of Elementary & Secondary
Education
EWIS District View (EW301)
Which grade levels or schools have the most students
that need targeted support in order to be on track to
meet the next academic goal?
What is the breakdown of high, moderate, and low risk
levels in a particular student subgroup (e.g. among
students with disabilities)?
Where does the district need to focus additional
resources to get students back on track for meeting
the next academic goal?
 Tip: This report is a great first place to start reviewing
EWIS data!
Massachusetts Department of Elementary & Secondary
Education
13
14
EWIS Student List (EW601)
What are the current risk levels of my students?
What are the prior risk levels of my current students?
What data points (indicators) went into the calculation
to determine the risk level?
 Tip: Avoid the temptation to go straight to the Student
List report when first reviewing EWIS data… it is
usually more helpful to start with the other reports to
get the lay of the land
Massachusetts Department of Elementary & Secondary
Education
15
Gr
School
Last
ad
Name
Name
e
First
Name
Non- Previo
Susp
Scho
Urban
Title1 ACCES ELA
Math Math
ELA
Sci
SS
Race/E
Risk Attend
Retain
Low SPED
Overag
Core us Year
SASID
ensio
ol
Gender Reside
School S for MCAS MCAS Course Course Course Course
thnicit
Level Rate
ed
Income Level
e
Course Risk
n
Move
nce
wide ELLs 2014
2014
Perf
Perf
Perf
Perf
y
Perf
Level
Any High 9
School
Aardvark Arnold
High
98%
0
No
No
Free
None
Female Yes
No
Yes
1.8
-
Pass
Pass
Pass
Pass
Pass
N/A
Hispani Y
c or
Latino
Any High 9
School
Bailey
High
90%
0
No
No
Free
Moderat Female Yes
e
Yes
Yes
None
Needs Warning Pass
Improve
ment
Pass
Fail
Fail
Fail
High
White
N
N/A
Any High 9
School
Colorado Carlos
High
88%
0
No
No
Reduced None
Female Yes
No
Yes
2.6
-
Pass
Pass
Pass
Pass
Pass
N/A
White
Y
Enrolled Not a
in
Special
sheltere Educati
d
on
English Student,
immersi age 6-21
on
Any High 9
School
Denton Daisy
High
99%
0
No
No
Free
None
Male
Yes
Yes
Yes
None
Proficie Needs Pass
nt
Improve
ment
Pass
Fail
Pass
Fail
High
Hispani N
c or
Latino
N/A
Not a
Special
Educati
on
Student,
age 6-21
Any High 9
School
Everett
High
74%
0
No
Yes
Free
None
Female Yes
No
Yes
None
Needs Warning Pass
Improve
ment
Pass
Pass
Pass
Fail
High
White
N/A
Not a
Special
Educati
on
Student,
age 6-21
Billy
Elena
-
ELL
Special
ELL Educat
Progra ion
m
Placem
ent
Enrolled Not a
in
Special
sheltere Educati
d
on
English Student,
immersi age 6-21
on
-
N
Massachusetts Department of Elementary & Secondary
Education
Full
Inclusio
n special
educatio
n
services
outside
the
general
educatio
n
classroo
m less
than
21% of
the time
16
EWIS Graphical View (EW302)
How does a selected group of students compare to the
district and state?
What are the previous risk levels of the currently
enrolled students over the past 5 years?
E.g., what are the previous risk levels of my current
4th grade male ELL students?
What are the risk levels of a specific student group
over the past 5 years?
E.g., what are the risk levels of 4th grade male ELL
students each year over the last 5 years?
Massachusetts Department of Elementary & Secondary
Education
17
18
EWIS Subgroup Analysis (EW318)
 How do EWIS risk levels within a school distribute over
different race/ethnicities or gender?
 Is there a noticeable difference in EWIS risk levels for
certain student subgroups (i.e. SWD, ELL, Low
Income, etc.)?
 How do the EWIS risk levels of a particular subgroup
of students compare to the state as a whole?
 Are there certain groups of students who could benefit
from additional services or interventions based on
their risk levels?
Massachusetts Department of Elementary & Secondary
Education
19
20
EWIS Risk Level Indicator Analysis
(EW317)
 How do attendance or suspension rates differ between
risk levels?
 Which courses do high risk students Pass or Fail at a
consistent rate?
 What are the areas of concern among high risk
students that need to be further investigated?
 Are there particular areas within a grade where
universal (Tier I) changes may help students?
21
Massachusetts Department of Elementary & Secondary
Education
22
Coming Soon!
Expanded EWIS
Predict readiness for postsecondary education
1. Enrollment : likelihood to enroll in postsecondary
education
2. Developmental/Remedial: readiness for creditbearing college courses
3. Persistence: likelihood for students to persist from
the first to second year
Massachusetts Department of Elementary & Secondary
Education
23
Early Warning Implementation Cycle
24
Massachusetts Department of Elementary & Secondary
Education
What is Early Warning Implementation?
 Multi-step process that districts and schools can follow
to effectively use early warning data to:
identify at-risk students
explore underlying causes for risk
provide interventions and supports
monitor student progress
make schoolwide and student level improvements
 Tip: This type of multi-step process is a key
foundation for implementing tiered systems of support
Massachusetts Department of Elementary & Secondary
Education
25
Early Warning Implementation
Theory of Action
COMPONENTS
INTERMEDIATE OUTCOMES
OUTCOMES
EWIS,
monitoring
indicators, and
diagnostic data
Early Warning
Implementation
Guide and other
resources,
training, and
supports
District and/or
school-based
team focused on
using early
warning data to
support students
A
Improved
use of
data
B
Improved
assignment
of
resources
C
Improved
progress
monitoring
D
Improved
likelihood
for
students
in need to
receive
apt
supports
and get
back on
track
E
Improved
student
success in
reaching
desired
academic
milestones
26
Massachusetts Department of Elementary & Secondary
Education
6 Step Early Warning
Implementation Cycle
27
Massachusetts Department of Elementary & Secondary
Education
For more information…
 Watch the Edwin website for updates, including materials and professional
development opportunities: www.doe.mass.edu/Edwin
 Follow Edwin on Twitter! @ Edwin_DESE
 Questions can be submitted to: EdwinAnalytics@doe.mass.edu
 If you need access to Edwin Analytics – contact your district directory
administrator:
http://www.doe.mass.edu/infoservices/data/diradmin/list.aspx
 For info on how your school and district are doing on the four academic
milestones…
 3rd & 6th grade MCAS and PARCC results – Profiles: http://profiles.doe.mass.edu
 9th grade course passing & high school graduation rate – Success After High
School DART: http://www.doe.mass.edu/apa/dart/DARTSAHS.xlsx
 Video highlights of Massachusetts schools and districts, created by the
national Data Quality Campaign (DQC):
https://www.youtube.com/watch?v=mUVCGZlQbQU
Massachusetts Department of Elementary & Secondary
Education
28
Early Warning Implementation Guide
 Section 1. What are Early Warning Data?
Includes overview of EWIS & definitions of key terms
 Section 2. The Six-Step Early Warning
Implementation Process. Each subsection briefly
describes the step, anticipated outcomes, and the
tasks and processes schools and districts can use to
implement the step
 “Notes from the Field” anecdotes derived from Massachusetts
districts and schools
 “Build off Your Foundation” includes prompts that encourage
schools and districts to build upon current practices
 Each step includes with a set of guiding questions
 Each step concludes with a checklist and other tools
Massachusetts Department of Elementary & Secondary
Education
29
Early Warning Implementation Guide
 Section 3. Additional Resources. This section
provides information and links to additional
Massachusetts and other resources that may be
beneficial
 Section 4. Appendixes. This section provides
additional tools and information to support
implementation and understanding of early warning
data
30
Massachusetts Department of Elementary & Secondary
Education