District and
School
Profiles
College and Career Readiness Webinar Series:
College and Career
Readiness Data
Tools
1
Massachusetts Department of Elementary & Secondary
Education
About the C&CR Webinar Series
September
Sept 15, 2015
Edwin: EWIS and Post Secondary
Reports
http://www.doe.mass.edu/co
nference/?ConferenceID=8169
October
Oct 13, 2015
9th Grade Transition
November
Nov 17, 2015
Labor Market Information
January
Jan 12, 2016
Individual Learning Plans
February
Feb 9, 2016
Graduation Rates
March
March 8, 2016
Career Development Education
http://www.doe.mass.edu/co
nference/?ConferenceID=8170
http://www.doe.mass.edu/co
nference/?ConferenceID=8171
http://www.doe.mass.edu/co
nference/?ConferenceID=8172
http://www.doe.mass.edu/co
nference/?ConferenceID=8173
http://www.doe.mass.edu/co
nference/?ConferenceID=8174
April
April 13, 2016
May
May 10, 2016
Maximizing Graduation Coaches and
Other Adult Advocates
Early College Pathways
http://www.doe.mass.edu/co
nference/?ConferenceID=8175
http://www.doe.mass.edu/co
nference/?ConferenceID=8176
Massachusetts Department of Elementary & Secondary
Education
2
Welcome!
Please type your question or comment in the chat
window at any point during the presentation
All phones will be muted
PowerPoint slides will be sent to all registered
participants
3
Massachusetts Department of Elementary & Secondary
Education
Goals for the Session
Understanding of the nature of data
Introduction to college and career readiness data
tools offered by ESE
“Information is a beacon, a cudgel, an olive
branch, a deterrent--all depending on who wields it
and how.” Steven Levitt
4
Massachusetts Department of Elementary & Secondary
Education
The Nature of things…
Qualitative data are everything that refers to the
quality of something. For example, a description of
experiences, observations and interviews are all
qualitative data.
Quantitative data are data that refers to a number.
The numbers of students, test scores, anything that
can be added, multiplied, subtracted or divided.
Simply put, data is information that helps in our
decision making.
Massachusetts Department of Elementary & Secondary
Education
5
What is the Cycle of Inquiry?
6
Massachusetts Department of Elementary & Secondary
Education
Making the Data Dance
 Data is a tool…
 Data can help inform our decisions…
 Behind the data are real human beings…
 Use data to tell the story…
 Data is our friend…
7
Massachusetts Department of Elementary & Secondary
Education
Overviews of College & Career Readiness
related data tools
School Profiles: http://profiles.doe.mass.edu/
Graduation rates, MCAS results, discipline data
Edwin Analytics: https://gateway.edu.state.ma.us/
Early Warning Indicator System (EWIS) reports
Postsecondary Readiness and Success reports
Success After High School DART:
http://www.doe.mass.edu/apa/dart/SAHS.xlsx
5-year trend lines for individual high schools
Massachusetts Department of Elementary & Secondary
Education
8
Profiles
9
Massachusetts Department of Elementary & Secondary
Education
First, a quick look at Profiles
http://profiles.doe.mass.edu/
Profiles are a great way to find basic
information about schools and districts
and also acts as an access point to a
great deal of other publicly available
data.
10
Massachusetts Department of Elementary & Secondary
Education
Edwin Analytics
11
Massachusetts Department of Elementary & Secondary
Education
What is Edwin Analytics?
 Goes beyond but includes reports previously available in the
Education Data Warehouse (EDW)
 A state-level platform that provides a more comprehensive view
of students, teachers, schools, districts and the state;
 Timely and actionable information that empowers educators and
informs better decision making at every level
 Tools and resources that will help educators improve teaching,
learning, school readiness and education outcomes
New reports are constantly being added
12
Massachusetts Department of Elementary & Secondary
Education
Out of the many reports in Edwin
Analytics, today we are focusing on…
Identifying current students in grades 1-12 who may be
at risk of missing key educational targets (Early Warning
Indicator System (EWIS) reports)
Identifying how your former students did after they
graduated from your school/district (Postsecondary
Readiness and Success)
13
Massachusetts Department of Elementary & Secondary
Education
Early Warning Indicator System (EWIS)
14
Massachusetts Department of Elementary & Secondary
Education
What are Early Warning Indicators?
Methodological way of looking at students and
their likelihood of meeting specific academic goals
Traditionally implemented at the high school level
to help predict which students are most likely to
drop out
Tool for educators to better design and target
student interventions in a timely way
15
Massachusetts Department of Elementary & Secondary
Education
Grade Groupings & Goals
=
Age
Groups
Grade
Levels
Academic Goals
(expected student outcomes for
each age group)
Use when setting
school and
district goals!
2013-14
Statewide
Results for
Goal
Early
Grades
Elementary 1-3
Reading by the end of third
grade: Reaching proficient or
advanced on 3rd grade ELA MCAS
57%
Late
Grades
Elementary 4-6
Middle school ready: Reaching
proficient or advanced on 6th grade
ELA and math MCAS
54%
Middle
Grades
Grades
7-9
High school ready: Passing grades
on all 9th grade courses
78%
High
School
Grades
10-12
High school graduation: Meeting
all local and state graduation
requirements
85%
Massachusetts Department of Elementary & Secondary
Education
16
EWIS Key Concepts
 Provides risk level information for all students
based on prior year data
 The risk levels are not based on a student’s
relative likelihood for reaching an academic
goal when compared with other students
 Individual indicators in the risk model are not
reflective of the level of risk - taken together,
the indicators (data points) collectively form a
risk level
 Only includes data from existing statewide
collections (SIMS, SSDR and SCS) and state
assessments (MCAS and ACCESS)
Massachusetts Department of Elementary & Secondary
Education
17
EWIS is / is not…
The Massachusetts EWIS is…
The Massachusetts EWIS is not…
A tool to better target interventions and
=
supports at the individual, small group,
A “life sentence” or a reason to label
students.
and whole school levels.
A systematic way to flag students that
need to be further examined to
determine what additional support are
necessary.
Related to measures that are included in
the state’s school and district
accountability system (PPI), but is not
itself an accountability measure.
Based on data and risk models that the
Department will reexamine each year to
continually improve the system.
A diagnostic tool that provides details
on the exact reason(s) why a student
is at risk for not meeting an expected
academic goal.
An accountability measure.
A stagnant system that will remain
exactly the same in future years.
Massachusetts Department of Elementary & Secondary
Education
18
Edwin Analytics: EWIS Reports
There are Five EWIS reports in Edwin:
EWIS District View (EW301)
EWIS Student List (EW601)
EWIS Graphical View (EW302)
EWIS Risk Level Indicator Analysis (EW317)
EWIS Subgroup Analysis (EW318)
 All can be filtered numerous way (e.g., selecting specific
schools, grade levels, student subgroups)
Massachusetts Department of Elementary & Secondary
Education
19
EW301: District Overview
20
Massachusetts Department of Elementary & Secondary
Education
EW601: The Student Level
Indicators in the risk model
Student ID info
Gr
School
Last
ad
Name
Name
e
First
Name
Student info
Non- Previo
Susp
Scho
Urban
Title1 ACCES ELA
Math Math
ELA
Sci
SS
Race/E
Risk Attend
Retain
Low SPED
Overag
Core us Year
SASID
ensio
ol
Gender Reside
School S for MCAS MCAS Course Course Course Course
thnicit
Level Rate
ed
Income Level
e
Course Risk
n
Move
nce
wide ELLs 2014
2014
Perf
Perf
Perf
Perf
y
Perf
Level
Any High 9
School
Aardvark Arnold
High
98%
0
No
No
Free
None
Female Yes
No
Yes
1.8
-
Pass
Pass
Pass
Pass
Pass
N/A
Hispani Y
c or
Latino
Any High 9
School
Bailey
High
90%
0
No
No
Free
Moderat Female Yes
e
Yes
Yes
None
Needs Warning Pass
Improve
ment
Pass
Fail
Fail
Fail
High
White
N
Any High 9
School
Colorado Carlos
High
88%
0
No
No
Reduced None
Female Yes
No
Yes
2.6
-
Pass
Pass
Pass
Pass
Pass
N/A
White
Y
Any High 9
School
Denton Daisy
High
99%
0
No
No
Free
None
Male
Yes
Yes
Yes
None
Proficie Needs Pass
nt
Improve
ment
Pass
Fail
Pass
Fail
High
Hispani N
c or
Latino
Any High 9
School
Everett
High
74%
0
No
Yes
Free
None
Female Yes
No
Yes
None
Needs Warning Pass
Improve
ment
Pass
Pass
Pass
Fail
High
White
Billy
Elena
-
ELL
-
N
Massachusetts Department of Elementary & Secondary
Education
Special
ELL Educat
Progra ion
m
Placem
ent
Enrolled Not a
in
Special
sheltere Educati
d
on
English Student,
immersi age 6-21
on
N/A
Full
Inclusio
n special
educatio
n
services
outside
the
general
educatio
n
classroo
m less
than
21% of
the time
Enrolled Not a
in
Special
sheltere Educati
d
on
English Student,
immersi age 6-21
on
N/A
Not a
Special
Educati
on
Student,
age 6-21
N/A
Not a
Special
Educati
on
Student,
age 6-21
21
EW302: Graphical View
22
Massachusetts Department of Elementary & Secondary
Education
THE NEWEST EWIS TOOLS!
EWIS Risk Level Indicator Analysis (EW317)
A tool to analyze specific attendance, behaviors, and course Early
Warning Indicator System (EWIS) indicators for students in each risk
level for a selected grade.
EWIS Subgroup Analysis (EW318)
Provides educators with aggregate student risk levels of not
achieving their next educational outcome as determined by the
Early Warning Indicator System (EWIS) by student subgroups
within a District
23
Massachusetts Department of Elementary & Secondary
Education
Insert EW 318 Screen shot
24
Massachusetts Department of Elementary & Secondary
Education
Insert EW317 Screen shot
25
Sample of larger
amount of data
Massachusetts Department of Elementary & Secondary
Education
Postsecondary Outcome Reports
26
Massachusetts Department of Elementary & Secondary
Education
Postsecondary Outcomes of Public High
School Students
Collecting and analyzing data about what college-bound students
actually do after graduating high school helps district and school
leaders assess the effectiveness of their programs in developing
graduates who are college ready.
The data for these reports are drawn from existing ESE data
sources such as SIMS, SCS and MCAS as well as College Board (for
SAT) , the Mass. Dept. of Higher Education HEIRS system and the
National Student Clearinghouse.
27
Massachusetts Department of Elementary & Secondary
Education
How did our students do after graduating?
The Three Basic Reports:
Postsecondary Outcomes of Public HS Graduates:
All Institutions (CR301)
Postsecondary Outcomes of Public HS Graduates:
MA Public Institutions Only (CR302)
Postsecondary Outcomes of Public HS Graduates:
Academic Preparation (CR303)
28
Massachusetts Department of Elementary & Secondary
Education
Use when
setting
school and
district
goals!
Comparing the Three Reports
Potential Questions
CR301
CR302
CR303
How many graduates go on to postsecondary?
What type of institutions (4-year/2-year, private/public) do
graduates attend?
Which specific institutions did students attended?
How many went to each?
What are the enrollment rates of students who complete
MassCore or AP coursework compared to those who don’t?
What are the persistence and degree obtainment rates of
students who complete MassCore or AP coursework?
What is the postsecondary experience of students from a
specific subgroup? (CVTE, ELL, SWD, high needs, etc)
What percentages of students enroll in and complete
remedial courses?
In what remedial subjects to do students enroll?
What are the MCAS scores of students who take remedial
coursework in math and reading?
How many credits, on average, do public high school
graduates earn in their first year of college?
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
29
X
Massachusetts Department of Elementary & Secondary
Education
X
How many high school graduates enroll
in postsecondary education?
… at MA public institutions?
Drill down
to student
level
records
30
On top: data from CR301; on bottom: data from CR302
Massachusetts Department of Elementary & Secondary
Education
What Types of Institutions are Your
Students Attending?
31
Massachusetts Department of Elementary & Secondary
Education
How many return for a 2nd year?
What does
persistence
look like
across
subgroups?
32
Massachusetts Department of Elementary & Secondary
Education
Where
did
students
enroll?
(Anywhere in
the US)
33
This data from CR301
See CR302 for MA Public Institutions Only
Massachusetts Department of Elementary & Secondary
Education
What are enrollment rates of students
who complete AP or MassCore?
34
Massachusetts Department of Elementary & Secondary
Education
Of those in MA public postsecondary,
what % enroll in remedial courses?
In which
subjects?
35
Data from CR302
Massachusetts Department of Elementary & Secondary
Education
How many
of your
graduates
obtain a
degree?
36
Massachusetts Department of Elementary & Secondary
Education
… and this is just the beginning of
what we can do!
We could also explore…
What is the relationship between
MCAS scores and the type of institution where students
enroll? (4-year, 2-year, public, or private)
What is the postsecondary experience of students from
a specific subgroup (CVTE, ELL, students with
disabilities, high needs, etc)?
37
Massachusetts Department of Elementary & Secondary
Education
Additional Post-Secondary Reports
Several additional new Edwin Analytics reports may assist schools and
districts in analyzing post-secondary data for program improvement.
CR304 Postsecondary Outcomes of Public HS Graduates View by MA Public
Postsecondary Institution:
Details postsecondary enrollment, achievement, and outcomes of MA public
graduating students from a fall enrollment class in a specific MA public
postsecondary institution. It also contains details by “feeder” MA public
high schools. (Postsecondary data source: MA Department of Higher
Education)
CR321 CVTE Graduate Pathway Summary: MA Public Postsecondary:
Details postsecondary enrollment, outcomes, and continuation of vocational
field of study for MA public school graduates who were CVTE concentrators
and who enrolled in a MA public postsecondary institution. (Postsecondary
data source: MA Department of Higher Education)
Massachusetts Department of Elementary & Secondary
Education
38
Student Level Data
Two other reports are available on the Student tab for those who have been
granted access to student-level data:
CR608 Student High School Academic Preparation and Postsecondary
Experience: This source provides student-level high school academic
experience, postsecondary enrollment, and outcomes for students graduating
from MA public schools and enrolling in public or private postsecondary
institutions.. (Postsecondary data source: National Student Clearinghouse)
CR601 Student Progress towards MassCore: While not specific to
postsecondary education, this report displays a list of high school students and
their progress towards completing the MassCore recommended program of
study. It identifies students who are falling behind, enables drill-drown to
investigate course completion for specific students, and is available for all
39
students in grades 9-12.
Massachusetts Department of Elementary & Secondary
Education
For more information…
 Watch the Edwin website for updates, including materials and professional
development opportunities: www.doe.mass.edu/Edwin
 Follow Edwin on Twitter! @ Edwin_DESE
 Questions can be submitted to: Edwin@doe.mass.edu
 If you need access to Edwin Analytics – contact your district directory
administrator (List is available at
http://www.doe.mass.edu/infoservices/data/diradmin/list.aspx)
 EWIS use workshop day: October 1:
http://www.surveygizmo.com/s3/2307782/Early-Warning-IndicatorSystem-EWIS-Workshop-10-1-15
 Early warning implementation guide available at
http://www.doe.mass.edu/edwin/analytics/ewis.html
Massachusetts Department of Elementary & Secondary
Education
40
Success After High School DART Detail
41
Massachusetts Department of Elementary & Secondary
Education
What is the DART?
The District Analysis and Review Tools
(DARTs) turn the Department's vast amount
of data into valuable, easily consumable
information.
The DARTs offer snapshots of district and
school performance, allowing users to
easily track select data elements over time,
and make sound, meaningful comparisons
to the state or to "comparable"
organizations.
DARTs are free and available to the public
on the ESE website
Massachusetts Department of Elementary & Secondary
Education
46
What is the Success After High
School DART?
 The Success After High School Dart is a DART: Detail tool
 DART: Detail tools provide in-depth data on targeted topic
areas
 The Department already released DART Detail’s: Staffing and
Finance and English Language Learners to go along with the
more comprehensive DART tools
 The intent of the Success After High School DART is to synthesize
and summarize existing information available concerning
secondary and post-secondary education in a freely, wellorganized, accessible format for schools, districts, non-profit
organizations, researchers, families and the general public.
Massachusetts Department of Elementary & Secondary
Education
43
Use when
setting
school and
district
goals!
Example elements for DART
Detail: Success After High School
High School Indicators: Graduation, dropout, suspension,
and retention rates.
High School Performance: Students scoring proficient on
MCAS, students scoring on SAT and AP tests, and
participation in SAT and AP tests.
Program of Study: Students completing MassCore and
Advanced Placement coursework. Students passing all
ninth grade courses, Students passing 12th grade
mathematics and science technology/engineering
coursework.
Massachusetts Department of Elementary & Secondary
Education
44
Use when
setting
school and
district
goals!
Example elements for DART Detail:
Success After High School (continued)
Post-Secondary Educational Outcomes: Students
enrolling in post-secondary education, students
persisting in post-secondary education, percentage of
students enrolling in developmental (remedial)
coursework in Massachusetts Public Colleges.
Workforce Development: Students participating
in career/technical education by cluster, Students
completing work based learning plans and Connecting
Activities, links to other regional workforce
development data.
45
Massachusetts Department of Elementary & Secondary
Education
DART for Success After High School is a publicly accessible
tool that provides a snapshot of high school,
postsecondary, and career development education data.
This tool allows users such as district and school staff, parents, students,
workforce partners and organizations to:
 Find comparable high schools based on demographic characteristics
 Look at trends over several years
 View school data in charts that are easy to interpret
 Evaluate and reflect on schools and their progress
46
Massachusetts Department of Elementary & Secondary
Education
Find comparable high schools based
on demographic characteristics
Select your school If desired, select a "comparable"
school, no school or any other school.
Malden - Malden High
Select comparable school
School Overview
*Schools most similar to your district in terms of grade span, total enrollment, and special
populations.
Orange-shaded row: Your school Blue-shaded row: Highest performing of the other 10 schools.
Comparable
schools
School
Gra de
s pa n
MA Level
2013
Brockton - Brockton HS*
Everett - Everett High*
Holyoke - Holyoke HS*
Lowell - Lowell HS*
Lynn - Classical High*
Lynn - Lynn English High*
Malden - Malden High*
Quincy - North Quincy High*
Quincy - Quincy High*
Revere - Revere High*
Worcester - Doherty Memorial HS*
09 - 12
09 - 12
09 - 12
09 - 12
09 - 12
09 - 12
09 - 12
09 - 12
09 - 12
09 - 12
09 - 12
Level 3
Level 3
Level 3
Level 3
Level 1
Level 3
Level 3
Level 1
Level 2
Level 1
Level 3
WIB Regi on
Brockton Area
Metro North
Hampden County
Greater Lowell
North Shore
North Shore
Metro North
South Shore
South Shore
Metro North
Central Mass
2012-13 Enrol l ment
Total
#
4,081
1,931
1,354
3,099
1,528
1,653
1,847
1,222
1,568
1,537
1,333
Hi gh
Needs
%
79.2
76.2
77.4
73.5
83.6
82.0
67.4
59.0
61.4
75.5
67.9
Low
Income
%
SWD
%
ELL
%
75.7
69.0
74.9
68.3
79.8
79.6
59.1
51.0
53.8
71.0
59.3
10.6
12.1
14.3
11.3
13.3
12.8
12.7
12.7
14.3
10.7
15.3
17.2
9.7
10.9
18.3
11.2
10.8
12.5
9.9
7.2
7.5
25.8
Massachusetts Department of Elementary & Secondary
Education
47
Trend over several years with easy to
interpret charts
4-year cohort graduation rate
100%
90%
80%
70%
Your School
60%
50%
Comparable
school
40%
30%
State
20%
10%
48
0%
2008
2009
2010
2011
2012
Massachusetts Department of Elementary & Secondary
Education
“The Waterfall”
49
Massachusetts Department of Elementary & Secondary
Education
And recently released!
100%
State Avg
Cambridge
90%
Needham
80%
70%
60%
50%
40%
30%
20%
10%
0%
2006 Cohort
50
5 Year Grad for 2006
Cohort
Enroll in college
(immediate fall)
Persist 1st-to-second
year*
Graduate PS within
6 years**
Massachusetts Department of Elementary & Secondary
Education
The Success After High School DART is available at
http://www.doe.mass.edu/apa/dart/DARTSAHS.xlsx
It is openly accessible to anyone, however it is a large
file due to all the data in the file, so…
a. it is best to download it to your computer and
b. won’t go through most email systems
Massachusetts Department of Elementary & Secondary
Education
51
Questions or comments?
52
Massachusetts Department of Elementary & Secondary
Education