College and Career
Readiness Webinar
Series:
Tools to be CCR
Data-Wise
September 17, 2013
Massachusetts Department of Elementary & Secondary
Education
Welcome!
Please type your question or comment in the
chat window at any point during the
presentation
All phones will be muted
PowerPoint slides will be sent to all registered
participants
2
Massachusetts Department of Elementary & Secondary
Education
About the C&CR Webinar Series
New for the 2013-14 school year – monthly
webinars on a variety of college and career
readiness topics
Typically held on the 3rd Tuesday of month at
2:00
The October 15th webinar on topic of career
development education
3
Massachusetts Department of Elementary & Secondary
Education
Webinar Agenda
Overviews of two College and Career
Readiness related data tools that were
released last winter through Edwin Analytics:
 Early Warning Indicator System (EWIS) reports
 Postsecondary Readiness and Success reports
Preview of the new College & Career
Readiness DART that will be available soon
Q&A
4
Massachusetts Department of Elementary & Secondary
Education
Edwin Analytics
5
Massachusetts Department of Elementary & Secondary
Education
• Student profiles
• An early warning
indicator system
• Dashboards &
reports
• Tools to assess
student understanding
of multiple standards
• Competency Tracking
System
• Aligned to standards
• Model
curriculum
units
• Curriculum
mapping and
Lesson
planning tools
• Aligned to
Standards
• High-quality
digital
resources to
support
instruction
• Aligned to
standards
6
Massachusetts Department of Elementary & Secondary
Education
What is Edwin Analytics?
What is the SAME: Education Data Warehouse
(EDW) reports are still available
What is NEW and NOW available:
Identify current students in grades 1-12
who may be at risk of missing key educational
targets (Early Warning Indicator System
(EWIS) reports)
Identify how your former students did after
they graduated from your school/district
(Postsecondary Readiness and Success)
Massachusetts Department of Elementary & Secondary
Education
7
Early Warning Indicator System
8
Massachusetts Department of Elementary & Secondary
Education
What are Early Warning Indicators?
Methodological way of looking at students and
their likelihood of meeting specific academic
goals
Traditionally implemented at the high school
level to help predict which students are most
likely to drop out
 Tool for educators to better design and target
student interventions in a timely way
9
Massachusetts Department of Elementary & Secondary
Education
MA EWIS Background
Department previously provided Early Warning
Indicator Index (EWII) to predict the likelihood
grade 9 students would graduate
Department worked with American Institutes
for Research (AIR) to develop the new risk
models
District feedback, e.g., focus groups and
webinars
First rollout of EWIS in 2012-13 school year
Massachusetts Department of Elementary & Secondary
Education
10
Grade Groupings & Goals
Grade
Levels
Academic Goals
(expected student outcomes for each
age group)
Early
Grades
Elementary 1-3
Proficient or advanced on 3rd grade ELA
MCAS
61%
Late
Grades
Elementary 4-6
Proficient or advanced on 6th grade ELA
and Mathematics MCAS
54%
Middle
Grades
Grades
7-9
Passing grades on all 9th grade courses
75%
High
School
Grades
10-12
High school graduation (four-year)
=
Age
Groups
2011-12
Statewide
Results for Goal
84.7%
11
Massachusetts Department of Elementary & Secondary
Education
Key Concepts, continued
 Provides risk level information for all students based
on prior year data
 The risk levels are not based on a student’s relative
likelihood for reaching an academic goal when
compared with other students
 Individual indicators in the risk model are not
reflective of the level of risk - taken together, the
indicators (data points) collectively form a risk level
 Only includes data from existing statewide collections
(SIMS and SCS) and state assessments (MCAS and
ACCESS)
Massachusetts Department of Elementary & Secondary
Education
12
EWIS is / is not…
The Massachusetts EWIS is…
=
A tool to better target interventions
and supports at the individual, small
group, and whole school levels.
A systematic way to flag students that
need to be further examined to
determine what additional support are
necessary.
Related to measures that are included
in the state’s school and district
accountability system (PPI), but is not
itself an accountability measure.
Based on data and risk models that the
Department will reexamine each year
to continually improve the system.
The Massachusetts EWIS is not…
A “life sentence” or a reason to label
students.
A diagnostic tool that provides details
on the exact reason(s) why a student is
at risk for not meeting an expected
academic goal.
An accountability measure.
A stagnant system that will remain
exactly the same in future years.
Massachusetts Department of Elementary & Secondary
Education
13
Edwin Analytics: EWIS Reports
There are three EWIS reports in Edwin:
EWIS Student List (EW601)
EWIS District View (EW301)
EWIS Graphical View (EW302)
All can be filtered numerous way (e.g.,
selecting specific schools, grade levels, student
subgroups)
14
Massachusetts Department of Elementary & Secondary
Education
EWIS Student List (EW601)
What are the current risk levels of my
students?
What are the prior risk levels of my current
students?
What data points (indicators) went into the
calculation to determine the risk level?
15
Massachusetts Department of Elementary & Secondary
Education
16
Massachusetts Department of Elementary & Secondary
Education
EWIS District View (EW301)
Which grade levels or schools have the most
students that need targeted support in order to be
on track to meet the next academic goal?
What is the breakdown of high, moderate, and low
risk levels in a particular student subgroup (e.g.
among students with disabilities)?
Where does the district need to focus additional
resources to get students back on track for meeting
the next academic goal?
17
Massachusetts Department of Elementary & Secondary
Education
18
Massachusetts Department of Elementary & Secondary
Education
EWIS Graphical View (EW302)
How does a selected group of students compare to the
district and state?
What are the previous risk levels of the currently
enrolled students over the past 5 years?
E.g., what are the previous risk levels of my current
4th grade male ELL students?
What are the risk levels of a specific student group
over the past 5 years?
E.g., what are the risk levels of 4th grade male ELL
students each year over the last 5 years?
Massachusetts Department of Elementary & Secondary
Education
19
20
Massachusetts Department of Elementary & Secondary
Education
Postsecondary Outcome Reports
21
Massachusetts Department of Elementary & Secondary
Education
How did our students do after graduating?
Three reports:
Postsecondary Outcomes of Public HS
Graduates:
All Institutions (CR301)
Postsecondary Outcomes of Public HS
Graduates:
MA Public Institutions Only (CR302)
Postsecondary Outcomes of Public HS
Graduates:
Academic Preparation (CR303)
Massachusetts Department of Elementary & Secondary
Education
22
How many public high school
graduates enroll postsecondary?
… at MA public institutions?
23
On top: data from CR301; on bottom: data from CR302
Massachusetts Department of Elementary & Secondary
Education
Where did
students
enroll?
(Anywhere
in the USA)
24
This data from CR301
See CR302 for MA Public Institutions Only
Massachusetts Department of Elementary & Secondary
Education
How many return for a 2nd year?
What does
persistence
look like
across
subgroups?
25
Data from CR301; See CR302 for MA Public Institutions Only
Massachusetts Department of Elementary & Secondary
Education
How many of your graduates
obtain a degree?
 How does that vary by subgroup?
 How does that vary by the type of institution where
students enroll? (4-year, 2-year, public, or private)
26
Data from CR301; See CR302 for MA Public Institutions Only
Massachusetts Department of Elementary & Secondary
Education
What are enrollment rates of students
who complete AP or MassCore?
27
Data from CR301; See CR302 for MA Public Institutions Only
Massachusetts Department of Elementary & Secondary
Education
Which type of institutions do they
attend?
28
Data from CR303
Massachusetts Department of Elementary & Secondary
Education
Of those in MA public postsecondary,
what % enroll in remedial courses?
In which
subjects?
29
Data from CR302
Massachusetts Department of Elementary & Secondary
Education
What are the MCAS scores of
students who take remedial courses?
30
Data from CR302
Massachusetts Department of Elementary & Secondary
Education
How many credits do graduates
earn in their first year of college?
What about developmental credits?
31
Data from CR302
Massachusetts Department of Elementary & Secondary
Education
… and this is just the beginning of
what we can do!
We could also explore…
What is the relationship between
MCAS scores and the type of institution where
students enroll? (4-year, 2-year, public, or private)
What is the postsecondary experience of students
from a specific subgroup (CVTE, ELL, students with
disabilities, high needs, etc)?
32
Massachusetts Department of Elementary & Secondary
Education
Comparing the Three Reports
Potential Questions
How many graduates go on to postsecondary?
What type of institutions (4-year/2-year, private/public) do
graduates attend?
Which specific institutions did students attended?
How many went to each?
What are the enrollment rates of students who complete MassCore
or AP coursework compared to those who don’t?
What are the persistence and degree obtainment rates of students
who complete MassCore or AP coursework?
What is the postsecondary experience of students from a specific
subgroup? (CVTE, ELL, SWD, high needs, etc)
What percentages of students enroll in and complete remedial
courses?
In what remedial subjects to do students enroll?
What are the MCAS scores of students who take remedial
coursework in math and reading?
How many credits, on average, do public high school graduates earn
in their first year of college?
CR301 CR302 CR303
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
33
Massachusetts Department of Elementary & Secondary
Education
Refining your search
34
34
Massachusetts Department of Elementary & Secondary
Education
For more information…
 Watch the Edwin website for updates, including
materials and professional development opportunities:
www.doe.mass.edu/Edwin
 Follow Edwin on Twitter! @ Edwin_DESE
 Questions can be submitted to: Edwin@doe.mass.edu
 If you need access to Edwin Analytics – contact your
district directory administrator (List is available at
http://www.doe.mass.edu/infoservices/data/diradmin/l
ist.aspx)
Massachusetts Department of Elementary & Secondary
Education
35
Coming soon:
Success After High School DART Detail
36
Massachusetts Department of Elementary & Secondary
Education
What is the DART?
The District Analysis and Review Tools
(DARTs) turn the Department's vast
amount of data into valuable, easily
consumable information.
The DARTs offer snapshots of district
and school performance, allowing users
to easily track select data elements over
time, and make sound, meaningful
comparisons to the state or to
"comparable" organizations.
DARTs are free and available to the
public on the ESE website
Massachusetts Department of Elementary & Secondary
Education
37
What is the Success After High School
DART?
The Success After High School Dart is a DART: Detail
tool
 DART: Detail tools provide in-depth data on targeted topic
areas
 The Department already released DART Detail’s: Staffing and
Finance and English Language Learners to go along with the
more comprehensive DART tools
The intent of the Success After High School DART is to
synthesize and summarize existing information
available concerning secondary and post-secondary
education in a freely, well-organized, accessible format
for schools, districts, non-profit organizations,
researchers, families and the general public.
Massachusetts Department of Elementary & Secondary
Education
38
Example elements for DART Detail:
Success After High School
High School Indicators: Graduation, dropout,
suspension, and retention rates.
High School Performance: Students scoring
proficient on MCAS, students scoring on SAT and AP
tests, and participation in SAT and AP tests.
Program of Study: Students completing MassCore
and Advanced Placement coursework. Students
passing all ninth grade courses, Students passing 12th
grade mathematics and science
technology/engineering coursework.
Massachusetts Department of Elementary & Secondary
Education
39
Example elements for DART Detail:
Success After High School (continued)
Post-Secondary Educational Outcomes: Students
enrolling in post-secondary education, students
persisting in post-secondary education, percentage of
students enrolling in developmental (remedial)
coursework in Massachusetts Public Colleges.
Workforce Development: Students participating in
career/technical education by cluster, Students
completing work based learning plans and Connecting
Activities, links to other regional workforce
development data.
40
Massachusetts Department of Elementary & Secondary
Education
Look for the Success After High School
DART on November 1, 2013!
Massachusetts Department of Elementary & Secondary
Education