Massachusetts Mathematics and Science Partnership Title IIB
Annual State-level Evaluation Report (Year 11)
Reporting Period: September 1, 2013 through August 31, 2014
Cumulative Reporting Period: February 2, 2004 through August 31, 2014
Prepared for the Massachusetts Department of Elementary and
Secondary Education
January 2015
MMSP State-level Evaluation (Year 11)
Contents
Contents
Introduction ................................................................................................................................................. 1
Program Overview ................................................................................................. 1
Evaluation Plan and Activities ................................................................................ 2
Report Organization ............................................................................................... 4
Year 11 Project Activity .............................................................................................................................. 5
State-level Participant Background Data ............................................................... 5
Partnership-level Participant Background Data ................................................... 15
Cumulative Summary: All Cohorts over All Funding Periods .............................................................. 21
Summary of Findings ............................................................................................................................... 37
Cumulative Findings ............................................................................................ 37
Year 11 Findings .................................................................................................. 39
Appendix A: Participant Background Survey – Year 11 ....................................................................... 41
Appendix B: Timeline for State-level Evaluation and TA Activities ..................................................... 49
Appendix C: Year 11 Participant Background Survey Results ............................................................ 52
Appendix D: High Need District Eligibility Criteria ................................................................................ 58
Appendix E: Enrollment and Attrition Rates by Course ....................................................................... 68
Appendix F: Mean Percentage Scores for Pre- & Post-course Tests .................................................. 69
Appendix G: High Need Districts for All Funding Periods, by Partnership ........................................ 70
UMass Donahue Institute
Research and Evaluation Group
I
MMSP State-level Evaluation (Year 11)
Tables
Tables Index
Table 1: Partnership Budgets (Year 11) ................................................................................................................ 5
Table 2: Professional Position of Participants..................................................................................................... 6
Table 3: Teaching Content Areas of Participants ................................................................................................ 6
Table 4: Experience of Participants ...................................................................................................................... 7
Table 5: Teaching Level of Participants ............................................................................................................... 7
Table 6: Participants by Type of Schools ............................................................................................................. 7
Table 7: Public School Participants by High Need District Status .................................................................... 8
Table 8: High Need District Participants by Partnership .................................................................................... 8
Table 9: Reasons for Participation, All Seats..................................................................................................... 10
Table 10: Repeat Participants .............................................................................................................................. 10
Table 11: Total Enrollment and Attrition Information ........................................................................................ 11
Table 12a: Science and Technology Teaching Areas – Regular Education .................................................... 12
Table 12b: Science and Technology Teaching Areas – Special Education ..................................................... 12
Table 12c: Science and Technology Teaching Areas – ELL Education ........................................................... 12
Table 13a: Mathematics Teacher Levels – Regular Education.......................................................................... 13
Table 13b: Mathematics Teacher Levels – Special Education .......................................................................... 13
Table 13c: Mathematics Teacher Levels – ELL Education ................................................................................ 14
Table 14: Pursuit of Degrees ............................................................................................................................... 14
Table 15: Number of Courses with Statistically Significant Gains in Mean Content Knowledge Scores.... 15
Table 16: Participant Background Info: EduTron – Revere-Everett-Saugus (Cohort 6 – Math) ................... 15
Table 17: Participant Background Info: Lesley – Brockton (Cohort 6 – Math) ............................................... 16
Table 18: Participant Background Info: Lesley – Springfield (Cohort 6 – Math) ............................................ 16
Table 19: Participant Background Info: MCLA (Cohort 6 – Math/Science) .................................................... 17
Table 20: Participant Background Info: Northeastern (Cohort 6 – Science) .................................................. 17
Table 21: Participant Background Info: Fitchburg – NCIS (Cohort 7 – Math) ................................................. 18
Table 22: Participant Background Info: Framingham (Cohort 7 – Science) ................................................... 18
Table 23: Participant Background Info: Global Learning Public Charter (Cohort 7 – Science) .................... 19
Table 24: Participant Background Info: Lesley – Springfield-Easthampton (Cohort 7 – Math) ................... 19
Table 25: Participant Background Info: Worcester Public Schools (Cohort 7 – Math) ................................ 20
Table 26: Overview of MMSP Partnership Participation: Cohort 1–Cohort 3 ................................................. 21
Table 27: Overview of MMSP Partnership Participation: Cohort 4–Cohort 7 ................................................. 22
Table 28: Budgets: All Partnerships, All Funding Periods ............................................................................... 23
Table 29: Professional Position of Participants – Cumulative ......................................................................... 25
Table 30: Teaching Areas: All Participants, All Funding Periods .................................................................... 25
Table 31. Types of Schools: All Unique Participants, All Funding Periods .................................................... 26
Table 32: High Need Status of All Unique Participants from Public Schools ................................................. 27
Table 33: Reasons for Participation of Participants: All Seats, All Funding Periods .................................... 28
Table 34: Repeat Participants – All Partnerships, All Funding Periods .......................................................... 28
Table 34: Repeat Participants – All Partnerships, All Funding Periods (Continued) .................................... 29
Table 35: MTEL Tests Taken by All Participants – Total to Date ..................................................................... 30
Table 36a: Science and Technology Teaching Areas – Regular Education .................................................... 31
Table 36b: Science and Technology Teaching Areas – Special Education ..................................................... 31
Table 36c: Science and Technology Teaching Areas – ELL Education ........................................................... 31
Table 37a: Mathematics Teacher Levels – Regular Education.......................................................................... 32
Table 37b: Mathematics Teacher Levels – Special Education .......................................................................... 32
UMass Donahue Institute
Research and Evaluation Group
II
MMSP State-level Evaluation (Year 11)
Tables
Table 37c: Mathematics Teacher Levels – ELL Education ................................................................................ 32
Table 38: Pursuit of Degrees ............................................................................................................................... 33
Table 39: Number of Courses with Statistically Significant Gains in Mean Content Knowledge Scores.... 34
UMass Donahue Institute
Research and Evaluation Group
III
MMSP State-level Evaluation (Year 11)
Introduction
Introduction
Program Overview
The purpose of the Massachusetts Mathematics and Science Partnership Program (MMSP) is to improve student
achievement in mathematics, science, and technology/engineering through intensive, high-quality professional
development activities that focus on deepening teachers’ content knowledge. This multi-year project is funded by
the U.S. Department of Education (USED) as part of the No Child Left Behind (NCLB) Act Title IIB funding
stream. Funding to local partnerships is administered by state education agencies; in Massachusetts this is the
Massachusetts Department of Elementary and Secondary Education (ESE), which awards funding through a
competitive grant process.
Partnerships awarded these grants are required to include 1) a core partner that has been identified as a high need
school district in the subject matter on which the partnership is focusing and 2) a core partner that is a science,
technology and/or engineering, or mathematics (STEM) department from an institution of higher education. The
partnerships are composed of higher education institutions, school districts, and, in some cases, private
organizations involved in providing both pre-service and in-service training to teachers. Partnership activities are
guided by the following goals and objectives:
Goal I
Develop and implement an effective and sustained course of study for in-service educators of STEM by
integrating the courses of study into schools of arts and sciences and/or education at institutions of
higher education.
Goal II
Identify credible, instructionally useful measures of student growth and select, develop and pilot
District Determined Measures (DDMs) that measure student growth relative to subject matter
standards1.
Goal III Increase the number of STEM teachers in the partner school districts who participate in effective
professional development and advance their knowledge of subject matter standards, disciplinary
practices, and student learning.
Goal IV Develop and implement a systemic approach to STEM education by integrating professional
development with district and school STEM improvement initiatives.
All professional development funded by the program must be implemented in accordance with the Massachusetts
Standards for Professional Development. Those standards encourage a set of coherent learning experiences that is
systematic, purposeful, and structured over a sustained period of time with the goal of improving teacher practice
and student outcomes. Each course offered by the cohort 7 partnerships must provide at least 45 hours of direct
instruction followed by 24 hours of supplemental activities to guide the implementation of standards-based
instruction and facilitate connections between the course(s) and DDMs2. Partnerships are encouraged to tailor the
model used to deliver the professional development and follow-up to best fit the objectives of their programs
along with their resources, expertise, and existing infrastructure. At least half of participating educators must be
from high need districts.
The program began in February 2004, and has had eleven funding periods, defined as follows:

1
2
Year 1: February 2, 2004 through August 31, 2004 (initial funding for Cohort 1)
This goal was introduced beginning with cohort 7
Prior to cohort 7, the follow-up requirement was 20 hours.
UMass Donahue Institute
Research and Evaluation Group
1
MMSP State-level Evaluation (Year 11)










Introduction
Year 2: September 1, 2004 through August 31, 2005 (initial funding for Cohort 2)
Year 3: September 1, 2005 through August 31, 2006
Year 4: September 1, 2006 through August 31, 2007 (initial funding for Cohort 3)
Year 5: September 1, 2007 through August 31, 2008
Year 6: September 1, 2008 through August 31, 2009 (initial funding for Cohort 4)
Year 7: September 1, 2009 through August 31, 2010
Year 8: September 1, 2010 through August 31, 2011 (initial funding for Cohort 5)
Year 9: September 1, 2011 through August 31, 2012 (initial funding for Cohort 6)
Year 10: September 1, 2012 through August 31, 2013
Year 11: September 1, 2013 through August 31, 2014 (initial funding for Cohort 7)
Evaluation Plan and Activities
The UMass Donahue Institute (UMDI) has served as the state-level evaluator of MMSP since the program’s
inception. To date, UMDI’s primary role has been to coordinate program-wide collection of outcome data on
behalf of the ESE. Previously, each partnership was required to conduct its own local evaluation with UMDI
providing limited technical assistance on design and implementation. However, beginning with Cohort 7, ESE
has selected UMDI to conduct both statewide and local evaluation research activities for the program.
The statewide and local evaluations will be guided by the following evaluation goals:
1. Facilitate partnership’s annual federal reporting and provide ESE with a statewide report of the data required
for federal reporting.
2. Provide ESE and partnerships with timely formative feedback to support program implementation and
refinement.
3. Assess statewide and partnership progress toward achievement of the program’s articulated goals.
4. Track changes in the achievement of middle school students 3taught by MMSP participants.
3
Although some partnerships may also include support for elementary and/or high school students, the focus of the program
has been defined as improving student achievement at the middle school level. Accordingly the evaluation is focused on
middle school (grades 6-8) where funding is consistently directed across partnerships. UMDI will provide technical
assistance to support individual partnerships’ analysis of DDMs for other levels as needed.
UMass Donahue Institute
Research and Evaluation Group
2
MMSP State-level Evaluation (Year 11)
Introduction
Data collection for the evaluation is organized around a basic logic model for professional development
initiatives, as shown below.
The statewide and local evaluations utilize a common approach and structure drawing on a core data sources
described in more detail below.
End-of-course report packages
Over the years, UMDI and ESE have established an ongoing reporting system for all partnerships, consisting
primarily of end-of-course summary report packages. Those report packages include: course enrollment and
completion rates; email addresses for all enrollees; individual pre/post-results of the required participant content
knowledge assessment; and completed course participant background surveys. In cases where partnerships are
providing data on individual participants (pre/post-results and the background surveys) those data are collected
using a prescribed individual coding system which allows data to be linked anonymously across various
instruments and program years.
Beginning in Year 12, UMDI is significantly revising this approach to collecting participant background
information and will rely on existing databases maintained by ESE. (The existing survey pre-dated the existence
of those databases). The end-of-course survey has been revised to focus more on gathering participant’s feedback
about their course experience.
Partnership Interviews
Interviews with the core partners of each grant will focus on gathering data relative to evaluation goals 2 (timely
formative feedback) and 3 (progress toward meeting the program goals). Other significant partners will be added
at the discretion of the evaluation team with input from the partnership and ESE.
Annual Participant Survey
In addition to end-of-course surveys, UMDI will develop and administer an annual survey of MMSP course
participants. This survey will provide an opportunity to gather participant feedback on the partnership’s
professional development program more broadly, including the follow-up component and how the individual
UMass Donahue Institute
Research and Evaluation Group
3
MMSP State-level Evaluation (Year 11)
Introduction
course offerings fit together into a coherent course of study. Participants will also be able to reflect upon the
extent to which their participation impacted their classroom practice.
District Determined Measures
UMDI will work with each partnership to fully understand the DDMs that have been selected to measure student
growth and develop a process through which those partnerships can provide middle school-level data for analysis
in relation to evaluation goal 4 (tracking changes in student achievement). This will result in DDM analysis
reports for each partnership as well as a statewide summary report for ESE. In cases where partnerships are also
serving elementary and/or high school students, UMDI will provide technical assistance to support the
partnerships’ analysis of DDMs for those levels.
A timeline listing the evaluation activities for Year 11 can be found in Appendix B.
Report Organization
The purpose of this report is twofold: 1) to provide details regarding participation for the 2013–2014 funding
period; and 2) to provide a cumulative summary regarding participation for all cohorts over all funding periods.
Data supporting the first purpose address the period of September 1, 2013 through August 31, 2014, and data
supporting the second purpose address the period February 2, 2004 through August 31, 2014. Participant data
were collected through the Participant Background Survey, an instrument developed by the UMass Donahue
Institute (UMDI) and administered by partnerships to each participant on the last day of each course. See
Appendix A for the survey used during Year 11. The purpose of this survey is to gather data about participants’
professional backgrounds and qualifications. This information provides a description of the participants, aids in
determining whether courses are reaching the teachers who most need professional development, and aids in
tracking how teacher qualifications change during the MMSP funding period. Unless noted, data from the survey
are unduplicated, meaning that they are reported in terms of unique individuals, regardless of the number of
courses taken by each individual.
Data related to strengthening of relationships between partnership members were collected through a section of
the local evaluation reports that partnerships were required to submit to ESE. Specifically, partnerships were
asked to describe the extent to which their courses had been integrated into activities of their higher education
partners.
UMass Donahue Institute
Research and Evaluation Group
4
MMSP State-level Evaluation (Year 11)
Year 11 Project Activity
Year 11 Project Activity
During Year 11, five Cohort 6 partnerships completed their third year, three offering mathematics professional
development, one offering science professional development, and one offering both math and science professional
development. The Year 11 funding period provided funding for six new partnerships as well, and of the six
funded, five completed courses during Year 11. Of the five partnerships that ran courses, three offered
mathematics professional development and two offered science professional development. While West
Springfield Public Schools was funded in Year 11 and began a course in August, the course was not completed
before October 1, 2014. Table 1 shows the funding received by each partnership for the period beginning in
September 2013 and ending in August 2014.
Table 1: Partnership Budgets (Year 11)
Partnership
Sep13-Aug14
Cohort 6 – Math
EduTron – Revere-Everett-Saugus
249,982
Cohort 6 – Math
Lesley – Brockton
314,383
Cohort 6 – Math
Lesley – Springfield
274,930
Cohort 6 – Math/Science
MCLA
Cohort 6 – Science
Northeastern
199,991
Cohort 7 – Math/Science
Fitchburg – North Central Integrated STEM (NCIS)
$71,404
Cohort 7 – Science
Framingham
$55,839
Cohort 7 – Science
Global Learning Public Charter
$75,207
Cohort 7 – Math
Lesley – Springfield-Easthampton
$73,818
Cohort 7 – Science
West Springfield Public Schools
$49,553
Cohort 7 – Math
TOTAL
Worcester Public Schools
17,326
$51,753
1,416,860
State-level Participant Background Data
In Year 11, there were 33 courses delivered in 2013–2014 – 25 mathematics courses and 8 science courses. Of
those 33 courses, 29 were unique, and 4 were repeat offerings. During Year 11, 328 unique participants completed
the Participant Background Survey at least once. Of those participants, 89 took two or more courses in 2013–
2014. The term “unique participant” refers to each individual who participated in the program, regardless of the
number of courses taken. Data for items from the survey that convey participants’ professional backgrounds and
motives for participation are presented in this section. Additional survey data are presented in Appendix C.
Responses to survey questions are presented as frequencies and percentages. Not all percentages total 100% as
many items allowed multiple responses and not all participants responded to all items.
Professional Position of Participants
As shown in Table 2, the vast majority of the 328 participants were teachers, predominantly regular education
teachers. Only 1% were math coaches and 2% were department heads or curriculum coordinators.
UMass Donahue Institute
Research and Evaluation Group
5
MMSP State-level Evaluation (Year 11)
Year 11 Project Activity
Table 2: Professional Position of Participants
Sep13–Aug14
Professional Position (N=327)
(Only one response permitted)
Teacher
Regular Education
Special Education or Special Education Inclusion
ELL or Sheltered English Immersion
OtherTeachers
Math Coach (Teaching and Non-Teaching)
Department Head or Curriculum Coordinator
Principal, Assistant Principal or Headmaster
Other (including no response)
n
%
305
247
44
8
5
4
8
0
11
93
76
13
2
2
1
2
0
3
Teaching Content Areas
The distribution of teaching areas of respondents at the time of the survey is shown in Table 3. Because
respondents identified all teaching areas that applied to their positions at the time of the survey, some selected
multiple responses, so frequency totals exceed the number of respondents and percentages exceed 100%. At the
time of their last Year 11 MMSP course, 79% of participants were teaching mathematics (at any level), 42% were
teaching science, and 18% were teaching all subjects at the elementary level.
Table 3: Teaching Content Areas of Participants
Sep13–Aug14
Teaching Areas (N=327)
(Multiple responses permitted)
Mathematics
Any science area
General Science
Biology
Earth Science
Chemistry
Physics
Technology/Engineering
Elementary (all subjects)
Elementary Mathematics
Other
Not currently teaching
n
%
258
137
40
18
11
20
20
10
59
10
4
1
79
42
12
6
3
6
6
3
18
3
1
<1
Teaching Experience
As shown in Table 4, approximately two-thirds of respondents had six or more years of experience. Roughly onequarter had been teaching between two and five years and six percent were in their first year of teaching.
UMass Donahue Institute
Research and Evaluation Group
6
MMSP State-level Evaluation (Year 11)
Year 11 Project Activity
Table 4: Experience of Participants
Sep13–Aug14
Year Employed in Education (N=327)
(Only one response permitted)
n
%
More than 20 years
11-20 years
6-10 years
4-5 years
2-3 years
First year
No response
35
85
97
40
45
19
6
11
27
30
13
14
6
2
Teaching Levels
For reporting purposes, schools in the participating districts were organized into categories of elementary schools
(grades K-5), middle schools (grades 6-8), K-8 schools, and high schools (grades 9-12). As shown in Table 5,
23% of Year 11 participants were teaching in an elementary or K-8 school, 54% were teaching in a middle school,
22% were teaching in a high school.
Table 5: Teaching Level of Participants
What grades do you currently teach?
(N=327)
(Coded as one response per individual)
Elementary or K-8
Middle School (Grades 6-8)
High School (Grades 9-12)
Middle and High School Grades
Not currently teaching
No response
Sep13–Aug14
n
%
76
174
71
1
0
3
23
54
22
<1
0
1
Types of Schools
As shown in Table 6, 97% of unique 2013–2014 participants worked in a public school setting, and 2% worked in
a non-public school setting.
Table 6: Participants by Type of Schools
Sep13–Aug14
What type of school are you employed in? (N=327)
Public School (includes public charter schools)
Non-public School
Not reported
n
%
318
8
1
97
2
<1
High Need Status of Districts
MMSP partnerships were required to include at least one high need district. Appendix D identifies the criteria for
the high need designation and includes lists of public school districts qualifying as high need through 2014. ESE
expected that at least 50% of participants in each partnership would be from high need districts, and further, set an
informal goal that at least 75% of participants in each partnership would be from high need districts. Table 7
UMass Donahue Institute
Research and Evaluation Group
7
MMSP State-level Evaluation (Year 11)
Year 11 Project Activity
shows that of all Year 11 participants, 78% came from high need districts and 81% of participants from public
schools had come from high need districts.
Table 7: Public School Participants by High Need District Status
Sep13–Aug14
High-Need Status of Participants (N=327)
High Need District
Non-high Need District/Unknown
n
%
256
71
78
22
256
62
81
20
High-Need Status of Public School Participants (N=318)
Public School High Need District
Non-high Need District/Unknown
Table 8 presents the number of participants from high need districts organized by partnership. An examination of
high need district participation in individual partnerships reveals that during the Year 11, eight out of ten
partnerships met or exceeded the informal goal of having at least 75% of the participants come from high need
districts. Furthermore, all participants were from high needs districts in four of the partnerships.
Table 8: High Need District Participants by Partnership
Sep13–Aug14
Partnership
Cohort 6 (M)
EduTron – Revere-Everett-Saugus
High Need District
n
%
Dracut
Everett
1
1
37
Haverhill
Hudson
Lowell
Lynn
Medford
New Bedford
Revere
Southbridge
Cohort 6 (M)
Cohort 6 (M)
Lesley – Brockton
Lesley – Springfield
UMass Donahue Institute
Research and Evaluation Group
25
1
1
1
4
3
1
11
1
1
1
1
1
6
4
1
16
1
Springfield
Subtotal
50
1
75
Total in Partnership
67
100
Brockton
Fall River
54
1
64
1
Quincy
Randolph
9
1
11
1
Stoughton
Weymouth
1
6
1
7
Subtotal
72
86
Total in Partnership
84
100
Easthampton
1
2
Franklin County Reg Voc
Holyoke
1
7
2
13
Monson
Springfield
1
30
2
55
Westfield
1
2
8
MMSP State-level Evaluation (Year 11)
Year 11 Project Activity
Table 8: High Need District Participants by Partnership
Sep13–Aug14
Partnership
Lesley-Springfield (continued)
Cohort 6 (M/S)
MCLA
High Need District
n
%
West Springfield
Holyoke Community CS
2
1
4
2
Subtotal
44
80
Total in Partnership
55
100
Adams-Cheshire
Berkshire Arts CS
2
25
50
North Adams
Subtotal
Cohort 6 (S)
Cohort 7 (M/S)
Cohort 7 (S)
Cohort 7 (S)
Cohort 7 (M)
Cohort 7 (M)
Northeastern
Fitchburg – NCIS
Framingham
Global Learning Public Charter
Lesley – Springfield-Easthampton
Worcester Public Schools
4
1
7
13
88
Total in Partnership
8
100
Boston
Quincy
8
1
21
3
Somerville
Stoughton
1
1
3
3
Weymouth
Subtotal
1
12
3
32
Total in Partnership
38
100
Fitchburg
Leominster
4
5
25
31
Winchendon
Narragansett
4
3
25
19
Subtotal
16
100
Total in Partnership
16
100
Framingham
Marlborough
8
53
47
Subtotal
7
15
100
Total in Partnership
15
100
Wareham
15
60
Global Learning CS
Subtotal
10
25
40
100
Total in Partnership
25
100
Easthampton
Springfield
1
5
10
50
Subtotal
6
60
Total in Partnership
10
100
Worcester
Subtotal
9
9
100
100
Total in Partnership
9
100
West Springfield Public Schools is not listed in this table because no courses were offered in Y11.
UMass Donahue Institute
Research and Evaluation Group
9
MMSP State-level Evaluation (Year 11)
Year 11 Project Activity
Reasons for Participation in MMSP Courses
For each course taken, respondents identified all of their reasons for participating. Unlike in the earlier sections of
this report where data were presented for unique participants, data for this topic are presented for all course seats,
since responses are uniquely relevant to each distinct course that a participant took. Table 9 presents findings for
all 500 seats for courses taken by participants during the Year 11 funding period. The most common reasons for
participating in a course were to obtain graduate credit (75%) and to increase knowledge in content (64%). Also,
nearly one-third took a course to pursue a personal interest.
Table 9: Reasons for Participation, All Seats
Sep12–Aug13
Reasons for Participation (N=500)
(Multiple responses permitted)
To obtain graduate credit
To increase knowledge in content
To pursue a personal interest
To earn PDPs for recertification
To get an additional license (certification)
To prepare for the Massachusetts Test for Educator Licensure (MTEL)
To follow an administrator’s suggestion
To obtain a first license (certification)
Other
N
%
376
321
153
111
68
51
29
18
35
75
64
31
22
14
10
6
4
7
Repeat Participation
Six of the eleven partnerships offered at least two courses in Year 11, each with some participants attending more
than one course. Table 10 presents information regarding repeat participation. (Tables 16 through 25 also include
data on repeat participation, as does Table 33.)
Table 10: Repeat Participants
Sep13–Aug14
Number of
Courses Offered
Partnership
Number of
Unique
Participants
Participants
Taking Multiple
Courses
Cohort 6 – Math
EduTron – Revere-Everett-Saugus
3
67
7
Cohort 6 – Math
Lesley – Brockton
10
84
28
Cohort 6 – Math
Lesley – Springfield
8
55
37
Cohort 6 – Math/Science
MCLA
2
8
0
Cohort 6 – Science
Northeastern
4
38
7
Cohort 7 – Math/Science
Fitchburg – NCIS
1
16
N/A
Cohort 7 – Science
Framingham
1
15
N/A
Cohort 7 – Science
Global Learning Public Charter
1
25
N/A
Cohort 7 – Math
Lesley – Springfield-Easthampton
2
10
2
Cohort 7 – Science
West Springfield Public Schools
0
N/A
N/A
Cohort 7 – Math
Worcester Public Schools
1
9
N/A
33
327
81
TOTAL
UMass Donahue Institute
Research and Evaluation Group
10
MMSP State-level Evaluation (Year 11)
Year 11 Project Activity
Attrition
Partnerships offered a total of 33 courses in Year 11, with an overall participant attrition rate of 4%. Of the 33
courses, only one (Global Learning Public Charter) had no attrition, four had attrition rates between 1% and 7%,
and five had attrition rates between 11% and 17%. Table 11 provides a breakdown, by partnership, of enrollment
and attrition rates. Appendix E provides a breakdown, by course, of enrollment and attrition rates.
Table 11: Total Enrollment and Attrition Information
Cohort 6 – Math
Cohort 6 – Math
Cohort 6 – Math
Cohort 6 – Math/Science
Cohort 6 – Science
EduTron – Revere-Everett-Saugus
Lesley – Brockton
Lesley – Springfield
MCLA
Northeastern
3
10
8
2
4
Number of
Participants
Enrolled First
Day
243
339
305
45
251
Cohort 7 – Math/Science
Cohort 7 – Science
Cohort 7 – Science
Cohort 7 – Math
Cohort 7 – Science
Cohort 7 – Math
All Courses/Partnerships
Fitchburg – NCIS
Framingham
Global Learning Public Charter
Lesley – Springfield-Easthampton
West Springfield Public Schools
Worcester Public Schools
1
1
1
2
0
1
18
18
25
17
N/A
12
16
15
25
15
N/A
10
11%
17%
0%
13%
N/A
17%
33
1,273
1,219
4%
Number of
Courses
Offered
Partnership
Number of
Participants
Completed
Course
238
328
298
39
235
Attrition Rate
2%
3%
2%
13%
7%
Licensure and Degrees Held in Subjects Taught
During Year 11, 90 regular education teachers, 13 special education teachers, and 7 ELL teachers reported
teaching subjects in science or technology/engineering. (These figures include elementary teachers who indicated
that they teach science.) Tables 12a, 12b, and 12c show the number of participants teaching each science or
technology/engineering subject during the 2013–2014 funding period. The tables also show the percentages of
teachers who indicated they were licensed in the subject they taught, and the percentages of teachers who
indicated they held a degree in the subject they taught.
To see the extent to which licensure and degree were appropriate for a particular type of teacher for a particular
subject, look in the relevant table at data in the row corresponding to the subject of interest. For example, to see
the extent to which licensure and degree were appropriate for just regular education biology teachers, look at
Table 12a in the row for the subject area “Biology” to learn that 13 teachers taught regular education biology and
46% of them were licensed in biology and 54% held degrees in biology.
Among regular education teachers, licenses were held by nearly half of the teachers in three subjects: general
science, biology, and chemistry. Thirty-eight percent of regular education teachers held a license in physics and
technology/engineering. Only in biology did more than half of the teachers hold a degree in the subject taught.
UMass Donahue Institute
Research and Evaluation Group
11
MMSP State-level Evaluation (Year 11)
Year 11 Project Activity
Table 12a: Science and Technology Teaching Areas – Regular Education
Sep13–Aug14
Subject
General Science
Biology
Chemistry
Physics
Earth Science
Technology/Engineering
Number of
Teachers
License in Subject
Taught
Degree in Subject
Taught
n
n*
%*
n*
%*
30
13
15
16
8
8
14
6
7
6
0
3
47
46
47
38
0
38
3
7
4
3
0
2
10
54
27
19
0
25
*Of the number of regular education participants teaching this subject for this period
Table 12b provides information for special education teachers, and Table 12c provides information for ELL
teachers. (The numbers in these tables exceed the number of teachers who reported teaching these subjects
because some teachers taught more than one subject.) Across all science subject areas for special education
teachers, the one teacher who taught chemistry was licensed in that subject. One of the three special education
teachers teaching physics was licensed and one of the five teachers teaching general science was licensed.
Among the seven ELL teachers, in both the general science and chemistry subjects, one of two teachers held a
license and had a degree. Both teachers teaching physics and earth science had a degree in the subject as well.
Table 12b: Science and Technology Teaching Areas – Special Education
Sep13–Aug14
Subject
General Science
Biology
Chemistry
Physics
Earth Science
Technology/Engineering
Number of
Teachers
License in Subject
Taught
Degree in Subject
Taught
n
n*
%*
n*
%*
5
3
1
3
0
1
1
0
1
1
0
0
20
0
100
33
0
0
0
0
0
0
0
0
0
0
0
0
0
0
*Of the number of special education participants teaching this subject for this period
Table 12c: Science and Technology Teaching Areas – ELL Education
Sep13–Aug14
Subject
General Science
Biology
Chemistry
Physics
Earth Science
Technology/Engineering
Number of
Teachers
License in Subject
Taught
Degree in Subject
Taught
n
n*
%*
n*
%*
2
1
2
1
1
0
1
0
1
0
0
0
50
0
50
0
0
0
1
0
1
1
1
0
50
0
50
100
100
0
*Of the number of ELL education participants teaching this subject for this period
UMass Donahue Institute
Research and Evaluation Group
12
MMSP State-level Evaluation (Year 11)
Year 11 Project Activity
During Year 11, 156 regular education teachers 22 special education teachers, and 4 ELL teachers reported
teaching mathematics. Tables 13a, 13b, and 13c show how many teachers taught at each mathematics level during
the 2013–2014 funding period. The tables also show the percentages of teachers who indicated they were licensed
in the mathematics level at which they taught, and the percentages of teachers who indicated they held a degree in
mathematics.
To see the extent to which licensure and degree were appropriate for a particular type of teacher for a particular
mathematics level taught, look in the relevant table at data in the row corresponding to the level of interest and in
the column corresponding to the time period of interest. For example, to see the extent to which licensure and
degree were appropriate for just regular education middle school math teachers, look at Table 13a in the row for
the level “Middle School” to learn that 104 teachers taught regular education middle school mathematics and that
77% of them were licensed to teach middle school mathematics and 39% held mathematics degrees.
From an “ease-of-reading perspective,” it would be ideal merely to report on the percentages of teachers who
were licensed to teach mathematics and the percentage of teachers who held mathematics degrees. Sometimes,
though, teachers hold multiple positions and may only be licensed to teach some of what they are teaching, and
what is really desired is to understand the extent to which what is being taught is done so by individuals licensed
to teach it and by individuals who hold relevant degrees. As a result, the explanations of data that follow are
presented in terms of teaching positions.
For MMSP mathematics teaching positions in regular education, the licensing appeared to be appropriate for
approximately 77% of positions held, and 46% of the positions were held by individuals with mathematics
degrees. The alignment was strongest for high school-level positions followed by middle school-level positions.
None of the special education and only one of the ELL teachers held degrees in math. Licensing appeared to be
appropriate for 18% of the positions held by special education teachers and 75% of the ELL teachers.
Table 13a: Mathematics Teacher Levels – Regular Education
Sep13–Aug14
Level
Elementary School
Middle School
High School
Number of
Teachers
License at Level
Degree in Math
n
n*
%*
n*
%*
12
103
41
2
79
39
17
77
95
1
39
31
8
38
76
*Of the number of regular education participants teaching mathematics at this level for this period
Table 13b: Mathematics Teacher Levels – Special Education
Sep13–Aug14
Level
Elementary School
Middle School
High School
Number of
Teachers
License at Level
Degree in Math
n
n*
%*
n*
%*
1
18
3
0
3
1
0
17
33
0
0
0
0
0
0
*Of the number of special education participants teaching mathematics at this level for this period
UMass Donahue Institute
Research and Evaluation Group
13
MMSP State-level Evaluation (Year 11)
Year 11 Project Activity
Table 13c: Mathematics Teacher Levels – ELL Education
Sep13–Aug14
Number of
Teachers
Level
Elementary School
Middle School
High School
License at Level
Degree in Math
n
n*
%*
n*
%*
0
2
2
0
1
2
0
50
100
0
1
0
0
50
0
*Of the number of ELL education participants teaching mathematics at this level for this period
Degrees Currently Pursued
Information on degrees currently being pursued in science, technology/engineering, and mathematics was derived
from the most recently completed survey of each individual. Table 14 provides details about the degrees being
pursued and the teaching areas of those who were pursuing the degrees, and it shows that these participants were
pursuing a total of 144 undergraduate and graduate degrees. Of all unique participants from the 2013–2014
funding period, 32 reported that they were pursuing a degree in general education, 100 were working on degrees
in mathematics or mathematics education, and 12 were pursuing degrees in science and technology/engineering.
Table 14: Pursuit of Degrees
Teaching Area
BA/BS
Degree Pursued
MA/MS
CAGS
Doctorate
General Education
0
19
9
4
Mathematics Education
0
62
14
2
Mathematics
0
13
4
2
Science Education
0
8
0
0
General Science
0
2
1
0
Biology
0
0
0
0
Chemistry
0
0
0
0
Physics
0
0
0
0
Earth Science
1
0
0
0
Technology/Engineering
1
0
0
0
Total
2
104
28
8
Content Knowledge Gains
As a grant condition, partnerships were required to use a pre-test and post-test for each MMSP course to assess
participants’ content knowledge growth. For each course, the same instrument served as both the pre- and posttest. While partnerships were permitted to locate and use an instrument with established reliability and validity, it
was most often the case that such instruments were not available because a priority was placed on utilizing
assessments that would reflect the precise content taught in each course. As a result, the faculty members who had
developed the courses usually developed the assessments, and typically, neither validity nor reliability was
determined for them because time and resource constraints prohibited doing so.
Data were analyzed only for participants who completed both pre- and post-course assessments. To determine if
the scores showed statistically significantly changes between pre- and post-course test administrations, a paired
samples t-test was used for each course with ten or more participants completing both pre- and post-course
UMass Donahue Institute
Research and Evaluation Group
14
MMSP State-level Evaluation (Year 11)
Year 11 Project Activity
assessments, and a Wilcoxon Matched Pairs test was used for each course with fewer than ten participants
completing both pre- and post-tests.
Pre- and post-tests were administered for all of the 33 courses offered in Year 11. Gains in average percentage of
items correct between pre- and post-test administrations occurred in all of the courses. Of the mathematics
courses offered, all but one showed statistically significant improvements in scores on content knowledge
assessments. Statistically significant improvements in scores also occurred for seven of the eight science courses
offered. The two courses not showing a statistically significant improvement in scores had six participants in one
and only two in the other complete both the pre- and post-test—the smallest sample size at which it is possible to
detect statistical significance at the level used for these analyses. Table 15 provides an overview, by subject
matter of courses delivered. For detailed information on mean pre- and post-test scores by partnership and by
course, see Appendix F.
Table 15: Number of Courses with Statistically Significant Gains
in Mean Content Knowledge Scores
Sep13–Aug14
Content Area
Courses Delivered
Significant Gains
Mathematics
Science & Technology/Engineering
24
7
96%
88%
TOTAL
31
94%
Partnership-level Participant Background Data
Tables 16 through 25 offer an overview of selected participant survey data for each year 11 partnership. These
data were collected through the participant background survey administered at the end of each course. The
responses are presented as frequencies and percentages. In cases where not all participants responded to all of the
items and in cases where multiple responses were permitted, percentages presented may not total 100%.
Table 16: Participant Background Info: EduTron – Revere-Everett-Saugus (Cohort 6 – Math)
Number of Participants
Sep11-Aug12
Total Number of Participants
69
Sep12-Aug13
Sep13-Aug14
67
52
1
(1%)
25
(48%)
7
(10%)
14
(20%)
34
(65%)
Teach Regular Education
51
(74%)
29
(56%)
28
46
(42%)
(69%)
Teach Special Education (sole and inclusion)
11
(16%)
13
(25%)
13
(19%)
1
(1%)
1
(2%)
2
(3%)
18
(26%)
22
(42%)
11
(16%)
7
(10%)
8
(15%)
3
(5%)
44
(64%)
20
(39%)
66
(99%)
1
(1%)
15
(29%)
7
(10%)
Teach in High Need District
47
(68%)
29
(56%)
50
(75%)
Have Passed One or More MTEL Exams
52
(75%)
38
(73%)
45
(67%)
Hold One or More Teaching Licenses
69
(100%)
31
(60%)
21
(31%)
Participants Who Took Multiple Courses
In relevant year
At any point in time
Teach ELL, ESL, or Sheltered English Immersion
Teach Elementary (all content areas)
Teach Elementary Math
Teach Mathematics
Teach Science or Technology/Engineering
UMass Donahue Institute
Research and Evaluation Group
15
MMSP State-level Evaluation (Year 11)
Year 11 Project Activity
Table 17: Participant Background Info: Lesley – Brockton (Cohort 6 – Math)
Number of Participants
Sep11-Aug12
Total Number of Participants
35
Sep12-Aug13
Sep13-Aug14
84
94
3
(9%)
30
(32%)
28
(33%)
18
(51%)
39
(42%)
22
(63%)
69
(73%)
51
62
(61%)
(74%)
Teach Special Education (sole and inclusion)
5
(14%)
12
(13%)
10
(12%)
Teach ELL, ESL, or Sheltered English Immersion
6
(17%)
8
(9%)
3
(4%)
10
(29%)
32
(34%)
31
(37%)
5
(14%)
6
(6%)
3
(4%)
22
(63%)
53
(56%)
80
(95%)
5
(14%)
43
(46%)
32
(38%)
Teach in High Need District
33
(94%)
77
(82%)
72
(86%)
Have Passed One or More MTEL Exams
23
(66%)
57
(61%)
63
(75%)
Hold One or More Teaching Licenses
32
(91%)
56
(60%)
11
(13%)
Participants Who Took Multiple Courses
In relevant year
At any point in time
Teach Regular Education
Teach Elementary (all content areas)
Teach Elementary Math
Teach Mathematics
Teach Science or Technology/Engineering
Table 18: Participant Background Info: Lesley – Springfield (Cohort 6 – Math)
Number of Participants
Sep13-Aug14
Sep11-Aug12
Sep12-Aug13
Total Number of Participants
27
55
51
5
(19%)
34
(67%)
37
(67%)
6
(22%)
37
(73%)
20
(74%)
29
(57%)
46
38
(84%)
(69%)
Teach Special Education (sole and inclusion)
4
(15%)
11
(22%)
11
(20%)
Teach ELL, ESL, or Sheltered English Immersion
0
(0%)
0
(0%)
0
(0%)
10
(37%)
14
(28%)
12
(22%)
7
(26%)
6
(12%)
3
(6%)
13
(48%)
24
(47%)
53
(96%)
2
(7%)
11
(22%)
17
(31%)
Teach in High Need District
22
(82%)
36
(71%)
44
(80%)
Have Passed One or More MTEL Exams
17
(63%)
36
(71%)
38
(69%)
Hold One or More Teaching Licenses
21
(78%)
27
(53%)
5
(9%)
Participants Who Took Multiple Courses
In relevant year
At any point in time
Teach Regular Education
Teach Elementary (all content areas)
Teach Elementary Math
Teach Mathematics
Teach Science or Technology/Engineering
UMass Donahue Institute
Research and Evaluation Group
16
MMSP State-level Evaluation (Year 11)
Year 11 Project Activity
Table 19: Participant Background Info: MCLA (Cohort 6 – Math/Science)
Number of Participants
Sep11-Aug12
Total Number of Participants
Sep12-Aug13
17
Sep13-Aug14
8
11
0
(0%)
0
(0%)
0
(0%)
2
(12%)
2
(18%)
Teach Regular Education
9
(53%)
7
(64%)
2
8
(25%)
(100%)
Teach Special Education (sole and inclusion)
0
(0%)
2
(18%)
0
(0%)
Teach ELL, ESL, or Sheltered English Immersion
0
(0%)
0
(0%)
0
(0%)
Teach Elementary (all content areas)
0
(0%)
0
(0%)
0
(0%)
Teach Elementary Math
0
(0%)
1
(9%)
0
(0%)
Teach Mathematics
5
(29%)
9
(82%)
5
(63%)
Teach Science or Technology/Engineering
4
(24%)
1
(9%)
4
(50%)
Teach in High Need District
6
(35%)
4
(36%)
7
(88%)
Have Passed One or More MTEL Exams
6
(35%)
7
(64%)
7
(88%)
Hold One or More Teaching Licenses
8
(47%)
10
(91%)
0
(0%)
Participants Who Took Multiple Courses
In relevant year
At any point in time
Table 20: Participant Background Info: Northeastern (Cohort 6 – Science)
Number of Participants
Sep11-Aug12
Total Number of Participants
85
Sep12-Aug13
65
Sep13-Aug14
38
12
(14%)
11
(17%)
7
(18%)
34
(40%)
31
(48%)
20
(53%)
76
(89%)
49
(75%)
32
(84%)
Teach Special Education (sole and inclusion)
4
(5%)
9
(14%)
3
(8%)
Teach ELL, ESL, or Sheltered English Immersion
1
(1%)
3
(5%)
1
(3%)
Teach Elementary (all content areas)
8
(9%)
7
(11%)
1
(3%)
Teach Elementary Math
2
(2%)
2
(3%)
0
(0%)
Teach Mathematics
8
(9%)
12
(19%)
8
(21%)
Teach Science or Technology/Engineering
63
(74%)
55
(85%)
38
(100%)
Teach in High Need District
29
(34%)
27
(42%)
12
(32%)
Have Passed One or More MTEL Exams
50
(59%)
39
(60%)
30
(79%)
Hold One or More Teaching Licenses
70
(82%)
47
(72%)
7
(18%)
Participants Who Took Multiple Courses
In relevant year
At any point in time
Teach Regular Education
UMass Donahue Institute
Research and Evaluation Group
17
MMSP State-level Evaluation (Year 11)
Year 11 Project Activity
Table 21: Participant Background Info: Fitchburg – NCIS (Cohort 7 – Math)
Number of Participants
Sep13-Aug14
Total Number of Participants
Participants Who Took Multiple Courses
16
In relevant year
At any point in time
0
(0%)
1
(6%)
14
(88%)
Teach Special Education (sole and inclusion)
1
(6%)
Teach ELL, ESL, or Sheltered English Immersion
0
(0%)
Teach Elementary (all content areas)
3
(19%)
Teach Elementary Math
0
(0%)
Teach Mathematics
8
(50%)
Teach Science or Technology/Engineering
9
(56%)
Teach in High Need District
16
(100%)
Have Passed One or More MTEL Exams
11
(69%)
2
(13%)
Teach Regular Education
Hold One or More Teaching Licenses
Table 22: Participant Background Info: Framingham (Cohort 7 – Science)
Number of Participants
Sep13-Aug14
Total Number of Participants
15
0
(0%)
2
(13%)
10
(67%)
Teach Special Education (sole and inclusion)
3
(20%)
Teach ELL, ESL, or Sheltered English Immersion
1
(7%)
Teach Elementary (all content areas)
0
(0%)
Teach Elementary Math
0
(0%)
Teach Mathematics
3
(20%)
Teach Science or Technology/Engineering
14
(93%)
Teach in High Need District
15
(100%)
Have Passed One or More MTEL Exams
6
(40%)
Hold One or More Teaching Licenses
5
(33%)
Participants Who Took Multiple Courses
Teach Regular Education
UMass Donahue Institute
Research and Evaluation Group
In relevant year
At any point in time
18
MMSP State-level Evaluation (Year 11)
Year 11 Project Activity
Table 23: Participant Background Info: Global Learning Public Charter
(Cohort 7 – Science)
Number of Participants
Sep13-Aug14
Total Number of Participants
25
0
(0%)
4
(16%)
22
(88%)
Teach Special Education (sole and inclusion)
2
(8%)
Teach ELL, ESL, or Sheltered English Immersion
1
(4%)
Teach Elementary (all content areas)
1
(4%)
Teach Elementary Math
2
(8%)
Teach Mathematics
18
(72%)
Teach Science or Technology/Engineering
14
(56%)
Teach in High Need District
25
(100%)
Have Passed One or More MTEL Exams
14
(56%)
8
(32%)
Participants Who Took Multiple Courses
In relevant year
At any point in time
Teach Regular Education
Hold One or More Teaching Licenses
Table 24: Participant Background Info: Lesley – Springfield-Easthampton
(Cohort 7 – Math)
Number of Participants
Sep13-Aug14
Total Number of Participants
Participants Who Took Multiple Courses
10
In relevant year
At any point in time
2
(20%)
8
(80%)
Teach Regular Education
9
(90%)
Teach Special Education (sole and inclusion)
1
(10%)
Teach ELL, ESL, or Sheltered English Immersion
0
(0%)
Teach Elementary (all content areas)
1
(10%)
Teach Elementary Math
3
(30%)
10
(100%)
Teach Science or Technology/Engineering
1
(10%)
Teach in High Need District
6
(60%)
Have Passed One or More MTEL Exams
6
(60%)
Hold One or More Teaching Licenses
0
(0%)
Teach Mathematics
UMass Donahue Institute
Research and Evaluation Group
19
MMSP State-level Evaluation (Year 11)
Year 11 Project Activity
Table 25: Participant Background Info: Worcester Public Schools
(Cohort 7 – Math)
Number of Participants
Sep13-Aug14
Total Number of Participants
9
0
(0%)
4
(44%)
Teach Regular Education
6
(67%)
Teach Special Education (sole and inclusion)
1
(11%)
Teach ELL, ESL, or Sheltered English Immersion
0
(0%)
Teach Elementary (all content areas)
2
(22%)
Teach Elementary Math
1
(11%)
Teach Mathematics
7
(78%)
Teach Science or Technology/Engineering
1
(11%)
Teach in High Need District
9
(100%)
Have Passed One or More MTEL Exams
6
(67%)
Hold One or More Teaching Licenses
5
(56%)
Participants Who Took Multiple Courses
UMass Donahue Institute
Research and Evaluation Group
In relevant year
At any point in time
20
MMSP State-level Evaluation (Year 11)
Cumulative Summary
Cumulative Summary: All Cohorts over All Funding Periods
Overview of Partnerships, Budgets, Courses, and Participants
Tables 26 and 27 provide an overview of partnership projects since the inception of the program. The first seven cohorts combined included 39 partnerships, with
22 offering mathematics professional development (PD), 14 offering science PD, and 3 offering PD in both mathematics and science content. Specifically, Cohort
1 consisted of eight partnerships, with six offering mathematics PD and two offering science PD. Cohort 2 consisted of two partnerships, both offering
mathematics PD. Cohort 3 consisted of nine partnerships, with four offering mathematics PD, three offering science PD, and two offering PD in both mathematics
and science. Cohort 4 consisted of eight partnerships, with four offering mathematics PD and four offering science PD. Cohort 5 consisted of two partnerships,
with one offering mathematics PD and one offering science PD. Cohort 6 consisted of five partnerships, with three offering mathematics PD, one offering science
PD, and one offering PD in both mathematics and science. Cohort 7 consisted of six partnerships, with two offering mathematics PD, three offering science PD,
and one offering PD in both mathematics and science.
Table 26: Overview of MMSP Partnership Participation: Cohort 1–Cohort 3
Funding Period
MMSP Year 1
Feb04-Aug04
Partnership Grouping
Cohort 1
MMSP Year 2
Sep04-Aug05
MMSP Year 3
Sep05-Aug06
MMSP Year 4
MMSP Year 5
MMSP Year 6
Sep06-Aug07
Sep07-Aug08
Sep08-Aug09
Grant Year 1
Grant Year 2
Grant Year 3
EduTron/Fitchburg-Math
Harvard-Math
Lesley-Math
MCLA-Science
Salem-Math
Springfield PS-Science
Wareham PS-Math
WPI-Math
EduTron/Fitchburg-Math
Harvard-Math
Lesley-Math
MCLA-Science
Salem-Math
Springfield PS-Science
Wareham PS-Math
WPI-Math
EduTron/Fitchburg-Math
Harvard-Math
Lesley-Math
MCLA-Science
Salem-Math
Springfield PS-Science
Wareham PS-Math
WPI-Math
Grant Year 1
Grant Year 2
Grant Year 3
MCLA-Math
PV STEMNET-Math
MCLA-Math
PV STEMNET-Math
MCLA-Math
PV STEMNET-Math
Cohort 2
Cohort 3
UMass Donahue Institute
Research and Evaluation Group
Grant
Year 3
Extension
Grant Year 1
Grant Year 2
Grant Year 3
EduTron/Lowell-Math/Sci
EduTron/Fitchburg-Math
Lesley-Math
North Shore-Science
PV STEMNET-Math/Sci
Salem-Math
SE/Cape-Science
WPI-Science
WPS-Math (discontinued)
EduTron/Lowell-Math/Sci
EduTron/Fitchburg-Math
Lesley-Math
North Shore-Science
PV STEMNET-Math/Sci
Salem-Math
SE/Cape-Science
WPI-Science
EduTron/Lowell-Math/Sci
EduTron/Fitchburg-Math
Lesley-Math
North Shore-Science
PV STEMNET-Math/Sci
Salem-Math
SE/Cape-Science
WPI-Science
21
MMSP State-level Evaluation (Year 11)
Cumulative Summary
Table 27: Overview of MMSP Partnership Participation: Cohort 4–Cohort 7
Funding Period
MMSP Year 6
Sep08-Aug09
Partnership Grouping
Cohort 4
MMSP Year 7
Sep09-Aug10
MMSP Year 8
MMSP Year 9
MMSP Year 10
MMSP Year 11
Sep10-Aug11
Sep11-Aug12
Sep12-Aug13
Sep13-Aug14
Grant Year 1
Grant Year 2
Grant Year 3
Boston PS-Math
Brockton PS-Math
Gateway RSD-Science
Lesley-Math
Northeastern-Science
Randolph PS-Science
Springfield Coll.-Science
Boston U.-Math
Boston PS-Math
Brockton PS-Math
Gateway RSD-Science
Lesley-Math
Northeastern-Science
Randolph PS-Science
Springfield Coll.-Science
Boston U.-Math
Boston PS-Math
Brockton PS-Math
Gateway RSD-Science
Lesley-Math
Northeastern-Science
Randolph PS-Science
Springfield Coll.-Science
Boston U.-Math
Cohort 5
Cohort 6
Grant Year 1
Grant Year 2
Grant Year 3
EduTron-Worc-M EverettUMass Boston-S
EduTron-Worc-M
Everett-UMass Boston-S
EduTron-Worc-M
Everett-UMass Boston-S
Grant Year 1
Grant Year 2
Grant Year 3
EduTron-Cohort 6-M
Lesley-Brockton-M
Lesley-Springfield-M
MCLA-S
Northeastern-S
EduTron-Cohort 6-M
Lesley-Brockton-M
Lesley-Springfield-M
MCLA-S
Northeastern-S
EduTron-Cohort 6-M
Lesley-Brockton-M
Lesley-Springfield-M
MCLA-M/S
Northeastern-S
Grant Year 1
Cohort 7
Fitchburg – NCIS-M/S
Framingham-S
Global Learning Public Charter-S
Lesley – Springfield-Easthampton-M
Worcester Public Schools-M
West Springfield Public Schools-S
Table 28 shows the funding received by each partnership since the inception of the program. While some partnerships were awarded funding in more than one
funding period, for evaluation purposes, a partnership was identified as a “new” partnership each time it received funding that resulted from a different
competition. Overall, partnerships have been awarded a total of $19,362,175 since the inception of MMSP.
UMass Donahue Institute
Research and Evaluation Group
22
MMSP State-level Evaluation (Year 11)
Cumulative Summary
Table 28: Budgets: All Partnerships, All Funding Periods
Feb04Aug06
Partnership
Sep06Aug07
Sep07Aug08
Sep08Aug09
Sep09Aug10
Sep10Aug11
Sep11Aug12
Sep12Aug13
Sep13Aug14
TOTAL
COHORT 1: Initially funded February 2004
EduTron (Math)
$770,000
$68,352
$838,352
Harvard Graduate School of Ed. (Math)
$489,899
$87,425
$577,324
Lesley University (Math)
$810,726
$43,838
$854,564
MCLA – Science (Science)
$133,192
$38,247
$171,439
Salem State College (Math)
$541,995
$43,648
$585,643
Springfield Public Schools (Science)
$500,044
$74,737
$574,781
Wareham Public Schools (Math)
$398,440
$43,962
$442,402
$601,778
$35,633
$637,411
Worcester Polytechnic Institute (Math)
COHORT 2: Initially funded September 2004
MCLA – Math (Math)
$111,494
$51,874
$163,368
UMass Amherst (Math)
$262,415
$181,581
$443,996
COHORT 3: Initially funded September 2006
EduTron Lowell (Math & Science)
$210,000
$220,000
$240,000
$670,000
EduTron Fitchburg (Math)
$102,000
$110,000
$120,000
$332,000
Lesley University (Math)
$347,911
$355,626
$355,357
$1,058,894
North Shore (Science)
$196,474
$194,729
$199,871
$591,074
UMass Amherst (Math/Science)
$107,424
$216,281
$169,064
$492,769
Salem State College (Math)
$120,882
$113,551
$36,604
$271,037
SE/Cape (Science)
$129,438
$181,420
$169,246
$480,104
$99,586
$70,734
$94,852
$265,172
Worcester Polytechnic Inst. (Science)
Worcester Public Schools (Math)
UMass Donahue Institute
Research and Evaluation Group
$231,210
$231,210
23
MMSP State-level Evaluation (Year 11)
Cumulative Summary
Sep08Aug09
Sep09Aug10
Sep10Aug11
Boston Public Schools (Math)
$157,975
$405,747
$218,986
$782,708
Brockton Public Schools (Math)
$180,145
$255,758
$251,263
$687,166
Gateway Regional (Science)
$186,609
$200,370
$172,379
$559,358
Lesley-Springfield (Math)
$228,593
$324,820
$308,416
$861,829
Greater North Shore (Science)
$265,917
$306,690
$266,480
$839,087
Randolph Public Schools (Science)
$176,993
$183,150
$151,178
$511,321
Springfield College (Science)
$161,062
$148,896
$156,832
$466,790
Boston University (Math)
$241,586
$245,180
$244,394
$731,160
Partnership
Feb04Aug06
Sep06Aug07
Sep07Aug08
Sep11Aug12
Sep12Aug13
Sep13Aug14
TOTAL
COHORT 4: Initially funded September 2008
COHORT 5: Initially funded September 2010
EduTron-Worcester-Lowell (Math)
$200,000
$205,000
$200,000
$605,000
Everett –UMass Boston (Science)
$149,250
$149,250
$149,250
$447,750
COHORT 6: Initially funded September 2011
EduTron-Revere-Everett-Saugus (Math)
$200,000
$201,500
$249,982
$651,482
Lesley-Brockton (Math)
$155,794
$290,750
$314,383
$760,927
Lesley-Springfield (Math)
$122,415
$311,523
$274,930
$708,868
MCLA (Math/Science)
$49,664
$40,386
$17,326
$90.050
Northeastern (Science)
$199,627
$199,947
$199,991
$599,565
Fitchburg – NCIS (Math/Science)
$71,404
$71,404
Framingham (Science)
$55,839
$55,839
Global Learning Public Charter (Science)
$75,207
$75,207
Lesley – Springfield-Easthampton (Math)
$73,818
$73,818
West Springfield Public Schools (Science)
$49,553
$49,553
$51,753
$51,753
COHORT 7: Initially funded April 2014
Worcester Public Schools (Math)
TOTAL
UMass Donahue Institute
Research and Evaluation Group
$4,619,983
$2,214,222
$1,462,341
$2,983,874
$2,070,611
$2,119,178
$1,081,750
$1,393,356
$1,416,860
$19,362,175
24
MMSP State-level Evaluation (Year 11)
Cumulative Summary
Through Year 11 (2013-2014) partnerships developed and implemented a total of 425 courses of which 231
(54%) were unique, and 194 (46%) were repeat offerings. Of the 425 courses, 288 (68%) offered mathematics
content, 135 (32%) offered science and/or technology/engineering content, and 2 (1%) offered both mathematics
and science/technology content. In total, there were 3,678 unique participants, 1,382 of whom took two or more
courses. The term “unique participant” refers to each individual who participated in the program, regardless of
how many courses he or she took. Data for items from the Participant Background Survey that help to convey
participants’ professional backgrounds and motives for participation are discussed in this section. In addition, this
section discusses information obtained from partnerships on the extent to which courses offered through MMSP
became institutionalized.
Position of Participants
As shown in Table 29, 89% of MMSP course participants identified themselves as teachers, predominantly
regular education teachers (69% of all respondents).
Table 29: Professional Position of Participants – Cumulative
Professional Position (N=3,678)
(Only one response permitted)
Teacher
Regular Education
Special Education or Special Education Inclusion
ELL or Sheltered English Immersion
Other Teachers
Math Coach (Teaching and Non-Teaching)
Department Head or Curriculum Coordinator
Principal, Assistant Principal or Headmaster
Other
All Funding Periods
n
%
3258
2541
551
117
49
92
57
38
202
89
69
15
3
1
3
2
1
6
Content Taught
The distribution of teaching areas of respondents at the time of the survey is shown in Table 30 – 49% were
teaching mathematics, 32% were teaching science, and 26% were teaching all subjects at the elementary level.
Because respondents identified all teaching areas that applied to their positions at the time of the survey, some
selected multiple responses, so frequency totals exceed the number of respondents and percentages exceed 100%.
Table 30: Teaching Areas: All Participants, All Funding Periods
Teaching Areas (N=3,678)
(Multiple responses permitted)
Mathematics
Any science area
General Science
Biology
Earth Science
Chemistry
Physics
Technology/Engineering
Elementary (all subjects)
Elementary Mathematics
Other
Not Currently Teaching
UMass Donahue Institute
Research and Evaluation Group
All Funding Periods
n
1726
1166
575
268
135
173
170
70
970
198
56
214
%
49
32
16
7
4
5
5
2
26
5
2
6
25
MMSP State-level Evaluation (Year 11)
Cumulative Summary
Types of Schools of Participants
For each funding period at least 89% of MMSP participants worked in a public school. Over the course of the program to date, 96% of worked in a public school,
and 3% worked in a non-public school. Table 31 provides a breakdown, by funding period, of the types of schools in which participants worked.
Table 31. Types of Schools: All Unique Participants, All Funding Periods
School Type
Public Schools
(includes
charters)
Non-public
School
Other or No
Response
TOTAL
Feb04Aug04
n
%
Sep04Aug05
n
%
Sep05Aug06
n
%
Sep06Aug07
n
%
Sep07Aug08
n
%
Sep08Aug09
n
%
Sep09Aug10
n
%
Sep10Aug11
n
%
Sep11Aug12
n
%
Sep12Aug13
n
%
Sep13Aug14
n
%
Total
n
%
332
97
448
98
455
98
533
96
462
97
3176
96
614
98
600
95
278
89
327
93
3538
96
10763
96
8
2
7
2
6
1
12
2
7
2
113
3
16
3
29
5
17
5
20
6
122
3
357
3
1
<1
1
<1
3
1
8
<1
8
2
19
1
0
0
0
0
19
6
6
<2
13
<1
78
1
341
100
456
100
464
100
553
100
477
100
3308
100
630
100
629
100
314
100
353
100
3673
100 11198
100
High Need Status of Districts of Participants
MMSP partnerships were required to include at least one high need district. (Appendix D identifies the criteria for the high need designation and includes lists of
public school districts qualifying as high need through 2013-2014.) In addition, ESE expected that at least 50% of participants in each partnership would come
from high need districts, and further, they set an informal goal that at least 75% of participants for each partnership would come from high need districts. The high
need status of some school districts changed across years. For the purposes of MMSP evaluation and reporting, any district identified as high need in the first year
of a partnership’s funding continued to be classified as high need in subsequent years of the partnership, even if the district’s status changed. Additionally, any
districts not on the high need list in the first year of a partnership’s funding but subsequently added to the high need list in later years of the partnership were then
identified as qualifying for high need district designation.
As a whole, across all years of funding, 69% of participants were from high need districts. Table 32 shows that across all years of funding, 69% of the public
school participants in the program as a whole had come from high need districts and that for each year of funding, over 56% of public school participants in the
program had come from high need districts. An examination of high need district participation in individual partnerships reveals that over the course of each
partnership’s involvement since the beginning of MMSP, 26 of the 39 partnerships had at least 50% of their participants coming from high need districts across all
years of their involvement, and (again, across all years of their involvement), and 20 of 39 partnerships exceeded the informal goal of having at least 75% of the
participants come from high need districts. Of the 3678 unique MMSP course participants, approximately 68% were from high need public school districts,
approximately 30% were from other public school districts, and 2% either were from private schools or did not provide information on their districts. In addition,
UMass Donahue Institute
Research and Evaluation Group
26
MMSP State-level Evaluation (Year 11)
Cumulative Summary
less than 1% were from high need districts for some of the courses they took but not for others4. The table in Appendix G shows the number of participants from
high need districts organized by partnership.
Table 32: High Need Status of All Unique Participants from Public Schools
School Type
High Need
District
Non-high
Need District
Unknown or
No Response
TOTAL
Feb04Aug04
n
%
Sep04Aug05
n
%
Sep05Aug06
n
%
Sep06Aug07
n
%
Sep07Aug08
n
%
Sep08Aug09
n
%
Sep09Aug10
n
%
Sep10Aug11
n
%
Sep11Aug12
n
%
Sep12Aug13
n
%
Sep13Aug14
n
%
Total
n
%
202
61
276
62
254
56
397
74
335
73
544
69
412
67
437
70
212
76
243
74
2479
70
5791
69
129
39
161
36
196
43
126
24
122
26
240
31
202
33
191
30
66
24
82
25
1038
29
2553
30
1
<1
11
2
5
1
10
2
5
1
4
<1
0
0
1
<1
0
0
2
1
14
<1
53
1
332
100
448
100
455
100
533
100
462
100
788
100
614
100
629
100
278
100
327
100
3531
100
8397
100
Participants who took MMSP courses from a math partnership when their districts were considered high need for only science were identified as having come from a “non-high
need” district, and teachers who took MMSP courses from a science partnership when their districts were considered high need for only math were identified as having come from
a “non-high need” district.
4
UMass Donahue Institute
Research and Evaluation Group
27
MMSP State-level Evaluation (Year 11)
Cumulative Summary
Reasons for Participation in MMSP Courses
For each course taken, respondents identified all of their reasons for participating. Unlike in most other portions
of this report where data were presented for unique participants, data for this issue are presented for all course
seats, since responses are uniquely relevant to each distinct course that a participant took. Table 33 presents
findings for all 7,305 seats for all courses taken by all participants across all funding periods.
Table 33: Reasons for Participation of Participants: All Seats, All
Funding Periods
Total
Reasons for Participation
%
n
(Multiple responses permitted)
of 7,305
course seats
To increase knowledge in content
5184
71%
To obtain graduate credit
5038
69%
To pursue a personal interest
2233
31%
To earn PDPs for recertification
2153
29%
To get an additional license (certification)
1226
17%
To prepare for the Massachusetts Test for Educator Licensure (MTEL)
1024
14%
To follow an administrator’s suggestion
496
7%
To obtain a first license (certification)
226
3%
Other
415
6%
Repeat Participation
Overall, partnerships were successful at encouraging and retaining repeat participants. Of the 39 partnerships, 34
offered multiple courses. In all, 1,368 participants (37%) attended multiple courses. Table 34 provides details
regarding repeat participation.
Table 34: Repeat Participants – All Partnerships, All Funding
Periods
Number of
Courses
Delivered
to Date
Total
Number of
Unique
Participants
to Date*
EduTron (Math)
7
101
31
Harvard (Math)
8
154
21
19
98
75
3
18
9
26
130
53
7
76
27
Partnership
Lesley Univ. C1 (Math)
MCLA (Science)
Salem State College (Math)
Springfield PS (Science)
UMass Donahue Institute
Research and Evaluation Group
Total Number
Taking
Multiple
Courses
28
MMSP State-level Evaluation (Year 11)
Cumulative Summary
Table 34: Repeat Participants – All Partnerships, All Funding
Periods (Continued)
Number of
Courses
Delivered
to Date
Total
Number of
Unique
Participants
to Date*
Wareham PS (Math)
3
40
10
WPI (Math)
6
127
37
MCLA (Math)
4
9
2
Partnership
Total Number
Taking
Multiple
Courses
UMass Amherst (Math)
11
53
29
EduTron Lowell (Math/Science)
10
147
29
7
118
54
Lesley Univ. C3 (Math)
40
159
105
North Shore (Science)
30
72
26
UMass Amherst C3 (Math/Science)
14
118
42
8
71
36
15
137
50
WPI (Science)
3
43
5
Worcester PS(Math)
3
37
5
Boston PS (Math)
29
265
83
Brockton PS (Math)
15
235
18
3
82
45
EduTron Fitchburg (Math)
Salem State C3 (Math)
SE/Cape (Science)
Gateway RSD (Science)
Lesley Springfield (Math)
21
98
62
Greater North Shore (Science)
28
209
97
Randolph PS (Science)
8
70
32
Springfield Coll. (Science)
5
51
24
Boston University (Math)
6
96
18
EduTron (Math)
9
156
48
Everett (Science)
6
85
28
EduTron-Revere-Everett-Saugus (Math)
7
136
55
Lesley-Brockton (Math)
21
163
85
Lesley-Springfield (Math)
19
83
56
MCLA (Math/Science)
5
25
5
Northeastern (Science)
13
141
64
Fitchburg – NCIS (Math/Science)
1
16
N/A
Framingham (Science)
1
15
N/A
Global Learning Public Charter (Science)
1
25
N/A
Lesley – Springfield-Easthampton(Math)
2
10
West Springfield Public Schools (Science)
0
N/A
N/A
Worcester Public Schools (Math)
1
9
N/A
Across All Partnerships
425
3,678
2
1,368
*Participants who participated in multiple courses across partnerships were counted only once in the
partnership of their most recent course
UMass Donahue Institute
Research and Evaluation Group
29
MMSP State-level Evaluation (Year 11)
Cumulative Summary
MTEL Information
One method by which teachers may demonstrate subject-matter competency is to pass the Massachusetts Tests for
Educator Licensure (MTEL) in the content areas that they teach. Table 35 cumulatively identifies the tests taken
by participants across all years of the program along with passage rates.
Table 35: MTEL Tests Taken by All Participants – Total to Date
Based on each participant’s last survey
Took Test
Passing Test
Failing Test
n
n
%
600
579
97
8
1
13
2
75
65
87
3
4
7
9
Elementary Mathematics
151
135
89
6
4
10
7
Mathematics
408
341
84
37
9
30
7
Middle School Mathematics
622
543
87
46
7
33
5
96
65
68
20
21
11
11
General Science
277
249
90
12
4
16
6
Biology
181
164
91
8
4
9
5
Chemistry
103
89
86
10
10
4
4
Physics
67
50
75
13
19
4
6
Earth Science
31
22
71
5
16
4
13
Technology/Engineering
16
14
86
2
13
0
0
771
653
567
70
87
48
45
General Curriculum (formerly
Elementary)
Early Childhood
Middle School
Mathematics/Science
TOTAL in STE Areas
n
%
Scores
Unknown
n
%
Licensure and Degrees Held in Content Area Taught
Tables 36a, 36b, and 36c show how many teachers taught in each science and technology/engineering area over
the course of the program. The tables also show the percentages of teachers who indicated they were licensed in
the area in which they taught, and they show the percentages of teachers who indicated they held a degree in the
area in which they taught. Table 36a provides information for regular education teachers, Table 36b provides
information for special education teachers, and Table 36c provides information for ELL teachers. (The numbers
presented in Tables 36a, 36b, and 36c exceed the number of teachers who reported teaching in these areas because
some teachers taught in more than one area.)
For regular education teachers, the licensing reported by approximately 57% appeared to be appropriate for the
content area taught, and the degrees held by approximately 27% corresponded to content area taught. For special
education teachers, the licensing reported by approximately 44% appeared to be appropriate for the content area
taught, and the degrees held by approximately 7% corresponded to content area taught. For ELL teachers, the
licensing reported by approximately 48% appeared to be appropriate for the content area taught, and the degrees
held by 31% corresponded to content area taught.
UMass Donahue Institute
Research and Evaluation Group
30
MMSP State-level Evaluation (Year 11)
Cumulative Summary
Table 36a: Science and Technology Teaching Areas – Regular Education
Sep13–Aug14
Subject
General Science
Biology
Chemistry
Physics
Earth Science
Technology/Engineering
Number of
Teachers
License in Subject
Taught
Degree in Subject
Taught
n
n*
%*
n*
%*
469
214
148
143
104
60
292
164
89
58
26
23
62
77
60
41
25
38
60
141
58
29
12
11
13
66
39
20
12
18
*Of the number of regular education participants teaching in this area across all program years
Table 36b: Science and Technology Teaching Areas – Special Education
Sep13–Aug14
Subject
General Science
Biology
Chemistry
Physics
Earth Science
Technology/Engineering
Number of
Teachers
License in Subject
Taught
Degree in Subject
Taught
n
n*
%*
n*
%*
69
33
14
20
21
6
32
12
8
12
7
1
46
36
57
60
33
17
2
2
1
6
1
0
3
6
7
30
5
0
*Of the number of special education participants teaching in this area across all program years
Table 36c: Science and Technology Teaching Areas – ELL Education
Sep13–Aug14
Subject
General Science
Biology
Chemistry
Physics
Earth Science
Technology/Engineering
Number of
Teachers
License in Subject
Taught
Degree in Subject
Taught
n
n*
%*
n*
%*
25
16
6
7
5
3
16
7
2
3
1
1
64
44
33
43
20
33
5
7
2
3
2
0
20
44
33
43
40
0
*Of the number of ELL participants teaching in this area across all program years
Tables 37a, 37b, and 37c show how many teachers taught at each non-elementary mathematics level over the
course of the program. The tables also show the percentages of teachers who indicated they were licensed in the
level at which they taught, and the percentages who indicated they held a degree in the area in which they taught.
UMass Donahue Institute
Research and Evaluation Group
31
MMSP State-level Evaluation (Year 11)
Cumulative Summary
Table 37a provides information for regular education teachers, Table 37b provides information for special
education teachers, and Table 37c provides information for ELL teachers.
For regular education teachers, the licensing reported by approximately 47% appeared to be appropriate for the
mathematics level taught, and 26% held mathematics degrees. For special education teachers, the licensing
reported by approximately 10% appeared to be appropriate for the level taught, and 2% held mathematics degrees.
For ELL teachers, the licensing reported by 17% appeared to be appropriate for the level taught, and 15% held
mathematics degrees.
Table 37a: Mathematics Teacher Levels – Regular Education
Sep13–Aug14
Level
Elementary School
Middle School
High School
Number of
Teachers
License at Level
Degree in Math
n
n*
%*
n*
%*
20
867
255
4
410
119
20
47
47
2
168
125
10
19
49
*Of the number of regular education participants teaching in this area for this year
Table 37b: Mathematics Teacher Levels – Special Education
Sep13–Aug14
Level
Elementary School
Middle School
High School
Number of
Teachers
License at Level
Degree in Math
n
n*
%*
n*
%*
1
202
41
0
21
3
0
10
7
0
3
3
0
2
7
*Of the number of special education participants teaching in this area for this year
Table 37c: Mathematics Teacher Levels – ELL Education
Sep13–Aug14
Level
Elementary School
Middle School
High School
Number of
Teachers
License at Level
Degree in Math
n
n*
%*
n*
%*
0
38
9
0
6
2
0
16
22
0
3
4
0
8
44
*Of the number of ELL participants teaching in this area for this year
Degrees Currently Pursued
Information on degrees currently being pursued was derived from the most recently completed survey of each
individual. Table 38 provides details about the degrees being pursued and the teaching areas of those who were
pursuing the degrees, and it shows that these participants were pursuing a total of 1,187 undergraduate and
graduate degrees. Of all unique participants through the end of the 2013–2014 funding period, 297 reported that
UMass Donahue Institute
Research and Evaluation Group
32
MMSP State-level Evaluation (Year 11)
Cumulative Summary
they were pursuing degrees in science and technology/engineering and 407 reported that they were pursuing
mathematics degrees.
Table 38: Pursuit of Degrees
Teaching Area
BA/BS
Degree Pursued
MA/MS
CAGS
Doctorate
General Science
3
49
6
0
Biology
2
17
0
1
Chemistry
1
5
0
0
Physics
1
9
2
0
Earth Science
2
3
0
0
Technology/Engineering
6
18
5
1
20
285
59
25
Math Education
8
253
59
8
Science Education
4
153
7
2
Mathematics
2
56
18
3
Other
5
67
16
6
Total
54
915
172
46
Education
Content Knowledge Gains
As a grant condition, partnerships were required to use a pre- test and post- test for each MMSP course to assess
participants’ knowledge growth. For each course, the same instrument served as both the pre- and post-test. While
partnerships were permitted to locate and use an instrument with established reliability and validity, it was most
often the case that such instruments were not available because a priority was placed on utilizing assessments that
would reflect the precise content taught in each course. As a result, the faculty members who had developed the
courses usually developed the assessments, and typically neither validity nor reliability was determined for them
because time and resource constraints prohibited doing so.
Data were analyzed only for participants who completed both pre- and post-tests. To determine if the scores
showed statistically significantly changes between pre- and post-test administrations, a paired samples t-test was
used for each course with ten or more participants completing both pre- and post-tests, and a Wilcoxon Matched
Pairs test was used for each course with fewer than ten participants completing both pre- and post-tests.
Of the 425 courses that were delivered across all partnerships through Year 11, content assessments were
administered for 417 courses. Of these 417 courses, 412 had gains in the average percentage of items correct
between pre- and post- test administrations. Statistically significant improvements in scores on content knowledge
assessments occurred in 87% of the 417 courses. Of the 54 courses not showing statistically significant
improvement in scores, 44 had fewer than six participants5, the smallest sample size at which it is possible to
detect statistical significance at the level used for these analyses. Table 39 provides an overview, by subject
matter, of the numbers of courses that showed statistically significant gains.
5
Some of these courses had fewer than six MMSP enrollees within the context of another course that included additional
non-MMSP enrollees; other courses were not able to administer content assessments to all participants because of logistical
challenges.
UMass Donahue Institute
Research and Evaluation Group
33
MMSP State-level Evaluation (Year 11)
Cumulative Summary
Table 39: Number of Courses with Statistically Significant
Gains in Mean Content Knowledge Scores
Total
Content Area
Delivered,
with Pre- and Posttests
Significant
Pre/Post
Gains
Math
281
251
Science & Technology/Engineering
Math and Science
134
2
110
2
TOTAL
417
363
Course Institutionalization
For systemic change to occur at the higher education institutions, departments of arts and sciences and education
departments are encouraged to work together through MMSP to support stronger content courses in mathematics
and science for teacher preparation, undergraduate and graduate degree requirements, and for in-service teachers
pursuing graduate-level content courses for recertification. Integration of Title IIB courses into graduate programs
at Institutes of Higher Education ensures sustainability over time. The intent behind encouraging the partnerships
is that the faculty from the Arts and Sciences Departments bring strong content expertise to the partnership table.
This integration creates greater opportunities for participants to complete coursework leading to a content-area
degree and/or to licensure along with the highly qualified designation.
Since the 2006–2007 funding period, partnerships have been asked to describe activities that were related to the
institutionalization of their courses. Many partnerships reported integration, plans for future integration, or—in
the case of partnerships with previously established involvement with MMSP—work toward sustaining prior
integration. As would be expected in a program involving partnerships with diverse structures and styles, the
extent and type of integration varied across partnerships. To convey a sense of how integration occurred,
following are significant related activities, grouped according to partnership:
EduTron Lowell Public Schools (Math/Science) and EduTron Fitchburg State College (Math)
 Two developmental courses, based on the EduTron model for MMSP courses, will continue to be offered
at Fitchburg State College (FSC). EduTron partners supported FSC in designing three pre-service courses
that are optimized for education majors. EduTron has begun working with FSC to help FSC apply the
EduTron model used in MMSP math courses to science courses.
 FSC has partnered with Lowell Public Schools to offer a teacher certification/CAGS program.
 Six math and four science courses were approved by FSC as offerings at the continuing education level.
Lesley University Cohort 3 (Math)
 Two courses created through Lesley University’s MMSP in 2007-2008 are now offered to Lesley’s oncampus pre-service teachers.
 Efforts through MMSP contributed to the development of an online Mathematics Education program at
Lesley leading to the Master of Arts degree for elementary and middle school teachers.
 Nine math content courses were developed through participation in the MMSP program in 2007-2008 and
in prior years. All of these courses are part of Lesley University’s mathematics major for undergraduates,
which would not have been possible without the MMSP program.
North Shore (Science)
 As a result of their joint involvement in MMSP through the North Shore partnership and the National
Science Foundation MSP program, Northeastern University has institutionalized all MMSP courses. Ten
UMass Donahue Institute
Research and Evaluation Group
34
MMSP State-level Evaluation (Year 11)
Cumulative Summary
MMSP courses can be used to fulfill degree requirements toward a Master’s in Education for Middle
School Science. In addition, this degree was developed as a result of these courses.
UMass Amherst Cohort 3 (Math/Science)
 Four courses developed through the UMass Amherst partnership were approved for graduate level credit.
Salem State Cohort 3 (Math)
 Salem State College offers courses developed through MMSP as part of a master’s level teaching
program in middle school mathematics. All courses developed by Salem State College through MMSP
can be applied towards earning a degree through that program.
Southeast/Cape (Science)
 Participants of the three courses offered through the SE/Cape partnership may apply credit for the courses
towards the Master of Arts in Teaching in Physical Science program that is offered through Bridgewater
State College.
Worcester Polytechnic Institute (Science)
 A Master of Science Education program was created through the physics department at Worcester
Polytechnic Institute, and the MMSP course that was offered through the WPI-Science partnership will
serve as the model for instruction of future courses that will be offered.
Worcester Public Schools (Math)
 As a result of the experience of working with Worcester Public Schools on MMSP, Clark University has
expressed interest in exploring the institutionalization of courses that were offered through MMSP.
Springfield College (Science)
 Springfield College has incorporated into its pre-service Best Practices of Teaching Science course
activities from an MMSP course that are designed to help teachers understand how to change
misconceptions that students have about life science.
Randolph Public Schools (Science)
 Four courses developed through the Randolph Public Schools partnership were approved for graduate
level credit at Bridgewater State University.
Edutron Cohort 6 (Math)
 The partners have been working with Worcester State University and Fitchburg State University to
transfer EduTron’s Intensive Immersion approach and its detailed diagnostics techniques to Worcester
and Fitchburg State Universities’ courses.
 Edutron has identified seven graduate-level math courses, developed for MMSP, that have been
integrated into the course offerings at Fitchburg State University between March 2012 and May 2014.
Lesley- Brockton Cohort 6 (Math), Lesley-Springfield Cohort 6 (Math), and Lesley-Springfield/
Easthampton (Cohort 7)
 Ten graduate-level math electives, developed as MMSP courses, were integrated into the Lesley
University School of Education course offerings between Fall 2013 and Fall 2014.
MCLA Cohort 6 (Math/Science)
 Four graduate-level courses (two math, 1 science and 1 blended), developed for MMSP, were integrated
into the MCLA course offerings between Fall 2013 and Fall 2014.
UMass Donahue Institute
Research and Evaluation Group
35
MMSP State-level Evaluation (Year 11)
Cumulative Summary
Framingham (Science)
 In its first year, the Framingham Partnership developed a new graduate-level blended math/science
practice course which has been incorporated into the course offerings at UMass Boston.
Scaling Up
While this evaluation did not set out to explore the reach of partnerships beyond documenting the numbers of
participants and their high need districts of origin, an exceptional instance of scaling up emerged through data
collection efforts. Since it speaks to the goals of MMSP and also is an indicator of project success, it is being
included here. As its professional development model, the Brockton Public Schools partnership used the 80-hour
Massachusetts Intel Math Initiative (MIMI) course and professional learning community follow-up. Through their
participation in MMSP, they—in effect—regionalized the program, expanding the model from a relatively small
partnership between school districts and higher education professors to include over 28 Southeastern
Massachusetts districts and three institutions of higher education.
UMass Donahue Institute
Research and Evaluation Group
36
MMSP State-level Evaluation (Year 11)
Summary of Findings
Summary of Findings
The MMSP partnership activities summarized in this section of the report occurred between February 2, 2004,
and August 31, 2014. This period spans the beginning of the MMSP program through the end of the 2013–2014
funding period. Since MMSP began in February 2004, progress has been made towards meeting the goals of the
program as evidenced by the following data for both the program as a whole, since its inception, and for the most
recent funding period of 2013–2014.
Cumulative Findings
Overview of Partnerships

A total of 39 partnerships were funded across the Commonwealth. Of these, 22 were organized around
mathematical content, 14 were organized around science content, and 3 were organized around both
mathematical and science content. Of the 39 MMSP partnerships, all delivered courses, 34 offered
multiple courses.

Cohort 1, which began in February 2004, consisted of eight partnerships, with six of the eight
partnerships offering mathematics professional development and two offering science professional
development.

Cohort 2, which began in September 2004, consisted of two partnerships, both offering mathematics
professional development.

Cohort 3, which began in September 2006, consisted of nine partnerships, with five of the nine
partnerships offering mathematics professional development, three offering science professional
development, and one offering professional development in both mathematics and science content.

Cohort 4, which began in September 2008, consisted of eight partnerships, with four of the eight
partnerships offering mathematics professional development and four offering science professional
development.

Cohort 5, which began in September 2010, consisted of two partnerships, with one partnership
offering mathematics professional development and one offering science professional development.

Cohort 6, which began in September 2011, consisted of five partnerships, with three of the five
partnerships offering mathematics professional development and two offering science professional
development.

Cohort 7, which began in April 2013, consisted of six partnerships, with two of the five offering
mathematics professional development, three offering science professional development, and one
offering professional development in both math and science.
Overview of Courses

In total, 425 MMSP courses were delivered by the end of Year 11 of MMSP funding. Of these 425
courses, 288 were mathematics courses, 135 were science and/or technology/engineering courses, and
two were courses offering both mathematics and science content.
Overview of Participants

In total, 3,678 unique participants participated in MMSP courses by the end of Year 11.
UMass Donahue Institute
Research and Evaluation Group
37
MMSP State-level Evaluation (Year 11)

1,368 participants (37% of all participants) attended multiple courses.

7,305 course seats were filled by all participants across all funding periods.
Summary of Findings
Reaching Targeted Participants

Types of Schools of Participants


Of all 3,546 unique participants, 96% came from public schools (including public charter schools),
3% came from non-public schools, and 1% did not indicate their school type.
High Need Status of Districts of Participants

The partnerships exceeded the ESE target of having at least 50% of all participants come from high
need districts, with 67% of all participants in the program coming from high need districts.

Across all years of their involvement, 26 of the 39 partnerships had at least 50% of their participants
coming from high need districts.

Across all years of their involvement, 20 of 39 partnerships had at least 75% of the participants come
from high need districts.
As the following data reveal, not all MMSP participants were licensed in their teaching areas or held relevant
degrees, indicating that course participants were, in fact, those in need of the courses:


Licensure in Mathematics and Science Content Areas for Middle and High School Teachers

Of the regular education mathematics courses taught by MMSP teachers, 47% were taught by
teachers who were licensed in mathematics.

Of the special education mathematics courses taught by MMSP teachers, 10% were taught by
teachers who were licensed in mathematics.

Of the ELL mathematics courses taught by MMSP teachers, 17% were taught by teachers who were
licensed in mathematics.

Of regular education science or technology/engineering courses taught by MMSP teachers, 57% were
taught by teachers who were licensed in the subject of the course.

Of special education science or technology/engineering courses taught by MMSP teachers, 44% were
taught by teachers who were licensed in the subject of the course.

Of ELL science or technology/engineering courses taught to students of MMSP teachers, 48% were
taught by teachers who were licensed in the subject of the course.
Degrees Held in Content Area in which Teaching

Of regular education mathematics courses taught by MMSP teachers, 26% were taught by teachers
who held mathematics degrees.

Of special education mathematics courses taught by MMSP teachers, 2% were taught by teachers
who held mathematics degrees.

Of ELL mathematics courses taught to students of MMSP teachers, 15% were taught by teachers who
held mathematics degrees.

Of regular education science or technology/engineering courses taught by MMSP teachers, 27% were
taught by teachers who held degrees that were relevant to the focus of the course.
UMass Donahue Institute
Research and Evaluation Group
38
MMSP State-level Evaluation (Year 11)
Summary of Findings

Of special education science or technology/engineering courses taught by MMSP teachers, 7% were
taught by teachers who held degrees that were relevant to the focus of the course.

Of ELL science or technology/engineering courses taught by MMSP teachers, 31% were taught by
teachers who held degrees that were relevant to the focus of the course.
Content Knowledge Gains
The content knowledge of participants increased:

Statistically significant improvements in scores on content knowledge assessments occurred in 87% of the
417 courses in which assessments were administered.
Year 11 Findings
Overview of Partnerships

Eleven partnerships were funded during the 2013–2014 funding period. Of these, five were organized
around mathematical content, four were organized around science content, and two addressed both
mathematics and science content. One partnership (West Springfield) did not complete a course during
the funding period, four offered a single course, and six offered at least two courses. Five partnerships had
participants who attended more than one course during the year.
Overview of Courses

In total, 33 courses were delivered during Year 11 of MMSP funding. Of these 33 courses, 25 were
mathematics courses and 8 were science/technology/engineering courses.
Overview of Participants

During Year 11, 328 unique participants participated in MMSP courses.

89 participants (27% of all Year 11 participants) attended multiple courses during 2013–2014.

500 course seats were filled during Year 11.

Course attrition rates were low averaging 4% across all courses offered by partnerships in Year 11.
Reaching Targeted Participants

Types of Schools of Participants


Of all 328 unique Year 11 participants, 97% came from public schools (including public charter
schools), and 3% came from non-public schools.
High Need Status of Districts of Participants

The Year 10 partnerships exceeded the ESE target of having at least 50% of all participants come
from high need districts, with 78% of all 2013–2014 participants coming from high need districts.

Nine of the ten partnerships offering courses had at least 50% of their participants coming from high
need districts.

Eight of the ten partnerships offering courses had at least 75% of the participants come from high
need districts.
UMass Donahue Institute
Research and Evaluation Group
39
MMSP State-level Evaluation (Year 11)
Summary of Findings
As the following data reveal, not all Year 11 participants were licensed in their teaching areas or held relevant
degrees, indicating that course participants were, in fact, those in need of the courses:


Licensure in Mathematics and Science Content Areas
 Of regular education mathematics courses taught by Year 11 participants, 77% were taught by
teachers who were licensed in mathematics.

Of special education mathematics courses taught by Year 11 participants, 18% were taught by
teachers who were licensed in mathematics.

Of ELL mathematics courses taught by Year 11 participants, 75% were taught by teachers who were
licensed in mathematics.

Of regular education science or technology/engineering courses taught by Year 11 participants, 40%
were taught by teachers who were licensed in the subject of the course.

Of special education science or technology/engineering courses taught by Year 11 participants, 23%
were taught by teachers who were licensed in the subject of the course.

Of ELL science or technology/engineering courses taught by Year 11 participants, 29% were taught
by teachers who were licensed in the subject of the course.
Degrees Held in Content Area in which Teaching

Of regular education mathematics courses taught by Year 11 participants, 46% were taught by
teachers who held mathematics degrees.

Of special education mathematics courses taught by Year 11 participants, none were taught by
teachers who held mathematics degrees.

Of ELL mathematics courses taught by Year 11 participants, one was taught by a teacher who held a
mathematics degree.

Of regular education science or technology/engineering courses taught by Year 11 participants, 21%
were taught by teachers who held degrees that were relevant to the focus of the course.

Of special education science or technology/engineering courses taught by Year 11 participants, none
were taught by teachers who held degrees that were relevant to the focus of the course.

Of ELL science or technology/engineering courses taught by Year 11 participants, 57% were taught
by teachers who held degrees that were relevant to the focus of the course.
Content Knowledge Gains
The content knowledge of Year 10 participants increased:

Statistically significant improvements in scores on content knowledge assessments occurred in 31 of the
33 courses.
UMass Donahue Institute
Research and Evaluation Group
40
MMSP State-level Evaluation (Year 11)
Appendix A: Participant Background Survey
Appendix A: Participant Background Survey – Year 11
UMass Donahue Institute
Research and Evaluation Group
41
MMSP State-level Evaluation (Year 11)
UMass Donahue Institute
Research and Evaluation Group
Appendix A: Participant Background Survey
42
MMSP State-level Evaluation (Year 11)
UMass Donahue Institute
Research and Evaluation Group
Appendix A: Participant Background Survey
43
MMSP State-level Evaluation (Year 11)
UMass Donahue Institute
Research and Evaluation Group
Appendix A: Participant Background Survey
44
MMSP State-level Evaluation (Year 11)
UMass Donahue Institute
Research and Evaluation Group
Appendix A: Participant Background Survey
45
MMSP State-level Evaluation (Year 11)
UMass Donahue Institute
Research and Evaluation Group
Appendix A: Participant Background Survey
46
MMSP State-level Evaluation (Year 11)
UMass Donahue Institute
Research and Evaluation Group
Appendix A: Participant Background Survey
47
MMSP State-level Evaluation (Year 11)
UMass Donahue Institute
Research and Evaluation Group
Appendix A: Participant Background Survey
48
MMSP State-level Evaluation (Year 11)
Appendix B: Evaluation Activities
Appendix B: Timeline for State-level Evaluation and TA Activities
The following is a summary timeline of state-level evaluation and technical assistance activities carried out between
February, 2004, and end of Year 11 of the MMSP.
February 2004 Held Kick-off Meeting for all partnerships and their evaluators at the Department of Education
Spring 2004
Conducted individual partnership meetings with local evaluators and partnership leaders to:
 Review the Minimum Expectations document along with the local evaluation and data collection
plan
 Explore potential modifications to implementation plans to create opportunities for experimental
or quasi experimental design
Spring 2004
Developed common measures for state-level data collection
June 2004
Attended federal meeting held for MSP projects across the country
Summer 2004 Disseminated and collected end-of-course documents designed to collect course-level data for the
statewide evaluation
Fall 2004
Conducted individual partnership meetings with local evaluators and partnership leaders to:
 Review the Minimum Expectations document along with the local evaluation and data collection
plan
 Review the Federal Reporting document to ensure the partnerships were collecting the data
needed to complete that report
Winter 2005
Conducted partnership meetings with the two new partnerships funded in the second round that
constitutes Cohort 2 to:
 Introduce the Minimum Expectations document along with the local evaluation and data
collection plan
 Introduce the Federal Reporting document to ensure the partnerships were collecting the data
needed to complete that report
June 2005
Held technical assistance meeting for all partnerships regarding the requirements of the USED Annual
report
June 2006
Participated in USED Annual Conference of MSP State Coordinators
August 2006
Held technical assistance meeting for all partnerships regarding evaluation requirements for MMSP
Fall 2006 to
Winter 2007
Conducted partnership meetings with the new Cohort 3 partnerships to:


Introduce the Minimum Expectations document along with the local evaluation and data
collection plan
Discuss the federal reporting requirements to ensure the partnerships were collecting the data
needed to complete federal report
UMass Donahue Institute
Research and Evaluation Group
49
MMSP State-level Evaluation (Year 11)
Appendix B: Evaluation Activities
December 2006 Participated in USED MSP Regional Conference
June 2007
Participated in USED Annual Conference of MSP State Coordinators
September 2007 Held technical assistance meeting for all partnerships regarding the requirements of the USED Annual
report
January 2008
Participated in USED MSP Regional Conference
April 2008
Participated in technical assistance workshop for bidders pursuing MSP funding for 2008-2009
April 2008
Participated in USED the Massachusetts MSP Statewide Conference
June 2008
Participated in USED MSP State Coordinators’ Meeting
October 2008
Held technical assistance meeting for all partnerships regarding evaluation requirements for MMSP
Fall 2008 to
Winter 2009
Conducted partnership meetings with the new Cohort 3 partnerships to:


Discuss evaluation expectations data collection plans
Discuss the federal reporting requirements to ensure the partnerships were collecting the data
needed to complete federal report
March 2009
Participated in USED MSP Regional Conference
May 2009
Participated in USED the Massachusetts MSP Statewide Conference
September 2009 Participated in technical assistance meeting for all partnerships regarding reporting requirements
January 2010
Participated in USED MSP Regional Conference
April 2010
Participated in technical assistance workshop for bidders pursuing MSP funding for 2010–2011
Spring 2010
Participated in continuation conferences for select partnerships
August 2010
Participated in technical assistance meeting for all partnerships regarding reporting requirements
February 2011
Participated in USED MSP Regional Conference
September 2011 Participated in technical assistance meeting for all partnerships regarding reporting requirements
November 2011 Participated in technical assistance workshop for bidders pursuing MSP funding for 2012–2013
April 2012
Participated in USED MSP Regional Conference
September 2012 Participated in technical assistance meeting for all partnerships regarding reporting requirements
September 2013 Participated in technical assistance meeting for all partnerships regarding reporting requirements
UMass Donahue Institute
Research and Evaluation Group
50
MMSP State-level Evaluation (Year 11)
April 2014
Appendix B: Evaluation Activities
Reached agreement with ESE regarding amended scope of services to include local evaluation services
for all partnerships beginning with Cohort 7.
September 2014 Led technical assistance meeting for all partnerships regarding reporting requirements
December 2014 Finalized revised end-of-course participants survey and developed a plan for obtaining participant
background data from existing ESE datasets.
The following activities were on going throughout the life of the project:
Disseminated and collected end-of-course documents designed to collect course-level data for the statewide
evaluation
Managed data collected from partnerships at the end of each course
Provided technical assistance to partnerships in support of local partnership evaluation efforts
Monitored local evaluation plans to see they include both formative and summative research questions and
corresponding activities
Monitored data collection and analysis around the basic logic model of professional development
Served as liaison to the U.S. Department of Education for evaluation and research issues including participation in
national meetings and periodic conference calls
Met with ESE MSP Team as needed to support integration of evaluation efforts with program goals
Until Steering Committee was disbanded, attended MMSP Steering Committee meetings in role of state level
evaluator and technical assistance
UMass Donahue Institute
Research and Evaluation Group
51
MMSP State-level Evaluation (Year 11)
Appendix C: Year 11 Background Survey Results
Appendix C: Year 11 Participant Background Survey Results
2013–2014
Item
n
%
How do you describe yourself?
American Indian or Alaskan native
Asian
Black or African American
Hispanic or Latino
Native Hawaiian or other Pacific Islander
White
Mixed Race
Other
No Response
2
11
12
12
0
280
5
11
5
1%
3%
4%
4%
0%
86%
2%
3%
2%
What best describes your current primary position?
Teacher (Regular Education)
Special Education Teacher (Sole Instructor)
Special Education Inclusion Teacher
Department Head or Curriculum Coordinator
Principal/Asst. Principal/Headmaster
Support Specialist (counselor, librarian, etc.)
Long-term Substitute
Paraprofessional
Superintendent or Asst. Superintendent
ELL, ESL, or Sheltered English Immersion Teacher
Gifted or Talented Teacher
AP or IB Program Teacher
Title I Teacher
Math Coach (Non-Teaching)
Math Coach (Teaching)
Science Coach (Non-Teaching)
Science Coach (Teaching)
Instructional Technology Director
Other
Unknown/No Response
UMass Donahue Institute
Research and Evaluation Group
247
21
23
8
0
1
2
1
0
8
4
0
1
1
3
0
0
1
3
3
76%
6%
7%
2%
0%
<1%
1%
<1%
0%
2%
1%
0%
<1%
<1%
1%
0%
0%
<1%
1%
1%
52
MMSP State-level Evaluation (Year 11)
Appendix C: Year 11 Background Survey Results
2013–2014
Item
n
%
What grades do you currently teach?
Pre-K
Elementary and K-8
Middle School (Grades 6-8)
High School (Grades 9-12)
Middle and High School grades
Adult Education
All levels
NA (doesn’t teach)
No Response
0
76
174
71
1
0
0
0
3
0%
23%
54%
22%
<1%
0%
0%
0%
1%
How many years have you been employed in education?
1st year
2-3 years
4-5 years
6-10 years
11-20 years
Over 20 years
No Response
19
45
40
97
85
35
6
6
14
12
30
26
11
2
Note: Percentages may not total 100% due to rounding error or items in which respondents may respond to
all that apply.
2013–2014
Item
Which of the following content areas are you currently
teaching?
Mathematics
Elementary (all subjects)
Middle School Mathematics
High School Mathematics
Any science area
General Science
Biology
Physics
Earth Science
Chemistry
Technology/Engineering
n
%
258
62
134
49
137
40
18
20
11
20
10
79%
19%
41%
15%
42%
12%
6%
6%
3%
6%
3%
318
8
1
97%
2%
<1%
2
2
1%
1%
In which of the following are you currently employed?
Public School/ Public Charter School
Private School
Unknown/No Response
Currently hold certification through the National Board for
Professional Teaching Standards.
In Mathematics
In General Science
UMass Donahue Institute
Research and Evaluation Group
53
MMSP State-level Evaluation (Year 11)
Appendix C: Year 11 Background Survey Results
2013–2014
Item
n
%
76
10
66
163
12
0
23%
3%
20%
50%
4%
0%
192
5
57
72
1
0
59%
2%
17%
22%
<1%
0%
108
47
54
57
3
58
33%
14%
17%
17%
1%
18%
157
35
63
21
0
51
48%
11%
19%
6%
0%
16%
31
109
159
10
3
15
10%
33%
49%
3%
1%
5%
54
155
67
24
3
24
17%
47%
21%
7%
1%
7%
Approximately how many math students do you teach annually?
0 students
1-10 students
11-40 students
41-150 students
151+ students
No Response
Approximately how many science students do you teach annually?
0 students
1-10 students
11-40 students
41-150 students
151+ students
No Response
Approximately how many students do you teach annually who are
Title I students?
0 students
1-10 students
11-40 students
41-150 students
151+ students
No Response
Approximately how many students do you teach annually who are
academically advanced students?
0 students
1-10 students
11-40 students
41-150 students
151+ students
No Response
Approximately how many students do you teach annually who are
Special Education students?
0 students
1-10 students
11-40 students
41-150 students
151+ students
No Response
Approximately how many students do you teach annually who are
English Language Learners?
0 students
1-10 students
11-40 students
41-150 students
151+ students
No Response
UMass Donahue Institute
Research and Evaluation Group
54
MMSP State-level Evaluation (Year 11)
Item
Appendix C: Year 11 Background Survey Results
2013–2014
n
%
Why did you participate in this course? *
To obtain graduate credit
To increase knowledge in content
To pursue a personal interest
To earn PDPs for recertification
To get an additional license (certification)
To prepare for the Massachusetts Test for
Educator Licensure (MTEL)
To follow an administrator’s suggestion
To obtain a first license (certification)
Other
376
321
153
111
68
75
64
31
22
14
51
10
29
18
35
6
4
7
High Need District
Yes
No
Unknown or N/A
256
62
9
78%
19%
3%
*Data for this item represents the number of seats filled from all courses, rather than the
number of unique participants.
Item
2013–2014
n
%
How many PDP hours do you have in your content area(s)?
Less than 48 PDP hours
48 to 100 PDP hours
101 to 250 PDP hours
251+ PDP hours
No Response
Please select any of the following licenses you currently
hold.
Vocational Technical
Specialist Teacher
Supervisor/Director
Principal/Asst. Principal
Superintendent/Assistant Superintendent
UMass Donahue Institute
Research and Evaluation Group
19
9
5
2
292
6%
3%
2%
1%
89%
4
46
4
19
3
1%
14%
1%
6%
1%
55
MMSP State-level Evaluation (Year 11)
Item
Appendix C: Year 11 Background Survey Results
2013–2014
Bachelors
n
%
Masters
n
%
CAGS
n
%
Doctorate
n
%
A degree currently held for each major.
Education
Math Education
Science Education
Math
General Science
Biology
Chemistry
Earth Science
Physics
Technology/Engineering
All science/technology combined
Other
74
19
4
44
4
20
6
3
4
15
56
93
23%
6%
1%
14%
1%
6%
2%
1%
1%
5%
17%
28%
140
28
15
7
2
0
4
1
0
6
28
41
43%
9%
5%
2%
1%
0%
1%
<1%
0%
2%
9%
13%
3
1
0
0
0
0
0
0
0
0
0
3
1%
<1%
0%
0%
0%
0%
0%
0%
0%
0%
0%
1%
0
0
0
0
0
0
0
1
0
0
1
0
0%
0%
0%
0%
0%
0%
0%
<1%
0%
0%
<1%
0%
0
0
0
0
0
0
0
1
0
1
2
0
0%
0%
0%
0%
0%
0%
0%
<1%
0%
<1%
1%
0%
19
62
8
13
2
0
0
0
0
0
10
8
6%
19%
2%
4%
1%
0%
0%
0%
0%
0%
3%
2%
9
14
1
4
1
0
0
0
0
0
2
3
3%
4%
<1%
1%
<1%
0%
0%
0%
0%
0%
1%
1%
4
2
0
2
0
0
0
0
0
0
0
1
1%
1%
0%
1%
0%
0%
0%
0%
0%
0%
0%
<1%
A degree currently being pursued for each major.
Education
Math Education
Science Education
Math
General Science
Biology
Chemistry
Earth Science
Physics
Technology/Engineering
All science/technology combined
Other
2013–2014
Item
MTEL
Taken
n
%
MTEL
Passed
n
%
Scores
Unknown
n
%
MTEL tests taken
General Curriculum (formerly Elementary)
Elementary Math
Early Childhood
Mathematics
Middle School Mathematics
General Science
Biology
Chemistry
Physics
Earth Science
Technology/Engineering
UMass Donahue Institute
Research and Evaluation Group
52
18
14
71
91
20
8
10
11
1
3
16%
6%
4%
22%
28%
6%
2%
3%
3%
<1%
1%
52
17
13
59
82
18
6
10
8
1
3
95%
95%
93%
89%
92%
95%
86%
100%
73%
100%
100%
0
0
0
2
2
1
1
0
0
0
0
0%
0%
0%
3%
2%
5%
14%
0%
0%
0%
0%
56
MMSP State-level Evaluation (Year 11)
Appendix C: Year 11 Background Survey Results
2013–2014
Item
n
%
1
0
7
9
3
12
16
2
2
93
21
5
6
1
22
3
72
13
108
14
1
8
27
22
2
3
<1%
0%
2%
3%
1%
4%
5%
1%
1%
28%
6%
2%
2%
<1%
7%
1%
22%
4%
33%
4%
<1%
2%
8%
7%
1%
1%
License Areas
Academically Advanced PreK-8
Adult Basic Education
Biology 5-8
Biology 8-12
Chemistry 5-8
Chemistry 8-12
Early Childhood PreK-2
Earth Science 5-8
Earth Science 8-12
Elementary 1-6
Elementary Mathematics 1-6
ELL PreK-6
ELL 5-12
General Science 1-6
General Science 5-8
Instructional Technology
Mathematics 8-12
Middle School
Middle School Mathematics 5-8
Middle School Math/Science 5-8
Physics 5-8
Physics 8-12
Students w/ Moderate Disability PreK-8
Students w/ Moderate Disability 5-12
Students w/ Severe Disability
Technology/Engineering 5-12
UMass Donahue Institute
Research and Evaluation Group
57
MMSP State-level Evaluation (Year 11)
Appendix D: High Need Districts
Appendix D: High Need District Eligibility Criteria
High Need Districts (See list below.):
1.
For proposals with a mathematics content focus: A district is considered to be a high need district if it has a
mathematics proficiency index for grades 4-8 that is below the state target for Cycle II for MMSP Year 1projects or
below the state target for Cycle III for MMSP Year 2 projects. Priority will be given to high need districts with two or
more schools identified for improvement in mathematics.
2. For proposals with a science and/or technology/engineering content focus: A district is considered to be a high need
district if it has a science proficiency index for grades 5-8 in 2003 that is at or below the 20th percentile for the state.
In addition, a high need district must demonstrate that there is a high number or percentage of teachers in the district who are
teaching in the academic subject or grade level for which they have not demonstrated subject matter competency through
licensure or completion of the professional development activities in their HOUSSE plans.
An interested district that is not identified as high need is encouraged to contact a high need district to explore becoming a
partner in the proposed program (e.g., vocational technical schools are encouraged to contact feeder school districts).
MA FY2004 High Need Districts
DISTRICT
AVON
BARNSTABLE
BOSTON
BROCKTON
CAMBRIDGE
CHELSEA
CHICOPEE
CLARKSBURG
EASTHAMPTON
EVERETT
FAIRHAVEN
FALL RIVER
FITCHBURG
FLORIDA
GARDNER
GREENFIELD
HAVERHILL
HOLBROOK
HOLYOKE
HULL
LAWRENCE
LOWELL
LYNN
MALDEN
MEDFORD
METHUEN
NEW BEDFORD
NORTH ADAMS
PITTSFIELD
PROVINCETOWN
RANDOLPH
REVERE
SALEM
SOMERVILLE
SOUTHBRIDGE
SPRINGFIELD
TAUNTON
MATH


























UMass Donahue Institute
Research and Evaluation Group
SCIENCE
T/E




































WALTHAM
WARE
DISTRICT
WAREHAM
WEBSTER
WEST SPRINGFIELD
WESTFIELD
WINCHENDON
WINTHROP
WORCESTER
ABBY KELLEY FOSTER CS
ATLANTIS CS
BENJAMIN BANNEKER CS
BOSTON RENAISSANCE CS
CONSERVATORY LAB CS
EDWARD BROOKE CS
FREDERICK DOUGLASS CS
LAWRENCE FAMILY DEV CS
LOWELL COMMUNITY CS
NEIGHBORHOOD HOUSE CS
NEW BEDFORD GLOBAL CS
NEW LEADERSHIP HMCS
NORTH CENTRAL REG CS
ROBERT M. HUGHES CS
SABIS INTERNATIONAL CS
SEVEN HILLS CS
SOMERVILLE CS
UPHAMS CORNER CS
ATHOL-ROYALSTON RSD
BERKSHIRE HILLS RSD
FRONTIER RSD
GILL-MONTAGUE RSD
HAMPSHIRE RSD
HAWLEMONT RSD
MOUNT GREYLOCK RSD
RALPH C MAHAR RSD

MATH























SCIENCE
T/E































58
MMSP State-level Evaluation (Year 11)
Appendix D: High Need Districts
MA FY2005 High Need Districts
DISTRICT
Grade 5
ATTLEBORO
SCI
Grade 8 Grades 4SCI
8 Math


AVON
BOSTON



BOURNE

BROCKTON
CAMBRIDGE






CHELSEA



CHICOPEE



CLARKSBURG



DEDHAM

DRACUT

EAST BRIDGEWATER

EASTHAMPTON

EVERETT



FAIRHAVEN
FALL RIVER



FITCHBURG



FLORIDA

GARDNER

HAVERHILL


GREENFIELD



HOLBROOK
HOLYOKE


HULL


LAWRENCE



LEOMINSTER

LOWELL



LYNN



MALDEN



MEDFORD


METHUEN


NEW BEDFORD



NORTH ADAMS



ORANGE

OXFORD

QUINCY

PITTSFIELD

RANDOLPH

REVERE




Grade 5
Grade 8 Grades 4-
SCI
SCI
8 Math
SOUTHBRIDGE



SPRINGFIELD
TAUNTON
WALTHAM
WARE
WAREHAM
WEBSTER
WESTFIELD
WINCHENDON
WINTHROP
WORCESTER
ACADEMY OF STRATEGIC CS
BENJAMIN BANNEKER CS
FREDERICK DOUGLASS CS
EDWARD BROOKE CS
CONSERVATORY LAB CS
COMMUNITY DAY CS
SABIS INTERNATIONAL CS
NEIGHBORHOOD HOUSE CS
ABBY KELLEY FOSTER REG CS
SO.BOSTON HARBOR ACAD CS
ROBERT M. HUGHES ACAD CS
LAWRENCE FAMILY DEV. CS
LOWELL COMMUNITY CS
NEW LEADERSHIP HMCS
NEW BEDFORD GLOBAL HMCS
NORTH CENTRAL CS
BOSTON RENAISSANCE CS
SEVEN HILLS CS
SOMERVILLE CS
PROSPECT HILL ACADEMY CS
UPHAMS CORNER CS
ATLANTIS CS
ADAMS-CHESHIRE
ATHOL-ROYALSTON
BERKSHIRE
FREETOWN-LAKEVILLE
GATEWAY
GILL-MONTAGUE
HAMPSHIRE
HAWLEMONT
NEW SALEM-WENDELL





































































ROCKLAND
SALEM



SOMERVILLE



UMass Donahue Institute
Research and Evaluation Group
DISTRICT
59
MMSP State-level Evaluation (Year 11)
Appendix D: High Need Districts
MA FY2006 High Need Districts
DISTRICT
ATTLEBORO
Grade 5
SCIENCE

Grade 8
SCIENCE
Grades 4-8
MATH

AVON
DISTRICT
Grade 5
SCIENCE
WALTHAM

WARE

BOSTON

BOURNE

BROCKTON



WESTFIELD
CAMBRIDGE



WINCHENDON
CHELSEA



WINTHROP

CHICOPEE






WORCESTER
ACADEMY OF STRATEGIC
CS
BENJAMIN BANNEKER CS

CLARKSBURG



DRACUT
EAST
BRIDGEWATER
EASTHAMPTON


EVERETT









EDWARD BROOKE CS

CONSERVATORY LAB CS


COMMUNITY DAY CS


SABIS INTERNATIONAL CS




FALL RIVER



NEIGHBORHOOD HOUSE CS


FITCHBURG



ABBY KELLEY FOSTER
REGIONAL CS


SOUTH BOSTON HARBOR
ACADEMY CS


ROBERT M. HUGHES
ACADEMY CS



LAWRENCE FAMILY
DEVELOPMENT CS



LOWELL COMMUNITY CS

FLORIDA

GARDNER

GREENFIELD
HAVERHILL


HOLBROOK

HOLYOKE


HULL


LAWRENCE


Grades 4-8
MATH


FREDERICK DOUGLASS CS

FAIRHAVEN
WAREHAM
WEBSTER
DEDHAM
Grade 8
SCIENCE





NEW LEADERSHIP HMCS


LOWELL





LYNN



NEW BEDFORD GLOBAL
HMCS
MALDEN



MEDFORD


METHUEN


NEW BEDFORD



NORTH ADAMS



ORANGE
LEOMINSTER
NORTH CENTRAL CS


BOSTON RENAISSANCE CS



SEVEN HILLS CS



SOMERVILLE CS

PROSPECT HILL ACADEMY
CS


UPHAMS CORNER CS
OXFORD


ATLANTIS CS
QUINCY


ADAMS-CHESHIRE RSD

ATHOL-ROYALSTON RSD

PITTSFIELD

RANDOLPH

BERKSHIRE HILLS RSD

FREETOWN-LAKEVILLE RSD


GATEWAY RSD


REVERE

ROCKLAND

SALEM



GILL-MONTAGUE RSD
SOMERVILLE



HAMPSHIRE RSD
SOUTHBRIDGE



HAWLEMONT RSD
SPRINGFIELD



NEW SALEM-WENDELL RSD
TAUNTON


UMass Donahue Institute
Research and Evaluation Group








60
MMSP State-level Evaluation (Year 11)
Appendix D: High Need Districts
MA FY2007 High Need Districts
DISTRICT
Science/Tech.
Engineering
ATTLEBORO
Math

BOSTON

BROCKTON

BROOKFIELD
Science/Tech.
Engineering
Math
REVERE

ROCKLAND


SALEM


SAUGUS


SEEKONK


BARNSTABLE
DISTRICT
CAMBRIDGE


SOMERVILLE
CHELSEA


SOUTHAMPTON
CHICOPEE

SOUTHBRIDGE
CLINTON

SOUTH HADLEY
DOUGLAS

SPRINGFIELD
EASTHAMPTON

STOUGHTON

ERVING

TAUNTON

EVERETT

WALTHAM

FAIRHAVEN

WARE

FALL RIVER

WAREHAM

WEBSTER
FRAMINGHAM

WESTFIELD

FREETOWN

WESTPORT

GARDNER

WEST SPRINGFIELD

GLOUCESTER

WINCHENDON

GRANVILLE

WINTHROP

GREENFIELD

WORCESTER
HAVERHILL

EXCEL ACADEMY CS

HOLBROOK

FOUR RIVERS CS

HOLYOKE

BERKSHIRE ARTS CS


ACADEMY OF STRATEGIC
CS
FITCHBURG


HUDSON
LAWRENCE


LEE

LEICESTER



















METHUEN

MIDDLEBOROUGH

MONSON

NAHANT





NORTHAMPTON

NORTH BROOKFIELD

NORTON

OXFORD

PALMER

PITTSFIELD

QUINCY
UMass Donahue Institute
Research and Evaluation Group


MASHPEE
RANDOLPH



NORTH ADAMS

CONSERVATORY LAB CS
SABIS INTERNATIONAL
CS
ROBERT M. HUGHES
ACAD CS
LAWRENCE FAMILY DEV
CS
LOWELL COMMUNITY CS
MALDEN
NEW BEDFORD


LUDLOW
LYNN

MURDOCH MIDDLE CS
LEOMINSTER
LOWELL
SMITH LEADERSHIP
ACAD CS
BENJAMIN BANNEKER CS

NEW LEADERSHIP HMCS
NEW BEDFORD GLOBAL
CS
NORTH CENTRAL CS










PIONEER VALLEY
PERFORMING
BOSTON RENAISSANCE
CS
SALEM ACADEMY CS






SEVEN HILLS CS
PROSPECT HILL ACAD
CS
SOUTH SHORE CS

UPHAMS CORNER CS





61
MMSP State-level Evaluation (Year 11)
DISTRICT
Science/Tech.
Engineering
Appendix D: High Need Districts
Math
ATLANTIS CS

ADAMS-CHESHIRE REG.

ATHOL-ROYALSTON

BERKSHIRE HILLS

FREETOWN-LAKEVILLE

GATEWAY

GILL-MONTAGUE

HAMPSHIRE

MOHAWK TRAIL

NARRAGANSETT

PIONEER VALLEY

RALPH C MAHAR

SILVER LAKE

UMass Donahue Institute
Research and Evaluation Group
62
MMSP State-level Evaluation (Year 11)
Appendix D: High Need Districts
MA FY2008 and FY2009 High Need Districts
MA FY2009 and FY2010 High Need Districts (same as MA FY2008 and FY2009 High Need Districts)
Criteria: A high-need district in science and technology/engineering is a district that has a grade 8 and a high school science
CPI of less than 60. A high-need district in mathematics is a district that has been identified for corrective action in
mathematics, or districts with one or more Commonwealth Priority Schools identified for mathematics.
DISTRICT
Science
Math


NEW BEDFORD

NORTH ADAMS
BOSTON
BRIDGEWATER-RAYNHAM
DISTRICT
Science
Math



BROCKTON


NORTH BROOKFIELD
CAMBRIDGE


PEABODY
CHICOPEE


PITTSFIELD
EASTHAMPTON

EVERETT


RANDOLPH


FALL RIVER


REVERE


FALMOUTH

SALEM
FITCHBURG

SOMERVILLE


SOUTHBRIDGE


GARDNER

GATEWAY







PLYMOUTH

SPENCER-EAST BROOKFIELD

GLOUCESTER

SPRINGFIELD

GREENFIELD

WAREHAM

HAVERHILL

WESTFIELD

HOLBROOK


WOBURN
HOLYOKE


WORCESTER

HULL

BERKSHIRE ARTS CS

LAWRENCE


LOWELL





MARLBOROUGH

NEW BEDFORD GLOBAL CS

MEDFORD

METHUEN



LYNN
UMass Donahue Institute
Research and Evaluation Group


BENJAMIN BANNEKER CS
COMMUNITY CS OF
CAMBRIDGE
NEW LEADERSHIP HMCS
LUDLOW



63
MMSP State-level Evaluation (Year 11)
Appendix D: High Need Districts
MA FY2010 and FY2011 High Need Districts
Criteria: High-need districts are defined as districts in corrective action or single school districts in corrective action or
restructuring status under No Child Left Behind.
DISTRICT
Agawam
Pittsfield
Boston
Ralph C. Mahar
Bridgewater-Raynham
Randolph
Brockton
Revere
Chelsea
Salem
Chicopee
Somerville
Everett
Southbridge
Fall River
Springfield
Fitchburg
Waltham
Gardner
Wareham
Gloucester
Westfield
Greater Lawrence Regional Vocational Technical
Weymouth
Greenfield
Woburn
Hampshire
Worcester
Haverhill
Abby Kelley Foster Charter Public
Holbrook
Benjamin Banneker Charter Public
Holyoke
Berkshire Arts and Technology Charter Public
Lawrence
Boston Renaissance Charter
Leominster
Lowell Community Charter Public
Lowell
Mystic Valley Regional Charter
Lynn
New Leadership Charter
Malden
North Central Charter Essential
Marlborough
Sabis International Charter
Medford
Seven Hills Charter
Methuen
Silver Hill Horace Mann Charter
New Bedford
Smith Leadership Academy Charter Public
Orange
South Shore Charter Public
Peabody
UMass Donahue Institute
Research and Evaluation Group
64
MMSP State-level Evaluation (Year 11)
Appendix D: High Need Districts
MA FY2012 High Need Districts
FY2012 Massachusetts Math/Science Partnership Qualifying High Needs Districts List
Lead LEA Criteria: The Lead LEA must be a Level 3 or 4 district as identified by the Massachusetts Department
of Elementary and Secondary Education (Department).
For the purpose of the MMSP, Level 3 & 4 districts are considered high need districts.
Level 3 districts with one or more schools among the lowest-performing 20% based on quantitative indicators.
Level 4 districts identified by quantitative and qualitative indicators through a district review; districts with one or
more schools among the lowest-performing and least improving 2% based on quantitative indicators.
District Name
District Name
Charter District
Athol-Royalston
North Andover
Abby Kelly Foster Charter
Bellingham
Northampton
Boston
Northampton-Smith
Atlantis Charter
Berkshire Arts and Technology
Charter
Brockton
Northeast Metro Voc
Boston Day and Evening Academy
Chelsea
Orange
Boston Renaissance Charter
Chicopee
Pittsfield
Holyoke Community Charter
Dennis-Yarmouth
Quabbin
Lowell Community Charter
Everett
Quaboag Regional
Mystic Valley Charter
Fall River
Quincy
Leadership Charter
Fitchburg
Randolph
North Central Essential Charter
Framingham
Ralph C Mahar
Sabis International Charter
Gardner
Gill-Montague
Gloucester
Revere
Salem
So Middlesex Voc Tech Reg
Salem Academy Charter
Seven Hills Charter
Silver Hill Charter
Greater Lawrence RVT
Somerville
Smith Leadership Academy Charter
Greater Lowell Voc Tec
Southbridge
South Shore Charter
Haverhill
Southeastern Reg Voc Tech
Holbrook
Springfield
Holyoke
Stoneham
Lawrence
Taunton
Leominster
Waltham
Lowell
Ware
Lynn
Wareham
Malden
Webster
Medford
West Springfield
Middleborough
Mohawk Trail
Nantucket
Narragansett
Westfield
Winchendon
Winthrop
Worcester
New Bedford
MMSP State-level Evaluation (Year 11)
Appendix D: High Need Districts
MA FY2013 High Need Districts
Massachusetts Math/Science Partnership Qualifying High Need Districts List (FY2012-13)
Lead LEA Criteria: The Lead LEA must be a Level 3 or 4 district as identified by the Massachusetts Department of Elementary and Secondary
Education (Department). (http://www.doe.mass.edu/apa/accountability/default.html)
For the purpose of the MMSP, Level 3 & 4 districts are considered high need districts.
District Name
Adams-Cheshire
Athol-Royalston
Amherst-Pelham
Beverly
Boston
Brockton
Chelsea
Chicopee
Dracut
Easthampton
Everett
Fairhaven
Fall River
Fitchburg
Framingham
Franklin County
Freetown-Lakeville Regional
Gardner
Gill-Montague
Gloucester
Greater Lawrence RVT
Greater Lowell Voc Tec
Greenfield
Haverhill
Holbrook
Holyoke
Lawrence
Leominster
Lowell
Lynn
Malden
Marlborough
Methuen
Monson
Narragansett
District Name
New Bedford
North Adams
Northampton
Northampton-Smith
Northbridge
Orange
Oxford
Palmer
Peabody
Pittsfield
Plymouth
Quincy
Randolph
Ralph C Mahar
Revere
Rockland
Salem
Saugus
Somerville
Southbridge
Southeastern Reg Voc Tech
Spencer-E Brookfield
Springfield
Stoughton
Taunton
Waltham
Ware
Wareham
Watertown
Webster
Westfield
West Springfield
Weymouth
Winchendon
Worcester
Charter District
Boston Renaissance Charter
Global Learning Charter
Martin Luther King Jr. Charter
New Leadership Charter
Phoenix Charter Academy
Seven Hills Charter
UP Academy Charter Boston
Note: For the purpose of this
continuation grant, previously
identified Level 3 & Level 4
districts under which the original
competitive grant was awarded
continue to be eligible as high
needs partners.
05/02/13
MMSP State-level Evaluation (Year 11)
Appendix D: High Need Districts
MA FY2014 High Need Districts
District Name
Adams-Cheshire
District Name
New Leadership Charter (District)
Athol-Royalston
North Adams
Billerica
Northampton
Boston
Northbridge
Boston Day and Evening Academy Charter (District)
Orange
Boston Renaissance Charter Public (District)
Oxford
Brockton
Palmer
Cambridge
Pathfinder Regional Vocational Technical
Chelsea
Peabody
Chicopee
Phoenix Charter Academy (District)
Dennis-Yarmouth
Pittsfield
Dracut
Quincy
Easthampton
Randolph
Everett
Revere
Fall River
Salem
Fitchburg
Somerville
Framingham
Southbridge
Gardner
Southeastern Regional Vocational Technical
Gateway
Spencer-E Brookfield
Gill-Montague
Springfield
Global Learning Charter Public (District)
Stoughton
Gloucester
Taunton
Greater Lawrence Regional Vocational Technical
Waltham
Haverhill
Ware
Holbrook
Wareham
Holyoke
Webster
Hudson
West Springfield
Lawrence
Westfield
Leominster
Weymouth
Lowell
Winchendon
Lynn
Worcester
Malden
Marlborough
Martin Luther King Jr. Charter School of Excellence (District)
Mashpee
Mendon-Upton
Methuen
Middleborough
Monson
Narragansett
New Bedford
MMSP State-level Evaluation (Year 11)
Appendix E: Enrollment and Attrition Rates by Course
Appendix E: Enrollment and Attrition Rates by Course
Enrollment and Attrition Information During 2013 – 2014 as Reported by Partnerships
Partnership
EduTron – RevereEverett-Saugus
Lesley – Brockton
Lesley – Springfield
MCLA
Northeastern
Fitchburg – NCIS
Framingham
Global Learning
Public Charter
Lesley – SpringfieldEasthampton
Worcester Public
Schools
West Springfield
Public Schools
TOTAL
Course Title
An Odyssey of Mathematics Grades 7-12
Common Core Mathematics 7-12
Common Core Mathematics Grades 4-8
Subtotal
Functions and Algebra II
Geometry and Measurement II
Number Theory
Operations and Algebraic Thinking
Probability
Ratio, Rate, and Proportion
Ratio, Rate, and Proportion
Science and Inquiry in the Elementary School
Statistics and Data Analysis
Using Fraction as Number
Subtotal
Calculus
Calculus
Functions and Algebra II
Functions and Algebra II
Geometry and Measurement II
Ratio, Rate, and Proportion
Statistics and Data Analysis
Statistics and Data Analysis
Subtotal
Alternative Energy for Middle School Teachers
Mathematical Modeling for Middle School Teachers
Subtotal
ES I – Earth Systems: Weather and Water
From Science to Engineering: Pre-engineering Design
Mathematics Essentials for Science Teachers
Physics II – Waves, Electricity & Magnetism
Subtotal
Scale, Proportion and Quantity
Subtotal
Strategies to Improve Instruction in Middle School Science
Subtotal
Engineering Design Process
Subtotal
Expressions and Equations
Using Fraction as Number
Subtotal
Curriculum and Knowing in Math (for Middle School Teachers)
Subtotal
N/A
Number
Enrolled First
Day
37
21
28
86
18
10
15
16
19
13
12
14
15
21
153
13
17
12
24
17
27
17
25
152
4
6
10
15
18
14
17
64
18
18
18
18
25
25
8
9
17
12
12
--
Number
Completed
Course
37
19
28
84
16
10
15
16
18
13
11
14
14
21
148
13
17
11
23
17
27
16
25
149
3
6
9
15
14
10
15
54
16
16
15
15
25
25
6
9
15
10
10
--
555
525
Attrition
Rate
0%
10%
0%
2%
11%
0%
0%
0%
5%
0%
8%
0%
7%
0%
3%
0%
0%
8%
4%
0%
0%
6%
0%
2%
25%
0%
10%
0%
22%
29%
12%
16%
11%
11%
17%
17%
0%
0%
25%
0%
12%
17%
17%
-5%
MMSP State-level Evaluation (Year 11)
Appendix F: Pre-Post Scores
Appendix F: Mean Percentage Scores for Pre- & Post-course Tests
Mean Percentage Scores for Pre- & Post-course Tests During 2013 – 2014
Partnership
EduTron – RevereEverett-Saugus
Lesley – Brockton
Lesley – Springfield
MCLA
Northeastern
Fitchburg – NCIS
Framingham
Global Learning Public
Charter
Lesley – SpringfieldEasthampton
Worcester Public
Schools
West Springfield Public
Schools
Number
Completed
Course
37
19
28
16
10
15
16
18
13
11
14
14
21
13
17
11
23
17
27
16
25
Mean
Pre-test
(%)
62
46
72
12
12
20
29
35
6
25
33
12
8
6
8
27
6
13
13
11
10
Mean
Post-test
(%)
81
63
90
28
26
42
66
56
22
44
48
21
15
32
50
62
17
50
25
23
22
Change in
Mean
(% points)
19
17
18
16
14
22
37
21
16
19
15
9
7
26
42
35
11
37
12
12
12
3
12
17
5
N/A
6
10
12
2
Yes
15
40
47
7
Yes
14
8
10
2
Yes
10
15
79
29
94
36
15
7
Yes
Yes
Scale, Proportion and Quantity
Strategies to Improve Instruction in Middle
School Science
Engineering Design Process
16
22
25
3
Yes
15
18
24
6
Yes
25
17
20
3
Yes
Expressions and Equations
Using Fraction as Number
Curriculum and Knowing in Mathematics (for
Middle School Teachers)
N/A
6
9
12
10
14
15
2
5
No
Yes
10
9
13
4
Yes
--
--
--
--
--
Course Title
An Odyssey of Mathematics Grades 7-12
Common Core Mathematics 7-12
Common Core Mathematics Grades 4-8
Functions and Algebra II
Geometry and Measurement II
Number Theory
Operations and Algebraic Thinking
Probability
Ratio, Rate, and Proportion
Ratio, Rate, and Proportion
Science and Inquiry in the Elementary School
Statistics and Data Analysis
Using Fraction as Number
Calculus
Calculus
Functions and Algebra II
Functions and Algebra II
Geometry and Measurement II
Ratio, Rate, and Proportion
Statistics and Data Analysis
Statistics and Data Analysis
Alternative Energy for Middle School
Teachers
Mathematical Modeling for Middle School
Teachers
ES I – Earth Systems: Weather and Water
From Science to Engineering: Pre-engineering
Design
Mathematics Essentials for Science Teachers
Physics II – Waves, Electricity & Magnetism
p <.05
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
MMSP State-level Evaluation (Year 11)
Appendix G: High Need Districts, by Partnership
Appendix G: High Need Districts for All Funding Periods, by Partnership
High Need Districts for Cohort 1–Cohort 3, by Partnership
Number of Participants from High Need Districts
Partnership
EduTron (M)
Harvard University (M)
Lesley University (M)
MCLA – Science (S)
UMass Donahue Institute
Research and Evaluation Group
High Need
Districts
Fitchburg
Gardner
Subtotal
Boston
Boston Renaiss. CS
Cambridge
Fall River
Lowell
Malden
New Bedford
Somerville
Somerville CS/
Prospect Hill
Academy
Southbridge
Subtotal
Malden
Adams-Cheshire
Clarksburg
Florida
Mount Greylock
North Adams
Feb04–
Aug04
Sep04–
Aug05
37
14
51 (79%)
1
3
5
0
0
0
0
0
53
4
57 (88%)
3
2
4
0
1
0
1
2
0
0
9 (39%)
21 (21%)
Planning Year
5
0
18 (33%)
16 (19%)
6
1
2
0
5
Sep05–
Aug06
28
9
37 (84%)
10
0
9
4
0
3
0
4
0
1
31 (39%)
14 (19%)
5
1
1
0
3
Sep06–
Aug07
Cohorts
1&2
N/A
N/A
N/A
0
2
3
4
0
1
0
2
1
0
13 (54%)
3 (25%)
5
1
2
2
3
Sep06–
Aug07
Cohort 3
Sep07–Aug08
Cohort 3
Sep08–
Aug09
Cohort 3
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
70
MMSP State-level Evaluation (Year 11)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 1–Cohort 3, by Partnership
Number of Participants from High Need Districts
Partnership
MCLA – Science (S)
Salem State College (M)
Springfield/Holyoke
Public Schools (S)
Wareham PS (M)
WPI – Math (M)
UMass Donahue Institute
Research and Evaluation Group
High Need
Districts
Subtotal
Boston
Chelsea
Haverhill PS
Lynn
Salem
Somerville
Subtotal
Holyoke
Holyoke Comm. CS
Springfield
Subtotal
Wareham
Abby Kelley
Foster CS
Athol-Royalston
Berkshire Hills
Boston
Brockton
Cambridge
Chicopee
Fall River
Fitchburg
Lawrence Fam.
Devt. CS
Lowell
Community CS
New Bedford
Feb04–
Aug04
Sep04–
Aug05
0
0
1
32
18
0
51 (93%)
6
0
32
38(100%)
17 (46%)
14 (100%)
0
1
20
32
16
0
69 (86%)
17
0
31
48(100%)
11 (61%)
Sep05–
Aug06
10 (100%)
1
1
16
41
14
0
73 (79%)
19
0
28
47 (96%)
N/A
Sep06–
Aug07
Cohorts
1&2
13 (93%)
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep06–
Aug07
Cohort 3
Sep07–Aug08
Cohort 3
Sep08–
Aug09
Cohort 3
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0
2
3
N/A
N/A
N/A
N/A
1
0
0
0
0
4
0
0
0
1
0
1
0
1
1
0
0
0
1
0
1
2
0
1
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0
0
3
N/A
N/A
N/A
N/A
0
0
1
N/A
N/A
N/A
N/A
2
4
0
N/A
N/A
N/A
N/A
71
MMSP State-level Evaluation (Year 11)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 1–Cohort 3, by Partnership
Number of Participants from High Need Districts
Partnership
WPI – Math (M)
High Need
Districts
Feb04–
Aug04
MCLA – Math (M)
North Adams
Pittsfield
Ralph C. Mahar
Seven Hills CS
Somerville
Webster
Winchendon
Worcester
Subtotal
Adams-Cheshire
0
0
0
4
15(63%)
Started: Year 2
UMass Amherst (M)
North Adams
Pittsfield
Subtotal
Athol-Royalston
N/A
N/A
N/A
Started: Year 2
Chicopee
Easthampton
Gateway
Gill-Montague
Greenfield
Holyoke
Holyoke
Community CS
Ludlow
North Adams
Ralph C. Mahar
Springfield
Westfield
N/A
N/A
N/A
N/A
N/A
N/A
UMass Donahue Institute
Research and Evaluation Group
2
0
2
0
Sep04–
Aug05
0
0
0
0
1
1
8
7
27 (41%)
Planning Year
N/A
N/A
N/A
0
Sep05–
Aug06
0
2
0
2
0
0
6
10
32 (43%)
0
Sep06–
Aug07
Cohorts
1&2
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
3
Sep06–
Aug07
Cohort 3
Sep07–Aug08
Cohort 3
Sep08–
Aug09
Cohort 3
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
1
0
1 (9%)
0
2
1
6 (86%)
1
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0
0
0
0
2
5
2
0
0
0
0
5
1
1
1
2
0
3
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0
0
1
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0
0
0
7
2
0
0
0
4
2
2
1
1
7
2
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
72
MMSP State-level Evaluation (Year 11)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 1–Cohort 3, by Partnership
Number of Participants from High Need Districts
Partnership
UMass Amherst (M)
EduTron Lowell (M/S)
EduTron Fitchburg (M)
Lesley University (M)
North Shore (S)
UMass Donahue Institute
Research and Evaluation Group
High Need
Districts
Subtotal
Lowell
Fitchburg
Gardner
Leominster
Subtotal
Attleboro
Brockton
Fairhaven
Fall River
Haverhill PS
Holyoke
Malden
Middleborough
New Bedford
Northampton
Randolph
Revere
Saugus
Silver Hill Charter
Somerville
Taunton
Ware
Subtotal
Boston
Fitchburg
Holyoke
Feb04–
Aug04
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep04–
Aug05
16 (64%)
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep05–
Aug06
13 (37%)
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep06–
Aug07
Cohorts
1&2
23 (34%)
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep06–
Aug07
Cohort 3
N/A
54(100%)
17
7
10
34 (100%)
0
13
3
26
29
29
1
0
4
0
13
0
2
0
0
0
0
120 (94%)
0
1
0
Sep07–Aug08
Cohort 3
N/A
66(100%)
20
7
26
53 (98%)
1
3
1
18
23
18
0
0
0
0
14
2
5
0
11
0
1
97 (90%)
0
0
0
Sep08–
Aug09
Cohort 3
N/A
72 (100%)
19
12
34
65 (97%)
1
5
1
15
22
17
1
1
0
1
11
8
3
1
17
2
0
106 (95%)
4
1
1
73
MMSP State-level Evaluation (Year 11)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 1–Cohort 3, by Partnership
Number of Participants from High Need Districts
Partnership
North Shore (S)
UMass Amherst C3 (M/S)
Salem State College (M)
UMass Donahue Institute
Research and Evaluation Group
Feb04–
Aug04
Sep04–
Aug05
Sep05–
Aug06
Lynn
Revere
Somerville
Lowell Comm. CS
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep06–
Aug07
Cohorts
1&2
N/A
N/A
N/A
N/A
Subtotal
Athol Royalston
Chicopee
Easthampton
Gateway
Greenfield
Gill-Montague
Holyoke
Ludlow
Lynn
New Leadership LS
North Adams
Pittsfield
South Hadley
Springfield
West Springfield
Subtotal
Boston
Chelsea
Everett
Gloucester
Haverhill PS
Lynn
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
High Need
Districts
Sep06–
Aug07
Cohort 3
Sep07–Aug08
Cohort 3
Sep08–
Aug09
Cohort 3
0
0
16
0
3
9
14
1
0
8
13
0
17 (41%)
1
5
0
1
1
0
3
2
0
1
0
0
2
3
3
22 (46%)
1
1
3
6
4
10
27 (40%)
0
3
1
0
0
1
4
0
0
0
1
1
0
9
0
19 (38%)
0
0
1
3
3
10
27 (53%)
0
3
1
1
0
1
2
0
1
0
1
2
0
12
1
25 (47%)
N/A
N/A
N/A
N/A
N/A
N/A
74
MMSP State-level Evaluation (Year 11)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 1–Cohort 3, by Partnership
Number of Participants from High Need Districts
Partnership
Salem State College (M)
SE/Cape (S)
WPI – Science (S)
Worcester PS (M)
UMass Donahue Institute
Research and Evaluation Group
High Need
Districts
Feb04–
Aug04
Sep04–
Aug05
Sep05–
Aug06
Malden
Methuen
Peabody
Revere
Salem
Winthrop
Worcester
Subtotal
Barnstable
Brockton
Fall River
Horace Mann CS
Lawrence
New Bedford
Subtotal
Worcester
Southbridge
Subtotal
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep06–
Aug07
Cohorts
1&2
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Worcester
N/A
N/A
N/A
N/A
Sep06–
Aug07
Cohort 3
1
0
0
0
3
1
1
41
1
20
0
2
0
8
31
3
0
3
(82%)
(66%)
(16%)
(16%)
34 (83%)
Sep07–Aug08
Cohort 3
2
1
2
1
3
0
0
26
5
22
5
0
0
8
40
7
0
7
(65%)
(51%)
(54%)
(54%)
N/A
Sep08–
Aug09
Cohort 3
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0
32
10
0
1
2
45(50%)
3
1
4 (22%)
N/A
75
MMSP State-level Evaluation (Year 11)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 4–Cohort 7, by Partnership
Partnership
Boston PS (M)
Brockton PS (M)
Gateway RSD (S)
UMass Donahue Institute
Research and Evaluation Group
High Need Districts
Boston
Medford
Subtotal
Bridgewater-Raynham
Brockton
Fall River
Falmouth
Freetown/Lakeville
New Bedford
Plymouth
Quincy
Randolph
Seekonk
South Shore CS
Swansea
Wareham
Weymouth
Subtotal
Agawam
Chicopee
Easthampton
Gateway
Hampshire
Holyoke
Springfield
Westfield
Subtotal
Sep08–
Aug09
Cohort 4
40
0
40 (100%)
0
21
9
0
0
0
0
0
0
0
0
0
0
0
30 (59%)
0
0
1
6
0
2
8
0
17 (39%)
Sep09–
Aug10
Cohort 4
174
0
174 (100%)
0
28
12
1
3
0
3
2
0
1
0
1
0
0
51 (45%)
0
0
3
7
0
9
0
0
19 (100%)
Sep10–
Aug11
Cohorts
4&5
112
1
113 (99%)
1
24
13
0
0
1
0
0
2
0
1
0
3
4
49 (56%)
4
2
2
6
4
2
2
12
34 (72%)
Sep11–
Aug12
Cohorts
5&6
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep12–
Aug13
Cohorts
5&6
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep13–
Aug14
Cohorts
6&7
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
76
MMSP State-level Evaluation (Year 11)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 4–Cohort 7, by Partnership
Partnership
High Need Districts
Lesley Springfield (M)
Cohort 4
Agawam
Chicopee
Holyoke
Springfield
Westfield
Subtotal
Boston
Bridgewater-Raynham
Fitchburg
Lawrence
Lynn
Malden
Medford
Pioneer Charter School of
Science
Quincy
Randolph
Revere
Somerville
Waltham
Weymouth
Subtotal
Randolph
Weymouth
Subtotal
New Leadership CS
Springfield
Subtotal
Greater North Shore (S)
Randolph PS (S)
Springfield College (S)
UMass Donahue Institute
Research and Evaluation Group
Sep08–
Aug09
Cohort 4
0
1
9
35
4
49 (94%)
26
0
0
0
3
0
Sep09–
Aug10
Cohort 4
0
1
6
59
5
71 (84%)
26
0
0
0
4
1
0
0
1
0
0
1
0
0
0
0
31 (50%)
6
0
6 (25%)
0
26
26 (96%)
0
1
8
12
0
0
52 (46%)
18
0
18 (41%)
0
30
30 (100%)
Sep10–
Aug11
Cohorts
4&5
1
0
5
29
5
40 (93%)
35
1
2
1
0
5
3
0
1
0
2
9
3
1
63 (50%)
7
1
8 (24%)
1
20
21 (100%)
Sep11–
Aug12
Cohorts
5&6
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep12–
Aug13
Cohorts
5&6
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep13–
Aug14
Cohorts
6&7
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
77
MMSP State-level Evaluation (Year 11)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 4–Cohort 7, by Partnership
Partnership
High Need Districts
Sep08–
Aug09
Cohort 4
Boston University (M)
Boston
Brockton
Cambridge
Chelsea
Chicopee
Falmouth
Haverhill
Lawrence
Lowell
Medford
Peabody
Randolph
Salem
Somerville
Waltham
Weymouth
Woburn
Worcester
Subtotal
Boston
Brockton
Chelsea
Lawrence
Lowell
Lynn
Revere
Worcester
Region Voc-Tech Greater
3
0
0
0
0
2
2
7
0
0
0
0
1
0
0
0
0
1
16 (39%)
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
EduTronWorc-Lowell (M)
UMass Donahue Institute
Research and Evaluation Group
Sep09–
Aug10
Cohort 4
3
0
1
0
0
0
0
1
2
0
0
0
1
1
0
0
1
0
10 (29%)
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep10–
Aug11
Cohorts
4&5
1
2
3
1
1
0
0
5
2
1
1
1
0
1
1
3
0
0
23 (58%)
1
1
2
3
28
2
1
27
1
Sep11–
Aug12
Cohorts
5&6
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0
0
0
0
27
0
0
22
0
Sep12–
Aug13
Cohorts
5&6
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0
0
0
0
27
0
0
22
0
Sep13–
Aug14
Cohorts
6&7
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
78
MMSP State-level Evaluation (Year 11)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 4–Cohort 7, by Partnership
Partnership
(continued)
Everett (S)
EduTron – RevereEverett-Saugus
Cohort 6 (M)
UMass Donahue Institute
Research and Evaluation Group
High Need Districts
Lawrence
Subtotal
Boston
Chelsea
Chicopee
Everett
Holbrook
Malden
Medford
Mystic Valley Regional CS
North Andover
Rockland
Seekonk
Somerville
Waltham
Subtotal
Dracut
Everett
Haverhill
Hudson
Framingham
Lawrence
Lowell
Lynn
Medford
Malden
New Bedford
North Central Charter
Sep08–
Aug09
Cohort 4
Sep09–
Aug10
Cohort 4
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep10–
Aug11
Cohorts
4&5
Sep11–
Aug12
Cohorts
5&6
66 (76%)
49 (100%)
2
2
2
1
0
0
6
6
1
0
1
6
11
10
0
3
0
1
1
0
1
0
1
0
1
0
27 (68%)
29 (85%)
N/A
N/A
N/A
23
N/A
N/A
N/A
N/A
N/A
3
N/A
1
N/A
N/A
N/A
N/A
N/A
N/A
N/A
2
N/A
N/A
N/A
1
Sep12–
Aug13
Cohorts
5&6
49 (100%)
0
0
1
4
0
7
11
0
0
0
0
0
1
24 (77%)
N/A
22
N/A
N/A
0
0
N/A
N/A
N/A
2
N/A
0
Sep13–
Aug14
Cohorts
6&7
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
1
25
1
1
N/A
N/A
1
4
3
N/A
1
N/A
79
MMSP State-level Evaluation (Year 11)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 4–Cohort 7, by Partnership
Partnership
(continued)
Lesley-Brockton (M)
Lesley-Springfield
Cohort 6 (M)
MCLA (M/S)
UMass Donahue Institute
Research and Evaluation Group
High Need Districts
Essential
Pittsfield
Revere
Southbridge
Springfield
Subtotal
Brockton
Fall River
New Bedford
Quincy
Randolph
Stoughton
Weymouth
Subtotal
Easthampton
Franklin County Reg Voc
Holyoke
Monson
Springfield
Westfield
West Springfield
Holyoke Community CS
Subtotal
Adams-Cheshire
Berkshire Arts and
Technology CS
North Adams
Pittsfield
Sep10–
Aug11
Cohorts
4&5
Sep08–
Aug09
Cohort 4
Sep09–
Aug10
Cohort 4
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep11–
Aug12
Cohorts
5&6
Sep12–
Aug13
Cohorts
5&6
1
0
16
5
N/A
N/A
N/A
N/A
47 (68%)
29 (56%)
30
56
N/A
N/A
0
1
3
18
0
2
N/A
N/A
N/A
N/A
33 (94%)
77 (82%)
N/A
N/A
N/A
N/A
5
7
N/A
N/A
14
24
1
2
0
3
N/A
N/A
20 (80%)
36 (71%)
N/A
N/A
4
N/A
N/A
11
1
1
50 (75%)
54
1
N/A
9
1
1
6
72 (86%)
1
1
7
1
30
1
2
1
44 (80%)
2
1
4
3
1
N/A
N/A
2
Sep13–
Aug14
Cohorts
6&7
80
MMSP State-level Evaluation (Year 11)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 4–Cohort 7, by Partnership
(continued)
Northeastern (S)
Subtotal
Boston
Brockton
Chelsea
Lynn
Malden
Medford
Quincy
Randolph
Somerville
Stoneham
Stoughton
Taunton
Waltham
Weymouth
Winthrop
Subtotal
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep10–
Aug11
Cohorts
4&5
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Fitchburg – NCIS (M/S)
Fitchburg
Leominster
Winchendon
Narragansett
Subtotal
Framingham
Marlborough
Subtotal
Wareham
Global Learning CS
Subtotal
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Partnership
Framingham (S)
Global Learning
Public Charter (S)
UMass Donahue Institute
Research and Evaluation Group
High Need Districts
Sep08–
Aug09
Cohort 4
Sep09–
Aug10
Cohort 4
Sep11–
Sep12–
Aug12
Aug13
Cohorts
Cohorts
5&6
5&6
6 (35%)
4 (36%)
11
20
4
0
1
0
0
1
1
0
1
0
7
4
1
0
1
0
0
1
N/A
N/A
1
0
0
1
N/A
N/A
1
0
29 (34%)
27 (43%)
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep13–
Aug14
Cohorts
6&7
7 (88%)
8
N/A
N/A
N/A
N/A
N/A
1
N/A
1
N/A
1
N/A
N/A
1
N/A
12 (32%)
4
5
4
3
16 (100%)
8
7
15 (100%)
15
10
25 (100%)
81
MMSP State-level Evaluation (Year 11)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 4–Cohort 7, by Partnership
Lesley – SpringfieldEasthampton (M)
Easthampton
Springfield
Subtotal
N/A
N/A
N/A
N/A
N/A
N/A
Sep10–
Aug11
Cohorts
4&5
N/A
N/A
N/A
Worcester Public
Schools (M)
Worcester
Subtotal
N/A
N/A
N/A
N/A
N/A
N/A
Partnership
High Need Districts
Sep08–
Aug09
Cohort 4
Sep09–
Aug10
Cohort 4
Sep11–
Aug12
Cohorts
5&6
N/A
N/A
N/A
Sep12–
Aug13
Cohorts
5&6
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep13–
Aug14
Cohorts
6&7
1
5
6 (60%)
9
9 (100%)
West Springfield Public Schools is not listed in this table because no courses were offered in Y11.
UMass Donahue Institute
Research and Evaluation Group
82