Massachusetts Mathematics and Science Partnership Title IIB
Annual State-level Evaluation Report (Year 10)
Reporting Period: September 1, 2012 through August 31, 2013
Cumulative Reporting Period: February 2, 2004 through August 31, 2013
Prepared for the Massachusetts Department of Elementary and
Secondary Education
April 25, 2014
MMSP State-level Evaluation (Year 10)
Contents
Contents
Program Description................................................................................................................................... 3
Report Organization .................................................................................................................................... 4
Evaluation Plan and Activities ................................................................................................................... 5
Year 10 Project Activity .............................................................................................................................. 6
State-level Participant Background Data ............................................................... 6
Partnership-level Participant Background Data ................................................... 15
Cumulative Summary: All Cohorts over All Funding Periods .............................................................. 19
Summary of Findings ............................................................................................................................... 34
Cumulative Findings ............................................................................................ 34
Year 10 Findings .................................................................................................. 36
Appendix A: Participant Background Survey – Year 10 ....................................................................... 38
Appendix B: Timeline for State-level Evaluation and TA Activities ..................................................... 46
Appendix C: Year 10 Participant Background Survey Results ............................................................ 49
Appendix D: High Need District Eligibility Criteria ................................................................................ 55
Appendix E: Enrollment and Attrition Rates by Course ....................................................................... 64
Appendix F: Mean Percentage Scores for Pre- & Post-course Tests .................................................. 65
Appendix G: High Need Districts for All Funding Periods, by Partnership ........................................ 66
UMass Donahue Institute
Research and Evaluation Group
I
MMSP State-level Evaluation (Year 10)
Tables
Tables Index
Table 1: Partnership Budgets (Year 10) ................................................................................................................ 6
Table 2: Professional Position of Participants..................................................................................................... 6
Table 3: Teaching Content Areas of Participants ................................................................................................ 7
Table 4: Experience of Participants ...................................................................................................................... 7
Table 5: Teaching Level of Participants ............................................................................................................... 8
Table 6: Participants by Type of Schools ............................................................................................................. 8
Table 7: Public School Participants by High Need District Status .................................................................... 8
Table 8: High Need District Participants by Partnership .................................................................................... 9
Table 9: Reasons for Participation, All Seats..................................................................................................... 10
Table 10: Repeat Participants .............................................................................................................................. 10
Table 11: Total Enrollment and Attrition Information ........................................................................................ 11
Table 12a: Science and Technology Teaching Areas – Regular Education .................................................... 11
Table 12b: Science and Technology Teaching Areas – Special Education ..................................................... 12
Table 12c: Science and Technology Teaching Areas – ELL Education ........................................................... 12
Table 13a: Mathematics Teacher Levels – Regular Education.......................................................................... 13
Table 13b: Mathematics Teacher Levels – Special Education .......................................................................... 13
Table 13c: Mathematics Teacher Levels – ELL Education ................................................................................ 13
Table 14: Pursuit of Degrees ............................................................................................................................... 14
Table 15: Number of Courses with Statistically Significant Gains in Mean Content Knowledge Scores.... 15
Table 16: Cohort 5 Math – EduTron-Worcester-Lowell Participant Background Information ...................... 15
Table 17:Cohort 5 Science – Everett-UMass Boston Participant Background Information ........................... 16
Table 18: Cohort 6 Math – EduTron-Revere-Everett-Saugus Participant Background Info ........................... 16
Table 19: Cohort 6 Math – Lesley-Brockton Participant Background Information ......................................... 17
Table 20: Cohort 6 Math – Lesley-Springfield Participant Background Information ...................................... 17
Table 21: Cohort 6 Science – MCLA Participant Background Information ...................................................... 18
Table 22: Cohort 6 Science – Northeastern Participant Background Information .......................................... 18
Table 23: Overview of MMSP Partnership Participation: Cohort 1–Cohort 3 ................................................. 19
Table 24: Overview of MMSP Partnership Participation: Cohort 4–Cohort 6 ................................................. 20
Table 25: Budgets: All Partnerships, All Funding Periods ............................................................................... 21
Table 26: Professional Position of Participants – Cumulative ......................................................................... 23
Table 27: Teaching Areas: All Participants, All Funding Periods .................................................................... 23
Table 28. Types of Schools: All Unique Participants, All Funding Periods .................................................... 24
Table 29: High Need Status of All Unique Participants from Public Schools ................................................. 25
Table 30: Reasons for Participation of Participants: All Seats, All Funding Periods .................................... 26
Table 31: Repeat Participants – All Partnerships, All Funding Periods .......................................................... 27
Table 32: MTEL Tests Taken by All Participants – Total to Date ..................................................................... 28
Table 33a: STE Teaching Areas of Regular Education Teachers .................................................................... 29
Table 33b: STE Teaching Areas of Special Education Teachers ..................................................................... 29
Table 33c: STE Teaching Areas of ELL Educators ............................................................................................ 29
Table 34a: Mathematics Teacher Levels of Regular Education Teachers ...................................................... 30
Table 34b: Mathematics Teacher Levels of Special Education Teachers ....................................................... 30
Table 34c: Mathematics Teacher Levels of ELL Educators .............................................................................. 30
Table 35: Pursuit of Degrees ............................................................................................................................... 31
Table 36: Number of Courses with Statistically Significant Gains in Mean Content Knowledge Scores.... 31
UMass Donahue Institute
Research and Evaluation Group
II
MMSP State-level Evaluation (Year 10)
Program Description
Program Description
The purpose of the Massachusetts Mathematics and Science Partnership Program (MMSP) is to improve student
achievement in mathematics, science, and technology/engineering through intensive, high-quality professional
development activities that focus on deepening teachers’ content knowledge. This multi-year project is funded by
the U.S. Department of Education (USED) as part of the No Child Left Behind (NCLB) Act Title IIB funding
stream. Funding to local partnerships is administered by state education agencies; in Massachusetts this is the
Massachusetts Department of Elementary and Secondary Education (ESE), which awards funding through a
competitive grant process.
Partnerships awarded these grants are required to include 1) a core partner that has been identified as a high need
school district in the subject matter on which the partnership is focusing and 2) a core partner that is a science,
technology and/or engineering, or mathematics (STEM) department from an institution of higher education. The
partnerships are composed of higher education institutions, school districts, and, in some cases, private
organizations involved in providing both pre-service and in-service training to teachers. Partnerships are required
to offer courses that supply at least 45 hours of direct instruction followed by at least 20 hours of follow-up
contact to support the implementation of course content in the classroom. Partnerships are encouraged to tailor the
model used to deliver the professional development and follow-up to best fit the objectives of their programs
along with their resources, expertise, and existing infrastructure. Partnership activities are guided by the following
goals:
Goal I
Develop and implement an effective and sustained course of study for in-service teachers of STEM by
integrating the courses of study into schools of arts and sciences and/or education at institutions of
higher education.
Goal II
Increase the number of STEM teachers in the partner school districts who are licensed in the subject
area(s) and grade level(s) they teach.
Goal III Increase the number of STEM teachers in the partner school districts who participate in high quality
professional development and advance their content knowledge.
Goal IV Develop and implement a systemic approach to STEM education by integrating professional
development with district and school STEM improvement initiatives.
The program began in February 2004, and has had ten funding periods, defined as follows:










Year 1: February 2, 2004 through August 31, 2004 (initial funding for Cohort 1)
Year 2: September 1, 2004 through August 31, 2005 (initial funding for Cohort 2)
Year 3: September 1, 2005 through August 31, 2006
Year 4: September 1, 2006 through August 31, 2007 (initial funding for Cohort 3)
Year 5: September 1, 2007 through August 31, 2008
Year 6: September 1, 2008 through August 31, 2009 (initial funding for Cohort 4)
Year 7: September 1, 2009 through August 31, 2010
Year 8: September 1, 2010 through August 31, 2011 (initial funding for Cohort 5)
Year 9: September 1, 2011 through August 31, 2012 (initial funding for Cohort 6)
Year 10: September 1, 2012 through August 31, 2013
UMass Donahue Institute
Research and Evaluation Group
3
MMSP State-level Evaluation (Year 10)
Report Organization
Report Organization
The purpose of this report is twofold: 1) to provide details regarding participation for the 2012–2013 funding
period; and 2) to provide a cumulative summary regarding participation for all cohorts over all funding periods.
Data supporting the first purpose address the period of September 1, 2012 through August 31, 2013, and data
supporting the second purpose address the period February 2, 2004 through August 31, 2013. Participant data
were collected through the Participant Background Survey, an instrument developed by the UMass Donahue
Institute (UMDI) and administered by partnerships to each participant on the last day of each course. See
Appendix A for the survey used during Year 10. The purpose of this survey is to gather data about participants’
professional backgrounds and qualifications. This information provides a description of the participants, aids in
determining whether courses are reaching the teachers who most need professional development, and aids in
tracking how teacher qualifications change during the MMSP funding period. Unless noted, data from the survey
are unduplicated, meaning that they are reported in terms of unique individuals, regardless of the number of
courses taken by each individual.
Data related to strengthening of relationships between partnership members were collected through a section of
the local evaluation reports that partnerships were required to submit to ESE. Specifically, partnerships were
asked to describe the extent to which their courses had been integrated into activities of their higher education
partners.
UMass Donahue Institute
Research and Evaluation Group
4
MMSP State-level Evaluation (Year 10)
Evaluation Plan and Activities
Evaluation Plan and Activities
ESE has contracted with UMDI to conduct a state-level evaluation of the MMSP since the program’s inception.
State-level evaluation reports are submitted to USED. UMDI’s primary role as state-level evaluator is to
coordinate program-wide collection of outcome data on behalf of the ESE. Data collection for the state-level
evaluation is organized around a basic logic model for professional development initiatives, as shown below.
In addition to the state-level data collection, each partnership is required to conduct its own local evaluation. In an
effort to support strong local evaluations, ESE required that partnerships sub-contract with UMDI to provide
technical assistance to support the design and implementation of their local evaluations.
A timeline listing the state-level evaluation and technical assistance activities for Year 10 can be found in
Appendix B.
UMass Donahue Institute
Research and Evaluation Group
5
MMSP State-level Evaluation (Year 10)
Year 10 Project Activity
Year 10 Project Activity
The Year 10 funding period provided funding for seven partnerships, four offering mathematics professional
development and three offering science professional development. Table 1 shows the funding received by each
partnership for the period beginning in September 2012 and ending in August 2013.
Table 1: Partnership Budgets (Year 10)
Partnership
Sep12-Aug13
Cohort 5 Math
EduTron-Worcester-Lowell
$200,000
Cohort 5 Science
Everett-UMass Boston
$149,250
Cohort 6 Math
EduTron-Revere-Everett-Saugus
$201,500
Cohort 6 Math
Lesley-Brockton
$290,750
Cohort 6 Math
Lesley-Springfield
$311,523
Cohort 6 Science
MCLA
Cohort 6 Science
TOTAL
Northeastern
$40,386
$199,947
$1,393,356
State-level Participant Background Data
In Year 10, there were 29 courses delivered in 2012–2013 – 22 mathematics courses and 7 science courses. Of
those 29 courses, 22 were unique, and seven were repeat offerings. During Year 10, 353 unique participants
completed the Participant Background Survey at least once. Of those participants, 106 took two or more courses
in 2012–2013. The term “unique participant” refers to each individual who participated in the program, regardless
of the number of courses taken. Data for items from the survey that convey participants’ professional backgrounds
and motives for participation are presented in this section. Additional survey data are presented in Appendix C.
Responses to survey questions are presented as frequencies and percentages. Not all percentages total 100% as
many items allowed multiple responses and not all participants responded to all items.
Professional Position of Participants
As shown in Table 2, the vast majority of the 353 participants were teachers, predominantly regular education
teachers. Only were math coaches and 1% were department heads or curriculum coordinators.
Table 2: Professional Position of Participants
Professional Position (N=353)
(Only one responses permitted)
Teacher
Regular Education
Special Education or Special Education Inclusion
ELL or Sheltered English Immersion
OtherTeachers
Math Coach (Teaching and Non-Teaching)
Department Head or Curriculum Coordinator
Principal, Assistant Principal or Headmaster
Other (including no response)
UMass Donahue Institute
Research and Evaluation Group
Sep12–Aug13
n
%
325
243
52
19
11
8
3
1
16
92
69
15
5
3
2
1
<1
5
6
MMSP State-level Evaluation (Year 10)
Year 10 Project Activity
Teaching Content Areas
The distribution of teaching areas of respondents at the time of the survey is shown in Table 3. Because
respondents identified all teaching areas that applied to their positions at the time of the survey, some selected
multiple responses, so frequency totals exceed the number of respondents and percentages exceed 100%. At the
time of their last Year 10 MMSP course, 46% of participants were teaching mathematics (at any level), 24% were
teaching science, and 31% were teaching all subjects at the elementary level.
Table 3: Teaching Content Areas of Participants
Teaching Areas (N=353)
(Multiple responses permitted)
Mathematics
Any science area
General Science
Biology
Earth Science
Chemistry
Physics
Technology/Engineering
Elementary (all subjects)
Elementary Mathematics
Other
Not currently teaching
Sep12–Aug13
n
%
136
84
36
29
4
19
26
4
110
26
6
4
39
24
10
8
1
5
7
1
31
7
2
1
Teaching Experience
As shown in Table 4, most respondents had four or more years of experience. Only one-fifth were relatively new
teachers with three of years of experience or less.
Table 4: Experience of Participants
Year Employed in Education (N=353)
(Only one response permitted)
More than 20 years
11-20 years
4-10 years
1-3 years
No response
Sep12–Aug13
n
%
35
107
133
74
4
10
30
38
21
1
Teaching Levels
For reporting purposes, schools in the participating districts were organized into categories of elementary schools
(grades K-5), middle schools (grades 6-8), K-8 schools, and high schools (grades 9-12). As shown in Table 5,
40% of Year 10 participants were teaching in an elementary or K-8 school, 37% were teaching in a middle school,
20% were teaching in a high school.
UMass Donahue Institute
Research and Evaluation Group
7
MMSP State-level Evaluation (Year 10)
Year 10 Project Activity
Table 5: Teaching Level of Participants
What grades do you currently teach? (N=353)
(Coded as one response per individual)
Elementary or K-8
Middle School (Grades 6-8)
High School (Grades 9-12)
Middle and High School Grades
Not currently teaching
No response
Sep12–Aug13
n
%
140
132
70
2
4
5
40
37
20
1
1
1
Types of Schools
As shown in Table 6, 93% of unique 2012–2013 participants worked in a public school setting, and 6% worked in
a non-public school setting.
Table 6: Participants by Type of Schools
What type of school are you employed? (N=353)
Public School (includes public charter schools)
Non-public School
Not reported
Sep12–Aug13
N
327
20
6
%
93
6
2
High Need Status of Districts
MMSP partnerships were required to include at least one high need district. Appendix D identifies the criteria for
the high need designation and includes lists of public school districts qualifying as high need through 2013. ESE
expected that at least 50% of participants in each partnership would be from high need districts, and further, set an
informal goal that at least 75% of participants in each partnership would be from high need districts. Of all Year
10 participants, 70% came from high need districts. Table 7 shows that by the end of the Year 10 funding period,
74% of participants from public schools had come from high need districts.
Table 7: Public School Participants by High Need District Status
High-Need Status of Public School Participants (N=327)
High Need District
Non-high Need District/Unknown
Sep12–Aug13
N
%
243
84
74
26
Table 8 presents the number of participants from high need districts organized by partnership. An examination of
high need district participation in individual partnerships reveals that during the Year 10, five partnerships had at
least 50% of their participants coming from high need districts including three partnerships that exceeded the
informal goal of having at least 75% of the participants come from high need districts.
UMass Donahue Institute
Research and Evaluation Group
8
MMSP State-level Evaluation (Year 10)
Year 10 Project Activity
Table 8: High Need District Participants by Partnership
Partnership
High Need District
Sep12-Aug13
Lowell
Worcester
27
22
55
45
Subtotal
Chicopee
49
1
100
3
4
7
13
23
Medford
Waltham
11
1
35
3
Subtotal
Everett
24
22
77
42
Malden
Revere
2
5
4
10
Subtotal
Brockton
29
56
56
60
New Bedford
Quincy
1
18
1
19
Randolph
Subtotal
2
77
2
82
Holyoke
Springfield
7
24
14
47
2
3
4
6
36
71
1
9
3
4
27
36
20
1
31
2
Quincy
Stoneham
4
1
6
2
Waltham
Subtotal
1
27
2
43
n
Cohort 5 Math
EduTron-Worcester-Lowell
Cohort 5 Science
Everett-UMass Boston
Everett
Malden
Cohort 6 Math
Cohort 6 Math
Cohort 6 Math
EduTron-Revere-Everett-Saugus
Lesley-Brockton
Lesley-Springfield
Westfield
West Springfield
Cohort 6 Science
MCLA
Subtotal
Berkshire Arts and
Technology CS
Pittsfield
Subtotal
Cohort 6 Science
Northeastern
Boston
Lynn
%
Reasons for Participation in MMSP Courses
For each course taken, respondents identified all of their reasons for participating. Unlike in the earlier sections of
this report where data were presented for unique participants, data for this topic are presented for all course seats,
since responses are uniquely relevant to each distinct course that a participant took. Table 9 presents findings for
all 511 seats for courses taken by participants during the Year 10 funding period.
UMass Donahue Institute
Research and Evaluation Group
9
MMSP State-level Evaluation (Year 10)
Year 10 Project Activity
Table 9: Reasons for Participation, All Seats
Sep12–Aug13
Reasons for Participation (N=511)
(Multiple responses permitted)
To obtain graduate credit
To increase knowledge in content
To earn PDPs for recertification
To pursue a personal interest
To get an additional license (certification)
To prepare for the Massachusetts Test for Educator Licensure (MTEL)
To follow an administrator’s suggestion
To obtain a first license (certification)
Other
N
%
380
304
125
108
68
50
27
19
18
74
60
24
21
13
10
5
4
4
Repeat Participation
Six of the seven partnerships offered at least two courses in Year 10, each with some participants attending more
than one course. Table 10 presents information regarding repeat participation. (Tables 16 through 22 also include
data on repeat participation, as does Table 31.)
Table 10: Repeat Participants
Sep12–Aug13
Number of
Courses Offered
Partnership
Number of Unique
Participants
Participants
Taking Multiple
Courses
Cohort 5 Math
EduTron-Worcester-Lowell
3
49
1
Cohort 5 Science
Everett-UMass Boston
2
31
5
Cohort 6 Math
EduTron-Revere-Everett-Saugus
2
52
25
Cohort 6 Math
Lesley-Brockton
9
94
30
Cohort 6 Math
Lesley-Springfield
8
51
34
Cohort 6 Science
MCLA
1
11
0
Cohort 6 Science
Northeastern
4
65
11
29
353
106
TOTAL
Attrition
Partnerships offered a total of 29 courses in Year 10, with an overall participant attrition rate of 4%. Of the 29
courses, 15 (52%) had no attrition, 10 (34%) had attrition rates between 1% and 10%, and 4 (14%) had attrition
rates greater than 10%. Table 11 provides a breakdown, by partnership, of enrollment and attrition rates.
Appendix E provides a breakdown, by course, of enrollment and attrition rates.
UMass Donahue Institute
Research and Evaluation Group
10
MMSP State-level Evaluation (Year 10)
Year 10 Project Activity
Table 11: Total Enrollment and Attrition Information
Partnership
3
2
2
9
8
1
4
Number of
Participants
Enrolled First
Day
67
35
82
144
125
14
90
Number of
Participants
Completed
Course
60
35
80
139
121
13
84
29
557
532
Number of
Courses
Offered
Cohort 5 Math
EduTron-Worcester-Lowell
Cohort 5 Science
Everett-UMass Boston
Cohort 6 Math
EduTron-Revere-Everett-Saugus
Cohort 6 Math
Lesley-Brockton (Math)
Cohort 6 Math
Lesley-Springfield (Math)
Cohort 6 Science
MCLA (Science)
Cohort 6 Science
Northeastern (Science)
All Courses/Partnerships
Attrition Rate
10%
0%
2%
3%
3%
7%
7%
4%
Licensure and Degrees Held in Subjects Taught
During Year 10, 131 regular education teachers, 24 special education teachers, and 13 ELL teachers reported
teaching subjects in science or technology/engineering. (These figures include elementary teachers who indicated
that they teach science.) Tables 12a, 12b, and 12c show the number of participants teaching each science or
technology/engineering subject during the 2012–2013 funding period. The tables also show the percentages of
teachers who indicated they were licensed in the subject they taught, and the percentages of teachers who
indicated they held a degree in the subject they taught.
To see the extent to which licensure and degree were appropriate for a particular type of teacher for a particular
subject, look in the relevant table at data in the row corresponding to the subject of interest. For example, to see
the extent to which licensure and degree were appropriate for just regular education biology teachers, look at
Table 12a in the row for the subject area “Biology” to learn that 23 teachers taught regular education biology and
74% of them were licensed in biology and 61% held degrees in biology.
Among regular education teachers, licensing appeared to be appropriate for approximately 58% of the science
subjects taught, and the degrees held corresponded to the subject for approximately 29% of the science subjects
taught. The strongest alignment of both licensure and degree was evident among those teaching biology, followed
by chemistry and physics.
Table 12a: Science and Technology Teaching Areas – Regular Education
Sep12–Aug13
Subject
General Science
Biology
Chemistry
Physics
Earth Science
Technology/Engineering
Number of
Teachers
License in Subject
Taught
Degree in Subject
Taught
n
%*
%*
23
23
13
22
2
2
56
74
54
46
0
50
9
61
31
23
0
0
*Of the number of regular education participants teaching this subject for this period
Table 12b provides information for special education teachers, and Table 12c provides information for ELL
teachers. (The numbers in these tables exceed the number of teachers who reported teaching these subjects
UMass Donahue Institute
Research and Evaluation Group
11
MMSP State-level Evaluation (Year 10)
Year 10 Project Activity
because some teachers taught more than one subject.) Across all science subject areas for special education
teachers, the licensing reported appeared to be appropriate for 24% of the science subjects taught, and the degrees
held corresponded to the science subject taught for 12% of the science subjects taught. Across all science subject
areas for ELL teachers, the licensing reported appeared to be appropriate for 67% of the subjects taught, and the
degree held by one teacher corresponded to the science subject taught.
Table 12b: Science and Technology Teaching Areas – Special Education
Sep12–Aug13
Subject
General Science
Biology
Chemistry
Physics
Earth Science
Technology/Engineering
Number of
Teachers
License in Subject
Taught
Degree in Subject
Taught
n
%*
%*
7
5
5
4
2
2
14
40
40
25
0
0
0
40
20
0
0
0
*Of the number of special education participants teaching this subject for this period
Table 12c: Science and Technology Teaching Areas – ELL Education
Sep12–Aug13
Subject
General Science
Biology
Chemistry
Physics
Earth Science
Technology/Engineering
Number of
Teachers
License in Subject
Taught
Degree in Subject
Taught
n
%*
%*
5
1
0
0
0
0
80
0
-
20
0
-
*Of the number of ELL education participants teaching this subject for this period
During Year 10, 117 regular education teachers 23 special education teachers, and 8 ELL teachers reported
teaching mathematics. Tables 13a, 13b, and 13c show how many teachers taught at each mathematics level during
the 2012–2013 funding period. The tables also show the percentages of teachers who indicated they were licensed
in the mathematics level at which they taught, and the percentages of teachers who indicated they held a degree in
mathematics.
To see the extent to which licensure and degree were appropriate for a particular type of teacher for a particular
mathematics level taught, look in the relevant table at data in the row corresponding to the level of interest and in
the column corresponding to the time period of interest. For example, to see the extent to which licensure and
degree were appropriate for just regular education middle school math teachers, look at Table 13a in the row for
the level “Middle School” to learn that 72 teachers taught regular education middle school mathematics and that
76% of them were licensed to teach middle school mathematics and 18% held mathematics degrees.
From an “ease-of-reading perspective,” it would be ideal merely to report on the percentages of teachers who
were licensed to teach mathematics and the percentage of teachers who held mathematics degrees. Sometimes,
though, teachers hold multiple positions and may only be licensed to teach some of what they are teaching, and
what is really desired is to understand the extent to which what is being taught is done so by individuals licensed
UMass Donahue Institute
Research and Evaluation Group
12
MMSP State-level Evaluation (Year 10)
Year 10 Project Activity
to teach it and by individuals who hold relevant degrees. As a result, the explanations of data that follow are
presented in terms of teaching positions.
For MMSP mathematics teaching positions in regular education, the licensing appeared to be appropriate for
approximately 74% of positions held, and 21% of the positions were held by individuals with mathematics
degrees. The alignment was strongest for high school-level positions followed by middle school-level positions.
None of the special education or ELL teachers held degrees in math. Licensing appeared to be appropriate for
30% of the positions held by special education teachers and 38% of the ELL teachers.
Table 13a: Mathematics Teacher Levels – Regular Education
Sep12–Aug13
Number of
Teachers
Licensed at Level
Degree in Math
n
%*
%*
Elementary School
Middle School
High School
Middle & High School
19
72
26
0
32
76
96
-
5
18
42
-
TOTAL Math
117
74
21
Level
*Of the number of regular education participants teaching mathematics at this level for this period
Table 13b: Mathematics Teacher Levels – Special Education
Sep12–Aug13
Number of
Teachers
Licensed at Level
Degree in Math
n
%*
%*
Elementary School
Middle School
High School
Middle & High School
0
20
1
2
35
0
0
0
0
0
TOTAL Math
23
30
0
Level
*Of the number of special education participants teaching mathematics at this level for this period
Table 13c: Mathematics Teacher Levels – ELL Education
Sep12–Aug13
Level
Number of
Teachers
Licensed at Level
Degree in Math
%*
n
%*
Elementary School
0
-
-
Middle School
High School
Middle & High School
8
0
0
38
-
0
-
TOTAL Math
8
38
0
*Of the number of ELL education participants teaching mathematics at this level for this period
UMass Donahue Institute
Research and Evaluation Group
13
MMSP State-level Evaluation (Year 10)
Year 10 Project Activity
Degrees Currently Pursued
Information on degrees currently being pursued in science, technology/engineering, and mathematics was derived
from the most recently completed survey of each individual. Table 14 provides details about the degrees being
pursued and the teaching areas of those who were pursuing the degrees, and it shows that these participants were
pursuing a total of 143 undergraduate and graduate degrees. Of all unique participants from the 2012–2013
funding period, four reported that they were pursuing degrees in science and technology/engineering and nine
reported that they were pursuing mathematics degrees.
Table 14: Pursuit of Degrees
Degree Pursued
Teaching Area
BA/BS
MA/MS
CAGS
Doctorate
General Science
1
14
0
0
Biology
0
13
1
0
Chemistry
0
5
0
0
Physics
0
5
0
0
Earth Science
0
1
0
0
Technology/Engineering
0
1
0
0
Elementary
2
24
5
0
Mathematics
1
62
7
1
Total
4
125
13
1
Content Knowledge Gains
As a grant condition, partnerships were required to use a pre-test and post-test for each MMSP course to assess
participants’ content knowledge growth. For each course, the same instrument served as both the pre- and posttest. While partnerships were permitted to locate and use an instrument with established reliability and validity, it
was most often the case that such instruments were not available because a priority was placed on utilizing
assessments that would reflect the precise content taught in each course. As a result, the faculty members who had
developed the courses usually developed the assessments, and typically, neither validity nor reliability was
determined for them because time and resource constraints prohibited doing so.
Data were analyzed only for participants who completed both pre- and post-course assessments. To determine if
the scores showed statistically significantly changes between pre- and post-course test administrations, a paired
samples t-test was used for each course with ten or more participants completing both pre- and post-course
assessments, and a Wilcoxon Matched Pairs test was used for each course with fewer than ten participants
completing both pre- and post-tests.
Pre- and post-tests were administered for all of the 29 courses offered in Year 10. Gains in average percentage of
items correct between pre- and post-test administrations occurred in all of the courses and statistically significant
improvements in scores on content knowledge assessments occurred in all but one of those courses. The course
not showing a statistically significant improvement in scores had fewer than six participants complete both the
pre- and post-test—the smallest sample size at which it is possible to detect statistical significance at the level
used for these analyses. Table 15 provides an overview, by subject matter of courses delivered. For detailed
information on mean pre- and post-test scores by partnership and by course, see Appendix F.
UMass Donahue Institute
Research and Evaluation Group
14
MMSP State-level Evaluation (Year 10)
Year 10 Project Activity
Table 15: Number of Courses with Statistically Significant Gains
in Mean Content Knowledge Scores
Content Area
Sep12–Aug13
Courses Delivered
Significant Gains
Mathematics
Science & Technology/Engineering
22
7
95%
100%
TOTAL
29
97%
Partnership-level Participant Background Data
Tables 16 through 22 offer an overview of selected participant survey data for each year 10 partnership. These
data were collected through the participant background survey administered at the end of each course. The
responses are presented as frequencies and percentages. In cases where not all participants responded to all of the
items and in cases where multiple responses were permitted, percentages presented may not total 100%.
Table 16: Cohort 5 Math – EduTron-Worcester-Lowell Participant Background Information
Sep10-Aug11
Number of Participants
Sep11-Aug12
Total Number of Participants
87
Participants Who Took Multiple Courses
In relevant year
At any point in time
10
15
(12%)
(17%)
0
17
(0%)
(36%)
1
16
(2%)
(33%)
Teach Regular Education
63
(72%)
38
(81%)
35
(71%)
Teach Special Education (sole and inclusion)
13
(15%)
3
(6%)
5
(10%)
Teach ELL, ESL, or Sheltered English Immersion
1
(1%)
1
(2%)
3
(6%)
Teach Elementary (all content areas)
9
(10%)
9
(19%)
25
(51%)
Teach Elementary Math
3
(3%)
1
(2%)
3
(6%)
Teach Mathematics Above Elementary
65
(75%)
36
(77%)
17
(35%)
Teach Science or Technology/Engineering
10
(12%)
9
(19%)
22
(45%)
Teach in High Need District
66
(76%)
47
(100%)
49
(100%)
Have Passed One or More MTEL Exams
49
(56%)
30
(64%)
28
(57%)
Hold One or More Teaching Licenses
81
(93%)
44
(94%)
28
(57%)
UMass Donahue Institute
Research and Evaluation Group
47
Sep12-Aug13
49
15
MMSP State-level Evaluation (Year 10)
Year 10 Project Activity
Table 17:Cohort 5 Science – Everett-UMass Boston Participant Background Information
Number of Participants
Sep10-Aug11
Total Number of Participants
Participants Who Took Multiple Courses
Sep11-Aug12
40
In relevant year
At any point in time
Sep12-Aug13
39
31
3
8
(8%)
(20%)
6
16
(15%)
(41%)
5
11
(16%)
(36%)
32
(80%)
32
(82%)
24
(77%)
Teach Special Education (sole and inclusion)
4
(10%)
3
(8%)
1
(3%)
Teach ELL, ESL, or Sheltered English Immersion
4
(10%)
2
(5%)
4
(13%)
Teach Elementary (all content areas)
7
(18%)
6
(15%)
10
(32%)
Teach Elementary Math
0
(0%)
0
(0%)
0
(0%)
Teach Mathematics Above Elementary
3
(8%)
3
(8%)
0
(0%)
Teach Science or Technology/Engineering
32
(80%)
35
(90%)
28
(90%)
Teach in High Need District
27
(68%)
31
(80%)
24
(77%)
Have Passed One or More MTEL Exams
26
(65%)
30
(77%)
19
(61%)
Hold One or More Teaching Licenses
40
(100%)
36
(92%)
21
(68%)
Teach Regular Education
Table 18: Cohort 6 Math – EduTron-Revere-Everett-Saugus Participant Background Info
Number of Participants
Sep11-Aug12
Total Number of Participants
Sep12-Aug13
69
52
1
(1%)
25
(48%)
14
(20%)
34
(65%)
Teach Regular Education
51
(74%)
29
(56%)
Teach Special Education (sole and inclusion)
11
(16%)
13
(25%)
1
(1%)
1
(2%)
18
(26%)
22
(42%)
7
(10%)
8
(15%)
44
(64%)
20
(39%)
1
(1%)
15
(29%)
Teach in High Need District
47
(68%)
29
(56%)
Have Passed One or More MTEL Exams
52
(75%)
38
(73%)
Hold One or More Teaching Licenses
69
(100%)
31
(60%)
Participants Who Took Multiple Courses
Teach ELL, ESL, or Sheltered English Immersion
Teach Elementary (all content areas)
Teach Elementary Math
Teach Mathematics Above Elementary
Teach Science or Technology/Engineering
UMass Donahue Institute
Research and Evaluation Group
In relevant year
At any point in time
16
MMSP State-level Evaluation (Year 10)
Year 10 Project Activity
Table 19: Cohort 6 Math – Lesley-Brockton Participant Background Information
Number of Participants
Sep11-Aug12
Total Number of Participants
Sep12-Aug13
35
94
3
(9%)
30
(32%)
18
(51%)
39
(42%)
22
(63%)
69
(73%)
Teach Special Education (sole and inclusion)
5
(14%)
12
(13%)
Teach ELL, ESL, or Sheltered English Immersion
6
(17%)
8
(9%)
10
(29%)
32
(34%)
5
(14%)
6
(6%)
22
(63%)
53
(56%)
5
(14%)
43
(46%)
Teach in High Need District
33
(94%)
77
(82%)
Have Passed One or More MTEL Exams
23
(66%)
57
(61%)
Hold One or More Teaching Licenses
32
(91%)
56
(60%)
Participants Who Took Multiple Courses
In relevant year
At any point in time
Teach Regular Education
Teach Elementary (all content areas)
Teach Elementary Math
Teach Mathematics Above Elementary
Teach Science or Technology/Engineering
Table 20: Cohort 6 Math – Lesley-Springfield Participant Background Information
Number of Participants
Sep11-Aug12
Total Number of Participants
Sep12-Aug13
27
51
5
(19%)
34
(67%)
6
(22%)
37
(73%)
20
(74%)
29
(57%)
Teach Special Education (sole and inclusion)
4
(15%)
11
(22%)
Teach ELL, ESL, or Sheltered English Immersion
0
(0%)
0
(0%)
10
(37%)
14
(28%)
7
(26%)
6
(12%)
13
(48%)
24
(47%)
2
(7%)
11
(22%)
Teach in High Need District
22
(82%)
36
(71%)
Have Passed One or More MTEL Exams
17
(63%)
36
(71%)
Hold One or More Teaching Licenses
21
(78%)
27
(53%)
Participants Who Took Multiple Courses
Teach Regular Education
Teach Elementary (all content areas)
Teach Elementary Math
Teach Mathematics Above Elementary
Teach Science or Technology/Engineering
UMass Donahue Institute
Research and Evaluation Group
In relevant year
At any point in time
17
MMSP State-level Evaluation (Year 10)
Year 10 Project Activity
Table 21: Cohort 6 Science – MCLA Participant Background Information
Number of Participants
Sep11-Aug12
Total Number of Participants
Sep12-Aug13
17
11
0
(0%)
0
(0%)
2
(12%)
2
(18%)
Teach Regular Education
9
(53%)
7
(64%)
Teach Special Education (sole and inclusion)
0
(0%)
2
(18%)
Teach ELL, ESL, or Sheltered English Immersion
0
(0%)
0
(0%)
Teach Elementary (all content areas)
0
(0%)
0
(0%)
Teach Elementary Math
0
(0%)
1
(9%)
Teach Mathematics Above Elementary
5
(29%)
9
(82%)
Teach Science or Technology/Engineering
4
(24%)
1
(9%)
Teach in High Need District
6
(35%)
4
(36%)
Have Passed One or More MTEL Exams
6
(35%)
7
(64%)
Hold One or More Teaching Licenses
8
(47%)
10
(91%)
Participants Who Took Multiple Courses
In relevant year
At any point in time
Table 22: Cohort 6 Science – Northeastern Participant Background Information
Number of Participants
Sep11-Aug12
Total Number of Participants
Sep12-Aug13
85
65
12
(14%)
11
(17%)
34
(40%)
31
(48%)
76
(89%)
49
(75%)
Teach Special Education (sole and inclusion)
4
(5%)
9
(14%)
Teach ELL, ESL, or Sheltered English Immersion
1
(1%)
3
(5%)
Teach Elementary (all content areas)
8
(9%)
7
(11%)
Teach Elementary Math
2
(2%)
2
(3%)
Teach Mathematics Above Elementary
8
(9%)
12
(19%)
Teach Science or Technology/Engineering
63
(74%)
55
(85%)
Teach in High Need District
29
(34%)
27
(42%)
Have Passed One or More MTEL Exams
50
(59%)
39
(60%)
Hold One or More Teaching Licenses
70
(82%)
47
(72%)
Participants Who Took Multiple Courses
In relevant year
At any point in time
Teach Regular Education
UMass Donahue Institute
Research and Evaluation Group
18
MMSP State-level Evaluation (Year 10)
Cumulative Summary
Cumulative Summary: All Cohorts over All Funding Periods
Overview of Partnerships, Budgets, Courses, and Participants
Tables 23 and 24 provide an overview of partnership projects since the inception of the program. The first six cohorts combined included 34 partnerships, with 20
offering mathematics professional development (PD), twelve offering science PD, and two offering PD in both mathematics and science content. Specifically,
Cohort 1 consisted of eight partnerships, with six offering mathematics PD and two offering science PD. Cohort 2 consisted of two partnerships, both offering
mathematics PD. Cohort 3 consisted of nine partnerships, with four offering mathematics PD, three offering science PD, and two offering PD in both mathematics
and science. Cohort 4 consisted of eight partnerships, with four offering mathematics PD and four offering science PD. Cohort 5 consisted of two partnerships,
with one offering mathematics PD and one offering science PD. Cohort 6 consisted of five partnerships, with three offering mathematics PD and two offering
science PD.
Table 23: Overview of MMSP Partnership Participation: Cohort 1–Cohort 3
Funding Period
Partnership Grouping
Cohort 1
MMSP Year 1
MMSP Year 2
MMSP Year 3
MMSP Year 4
MMSP Year 5
MMSP Year 6
Feb04-Aug04
Sep04-Aug05
Sep05-Aug06
Sep06-Aug07
Sep07-Aug08
Sep08-Aug09
Grant Year 1
Grant Year 2
Grant Year 3
EduTron/Fitchburg-Math
Harvard-Math
Lesley-Math
MCLA-Science
Salem-Math
Springfield PS-Science
Wareham PS-Math
WPI-Math
EduTron/Fitchburg-Math
Harvard-Math
Lesley-Math
MCLA-Science
Salem-Math
Springfield PS-Science
Wareham PS-Math
WPI-Math
EduTron/Fitchburg-Math
Harvard-Math
Lesley-Math
MCLA-Science
Salem-Math
Springfield PS-Science
Wareham PS-Math
WPI-Math
Grant Year 1
Grant Year 2
Grant Year 3
MCLA-Math
PV STEMNET-Math
MCLA-Math
PV STEMNET-Math
MCLA-Math
PV STEMNET-Math
Cohort 2
Cohort 3
UMass Donahue Institute
Research and Evaluation Group
Grant
Year 3
Extension
Grant Year 1
Grant Year 2
Grant Year 3
EduTron/Lowell-Math/Sci
EduTron/Fitchburg-Math
Lesley-Math
North Shore-Science
PV STEMNET-Math/Sci
Salem-Math
SE/Cape-Science
WPI-Science
WPS-Math (discontinued)
EduTron/Lowell-Math/Sci
EduTron/Fitchburg-Math
Lesley-Math
North Shore-Science
PV STEMNET-Math/Sci
Salem-Math
SE/Cape-Science
WPI-Science
EduTron/Lowell-Math/Sci
EduTron/Fitchburg-Math
Lesley-Math
North Shore-Science
PV STEMNET-Math/Sci
Salem-Math
SE/Cape-Science
WPI-Science
19
MMSP State-level Evaluation (Year 10)
Cumulative Summary
Table 24: Overview of MMSP Partnership Participation: Cohort 4–Cohort 6
Partnership Grouping
Funding Period
Cohort 4
MMSP Year 6
MMSP Year 7
MMSP Year 8
MMSP Year 9
MMSP Year 10
Sep08-Aug09
Sep09-Aug10
Sep10-Aug11
Sep11-Aug12
Sep12-Aug13
Grant Year 1
Grant Year 2
Grant Year 3
Boston PS-Math
Brockton PS-Math
Gateway RSD-Science
Lesley-Math
Northeastern-Science
Randolph PS-Science
Springfield Coll.Science
Boston U.-Math
Boston PS-Math
Brockton PS-Math
Gateway RSD-Science
Lesley-Math
Northeastern-Science
Randolph PS-Science
Springfield Coll.Science
Boston U.-Math
Boston PS-Math
Brockton PS-Math
Gateway RSD-Science
Lesley-Math
Northeastern-Science
Randolph PS-Science
Springfield Coll.Science
Boston U.-Math
Cohort 5
Cohort 6
Grant Year 1
Grant Year 2
EduTron-Worc-M
EduTron-Worc-M
Everett-UMass Boston-S Everett-UMass Boston-S
Grant Year 3
EduTron-Worc-M
Everett-UMass Boston-S
Grant Year 1
Grant Year 2
EduTron-Cohort 6-M
Lesley-Brockton-M
Lesley-Springfield-M
MCLA-S
Northeastern-S
EduTron-Cohort 6-M
Lesley-Brockton-M
Lesley-Springfield-M
MCLA-S
Northeastern-S
Table 25 shows the funding received by each partnership since the inception of the program. While some
partnerships were awarded funding in more than one funding period, for evaluation purposes, a partnership was
identified as a “new” partnership each time it received funding that resulted from a different competition. Overall,
partnerships have been awarded a total of $17,945,315 since the inception of MMSP.
UMass Donahue Institute
Research and Evaluation Group
20
MMSP State-level Evaluation (Year 10)
Cumulative Summary
Table 25: Budgets: All Partnerships, All Funding Periods
Feb04Aug06
Sep06Aug07
EduTron (Math)
$770,000
$68,352
$838,352
Harvard Graduate School of Ed. (Math)
$489,899
$87,425
$577,324
Lesley University (Math)
$810,726
$43,838
$854,564
MCLA – Science (Science)
$133,192
$38,247
$171,439
Salem State College (Math)
$541,995
$43,648
$585,643
Springfield Public Schools (Science)
$500,044
$74,737
$574,781
Wareham Public Schools (Math)
$398,440
$43,962
$442,402
Worcester Polytechnic Institute (Math)
$601,778
$35,633
$637,411
Partnership
Sep07Aug08
Sep08Aug09
Sep09Aug10
Sep10Aug11
Sep11Aug12
Sep12Aug13
TOTAL
COHORT 1
Initially funded February 2004
COHORT 2
Initially funded September 2004
MCLA – Math (Math)
$111,494
$51,874
$163,368
UMass Amherst (Math)
$262,415
$181,581
$443,996
COHORT 3
Initially funded September 2006
EduTron Lowell (Math & Science)
$210,000
$220,000
$240,000
$670,000
EduTron Fitchburg (Math)
$102,000
$110,000
$120,000
$332,000
Lesley University (Math)
$347,911
$355,626
$355,357
$1,058,894
North Shore (Science)
$196,474
$194,729
$199,871
$591,074
UMass Amherst (Math/Science)
$107,424
$216,281
$169,064
$492,769
Salem State College (Math)
$120,882
$113,551
$36,604
$271,037
SE/Cape (Science)
$129,438
$181,420
$169,246
$480,104
$99,586
$70,734
$94,852
$265,172
Worcester Polytechnic Inst. (Science)
Worcester Public Schools (Math)
UMass Donahue Institute
Research and Evaluation Group
$231,210
$231,210
21
MMSP State-level Evaluation (Year 10)
Cumulative Summary
Sep08Aug09
Sep09Aug10
Sep10Aug11
Boston Public Schools (Math)
$157,975
$405,747
$218,986
$782,708
Brockton Public Schools (Math)
$180,145
$255,758
$251,263
$687,166
Gateway Regional (Science)
$186,609
$200,370
$172,379
$559,358
Lesley-Springfield (Math)
$228,593
$324,820
$308,416
$861,829
Greater North Shore (Science)
$265,917
$306,690
$266,480
$839,087
Randolph Public Schools (Science)
$176,993
$183,150
$151,178
$511,321
Springfield College (Science)
$161,062
$148,896
$156,832
$466,790
Boston University (Math)
$241,586
$245,180
$244,394
$731,160
Partnership
Feb04Aug06
Sep06Aug07
Sep07Aug08
Sep11Aug12
Sep12Aug13
TOTAL
COHORT 4
Initially funded September 2008
COHORT 5
Initially funded September 2010
EduTron-Worcester-Lowell (Math)
$200,000
$205,000
$200,000
$605,000
Everett –UMass Boston (Science)
$149,250
$149,250
$149,250
$447,750
EduTron-Revere-Everett-Saugus (Math)
$200,000
$201,500
$401,500
Lesley-Brockton (Math)
$155,794
$290,750
$446,544
Lesley-Springfield (Math)
$122,415
$311,523
$433,938
MCLA (Science)
$49,664
$40,386
$90,050
COHORT 6
Initially funded September 2011
Northeastern (Science)
TOTAL
UMass Donahue Institute
Research and Evaluation Group
$199,627
$4,619,983
$2,214,222
$1,462,341
$2,983,874
$2,070,611
$2,119,178
$1,081,750
$199,947
$399,574
$1,393,356
$17,945,315
22
MMSP State-level Evaluation (Year 10)
Cumulative Summary
Through Year 10 (2012-2013) partnerships developed and implemented a total of 392 courses of which 213
(54%) were unique, and 179 (46%) were repeat offerings. Of the 392 courses, 264 (67%) offered mathematics
content, 126 (32%) offered science and/or technology/engineering content, and 2 (1%) offered both mathematics
and science/technology content. In total, there were 3,546 unique participants, 1,303 of whom took two or more
courses. The term “unique participant” refers to each individual who participated in the program, regardless of
how many courses he or she took. Data for items from the Participant Background Survey that help to convey
participants’ professional backgrounds and motives for participation are discussed in this section. In addition, this
section discusses information obtained from partnerships on the extent to which courses offered through MMSP
became institutionalized.
Position of Participants
As shown in Table 26, 89% of MMSP course participants identified themselves as teachers, predominantly
regular education teachers (69% of all respondents).
Table 26: Professional Position of Participants – Cumulative
Professional Position (N=3,546)
(Only one responses permitted)
All Funding Periods
%
n
Teacher
Regular Education
Special Education or Special Education Inclusion
ELL or Sheltered English Immersion
Other Teachers
Math Coach (Teaching and Non-Teaching)
Department Head or Curriculum Coordinator
Principal, Assistant Principal or Headmaster
Other (including no response)
3143
2429
532
114
68
89
51
42
221
89
69
15
3
2
3
1
1
6
Content Taught
The distribution of teaching areas of respondents at the time of the survey is shown in Table 27 – 39% were
teaching mathematics, 31% were teaching science, and 27% were teaching all subjects at the elementary level.
Because respondents identified all teaching areas that applied to their positions at the time of the survey, some
selected multiple responses, so frequency totals exceed the number of respondents and percentages exceed 100%.
Table 27: Teaching Areas: All Participants, All Funding Periods
Teaching Areas (N=3,546)
(Multiple responses permitted)
Mathematics
Any science area
General Science
Biology
Earth Science
Chemistry
Physics
Technology/Engineering
Elementary (all subjects)
Elementary Mathematics
Other
Not Currently Teaching
UMass Donahue Institute
Research and Evaluation Group
All Funding Periods
n
1399
1096
552
256
120
161
156
65
956
215
58
222
%
39
31
16
7
3
5
4
2
27
6
2
6
23
MMSP State-level Evaluation (Year 10)
Cumulative Summary
Types of Schools of Participants
For each funding period at least 89% of MMSP participants worked in a public school. Over the course of the program to date, 96% of worked in a public school,
and 4% worked in a non-public school. Table 28 provides a breakdown, by funding period, of the types of schools in which participants worked.
Table 28. Types of Schools: All Unique Participants, All Funding Periods
School Type
Public Schools
(includes charters)
Non-public School
Other or No
Response
TOTAL
Feb04Aug04
n
%
Sep04Aug05
n
%
Sep05Aug06
n
%
Sep06Aug07
n
%
Sep07Aug08
n
%
Sep08Aug09
n
n
Sep09Aug10
n
%
Sep10Aug11
n
%
Sep11Aug12
n
%
Sep12Aug13
n
%
Total
n
%
332
97
448
98
455
98
533
96
462
97
3176
3176
614
98
600
95
278
89
327
93
3399
96
8
2
7
2
6
1
12
2
7
2
113
113
16
3
29
5
17
5
20
6
123
4
1
<1
1
<1
3
1
8
<1
8
2
19
19
0
0
0
0
19
6
6
<2
24
<1
341
100
456
100
464
100
553
100
477
100
3308
3308
630
100
629
100
314
100
353
100
3546
100
High Need Status of Districts of Participants
MMSP partnerships were required to include at least one high need district. 9Appendix D identifies the criteria for the high need designation and includes lists of
public school districts qualifying as high need through 2012-2013.) In addition, ESE expected that at least 50% of participants in each partnership would come
from high need districts, and further, they set an informal goal that at least 75% of participants for each partnership would come from high need districts. The high
need status of some school districts changed across years. For the purposes of MMSP evaluation and reporting, any district identified as high need in the first year
of a partnership’s funding continued to be classified as high need in subsequent years of the partnership, even if the district’s status changed. Additionally, any
districts not on the high need list in the first year of a partnership’s funding but subsequently added to the high need list in later years of the partnership were then
identified as qualifying for high need district designation.
As a whole, across all years of funding, 67% of participants were from high need districts. Table 29 shows that across all years of funding, 69% of the public
school participants in the program as a whole had come from high need districts and that for each year of funding, over 55% of public school participants in the
program had come from high need districts. An examination of high need district participation in individual partnerships reveals that over the course of each
partnership’s involvement since the beginning of MMSP, 24 of the 34 partnerships had at least 50% of their participants coming from high need districts across all
years of their involvement, and (again, across all years of their involvement) 18 of 34 partnerships exceeded the informal goal of having at least 75% of the
participants come from high need districts. If individual years of participation are examined, 18 of the 34 partnerships had at least 50% of their participants coming
from high need districts for each and every year of funding, and 13 exceeded the informal goal of having at least 75% of the participants come from high need
districts for each and every year of funding. As of their last course in MMSP, of the 1,303 individuals who took multiple courses, approximately 67% were from
high need public school districts, approximately 31% were from other public school districts, and 2% either were from private schools or did not provide
UMass Donahue Institute
Research and Evaluation Group
24
MMSP State-level Evaluation (Year 10)
Cumulative Summary
information on their districts. In addition, less than 1% were from high need districts for some of the courses they took but not for others1. The table in Appendix G
shows the number of participants from high need districts organized by partnership.
Table 29: High Need Status of All Unique Participants from Public Schools
School Type
High Need
District
Non-high Need
District
Unknown or No
Response
TOTAL
Feb04Aug04
n
%
Sep04Aug05
n
%
Sep05Aug06
n
%
Sep06Aug07
n
%
Sep07Aug08
n
%
Sep08Aug09
n
%
Sep09Aug10
n
%
Sep10Aug11
n
%
Sep11Aug12
n
%
Sep12Aug13
n
%
Total
n
%
202
61
276
62
254
56
397
74
335
73
544
69
412
67
437
70
212
76
243
74
2352
69
129
39
161
36
196
43
126
24
122
26
240
31
202
33
191
30
66
24
82
25
1031
30
1
<1
11
2
5
1
10
2
5
1
4
<1
0
0
1
<1
0
0
2
1
16
<1
332
100
448
100
455
100
447
100
462
100
788
100
614
100
629
100
278
100
327
100
3399
100
Participants who took MMSP courses from a math partnership when their districts were considered high need for only science were identified as having come from a “non-high
need” district, and teachers who took MMSP courses from a science partnership when their districts were considered high need for only math were identified as having come from
a “non-high need” district.
1
UMass Donahue Institute
Research and Evaluation Group
25
MMSP State-level Evaluation (Year 10)
Cumulative Summary
Reasons for Participation in MMSP Courses
For each course taken, respondents identified all of their reasons for participating. Unlike in most other portions
of this report where data were presented for unique participants, data for this issue are presented for all course
seats, since responses are uniquely relevant to each distinct course that a participant took. Table 30 presents
findings for all 6,805 seats for all courses taken by all participants across all funding periods.
Table 30: Reasons for Participation of Participants: All Seats, All
Funding Periods
Total
Reasons for Participation
(Multiple responses permitted)
%
n
of 6,805 course
seats
To increase knowledge in content
4863
71%
To obtain graduate credit
4662
69%
To pursue a personal interest
2080
31%
To earn PDPs for recertification
2042
30%
To get an additional license (certification)
1158
17%
To prepare for the Massachusetts Test for Educator Licensure (MTEL)
973
14%
To follow an administrator’s suggestion
467
7%
To earn PDPs for HOUSSE plan requirement
331
5%
To obtain a first license (certification)
208
3%
Other
380
6%
Repeat Participation
Overall, partnerships were successful at encouraging and retaining repeat participants. Of the 34 partnerships, all
offered multiple courses, and all had participants who attended more than one MMSP course. In all, 1,303
participants (37%) attended multiple courses. Table 31 provides details regarding repeat participation.
UMass Donahue Institute
Research and Evaluation Group
26
MMSP State-level Evaluation (Year 10)
Cumulative Summary
Table 31: Repeat Participants – All Partnerships, All Funding
Periods
Number of
Courses
Delivered
to Date
Total
Number of
Unique
Participants
to Date*
EduTron (Math)
7
105
31
Harvard (Math)
8
157
21
19
98
75
Partnership
Lesley Univ. C1 (Math)
MCLA (Science)
Total Number
Taking
Multiple
Courses
3
19
10
26
133
55
Springfield PS (Science)
7
77
27
Wareham PS (Math)
3
40
10
WPI (Math)
6
130
40
Salem State College (Math)
MCLA (Math)
4
9
2
UMass Amherst (Math)
11
53
28
EduTron Lowell (Math/Science)
10
144
25
7
118
54
Lesley Univ. C3 (Math)
40
170
104
North Shore (Science)
30
75
25
UMass Amherst C3 (Math/Science)
14
117
41
EduTron Fitchburg (Math)
Salem State C3 (Math)
SE/Cape (Science)
WPI (Science)
Worcester PS(Math)
8
71
37
15
141
49
3
43
5
3
38
5
Boston PS (Math)
29
266
86
Brockton PS (Math)
15
235
24
3
82
45
Lesley Springfield (Math)
21
107
66
Greater North Shore (Science)
Gateway RSD (Science)
28
208
98
Randolph PS (Science)
8
70
32
Springfield Coll. (Science)
5
51
24
Boston University (Math)
6
98
19
EduTron (Math)
9
166
49
Everett (Science)
6
85
24
EduTron-Revere-Everett-Saugus (Math)
4
94
42
Lesley-Brockton (Math)
11
121
54
Lesley-Springfield (Math)
11
65
37
3
27
4
MCLA (Science)
Northeastern (Science)
Across All Partnerships
9
133
55
392
3,546
1,303
*Participants who participated in multiple courses across partnerships were counted only once in the
partnership of their most recent course
UMass Donahue Institute
Research and Evaluation Group
27
MMSP State-level Evaluation (Year 10)
Cumulative Summary
MTEL Information
One method by which teachers may demonstrate subject-matter competency is to pass the Massachusetts Tests for
Educator Licensure (MTEL) in the content areas that they teach. Table 32 cumulatively identifies the tests taken
by participants across all years of the program along with passage rates.
Table 32: MTEL Tests Taken by All Participants – Total to Date
Based on each participant’s last survey
Took Test
Passing Test
Failing Test
n
n
%
590
568
96
9
2
13
2
64
55
86
3
5
6
9
Elementary Mathematics
148
131
89
7
5
10
7
Mathematics
364
306
84
34
9
24
7
Middle School Mathematics
581
504
87
42
7
35
6
Middle School
Mathematics/Science
104
71
68
20
19
13
13
General Science
266
240
90
12
5
14
5
Biology
177
159
90
10
6
8
5
Chemistry
98
84
86
10
10
4
4
Physics
63
46
73
11
18
6
10
Earth Science
29
22
76
3
10
4
14
Technology/Engineering
14
11
79
3
21
0
0
751
633
84
69
9
49
7
General Curriculum (formerly
Elementary)
Early Childhood
TOTAL in STE Areas
n
%
Scores
Unknown
n
%
Licensure and Degrees Held in Content Area Taught
Over the course of the program, 897 regular education teachers, 120 special education teachers, and 53 ELL
teachers reported teaching science or technology/engineering. Tables 33a, 33b, and 33c show how many teachers
taught in each science and technology/engineering area over the course of the program. The tables also show the
percentages of teachers who indicated they were licensed in the area in which they taught, and they show the
percentages of teachers who indicated they held a degree in the area in which they taught. Table 33a provides
information for regular education teachers, Table 33b provides information for special education teachers, and
Table 33c provides information for ELL teachers. (The numbers presented in Tables 33a, 33b, and 33c exceed the
number of teachers who reported teaching in these areas because some teachers taught in more than one area.)
For regular education teachers, the licensing reported by approximately 59% appeared to be appropriate for the
content area taught, and the degrees held by approximately 28% corresponded to content area taught. For special
education teachers, the licensing reported by approximately 44% appeared to be appropriate for the content area
taught, and the degrees held by approximately 3% corresponded to content area taught. For ELL teachers, the
licensing reported by approximately 53% appeared to be appropriate for the content area taught, and the degrees
held by 25% corresponded to content area taught.
UMass Donahue Institute
Research and Evaluation Group
28
MMSP State-level Evaluation (Year 10)
Cumulative Summary
Table 33a: STE Teaching Areas of Regular Education Teachers
Total
Content Area
General Science
Biology
Chemistry
Earth Science
Physics
Technology/Engineering
Teach in Area
License in Area
Taught
Degree in Area
Taught
n
%*
%*
449
204
141
91
131
56
64
78
61
25
41
41
13
66
40
11
20
16
*Of the number of regular education participants teaching in this area across all program years
Table 33b: STE Teaching Areas of Special Education Teachers
Content Area
General Science
Biology
Chemistry
Earth Science
Physics
Technology/Engineering
Teach in Area
Total
License in Area
Taught
Degree in Area
Taught
n
%*
%*
67
31
14
22
18
6
46
39
50
32
61
17
3
7
7
0
0
0
*Of the number of special education participants teaching in this area across all program years
Table 33c: STE Teaching Areas of ELL Educators
Content Area
General Science
Biology
Chemistry
Earth Science
Physics
Technology/Engineering
Teach in Area
Total
License in Area
Taught
Degree in Area
Taught
n
%*
%*
25
15
3
4
5
3
64
47
33
25
60
33
16
40
33
25
40
0
*Of the number of ELL participants teaching in this area across all program years
Over the course of the program and by the end of the Year 10 funding period, 1,070 regular education teachers,
243 special education teachers, and 46 ELL teachers reported teaching non-elementary level mathematics. Tables
34a, 34b, and 34c show how many teachers taught at each non-elementary mathematics level over the course of
the program. The tables also show the percentages of teachers who indicated they were licensed in the level at
which they taught, and the percentages who indicated they held a degree in the area in which they taught. Table
34a provides information for regular education teachers, Table 34b provides information for special education
teachers, and Table 34c provides information for ELL teachers.
UMass Donahue Institute
Research and Evaluation Group
29
MMSP State-level Evaluation (Year 10)
Cumulative Summary
For regular education teachers, the licensing reported by approximately 44% appeared to be appropriate for the
mathematics level taught, and 23% held mathematics degrees. For special education teachers, the licensing
reported by approximately 10% appeared to be appropriate for the level taught, and 4% held mathematics degrees.
For ELL teachers, the licensing reported by 11% appeared to be appropriate for the level taught, and 13% held
mathematics degrees.
Table 34a: Mathematics Teacher Levels of Regular Education Teachers
Level
Middle School
High School
MS & HS grades
TOTAL Math
Teach in Area
Total
License in Area
Taught
Degree in Math
n
%*
%*
824
234
12
46
39
33
17
44
25
1070
44
23
*Of the number of regular education participants teaching in this area for this year
Table 34b: Mathematics Teacher Levels of Special Education Teachers
Level
Middle School
High School
MS & HS grades
TOTAL Math
Teach in Area
Total
License in Area
Taught
Degree in Math
n
%*
%*
194
11
2
40
9
5
0
10
11
243
10
4
*Of the number of special education participants teaching in this area for this year
Table 34c: Mathematics Teacher Levels of ELL Educators
Level
Teach in Area
Total
License in Area
Taught
Degree in Math
n
%*
%*
Middle School
High School
MS & HS grades
38
7
1
13
0
0
5
57
0
TOTAL Math
46
11
13
*Of the number of ELL participants teaching in this area for this year
Degrees Currently Pursued
Information on degrees currently being pursued was derived from the most recently completed survey of each
individual. Table 35 provides details about the degrees being pursued and the teaching areas of those who were
pursuing the degrees, and it shows that these participants were pursuing a total of 1,194 undergraduate and
graduate degrees. Of all unique participants through the end of the 2012–2013 funding period, 243 reported that
they were pursuing degrees in science and technology/engineering and 273 reported that they were pursuing
mathematics degrees.
UMass Donahue Institute
Research and Evaluation Group
30
MMSP State-level Evaluation (Year 10)
Cumulative Summary
Table 35: Pursuit of Degrees
Degree Pursued
Teaching Area
BA/BS
MA/MS
General Science
CAGS
Doctorate
11
155
23
4
Biology
8
91
13
1
Chemistry
7
50
9
2
Physics
2
57
6
1
Earth Science
2
29
2
1
Technology/Engineering
2
18
4
1
Elementary
9
186
40
9
Mathematics
13
367
56
15
Total
54
953
153
34
Content Knowledge Gains
As a grant condition, partnerships were required to use a pre- test and post- test for each MMSP course to assess
participants’ knowledge growth. For each course, the same instrument served as both the pre- and post-test. While
partnerships were permitted to locate and use an instrument with established reliability and validity, it was most
often the case that such instruments were not available because a priority was placed on utilizing assessments that
would reflect the precise content taught in each course. As a result, the faculty members who had developed the
courses usually developed the assessments, and typically neither validity nor reliability was determined for them
because time and resource constraints prohibited doing so.
Data were analyzed only for participants who completed both pre- and post-tests. To determine if the scores
showed statistically significantly changes between pre- and post-test administrations, a paired samples t-test was
used for each course with ten or more participants completing both pre- and post-tests, and a Wilcoxon Matched
Pairs test was used for each course with fewer than ten participants completing both pre- and post-tests.
Of the 392 courses that were delivered across all partnerships through Year 10, content assessments were
administered for 384 courses. Of these 384 courses, 380 had gains in the average percentage of items correct
between pre- and post- test administrations. Statistically significant improvements in scores on content knowledge
assessments occurred in 86% of the 384 courses. Of the 52 courses not showing statistically significant
improvement in scores, 35 had fewer than six participants2, the smallest sample size at which it is possible to
detect statistical significance at the level used for these analyses. Table 36 provides an overview, by subject
matter, of the numbers of courses that showed statistically significant gains.
Table 36: Number of Courses with Statistically Significant
Gains in Mean Content Knowledge Scores
Total
Content Area
Delivered,
with Pre- and Posttests
Significant
Pre/Post
Gains
Math
257
228
Science & Technology/Engineering
Math and Science
125
2
102
2
TOTAL
384
332
2
Some of these courses had fewer than six MMSP enrollees within the context of another course that included additional
non-MMSP enrollees; other courses were not able to administer content assessments to all participants because of logistical
challenges.
UMass Donahue Institute
Research and Evaluation Group
31
MMSP State-level Evaluation (Year 10)
Cumulative Summary
Course Institutionalization
For systemic change to occur at the higher education institutions, departments of arts and sciences and education
departments are encouraged to work together through MMSP to support stronger content courses in mathematics
and science for teacher preparation, undergraduate and graduate degree requirements, and for in-service teachers
pursuing graduate-level content courses for recertification. Integration of Title IIB courses into graduate programs
at Institutes of Higher Education ensures sustainability over time. The intent behind encouraging the partnerships
is that the faculty from the Arts and Sciences Departments bring strong content expertise to the partnership table.
This integration creates greater opportunities for participants to complete coursework leading to a content-area
degree and/or to licensure along with the highly qualified designation.
Since the 2006–2007 funding period, partnerships have been asked to describe activities that were related to the
institutionalization of their courses. Many partnerships reported integration, plans for future integration, or—in
the case of partnerships with previously established involvement with MMSP—work toward sustaining prior
integration. As would be expected in a program involving partnerships with diverse structures and styles, the
extent and type of integration varied across partnerships. To convey a sense of how integration occurred,
following are significant related activities, grouped according to partnership:
EduTron Lowell Public Schools (Math/Science) and EduTron Fitchburg State College (Math)
 Two developmental courses, based on the EduTron model for MMSP courses, will continue to be offered
at Fitchburg State College (FSC). EduTron partners supported FSC in designing three pre-service courses
that are optimized for education majors. EduTron has begun working with FSC to help FSC apply the
EduTron model used in MMSP math courses to science courses.
 FSC has partnered with Lowell Public Schools to offer a teacher certification/CAGS program.
 Six mathematics and four science courses were approved by FSC as offerings at the continuing education
level.
Lesley University Cohort 3 (Math)
 Two courses created through Lesley University’s MMSP in 2007-2008 are now offered to Lesley’s oncampus pre-service teachers.
 Efforts through MMSP contributed to the development of an online Mathematics Education program at
Lesley leading to the Master of Arts degree for elementary and middle school teachers.
 Nine math content courses were developed through participation in the MMSP program in 2007-2008 and
in prior years. All of these courses are part of Lesley University’s mathematics major for undergraduates,
which would not have been possible without the MMSP program.
North Shore (Science)
 As a result of their joint involvement in MMSP through the North Shore partnership and the National
Science Foundation MSP program, Northeastern University has institutionalized all MMSP courses. Ten
MMSP courses can be used to fulfill degree requirements toward a Master’s in Education for Middle
School Science. In addition, this degree was developed as a result of these courses.
UMass Amherst Cohort 3 (Math/Science)
 Four courses developed through the UMass Amherst partnership were approved for graduate level credit.
Salem State Cohort 3 (Math)
 Salem State College offers courses developed through MMSP as part of a master’s level teaching
program in middle school mathematics. All courses developed by Salem State College through MMSP
can be applied towards earning a degree through that program.
UMass Donahue Institute
Research and Evaluation Group
32
MMSP State-level Evaluation (Year 10)
Cumulative Summary
Southeast/Cape (Science)
 Participants of the three courses offered through the SE/Cape partnership may apply credit for the courses
towards the Master of Arts in Teaching in Physical Science program that is offered through Bridgewater
State College.
Worcester Polytechnic Institute (Science)
 A Master of Science Education program was created through the physics department at Worcester
Polytechnic Institute, and the MMSP course that was offered through the WPI-Science partnership will
serve as the model for instruction of future courses that will be offered.
Worcester Public Schools (Math)
 As a result of the experience of working with Worcester Public Schools on MMSP, Clark University has
expressed interest in exploring the institutionalization of courses that were offered through MMSP.
Springfield College (Science)
 Springfield College has incorporated into its pre-service Best Practices of Teaching Science course
activities from an MMSP course that are designed to help teachers understand how to change
misconceptions that students have about life science.
Randolph Public Schools (Science)
 Four courses developed through the Randolph Public Schools partnership were approved for graduate
level credit at Bridgewater State University.
Scaling Up
While this evaluation did not set out to explore the reach of partnerships beyond documenting the numbers of
participants and their high need districts of origin, an exceptional instance of scaling up emerged through data
collection efforts. Since it speaks to the goals of MMSP and also is an indicator of project success, it is being
included here. As its professional development model, the Brockton Public Schools partnership used the 80-hour
Massachusetts Intel Math Initiative (MIMI) course and professional learning community follow-up. Through their
participation in MMSP, they—in effect—regionalized the program, expanding the model from a relatively small
partnership between school districts and higher education professors to include over 28 Southeastern
Massachusetts districts and three institutions of higher education.
UMass Donahue Institute
Research and Evaluation Group
33
MMSP State-level Evaluation (Year 10)
Summary of Findings
Summary of Findings
The MMSP partnership activities summarized in this section of the report occurred between February 2, 2004,
and August 31, 2013. This period spans the beginning of the MMSP program through the end of the 2012–2013
funding period. Since MMSP began in February 2004, progress has been made towards meeting the goals of the
program as evidenced by the following data for both the program as a whole, since its inception, and for the most
recent funding period of 2012–2013.
Cumulative Findings
Overview of Partnerships

A total of 34 partnerships were funded across the Commonwealth. Of these, 20 were organized around
mathematical content, twelve were organized around science content, and two were organized around
both mathematical and science content. Of the 34 MMSP partnerships, all delivered courses, all offered
multiple courses, and all had participants who attended more than one course.

Cohort 1, which began in February 2004, consisted of eight partnerships, with six of the eight
partnerships offering mathematics professional development and two offering science professional
development.

Cohort 2, which began in September 2004, consisted of two partnerships, both offering mathematics
professional development.

Cohort 3, which began in September 2006, consisted of nine partnerships, with five of the nine
partnerships offering mathematics professional development, three offering science professional
development, and one offering professional development in both mathematics and science content.

Cohort 4, which began in September 2008, consisted of eight partnerships, with four of the eight
partnerships offering mathematics professional development and four offering science professional
development.

Cohort 5, which began in September 2010, consisted of two partnerships, with one partnership
offering mathematics professional development and one offering science professional development.

Cohort 6, which began in September 2011, consisted of five partnerships, with three of the five
partnerships offering mathematics professional development and two offering science professional
development.
Overview of Courses

In total, 392 MMSP courses were delivered by the end of Year 10 of MMSP funding. Of these 392
courses, 264 were mathematics courses, 126 were science and/or technology/engineering courses, and
two were courses offering both mathematics and science content.
Overview of Participants

In total, 3,546 unique participants participated in MMSP courses by the end of Year 10.

1,303 participants (37% of all participants) attended multiple courses.

6,805 course seats were filled by all participants across all funding periods.
UMass Donahue Institute
Research and Evaluation Group
34
MMSP State-level Evaluation (Year 10)
Summary of Findings
Reaching Targeted Participants

Types of Schools of Participants


Of all 3,546 unique participants, 96% came from public schools (including public charter schools)
and 4% came from non-public schools, and <1% did not indicate their school type.
High Need Status of Districts of Participants

The partnerships exceeded the ESE target of having at least 50% of all participants come from high
need districts, with 67% of all participants in the program coming from high need districts.

Across all years of their involvement, 24 of the 34 partnerships had at least 50% of their participants
coming from high need districts.

For each and every individual year of funding, 18 of the 34 partnerships had at least 50% of their
participants coming from high need districts.

Across all years of their involvement, 18 of 34 partnerships had at least 75% of the participants come
from high need districts.

For each and every individual year of funding, 13 of the 34 partnerships had at least 75% of the
participants come from high need districts.
As the following data reveal, not all MMSP participants were licensed in their teaching areas or held relevant
degrees, indicating that course participants were, in fact, those in need of the courses:


Licensure in Mathematics and Science Content Areas for Middle and High School Teachers

Of the regular education mathematics courses taught by MMSP teachers, 44% were taught by
teachers who were licensed in mathematics.

Of the special education mathematics courses taught by MMSP teachers, 10% were taught by
teachers who were licensed in mathematics.

Of the ELL mathematics courses taught by MMSP teachers, 11% were taught by teachers who were
licensed in mathematics.

Of regular education science or technology/engineering courses taught by MMSP teachers, 59% were
taught by teachers who were licensed in the subject of the course.

Of special education science or technology/engineering courses taught by MMSP teachers, 44% were
taught by teachers who were licensed in the subject of the course.

Of ELL science or technology/engineering courses taught to students of MMSP teachers, 53% were
taught by teachers who were licensed in the subject of the course.
Degrees Held in Content Area in which Teaching

Of regular education mathematics courses taught by MMSP teachers, 23% were taught by teachers
who held mathematics degrees.

Of special education mathematics courses taught by MMSP teachers, 4% were taught by teachers
who held mathematics degrees.

Of ELL mathematics courses taught to students of MMSP teachers, 13% were taught by teachers who
held mathematics degrees.

Of regular education science or technology/engineering courses taught by MMSP teachers, 28% were
taught by teachers who held degrees that were relevant to the focus of the course.
UMass Donahue Institute
Research and Evaluation Group
35
MMSP State-level Evaluation (Year 10)
Summary of Findings

Of special education science or technology/engineering courses taught by MMSP teachers, 3% were
taught by teachers who held degrees that were relevant to the focus of the course.

Of ELL science or technology/engineering courses taught by MMSP teachers, 25% were taught by
teachers who held degrees that were relevant to the focus of the course.
Content Knowledge Gains
The content knowledge of participants increased:

Statistically significant improvements in scores on content knowledge assessments occurred in 86% of the
384 courses in which assessments were administered.
Year 10 Findings
Overview of Partnerships

Seven partnerships were funded during the 2012–2013 funding period. Of these, four were organized
around mathematical content and three were organized around science content. Six of the seven
partnerships delivered at least two courses. Of the seven partnerships, six had participants who took more
than one course within the 2012–2013 funding period, and all had participants who had attended more
than one course across the entire duration of MMSP.
Overview of Courses

In total, 29 courses were delivered during Year 10 of MMSP funding. Of these 29 courses, 22 were
mathematics courses and seven were science/technology/engineering courses.
Overview of Participants

During Year 10, 353 unique participants participated in MMSP courses.

106 participants (30% of all Year 10 participants) attended multiple courses during 2012–2013.

511 course seats were filled during Year 10.

Course attrition rates were low averaging 4% across all courses offered by partnerships in Year 10.
Reaching Targeted Participants

Types of Schools of Participants


Of all 353 unique Year 10 participants, 93% came from public schools (including public charter
schools), and 6% came from non-public schools.
High Need Status of Districts of Participants

The Year 10 partnerships exceeded the ESE target of having at least 50% of all participants come
from high need districts, with 70% of all 2012–2013 participants coming from high need districts.

Five of the seven partnerships had at least 50% of their participants coming from high need districts.

Three of the seven partnerships had at least 75% of the participants come from high need districts.
As the following data reveal, not all Year 10 participants were licensed in their teaching areas or held relevant
degrees, indicating that course participants were, in fact, those in need of the courses:

Licensure in Mathematics and Science Content Areas
UMass Donahue Institute
Research and Evaluation Group
36
MMSP State-level Evaluation (Year 10)

Summary of Findings

Of regular education mathematics courses taught by Year 10 participants, 74% were taught by
teachers who were licensed in mathematics.

Of special education mathematics courses taught by Year 10 participants, 30% were taught by
teachers who were licensed in mathematics.

Of ELL mathematics courses taught by Year 10 participants, 38% were taught by teachers who were
licensed in mathematics.

Of regular education science or technology/engineering courses taught by Year 10 participants, 58%
were taught by teachers who were licensed in the subject of the course.

Of special education science or technology/engineering courses taught by Year 10 participants, 24%
were taught by teachers who were licensed in the subject of the course.

Of ELL science or technology/engineering courses taught by Year 10 participants, 67% were taught
by teachers who were licensed in the subject of the course.
Degrees Held in Content Area in which Teaching

Of regular education mathematics courses taught by Year 10 participants, 21% were taught by
teachers who held mathematics degrees.

Of special education mathematics courses taught by Year 10 participants, none were taught by
teachers who held mathematics degrees.

Of ELL mathematics courses taught by Year 10 participants, none were taught by teachers who held
mathematics degrees.

Of regular education science or technology/engineering courses taught by Year 10 participants, 29%
were taught by teachers who held degrees that were relevant to the focus of the course.

Of special education science or technology/engineering courses taught by Year 10 participants, 12%
were taught by teachers who held degrees that were relevant to the focus of the course.

Of ELL science or technology/engineering courses taught by Year 10 participants, one was taught by
a teacher who held a degree that was relevant to the focus of the course.
Content Knowledge Gains
The content knowledge of Year 10 participants increased:

Statistically significant improvements in scores on content knowledge assessments occurred in 28 of the
29 courses.
UMass Donahue Institute
Research and Evaluation Group
37
MMSP State-level Evaluation (Year 10)
Appendix A: Participant Background Survey
Appendix A: Participant Background Survey – Year 10
UMass Donahue Institute
Research and Evaluation Group
38
MMSP State-level Evaluation (Year 10)
UMass Donahue Institute
Research and Evaluation Group
Appendix A: Participant Background Survey
39
MMSP State-level Evaluation (Year 10)
UMass Donahue Institute
Research and Evaluation Group
Appendix A: Participant Background Survey
40
MMSP State-level Evaluation (Year 10)
UMass Donahue Institute
Research and Evaluation Group
Appendix A: Participant Background Survey
41
MMSP State-level Evaluation (Year 10)
UMass Donahue Institute
Research and Evaluation Group
Appendix A: Participant Background Survey
42
MMSP State-level Evaluation (Year 10)
UMass Donahue Institute
Research and Evaluation Group
Appendix A: Participant Background Survey
43
MMSP State-level Evaluation (Year 10)
UMass Donahue Institute
Research and Evaluation Group
Appendix A: Participant Background Survey
44
MMSP State-level Evaluation (Year 10)
UMass Donahue Institute
Research and Evaluation Group
Appendix A: Participant Background Survey
45
MMSP State-level Evaluation (Year 10)
Appendix B: Evaluation Activities
Appendix B: Timeline for State-level Evaluation and TA Activities
The following is a summary timeline of state-level evaluation and technical assistance activities carried out between
February, 2004, and end of Year 10 of the MMSP.
February 2004 Held Kick-off Meeting for all partnerships and their evaluators at the Department of Education
Spring 2004
Conducted individual partnership meetings with local evaluators and partnership leaders to:
 Review the Minimum Expectations document along with the local evaluation and data collection
plan
 Explore potential modifications to implementation plans to create opportunities for experimental
or quasi experimental design
Spring 2004
Developed common measures for state-level data collection
June 2004
Attended federal meeting held for MSP projects across the country
Summer 2004 Disseminated and collected end-of-course documents designed to collect course-level data for the
statewide evaluation
Fall 2004
Conducted individual partnership meetings with local evaluators and partnership leaders to:
 Review the Minimum Expectations document along with the local evaluation and data collection
plan
 Review the Federal Reporting document to ensure the partnerships were collecting the data
needed to complete that report
Winter 2005
Conducted partnership meetings with the two new partnerships funded in the second round that
constitutes Cohort 2 to:
 Introduce the Minimum Expectations document along with the local evaluation and data
collection plan
 Introduce the Federal Reporting document to ensure the partnerships were collecting the data
needed to complete that report
June 2005
Held technical assistance meeting for all partnerships regarding the requirements of the USED Annual
report
June 2006
Participated in USED Annual Conference of MSP State Coordinators
August 2006
Held technical assistance meeting for all partnerships regarding evaluation requirements for MMSP
Fall 2006 to
Winter 2007
Conducted partnership meetings with the new Cohort 3 partnerships to:


Introduce the Minimum Expectations document along with the local evaluation and data
collection plan
Discuss the federal reporting requirements to ensure the partnerships were collecting the data
needed to complete federal report
UMass Donahue Institute
Research and Evaluation Group
46
MMSP State-level Evaluation (Year 10)
Appendix B: Evaluation Activities
December 2006 Participated in USED MSP Regional Conference
June 2007
Participated in USED Annual Conference of MSP State Coordinators
September 2007 Held technical assistance meeting for all partnerships regarding the requirements of the USED Annual
report
January 2008
Participated in USED MSP Regional Conference
April 2008
Participated in technical assistance workshop for bidders pursuing MSP funding for 2008-2009
April 2008
Participated in USED the Massachusetts MSP Statewide Conference
June 2008
Participated in USED MSP State Coordinators’ Meeting
October 2008
Held technical assistance meeting for all partnerships regarding evaluation requirements for MMSP
Fall 2008 to
Winter 2009
Conducted partnership meetings with the new Cohort 3 partnerships to:


Discuss evaluation expectations data collection plans
Discuss the federal reporting requirements to ensure the partnerships were collecting the data
needed to complete federal report
March 2009
Participated in USED MSP Regional Conference
May 2009
Participated in USED the Massachusetts MSP Statewide Conference
September 2009 Participated in technical assistance meeting for all partnerships regarding reporting requirements
January 2010
Participated in USED MSP Regional Conference
April 2010
Participated in technical assistance workshop for bidders pursuing MSP funding for 2010–2011
Spring 2010
Participated in continuation conferences for select partnerships
August 2010
Participated in technical assistance meeting for all partnerships regarding reporting requirements
February 2011
Participated in USED MSP Regional Conference
September 2011 Participated in technical assistance meeting for all partnerships regarding reporting requirements
November 2011 Participated in technical assistance workshop for bidders pursuing MSP funding for 2012–2013
April 2012
Participated in USED MSP Regional Conference
September 2012 Participated in technical assistance meeting for all partnerships regarding reporting requirements
UMass Donahue Institute
Research and Evaluation Group
47
MMSP State-level Evaluation (Year 10)
Appendix B: Evaluation Activities
The following activities were on going throughout the life of the project:
Disseminated and collected end-of-course documents designed to collect course-level data for the statewide
evaluation
Managed data collected from partnerships at the end of each course
Provided technical assistance to partnerships in support of local partnership evaluation efforts
Monitored local evaluation plans to see they include both formative and summative research questions and
corresponding activities
Monitored data collection and analysis around the basic logic model of professional development
Served as liaison to the U.S. Department of Education for evaluation and research issues including participation in
national meetings and periodic conference calls
Met with ESE MSP Team as needed to support integration of evaluation efforts with program goals
Until Steering Committee was disbanded, attended MMSP Steering Committee meetings in role of state level
evaluator and technical assistance
UMass Donahue Institute
Research and Evaluation Group
48
MMSP State-level Evaluation (Year 10)
Appendix C: Year 10 Background Survey Results
Appendix C: Year 10 Participant Background Survey Results
2012–2013
Item
n
%
How do you describe yourself?
American Indian or Alaskan native
Asian
Black or African American
Hispanic or Latino
Native Hawaiian or other Pacific Islander
White
Mixed Race
Other
No Response
0
13
10
14
1
300
5
7
3
0%
4%
3%
4%
<1%
85%
1%
2%
1%
What best describes your current primary position?
Teacher (Regular Education)
Special Education Teacher (Sole Instructor)
Special Education Inclusion Teacher
Department Head or Curriculum Coordinator
Principal/Asst. Principal/Headmaster
Support Specialist (counselor, librarian, etc.)
Long-term Substitute
Paraprofessional
Superintendent or Asst. Superintendent
ELL, ESL, or Sheltered English Immersion Teacher
Gifted or Talented Teacher
AP or IB Program Teacher
Title I Teacher
Math Coach (Non-Teaching)
Math Coach (Teaching)
Science Coach (Non-Teaching)
Science Coach (Teaching)
Instructional Technology Director
Other
Unknown/No Response
UMass Donahue Institute
Research and Evaluation Group
243
24
28
3
1
0
6
7
0
19
1
1
3
4
4
0
0
1
7
1
69%
7%
8%
1%
<1%
0%
2%
2%
0%
5%
<1%
<1%
1%
1%
1%
0%
0%
<1%
2%
<1%
49
MMSP State-level Evaluation (Year 10)
Appendix C: Year 10 Background Survey Results
2012–2013
Item
n
%
What grades do you currently teach?
Pre-K
Elementary and K-8
Middle School (Grades 6-8)
High School (Grades 9-12)
Middle and High School grades
Adult Education
All levels
NA (doesn’t teach)
No Response
0
140
132
70
2
0
0
4
5
0%
40%
37%
20%
1%
0%
0%
1%
1%
21
53
35
98
107
35
07
4
6%
15%
10%
28%
30%
10%
1%
How many years have you been employed in education?
1st year
2-3 years
4-5 years
6-10 years
11-20 years
Over 20 years
0 or No Response
Note: Percentages may not total 100% due to rounding error or items in which respondents may respond to
all that apply.
Item
Which of the following content areas are you currently
teaching?
Mathematics
Elementary (all subjects)
Elementary Mathematics
General Science
Biology
Physics
Earth Science
Chemistry
Technology/Engineering
Any science area
Other
Do not teach currently
2012–2013
n
%
136
110
26
36
29
26
4
19
4
84
6
4
39%
31%
7%
10%
8%
7%
1%
5%
1%
24%
2%
1%
327
20
6
93%
6%
<2%
6
1
2%
<1%
In which of the following are you currently employed?
Public School/ Public Charter School
Private School
Unknown/No Response
Currently hold certification through the National Board for
Professional Teaching Standards.
In Mathematics
In General Science
UMass Donahue Institute
Research and Evaluation Group
50
MMSP State-level Evaluation (Year 10)
Appendix C: Year 10 Background Survey Results
2012–2013
Item
n
%
74
10
120
129
6
14
21%
3%
34%
37%
2%
4%
142
6
91
91
2
21
40%
2%
26%
26%
<1%
6%
111
63
54
43
4
78
31%
18%
15%
12%
1%
22%
146
58
46
22
0
81
41%
16%
13%
6%
0%
23%
42
135
127
15
0
34
12%
38%
36%
4%
0%
10%
53
134
94
24
3
45
15%
38%
27%
7%
1%
13%
Approximately how many math students do you teach annually?
0 students
1-10 students
11-40 students
41-150 students
151+ students
No Response
Approximately how many science students do you teach annually?
0 students
1-10 students
11-40 students
41-150 students
151+ students
No Response
Approximately how many students do you teach annually who are
Title I students?
0 students
1-10 students
11-40 students
41-150 students
151+ students
No Response
Approximately how many students do you teach annually who are
academically advanced students?
0 students
1-10 students
11-40 students
41-150 students
151+ students
No Response
Approximately how many students do you teach annually who are
Special Education students?
0 students
1-10 students
11-40 students
41-150 students
151+ students
No Response
Approximately how many students do you teach annually who are
English Language Learners?
0 students
1-10 students
11-40 students
41-150 students
151+ students
No Response
UMass Donahue Institute
Research and Evaluation Group
51
MMSP State-level Evaluation (Year 10)
Appendix C: Year 10 Background Survey Results
2012–2013
Item
n
%
Why did you participate in this course? *
To obtain graduate credit
To increase knowledge in content
To pursue a personal interest
To earn PDPs for recertification
To get an additional license (certification)
To prepare for the Massachusetts Test for
Educator Licensure (MTEL)
To follow an administrator’s suggestion
To obtain a first license (certification)
Other
380
304
108
125
68
74
60
21
24
13
50
10
27
19
18
5
4
4
High Need District
Yes
No
Unknown or N/A
246
95
12
70%
27%
3%
*Data for this item represents the number of seats filled from all courses, rather than the
number of unique participants.
2012–2013
Item
n
%
How many PDP hours do you have in your content area(s)?
Less than 48 PDP hours
48 to 100 PDP hours
101 to 250 PDP hours
251+ PDP hours
No Response
Please select any of the following licenses you currently
hold.
Vocational Technical
Specialist Teacher
Supervisor/Director
Principal/Asst. Principal
Superintendent/Assistant Superintendent
UMass Donahue Institute
Research and Evaluation Group
23
19
12
4
295
7%
5%
3%
1%
84%
2
65
3
13
0
1%
18%
1%
4%
0%
52
MMSP State-level Evaluation (Year 10)
Appendix C: Year 10 Background Survey Results
2012–2013
Item
Bachelors
n
%
Masters
n
%
CAGS
n
%
Doctorate
n
%
A degree currently held for each major.
Education
Math Education
Science Education
Math
General Science
Biology
Chemistry
Earth Science
Physics
Technology/Engineering
All science/technology combined
Other
78
9
7
36
5
29
4
4
6
15
63
92
22%
3%
2%
10%
1%
8%
1%
1%
2%
4%
18%
26%
181
21
12
3
1
1
3
0
0
3
8
41
51%
6%
3%
1%
<1%
<1%
1%
0%
0%
1%
2%
12%
6
2
0
0
0
0
0
0
0
0
0
3
2%
1%
0%
0%
0%
0%
0%
0%
0%
0%
0%
1%
2
0
0
0
0
0
0
1
0
0
1
0
1%
0%
0%
0%
0%
0%
0%
<1%
0%
0%
<1%
0%
3
2
1
0
1
0
0
0
0
0
1
1
1%
1%
<1%
0%
<1%
0%
0%
0%
0%
0%
<1%
<1%
26
65
16
8
0
1
0
0
1
1
3
5
7%
18%
5%
2%
0%
<1%
0%
0%
<1%
<1%
1%
1%
6
4
1
1
0
0
0
0
0
0
0
2
2%
1%
<1%
<1%
0%
0%
0%
0%
0%
0%
0%
1%
1
1
0
0
0
0
0
0
0
0
0
0
<1%
<1%
0%
0%
0%
0%
0%
0%
0%
0%
0%
0%
A degree currently being pursued for each major.
Education
Math Education
Science Education
Math
General Science
Biology
Chemistry
Earth Science
Physics
Technology/Engineering
All science/technology combined
Other
2012–2013
Item
MTEL
Taken
n
%
MTEL
Passed
n
%
Scores
Unknown
n
%
MTEL tests taken
General Curriculum (formerly Elementary)
Elementary Math
Early Childhood
Mathematics
Middle School Mathematics
Middle School Mathematics/Science
General Science
Biology
Chemistry
Physics
Earth Science
Technology/Engineering
UMass Donahue Institute
Research and Evaluation Group
81
33
14
46
67
11
21
27
12
11
3
2
23%
9%
4%
13%
19%
3%
6%
8%
3%
3%
1%
1%
77
31
12
42
61
10
20
21
10
9
2
1
95%
97%
86%
91%
91%
91%
95%
78%
83%
82%
67%
50%
4
1
1
3
6
1
0
3
2
1
1
0
5%
3%
7%
7%
9%
9%
0%
11%
17%
9%
33%
0%
53
MMSP State-level Evaluation (Year 10)
Appendix C: Year 10 Background Survey Results
2012–2013
Item
n
%
4
0
12
30
2
11
17
2
5
140
36
15
13
4
30
0
52
11
83
9
0
12
43
21
2
3
1%
0%
3%
9%
1%
3%
5%
1%
1%
40%
10%
4%
4%
1%
9%
0%
15%
3%
24%
3%
0%
3%
12%
6%
1%
1%
License Areas
Academically Advanced PreK-8
Adult Basic Education
Biology 5-8
Biology 8-12
Chemistry 5-8
Chemistry 8-12
Early Childhood PreK-2
Earth Science 5-8
Earth Science 8-12
Elementary 1-6
Elementary Mathematics 1-6
ELL PreK-6
ELL 5-12
General Science 1-6
General Science 5-8
Instructional Technology
Mathematics 8-12
Middle School
Middle School Mathematics 5-8
Middle School Math/Science 5-8
Physics 5-8
Physics 8-12
Students w/ Moderate Disability PreK-8
Students w/ Moderate Disability 5-12
Students w/ Severe Disability
Technology/Engineering 5-12
UMass Donahue Institute
Research and Evaluation Group
54
MMSP State-level Evaluation (Year 10)
Appendix D: High Need Districts
Appendix D: High Need District Eligibility Criteria
High Need Districts (See list below.):
1.
For proposals with a mathematics content focus: A district is considered to be a high need district if it has a
mathematics proficiency index for grades 4-8 that is below the state target for Cycle II for MMSP Year 1projects or
below the state target for Cycle III for MMSP Year 2 projects. Priority will be given to high need districts with two or
more schools identified for improvement in mathematics.
2. For proposals with a science and/or technology/engineering content focus: A district is considered to be a high need
district if it has a science proficiency index for grades 5-8 in 2003 that is at or below the 20th percentile for the state.
In addition, a high need district must demonstrate that there is a high number or percentage of teachers in the district who are
teaching in the academic subject or grade level for which they have not demonstrated subject matter competency through
licensure or completion of the professional development activities in their HOUSSE plans.
An interested district that is not identified as high need is encouraged to contact a high need district to explore becoming a
partner in the proposed program (e.g., vocational technical schools are encouraged to contact feeder school districts).
MA FY2004 High Need Districts
DISTRICT
AVON
BARNSTABLE
BOSTON
BROCKTON
CAMBRIDGE
CHELSEA
CHICOPEE
CLARKSBURG
EASTHAMPTON
EVERETT
FAIRHAVEN
FALL RIVER
FITCHBURG
FLORIDA
GARDNER
GREENFIELD
HAVERHILL
HOLBROOK
HOLYOKE
HULL
LAWRENCE
LOWELL
LYNN
MALDEN
MEDFORD
METHUEN
NEW BEDFORD
NORTH ADAMS
PITTSFIELD
PROVINCETOWN
RANDOLPH
REVERE
SALEM
SOMERVILLE
SOUTHBRIDGE
SPRINGFIELD
TAUNTON
MATH


























UMass Donahue Institute
Research and Evaluation Group
SCIENCE
T/E




































WALTHAM
WARE
DISTRICT
WAREHAM
WEBSTER
WEST SPRINGFIELD
WESTFIELD
WINCHENDON
WINTHROP
WORCESTER
ABBY KELLEY FOSTER CS
ATLANTIS CS
BENJAMIN BANNEKER CS
BOSTON RENAISSANCE CS
CONSERVATORY LAB CS
EDWARD BROOKE CS
FREDERICK DOUGLASS CS
LAWRENCE FAMILY DEV CS
LOWELL COMMUNITY CS
NEIGHBORHOOD HOUSE CS
NEW BEDFORD GLOBAL CS
NEW LEADERSHIP HMCS
NORTH CENTRAL REG CS
ROBERT M. HUGHES CS
SABIS INTERNATIONAL CS
SEVEN HILLS CS
SOMERVILLE CS
UPHAMS CORNER CS
ATHOL-ROYALSTON RSD
BERKSHIRE HILLS RSD
FRONTIER RSD
GILL-MONTAGUE RSD
HAMPSHIRE RSD
HAWLEMONT RSD
MOUNT GREYLOCK RSD
RALPH C MAHAR RSD

MATH























SCIENCE
T/E































55
MMSP State-level Evaluation (Year 10)
Appendix D: High Need Districts
MA FY2005 High Need Districts
DISTRICT
Grade 5
ATTLEBORO
SCI
Grade 8 Grades 4SCI
8 Math


AVON
BOSTON



BOURNE

BROCKTON
CAMBRIDGE






CHELSEA



CHICOPEE



CLARKSBURG



DEDHAM

DRACUT

EAST BRIDGEWATER

EASTHAMPTON

EVERETT



FAIRHAVEN
FALL RIVER



FITCHBURG



FLORIDA

GARDNER

HAVERHILL


GREENFIELD



HOLBROOK
HOLYOKE


HULL


LAWRENCE



LEOMINSTER

LOWELL



LYNN



MALDEN



MEDFORD


METHUEN


NEW BEDFORD



NORTH ADAMS



ORANGE

OXFORD

QUINCY

PITTSFIELD

RANDOLPH

REVERE




Grade 5
Grade 8 Grades 4-
SCI
SCI
8 Math
SOUTHBRIDGE



SPRINGFIELD
TAUNTON
WALTHAM
WARE
WAREHAM
WEBSTER
WESTFIELD
WINCHENDON
WINTHROP
WORCESTER
ACADEMY OF STRATEGIC CS
BENJAMIN BANNEKER CS
FREDERICK DOUGLASS CS
EDWARD BROOKE CS
CONSERVATORY LAB CS
COMMUNITY DAY CS
SABIS INTERNATIONAL CS
NEIGHBORHOOD HOUSE CS
ABBY KELLEY FOSTER REG CS
SO.BOSTON HARBOR ACAD CS
ROBERT M. HUGHES ACAD CS
LAWRENCE FAMILY DEV. CS
LOWELL COMMUNITY CS
NEW LEADERSHIP HMCS
NEW BEDFORD GLOBAL HMCS
NORTH CENTRAL CS
BOSTON RENAISSANCE CS
SEVEN HILLS CS
SOMERVILLE CS
PROSPECT HILL ACADEMY CS
UPHAMS CORNER CS
ATLANTIS CS
ADAMS-CHESHIRE
ATHOL-ROYALSTON
BERKSHIRE
FREETOWN-LAKEVILLE
GATEWAY
GILL-MONTAGUE
HAMPSHIRE
HAWLEMONT
NEW SALEM-WENDELL





































































ROCKLAND
SALEM



SOMERVILLE



UMass Donahue Institute
Research and Evaluation Group
DISTRICT
56
MMSP State-level Evaluation (Year 10)
Appendix D: High Need Districts
MA FY2006 High Need Districts
DISTRICT
ATTLEBORO
Grade 5
SCIENCE

Grade 8
SCIENCE
Grades 4-8
MATH

AVON
DISTRICT
Grade 5
SCIENCE
WALTHAM

WARE

BOSTON

BOURNE

BROCKTON



WESTFIELD
CAMBRIDGE



WINCHENDON
CHELSEA



WINTHROP

CHICOPEE






WORCESTER
ACADEMY OF STRATEGIC
CS
BENJAMIN BANNEKER CS

CLARKSBURG


DRACUT


EASTHAMPTON

EVERETT

FAIRHAVEN
WAREHAM
WEBSTER
DEDHAM
EAST
BRIDGEWATER












EDWARD BROOKE CS

CONSERVATORY LAB CS

COMMUNITY DAY CS

SABIS INTERNATIONAL CS


NEIGHBORHOOD HOUSE CS



FALL RIVER



FITCHBURG



ABBY KELLEY FOSTER
REGIONAL CS

SOUTH BOSTON HARBOR
ACADEMY CS


ROBERT M. HUGHES
ACADEMY CS



LAWRENCE FAMILY
DEVELOPMENT CS


LOWELL COMMUNITY CS

FLORIDA

GARDNER

GREENFIELD
HAVERHILL


HOLBROOK

HOLYOKE


HULL


LAWRENCE


LEOMINSTER

LOWELL



LYNN



NORTH CENTRAL CS
MALDEN



BOSTON RENAISSANCE CS

MEDFORD


SEVEN HILLS CS

METHUEN


SOMERVILLE CS
PROSPECT HILL ACADEMY
CS

UPHAMS CORNER CS

ATLANTIS CS

ADAMS-CHESHIRE RSD

ATHOL-ROYALSTON RSD


NEW BEDFORD



NORTH ADAMS



ORANGE

OXFORD
QUINCY


PITTSFIELD

RANDOLPH

REVERE

ROCKLAND
SALEM









SOUTHBRIDGE



SPRINGFIELD



TAUNTON


SOMERVILLE
UMass Donahue Institute
Research and Evaluation Group
Grades 4-8
MATH


FREDERICK DOUGLASS CS

Grade 8
SCIENCE




NEW LEADERSHIP HMCS


NEW BEDFORD GLOBAL
HMCS












BERKSHIRE HILLS RSD
FREETOWN-LAKEVILLE RSD

GATEWAY RSD

GILL-MONTAGUE RSD

HAMPSHIRE RSD

HAWLEMONT RSD
NEW SALEM-WENDELL RSD




57
MMSP State-level Evaluation (Year 10)
Appendix D: High Need Districts
MA FY2007 High Need Districts
DISTRICT
Science/Tech.
Engineering
ATTLEBORO
Math

BOSTON

BROCKTON

BROOKFIELD
Science/Tech.
Engineering
Math
REVERE

ROCKLAND


SALEM


SAUGUS


SEEKONK


BARNSTABLE
DISTRICT
CAMBRIDGE


SOMERVILLE
CHELSEA


SOUTHAMPTON
CHICOPEE

SOUTHBRIDGE
CLINTON

SOUTH HADLEY
DOUGLAS

SPRINGFIELD
EASTHAMPTON

STOUGHTON

ERVING

TAUNTON

EVERETT

WALTHAM

FAIRHAVEN

WARE

FALL RIVER

WAREHAM

WEBSTER
FRAMINGHAM

WESTFIELD

FREETOWN

WESTPORT

GARDNER

WEST SPRINGFIELD

GLOUCESTER

WINCHENDON

GRANVILLE

WINTHROP

GREENFIELD

WORCESTER
HAVERHILL

EXCEL ACADEMY CS

HOLBROOK

FOUR RIVERS CS

HOLYOKE

BERKSHIRE ARTS CS


ACADEMY OF STRATEGIC
CS
FITCHBURG


HUDSON
LAWRENCE


LEE

LEICESTER



















METHUEN

MIDDLEBOROUGH

MONSON

NAHANT





NORTHAMPTON

NORTH BROOKFIELD

NORTON

OXFORD

PALMER

PITTSFIELD

QUINCY
UMass Donahue Institute
Research and Evaluation Group


MASHPEE
RANDOLPH



NORTH ADAMS

CONSERVATORY LAB CS
SABIS INTERNATIONAL
CS
ROBERT M. HUGHES
ACAD CS
LAWRENCE FAMILY DEV
CS
LOWELL COMMUNITY CS
MALDEN
NEW BEDFORD


LUDLOW
LYNN

MURDOCH MIDDLE CS
LEOMINSTER
LOWELL
SMITH LEADERSHIP
ACAD CS
BENJAMIN BANNEKER CS

NEW LEADERSHIP HMCS
NEW BEDFORD GLOBAL
CS
NORTH CENTRAL CS










PIONEER VALLEY
PERFORMING
BOSTON RENAISSANCE
CS
SALEM ACADEMY CS






SEVEN HILLS CS
PROSPECT HILL ACAD
CS
SOUTH SHORE CS

UPHAMS CORNER CS





58
MMSP State-level Evaluation (Year 10)
DISTRICT
Science/Tech.
Engineering
Appendix D: High Need Districts
Math
ATLANTIS CS

ADAMS-CHESHIRE REG.

ATHOL-ROYALSTON

BERKSHIRE HILLS

FREETOWN-LAKEVILLE

GATEWAY

GILL-MONTAGUE

HAMPSHIRE

MOHAWK TRAIL

NARRAGANSETT

PIONEER VALLEY

RALPH C MAHAR

SILVER LAKE

UMass Donahue Institute
Research and Evaluation Group
59
MMSP State-level Evaluation (Year 10)
Appendix D: High Need Districts
MA FY2008 and FY2009 High Need Districts
MA FY2009 and FY2010 High Need Districts (same as MA FY2008 and FY2009 High Need Districts)
Criteria: A high-need district in science and technology/engineering is a district that has a grade 8 and a high school science
CPI of less than 60. A high-need district in mathematics is a district that has been identified for corrective action in
mathematics, or districts with one or more Commonwealth Priority Schools identified for mathematics.
DISTRICT
Science
Math


NEW BEDFORD

NORTH ADAMS
BOSTON
BRIDGEWATER-RAYNHAM
DISTRICT
Science
Math



BROCKTON


NORTH BROOKFIELD
CAMBRIDGE


PEABODY
CHICOPEE


PITTSFIELD
EASTHAMPTON

EVERETT


RANDOLPH


FALL RIVER


REVERE


FALMOUTH

SALEM
FITCHBURG

SOMERVILLE


SOUTHBRIDGE


GARDNER

GATEWAY







PLYMOUTH

SPENCER-EAST BROOKFIELD

GLOUCESTER

SPRINGFIELD

GREENFIELD

WAREHAM

HAVERHILL

WESTFIELD

HOLBROOK


WOBURN
HOLYOKE


WORCESTER

HULL

BERKSHIRE ARTS CS

LAWRENCE


LOWELL





MARLBOROUGH

NEW BEDFORD GLOBAL CS

MEDFORD

METHUEN



LYNN
UMass Donahue Institute
Research and Evaluation Group


BENJAMIN BANNEKER CS
COMMUNITY CS OF
CAMBRIDGE
NEW LEADERSHIP HMCS
LUDLOW



60
MMSP State-level Evaluation (Year 10)
Appendix D: High Need Districts
MA FY2010 and FY2011 High Need Districts
Criteria: High-need districts are defined as districts in corrective action or single school districts in corrective action or
restructuring status under No Child Left Behind.
DISTRICT
Agawam
Pittsfield
Boston
Ralph C. Mahar
Bridgewater-Raynham
Randolph
Brockton
Revere
Chelsea
Salem
Chicopee
Somerville
Everett
Southbridge
Fall River
Springfield
Fitchburg
Waltham
Gardner
Wareham
Gloucester
Westfield
Greater Lawrence Regional Vocational Technical
Weymouth
Greenfield
Woburn
Hampshire
Worcester
Haverhill
Abby Kelley Foster Charter Public
Holbrook
Benjamin Banneker Charter Public
Holyoke
Berkshire Arts and Technology Charter Public
Lawrence
Boston Renaissance Charter
Leominster
Lowell Community Charter Public
Lowell
Mystic Valley Regional Charter
Lynn
New Leadership Charter
Malden
North Central Charter Essential
Marlborough
Sabis International Charter
Medford
Seven Hills Charter
Methuen
Silver Hill Horace Mann Charter
New Bedford
Smith Leadership Academy Charter Public
Orange
South Shore Charter Public
Peabody
UMass Donahue Institute
Research and Evaluation Group
61
MMSP State-level Evaluation (Year 10)
Appendix D: High Need Districts
MA FY2012 High Need Districts
FY2012 Massachusetts Math/Science Partnership Qualifying High Needs Districts List
Lead LEA Criteria: The Lead LEA must be a Level 3 or 4 district as identified by the Massachusetts Department
of Elementary and Secondary Education (Department).
For the purpose of the MMSP, Level 3 & 4 districts are considered high need districts.
Level 3 districts with one or more schools among the lowest-performing 20% based on quantitative indicators.
Level 4 districts identified by quantitative and qualitative indicators through a district review; districts with one or
more schools among the lowest-performing and least improving 2% based on quantitative indicators.
District Name
District Name
Charter District
Athol-Royalston
North Andover
Abby Kelly Foster Charter
Bellingham
Northampton
Boston
Northampton-Smith
Atlantis Charter
Berkshire Arts and Technology
Charter
Brockton
Northeast Metro Voc
Boston Day and Evening Academy
Chelsea
Orange
Boston Renaissance Charter
Chicopee
Pittsfield
Holyoke Community Charter
Dennis-Yarmouth
Quabbin
Lowell Community Charter
Everett
Quaboag Regional
Mystic Valley Charter
Fall River
Quincy
Leadership Charter
Fitchburg
Randolph
North Central Essential Charter
Framingham
Ralph C Mahar
Sabis International Charter
Gardner
Gill-Montague
Gloucester
Revere
Salem
So Middlesex Voc Tech Reg
Salem Academy Charter
Seven Hills Charter
Silver Hill Charter
Greater Lawrence RVT
Somerville
Smith Leadership Academy Charter
Greater Lowell Voc Tec
Southbridge
South Shore Charter
Haverhill
Southeastern Reg Voc Tech
Holbrook
Springfield
Holyoke
Stoneham
Lawrence
Taunton
Leominster
Waltham
Lowell
Ware
Lynn
Wareham
Malden
Webster
Medford
West Springfield
Middleborough
Mohawk Trail
Nantucket
Narragansett
Westfield
Winchendon
Winthrop
Worcester
New Bedford
MMSP State-level Evaluation (Year 10)
Appendix D: High Need Districts
MA FY2013 High Need Districts
Massachusetts Math/Science Partnership Qualifying High Need Districts List (FY2012-13)
Lead LEA Criteria: The Lead LEA must be a Level 3 or 4 district as identified by the Massachusetts Department of Elementary and Secondary
Education (Department). (http://www.doe.mass.edu/apa/accountability/default.html)
For the purpose of the MMSP, Level 3 & 4 districts are considered high need districts.
District Name
Adams-Cheshire
Athol-Royalston
Amherst-Pelham
Beverly
Boston
Brockton
Chelsea
Chicopee
Dracut
Easthampton
Everett
Fairhaven
Fall River
Fitchburg
Framingham
Franklin County
Freetown-Lakeville Regional
Gardner
Gill-Montague
Gloucester
Greater Lawrence RVT
Greater Lowell Voc Tec
Greenfield
Haverhill
Holbrook
Holyoke
Lawrence
Leominster
Lowell
Lynn
Malden
Marlborough
Methuen
Monson
Narragansett
District Name
New Bedford
North Adams
Northampton
Northampton-Smith
Northbridge
Orange
Oxford
Palmer
Peabody
Pittsfield
Plymouth
Quincy
Randolph
Ralph C Mahar
Revere
Rockland
Salem
Saugus
Somerville
Southbridge
Southeastern Reg Voc Tech
Spencer-E Brookfield
Springfield
Stoughton
Taunton
Waltham
Ware
Wareham
Watertown
Webster
Westfield
West Springfield
Weymouth
Winchendon
Worcester
Charter District
Boston Renaissance Charter
Global Learning Charter
Martin Luther King Jr. Charter
New Leadership Charter
Phoenix Charter Academy
Seven Hills Charter
UP Academy Charter Boston
Note: For the purpose of this
continuation grant, previously
identified Level 3 & Level 4
districts under which the original
competitive grant was awarded
continue to be eligible as high
needs partners.
05/02/13
MMSP State-level Evaluation (Year 10)
Appendix E: Enrollment and Attrition Rates by Course
Appendix E: Enrollment and Attrition Rates by Course
Enrollment and Attrition Information as Reported by Partnerships
Partnership
EduTronWorc-Lowell
Year
Offered
12/13
12/13
12/13
Everett
Edutron-Cohort 6
Subtotal
12/13
12/13
Subtotal
12/13
12/13
Lesley-Springfield
Northeastern
Subtotal
12/13
12/13
12/13
12/13
12/13
12/13
12/13
12/13
Subtotal
12/13
12/13
12/13
12/13
MCLA
Lesley-Brockton
Subtotal
12/13
Subtotal
12/13
12/13
12/13
12/13
12/13
12/13
12/13
12/13
12/13
Subtotal
TOTAL
Course Title
Decoding Common Core Mathematics in
Grades 2-6
Mathematics Learning Community
Focus, Coherence, and Rigor in Common
Core Mathematics Standards in Grades 5-12
53
Physics - Motion and Forces in CAPS
(Cars, Amusement Parks & Sports)
Integrating Sciences through Energy
53
Building the Bridge from Arithmetic to
Algebra
Implementing Focus, Coherence, and
Rigor in Common Core Mathematics
Standards
53
Number Theory
Geometry and Measurement I
Probability
Geometry and Measurement II
Functions and Algebra I
Number Theory
Geometry and Measurement I
Probability
53
Chemistry II: The Energetics of Chemical
Change
Physics I: Forces, Energy & Motion
Biology II: Ecology, Evolution and
Diversity
Mathematics I: Mathematics for Middle
School Science Teachers
53
Expressions and Equations for Middle
School Teachers
53
Number and Operations
Number Theory
Ratio, Rate, and Proportion
Using Fraction as Number
Geometry and Measurement I
Geometry and Measurement I
The Mathematics of Engineering and
Design
Number and Operations
Functions and Algebra I
53
53
Number
Enrolled First
Day
27
Number
Completed
Course
25
Attrition
Rate
7%
18
22
13
22
28%
0%
67 49
22
60 8%
22
10%
0%
13
35 49
36
13
35 8%
35
0%
0%
3%
46
45
2%
82 49
9
13
12
9
18
28
25
11
125 49
18
80 8%
8
12
12
9
17
27
25
11
121 8%
18
2%
11%
8%
0%
0%
6%
4%
0%
0%
3%
0%
28
25
27
25
4%
0%
19
14
26%
90 49
14
84 8%
13
7%
7%
14 49
16
12
16
14
22
19
6
13 8%
16
11
13
13
22
19
6
7%
0%
8%
19%
7%
0%
0%
0%
20
19
144 49
557 49
20
19
139 8%
532 8%
0%
0%
3%
4%
MMSP State-level Evaluation (Year 10)
Appendix F: Pre-Post Scores
Appendix F: Mean Percentage Scores for Pre- & Post-course Tests
Mean Percentage Scores for Pre- & Post-course Tests
Partnership
EduTronWorc-Lowell
Year
Offered
12/13
12/13
12/13
Everett
EdutronCohort 6
12/13
12/13
12/13
12/13
LesleySpringfield
Northeastern
12/13
12/13
12/13
12/13
12/13
12/13
12/13
12/13
12/13
12/13
12/13
12/13
MCLA
12/13
LesleyBrockton
12/13
12/13
12/13
12/13
12/13
12/13
12/13
12/13
12/13
25
Mean
Pre-test
(%)
58
Mean
Post-test
(%)
76
13
22
55
84
74
91
19
7
Yes
Yes
22
39
74
35
Yes
12
35
26
67
28
85
2
18
Yes
Yes
45
67
89
22
Yes
6
21
57
36
Yes
Geometry and Measurement I
Probability
Geometry and Measurement II
Functions and Algebra I
Number Theory
Geometry and Measurement I
Probability
Chemistry II: The Energetics of Chemical
Change
Physics I: Forces, Energy & Motion
Biology II: Ecology, Evolution and
Diversity
Mathematics I: Mathematics for Middle
School Science Teachers
Expressions and Equations for Middle
School Teachers
12
11
9
14
24
24
11
17
36
28
7
48
19
33
5
41
67
71
25
74
65
61
14
71
31
43
18
26
46
28
9
30
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
27
25
32
61
40
75
8
14
Yes
Yes
14
71
86
15
Yes
11
42
72
30
Yes
Number and Operations
15
41
64
23
Yes
Number Theory
Ratio, Rate, and Proportion
Using Fraction as Number
Geometry and Measurement I
Geometry and Measurement I
The Mathematics of Engineering and
Design
Number and Operations
Functions and Algebra I
11
12
12
22
19
5
19
39
29
19
18
5
66
71
45
25
26
8
47
32
16
6
8
3
Yes
Yes
Yes
Yes
Yes
N/A
19
19
11
60
32
77
21
17
Yes
Yes
Course Title
Decoding Common Core Mathematics in
Grades 2-6
Mathematics Learning Community
Focus, Coherence, and Rigor in Common
Core Mathematics Standards in Grades 512
Physics - Motion and Forces in CAPS
(Cars, Amusement Parks & Sports)
Integrating Sciences through Energy
Building the Bridge from Arithmetic to
Algebra
Implementing Focus, Coherence, and
Rigor in Common Core Mathematics
Standards
Number Theory
N
Change in
Mean
(% points)
18
p <.05
Yes
MMSP State-level Evaluation (Year 10)
Appendix G: High Need Districts, by Partnership
Appendix G: High Need Districts for All Funding Periods, by Partnership
High Need Districts for Cohort 1–Cohort 3, by Partnership
Number of Participants from High Need Districts
Partnership
EduTron (M)
Harvard University (M)
Lesley University (M)
MCLA – Science (S)
UMass Donahue Institute
Research and Evaluation Group
High Need
Districts
Fitchburg
Gardner
Subtotal
Boston
Boston Renaiss. CS
Cambridge
Fall River
Lowell
Malden
New Bedford
Somerville
Somerville CS/
Prospect Hill
Academy
Southbridge
Subtotal
Malden
Adams-Cheshire
Clarksburg
Florida
Mount Greylock
North Adams
Feb04–
Aug04
Sep04–
Aug05
37
14
51 (79%)
1
3
5
0
0
0
0
0
53
4
57 (88%)
3
2
4
0
1
0
1
2
0
0
9 (39%)
21 (21%)
Planning Year
5
0
18 (33%)
16 (19%)
6
1
2
0
5
Sep05–
Aug06
28
9
37 (84%)
10
0
9
4
0
3
0
4
0
1
31 (39%)
14 (19%)
5
1
1
0
3
Sep06–
Aug07
Cohorts
1&2
N/A
N/A
N/A
0
2
3
4
0
1
0
2
1
0
13 (54%)
3 (25%)
5
1
2
2
3
Sep06–
Aug07
Cohort 3
Sep07–Aug08
Cohort 3
Sep08–
Aug09
Cohort 3
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
66
MMSP State-level Evaluation (Year 10)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 1–Cohort 3, by Partnership
Number of Participants from High Need Districts
Partnership
MCLA – Science (S)
Salem State College (M)
Springfield/Holyoke
Public Schools (S)
Wareham PS (M)
WPI – Math (M)
UMass Donahue Institute
Research and Evaluation Group
High Need
Districts
Subtotal
Boston
Chelsea
Haverhill PS
Lynn
Salem
Somerville
Subtotal
Holyoke
Holyoke Comm. CS
Springfield
Subtotal
Wareham
Abby Kelley
Foster CS
Athol-Royalston
Berkshire Hills
Boston
Brockton
Cambridge
Chicopee
Fall River
Fitchburg
Lawrence Fam.
Devt. CS
Lowell
Community CS
New Bedford
Feb04–
Aug04
Sep04–
Aug05
0
0
1
32
18
0
51 (93%)
6
0
32
38(100%)
17 (46%)
14 (100%)
0
1
20
32
16
0
69 (86%)
17
0
31
48(100%)
11 (61%)
Sep05–
Aug06
10 (100%)
1
1
16
41
14
0
73 (79%)
19
0
28
47 (96%)
N/A
Sep06–
Aug07
Cohorts
1&2
13 (93%)
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep06–
Aug07
Cohort 3
Sep07–Aug08
Cohort 3
Sep08–
Aug09
Cohort 3
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0
2
3
N/A
N/A
N/A
N/A
1
0
0
0
0
4
0
0
0
1
0
1
0
1
1
0
0
0
1
0
1
2
0
1
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0
0
3
N/A
N/A
N/A
N/A
0
0
1
N/A
N/A
N/A
N/A
2
4
0
N/A
N/A
N/A
N/A
67
MMSP State-level Evaluation (Year 10)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 1–Cohort 3, by Partnership
Number of Participants from High Need Districts
Partnership
WPI – Math (M)
High Need
Districts
Feb04–
Aug04
MCLA – Math (M)
North Adams
Pittsfield
Ralph C. Mahar
Seven Hills CS
Somerville
Webster
Winchendon
Worcester
Subtotal
Adams-Cheshire
0
0
0
4
15(63%)
Started: Year 2
UMass Amherst (M)
North Adams
Pittsfield
Subtotal
Athol-Royalston
N/A
N/A
N/A
Started: Year 2
Chicopee
Easthampton
Gateway
Gill-Montague
Greenfield
Holyoke
Holyoke
Community CS
Ludlow
North Adams
Ralph C. Mahar
Springfield
Westfield
N/A
N/A
N/A
N/A
N/A
N/A
UMass Donahue Institute
Research and Evaluation Group
2
0
2
0
Sep04–
Aug05
0
0
0
0
1
1
8
7
27 (41%)
Planning Year
N/A
N/A
N/A
0
Sep05–
Aug06
0
2
0
2
0
0
6
10
32 (43%)
0
Sep06–
Aug07
Cohorts
1&2
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
3
Sep06–
Aug07
Cohort 3
Sep07–Aug08
Cohort 3
Sep08–
Aug09
Cohort 3
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
1
0
1 (9%)
0
2
1
6 (86%)
1
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0
0
0
0
2
5
2
0
0
0
0
5
1
1
1
2
0
3
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0
0
1
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0
0
0
7
2
0
0
0
4
2
2
1
1
7
2
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
68
MMSP State-level Evaluation (Year 10)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 1–Cohort 3, by Partnership
Number of Participants from High Need Districts
Partnership
UMass Amherst (M)
EduTron Lowell (M/S)
EduTron Fitchburg (M)
Lesley University (M)
North Shore (S)
UMass Donahue Institute
Research and Evaluation Group
High Need
Districts
Subtotal
Lowell
Fitchburg
Gardner
Leominster
Subtotal
Attleboro
Brockton
Fairhaven
Fall River
Haverhill PS
Holyoke
Malden
Middleborough
New Bedford
Northampton
Randolph
Revere
Saugus
Silver Hill Charter
Somerville
Taunton
Ware
Subtotal
Boston
Fitchburg
Holyoke
Feb04–
Aug04
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep04–
Aug05
16 (64%)
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep05–
Aug06
13 (37%)
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep06–
Aug07
Cohorts
1&2
23 (34%)
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep06–
Aug07
Cohort 3
N/A
54(100%)
17
7
10
34 (100%)
0
13
3
26
29
29
1
0
4
0
13
0
2
0
0
0
0
120 (94%)
0
1
0
Sep07–Aug08
Cohort 3
N/A
66(100%)
20
7
26
53 (98%)
1
3
1
18
23
18
0
0
0
0
14
2
5
0
11
0
1
97 (90%)
0
0
0
Sep08–
Aug09
Cohort 3
N/A
72 (100%)
19
12
34
65 (97%)
1
5
1
15
22
17
1
1
0
1
11
8
3
1
17
2
0
106 (95%)
4
1
1
69
MMSP State-level Evaluation (Year 10)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 1–Cohort 3, by Partnership
Number of Participants from High Need Districts
Partnership
North Shore (S)
UMass Amherst C3 (M/S)
Salem State College (M)
UMass Donahue Institute
Research and Evaluation Group
Feb04–
Aug04
Sep04–
Aug05
Sep05–
Aug06
Lynn
Revere
Somerville
Lowell Comm. CS
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep06–
Aug07
Cohorts
1&2
N/A
N/A
N/A
N/A
Subtotal
Athol Royalston
Chicopee
Easthampton
Gateway
Greenfield
Gill-Montague
Holyoke
Ludlow
Lynn
New Leadership LS
North Adams
Pittsfield
South Hadley
Springfield
West Springfield
Subtotal
Boston
Chelsea
Everett
Gloucester
Haverhill PS
Lynn
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
High Need
Districts
Sep06–
Aug07
Cohort 3
Sep07–Aug08
Cohort 3
Sep08–
Aug09
Cohort 3
0
0
16
0
3
9
14
1
0
8
13
0
17 (41%)
1
5
0
1
1
0
3
2
0
1
0
0
2
3
3
22 (46%)
1
1
3
6
4
10
27 (40%)
0
3
1
0
0
1
4
0
0
0
1
1
0
9
0
19 (38%)
0
0
1
3
3
10
27 (53%)
0
3
1
1
0
1
2
0
1
0
1
2
0
12
1
25 (47%)
N/A
N/A
N/A
N/A
N/A
N/A
70
MMSP State-level Evaluation (Year 10)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 1–Cohort 3, by Partnership
Number of Participants from High Need Districts
Partnership
Salem State College (M)
SE/Cape (S)
WPI – Science (S)
Worcester PS (M)
UMass Donahue Institute
Research and Evaluation Group
High Need
Districts
Feb04–
Aug04
Sep04–
Aug05
Sep05–
Aug06
Malden
Methuen
Peabody
Revere
Salem
Winthrop
Worcester
Subtotal
Barnstable
Brockton
Fall River
Horace Mann CS
Lawrence
New Bedford
Subtotal
Worcester
Southbridge
Subtotal
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep06–
Aug07
Cohorts
1&2
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Worcester
N/A
N/A
N/A
N/A
Sep06–
Aug07
Cohort 3
1
0
0
0
3
1
1
41
1
20
0
2
0
8
31
3
0
3
(82%)
(66%)
(16%)
(16%)
34 (83%)
Sep07–Aug08
Cohort 3
2
1
2
1
3
0
0
26
5
22
5
0
0
8
40
7
0
7
(65%)
(51%)
(54%)
(54%)
N/A
Sep08–
Aug09
Cohort 3
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0
32
10
0
1
2
45(50%)
3
1
4 (22%)
N/A
71
MMSP State-level Evaluation (Year 10)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 4–Cohort 6, by Partnership
Partnership
Boston PS (M)
Brockton PS (M)
Gateway RSD (S)
UMass Donahue Institute
Research and Evaluation Group
High Need
Districts
Boston
Medford
Subtotal
BridgewaterRaynham
Brockton
Fall River
Falmouth
Freetown/Lakeville
New Bedford
Plymouth
Quincy
Randolph
Seekonk
South Shore CS
Swansea
Wareham
Weymouth
Subtotal
Agawam
Chicopee
Easthampton
Gateway
Hampshire
Holyoke
Springfield
Westfield
Subtotal
40
0
40 (100%)
0
174
0
174 (100%)
0
Sep10–
Aug11
Cohorts
4&5
112
1
113 (99%)
1
21
9
0
0
0
0
0
0
0
0
0
0
0
30 (59%)
0
0
1
6
0
2
8
0
17 (39%)
28
12
1
3
0
3
2
0
1
0
1
0
0
51 (45%)
0
0
3
7
0
9
0
0
19 (100%)
24
13
0
0
1
0
0
2
0
1
0
3
4
49 (56%)
4
2
2
6
4
2
2
12
34 (72%)
Sep08–
Aug09
Cohort 4
Sep09–
Aug10
Cohort 4
Sep11–
Aug12
Cohorts
5&6
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep12–
Aug13
Cohorts
5&6
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
72
MMSP State-level Evaluation (Year 10)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 4–Cohort 6, by Partnership
Partnership
Lesley Springfield (M)
Cohort 4
Greater North Shore (S)
Randolph PS (S)
Springfield College (S)
UMass Donahue Institute
Research and Evaluation Group
High Need
Districts
Agawam
Chicopee
Holyoke
Springfield
Westfield
Subtotal
Boston
BridgewaterRaynham
Fitchburg
Lawrence
Lynn
Malden
Medford
Pioneer Charter
School of Science
Quincy
Randolph
Revere
Somerville
Waltham
Weymouth
Subtotal
Randolph
Weymouth
Subtotal
New Leadership
CS
Sep08–
Aug09
Cohort 4
0
1
9
35
4
49 (94%)
26
Sep09–
Aug10
Cohort 4
0
1
6
59
5
71 (84%)
26
Sep10–
Aug11
Cohorts
4&5
1
0
5
29
5
40 (93%)
35
0
0
1
0
0
3
0
0
0
4
1
0
0
2
1
0
5
3
1
0
0
0
1
0
0
0
0
31 (50%)
6
0
6 (25%)
0
0
1
8
12
0
0
52 (46%)
18
0
18 (41%)
0
1
0
2
9
3
1
63 (50%)
7
1
8 (24%)
1
Sep11–
Aug12
Cohorts
5&6
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep12–
Aug13
Cohorts
5&6
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
73
MMSP State-level Evaluation (Year 10)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 4–Cohort 6, by Partnership
Partnership
Springfield College (S)
Boston University (M)
EduTronWorc-Lowell (M)
UMass Donahue Institute
Research and Evaluation Group
High Need
Districts
Springfield
Subtotal
Boston
Brockton
Cambridge
Chelsea
Chicopee
Falmouth
Haverhill
Lawrence
Lowell
Medford
Peabody
Randolph
Salem
Somerville
Waltham
Weymouth
Woburn
Worcester
Subtotal
Boston
Brockton
Chelsea
Lawrence
Lowell
Lynn
Revere
Sep08–
Aug09
Cohort 4
Sep09–
Aug10
Cohort 4
26
26 (96%)
3
0
0
0
0
2
2
7
0
0
0
0
1
0
0
0
0
1
16 (39%)
N/A
N/A
N/A
N/A
N/A
N/A
N/A
30
30 (100%)
3
0
1
0
0
0
0
1
2
0
0
0
1
1
0
0
1
0
10 (29%)
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep10–
Aug11
Cohorts
4&5
20
21 (100%)
1
2
3
1
1
0
0
5
2
1
1
1
0
1
1
3
0
0
23 (58%)
1
1
2
3
28
2
1
Sep11–
Aug12
Cohorts
5&6
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0
0
0
0
27
0
0
Sep12–
Aug13
Cohorts
5&6
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0
0
0
0
27
0
0
74
MMSP State-level Evaluation (Year 10)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 4–Cohort 6, by Partnership
Partnership
EduTronWorc-Lowell (M)
(continued)
Everett (S)
EduTron-Cohort 6 (M)
UMass Donahue Institute
Research and Evaluation Group
High Need
Districts
Worcester
Region Voc-Tech
Greater Lawrence
Subtotal
Boston
Chelsea
Chicopee
Everett
Holbrook
Malden
Medford
Mystic Valley
Regional CS
North Andover
Rockland
Seekonk
Somerville
Waltham
Subtotal
Everett
Framingham
Lawrence
Malden
North Central
Charter Essential
Pittsfield
Revere
Subtotal
N/A
Sep10–
Aug11
Cohorts
4&5
27
Sep11–
Aug12
Cohorts
5&6
22
Sep12–
Aug13
Cohorts
5&6
22
N/A
N/A
1
0
0
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep08–
Aug09
Cohort 4
Sep09–
Aug10
Cohort 4
N/A
66 (76%)
2
2
0
6
1
1
11
0
0
1
1
1
1
27 (68%)
N/A
N/A
N/A
N/A
49 (100%)
2
1
0
6
0
6
10
3
1
0
0
0
0
29 (85%)
23
3
1
2
1
1
16
47 (68%)
49 (100%)
0
0
1
4
0
7
11
0
0
0
0
0
1
24 (77%)
22
0
0
2
0
0
5
29 (56%)
75
MMSP State-level Evaluation (Year 10)
Appendix G: High Need Districts, by Partnership
High Need Districts for Cohort 4–Cohort 6, by Partnership
Partnership
Lesley-Brockton (M)
Lesley-Springfield (M)
Cohort 6
MCLA (S)
Northeastern
UMass Donahue Institute
Research and Evaluation Group
High Need
Districts
Brockton
New Bedford
Quincy
Randolph
Subtotal
Holyoke
Springfield
Westfield
West Springfield
Subtotal
Pittsfield
Berkshire Arts and
Technology CS
Subtotal
Boston
Brockton
Chelsea
Lynn
Malden
Medford
Quincy
Randolph
Somerville
Stoneham
Taunton
Waltham
Winthrop
Subtotal
Sep08–
Aug09
Cohort 4
Sep09–
Aug10
Cohort 4
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep10–
Aug11
Cohorts
4&5
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Sep11–
Aug12
Cohorts
5&6
30
0
3
0
33 (94%)
5
14
1
0
20 (80%)
2
4
6 (35%)
11
4
1
0
1
1
7
1
1
0
1
0
1
29 (34%)
Sep12–
Aug13
Cohorts
5&6
56
1
18
2
77 (82%)
7
24
2
3
36 (71%)
3
1
4 (36%)
20
0
0
1
0
0
4
0
0
1
0
1
0
27 (43%)
76