Download MS WORD Document size: 1.7MB

Massachusetts Mathematics and Science Partnership Title IIB

Annual State-level Evaluation Report

Cohort 3 Reporting Period: September 1, 2006, through August 31, 2009

Cohort 4 Reporting Period: September 1, 2008, through August 31, 2009

Cumulative Reporting Period: February 2, 2004, through August 31, 2009

Prepared for the Massachusetts Department of Elementary and

Secondary Education

December 2010

Annual Report of the MMSP State-level Evaluation Contents

Contents

Program Description ................................................................................................................................... 4

Report Organization .................................................................................................................................... 6

Evaluation Plan and Activities ................................................................................................................... 7

State-level Evaluation ............................................................................................ 7

Local Evaluation and Related Technical Assistance ............................................. 7

Cohort 3 Activity: September 1, 2006 through August 31, 2009 ............................................................ 8

State-level Participant Background Data ............................................................... 8

Partnership-level Participant Background Data ................................................... 22

Cohort 4 Activity: September 1, 2008 through August 31, 2009 .......................................................... 31

State-level Participant Background Data ............................................................. 31

Partnership-level Participant Background Data ................................................... 41

Cumulative Summary: All Cohorts over All Funding Periods .............................................................. 50

Summary of Findings ............................................................................................................................... 66

Cumulative Findings ............................................................................................ 66

Cohort 3 Findings ................................................................................................. 68

Cohort 4 Findings ................................................................................................. 70

Appendix A: Participant Background Survey – September 2008 – August 2009 .... Ошибка! Закладка не определена.

Appendix B: Timeline for State-level Evaluation and TA Activities ..................... Ошибка! Закладка не определена.

Appendix C: Results of the Participant Background Survey for 2008-2009 ....... Ошибка! Закладка не определена.

Appendix D: High Need District Eligibility Criteria .......................... Ошибка! Закладка не определена.

Appendix E: Enrollment and Attrition Rates by Course ................. Ошибка! Закладка не определена.

Appendix F: Subject Matter Competency Demonstration Options Ошибка! Закладка не определена.

Appendix G: Mean Percentage Scores for Pre- & Post-course Tests ................. Ошибка! Закладка не определена.

Appendix H: High Need Districts for All Funding Periods, by Partnership ........ Ошибка! Закладка не определена.

Appendix I: Criteria that Account for Gain in Highly Qualified Status ................ Ошибка! Закладка не определена.

UMass Donahue Institute

Research and Evaluation Group I

Annual Report of the MMSP State-level Evaluation Tables

Tables Index

Table 1: Budgets: Cohort 3 Partnerships ............................................................................................................. 8

Table 2: Teaching Areas: Cohort 3 Participants ................................................................................................... 9

Table 3: Types of Schools of Unique Participants: Cohort 3 Participants ...................................................... 10

Table 4: High Need Status of Unique Participants from Public Schools: Cohort 3 Participants .................. 10

Table 5: High Need District Participants by Partnership: Cohort 3 Participants ........................................... 11

Table 6: Reasons for Participation: Cohort 3, All Seats .................................................................................... 13

Table 7: Repeat Participants: Cohort 3 Partnerships ........................................................................................ 14

Table 8: Total Enrollment and Attrition Information: Cohort 3 Partnerships ................................................. 15

Table 9a: Cohort 3 Science and Technology Teaching Areas – Regular Education ..................................... 17

Table 9b: Cohort 3 Science and Technology Teaching Areas – Special Education ...................................... 17

Table 9c: Cohort 3 Science and Technology Teaching Areas – ELL Education ............................................ 18

Table 10a: Cohort 3 Mathematics Teacher Levels – Regular Education ......................................................... 19

Table 10b: Cohort 3 Mathematics Teacher Levels – Special Education ......................................................... 19

Table 10c: Cohort 3 Mathematics Teacher Levels – ELL Education ............................................................... 20

Table 11: Number of Courses with Statistically Significant Gains in Mean Content Knowledge Scores:

Cohort 3 ................................................................................................................................................. 21

Table 12: EduTron Lowell Participant Background Information (M/S) ............................................................ 22

Table 13: EduTron Fitchburg Participant Background Information (M) .......................................................... 23

Table 14: Lesley University Participant Background Information (M) ............................................................. 24

Table 15: North Shore Participant Background Information (S) ...................................................................... 25

Table 16: UMass Amherst Participant Background Information (M/S) ............................................................ 26

Table 17: Salem State College Participant Background Information (M) ........................................................ 27

Table 18: SE/Cape Participant Background Information (S) ............................................................................ 28

Table 19: WPI Participant Background Information (S) .................................................................................... 29

Table 20: Worcester Public Schools Participant Background Information (M) .............................................. 30

Table 21: Budgets: Cohort 4 Partnerships ......................................................................................................... 31

Table 22: Teaching Areas: Cohort 4 Participants .............................................................................................. 32

Table 23: Types of Schools of Unique Participants: Cohort 4 Participants ................................................... 33

Table 24: High Need Status of Unique Participants from Public Schools: Cohort 4 Participants ............... 33

Table 25: High Need District Participants by Partnership: Cohort 4 Participants ......................................... 33

Table 26: Reasons for Participation: Cohort 4, All Seats ................................................................................. 35

Table 27: Repeat Participants: Cohort 4 Partnerships ...................................................................................... 35

Table 28: Total Enrollment and Attrition Information: Cohort 4 Partnerships ............................................... 36

Table 29a: Cohort 4 Science and Technology Teaching Areas – Regular Education ................................... 37

Table 29b: Cohort 4 Science and Technology Teaching Areas – Special Education .................................... 37

Table 29c: Cohort 4 Science and Technology Teaching Areas – ELL Education .......................................... 37

Table 30a: Cohort 4 Mathematics Teacher Levels – Regular Education ......................................................... 39

Table 30b: Cohort 4 Mathematics Teacher Levels – Special Education ......................................................... 39

Table 30c: Cohort 4 Mathematics Teacher Levels – ELL Education ............................................................... 39

Table 31: Number of Courses with Statistically Significant Gains in Mean Content Knowledge Scores for

Cohort 4 ................................................................................................................................................. 41

Table 32: Boston Public Schools Participant Background Information (M) ................................................... 42

Table 33: Brockton Public Schools Participant Background Information (M) ................................................ 43

Table 34: Gateway Regional School District Participant Background Information (S) ................................. 44

Table 35: Lesley University C4 Participant Background Information (M) ....................................................... 45


Research and Evaluation Group II

Annual Report of the MMSP State-level Evaluation Tables

Table 36: Greater North Shore Participant Background Information (S) ........................................................ 46

Table 37: Randolph Public Schools Participant Background Information (S) ............................................... 47

Table 38: Springfield College Participant Background Information (S) .......................................................... 48

Table 39: Boston University Participant Background Information (M) ........................................................... 49

Table 40: Overview of MMSP Partnership Participation ................................................................................... 50

Table 41: Budgets: All Partnerships, All Funding Periods ............................................................................... 51

Table 42: Teaching Areas: All Participants, All Funding Periods .................................................................... 53

Table 43. Types of Schools: All Unique Participants, All Funding Periods .................................................... 53

Table 44: High Need Status of All Unique Participants from Public Schools ................................................. 54

Table 45: Reasons for Participation of Participants: All Seats, All Funding Periods .................................... 55

Table 46: Repeat Participants: All Partnerships, All Funding Periods ............................................................ 55

Table 47: Highly Qualified Status: All Unique Participants, All Funding Periods .......................................... 58

Table 48: MTEL Tests Taken by All Participants – Total to Date ..................................................................... 59

Table 49a: Science and Tech/Engineering Teaching Areas of All Participants – Regular Education ......... 60

Table 49b: Science and Tech/Engineering Teaching Areas of All Participants – Special Education .......... 60

Table 49c: Science and Tech/Engineering Teaching Areas of All Participants – ELL Education ................ 60

Table 50a: Mathematics Teacher Levels of All Participants – Regular Education ......................................... 61

Table 50b: Mathematics Teacher Levels of All Participants – Special Education ......................................... 61

Table 50c: Mathematics Teacher Levels of All Participants – ELL Education ............................................... 61

Table 51: Number of Courses with Statistically Significant Gains in Mean Content Knowledge Scores.... 63


Research and Evaluation Group III

Annual Report of the MMSP State-level Evaluation Program Description

Program Description

The purpose of the Massachusetts Mathematics and Science Partnership Program (MMSP) is to improve student achievement in mathematics, science, and technology/engineering through intensive, high-quality professional development activities that focus on deepening teachers’ content knowledge. This multi-year project is funded by the U.S. Department of Education as part of the No Child Left Behind (NCLB) Act Title IIB funding stream.

Funding to local partnerships is administered by state education agencies; in Massachusetts this is the

Massachusetts Department of Elementary and Secondary Education (ESE), which awards funding through a competitive grant process.

Partnerships awarded these grants are required to include 1) a core partner that has been identified as a high need school district in the subject matter on which the partnership is focusing and 2) a core partner that is a science, technology and/or engineering, or mathematics (STEM) department from an institution of higher education. The partnerships are composed of higher education institutions, school districts, and, in some cases, private organizations involved in providing both pre-service and in-service training to teachers. Partnerships are required to offer courses that equal at least 45 hours of direct instruction followed by at least 20 hours of follow-up contact to support the implementation of course content in the classroom. Partnerships are encouraged to tailor the model used to deliver the professional development and follow-up to best fit the objectives of their programs along with their resources, expertise, and existing infrastructure. Partnership activities are guided by the following goals 1 :

Goal I Develop and implement an effective and sustained course of study for in-service teachers of STEM by integrating the courses of study into schools of arts and sciences and/or education at institutions of higher education.

Goal II Increase the number of STEM teachers in the partner school districts who are licensed in the subject area(s) and grade level(s) they teach.

Goal III Increase the number of STEM teachers in the partner school districts who participate in high quality professional development and advance their content knowledge.

Goal IV Develop and implement a systemic approach to STEM education by integrating professional development with district and school STEM improvement initiatives.

The program began in February 2004, and has had six funding periods, defined as follows:



Year 1: February 2, 2004 through August 31, 2004



Year 2: September 1, 2004 through August 31, 2005










1 Program goals were modified slightly between the beginning of the program and the beginning of the most recent funding period.


Research and Evaluation Group 4

Annual Report of the MMSP State-level Evaluation Program Description



Year 6: September 1, 2008 through August 31, 2009.

The partnerships who received initial funding in Year 1 are referred to as Cohort 1; those who received initial funding in Year 2 are referred to as Cohort 2; those who received initial funding in Year 4 are referred to as

Cohort 3; and those who received initial funding in Year 6 are referred to as Cohort 4.



Annual Report of the MMSP State-level Evaluation Report Organization

Report Organization

The purpose of this report is threefold: 1) to provide details regarding only Cohort 3 participation for all three years of funding for it, 2) to provide details regarding only Cohort 4 participation for the only year of funding for it thus far, 3) to provide a cumulative summary regarding participation for all cohorts over all funding periods.

Data supporting the first purpose address the period of September 1, 2006, through August 31, 2009; data supporting the second purpose address the period of September 1, 2008, through August 31, 2009; and data supporting the third purpose address the period of February 2, 2004, through August 31, 2009. Participant data were collected through the Participant Background Survey, an instrument developed by the UMass Donahue

Institute (UMDI) and administered by partnerships to each participant on the last day of each course. See

Appendix A for the survey used during Year 6. The purpose of this survey is to gather data about participants’ professional backgrounds and qualifications. This information provides a picture of who the participants are, aids in determining whether the courses are reaching the teachers who most need professional development, and aids in tracking how teacher qualifications may change during the MMSP funding period. Data from the survey regarding teacher licensure, possession of and progress towards earning degrees, and status in terms of

Massachusetts Tests for Educator Licensure (MTEL) exams allows determination of the number of teachers who meet criteria defining highly qualified status. Unless noted, data from the survey are reported in terms of unique individuals, regardless of the number of courses taken by each individual.

Data speaking to the strengthening of relationships between partnership members were collected through a section of the local evaluation reports that partnerships were required to submit to the ESE. In this section, partnerships were asked to describe the extent to which their courses had been integrated into activities of their higher education partners.



Annual Report of the MMSP State-level Evaluation Evaluation Plan and Activities

Evaluation Plan and Activities

State-level Evaluation

Although not required by the U.S. Department of Education, the ESE contracted with UMDI to conduct a statelevel evaluation of the MMSP. UMDI’s primary role as state-level evaluator is to coordinate program-wide collection of outcome data on behalf of the ESE. Data collection for the state-level evaluation is organized around a basic logic model for professional development initiatives shown below.

Local Evaluation and Related Technical Assistance

In addition to the state-level data collection, each partnership is required to conduct its own local evaluation. In an effort to support strong local evaluations, ESE required that partnerships sub-contract with UMDI to provide technical assistance on design and implementation of their local evaluations. The timeline listing the evaluation activities is found in Appendix B.



Annual Report of the MMSP State-level Evaluation Cohort 3 Activity

Cohort 3 Activity: September 1, 2006 through August 31, 2009

Cohort 3, which began in the 2006-2007 funding period, consisted of nine partnerships. Table 1 shows the funding received by Cohort 3 partnerships for the period beginning in September 2006 and ending in August

2007, for the period beginning in September 2007 and ending in August 2008, for the period beginning in

September 2008 and ending in August 2009, and for the cumulation of the three.

Table 1: Budgets: Cohort 3 Partnerships

Partnership

EduTron Lowell (M/S)

EduTron Fitchburg (M)

Lesley University C3 (M)

North Shore (S)

UMass Amherst C3 (M/S)

Salem State College C3 (M)

SE Cape (S)

WPI – Science (S)

Worcester PS (M)

TOTAL

Sep06-Aug07

$210,000

$102,000

$347,911

$196,474

$107,424

$120,882

$129,438

$99,586

$231,210

$1,544,925

State-level Participant Background Data

Sep07-Aug08

$220,000

$110,000

$355,626

$194,729

$216,281

$113,551

$181,420

$70,734

$1,462,341

Sep08-Aug09

$240,000

$120,000

$355,357

$199,871

$169,064

$36,604

$169,246

$94,852

$1,384,994

TOTAL

$670,000

$332,000

$1,058,894

$591,074

$492,769

$271,037

$480,104

$265,172

$231,210

$4,392,260

Cohort 3 consisted of nine partnerships, with four of the nine partnerships offering mathematics professional development, three offering science professional development, and two offering both mathematics and science professional development.

Across all three years of funding, 130 Cohort 3 courses were delivered. Of these 130 courses, 70 were mathematics courses, 58 were science courses, and two courses covered both math and science content. Of those

130 courses, 64 (49%) were unique, and 66 (51%) were repeat offerings. This section of the report summarizes data collected from participants in these courses.

Across all three years of funding for Cohort 3, there were 1,076 participants, and 389 of them took two or more courses within Cohort 3. By the end of the 2008-2009 funding period, 1,076 unique participants completed the

Participant Background Survey on one or more occasions. The term “unique participant” refers to each individual who participated in the program, regardless of how many courses he or she took. Data for items from the survey that help to convey participants’ professional backgrounds and motives for participation are discussed in the remainder of this section. All survey data for this group may be found in Appendix C. The responses to the survey questions are presented as frequencies and percentages. Not all percentages total 100% because many items allowed multiple responses and not all of the participants responded to all of the items.



Annual Report of the MMSP State-level Evaluation

Position of Participants

Cohort 3 Activity

At the time of their last completed survey from a Cohort 3 MMSP course, 92% of course participants identified themselves as teachers. Of all respondents, 75% were regular education teachers; 11% were special education or special education inclusion teachers; 3% were ELL teachers; 2% were department heads or curriculum coordinators; 2% were principals, assistant principals, or headmasters; 1% were support specialists; 1% were paraprofessionals; 1% were long-term substitutes; <1% were superintendents or assistant superintendents; and 4% indicated that they held “other” positions.

Content Taught

The distribution of teaching areas of respondents at the time of the survey is shown in Table 2. Because respondents identified all teaching areas that applied to their positions at the time of the survey, some selected multiple responses, so frequency totals exceed the number of respondents and percentages exceed 100%.

At the time of their last MMSP course, 38% of Cohort 3 participants were teaching mathematics (including elementary mathematics), 31% were teaching science, and 25% were teaching all subjects at the elementary level.

Table 2: Teaching Areas: Cohort 3 Participants

Sep06-Aug07 Sep08-Aug09

Teaching Areas

N=458

(Multiple responses permitted) n

N=477 N=463

% n

TOTAL

N=1076

%

Mathematics

Any science area

General Science

Biology

Earth Science

Chemistry

Physics

Technology/Engineering

153

151

72

24

13

18

15

9

Computer Science

Elementary (all subjects)

Elementary Mathematics

Other

5

148

41

21

Not Currently Teaching 19

Teaching Experience of Participants

33

33

16

5

3

4

3

2

161

213

107

25

15

27

25

14

1 N/A N/A

32 117 25

9

5

4

36

14

39

8

3

8

3

6

5

3

34

45

22

5

125

143

73

36

19

34

29

14

27 326

31 337

16 197

8 70

4

7

6

3

36

64

56

32

N/A N/A

150 32

5

265

44

14

10

3

85

41

38 8 84

At the time of their last completed survey from an MMSP course, the teaching experience of the 1,076 unique

Cohort 3 participants was as follows: 17% were in their first to third year of teaching, 40% had between four and ten years experience in education, 28% had between 11 and 20 years of experience, 15% reported over 20 years of experience, and 1% did not report.

Teaching Levels of Participants

For reporting purposes, schools in the participating districts were organized into categories of elementary schools

(grades K-5), K-8 schools, middle schools (grades 6-8), and high schools (grades 9-12). At the time of their last completed survey from an MMSP course, 34% of Cohort 3 participants were teaching in an elementary or K-8

3

6

5

3

30

31

18

7

1

25

8

4

8



Annual Report of the MMSP State-level Evaluation Cohort 3 Activity school, 42% were teaching in a middle school, 15% were teaching in a high school, 1% were teaching at both the middle and high school levels, and less than 1% were teaching at either all levels or other levels such as pre-K or an adult level. Those remaining either were not currently teaching or the level at which they taught was unknown.

Types of Schools of Participants

As shown in Table 3, 97% of unique Cohort 3 participants worked in a public school setting, and 2% worked in a non-public school setting.

Table 3: Types of Schools of Unique Participants: Cohort 3 Participants

School Type

Sep06-Aug07 Sep08-Aug09 n %

TOTAL n %

Public Schools (includes public charter schools)

Non-public School

Other or No Response

447

7

4

98

2

<1

462

7

8

TOTAL 458 100

High Need Status of Districts of Participants

477

97

2

2

100

452 98 1042

8

3

2

<1

19

15

463 100 1076

97

2

1

100

MMSP partnerships were required to include at least one high need district. Appendix D identifies the criteria for the high need designation.

The ESE expected that at least 50% of participants in each partnership would come from high need districts, and further, they set an informal goal that at least 75% of participants for each partnership would come from high need districts. Of all Cohort 3 participants, 72% came from high need districts.

Table 4 shows that by the end of the

2008-2009 funding period, 74% of Cohort 3 participants from public schools had come from high need districts.

Table 4: High Need Status of Unique Participants from Public Schools: Cohort 3 Participants

Sep06-Aug07 Sep08-Aug09 TOTAL

School Type n % n %

High Need District

Non-high Need District

Unknown

343

94

10

77 335

21 122

2 5

73

26

1

339

112

1

75

25

<1

772

265

5

74

25

1

TOTAL 447 100% 462 100% 452 100% 1042 100%

Table 5 presents the number of participants from high need districts organized by each partnership. An examination of high need district participation in individual partnerships reveals that over the course of each partnership’s involvement in MMSP since the beginning of MMSP, six of the nine partnerships had at least 50% of their participants coming from high need districts across all years of their involvement and five of nine partnerships exceeded the informal goal of having at least 75% of the participants come from high need districts.

If individual years of participation are examined, it is seen that six of the nine partnerships had at least 50% of



Partnership

High Need

District

EduTron Lowell

(M/S)

EduTron Fitchburg

(M)

Lesley University

(M)

North Shore (S)

UMass Amherst

(M/S)

Lowell

Fitchburg

Gardner

Leominster

Subtotal

Attleboro

Brockton

Fairhaven

Fall River

Haverhill PS

Holyoke

Malden

Middleborough

New Bedford

Northampton

Randolph

Revere

Saugus

Silver Hill Charter

Somerville

Taunton

Ware

Subtotal

Boston

Fitchburg

Holyoke

Lynn

Revere

Somerville

Lowell Community

CS

Subtotal

Athol-Royalston

Chicopee

Easthampton

Gateway


Research and Evaluation Group

Annual Report of the MMSP State-level Evaluation Cohort 3 Activity their participants coming from high need districts for each and every year of funding, and four of the nine partnerships had at least 75% of the participants come from high need districts for each and every year of funding.

As of their last Cohort 3 course, of the 389 individuals who took multiple courses within Cohort 3, approximately

77% were from high need public school districts, approximately 21% were from other public school districts, and approximately 2% either were from private schools or did not provide information on their districts.

Table 5: High Need District Participants by Partnership: Cohort 3 Participants

Sep06-Aug07 Sep08-Aug09 TOTAL n % n* %

3

26

29

29

1

0

4

0

13

0

54 100% 66 100%

17

7

20

7

10 26

34 100% 53 98%

0

13

1

3

1

18

23

18

0

0

0

0

14

2

22

17

1

1

0

1

5

1

15

1

11

8

72 100% 166 100%

19

12

46

21

34 55

65 97% 122 98%

36

42

1

14

1

31

1

1

4

1

18

9

2

0

0

5

0

11

3

1

17

7

1

17

0

0

0

1

2

0

2

1 ^

120 94% 97 90% 106 95% 186

0

1

0

0

0

16

0

0

0

3

9

14

4

1

1

0

8

13

4

2

1

3

12

28

94%

0 1

17 41% 27 40%

1

5

0

1

0

3

1

0

0 1

0

27 53% 51 42%

1 ^

3

1

10

2

1 2

11


Table 5: High Need District Participants by Partnership: Cohort 3 Participants


Partnership

High Need

District

Sep06-Aug07 Sep08-Aug09 TOTAL n % n* %

UMass Amherst

(M/S)

Greenfield

Gill- Montague

Holyoke

Ludlow

Lynn

New Leadership CS

North Adams

Pittsfield

South Hadley

Springfield

West Springfield

Subtotal

Salem State College Boston

(M) Chelsea

Everett

Gloucester

Haverhill PS

Lynn

Malden

Methuen

Peabody

Revere

Salem

Winthrop

Worcester

Subtotal

1

0

3

2

0

1

0

0

2

3

0

0

1

1

0

9

0

1

4

0

3 0

22 46% 19 38%

1 0

1

3

6

0

1

3

4

19

1

0

0

0

3

10

2

1

2

1

3

1

1

3

0

0

41 82% 26 65%

1 5

20 22

N/A

N/A

N/A

N/A

N/A

N/A

1

25

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

0

32

1

0

1

2

0

12

0

1

2

0

N/A

N/A

N/A

N/A

N/A

N/A

47%

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

3

2

23

1

1

2

1

1

8

2

3

61 45%

1

1

3

6

6

29

2

1

2

1

6

1

1

60 77%

6

56

SE/Cape (S)

WPI-Science (S)

Barnstable

Brockton

Fall River

Horace Mann CS

Lawrence

New Bedford

Subtotal

Worcester

0

2

0

8

5

0

0

8

31 66% 40 51%

3 7

0 0

10

0

1

2

45

3

1

50%

13

2

1

14

92 52%

15

1 Southbridge

Subtotal 3 16% 7 54% 4 22% 16 36%

Worcester PS (M) Worcester 34 83% N/A N/A N/A N/A 34 83%

*Note that if this column is summed, the total will not correspond to relevant data in Table 4 for two reasons: 1) Table 4 presents data for unique participants across all partnerships while Table 5 presents data for unique participants only within partnerships (and some participants took courses through multiple partnerships) and 2) The districts of some of those participants who crossed partnerships were not consistently classified as high need districts (either because of the content of the course or because of the cohort of the partnership offering the course).

^This figure is not included in the total for this partnership because the individual this figure represents took an additional course in a later year in a different high need district.




Reasons for Participation in MMSP Courses


For each course taken, respondents identified all of their reasons for participating. Unlike in the earlier portions of this report where data were presented for unique participants, data for this topic are presented for all course seats, since responses are uniquely relevant to each distinct course that a participant took. Table 6 presents findings for all 1,984 seats for all courses taken by Cohort 3 participants during the 2006-2007, 2007-2008, and 2008-2009 funding periods.

Table 6: Reasons for Participation: Cohort 3, All Seats

Reasons for Participation

(Multiple responses permitted)

To obtain graduate credit 464

To increase knowledge in content 438

To pursue a personal interest 213

To earn PDPs for recertification 187

To get an additional license

(certification)

To prepare for the Massachusetts

Test for Educator Licensure

(MTEL)

To earn PDPs for HOUSSE plan

134

113

62 requirement

To follow an administrator's suggestion

To obtain a first license

(certification)

Other

41

13

43

Sep06-Aug07 n

% of 632 course seats

73%

69%

34%

30%

21%

18%

10%

6%

2%

7%

Sep07-Aug08 n

%

of 701 course seats

525 75%

436 62%

174 25%

192 27%

124 18%

105 15%

N/A N/A

30

25

46

4%

4%

7%

Sep08-Aug09 n

466

442

218

163


72%

68%

34%

25%

124 19%

84

N/A

23

13

51

13%

N/A

4%

2%

8%

TOTAL n


1455 73%

1316 66%

605 30%

542 27%

382 19%

302 15%

62 3%

94 5%

51 3%

140 7%




Repeat Participation


Cohort 3 partnerships were successful at encouraging and retaining repeat participants. All of the nine partnerships offered multiple courses and all had participants who attended more than one course within that partnership.

In all, 389 participants attended multiple courses within Cohort 3. (Ten of these participants took courses from other Cohort 3 partnerships.) Across all MMSP courses ever, 424 Cohort 3 participants attended multiple courses, with 74 repeat participants taking courses from partnerships from previous MMSP cohorts.

Table 7 provides details regarding repeat participation. (The Table 12 through Table 20 also include data on repeat participation, as does Table 46.)

Table 7: Repeat Participants: Cohort 3 Partnerships

Partnership



Lesley University (M)

North Shore (S)

UMass Amherst (M/S)

Salem State (M)

SE/Cape (S)

WPI—Science (S)

Worcester Public Schools (M)

8

15

3

3

7

40

30

14

Number of

Courses

Offered to Date

10

Total

Number of

Unique*

Participants to Date

166

Number

Taking

Multiple

Courses in

Cohort 3

34

77

177

44

40

125

194

120

133

25

68

5

5

46

120

52

34

Number Taking

One Course in

Cohort 3 & One or

More Courses in

Previous Cohorts

0

14

1

0

0

11

1

0

12

TOTAL 130 1076 389 39

*Participants who participated in multiple courses across partnerships were counted only once in the partnership of their most recent course

Number Taking

Multiple Courses in

Cohort 3 & Previous

Cohorts

1

0

7

6

0

0

14

5

2

35




Attrition

Course attrition rates were generally low and averaged 4% across all courses across all partnerships for Cohort 3. Of the 130 Cohort 3 courses delivered across all three years, data on attrition rates were available for 128. Of the 128 courses for which data were available, 82 (64%) had an attrition rate of 0%, 25 (20%) had an attrition rate ranging between 1% and 10%, and 21 (16%) had an attrition rate of greater than 10%. Table 8 provides a breakdown, by partnership, of enrollment and attrition rates. Appendix E provides a breakdown, by course, of enrollment and attrition rates.

Table 8: Total Enrollment and Attrition Information: Cohort 3 Partnerships

Partnership Number of Courses Offered

Number of Participants

Enrolled First Day


Completed Courses

Attrition Rate

Sep 06 -

Aug 07


EduTron Lowell (M)


North Shore (S)

UMass Amherst (M/S)

Salem State (M)

SE/Cape (S)

WPI (S)

Worcester PS (M)

All Courses/Partnerships 37

*Excludes data for one course, which were missing.

2

1

10

9

4

4

3

1

3

Sep 07 -

Aug 08

4

3

13

9

5

4

6

1

N/A

45

Sep08-

Aug09

4

3

17

12

5

0

6

1

N/A

48

Sep 06 -

Aug 07

76

36

271

74

57

73

57

20

54

718

Sep 07 -

Aug 08

Sep08-

Aug09

Sep 06 -

Aug 07

85 74 76

91

250

101

68

84

231

36

260

92* 69

77 54

54* N/A

117 114

13

N/A

20

N/A

67

54

19

49

779 692 684

Sep 07 -

Aug 08

Sep08-

Aug09

Sep 06 -

Aug 07

81 74 0%

85

243

99

59

84

225

0%

4%

91* 7%

68 5%

47* N/A

111 109

13

N/A

18

N/A

8%

5%

5%

9%

738 669 5%

Sep 07 -

Aug 08

5%

7%

3%

2%

13%

13%

5%

0%

N/A

5%

Sep08-

Aug09

0%

0%

3%

1%

12%

N/A

4%

10%

N/A

3%




Highly Qualified Status


For Cohort 3 through the end of the 2008-2009 funding period, 32 unique participants attained highly qualified status. Of the 32 who attained highly qualified status, ten did so by passing the appropriate MTEL, five did so by earning a teaching license, two did so by obtaining undergraduate equivalents in content areas, one did so by completing a sufficient number of PDPs on a HOUSSE plan, one did so by meeting two or more criteria simultaneously, two did so by passing appropriate MTEL tests and changing their positions, and one did so by earning a relevant degree and changing positions. In addition, ten others did so merely by changing their positions. Additionally, it is likely that MMSP participation contributed to the attainment of highly qualified status for more participants after the 2008-2009 period had ended, but it was not possible to capture those gains during this reporting period.

Of the 32 who attained highly qualified status, 11 were from the Lesley University (M) partnership; five were from the UMass Amherst (M/S) partnership; four were from each of the EduTron Lowell (M/S) and SE/Cape (S) partnerships; three were from the North Shore (S) partnership, two each were from the EduTron Fitchburg (M) and Salem State (M) partnerships; and one was from the WPI (S) partnership.

Of the 32 who attained highly qualified status, 26 began participating during Cohort 3 while six began participating prior to Cohort 3. Of the 26 whose participation began during Cohort 3, ten were from the Lesley

University (M) partnership; four were from the EduTron Lowell (M/S) partnership, three each were from the

North Shore (S), SE/Cape (S), and UMass Amherst (M/S) partnerships; two were from the EduTron Fitchburg

(M) partnership; and one was from the WPI (S) partnership. Of the six whose participation began prior to Cohort

3, two each were from the Salem State (M) and UMass Amherst (M/S) partnerships and one came from each of the Lesley (M) and SE/Cape (S) partnerships. An identification of the criteria by which teachers who gained highly qualified status could demonstrate competency in their subject matter may be found in Appendix F.

Licensure and Degrees Held in Subjects Taught

By the end of the 2008-2009 funding period for Cohort 3, 297 regular education teachers, 23 special education teachers, and 11 ELL teachers reported teaching subjects in science or technology/engineering. Tables 9a, 9b, and

9c show how many teachers taught each individual science or technology/engineering subject from Cohort 3 through the 2008-2009 funding period. The tables also show the percentages of teachers whose survey responses indicated they were licensed in the subject they taught, and they show the percentages of teachers whose survey responses indicated they held a degree in the subject they taught. Table 9a provides information for regular education teachers, Table 9b provides information for special education teachers, and Table 9c provides information for ELL teachers. (The numbers presented in these tables exceed the number of teachers who reported teaching these subjects because some teachers taught more than one subject.) Across all science subject areas for regular education teachers, the licensing reported appeared to be appropriate for approximately 66% of the science subjects taught, and the degrees held corresponded to the science subject taught for approximately 25% of the science subjects taught. Across all science subject areas for special education teachers, the licensing reported appeared to be appropriate for approximately 49% of the science subjects taught, and the degree held by one teacher translated to a 3% correspondence between degree and subject taught. Across all science subject areas for

ELL teachers, the licensing reported appeared to be appropriate for 39% of the subjects taught, and the degree held by one teacher translated to a 6% correspondence between degree and subject taught.

To see the extent to which licensure and degree were appropriate for any particular type of teacher for any particular subject taught, look in the relevant table at data in the row corresponding to the subject of interest and in the column corresponding to the time period of interest. For example, to see the extent to which licensure and degree were appropriate for just regular education biology teachers in all of Cohort 3, look at Table 9a in the cells of the “Total” column in the row for the subject area “Biology” to learn that 57 teachers taught regular education biology and that of those 57, 74% were licensed in biology and 56% held degrees in biology.




Table 9a: Cohort 3 Science and Technology Teaching Areas – Regular Education

Sep06-Aug07

Subject

General Science

Biology

Chemistry

Physics

Earth Science

Technology/Engineering

Number of

Teachers n

59

21

18

14

11

8

License in

Subject

Taught

%*

59

62

33

43

45

13

Degree in

Subject

Taught

%*

20

67

17

14

27

13

*Of the number of regular education participants teaching this subject for this period

Number of

Teachers n

90

18

24

23

12

13

Sep07-Aug08

License in

Subject

Taught

%*

79

78

58

44

33

31

Degree in

Subject

Taught

%*

22

61

38

30

25

23

Number of

Teachers n

62

32

32

28

17

14

Sep08- Aug09

License in

Subject

Taught

%*

90

81

56

61

41

71

Degree in

Subject

Taught

%*

5

53

34

21

6

0

Number of

Teachers n

166

57

59

52

30

31

Total

License in

Subject

Taught

%*

78

74

56

54

40

48

Degree in

Subject

Taught

%*

16

56

34

25

17

13

Table 9b: Cohort 3 Science and Technology Teaching Areas – Special Education

Sep06-Aug07

Subject

General Science

Biology

Chemistry

Physics

Earth Science

Number of

Teachers n

10

3

2

0

1

License in

Subject

Taught

%*

10

0

0

0

0

Degree in

Subject

Taught

%*

10

0

0

0

0

Technology/Engineering 0 0 0

*Of the number of special education participants teaching this subject for this period

Number of

Teachers n

9

6

3

2

3

0

Sep07-Aug08

License in

Subject

Taught

%*

44

33

67

100

67

0

Degree in

Subject

Taught

%*

0

0

0

0

0

0

Number of

Teachers n

4

1

0

0

1

0

Sep08- Aug09

License in

Subject

Taught

%*

75

100

0

0

100

0

Degree in

Subject

Taught

%*

0

0

0

0

0

0

Number of

Teachers n

16

9

3

2

5

0

Total

License in

Subject

Taught

%*

44

33

67

100

60

0

Degree in

Subject

Taught

%*

6

0

0

0

0

0




Table 9c: Cohort 3 Science and Technology Teaching Areas – ELL Education


Sep07-Aug08 Sep08-Aug09 Total

Subject

General Science

Biology

Chemistry

Physics

Earth Science

Number of

Teachers n

6

1

0

0

0

License in

Subject

Taught

%*

33

100

0

0

0

Degree in

Subject

Taught

%*

0

100

0

0

0

Number of

Teachers n

6

3

2

1

1

License in

Subject

Taught

% *

50

33

0

0

0

Degree in

Subject

Taught

% *

0

33

0

0

0

Number of

Teachers n

10

3

2

1

1

License in

Subject

Taught

%*

50

33

0

0

0

Technology/Engineering 1 100 0 0 0 0 1 100 0

*Of the number of ELL education participants teaching this subject for this period

By the end of the 2008-2009 funding period for Cohort 3, 315 regular education teachers 51 special education teachers, and 14 ELL teachers reported teaching mathematics. Tables 10a, 10b, and 10c show how many Cohort 3 teachers taught at each mathematics level through the 2008-2009 funding period. The tables also show the percentages of teachers whose survey responses indicated they were licensed in the mathematics level at which they taught, and they show the percentages of teachers whose survey responses indicated they held a degree in mathematics. Table 10a provides information for regular education teachers, Table 10b provides information for special education teachers, and Table 10c provides information for ELL teachers. (The sum of the numbers presented in each of Tables 10a and 10b exceeds the number of teachers who reported teaching mathematics because some teachers taught mathematics at both the elementary and middle school levels.)

From an “ease-of-reading perspective,” it would be ideal merely to report on the percentages of teachers who were licensed to teach mathematics and the percentage of teachers who held mathematics degrees. Sometimes, though, teachers hold multiple positions and may only be licensed to teach some of what they are teaching, and what is really desired is to understand the extent to which what is being taught is done so by individuals licensed to teach it and by individuals who hold relevant degrees. As a result, the explanations of data that follow are done so in terms of teaching positions.

For MMSP mathematics teaching positions in regular education, the licensing appeared to be appropriate for approximately 81% of positions held, and 17% of the positions were held by individuals with mathematics degrees. For special education teachers, the licensing appeared to be appropriate for approximately 36% of the positions held, and 2% of the positions were held by individuals with mathematics degrees. For ELL teachers, the licensing appeared to be appropriate for approximately 43% of the positions held, and the mathematics degrees held by two teachers also translated to a 14% correspondence between degree and position when viewed across all

ELL mathematics courses taught.

To see the extent to which licensure and degree were appropriate for a particular type of teacher for a particular mathematics level taught, look in the relevant table at data in the row corresponding to the level of interest and in the column corresponding to the time period of interest. For example, to see the extent to which licensure and degree were appropriate for just regular education middle school math teachers in only 2007-2008, look at Table

10a in the cells of the “Sep07-Aug08” column in the row for the subject area “Middle School” to learn that 103

Degree in

Subject

Taught

%*

0

33

0

0

0



Annual Report of the MMSP State-level Evaluation Cohort 3 Activity teachers taught regular education middle school mathematics and that of those 103, 70% were licensed to teach middle school mathematics and 21% held mathematics degrees.

Table 10a: Cohort 3 Mathematics Teacher Levels – Regular Education

Sep06-Aug07 Sep07-Aug08

Level

Elementary School

Middle School

High School

Middle & High School

TOTAL Math

Number of

Teachers n

41

106

19

0

166

Licensed at Level

%*

42

69

79

0

63

Degree in

Math

%*

0

10

42

0

11

Number of

Teachers n

31

103

18

3

155

*Of the number of regular education participants teaching mathematics at this level for this period

Licensed at Level

%*

100

70

78

67

77

Degree in

Math

%*

16

21

50

0

23

Sep08- Aug09

Number of

Teachers n

34

94

11

0

139

Licensed at Level

%*

71

90

91

0

86

Degree in

Math

%*

3

13

55

0

14

Total

Number of

Teachers n

66

215

41

3

325

Licensed

at Level

%*

82

80

83

100

81

Table 10b: Cohort 3 Mathematics Teacher Levels – Special Education

Degree in

Math

%*

5

16

44

0

17

Sep06-Aug07

Level

Elementary School

Middle School

High School


TOTAL Math

Number of

Teachers n

1

16

6

1

24

Licensed at Level

%*

0

25

0

0

17

Degree in

Math

%*

0

0

0

0

0

Number of

Teachers n

1

23

4

0

28

*Of the number of special education participants teaching mathematics at this level for this period

Sep07-Aug08

Licensed at Level

%*

100

35

0

0

32

Degree in

Math

%*

0

9

0

0

7

Sep08- Aug09

Number of

Teachers n

4

9

1

0

14

Licensed

at Level

%*

75

67

0

0

64

Degree in

Math

%*

0

0

0

0

0

Total

Number of

Teachers n

6

38

8

1

53

Licensed

at Level

%*

67

37

13

0

36


Research and Evaluation Group

Degree in

Math

%*

0

3

0

0

2

19


Table 10c: Cohort 3 Mathematics Teacher Levels – ELL Education


Sep07-Aug08 Sep08-Aug09 Total

Level

Elementary School

Middle School

High School


Number of

Teachers n

0

5

1

0

License in

Subject

Taught

%*

0

20

0

0

Degree in

Subject

Taught

%*

0

0

100

0

Number of

Teachers n

0

7

2

0

TOTAL Math 6 17 17 9

*Of the number of ELL education participants teaching mathematics at this level for this period

Degrees Currently Pursued

License in

Subject

Taught

% *

0

57

100

0

67

Degree in

Subject

Taught

% *

0

0

50

0

11

Number of

Teachers n

0

11

3

0

14

License in

Subject

Taught

%*

0

36

67

0

43

Information on degrees currently being pursued in science, technology/engineering, and mathematics was derived from the most recently completed survey of each individual. Of the 297 regular education teachers, 23 special education teachers, and 11 ELL teachers reported teaching subjects in science or technology/engineering in

Cohort 3 through the 2008-2009 funding period, 38 science and technology/engineering teachers reported that they were pursuing degrees in science and technology/engineering. Of these 38 teachers, 31 were pursuing science degrees in their current areas of teaching and 29 were pursuing advanced degrees (master’s degrees and

CAGS degrees) in their current area of teaching.

Seven teachers taught multiple science and technology/engineering subjects. Of these, four were pursuing degrees that were relevant to at least one area that they were teaching (three were pursuing master’s degrees and one was pursuing a CAGS degree). Two of the teachers who taught multiple science and technology/engineering subjects were pursuing degrees in an area they taught and in an other science field (one was pursuing a master’s degree and one was pursuing a bachelor’s degree). One teacher who taught multiple science and technology/engineering subjects was pursuing a master’s degree in a science field not currently taught. Twenty-one teachers who were pursuing a degree in science and technology/engineering taught only general science. Of the 21 general science teachers who were pursuing a degree in science and technology/engineering, 17 were pursuing a degree in general science (15 were pursuing master’s degrees, one was pursuing a bachelor’s degree, one was pursuing a master’s degree and a CAGS degree). One general science teacher was pursuing a master’s degree in general science and in another science field not currently taught. Three general science teachers were pursuing master’s degrees in science fields in which they did not currently teach. Three chemistry teachers were pursuing degrees in science and technology/engineering. Of these three, two were pursuing master’s degrees in chemistry and one was pursuing a master’s degree in another science field. Of five physics teachers who were pursuing degrees in science and technology/engineering, four were pursuing their master’s degree in physics and one was pursuing a master’s degree in physics and in an additional science field not currently taught. Two earth science teachers were pursuing master’s degrees in science fields in which they did not currently teach.

Of the 315 regular education teachers, 51 special education teachers, and 14 ELL teachers who reported teaching mathematics in Cohort 3 through the 2008-2009 funding period, 21 were pursuing mathematics degrees. Of these,

19 were pursuing advanced degrees (master’s degree or CAGS degree). Of the 21 mathematics teachers who were pursuing a degree in mathematics, 12 taught at the middle school level. Ten of the middle school teachers were pursuing their master’s degrees and one was pursuing a bachelor’s degree, and one was pursuing both a master’s

Degree in

Subject

Taught

%*

0

0

67

0

14



Annual Report of the MMSP State-level Evaluation Cohort 3 Activity degree and a CAGS degree. Seven of the mathematics teachers pursuing mathematics degrees taught at the high school level. Of the high school teachers, six were pursuing their master’s degree and one was pursuing both a master’s degree and a CAGS degree. One elementary mathematics teacher was pursuing a bachelor’s degree and one teacher who taught both middle school and high school mathematics was pursuing a master’s degree.

Content Knowledge Gains

As a grant condition, partnerships were required to use a pre-course test and post-course test for each MMSP course to assess participants’ knowledge growth of the content taught in the course. For each course, the same instrument served as both the pre- and post-course test. While partnerships were permitted to locate and use an instrument with established reliability and validity, it was most often the case that such instruments were not available because a priority was placed on utilizing assessments that would reflect the precise content taught in each course. As a result, the faculty members who had developed the courses usually developed the assessments, and typically neither validity nor reliability was determined for them because time and resource constraints prohibited doing so.

Data were analyzed only for participants who completed both pre- and post-course assessments. To determine if the scores showed statistically significantly changes between pre- and post-course test administrations, a paired samples t -test was used for each course for which ten or more participants completed both pre- and post-course assessments, and a Wilcoxon Matched Pairs test was used for each course for which fewer than ten participants completed both pre- and post-course assessments.

Of the 130 courses that were delivered across all Cohort 3 partnerships through the 2008-2009 funding period, content assessments were administered for 128 courses. Changes in average percentage of items correct between pre- and post-course test administrations could be measured in 127 of these courses. (One course had only one participant and therefore it was not appropriate to report the data.) Of these 127 courses, all had gains in the average percentage of items correct between pre- and post- test administrations. Statistically significant improvements in scores on content knowledge assessments occurred in 87% of the 127 courses. Of the 17 courses not showing statistically significant improvement in scores, though, 12 had fewer than six participants, the smallest sample size at which it is possible to detect statistical significance at the level used for these analyses.

Table 11 provides an overview, by subject matter of courses delivered, of the numbers of courses that showed statistically significant gains. For detailed information on mean pre- and post-course content knowledge scores by partnership and by course, see Appendix G.

Table 11: Number of Courses with Statistically Significant Gains in Mean Content

Knowledge Scores: Cohort 3

Content Area

Sep06-Aug07 Sep07-Aug08 Sep08- Aug09 Total

Delivered, with Pre- and Posttests

Significant

Pre/Post

Gains


Significant

Pre/Post

Gains


Significant

Pre/Post

Gains


Significant

Pre/Post

Gains

Math

Science & Technology/

Engineering

21

15

21

11

23

22

23

19

24*

22

22

13

68*

58

66

43

Math and Science 1 0 0 0 1 1 2 1

TOTAL 37 32 45 42 47 * 36 128 * 110

*One partnership offering math courses did not administer pre- or post-course content knowledge assessments for two courses; those two courses were excluded from this figure.




Partnership-level Participant Background Data


Presented in Table 12 through Table 20, this section offers an overview of selected participant survey data for each Cohort 3 partnership. These data were collected at the end of each course that was offered by each partnership. The responses to the survey questions are presented as frequencies and percentages of the total number of participants in the partnership for the 2006-2007, 2007-2008, and 2008-2009 periods and a cumulation of the three. In cases where not all participants responded to all of the items and in cases where multiple responses were permitted, percentages presented may not total 100%.

The section of each table identified as “Participants Who Took Multiple Courses” contains two categories. Each is defined as follows: “In Cohort 3” refers to participants who took two or more courses within the context of Cohort

3 – they participated in no courses that had been offered during previous cohorts. “In previous cohorts” refers to participants who took one course through Cohort 3 and at least one additional course that had been offered during a previous cohort.

The “Highly Qualified” section of each table includes the response option “Private school/Not applicable.”

Participants categorized under this option refer to those who either were teaching at a private school or for whom highly qualified status would be irrelevant, such as school administrators.

Table 12: EduTron Lowell Participant Background Information (M/S)

Total Number of Participants

Participants Who Took Multiple Courses

In Cohort 3

In previous cohorts

Teach Regular Education

Teach Special Education (sole and inclusion)

Teach ELL, ESL, or Sheltered English

Immersion

Teach Elementary (all content areas)

Teach Elementary Math

Teach Mathematics Above Elementary

Teach Science or


Teaching in High Need District

Highly Qualified Yes

No

In some, but not all areas

Cannot be determined

Private school/Not applicable

Have Passed One or More MTEL Exams

Hold One or More Teaching Licenses

Number of

Participants

Sep06 - Aug07

54

12 (22%)

0 (0%)

24 (44%)

7 (13%)

15 (28%)

7 (13%)

16 (30%)

9 (17%)

54 (100%)

11 (20%)

23 (43%)

2 (4%)

1 (2%)

17 (31%)

17 (31%)

49 (91%)

Number of

Participants

Sep07 - Aug08

66

16 (24%)

0 (0%)

39 (59%)

7

2

(11%)

(3%)

17 (26%)

10 (15%)

25 (38%)

22 (33%)

66 (100%)

30 (45%)

18 (27%)

4 (6%)

7

7

(11%)

(11%)

26 (40%)

64 (97%)

Number of

Participants

Sep08 - Aug09

72

17 (24%)

0 (0%)

53 (74%)

6 (8%)

4 (6%)

13 (18%)

4 (6%)

29 (40%)

19 (26%)

72 (100%)

32 (44%)

16 (22%)

6 (8%)

9 (13%)

9 (13%)

29 (40%)

69 (96%)

Number of

Participants*

TOTAL

166

34 (21%)

0 (0%)

109 (66%)

21 (13%)

5 (3%)

42 (25%)

15 (9%)

60 (36%)

40 (24%)

166 (100%)

62 (37%)

52

6

(31%)

(4%)

18

28

(11%)

(17%)

63 (38%)

156 (94%)

*The total number of participants refers to unique participants across all years, combined. Because some participants took more than one course across years, this total is not equivalent to the sum of participants over all three years.




Table 13: EduTron Fitchburg Participant Background Information (M)




In Cohort 3

In previous cohorts


Number of

Participants

Sep06 - Aug07

34

0 (0%)

18 (53%)

23 (68%)

Number of

Participants

Sep07 - Aug08

54

16 (30%)

1 (2%)

33 (61%)

Number of

Participants

Sep08 - Aug09

67

30 (45%)

12 (18%)

46 (69%)

Number of

Participants*

TOTAL

125

46 (37%)

25 (20%)

84 (67%)



Immersion




Teach Science or Technology/Engineering


8 (24%)

13 (38%)

1 (3%)

17 (50%)

4 (12%)

34 (100%)

11 (20%)

0 (0%)

29 (54%)

5 (9%)

15 (28%)

2 (4%)

53 (98%)

11 (16%)

24 (19%)

0 (0%)

34 (51%)

13 (19%)

0 (0%)

58 (46%)

15 (12%)

16 (24%)

3 (5%)

37

6

(30%)

(5%)

65 (97%) 122 (98%)


No






19 (56%)

8 (24%)

3 (9%)

2 (6%)

2 (6%)

17 (50%)

31 (91%)

18 (33%)

22 (41%)

1 (2%)

6 (11%)

7 (13%)

23 (43%)

54 (100%)

32 (48%) 52 (38%)

19 (28%)

3 (5%)

4 (6%)

47 (42%)

2 (2%)

9 (7%)

9 (13%)

34 (51%)

15

59

(12%)

(47%)

63 (94%) 119 (95%)





Table 14: Lesley University Participant Background Information (M)


Number of

Participants

Sep06 - Aug07

Number of

Participants

Sep07 - Aug08

Number of

Participants

Sep08 - Aug09

Number of

Participants*

TOTAL



In Cohort 3

In previous cohorts




127

66 (52%)

0 (0%)

92 (72%)

12 (9%)

108

73

15

(68%)

1 (1%)

62 (57%)

(14%)

112

89 (80%)

6 (5%)

124 (63%)

7 (4%)

82 (73%) 140 (71%)

10 (9%)

198

22 (11%)

Immersion




Teach Science or




No



49 (38%)

15 (12%)

53 (42%)

3 (2%)

120 (94%)

63 (50%)

45 (35%)

2 (2%)

0 (0%)

2 (2%)

38 (35%)

13

46

(12%)

(43%)

2 (2%)

97 (90%)

52 (48%)

34 (32%)

0 (0%)

10 (9%)

7 (6%)

37 (33%)

15

48

(13%)

(43%)

10 (5%)

61 (31%)

25

84

(13%)

(42%)

3 (3%) 6 (3%)

106 (95%) 186 (94%)

65 (58%) 66 (33%)

26 (23%) 98 (50%)

2 (2%)

6 (5%)

1 (1%)

12 (6%)



17 (13%)

61 (48%)

12 (11%)

55 (51%)

13 (12%) 21 (11%)

67 (60%) 103 (52%)

Hold One or More Teaching Licenses 119 (94%) 103 (95%) 108 (96%) 189 (96%)





Table 15: North Shore Participant Background Information (S)




In Cohort 3

In previous cohorts




Immersion




Teach Science or




No






Number of

Participants

Sep06 - Aug07

41

9 (22%)

0 (0%)

28 (68%)

10 (24%)

2 (5%)

0 (0%)

7 (17%)

33 (80%)

16 (39%)

19 (46%)

14 (34%)

5 (12%)

1 (2%)

2 (5%)

18 (44%)

39 (95%)

Number of

Participants

Sep07 - Aug08

68

24 (35%)

0 (0%)

56 (82%)

7 (10%)

1 (2%)

3 (4%)

0 (0%)

8 (12%)

60 (88%)

27 (40%)

38 (56%)

18 (27%)

7 (10%)

2 (3%)

3 (4%)

46 (68%)

65 (96%)

Number of

Participants

Sep08 - Aug09

54

31 (57%)

0 (0%)

48 (89%)

2 (4%)

3 (6%)

3 (6%)

0 (0%)

5 (9%)

47 (87%)

27 (50%)

31 (57%)

15 (28%)

4 (7%)

3 (6%)

1 (2%)

32 (59%)

53 (98%)

Number of

Participants*

TOTAL

121

53 (44%)

2 (2%)

102 (84%)

13 (11%)

4 (3%)

4 (3%)

0 (0%)

14 (12%)

103 (85%)

51 (42%)

69 (57%)

35 (29%)

7 (6%)

4 (3%)

6 (5%)

74 (61%)

117 (97%)





Table 16: UMass Amherst Participant Background Information (M/S)


Number of

Participants

Number of

Participants

Number of

Participants

Number of

Participants*



In Cohort 3

In previous cohorts




Immersion




Teach Science or




No






Sep06 - Aug07

48

4 (8%)

7 (15%)

37 (77%)

6 (13%)

8 (17%)

2 (4%)

18 (38%)

20 (42%)

22 (46%)

27 (56%)

15 (31%)

2 (4%)

1 (2%)

3 (6%)

24 (50%)

46 (96%)

Sep07 - Aug08

50

11 (22%)

3 (6%)

38 (76%)

5 (10%)

2 (4%)

5 (10%)

0 (0%)

23 (46%)

17 (34%)

20 (40%)

31 (62%)

10 (20%)

2 (4%)

2 (4%)

5 (10%)

32 (64%)

41 (82%)

Sep08 - Aug09

53

TOTAL

135

25 (47%)

7 (13%)

36

20

(27%)

(15%)

43 (81%) 105 (78%)

5 (9%)

3 (6%)

10 (19%)

3 (6%)

21 (40%)

13 (10%)

5 (4%)

20 (15%)

5 (4%)

55 (41%)

16 (30%)

25 (47%)

30 (57%)

16 (30%)

4 (8%)

0 (0%)

3 (6%)

45

61 (45%)

79 (59%)

38

2

2

14

(33%)

(28%)

(2%)

(2%)

(10%)

28 (53%) 73 (54%)

49 (93%) 121 (90%)





Table 17: Salem State College Participant Background Information (M)




In Cohort 3

In previous cohorts




Immersion




Teach Science or




No






Number of

Participants

Sep06 - Aug07

50

9 (18%)

17 (34%)

38 (76%)

9 (18%)

7 14%)

4 (8%)

35 (70%)

8 (16%)

41 (82%)

33 (66%)

11 (22%)

0 (0%)

2 (4%)

4 (8%)

26 (52%)

47 (94%)

Number of

Participants

Sep07 - Aug08

40

Number of

Participants

Sep08 - Aug09

N/A

16 (40%) N/A

0 (0%) N/A

27 (68%) N/A

6 (15%) N/A

2 (5%) N/A

1 (3%) N/A

0 (0%) N/A

31 (78%) N/A

5 (13%) N/A

26 (65%) N/A

21 (53%) N/A

10 (25%) N/A

1 (3%) N/A

3 (8%) N/A

5 (13%) N/A

22 (55%) N/A

36 (90%) N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

Number of

Participants*

TOTAL

78

26 (33%)

21 (27%)

54 (69%)

14 (18%)

2 (3%)

5 (6%)

3 (4%)

55 (71%)

11 (14%)

60 (77%)

43 (55%)

20 (26%)

1 (1%)

5 (6%)

9 (12%)

40 (51%)

71 (91%)





Table 18: SE/Cape Participant Background Information (S)


Number of

Participants

Sep06 - Aug07

47

Number of

Participants

Sep07 - Aug08

78


Number of

Participants

Sep08 - Aug09

90

Number of

Participants*

TOTAL

178


In Cohort 3

In previous cohorts




Immersion




Teach Science or




No





7 (15%)

0 (0%)

42 (89%)

4 (9%)

20 (43%)

8 (17%)

5 (11%)

22 (47%)

31 (66%)

19 (40%)

22 (47%)

6 (13%)

0 (0%)

0 (0%)

22 (47%)

35 (45%)

1 (1%)

67 (86%)

2 (3%)

4 (5%)

23 (30%)

8 (10%)

11 (14%)

51 (65%)

40 (51%)

39 (50%)

28 (36%)

4 (5%)

3 (4%)

4 (5%)

42 (54%)

36 (40%) 69 (39%)

1 (1%) 2 (1%)

74 (82%) 151 (85%)

3 (3%)

4 (4%)

27 (30%)

9 (10%)

6 (7%)

45 (50%)

45 (50%)

53 (59%)

19 (21%)

6 (7%)

6 (7%)

6 (7%)

51 (57%)

9 (5%)

7 (4%)

43 (24%)

17 (10%)

18 (10%)

95 (53%)

92 (52%)

89 (50%)

65 (37%)

5 (3%)

9 (5%)

10 (6%)

92 (52%)

Hold One or More Teaching Licenses 45 (96%) 76 (97%) 89 (99%) 173 (97%)





Table 19: WPI Participant Background Information (S)


Number of

Participants

Number of

Participants

Number of

Participants



In Cohort 3

In previous cohorts




Immersion




Teach Science or




No






Sep06 - Aug07

19

0 (0%)

0 (0%)

16 (84%)

0 (0%)

4 (21%)

0 (11%)

2 (0%)

14 (74%)

3 (16%)

5 (26%)

10 (53%)

0 (0%)

1 (5%)

3 (16%)

5 (26%)

17 (89%)

Sep07 - Aug08

13

2 (15%)

1 (8%)

10 (77%)

1 (8%)

0 (0%)

1 (8%)

0 (0%)

2 (15%)

10 (77%)

7 (54%)

6 (46%)

4 (31%)

1 (8%)

2 (15%)

0 (0%)

5 (39%)

12 (92%)

Sep08 - Aug09

18

4 (22%)

0 (0%)

16 (89%)

1 (6%)

0 (0%)

1 (6%)

0 (0%)

1 (6%)

11 (61%)

4 (22%)

4 (22%)

7 (39%)

2 (11%)

1 (6%)

4 (22%)

6 (33%)

17 (94%)

Number of

Participants*

TOTAL

44

5 (11%)

0 (0%)

38 (86%)

2 (5%)

0 (0%)

5 (11%)

0 (0%)

5 (11%)

30 (68%)

16 (36%)

14 (32%)

20 (46%)

1 (2%)

2 (5%)

7 (16%)

15 (34%)

41 (93%)





Table 20: Worcester Public Schools Participant Background

Information (M)





Teach ELL, ESL, or Sheltered English Immersion

In Cohort 3

In previous cohorts





Teach in High Need District


No







Sep06-Aug07

41

30

5

1

3

34

17

16

2

1

5

16

38

6

0

29

5

2

(15%)

0%

(71%)

(12%)

(5%)

(73%)

(12%)

(2%)

(7%)

(83%)

(42%)

(39%)

(5%)

(2%)

(12%)

(39%)

(93%)





Cohort 4 Activity: September 1, 2008 through August 31, 2009

Cohort 4, which began in the 2008-2009 funding period, consisted of eight partnerships. Table 21 shows the funding received by Cohort 4 partnerships for the period beginning in September 2008 and ending in August

2009.

Table 21: Budgets: Cohort 4

Partnerships

Partnership

Boston Public Schools (M)

Brockton Public Schools (M)

Gateway Regional School Dist. (S)

Lesley Springfield (S)

Greater North Shore (S)

Randolph Public Schools (S)

Springfield College (S)

Boston University (M)

TOTAL

State-level Participant Background Data

Sep08-Aug09

$157,975

$180,145

$186,609

$228,593

$265,917

$176,993

$161,062

$241,586

$1,598,880

Cohort 4 consisted of eight partnerships, with four of the eight partnerships offering mathematics professional development and four offering science professional development. There were 27 Cohort 4 courses delivered. Of these 27 courses, 13 were mathematics courses and 14 were science courses. Of those 27 courses, 13 (48%) were unique, and 14 (52%) were repeat offerings. This section of the report summarizes data collected from participants in these courses.

During this first year of funding for Cohort 4, there were 341 participants, and 46 of them took two or more courses. By the end of the 2008-2009 funding period, 341 unique participants completed the Participant

Background Survey on one or more occasions. The term “unique participant” refers to each individual who participated in the program, regardless of how many courses he or she took. Data for items from the survey that help to convey participants’ professional backgrounds and motives for participation are discussed in the remainder of this section. All survey data for this group may be found in Appendix C. The responses to the survey questions are presented as frequencies and percentages. Not all percentages total 100% because many items allowed multiple responses and not all of the participants responded to all of the items.


At the time of their last completed survey from a Cohort 4 MMSP course, 98% of course participants identified themselves as teachers. Of all respondents, 80% were regular education teachers; 15% were special education or special education inclusion teachers; 3% were ELL teachers; 2% were long-term substitutes; 1% were paraprofessionals; <1% were principals, assistant principals, or headmasters; <1% were support specialists; and

3% indicated that they held “other” positions.




Content Taught


The distribution of teaching areas of respondents at the time of the survey is shown in Table 22. Because respondents identified all teaching areas that applied to their positions at the time of the survey, some selected multiple responses, so frequency totals exceed the number of respondents and percentages exceed 100%.

At the time of their last MMSP course, 30% of Cohort 4 participants were teaching mathematics, 41% were teaching science, and 26% were teaching all subjects at the elementary level.

Table 22: Teaching Areas: Cohort 4 Participants

Teaching Areas


n

Sep08-Aug09

N =341

%

Mathematics

Any science area

General Science

Biology

Earth Science

Chemistry

Physics




Other

Not Currently Teaching

Teaching Experience of Participants

14

87

21

11

7

102

138

98

27

22

17

17

4

26

6

3

2

30

41

29

8

7

5

5

At the time of their last completed survey from an MMSP course, the teaching experience of the 341 unique

Cohort 4 participants was as follows: 18% were in their first to third year of teaching, 42% had between four and ten years experience in education, 24% had between 11 and 20 years of experience, 15% reported over 20 years of experience, and <1% did not report.

Teaching Levels of Participants

For reporting purposes, schools in the participating districts were organized into categories of elementary schools

(grades K-5), K-8 schools, middle schools (grades 6-8), and high schools (grades 9-12). At the time of their last completed survey from an MMSP course, 30% of Cohort 4 participants were teaching in an elementary or K-8 school, 51% were teaching in a middle school, 16% were teaching in a high school, and less than 1% were teaching at the pre-K level. Those remaining either were not currently teaching or the level at which they taught was unknown.


As shown in Table 23, 98% of unique Cohort 4 participants worked in a public school setting, and 2% worked in a non-public school setting.




Table 23: Types of Schools of Unique Participants: Cohort 4 Participants

Sep08-Aug09

School Type

Public School (includes public charter schools)

Non-public School

Other or No Response

TOTAL


N

333

7

1

341

%

98

2

<1

100

MMSP partnerships were required to include at least one high need district. Appendix D identifies the criteria for the high need designation.

The ESE expected that at least 50% of participants in each partnership would come from high need districts, and further, they set an informal goal that at least 75% of participants for each partnership would come from high need districts. Of all Cohort 4 participants, 63% came from high need districts. Table 24 shows that by the end of the

2008-2009 funding period, 64% of Cohort 4 participants from public schools had come from high need districts.

Table 24: High Need Status of Unique Participants from Public Schools: Cohort

4 Participants

Sep08-Aug09

School Type

N %

High Need District

Non-high Need District

TOTAL

213

120

333

64

36

100

Table 25 presents the number of participants from high need districts organized by each partnership. An examination of high need district participation in individual partnerships reveals that over the course of each partnership’s involvement in MMSP, five of the eight partnerships had at least 50% of their participants coming from high need districts and three of eight partnerships exceeded the informal goal of having at least 75% of the participants come from high need districts.

Table 25: High Need District Participants by

Partnership: Cohort 4 Participants

Partnership

Boston PS (M)

Brockton PS (M)

High Need

District

Boston

Brockton

Fall River

Subtotal

Sep08-Aug09 n*

40

21

9

30

%

100%

59%




Table 25: High Need District Participants by

Partnership: Cohort 4 Participants

Partnership

High Need

District

Sep08-Aug09

Gateway RSD (S) Easthampton

Gateway

Holyoke

Springfield

Subtotal

Chicopee

Holyoke

Springfield

1

6

2

8

17 39%

Lesley Springfield (M)

Westfield

Subtotal

Boston

1

9

35

4


49

26

3

94%

Lynn

Pioneer Charter

School of Science

1

Randolph PS (S)

Springfield Coll. (S)

Randolph

Subtotal

Randolph

Springfield

Boston

Falmouth

Haverhill

Lawrence

1

31

6

26

3

50%

25%

96%


2

2

7

1 Salem

Worcester 1

Subtotal 16 39%

*Note that if this column is summed, the total will not correspond to relevant data in Table 4 for two reasons: 1) Table 4 presents data for unique participants in public schools across all partnerships while

Table 5 presents data for all unique participants only within partnerships (and some participants took courses through multiple partnerships) and 2) The districts of some of those participants who crossed partnerships were not consistently classified as high need districts because of the content of the course.


For each course taken, respondents identified all of their reasons for participating. Unlike in the earlier portions of this report where data were presented for unique participants, data for this topic are presented for all course seats, since responses are uniquely relevant to each distinct course that a participant took. Table 26 presents findings for all 391 seats for all courses taken by Cohort 4 participants during the 2008-2009 funding period.


Cohort 4 partnerships were successful at encouraging and retaining repeat participants. Six of the eight partnerships offered multiple courses. Half of all partnerships had participants who attended more than one course within Cohort 4. Of all unique Cohort 4 participants, 46 attended multiple courses. Table 27 provides details regarding repeat participation, including information on the 20 repeat participants who took courses from partnerships from previous MMSP cohorts. (Table 32 through Table 39 also include data on repeat participation, as does Table 46.)




Table 26: Reasons for Participation: Cohort 4, All Seats



To increase knowledge in content

To obtain graduate credit

To pursue a personal interest

To earn PDPs for recertification

To get an additional license (certification)

To prepare for the Massachusetts Test for Educator Licensure

(MTEL)

To follow an administrator’s suggestion

To obtain a first license (certification)

Other

287

240

118

108

52

Sep08-Aug09

n


73%

61%

30%

28%

13%

45

30

11

28

12%

8%

3%

7%


Table 27: Repeat Participants: Cohort 4 Partnerships

Partnership

Number of

Courses

Offered to Date

2 Boston Public Schools (M)

Brockton Public Schools (M) 3

Gateway Regional Sch Dis (S) 1






5

11

2

2

2

Total

Number of

Unique*


40

Number

Taking

Multiple

Courses in

Cohort 4

0

24

27

41

51

44

52

62

4

1

0

19

22

0

0

Number Taking

One Course in

Cohort 4 & One or

More Courses in

Previous Cohorts

1

0

7

1

1

3

1

6

Number Taking

Multiple Courses in

Cohort 4 & Previous

Cohorts

TOTAL 28 341 46 20 0

* Participants who participated in multiple courses across partnerships were counted only once in the partnership of their most recent course

Attrition

The course attrition rate over all courses over all partnerships averaged 5%. Of the 28 Cohort 4 courses delivered, data on attrition rates were available for 27. Of the 27 courses for which data were available, 20 (74%) had an attrition rate of 0%, three (11%) had an attrition rate ranging between 1% and 10%, and four (15%) had an attrition rate of greater than 10%. Table 28 provides a breakdown, by partnership, of enrollment and attrition rates. Appendix E provides a breakdown, by course, of enrollment and attrition rates.

0

0

0

0

0

0

0

0




Table 28: Total Enrollment and Attrition Information: Cohort 4 Partnerships

Partnership

Number of

Courses

Offered

Number of

Participants

Enrolled First

Day

Number of

Participants

Completed

Course

Attrition Rate

Sep08-Aug09 Sep08-Aug09 Sep08-Aug 09 Sep08-Aug09

Boston PS (M)

Brockton PS (M)

Gateway Regional SD (S)



Randolph PS (S)



2

3

1

5

11

2

2

2

202

52

47

74

95

28

27

69

184

51

47

72

89

28

27

67

All Courses/Partnerships 28 594 565* 5%

* This value exceeds the number of seats reported on page 34 and in Table 26 of this report because one partnership did not administer

Partnership Background Surveys to 144 of its participants.


9%

2%

0%

3%

6%

0%

0%

3%

For Cohort 4 during the 2008-2009 funding period, five unique participants attained highly qualified status. Of the

5 who attained highly qualified status, one did so by passing the appropriate MTEL, one did so by earning a teaching license, and three did so by changing their positions. Additionally, it is likely that MMSP participation contributed to the attainment of highly qualified status for more participants after the 2008-2009 period had ended, but it was not possible to capture those gains during this reporting period.

Of the five who attained highly qualified status, two were from the Greater North Shore Northeastern (S) partnership and one each were from the Lesley (M), Gateway (S) and Randolph (S) partnerships. All but one had begun their participation in MMSP during Cohort 4. The one who had begun participating earlier was from the

Gateway partnership and had begun participating in Cohort 3. An identification of the criteria by which teachers who gained highly qualified status could demonstrate competency in their subject matter may be found in

Appendix F.

Licensure and Degrees Held in Subjects Taught

During the 2008-2009 funding period for Cohort 4, 117 regular education teachers, 15 special education teachers, and six ELL teachers reported teaching subjects in science or technology/engineering. Tables 29a, 29b, and 29c show how many teachers taught each individual science or technology/engineering subject from Cohort 4 during the 2008-2009 funding period. The tables also show the percentages of teachers whose survey responses indicated they were licensed in the subject they taught, and they show the percentages of teachers whose survey responses indicated they held a degree in the subject they taught. Table 29a provides information for regular education teachers, Table 29b provides information for special education teachers, and Table 29c provides information for

ELL teachers. (The numbers in these tables exceed the number of teachers who reported teaching these subjects because some teachers taught more than one subject.) Across all science subject areas for regular education teachers, the licensing reported appeared to be appropriate for approximately 72% of the science subjects taught, and the degrees held corresponded to the science subject taught for approximately 22% of the science subjects taught. Across all science subject areas for special education teachers, the licensing reported appeared to be appropriate for approximately 63% of the science subjects taught, and no teachers held degrees that corresponded



Annual Report of the MMSP State-level Evaluation Cohort 4 Activity to the subject taught. Across all science subject areas for ELL teachers, the licensing reported appeared to be appropriate for 67% of the subjects taught, and no teachers held degrees that corresponded to the subject taught.

Table 29a: Cohort 4 Science and Technology Teaching Areas

– Regular

Education

Subject

Number of

Teachers n

General Science

Biology

81

25

Chemistry

Physics

Earth Science

17

16

20

Technology/Engineering 14

*Of the number of regular education participants teaching this subject for this period

Sep08-Aug09

License in Subject

Taught

%*

91

72

65

44

45

36

Degree in Subject

Taught

%*

15

60

35

6

10

14

Table 29b: Cohort 4 Science and Technology Teaching Areas – Special

Education

Subject

Number of

Teachers n

General Science

Biology

Chemistry

Physics

12

1

0

1

Earth Science


2

0

*Of the number of special education participants teaching this subject for this period

Sep08-Aug09


Taught

%*

58

100

0

100

50

0

Degree in Subject

Taught

%*

0

0

0

0

0

0

Table 29c: Cohort 4 Science and Technology Teaching Areas – ELL Education

Subject

Number of

Teachers n

General Science

Biology

Chemistry

Physics

5

1

0

0

Earth Science 0

Technology/Engineering 0

*Of the number of ELL education participants teaching this subject for this period

Sep08-Aug09


Taught

%*

80

0

0

0

0

0

Degree in Subject

Taught

%*

0

0

0

0

0

0




To see the extent to which licensure and degree were appropriate for a particular type of teacher for a particular subject, look in the relevant table at data in the row corresponding to the subject of interest. For example, to see the extent to which licensure and degree were appropriate for just regular education biology teachers, look at

Table 29a in the row for the subject area “Biology” to learn that 25 teachers taught regular education biology and that of those 25, 72% were licensed in biology and 60% held degrees in biology.

During the 2008-2009 funding period for Cohort 4, 94 regular education teachers 25 special education teachers, and five ELL teachers reported teaching mathematics. Tables 30a, 30b, and 30c show how many Cohort 4 teachers taught at each mathematics level during the 2008-2009 funding period. The tables also show the percentages of teachers whose survey responses indicated they were licensed in the mathematics level at which they taught, and they show the percentages of teachers whose survey responses indicated they held a degree in mathematics. Table 30a provides information for regular education teachers, Table 30b provides information for special education teachers, and Table 30c provides information for ELL teachers.

From an “ease-of-reading perspective,” it would be ideal merely to report on the percentages of teachers who were licensed to teach mathematics and the percentage of teachers who held mathematics degrees. Sometimes, though, teachers hold multiple positions and may only be licensed to teach some of what they are teaching, and what is really desired is to understand the extent to which what is being taught is done so by individuals licensed to teach it and by individuals who hold relevant degrees. As a result, the explanations of data that follow are done so in terms of teaching positions.

For MMSP mathematics teaching positions in regular education, the licensing appeared to be appropriate for approximately 82% of positions held, and 13% of the positions were held by individuals with mathematics degrees. For special education teachers, the licensing appeared to be appropriate for approximately 56% of the positions held, and 8% of the positions were held by individuals with mathematics degrees. For ELL teachers, the licensing appeared to be appropriate for 60% of the positions held, and no teachers held mathematics degrees.

To see the extent to which licensure and degree were appropriate for a particular type of teacher for a particular mathematics level taught, look in the relevant table at data in the row corresponding to the level of interest and in the column corresponding to the time period of interest. For example, to see the extent to which licensure and degree were appropriate for just regular education middle school math teachers, look at Table 30a in the row for the level “Middle School” to learn that 60 teachers taught regular education middle school mathematics and that of those 60, 87% were licensed to teach middle school mathematics and 8% held mathematics degrees.




Table 30a: Cohort 4 Mathematics Teacher Levels – Regular Education

Sep08-Aug09

Level

Elementary School

Middle School

High School


TOTAL Math

Number of

Teachers n

17

60

17

0

94

Licensed at Level

*Of the number of regular education participants teaching mathematics at this level for this period

%*

65

87

82

0

82


Degree in Math

%*

0

8

41

0

13

Table 30b: Cohort 4 Mathematics Teacher Levels

– Special Education

Sep08-Aug09

Level

Elementary School

Middle School

High School


TOTAL Math

Number of

Teachers n

3

17

5

0

25

Licensed at Level

*Of the number of special education participants teaching mathematics at this level for this period

%*

100

53

40

0

56

Degree in Math

%*

0

12

0

0

8

Table 30c: Cohort 4 Mathematics Teacher Levels – ELL Education

Sep08-Aug09

Level

Elementary School

Middle School

High School


TOTAL Math

Number of

Teachers n

1

4

0

0

5

Licensed at Level

%*

100

50

0

0

60

Degree in Math

%*

0

0

0

0

0

*Of the number of ELL education participants teaching mathematics at this level for this period


Information on degrees currently being pursued in science, technology/engineering, and mathematics was derived from the most recently completed survey of each individual. Of the 117 regular education teachers, 15 special



Annual Report of the MMSP State-level Evaluation Cohort 4 Activity education teachers, and six ELL teachers who reported teaching subjects in science or technology/engineering in

Cohort 4 in the 2008-2009 funding period, 14 reported that they were pursuing degrees in science and technology/engineering. Of these 14 teachers, 12 were pursuing advanced degrees (master’s degrees and CAGS degrees) in their current area of teaching. Four teachers taught multiple science and technology/engineering subjects. Of these, three were pursuing degrees that were relevant to at least one area that they were teaching (two were pursuing master’s degrees and one was pursuing a CAGS degree). One teacher who taught multiple science and technology/engineering subjects was pursuing a master’s degree in a science field not currently taught. Nine teachers who were pursuing a degree in science and technology/engineering taught only general science. Of the nine general science teachers who were pursuing a degree in science and technology/engineering, eight were pursuing a degree in general science (seven were pursuing master’s degrees and one was pursuing a bachelor’s degree and a CAGS degree). One general science teacher was pursuing master’s degree in a science field not currently taught. One biology teacher was pursuing a master’s degree in biology.

Of the 94 regular education teachers 25 special education teachers, and five ELL teachers who reported teaching mathematics in Cohort 4 during the 2008-2009 funding period, five were pursuing mathematics degrees. Of these, four were pursuing advanced degrees (master’s degree or CAGS degree). Three middle school mathematics teachers were pursuing master’s degrees, one elementary school mathematics teacher was pursuing a master’s degree, and one high school mathematics teacher was pursuing a bachelor’s degree.


As a grant condition, partnerships were required to use a pre-course test and post-course test for each MMSP course to assess participants’ knowledge growth of the content taught in the course. For each course, the same instrument served as both the pre- and post-course test. While partnerships were permitted to locate and use an instrument with established reliability and validity, it was most often the case that such instruments were not available because a priority was placed on utilizing assessments that would reflect the precise content taught in each course. As a result, the faculty members who had developed the courses usually developed the assessments, and typically, neither validity nor reliability were determined for them because time and resource constraints prohibited doing so.


Gains in average percentage of items correct between pre- and post-course test administrations occurred in 26 of the 27 courses delivered across all Cohort 4 partnerships for the 2008-2009 funding period, and statistically significant improvements in scores on content knowledge assessments occurred in 63% of all 27 courses. Of the

10 courses not showing statistically significant improvement in scores, though, six had fewer than six participants, the smallest sample size at which it is possible to detect statistical significance at the level used for these analyses.

Table 31 provides an overview, by subject matter of courses delivered, of the numbers of courses that did and did not show statistically significant gains. For detailed information on mean pre- and post-course content knowledge scores by partnership and by course, see Appendix G.




Table 31: Number of Courses with Statistically Significant

Gains in Mean Content Knowledge Scores for Cohort 4

Content Area

Math

Science & Technology/Engineering

TOTAL

Partnership-level Participant Background Data

13

14

27

Sep08-Aug09

Delivered

Significant

Pre/Post

Gains

10

7

17


Presented in Table 32 through Table 39, this section offers an overview of selected participant survey data for each Cohort 4 partnership. These data were collected at the end of each course that was offered by each partnership. The responses to the survey questions are presented as frequencies and percentages of the total number of participants in the partnership for the 2008-2009 period. In cases where not all participants responded to all of the items and in cases where multiple responses were permitted, percentages presented may not total

100%.

The section of each table identified as “Participants Who Took Multiple Courses” contains two categories. Each is defined as follows: “In Cohort 4” refers to participants who took two or more courses within the context of Cohort

4 – they participated in no courses that had been offered during previous cohorts. “In previous cohorts” refers to participants who took one course during Cohort 4 and at least one additional course that had been offered during a previous cohort.

The “Highly Qualified” section of each table includes the response option “Private school/Not applicable.”

Participants categorized under this option refer to those who either were teaching at a private school or for whom highly qualified status would be irrelevant, such as school administrators.




Table 32: Boston Public Schools Participant Background

Information (M)


Sep08-Aug09


Participants Who Took Multiple Courses In Cohort 4

In previous cohorts








40

0

1

24

15

1

20

5

18

4

(50%)

(13%)

(45%)

(10%)



No





40

27

12

1

0

0

24

(100%)

(68%)

(30%)

(3%)

(0%)

(0%)

(60%)

Hold One or More Teaching Licenses 40 (100%)

* This value underreports by 144 the number of participants who completed courses for this partnership. This partnership, though, only administered Partnership Background Surveys to 40 of its 184 participants, and data from those surveys are reported here.

(0%)

(3%)

(60%)

(38%)

(3%)





Table 33: Brockton Public Schools Participant Background

Information (M)





In Cohort 4

In previous cohorts


Sep08-Aug09

51

6

1

38

10

(12%)

(2%)

(75%)

(20%)








No






15

5

30

30

16

2

0

3

29

50

1

29

5

(2%)

(57%)

(10%)

(29%)

(10%)

(59%)

(59%)

(31%)

(4%)

(0%)

(6%)

(57%)

(98%)





Table 34: Gateway Regional School District Participant

Background Information (S)





In Cohort 4

In previous cohorts


Sep08-Aug09

46

2

6

43

2

(4%)

(13%)

94%

4%








No






6

39

17

19

19

6

2

0

20

45

1

4

0

2%

9%

0%

13%

(85%)

(37%)

(41%)

(41%)

(13%)

(4%)

(0%)

(44%)

(98%)





Table 35: Lesley University C4 Participant Background

Information (M)





In Cohort 4

In previous cohorts


Sep08-Aug09

52

22

1

40

10

(42%)

(2%)

(77%)

(19%)








No






28

4

49

28

17

2

4

1

29

49

2

15

7

(4%)

(29%)

(14%)

(54%)

(8%)

(94%)

(54%)

(33%)

(4%)

(8%)

(2%)

(56%)

(94%)





Table 36: Greater North Shore Participant Background

Information (S)





In Cohort 4

In previous cohorts


Sep08-Aug09

62

26

3

54

5

(42%)

(5%)

(87%)

(8%)








No






4

54

31

36

13

8

4

1

36

59

2

2

0

(3%)

(3%)

(0%)

(7%)

(87%)

(50%)

(58%)

(21%)

(13%)

(7%)

(2%)

(58%)

(95%)





Table 37: Randolph Public Schools Participant Background

Information (S)



In previous cohorts










No







Sep08-Aug09

10

2

2

10

6

14

5

3

1

1

15

24

24

5

0

20

3

0

(42%)

(8%)

(8%)

(42%)

(25%)

(58%)

(21%)

(13%)

(4%)

(4%)

(63%)

(100%)

(21%)

(0%)

(83%)

(13%)

(0%)





Table 38: Springfield College Participant Background Information

(S)






In Cohort 4

In previous cohorts








No







Sep08-Aug09

2

19

1

0

26

12

9

3

3

0

12

25

27

3

7

22

4

1

(4%)

(0%)

(7%)

(70%)

(96%)

(44%)

(33%)

(11%)

(11%)

(0%)

(44%)

(93%)

(11%)

(26%)

(82%)

(15%)

(4%)




Table 39: Boston University Participant Background Information

(M)




In previous cohorts










No







Sep08-Aug09

27

5

6

2

16

23

8

1

1

8

26

36

41

0

1

33

3

2

(15%)

(5%)

(66%)

(12%)

(39%)

(56%)

(20%)

(2%)

(2%)

(20%)

(63%)

(88%)

(0%)

(2%)

(81%)

(7%)

(5%)



Annual Report of the MMSP State-level Evaluation Cumulative Summary

Cumulative Summary: All Cohorts over All Funding Periods

Overview of Partnerships, Budgets, Courses, and Participants

Table 40 provides an overview of partnership projects since the inception of the program.

Table 40: Overview of MMSP Partnership Participation

Cohort 1

Cohort 2

MMSP Year 1

Feb04-Aug04

Grant Year 1

EduTron/FitchburgMath

HarvardMath

LesleyMath

MCLAScience

SalemMath

Springfield PSScience

Wareham PSMath

WPIMath

MMSP Year 2

Sep04-Aug05

Grant Year 2

EduTron/Fitchburg-Math

Harvard-Math

Lesley-Math

MCLA-Science

Salem-Math

Springfield PS-Science

Wareham PS-Math

WPI-Math

Grant Year 1

MCLA-Math

PV STEMNET-Math

Cohort 3

Funding Period

MMSP Year 3 MMSP Year 4

Sep05-Aug06

Grant Year 3


Harvard-Math

Lesley-Math

MCLA-Science

Salem-Math

Springfield PS-Science

Wareham PS-Math

WPI-Math

Grant Year 2

MCLA-Math

PV STEMNET-Math

Sep06-Aug07

Grant

Year 3

Extension

Grant Year 3

MCLA-Math

PV STEMNET-Math

Grant Year 1

EduTron/Lowell-Math/Sci


Lesley-Math

North Shore-Science

PV STEMNET-Math/Sci

Salem-Math

SE/Cape-Science

WPI-Science

WPS-Math (discontinued)

MMSP Year 5

Sep07-Aug08

Grant Year 2



Lesley-Math

North Shore-Science

PV STEMNET-Math/Sci

Salem-Math

SE/Cape-Science

WPI-Science

Cohort 4

MMSP Year 6

Sep08-Aug09

Grant Year 3



Lesley-Math

North Shore-Science

PV STEMNET-Math/Sci

Salem-Math

SE/Cape-Science

WPI-Science

Grant Year 1

Boston PS-Math

Brockton PS-Math

Gateway RSD-Science

Lesley-Math

Northeastern-Science

Randolph PS-Science

Springfield Coll.-Science

Boston U. Trustees-Math


Research and Evaluation

Group 50


Cohorts 1, 2, 3, and 4 combined consisted of 27 partnerships, with 16 of the 27 partnerships offering mathematics professional development, nine offering science professional development, and two offering professional development in both mathematics and science content. Specifically, Cohort 1 consisted of eight partnerships, with six of the eight partnerships offering mathematics professional development and two offering science professional development. Cohort 2 consisted of two partnerships, both offering mathematics professional development.

Cohort 3 consisted of nine partnerships, with four of the nine partnerships offering mathematics professional development, three offering science professional development, and two offering professional development in both mathematics and science. Cohort 4 consisted of eight partnerships, with four of the eight partnerships offering mathematics professional development and four offering science professional development.

Table 41 shows the funding received by each partnership since the inception of the program. While some partnerships were awarded funding in more than one funding period, for evaluation purposes, a partnership was identified as a “new” partnership each time it received funding that resulted from a different competition. Overall, partnerships have been awarded a total of $11,280,420 since the inception of MMSP.

Table 41: Budgets: All Partnerships, All Funding Periods

Partnership

Feb04-

Aug06

COHORT 1

Initially funded February 2004

EduTron (M)

Harvard Graduate School of Ed. (M)


MCLA – Science (S)

Salem State College (M)

Springfield Public Schools (S)

Wareham Public Schools (M)

Worcester Polytechnic Institute (M)

COHORT 2

Initially funded September 2004

MCLA – Math (M)

UMass Amherst (M)

COHORT 3





North Shore (S)

UMass Amherst (M/S)

Salem State College (M)

SE/Cape (S)

Worcester Polytechnic Institute (S)

Worcester Public Schools (M)

$111,494

$262,415

$770,000

$489,899

$810,726

$133,192

$541,995

$500,044

$398,440

$601,778

Sep06-

Aug07

$68,352

$87,425

$43,838

$38,247

$43,648

$74,737

$43,962

$35,633

$51,874

$181,581

$210,000

$102,000

$347,911

$196,474

$107,424

$120,882

$129,438

$99,586

$231,210

Sep07-

Aug08

$220,000

$110,000

$355,626

$194,729

$216,281

$113,551

$181,420

$70,734

Sep08-

Aug09

$240,000

$120,000

$355,357

$199,871

$169,064

$36,604

$169,246

$94,852

TOTAL

$838,352

$577,324

$854,564

$171,439

$585,643

$574,781

$442,402

$637,411

$163,368

$443,996

$670,000

$332,000

$1,058,894

$591,074

$492,769

$271,037

$480,104

$265,172

$231,210




Table 41: Budgets: All Partnerships, All Funding Periods

Cumulative Summary

Partnership

Feb04-

Aug06

Sep06-

Aug07

Sep07-

Aug08

Sep08-

Aug09

TOTAL

COHORT 4


Boston Public Schools (M)

Brockton Public Schools (M)

Gateway Regional School Dist (S)






$265,917

$161,062

$176,993

$186,609

$157,975

$228,593

$180,145

$241,586

$265,917

$161,062

$176,993

$186,609

$157,975

$228,593

$180,145

$241,586

TOTAL $4,619,983 $2,214,222 $1,462,341 $2,983,874 $11,280,420

Over the span of MMSP, Cohort 1, 2, 3, and 4 partnerships developed and implemented a total of 252 courses. Of those 252 courses, 137 (54%) were unique, and 115 (46%) were repeat offerings. Of the 252 courses, 169 (67%) of the courses offered mathematics content, 81 (32%) offered science and/or technology/engineering content, and two (1%) offered both mathematics and science/technology content. This section of the report summarizes data collected from participants in these courses. In total, there were 2,262 unique participants, and 811 of them took two or more courses. The term “unique participant” refers to each individual who participated in the program, regardless of how many courses he or she took. Data for items from the Participant Background Survey that help to convey participants’ professional backgrounds and motives for participation are discussed in the remainder of this section. In addition, this section discusses information obtained from partnerships on the extent to which courses offered through MMSP became institutionalized.


At the time of their last completed survey from an MMSP course, 91% of course participants identified themselves as teachers. Of all respondents, 76% were regular education teachers; 13% were special education or special education inclusion teachers; 2% were ELL, ESL, or Sheltered English Immersion teachers; 2% were department heads or curriculum coordinators; 2% were principals, assistant principals, or headmasters; 1% were support specialists; 1% were paraprofessionals; 1% were long-term substitutes; <1% were superintendents or assistant superintendents; and 3% indicated that they held “other” positions.

Content Taught

The distribution of teaching areas of respondents at the time of the survey is shown in Table 42. Because respondents identified all teaching areas that applied to their positions at the time of the survey, some selected multiple responses, so frequency totals exceed the number of respondents and percentages exceed 100%. Also, figures reported in “Total” column may be smaller than figures for any individual year because the total is based on data from the last survey completed and some repeat participants changed teaching areas over the course of their participation.

At the time of their last MMSP course, 48% were teaching mathematics, 28% were teaching science, and 23% were teaching all subjects at the elementary level.




Table 42: Teaching Areas: All Participants, All Funding Periods


Teaching Areas

(Multiple responses permitted) n

Total

N =2262

%

Mathematics

Any science area

General Science

Biology

Earth Science

Chemistry

Physics




Other

Not Currently Teaching

921

634

403

137

83

98

92

54

525

161

66

196

41%

28%

18%

6%

4%

4%

4%

2%

23%

7%

3%

8%


For each funding period of the program, at least 96% of MMSP participants worked in a public school setting.

Over the course of the program to date, 97% of MMSP participants worked in a public school setting, and 2% worked in a non-public school setting. Table 43 provides a breakdown, by funding period, of the types of schools in which participants worked.

Table 43. Types of Schools: All Unique Participants, All Funding Periods

Feb04Sep04Sep05Sep06-

Aug04 Aug05 Aug06 Aug07

Sep07-

Aug08

Sep08-

Aug09

Total

School Type n

Public Schools

(includes public charter schools)

332 97 448 98 455 98 533

Non-public School

Other or No

Response

8 2 7 2 6 1 12

1 <1 1 <1 3 1 8

TOTAL 341 100 456 100 464 100 553


96 462

2 7

<1 8

100 477

% n

97

2

2

100

%

769 98 2195 97

15 2 47 2

4 <1 20 1

788 100 2262 100

MMSP partnerships were required to include at least one high need district. Appendix D identifies the criteria for the high need designation. In addition, the ESE expected that at least 50% of participants in each partnership would come from high need districts, and further, they set an informal goal that at least 75% of participants for each partnership would come from high need districts. The high need status of some school districts changed across years. To classify participants from districts that were high need at one point in time but not high need at another, a process was used that took into account the high need status of participants’ districts from the



Annual Report of the MMSP State-level Evaluation Cumulative Summary beginning of each partnership’s MMSP involvement with the program. A district identified as high need in the first year of a partnership’s funding continued to be classified as high need in subsequent years of the partnership, even if the district's status changed. Additionally, any districts not on the high need list in the first year of a partnership’s funding but subsequently added to the high need list in later years of the partnership were then identified as qualifying for high need district designation.

As a whole, across all years of funding, 65% of participants were from high need districts. Table 44 shows that across all years of funding, 67% of the public school participants in the program as a whole had come from high need districts and that for each year of funding, over 50% of public school participants in the program had come from high need districts. An examination of high need district participation in individual partnerships reveals that over the course of each partnership’s involvement in MMSP since the beginning of MMSP, 18 of the 27 partnerships had at least 50% of their participants coming from high need districts across all years of their involvement, and (again, across all years of their involvement) 12 of 27 partnerships exceeded the informal goal of having at least 75% of the participants come from high need districts. If individual years of participation are examined, it is seen that 15 of the 27 partnerships had at least 50% of their participants coming from high need districts for each and every year of funding, and 10 of the 27 partnerships exceeded the informal goal of having at least 75% of the participants come from high need districts for each and every year of funding. The table in

Appendix H shows the number of participants from high need districts organized by partnership.

As of their last course in MMSP, of the 811 individuals who took multiple courses, approximately 70% were from high need public school districts, approximately 27% were from other public school districts, and approximately

2% either were from private schools or did not provide information on their districts. In addition, less than 1% were from high need districts for some of the courses they took but not for others 2 .

Table 44: High Need Status of All Unique Participants from Public Schools

School Type

Feb04Sep04Sep05Sep06-

Aug04 Aug05 Aug06 Aug07 n

202 61 276 62

Sep07-

Aug08

Sep08-

Aug09 n

Total

%

254 56 397 74 335 73 544 69 1464 67 High Need District

Non-high Need District 129 39 161 36 196 43 126 24 122 26 240 31 705 32

Other* 1 <1 11 2 5 1 10 2 5 1 4 <1 26 1

TOTAL 332 100 448 100 455 100 447 100 462 100 788 100 2195 100

*Includes those who did not identify their districts and public school participants who took multiple courses whose districts were considered high need for only some of the courses those participants took.


For each course taken, respondents identified all of their reasons for participating. Unlike in most other portions of this report where data were presented for unique participants, data for this issue are presented for all course seats, since responses are uniquely relevant to each distinct course that a participant took. Table 45 presents findings for all 4,333 seats for all courses taken by all participants across all funding periods.

2 Teachers who took MMSP courses from a math partnership when their districts were considered high need for only science were identified as having come from a “non-high need” district, and teachers who took MMSP courses from a science partnership when their districts were considered high need for only math were identified as having come from a “non-high need” district.




Table 45: Reasons for Participation of Participants: All Seats, All

Funding Periods



To increase knowledge in content n

3126

Total

% of 4,333 course seats

72%

To obtain graduate credit

To earn PDPs for recertification

2948

1400

68%

32%

To pursue a personal interest

To get an additional license (certification)

1366

800

32%

18%

To prepare for the Massachusetts Test for Educator Licensure

(MTEL)

To earn PDPs for HOUSSE plan requirement

To follow an administrator’s suggestion

731

331

281

17%

8%

6%

To obtain a first license (certification) 131 3%

Other 262 6%


Partnerships were very successful at encouraging and retaining repeat participants. Of the 27 partnerships, all but two offered multiple courses. Of the 25 offering multiple courses, all had participants who attended more than one course. In all, 811 participants (36% of all participants) attended multiple courses. Of those participating in multiple courses, 135 took courses across partnerships. Table 46 provides details regarding repeat participation.

Table 46: Repeat Participants: All Partnerships, All Funding Periods

Partnership

EduTron (M)

Harvard (M)

Lesley Univ. C1

(M)

MCLA (S)

Salem State

College (M)

Springfield PS

(S)

Wareham PS

(M)

Number of

Courses

Delivered to Date

7

8

Total

Number of

Unique


128

166

Number Taking

Multiple

Courses within

Cohorts

1 & 2

36

Number Taking

Multiple

Courses within

Cohort 3

Number Taking

Multiple

Courses within

Cohort 4

N/A N/A

Total Number

Taking

Multiple

Courses

54

24 N/A N/A 26

19

3

26

107

23

162

83

13

59

N/A

N/A

N/A

N/A

N/A

N/A

85

13

81

7

3

96

43

41

12

N/A

N/A

N/A

N/A

43

12




Table 46: Repeat Participants: All Partnerships, All Funding Periods


Partnership

Number of

Courses

Delivered to Date

Total

Number of

Unique


145

16

Number Taking

Multiple

Courses within

Cohorts

1 & 2

47

Number Taking

Multiple

Courses within

Cohort 3

Number Taking

Multiple

Courses within

Cohort 4

N/A N/A

Total Number

Taking

Multiple

Courses

48

9 N/A N/A 9

WPI (M)

MCLA (M)

UMass Amherst

(M)

EduTron Lowell

(M/S)

EduTron

Fitchburg (M)

Lesley Univ. C3

(M)

North Shore (S)

UMass Amherst

C3 (M/S)

Salem State C3

(M)

SE/Cape (S)

WPI (S)

Worcester PS(M)

Boston PS (M)

6

4

11

10

7

40

30

14

8

15

3

3

2

76

166

125

198

121

135

78

178

44

41

40

39

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

34

46

124

53

36

26

69

5

6

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

0

52

34

56

124

52

46

40

70

5

5

1

Brockton PS (M)

Gateway RSD

(S)

Lesley

Springfield (M)

Greater North

Shore (S)

Randolph PS (S)

3

1

5

11

51

46

52

62

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

6

2

22

26

6

9

23

27

2 24 N/A N/A 5 5

Springfield Coll.

(S)

Boston

University (M)

2

2

27

41

N/A

N/A

N/A

N/A

3

0

9

1

Across All

252 2262 386 365 46 811

Partnerships


To comply with the federal No Child Left Behind legislation, public school teachers were required to meet the federal definition of highly qualified by the end of the 2005-2006 school year. One of the expectations of the




MMSP is that providing high quality professional development would contribute to teachers’ attainment of federal highly qualified status.

Information regarding the following areas was used to determine participants’ highly qualified status: licensure, years in education, subject areas taught, High Objective Uniform State Standard of Evaluation (HOUSSE) plans held, Professional Development Points (PDPs) held, MTELs passed, degrees held, undergraduate degree equivalents, and advanced or national certifications.

To be considered highly qualified, a teacher must be licensed and demonstrate subject matter competency in the areas of teaching. Demonstration of subject matter competency for elementary teachers was satisfied either by passing the appropriate MTEL (general curriculum MTEL if teaching multiple core subjects or elementary math

MTEL if teaching mainly elementary math) or by having a HOUSSE plan and having earned a particular number of PDPs. (The HOUSSE plan option was phased out in 2007.) Demonstration of subject matter competency for middle and secondary school teachers was satisfied by one of the following means: passing the appropriate

MTEL, completing an appropriate undergraduate major or graduate degree, completing appropriate coursework comparable to an undergraduate major, holding advanced or national certification in the appropriate subject area, or by having a HOUSSE plan and having earned a particular number of PDPs. For all teachers with HOUSSE plans, the minimum numbers of PDPs needed varied in relation to the date of June 30, 2006: Prior to June 30,

2006, the minimum was 48, and following June 30, 2006, the minimum was 96. Appendix F outlines options available for demonstrating subject matter competency. A participant was identified as highly qualified if the criteria for meeting highly qualified status were met for all subjects that a participant taught. If a participant taught multiple subject areas and only met the highly qualified criteria for some of the subjects taught, he or she was determined to be “highly qualified in some, but not all” content areas.

In the first MMSP funding period, the Participant Background Survey did not adequately capture information about teachers that could be used to determine highly qualified status. The survey was re-designed for Year 2 to capture this information. For the second and subsequent funding periods, though, survey responses indicated that participants misunderstood the meaning of HOUSSE plans, and some who completed more than one survey reported inconsistent data across surveys.

The Participant Background Survey permitted determination of the impact of MMSP courses on highly qualified status only for 1) those participants who held HOUSSE plans and 2) those participants who took more than one course (i.e., completed more than one survey). Because surveys were administered after participants had completed MMSP courses, if a participant completed only one course and did not have a HOUSSE plan it was not possible to determine whether that participant became highly qualified prior to or as a result of MMSP course participation.

Over the span of the program for all Cohorts, at least 139 participants attained highly qualified status. Table 47 presents the highly qualified status of participants across all years of the program. This table provides an unduplicated count of participants. Because the number of courses a participant took was relevant to the process used to determine how many participants attained highly qualified status while participating in MMSP, the findings are organized according to number of courses taken (only one vs. multiple). In Appendix I, a more detailed version of the table is presented in which the data are further broken down by cohort of participation.




Table 47: Highly Qualified Status: All Unique Participants, All Funding

Periods

Status

Took Only One Course

n = 1451

Took Multiple Courses

n = 811

TOTAL

n = 2262

Became Highly Qualified

Became Highly Qualified in only some content areas

Highly Qualified (unable to determine when became HQ)

Highly Qualified in some content areas but not all

(unable to determine when became HQ)

Not Highly Qualified

14

0

732

36

125

0

506

18

139

5

1238

54

419 176 595

Private school or not teaching

Unknown

122 63 185

142 48 190

MTEL Information

One method by which teachers may demonstrate subject-matter competency is to pass the Massachusetts Tests for

Educator Licensure (MTEL) in the content areas that they teach. Table 48 cumulatively identifies the tests taken by participants across all years of the program along with passage rates. Of the 811 participants taking multiple courses, 26% earned one or more new licenses by the end of the sixth year of MMSP, and 12% took and passed an MTEL test. In addition, over 1% of these participants learned that they had passed an MTEL that they had taken during a prior MMSP course.

As shown in Table 48, based on data from the last survey completed by each participant, of the 208 participants who had taken the Mathematics MTEL, 157 (76%) reported passing the test, and 16 (8%) had not yet received their scores at the time of survey completion. Of the 378 respondents who had taken the Middle School

Mathematics MTEL, 326 (86%) passed and 20 (5%) had not yet received their scores. Of the 70 participants who completed the Middle School Mathematics/Science MTEL, 45 (64%) passed and ten (14%) had not yet received their scores. Of the 183 participants completing the General Science MTEL, 160 (87%) passed and 13 (7%) had not yet received their scores. Eighty-one respondents attempted the Biology MTEL, and 72 (89%) passed and two

(3%) had not yet received scores. Of the 49 participants who took the Chemistry MTEL, 42 (86%) passed and two

(4%) had not yet received scores. Of the 18 who took the Earth Science MTEL, 14 (78%) passed and two (11%) had not yet received scores. Thirty participants attempted the Physics MTEL, and 20 (67%) passed while four

(13%) still awaited scores. Ten individuals completed the Technology/Engineering MTEL, and eight (80%) passed. Of the 327 participants who reported taking General Curriculum (formerly elementary) MTEL, 315 (96%) reported passing, and five (2%) had not yet received scores.




Table 48: MTEL Tests Taken by All Participants – Total to Date

Based on each participant’s last survey


Taking Test Passing Test Failing Test n

327 n %

315 96 n %

7 2

Scores

Unknown n %

5 2

General Curriculum (formerly

Elementary)

Early Childhood 24 20 83 2 8 2 8


Mathematics

Middle School Mathematics

Middle School

Mathematics/Science

General Science

Biology

Chemistry

Physics

Earth Science


73

208

378

70

183

81

49

30

18

10

63 86

157 76

326 86

45 64

160 87

72 89

42 86

20 67

14 78

8 80

5 7

35 17

32 9

15 21

10 6

7 9

5 10

6 20

2 11

2 20

5 7

16 8

20 5

10 14

13 7

2 3

2 4

4 13

2 11

0 0

TOTAL in STE Areas 441 361 82 47 11 33 7

Licensure and Degrees Held in Content Area Taught

Over the course of the program, 548 regular education teachers 59 special education teachers, and 20 ELL teachers reported teaching in science or technology/engineering areas. Tables 49a, 49b, and 49c show how many teachers taught in each science and technology/engineering area over the course of the program. The tables also show the percentages of teachers whose survey responses indicated they were licensed in the area in which they taught, and they show the percentages of teachers whose survey responses indicated they held a degree in the area in which they taught. Table 49a provides information for regular education teachers, Table 49b provides information for special education teachers, and Table 49c provides information for ELL teachers. (The numbers presented in Tables 49a, 49b, and 49c exceed the number of teachers who reported teaching in these areas because some teachers taught in more than one area.) For regular education teachers, the licensing reported by approximately 58% appeared to be appropriate for the content area taught, and the degrees held by approximately

23% corresponded to content area taught. For special education teachers, the licensing reported by approximately

35% appeared to be appropriate for the content area taught, and the degree held by one teacher (1%) corresponded to content area taught. For ELL teachers, the licensing reported by approximately 38% appeared to be appropriate for the content area taught, and the degrees held by 19% corresponded to content area taught.




Table 49a: Science and Tech/Engineering Teaching Areas of All

Participants – Regular Education

Content Area

General Science

Biology

Chemistry

Earth Science

Physics


Teach in Area n

337

114

90

68

84

52

*Of the number of regular education participants teaching in this area across all program years

Total

License in Area

Taught

%*

66%

73%

56%

31%

44%

42%


Degree in Area

Taught

%*

13%

61%

34%

12%

17%

12%

Table 49b: Science and Tech/Engineering Teaching Areas of All

Participants – Special Education

Content Area

General Science

Biology

Chemistry

Earth Science

Physics


Teach in Area n

44

16

5

12

4

0

*Of the number of special education participants teaching in this area across all program years

Total

License in Area

Taught

%*

32%

25%

60%

33%

75%

0%

Degree in Area

Taught

%*

2%

0%

0%

0%

0%

0%

Table 49c: Science and Tech/Engineering Teaching Areas of All

Participants – ELL Education

Content Area

General Science

Biology

Chemistry

Earth Science

Physics


Teach in Area n

17

5

3

3

2

2

*Of the number of ELL participants teaching in this area across all program years

Total

License in Area

Taught

%*

53%

20%

33%

0%

0%

50%

Degree in Area

Taught

%*

6%

40%

33%

33%

50%

0%




Over the course of the program and by the end of the 2008-2009 funding period, 758 regular education teachers,

130 special education teachers, and 21 ELL teachers reported teaching non-elementary level mathematics. Tables

50a, 50b, and 50c show how many teachers taught at each non-elementary mathematics level over the course of the program. The tables also show the percentages of teachers whose survey responses indicated they were licensed in the level at which they taught, and they show the percentages of teachers whose survey responses indicated they held a degree in the area in which they taught. Table 50a provides information for regular education teachers, Table 50b provides information for special education teachers, and Table 50c provides information for ELL teachers. For regular education teachers, the licensing reported by approximately 56% appeared to be appropriate for the mathematics level taught, and 19% held mathematics degrees. For special education teachers, the licensing reported by approximately 15% appeared to be appropriate for the level taught, and four teachers (3%) held mathematics degrees. For ELL teachers, the licensing reported by two teachers (10%) appeared to be appropriate for the level taught, and two teachers (10%) held mathematics degrees.

Table 50a: Mathematics Teacher Levels of All Participants – Regular

Education

Level

Middle School

High School

MS & HS grades

Teach in Area n

632

117

9

TOTAL Math 758

*Of the number of regular education participants teaching in this area for this year

Total

License in Area

Taught

%*

55%

61%

44%

56%

Degree in Math

%*

15%

43%

22%

19%

Table 50b: Mathematics Teacher Levels of All Participants – Special

Education

Level

Middle School

High School

MS & HS grades

Teach in Area n

105

22

3

TOTAL Math 130

*Of the number of special education participants teaching in this area for this year

Total

License in Area

Taught

%*

15%

14%

0%

15%

Degree in Math

%*

4%

0%

0%

3%

Table 50c: Mathematics Teacher Levels of All Participants – ELL Education

Level

Teach in Area n

Middle School

High School

18

3

MS & HS grades 0

TOTAL Math 21

*Of the number of ELL participants teaching in this area for this year

Total

License in Area

Taught

%*

11%

0%

0%

10%

Degree in Math

%*

0%

67%

0%

10%






Information on degrees currently being pursued in science, technology/engineering, and mathematics was derived from the most recently completed survey of each individual. Of the 548 regular education teachers, 59 special education teachers, and 20 ELL teachers reported teaching in science or technology/engineering areas, 49 science and technology/engineering teachers reported that they were pursuing degrees in science and technology/ engineering. Of these 49 teachers, 42 were pursuing science degrees in their current areas of teaching. Forty science and technology/engineering teachers were pursuing advanced degrees (master’s degrees and CAGS degrees) in their current area of teaching.

Eleven teachers taught multiple science and technology/engineering subjects. Of these, seven were pursuing degrees that were relevant to at least one area that they were teaching (four were pursuing master’s degrees, two were pursuing CAGS degrees, and one was pursuing a bachelor’s and a master’s degree). Two of the teachers who taught multiple science and technology/engineering subjects were pursuing degrees in an area they taught and in an other science field (one was pursuing a master’s degree and one was pursuing a bachelor’s degree). Two teachers who taught multiple science and technology/engineering subjects were pursuing master’s degrees in science fields they did not currently teach. Twenty-six teachers who were pursuing a degree in science and technology/engineering taught general science. Of the 26 general science teachers who were pursuing a degree in science and technology/engineering, 22 were pursuing a degree in general science (19 were pursuing master’s degrees, one was pursuing a bachelor’s degree, one was pursuing a master’s degree and a CAGS degree, and one was pursuing a bachelor’s degree and a CAGS degree). One general science teacher was pursuing a master’s degree in general science and in another science field not currently taught. Three general science teachers were pursuing master’s degrees in science fields they did not currently teach. Three chemistry teachers were pursuing degrees in science and technology/engineering. Of these three, two were pursuing master’s degrees in chemistry and one was pursuing a master’s degree in another science field. Of five physics teachers who were pursuing degrees in science and technology/engineering, four were pursuing their master’s degree in physics and one was pursuing a master’s degree in physics and an additional science field not currently taught. Two biology teachers were pursuing master’s degrees in biology. Two earth science teachers were pursuing master’s degrees in science fields they did not currently teach.

Of the 758 regular education teachers, 130 special education teachers, and 21 ELL teachers reported teaching mathematics over the course of the program, 24 were pursuing degrees in mathematics. Of these, 22 were pursuing advanced degrees (master’s degree or CAGS degree). Of the 24 mathematics teachers pursuing a degree in mathematics, 14 taught mathematics at the middle school level. Thirteen of the middle school mathematics teachers were pursuing master’s degrees and one was pursuing both a master’s degree and a CAGS degree. Seven of the mathematics teachers pursuing mathematics degrees taught at the high school level. Five of the high school math teachers were pursuing a master’s degree, one was pursuing a CAGS degree, and one was pursuing a bachelor’s degree. Two mathematics teachers who were pursuing mathematics degrees taught at the elementary level. One elementary school teacher was pursuing a master’s degree and one elementary school teacher was pursuing a bachelor’s degree. One mathematics teacher who was pursuing a mathematics degree taught both middle school and high school level math and was pursuing a master’s degree.


As a grant condition, partnerships were required to use a pre-course test and post-course test for each MMSP course to assess participants’ knowledge growth of the content taught in the course. For each course, the same instrument served as both the pre- and post-course test. While partnerships were permitted to locate and use an instrument with established reliability and validity, it was most often the case that such instruments were not available because a priority was placed on utilizing assessments that would reflect the precise content taught in each course. As a result, the faculty members who had developed the courses usually developed the assessments,



Annual Report of the MMSP State-level Evaluation Cumulative Summary and typically neither validity nor reliability was determined for them because time and resource constraints prohibited doing so.


Of the 252 courses that were delivered across all partnerships through the 2008-2009 funding period, content assessments were administered for 249 courses. Changes in average percentage of items correct between pre- and post-course test administrations could be measured in 248 of these courses. (One course had only one participant and therefore it was not appropriate to report the data.) Of these 248 courses, 246 had gains in the average percentage of items correct between pre- and post- test administrations. Statistically significant improvements in scores on content knowledge assessments occurred in 86% of the 248 courses. Of the 35 courses not showing statistically significant improvement in scores, though, 21 had fewer than six participants, the smallest sample size at which it is possible to detect statistical significance at the level used for these analyses. Table 51 provides an overview, by subject matter of courses delivered, of the numbers of courses that showed statistically significant gains. For detailed information on mean pre- and post-course content knowledge scores by partnership and by course, see Appendix G.

Table 51: Number of Courses with Statistically Significant

Gains in Mean Content Knowledge Scores

Content Area

Math

Science & Technology/Engineering

Math and Science

Total


166*

81

2

Significant

Pre/Post

Gains

152

59

2

TOTAL 249* 213

*Two partnerships offering math courses did not administer pre- or post-course content knowledge assessments for three courses; those three courses were excluded from this figure.

Course Institutionalization

For systemic change to occur at the higher education institutions, departments of arts and sciences and education departments are encouraged to work together through MMSP to support stronger content courses in mathematics and science for teacher preparation, undergraduate and graduate degree requirements, and for in-service teachers pursuing graduate-level content courses for recertification. Integration of Title IIB courses into graduate programs at Institutes of Higher Education ensures sustainability over time. The intent behind encouraging the partnerships is that the faculty from the Arts and Sciences Departments bring strong content expertise to the partnership table.

This integration creates greater opportunities for participants to complete coursework leading to a content-area degree and/or to licensure along with the highly qualified designation.

During the three most recent funding periods—2006-2007, 2007-2008, and 2008-2009—partnerships were asked to describe activities during those funding periods that were related to the institutionalization of their courses.

Many partnerships evinced integration, plans for future integration, or—in the case of partnerships with



Annual Report of the MMSP State-level Evaluation Cumulative Summary previously established involvement with MMSP—work toward sustaining prior integration. As would be expected in a program involving partnerships with diverse structures and styles, the extent and type of integration varied across partnerships. To convey a sense of how integration occurred, following are significant activities, grouped according to partnership:

EduTron Lowell Public Schools (M/S) and EduTron Fitchburg State College (M)



Two developmental courses, based on the EduTron model for MMSP courses, will continue to be offered at Fitchburg State College (FSC). EduTron partners supported FSC in designing three pre-service courses that are optimized for education majors. EduTron has begun working with FSC to help FSC apply the

EduTron model used in MMSP math courses to science courses.



FSC has partnered with Lowell Public Schools to offer a teacher certification/CAGS program.



Six mathematics and four science courses were approved by FSC as offerings at the continuing education level.

Lesley University C3 (M )

 Two courses created through Lesley University’s MMSP in 2007-2008 are now offered to Lesley’s on-

 campus pre-service teachers.

Efforts through MMSP contributed to the development of an online Mathematics Education program at

Lesley leading to the Master of Arts degree for elementary and middle school teachers.



Nine math content courses were developed through participation in the MMSP program in 2007-2008 and in prior years. All of these courses are part of Lesley University’s mathematics major for undergraduates, which would not have been possible without the MMSP program.

North Shore (S)



As a result of their joint involvement in MMSP through the North Shore partnership and the National

Science Foundation MSP program, Northeastern University has institutionalized all MMSP courses. Ten

MMSP courses can be used to fulfill degree requirements toward a Master’s in Education for Middle

School Science. In addition, this degree was developed as a result of these courses.

UMass Amherst C3 (M/S)



Four courses developed through the UMass Amherst partnership were approved for graduate level credit.

Salem State C3 (M)

 Salem State College offers courses developed through MMSP as part of a master’s level teaching program in middle school mathematics. All courses developed by Salem State College through MMSP can be applied towards earning a degree through that program.

Southeast/Cape (S )



Participants of the three courses offered through the SE/Cape partnership may apply credit for the courses towards the Master of Arts in Teaching in Physical Science program that is offered through Bridgewater

State College.

Worcester Polytechnic Institute (S)



A Master of Science Education program was created through the physics department at Worcester

Polytechnic Institute, and the MMSP course that was offered through the WPI-Science partnership will serve as the model for instruction of future courses that will be offered.

Worcester Public Schools (M)



As a result of the experience of working with Worcester Public Schools on MMSP, Clark University has expressed interest in exploring the institutionalization of courses that were offered through MMSP.




Springfield College (S)



Springfield College has incorporated into its pre-service Best Practices of Teaching Science course activities from an MMSP course that are designed to help teachers understand how to change misconceptions that students have about life science.



Annual Report of the MMSP State-level Evaluation Summary of Findings

Summary of Findings

The MMSP partnership activities summarized in this section of the report occurred between February 2, 2004, and August 31, 2009. This period spans the beginning of the MMSP program through the end of the 2008-2009 funding period. Since MMSP began in February 2004, progress has been made towards meeting the goals of the program as evidenced by the following data for both the program as a whole, since its inception, and for both

Cohort 3 and Cohort 4.

Cumulative Findings

Overview of Partnerships



A total of 27 partnerships were funded across the Commonwealth. Of these, 16 were organized around mathematical content, nine were organized around science content, and two were organized around both mathematical and science content. Of the 27 MMSP partnerships, all delivered courses, all but two offered multiple courses, and all had participants who attended more than one course.



Cohort 1, which began in February 2004, consisted of eight partnerships, with six of the eight partnerships offering mathematics professional development and two offering science professional development.



Cohort 2, which began in September 2004, consisted of two partnerships, both offering mathematics professional development.

 Cohort 3, which began in September 2006, consisted of nine partnerships, with five of the nine partnerships offering mathematics professional development, three offering science professional development, and one offering professional development in both mathematics and science content.

 Cohort 4, which began in September 2008, consisted of eight partnerships, with four of the eight partnerships offering mathematics professional development and four offering science professional development.

Overview of Courses



In total, 252 MMSP courses were delivered by the end of Year 6 of MMSP funding. Of these 252 courses, 169 were mathematics courses, 81 were science and/or technology/engineering courses, and two were courses offering both mathematics and science content.

Overview of Participants



In total, 2,262 unique participants participated in MMSP courses by the end of Year 6.

 811 participants (36% of all participants) attended multiple courses. Of the 811 participating in multiple courses, 135 took courses across partnerships.



4,333 course seats were filled by all participants across all funding periods.



Types of Schools of Participants

 Of all 2,262 unique participants, 97% came from public schools (including public charter schools),

2% came from non-public schools, and 1% did not indicate their school type.






High Need Status of Districts of Participants

 The partnerships exceeded the ESE target of having at least 50% of all participants come from high need districts, with 65% of all participants in the program coming from high need districts.



Across all years of their involvement, 18 of the 27 partnerships had at least 50% of their participants coming from high need districts.

 For each and every individual year of funding, 15 of the 27 partnerships had at least 50% of their participants coming from high need districts.



Across all years of their involvement, 12 of 27 partnerships had at least 75% of the participants come from high need districts.

 For each and every individual year of funding, ten of the 27 partnerships had at least 75% of the participants come from high need districts.

As the following data reveal, not all MMSP participants were licensed in their teaching areas or held relevant degrees, indicating that course participants were, in fact, those in need of the courses:



Licensure in Mathematics and Science Content Areas for Middle and High School Teachers

 Of the regular education mathematics courses taught by MMSP teachers, 56% were taught by teachers who were licensed in mathematics.



Of the special education mathematics courses taught by MMSP teachers, 15% were taught by teachers who were licensed in mathematics.

 Of the ELL mathematics courses taught by MMSP teachers, 10% were taught by teachers who were licensed in mathematics.



Of regular education science or technology/engineering courses taught by MMSP teachers, 58% were taught by teachers who were licensed in the subject of the course.

 Of special education science or technology/engineering courses taught by MMSP teachers, 35% were taught by teachers who were licensed in the subject of the course.



Of ELL science or technology/engineering courses taught to students of MMSP teachers, 38% were taught by teachers who were licensed in the subject of the course.



Degrees Held in Content Area in which Teaching



Of regular education mathematics courses taught by MMSP teachers, 19% were taught by teachers who held mathematics degrees.

 Of special education mathematics courses taught by MMSP teachers, 3% were taught by teachers who held mathematics degrees.



Of ELL mathematics courses taught to students of MMSP teachers, 10% were taught by teachers who held mathematics degrees.

 Of regular education science or technology/engineering courses taught by MMSP teachers, 23% were taught by teachers who held degrees that were relevant to the focus of the course.






Of special education science or technology/engineering courses taught by MMSP teachers, 1% were taught by teachers who held degrees that were relevant to the focus of the course.

 Of ELL science or technology/engineering courses taught by MMSP teachers, 19% were taught by teachers who held degrees that were relevant to the focus of the course.


MMSP has contributed to increasing the numbers of highly qualified public school teachers.



Attaining Highly Qualified Status



By the end of Year 6, of the participants who had entered MMSP as not highly qualified, 139 had attained highly qualified status.



New Licensures



Of the 811 participants taking multiple courses, 26% earned one or more new licenses by the end of the Year 6 of MMSP, and 12% took and passed an MTEL test.

Cohort 3 Findings




Nine Cohort 3 partnerships were funded through the 2008-2009 funding period. Of these, four were organized around mathematical content, three were organized around science content, and two were organized around both mathematical and science content. Of the nine Cohort 3 MMSP partnerships, all delivered courses. Of the nine partnerships, all offered multiple courses, and all had participants who attended more than one course.




In total, 130 Cohort 3 courses were delivered by the end of Year 6 of MMSP funding. (Plus one additional course was offered in conjunction with a program with a different funding source, but there were no MMSP participants.) Of these 130 courses, 70 were mathematics courses, 58 were science or

 technology/engineering courses, and two were courses offering both mathematics and science content.


By the end of Year 6, 1,076 unique Cohort 3 participants participated in MMSP courses.

 389 participants (36% of all Cohort 3 participants) attended multiple courses in Cohort 3 across the

2006-2007, 2007-2008, and 2008-2009 years.



424 participants (39% of all Cohort 3 participants) attended multiple courses across all MMSP funding periods.

 1984 course seats were filled by Cohort 3 participants across 2006-2007, 2007-2008, and 2008-2009.

 Course attrition rates were generally low and averaged 4% across all Cohort 3 courses.






Of all 1,076 unique Cohort 3 participants, 97% came from public schools (including public charter schools), and 2% came from non-public schools.







 The Cohort 3 partnerships exceeded the ESE target of having at least 50% of all participants come from high need districts, with 72% of all Cohort 3 participants in the program coming from high need districts.

 Six of the nine partnerships had at least 50% of their participants coming from high need districts.

 Five of the nine partnerships had at least 75% of the participants come from high need districts.

As the following data reveal, not all Cohort 3 participants were licensed in their teaching areas or held relevant degrees, indicating that course participants were, in fact, those in need of the courses:






Of regular education mathematics courses taught by Cohort 3 teachers, 81% were taught by teachers who were licensed in mathematics.

 Of special education mathematics courses taught by Cohort 3 teachers, 36% were taught by teachers who were licensed in mathematics.



Of ELL mathematics courses taught by Cohort 3 teachers, 43% were taught by teachers who were licensed in mathematics.

 Of regular education science or technology/engineering courses taught by Cohort 3 teachers, 66% were taught by teachers who were licensed in the subject of the course.



Of special education science or technology/engineering courses taught by Cohort 3 teachers, 49% were taught by teachers who were licensed in the subject of the course.

 Of ELL science or technology/engineering courses taught by Cohort 3 teachers, 39% were taught by teachers who were licensed in the subject of the course.




 Of regular education mathematics courses taught by Cohort 3 teachers, 17% were taught by teachers who held mathematics degrees.



Of special education mathematics courses taught by Cohort 3 teachers, 2% were taught by teachers who held mathematics degrees.

 Of ELL mathematics courses taught by Cohort 3 teachers, 14% were taught by teachers who held mathematics degrees.



Of regular education science or technology/engineering courses taught by Cohort 3 teachers, 25% were taught by teachers who held degrees that were relevant to the focus of the course.

 Of special education science or technology/engineering courses taught by Cohort 3 teachers, 3% were taught by teachers who held degrees that were relevant to the focus of the course.



Of ELL science or technology/engineering courses taught by Cohort 3 teachers, 6% were taught by teachers who held degrees that were relevant to the focus of the course.





Summary of Findings







By the end of the 2008-2009 funding period, of the Cohort 3 participants who had entered MMSP as not highly qualified, 32 had attained highly qualified status.


The content knowledge of Cohort 3 participants was increased:



Statistically significant improvements in scores on content knowledge assessments occurred in 110 of the

115 (96%) Cohort 3 courses for which it was possible to determine statistical significance of score changes.



Gains in average percentage of items correct between pre- and post-course test administrations occurred in all 127 of the 127 courses delivered across all Cohort 3 partnerships for which tests were administered and scores were obtained.

Integrating Courses into Higher Education Institutions

Integration or plans for future integration of courses into institutions of higher education occurred within all

Cohort 3 partnerships.

Cohort 4 Findings




Eight Cohort 4 partnerships were funded during the 2008-2009 funding period. Of these, four were organized around mathematical content and four were organized around science content. Of the eight

Cohort 4 MMSP partnerships, all delivered courses. Of the eight partnerships, all but two offered multiple courses, four had participants who took more than one course within Cohort 4, and all had participants who had attended more than one course across the entire duration of MMSP.




In total, 27 Cohort 4 courses were delivered by the end of Year 6 of MMSP funding. Of these 27 courses,

13 were mathematics courses and 14 were science/technology/engineering courses.




During Year 6, 341 unique Cohort 4 participants participated in MMSP courses.



46 participants (13% of all Cohort 4 participants) attended multiple courses in Cohort 4 during 2008-

2009.

 20 participants (6% of all Cohort 4 participants) attended multiple courses across all MMSP funding periods.



391 course seats were filled by Cohort 4 participants during 2008-2009.



Course attrition rates were generally low and averaged 5% across all courses offered by Cohort 4 partnerships.




Reaching Targeted Participants




 Of all 341 unique Cohort 4 participants, 98% came from public schools (including public charter schools), and 2% came from non-public schools.




 The Cohort 4 partnerships exceeded the ESE target of having at least 50% of all participants come from high need districts, with 64% of all Cohort 4 participants in the program coming from high need districts.



Five of the eight partnerships had at least 50% of their participants coming from high need districts.

 Three of the eight partnerships had at least 75% of the participants come from high need districts.

As the following data reveal, not all Cohort 4 participants were licensed in their teaching areas or held relevant degrees, indicating that course participants were, in fact, those in need of the courses:






Of regular education mathematics courses taught by Cohort 4 teachers, 82% were taught by teachers who were licensed in mathematics.

 Of special education mathematics courses taught by Cohort 4 teachers, 56% were taught by teachers who were licensed in mathematics.



Of ELL mathematics courses taught by Cohort 4 teachers, 60% were taught by teachers who were licensed in mathematics.

 Of regular education science or technology/engineering courses taught by Cohort 4 teachers, 72% were taught by teachers who were licensed in the subject of the course.



Of special education science or technology/engineering courses taught by Cohort 4 teachers, 63% were taught by teachers who were licensed in the subject of the course.

 Of ELL science or technology/engineering courses taught by Cohort 4 teachers, 67% were taught by teachers who were licensed in the subject of the course.




 Of regular education mathematics courses taught by Cohort 4 teachers, 13% were taught by teachers who held mathematics degrees.



Of special education mathematics courses taught by Cohort 4 teachers, 8% were taught by teachers who held mathematics degrees.

 Of ELL mathematics courses taught by Cohort 4 teachers, none were taught by teachers who held mathematics degrees.



Of regular education science or technology/engineering courses taught by Cohort 4 teachers, 22% were taught by teachers who held degrees that were relevant to the focus of the course.

 Of special education science or technology/engineering courses taught by Cohort 4 teachers, none were taught by teachers who held degrees that were relevant to the focus of the course.




 Of ELL science or technology/engineering courses taught by Cohort 4 teachers, none were taught by teachers who held degrees that were relevant to the focus of the course.








By the end of the 2008-2009 funding period, of the Cohort 4 participants who had entered MMSP as not highly qualified, five had attained highly qualified status.


The content knowledge of Cohort 4 participants was increased:



Statistically significant improvements in scores on content knowledge assessments occurred in 17 (63%) of the 27 Cohort 4 courses.



Gains in average percentage of items correct between pre- and post-course test administrations occurred in 26 of the 27 courses delivered through Cohort 4 partnerships.

Integrating Courses into Higher Education Institutions

Integration or plans for future integration of courses into institutions of higher education occurred within some

Cohort 4 partnerships.



Download MS WORD Document size: 1.7MB

Annual State-level Evaluation Report

Contents

Tables Index

Program Description

Report Organization

Evaluation Plan and Activities

State-level Evaluation

Local Evaluation and Related Technical Assistance

Cohort 3 Activity: September 1, 2006 through August 31, 2009

State-level Participant Background Data

Partnership-level Participant Background Data

Cohort 4 Activity: September 1, 2008 through August 31, 2009

State-level Participant Background Data

Partnership-level Participant Background Data

Cumulative Summary: All Cohorts over All Funding Periods

Summary of Findings

Cumulative Findings

Cohort 3 Findings

Cohort 4 Findings

Related documents

Products

Support

Download MS WORD Document size: 1.7MB

Annual State-level Evaluation Report

Contents

Tables Index

Program Description

Report Organization

Evaluation Plan and Activities

State-level Evaluation

Local Evaluation and Related Technical Assistance

Cohort 3 Activity: September 1, 2006 through August 31, 2009

State-level Participant Background Data

Partnership-level Participant Background Data

Cohort 4 Activity: September 1, 2008 through August 31, 2009

State-level Participant Background Data

Partnership-level Participant Background Data

Cumulative Summary: All Cohorts over All Funding Periods

Summary of Findings

Cumulative Findings

Cohort 3 Findings

Cohort 4 Findings

Related documents

Add this document to collection(s)

Add this document to saved

Suggest us how to improve StudyLib