Download MS WORD Document size: 3.4 MB

Massachusetts Mathematics and Science Partnership Title IIB

Annual State-level Evaluation Report

Reporting Period:

February 2, 2004 through August 31, 2006

Prepared for the Massachusetts Department of Education

March 2007

Annual Report of the MMSP State-level Evaluation Contents

Contents

Executive Summary ....................................................................................................................... 3

Program Description...................................................................................................................... 9

Evaluation Plan and Activities .................................................................................................... 11

State-level Evaluation .......................................................................................... 11

Local Evaluation and Related Technical Assistance ........................................... 11

MMSP Participant Background Data .......................................................................................... 13

State-level Participant Background Data ............................................................. 13

Partnership-level Participant Background Data ................................................... 20

Progress Toward Meeting MMSP Goals .................................................................................... 31

Goal I. ................................................................................................................... 31

Goal II. .................................................................................................................. 37

Goal III. ................................................................................................................. 44

Goal IV. ................................................................................................................ 47

Goal V. ................................................................................................................. 53

Discussion and Recommendations ........................................................................................... 54

Appendix A: Timeline for State-level Evaluation and TA Activities ........................................ 57

Appendix B: Minimum Expectations for Evaluation ................................................................ 59

Appendix C: Participant Background Survey – Year 1 ............................................................ 62

Appendix D: Participant Background Survey – Year 2 ............................................................ 66

Appendix E: Participant Background Survey – Year 3 ............................................................ 73

Appendix F: Results of the Participant Background Survey .................................................. 81

Appendix G: High Need District Eligibility ................................................................................ 92

Appendix H: Subject Matter Competency Demonstration Options ........................................ 95

Appendix I: Highly Qualified Status of Unique Participants .................................................. 96

Appendix J: Criteria that Account for Gain in Highly Qualified Status .................................. 97

UMass Donahue Institute

Research and Evaluation Group I

Annual Report of the MMSP State-level Evaluation Tables

Tables Index

Table 1: Partnership Budgets by Program Year .......................................................................................... 9

Table 2: Teaching Areas .................................................................................................................................. 14

Table 3: Degrees Held and Pursued in Relation to Subjects Taught and Licensure ....................... 15

Table 4: Types of Schools of Unique Participants .................................................................................... 16

Table 5: High Need Status of Unique Participants from Public Schools ............................................ 17

Table 6: High Need Districts by Partnership .............................................................................................. 17

Table 7: Reasons for Participation in the Course – All Seats

................................................................ 19

Table 8: EduTron Participant Background Information ........................................................................... 21

Table 9: Harvard Participant Background Information ............................................................................ 22

Table 10: Lesley University Participant Background Information ........................................................ 23

Table 11: MCLA Science Participant Background Information ............................................................. 24

Table 12: Salem State College Participant Background Information ................................................... 25

Table 13: Springfield/Holyoke Participant Background Information ................................................... 26

Table 14: Wareham Participant Background Information ....................................................................... 27

Table 15: WPI Participant Background Information ................................................................................. 28

Table 16: MCLA Math Participant Background Information ................................................................... 29

Table 17: University of Massachusetts Amherst Participant Background Information .................. 30

Table 18: Repeat Participants by Partnership ............................................................................................ 31

Table 19: Enrollment and Attrition Information as Reported by Each MMSP Partnership ............ 32

Table 20: Highly Qualified Status of Unique Participants ....................................................................... 38

Table 21: Number of Unique Participants Gaining Highly Qualified Status, by Partnership ......... 39

Table 22: MTEL Tests Taken by MMSP Participants – Total to Date

.................................................... 40

Table 23: HOUSSE Plan Status of Unique Public School Teachers ..................................................... 41

Table 24a: MMSP Science and Technology Teaching Areas

—Regular Education

.......................... 41

Table 24b: MMSP Science and Technology Teaching Areas —Special Education

........................... 42

Table 25a: MMSP Mathematics Teacher Levels —Regular Education

................................................ 42

Table 25b: MMSP Mathematics Teacher Levels —Special Education

................................................. 43

Table 26: Mean Percentage Scores for Pre-course & Post-course Tests, Including Gains in

Mean Scores ......................................................................................................................................................... 47


Research and Evaluation Group II

Annual Report of the MMSP State-level Evaluation Executive Summary

Executive Summary

The purpose of the Massachusetts Mathematics and Science Partnership (MMSP) Program is to provide high quality professional development for teachers in Massachusetts in the content areas of mathematics, science, and technology/engineering (MSTE). This multi-year project is funded through Title IIB of the No Child Left Behind

(NCLB) Act. The intention of the MMSP funding is to increase the number of highly qualified teachers in the specified content areas in the Commonwealth, particularly in high need districts, with an ultimate outcome of increased student achievement. The funding is administered by the Massachusetts Department of Education

(MADOE).

The Program began in February 2004, and has had three funding periods, defined as follows:



Year 1: February 2, 2004 through August 31, 2004



Year 2: September 1, 2004 through August 31, 2005.




The partnerships who received initial funding in Year 1 are referred to as Cohort 1; those who received initial funding in Year 2 are referred to as Cohort 2. Cohort 1 consisted of eight partnerships, with six of the eight partnerships offering mathematics professional development and two offering science professional development.

Cohort 2 consisted of two partnerships, both offering mathematics professional development.

The University of Massachusetts Donahue Institute was contracted to coordinate state-level collection of outcome data and to provide evaluation-related technical assistance to the partnerships.

The MADOE established the following five goals for the partnerships to achieve through MMSP funding. Data supporting each goal were collected from February 2, 2004 through August 31, 2006, for both Cohort 1 and

Cohort 2 1 .

Goal I. Develop and implement an effective and sustained course of study for current teachers of grades

4-8 mathematics and/or grades 4-12 science and technology/engineering.



A total of ten partnerships were funded across the Commonwealth. Eight were organized around mathematical content, and two were organized around science content. Of the ten MMSP partnerships, all

 delivered courses.

In total, 85 MMSP courses were offered by the end of Year 3 of MMSP funding. Of these 85 courses, 76 were mathematics courses, and eight were science courses.

1 Most data are reported as Year 1, Year 2, Year 3, and cumulative totals. The data from Year 1 were collected from February 2, 2004 through August 31, 2004. The data collected in Year 2 were collected from September 1, 2004, to August 31, 2005. The data collected in

Year 3 were collected from September 1, 2005, to August 31, 2006. There are also cumulative totals reported that include all three years’ years’ data. The data reported are largely descriptive in nature. Any participant data that could be aggregated at the state level were collected from the Participant Background Survey, which each participant completed on the last day of each course. Participant data, unless noted, are reported by unique individuals regardless of the number of courses taken by each individual. Partnership data regarding integration of MMSP courses into education and/or arts and sciences at the partnerships’ institutes of higher education was obtained through a series of items that appeared in the context of the Annual Report Addendum that was completed by partnerships in November

2006.


Research and Evaluation Group 3




In total, 909 unique participants participated in MMSP courses by the end of Year 3. Of all 909 unique participants, 97% came from public schools (including 3% from public charter schools), 2% came from non-public schools, and less than 1% did not indicate their school type. Approximately 60% of participants from public schools were from high need districts.



There were 354 participants who took more than one MMSP course by the end of Year 3. Of the 354 participants who took more than one course, approximately 60% were from high need districts.

Goal II. Increase the number of teachers currently employed in the partnership school districts who are licensed in the areas they teach and/or have completed their High Objective Uniform State

Standard of Evaluation (HOUSSE) plans.

Public school teachers must meet the federal definition of highly qualified to comply with the federal No Child

Left Behind legislation. Goal II addresses the spirit of the federal No Child Left Behind legislation regarding teacher licensure, professional development, and competency in subject area taught.



Licensure in Mathematics and Science Content Areas for Middle and High School Teachers



67% of regular education middle and high school MMSP teachers who taught mathematics were licensed in mathematics.

 18% of special education middle and high school MMSP teachers who taught mathematics were licensed in mathematics.



35% of regular education middle and high school MMSP teachers who taught a science or technology content area were licensed in the area in which they taught.

 3% of special education middle and high school teachers who taught a science or technology content area were licensed in the area in which they taught.



Highly Qualified Status



By the end of Year 3, of the participants who had entered MMSP as not highly qualified, 111 had attained highly qualified status. The breakdown of how highly qualified status was attained is as follows:



Sixteen passed the appropriate Massachusetts Tests for Educator Licensure (MTEL).

 Five obtained a degree in content areas taught.



Three earned a teaching license.

 Three obtained undergraduate degree equivalents in content areas taught.



Fourteen simultaneously met two or more criteria.

 Seventy of the 332 who indicated they had HOUSSE plans completed a sufficient number of

Professional Development Points (PDPs) on HOUSSE plans.






HOUSSE Plan Information 2



Of 883 public school participants, 38% reported having a HOUSSE Plan, 32% reported not having a

HOUSSE Plan, and 27% were unsure.

 By the end of Year 3, of the 332 public school participants who indicated they had HOUSSE Plans,

253 (76%) reported having 48 or more PDP hours, 32 (10%) reported having fewer than 48 PDP hours, and 47 (14%) did not respond.

Goal III. Increase the number of highly qualified teachers in mathematics, science, and/or technology/engineering by integrating the courses of study into schools of education and/or arts and sciences at institutes of higher education.

For Year 3, partnerships were asked to describe the activities of their partnerships during the Year 3 funding period that spoke to the “institutionalization” of their courses, the extent to which their courses have been integrated into activities of their higher education partners. The extent and type of integration varied across partnerships. Following are summaries for partnerships that indicated that additional integration was occurring in

Year 3:



In Year 3, all MMSP courses that EduTron created were offered through the Center for Professional

Studies at Fitchburg State College. All courses were offered to in-service teachers for graduate credit. In addition, the partnership united Mathematics faculty and Education faculty at Fitchburg State College in a fundamental way: All parties are now working together to improve teacher preparation in mathematics through more stringent requirements in math courses and by improving mathematics offerings. The education department is in the process of hosting a math education summit to articulate the problems and present potential solutions that are arrived at through the joint efforts of math and education faculty.



In addition to continuing with the programs that were created in prior MMSP years, Lesley University has begun developing an online mathematics program that will offer a master’s degree in elementary or middle school mathematics education. Through needs that have arisen through the classroom implementation component of their participation in MMSP, Lesley also has developed a relationship with

Cisco Corporation who will be providing the online platform for the online mathematics program.



Massachusetts College of Liberal Arts (MCLA) Science now offers two new variable credit science for educators courses for in-service teachers.



Massachusetts College of Liberal Arts (MCLA) Math now offers one new variable credit math for educators courses for in-service teachers.

 Salem State College continues to offer courses developed through MMSP as part of a master’s level teaching program in middle school mathematics. All courses developed by Salem State College through

MMSP can be applied towards earning a degree through that program.



The three science courses offered by the Springfield partnership through the MMSP program in Year 3 were offered for graduate credit through University of Massachusetts (UMass) Continuing Education. A goal of the partnership is to have these courses institutionalized in the UMass School of Continuing

Education through the School of Education.

2 Less than half of all course participants who took more than one course were consistent across surveys in their responses to the survey item asking them if they had a HOUSSE Plan, indicating that many were confused about the issue. Consequently, data regarding HOUSSE plans should be viewed with caution.






All courses developed through the UMass Amherst partnership can be taken for graduate level credits that can be transferred to a variety of graduate level programs at UMass. In addition, the two courses are being reviewed by the School of Education Academic matters Committee for permanent course approval.

 The one new course created by WPI in Year 3 was embedded into the master’s level graduate program in mathematics education offered by WPI’s Mathematics Department.

Goal IV. Increase the number of MSTE teachers currently employed in the partner school districts who participate in content-based professional development activities and substantially increase their content knowledge in order to be able to teach effectively the state learning standards.



Of the 85 courses offered across all partnerships by the end of Year 3, pre- to post-course test score gains were statistically significant in 79 (93%), indicating the courses had a positive impact on participants’ content knowledge.



Content assessments for three courses offered through one partnership were administered to both treatment (i.e., MMSP) and comparison groups. For each course, the treatment group post-course scores were statistically significantly higher than those of the comparison group, although in one case the treatment group pre-course scores also were statistically significantly higher than those of the comparison group.



78% of the participants reported taking MMSP courses to increase knowledge in the content offered.

Goal V. Improve student academic achievement as measured by Massachusetts Comprehensive

Assessment System (MCAS) mathematics and science and technology/engineering assessments and other assessments.



Partnerships reported student data directly to MADOE and to the United States Education Department under the federal reporting system.



The Donahue Institute state-level data collection effort for Title IIB did not include student outcome data; however, more than 89,000 students were taught annually by MMSP participants.

Data Quality

While in many instances for this project the data are sound, there are some areas for which this was not the case.

The Donahue Institute became aware that problems existed with some of the data when 1) surveys received from participants taking multiple courses revealed inconsistencies in reporting across surveys for individual participants, especially for items used in determining highly qualified status, and 2) surveys indicated that many participants were confused about whether or not they had a HOUSSE plan.

The Participant Background Survey was modified after Year 1 to improve the likelihood of obtaining good quality data, but similar problems with the data arose again in Year 2. The survey was modified again in Year 3, but some problems still existed, and totals to date will be affected by problems from Year 1 and Year 2; therefore, information regarding the highly qualified status of teachers must be viewed with caution.

Reaching Targeted Participants

The partnerships exceeded the MADOE goal of having at least 50% of all participants come from high need districts. Half of the ten partnerships providing courses by the end of Year 3 met that goal as individual partnerships.




One obstacle that interfered with efforts to raise the high need district participation involved one partnership, in particular, that had committed to one cohort going through multiple courses over the life of the project. For this partnership, it was not possible to change their participants mid-program.



MADOE should continue to remind partnerships that enrollment must include a high percentage of participants from high need districts.



MADOE should encourage partnerships to recruit additional participants from their high need partners and/or identify additional high need partners, even if they meet the minimum participant rate of 50%.



MADOE has already begun revisiting the procedure by which districts are identified as high need districts. MADOE also should consider developing criteria to identify individual schools as high need.

Local Evaluation Designs

It was determined early in the initial meetings with partnerships that implementing a rigorous evaluation design – using randomized-controlled trials or strong comparison groups – would be difficult, if not impossible. The limitations that arose were limited resources, the lack of sufficient participant numbers to form reasonable treatment and control groups, and insufficient time to recruit participants into a control group before the start of the program. At the end of Year 3, only one partnership had implemented a weak quasi-experimental evaluation design. The remaining partnerships implemented basic evaluation models utilizing pre-post comparisons. To address the difficulties in creating rigorous evaluation designs, the Request for Proposals for MMSP programs for

2006-2007 included information on how partnerships might obtain additional support through MADOE and the

Donahue Institute for this purpose.



MADOE and the Donahue Institute should continue to work closely with partnerships to provide technical support and resources to conduct rigorous evaluations in the continuation process.

Collecting Student Data

Perhaps the most pervasively problematic issue regarding evaluation of MMSP projects has been difficulty accessing student data. This has been both a logistical and legal challenge for evaluators. Logistically, many school districts do not routinely code their student data by teacher; thus, it is difficult to obtain data aggregated at the participant level. Partnerships have trouble enlisting the necessary cooperation from participating districts with few teachers enrolled in MMSP courses. Finally, Family Educational Rights and Privacy Act (FERPA) regulations present a real legal barrier in situations where the evaluation contract is not managed and paid for by the participating districts – the only relationship that would enable student-level data to be released to the evaluator. To date, it appears as if only two partnerships, EduTron and Lesley, have made a concerted effort to obtain student outcome data at the participant (as opposed to school) level.

Because student data was unattainable for most partnerships, they also encountered problems for meeting the federal reporting requirements regarding student data.



MADOE should give partnerships guidance on forging agreements with their partner districts to have access to student outcome data for participating teachers.

Integrating Courses into Higher Education Institutions

In the second year of the project, efforts were made to define integration and provide partnerships with structured guidance on reporting progress at integration. In the third year, structured guidance was again provided, and the data provided evidence that integration is occurring across almost all partnerships and that integration efforts will continue in the future for many of the partnerships, with at least one partnership (Lesley University) extending integration efforts beyond the scope of the MMSP grant. While the extent and types of integration that have occurred varied across partnerships, integration has happened primarily in the following ways that encourage



Annual Report of the MMSP State-level Evaluation Executive Summary sustainability beyond the duration of the program: embedding MMSP courses into pre-existing programs, integrating courses into pre-existing continuing education programs, and creating new degree programs.



MADOE should encourage partnerships to continue efforts to integrate MMSP courses into degree programs of partner higher education institutions.

Steering Committee

The Steering Committee has continued to provide guidance and feedback to the MADOE on issues regarding the

MMSP partnerships, on broader issues of professional development and mathematics, science, technology, and engineering. The MADOE is fully aware how fortunate it is to have such an engaged and invested Steering

Committee for this initiative. Steering Committee members have made observation visits to most of the partnerships and have offered programmatic suggestions and recommendations to the department.



MADOE should continue the role of the Steering Committee providing advice and guidance and make all efforts to follow up on committee suggestions and requests.



Annual Report of the MMSP State-level Evaluation Program Description

Program Description

The purpose of the Massachusetts Mathematics and Science Partnership Program (MMSP) is to provide high quality professional development for teachers in the content areas of mathematics, science, and technology/ engineering with the goal of increasing the number of highly qualified teachers, particularly in high need districts.

As with most professional development initiatives, the end goal is to improve student achievement levels. This multi-year project is funded by the U.S. Department of Education as part of the No Child Left Behind (NCLB)

Act Title IIB funding stream. Funding to local partnerships is administered by state education agencies; in

Massachusetts this is the Massachusetts Department of Education (MADOE), which awards funding through a competitive grant process.

The Program began in February 2004, and has had three funding periods, defined as follows:



Year 1: February 2, 2004 through August 31, 2004







The partnerships who received initial funding in Year 1 are referred to as Cohort 1; those who received initial funding in Year 2 are referred to as Cohort 2. Table 1 shows the funding received by each partnership for each year of the program.

Table 1: Partnership Budgets by Program Year

Partnership

EduTron

Harvard Graduate School of Education

Year 1

$210,000

$96,743

Year 2

$237,000

$188,856

Year 3

$323,000

$204,300

Lesley University

MCLA - Science

Salem State College

$220,007

$30,350

$118,395

$310,110

$51,912

$209,331

$280,609

$50,930

$214,269

Springfield Public Schools

Wareham Public Schools

$175,000

$120,930

$151,707

$162,122

$173,337

$115,388

Worcester Polytechnic Institute

MCLA – Math

PV Stemnet

$202,322

N/A

N/A

$203,257

$32,864

$88,264

$196,199

$78,630

$174,151

TOTAL $1,173,747 $1,635,423 $1,810,813

Cohort 1 consisted of eight partnerships, with six of the eight partnerships offering mathematics professional development and two offering science professional development. Cohort 2 consisted of two partnerships, both offering mathematics professional development.



Annual Report of the MMSP State-level Evaluation Program Description

This report includes data collected from February 2, 2004 through August 31, 2006, for both Cohort 1 and Cohort

2 3 . Any participant data that could be aggregated at the state level were collected from the Participant Background

Survey, which each participant completed on the last day of each course. Participant data, unless noted, are reported by unique individuals regardless of the number of courses taken by each individual.

The partnerships are composed of higher education institutions, school districts, and, in some cases, private organizations involved in providing both pre-service and in-service training to teachers. In order to provide a deep conceptual understanding of the content, partnerships are encouraged to work with mathematicians and scientists from the arts and sciences or engineering departments of their higher education partners.

Partnerships are required to offer courses that equal at least 45 hours of course time followed by at least 20 hours of follow-up support during the school year. Partnerships are encouraged to tailor the model used to deliver the professional development and follow-up to best fit the objectives of their programs along with their resources, expertise, and existing infrastructure.

Regardless of the variation of models across the partnerships, the programs are required to meet the following main goals of the MMSP:

Goal I. Develop and implement an effective and sustained course of study for current teachers of grades 4-8 mathematics and/or grades 4-12 science and technology/engineering.

Goal II. Increase the number of teachers currently employed in the partnership school districts who are licensed in the areas they teach and/or have completed their High Objective Uniform State Standard of

Evaluation (HOUSSE) plans.

Goal III. Increase the number of highly qualified teachers in mathematics, science, and/or technology/ engineering by integrating the courses of study into schools of education and/or arts and sciences at institutes of higher education.

Goal IV. Increase the number of mathematics, science, and technology/engineering teachers currently employed in the partner school districts who participate in content-based professional development activities and substantially increase their content knowledge in order to be able to teach effectively the state learning standards.

Goal V. Improve student academic achievement as measured by Massachusetts Comprehensive Assessment

System (MCAS) mathematics, science, and technology/engineering assessments and other assessments.

A final component of the program is the formation and on-going role of the MMSP Steering Committee. This group is composed of mathematicians and scientists, mostly from higher education institutions, as well as public schools and private organizations. The committee members have a common interest in mathematics, science, technology, and engineering issues and education. As a group, they provide the MADOE with guidance regarding programmatic and funding decisions related to the MMSP. This group also provides a mechanism for the

MADOE to receive expert input regarding the mathematical and scientific content being offered by MMSP partnerships. The Steering Committee is coordinated by the MADOE, and meetings are held periodically throughout the year.

3 Most data are reported as Year 1, Year 2, Year 3, and cumulative totals. The data from Year 1 were collected from February 2, 2004 through August 31, 2004. The data collected in Year 2 were collected from September 1, 2004, to August 31, 2005. The data collected in

Year 3 were collected from September 1, 2005, to August 31, 2006. There are also cumulative totals reported that include both years’ data.

The data reported are largely descriptive in nature.



Annual Report of the MMSP State-level Evaluation Evaluation Plan and Activities

Evaluation Plan and Activities

State-level Evaluation

Although not required by the U.S. Department of Education, the Massachusetts Department of Education contracted with the University of Massachusetts Donahue Institute to conduct a state-level evaluation of the

MMSP. The Donahue Institute’s primary role as state-level evaluator is to coordinate program-wide collection of outcome data on behalf of the MADOE. Data collection for the state-level evaluation is organized around a basic logic model for professional development initiatives shown below.

Local Evaluation and Related Technical Assistance

In addition to the state-level data collection, each partnership is required to conduct its own local evaluation. In an effort to support strong local evaluations, MADOE required that partnerships sub-contract with the Donahue

Institute to provide technical assistance on design and implementation of their local evaluations. The timeline listing the evaluation activities for this time period is found in Appendix A.

Year 1

Technical assistance activities began at the MMSP Project Kickoff Meeting in February 2004 where the Donahue

Institute presented the following general guidelines for strong local evaluation:

 Include both formative and summative research questions with data collection and analysis plans adequate to address those questions

 Utilize experimental or quasi-experimental design where feasible

 Organize data collection around the basic logic model presented above

Following the kickoff meeting, Donahue Institute staff conducted individual meetings with representatives from each partnership, including evaluators if they had been hired. These meetings provided an opportunity to reinforce



Annual Report of the MMSP State-level Evaluation Evaluation Plan and Activities the above guidelines for strong local evaluation and present MADOE’s expectations. The latter were documented in a “minimum expectations” document, a copy of which can be found in Appendix B.

Each of these meetings also included a discussion of whether it might be feasible to adjust the program design to create an opportunity for an experimental or quasi-experimental evaluation design. It appeared as if the Harvard partnership might have been able to use a random-assignment design since it was expecting its first course to be over-enrolled. However, to date that expectation has not come to pass and the program has not been able to establish a control group for participating teachers. At the end of Year 2, only one program had been able to implement a weak quasi-experimental design regarding only teacher outcomes. Finally, the Lesley partnership is hoping to access student assessment data that will allow them to compare outcomes for students of participating teachers to those for students of teachers who are not participating. It remains to be seen whether those efforts will be successful and whether student MCAS results will be sensitive enough to provide measurable evidence of improved student achievement as a result of teachers’ MMSP course participation.

After these initial meetings, the Donahue Institute provided follow-up support for individual partnerships as necessary.

Year 2

In Year 2, technical assistance was provided for Cohort 1 partnerships as it was needed. The most common request for assistance was related to completing the end-of-course forms. These questions lessened as staff became accustomed to filling out the forms.

In the fall of 2004, there was a round of meetings held with key members of each of the Cohort 1 partnerships and their evaluators to review evaluation plans and provide any assistance necessary. This provided an opportunity to hear what challenges were faced by evaluators in carrying out their evaluations. The biggest challenge at that time was the difficulty faced by partnerships in collecting student data. It was clear that unless a relationship had been forged and the expectation set to collect student achievement data from participating school districts, it would be very difficult if not impossible to collect these data.

The round of fall meetings also provided an opportunity to introduce the federal reporting tool that had been released at that time. Although the final draft and timeline had not been set for the reporting period, introducing the data collection tool gave partnerships the opportunity to collect the necessary data ahead of time.

Technical assistance activities began for the two Cohort 2 partnerships with initial meetings similar to those held with Cohort 1 partnerships. Because there were only two new partnerships in Cohort 2, a kickoff meeting was not held. The evaluation models and expectations were presented to the new partnerships at their individual meetings.

The emphasis was on designing solid evaluations to include both formative and summative research questions with data collection and analysis plans adequate to address those questions. After the initial meetings, follow-up support was provided for individual partnerships.

In the spring of 2005, the due date and final draft of the federal reporting tool were released. The Donahue

Institute worked with the federal DOE to clarify some of the requirements of the report to fit the Massachusetts model of professional development. In doing so, it became apparent that partnerships would have many similar questions. In anticipation of these questions and data requests from partnerships, the Donahue Institute provided a technical assistance workshop for partnerships to walk through the requirements of the federal reporting tool.

Although partnerships had access to the data required, the Donahue Institute had most of it centralized and easily accessible. Therefore, the Donahue Institute provided partnerships with the majority of data required for the federal report.



Annual Report of the MMSP State-level Evaluation MMSP Participant Background Data

MMSP Participant Background Data

There are a total of ten MMSP partnerships. The eight partnerships who were initially funded in Year 1, in the first round of funding, are referred to as Cohort 1. The two partnerships who were initially funded in Year 2 are referred to as Cohort 2. All MMSP partnerships delivered courses in the three years of MMSP partnership activity

(which, in actuality, spanned over the course of only 30 months). Eight of these partnerships delivered mathematics content courses, and two delivered science content courses. In total, there were 85 courses delivered.

Of these 85 courses, 76 were mathematics courses, eight were science courses, and one was a technology/ engineering course. This section of the report summarizes data collected from participants in these courses. In total, there were 909 participants, and 354 of them took two or more courses.

Course participants completed the Participant Background Survey upon completion of each course. See Appendix

C for the survey used in Year 1, see Appendix D for the survey used in Year 2, and see Appendix E for the survey used in Year 3. The purpose of this survey is to gather data about participants’ professional backgrounds and qualifications. This information provides a picture of who the participants are, aids in determining whether the courses are reaching the teachers who most need professional development, and aids in tracking how teacher qualifications may change during the MMSP funding period. Data from the survey regarding teacher licensure, possession of and progress towards earning degrees, and status in terms of Massachusetts Tests for Educator

Licensure (MTEL) exams, allows determination of the number of teachers who meet criteria defining highly qualified status.

State-level Participant Background Data

By the end of Year 3, 909 unique participants completed the participant background survey on one or more occasions. The term “unique participant” refers to each individual who participated in the program, regardless of how many courses he or she took. Data for items from the survey that help to convey participants’ professional backgrounds and motives for participation are discussed in the remainder of this section. All survey data for this group may be found in the “Total to Date” column of Appendix F.

Appendix F also contains participant background survey results for each item of the survey for Year 1, Year 2, and Year 3. Data for each of these years includes only unique participants for that particular year. For example,

Year 3 data includes those participants who completed a course in Year 3 regardless of their participation in either

Year 1 or Year 2, but if a participant completed more than one course in Year 3, his or her data is only counted once in Year 3.

The responses to the survey questions are presented as frequencies and percentages. Not all percentages total

100% because many items allowed multiple responses and not all of the participants responded to all of the items.

Teaching Experience of Participants

At the time of their last completed survey from an MMSP course, the teaching experience of the 909 unique participants was as follows: 15% were in their first to third year of teaching, 38% had between 4 and 10 years experience in education, 27% had between 11 and 20 years of experience, and 18% reported over twenty years of experience.

Teaching Levels of Participants

The schools in the participating districts were organized into categories of elementary schools (grades K-5), K-8 schools, middle schools (grades 6-8), and high schools (grades 9-12). At the time of their last completed survey



Annual Report of the MMSP State-level Evaluation MMSP Participant Background Data from an MMSP course, 22% of participants were teaching in an elementary or K-8 school, 57% were teaching in a middle school, and 12% were teaching in a high school.

Content Taught

The distribution of teaching areas of respondents at the time of the survey is shown in Table 2. Because respondents identified all teaching areas that applied to their positions at the time of the survey, some selected multiple responses, so frequency totals exceed the number of respondents and percentages exceed 100%. Also, figures reported in “Total” column may be smaller than figures for any individual year because the total is based on data from the last survey completed and some repeat participants changed teaching areas over the course of their participation.

At the time of their last MMSP course, 59% were teaching mathematics, 20% were teaching science, and 21% were teaching all subjects at the elementary level.

Table 2: Teaching Areas

Teaching Areas

(Multiple responses permitted)

Year 1 n

N = 341

%*

Year 2

n

N = 456

%*

Year 3

n

N = 464

%*

Total

n

N = 909

%*

Mathematics

Any science area

General Science

Biology

Earth Science

Chemistry

Physics

Technology/Engineering

Computer Science

Elementary (all subjects)

Elementary Mathematics

Other

Not Currently Teaching

TOTAL

190 56%

131 38%

65 19%

22

21

12

6%

6%

4%

11

10

N/A**

3%

3%

103 30%

N/A**

61 18%

13

508

4%

252 55% 288 62% 535 59%

141 31% 107 23% 244 27%

71 16% 56 12% 131 14%

24

16

14

5%

4%

3%

16 3%

14 3%

11 2%

41 5%

30 3%

22 2%

16

3

2

107

31

25

10

571

4%

1%

<1%

23%

7%

5%

2%

10 2%

4 1%

3 1%

20 2%

10 1%

4 <1%

79 17% 189 21%

26 6% 45 5%

22 5%

18 4%

547

59 6%

32 4%

1118

*Percentage of the total unique respondents for relevant year indicating that they currently teach this content area

**This teaching area not offered as a response option on the Year 1 survey

Position of Participants

At the time of their last completed survey from an MMSP course, 90% of course participants identified themselves as teachers. Of all respondents, 74% were regular education teachers; 15% were special education or special education inclusion teachers; 2% were department heads or curriculum coordinators; 1% were principals, assistant principals, or headmasters; 1% were support specialists; <1% were long-term substitutes; <1% were superintendents or assistant superintendents; and 5% indicated that they held “other” positions.




Degrees Held

In Year 2, elementary-level teachers were not asked about currently held or pursued degrees, and only secondarylevel mathematics teachers were asked about degrees held or pursued in the area of education. The survey was subsequently modified to gather degree information from all participants in the third year of this project.

Information on degrees held was derived from the most recently completed survey of each individual. Of the 854 public school teachers who were currently teaching at the time of their last survey, only 688 (81%) responded to the item asking about degrees held. Data indicated that these 688 reporting teachers held a total of 469 bachelor’s degrees and 428 master’s degrees. Of the 688 teachers,

 244 (36%) indicated that a bachelor’s degree was their highest degree: o 106 were in education o 28 were in science o 36 were in mathematics o 8 were in mathematics education o 2 were in science education o 81 were in other areas. (Figures total to more than 244 because some indicated that they held multiple bachelor’s degrees.)

 428 (62%) indicated that a master’s degree was their highest degree o 283 were in education o 27 were in science o 22 were in mathematics education o 14 were in mathematics o 6 were in science education o 105 were in other areas. (Figures total to more than 428 because some indicated that they held multiple master’s degrees.)



11 (2%) reported that a Certificate of Advanced Graduate Study (CAGS) was their highest degree: 9 in education and 2 in other areas.



8 (1%) reported that a doctorate was their highest degree, with 2 holding doctorates in education and 6 holding doctorates in other areas.

Table 3 shows the numbers of individuals – as subgroups of those currently teaching and those licensed – who currently held degrees in either mathematics or any science, technology, or engineering subject.

Table 3: Degrees Held and Pursued in Relation to Subjects Taught and Licensure

Area

Mathematics*

Any STE Subject**

Currently

Teach

570

186

Licensed in

Subject

316

72

Current Degree

BA MA

45 10

28 5

BA

Pursuing Degree

MA CAGS

2 11 2

0 0 1

*Includes Math, Elementary Math, MS Math, MS Math and Science

** Excludes Computer Science because licensure information was not available.

Degrees Currently Pursued

Information on degrees currently being pursued was derived from the most recently completed survey of each individual. Of the 854 unique public school teachers, 271 (32%) reported pursuing at least one degree. Ten (4%)



Annual Report of the MMSP State-level Evaluation MMSP Participant Background Data reported pursuing bachelor’s degrees: 20% in education; 20% in mathematics education; 10% in each of mathematics, science, science education, and a combination of mathematics education and science education; and

20% in other areas.

Of the 271, 218 (80%) reported a master’s degree as the highest degree pursued: 45% in mathematics education;

29% in education; 12% in mathematics; 2% in science education; 2% in a combination of mathematics education and science education; 1% in each of the following: science, a combination of mathematics and education, a combination of mathematics and mathematics education, a combination of education and mathematics education, a combination of education and science, a combination of mathematics education and another area; and 6% in other areas. (Percentages total to more than 100% as a result of rounding.)

Of the 271, 35 (13%) reported that a Certificate of Advanced Graduate Study (CAGS) was the highest degree pursued: 37% in education, 31% in mathematics education, 6% in mathematics, 6% in science, 20% in other areas.

Of the 271, eight (3%) reported a doctorate degree as the highest degree pursued: 50% in education, and one in each of the following areas: science, math education, a combination of education and math education, and another area. (Percentages total to more than 100% as a result of rounding.)

Table 3 shows the numbers of individuals – as subgroups of those currently teaching and those licensed – who were pursuing degrees in either mathematics or any science, technology, or engineering subject.

Types of Schools of Participants

As shown in Table 4, 97% of MMSP participants worked in a public school setting, and 2% worked in a nonpublic school setting.

Table 4: Types of Schools of Unique Participants

Year 1 Year 2 Year 3 Total to Date

School Type

N % N % N % N %

Public School (includes public charter schools)

Non-public School

Other or No Response

TOTAL

332

8

1

97.4% 448

2.4% 7

0.3% 1

341 100.1% 456

98.2% 455

1.5% 6

0.2% 3

98.1%

1.3%

0.7%

99.9% 464 100.1%

883

20

6

97.1%

2.2%

0.7%

909 100.0%

High Need Status of Districts of Participants

MMSP partnerships were required to include at least one high need district. Appendix G identifies the criteria for the high need designation. The high need status of some school districts changed between Year 1 and Year 2. To classify participants from districts that were high need at one point in time but not high need at another, a process was used that took into account the high need status of participants’ districts from the beginning of each partnership’s MMSP involvement with the program. If a district was identified as high need when it first joined

MMSP, that classification continued as long as the district was involved. Specifically, for Year 1, high need status was determined using only Year 1 eligibility criteria. For Year 2, high need status of Cohort 1 participants was determined using both Year 1 and Year 2 eligibility criteria, and high need status of Cohort 2 participants was determined using only Year 2 eligibility criteria. For Year 3, high need status of Cohort 1 participants was determined using eligibility criteria from Year 1, Year 2, and Year 3. For Year 3, high need status of Cohort 2 participants was determined using eligibility criteria from Year 2 and Year 3.




As of their last course in MMSP, of the 354 individuals who took multiple courses, approximately 60% were from high need public school districts, approximately 36% were from other public school districts, and approximately

3% either were from private schools or did not provide information on their districts. In addition, approximately

2% were from high need districts for some of the courses they took but not for others 4 . Table 5 shows that of the

883 unique participants working in public schools, approximately 60% were employed in high need districts.

Table 5: High Need Status of Unique Participants from Public Schools


Item

N % N % N % N %

High Need District

Non-high Need District

Other*

TOTAL

202 60.8% 276 61.6% 254 55.8% 530 60.0%

129 38.9% 161 35.9% 196 43.1% 337 38.2%

1 0.3% 11 2.5% 5 1.1% 16 1.9%

332 100.0% 448 100.0% 455 100.0% 883 100.1%

*Includes those who did not identify their districts and public school participants who took multiple courses whose districts were considered high need for only some of the courses those participants took.

The MADOE expected that at least 50% of participants in each partnership would come from high need districts, and further, they set an informal goal that at least 75% of participants for each partnership would come from high need districts. Over the course of each partnerships involvement in MMSP, five of the ten partnerships had at least 50% of their participants coming from high need districts, and four of the ten partnerships exceeded the informal goal of having at least 75% of the participants come from high need districts. Table 6 presents the number of participants from high need districts organized by each partnership. Note that if columns for Table 6 are summed, the totals will not correspond to relevant data in Table 5 for two reasons: 1) because Table 5 presents data for unique participants across all partnerships while Table 6 presents data for unique participants only within partnerships (and some participants took courses through multiple partnerships over all three years) and 2) the districts of some of those participants who crossed partnerships were not consistently classified as high need districts (either because of the content of the course or because of the cohort of the partnership offering the course).

Table 6: High Need Districts by Partnership

Partnership

EduTron

Harvard University

High Need Districts

Fitchburg

Gardner

Subtotal

Boston

Boston Renaissance Charter

Cambridge

Fall River

Number of Participants* from High Need Districts

Year 1 Year 2 Year 3

Total to

Date

37

14

53

4

51 (79%) 57 (88%)

1 3

3

5

0

2

4

0

28

9

85*

19

37 (84%) 104 (80%)

10 13

0

9

4

5

17

4

4 Teachers who took MMSP math courses when their districts were considered high need for only science were identified as having come from a “non-high need” district, and teachers who took MMSP science courses when their districts were considered high need for only math were identified as having come from a “non-high need” district.



Annual Report of the MMSP State-level Evaluation



Partnership High Need Districts

Harvard University

Lesley University

MCLA – Science

(continued)

Salem State College


Springfield/Holyoke Public Schools


Lowell

Malden

New Bedford

Somerville

Somerville Charter School

Southbridge

Subtotal

Malden

Adams-Cheshire

Clarksburg

Florida

North Adams

Subtotal

Boston

Chelsea

Haverhill Public Schools

Lynn

Salem

Subtotal

Holyoke

Springfield

Subtotal

Wareham

Abby Kelley Foster Charter

Athol-Royalston

β

Berkshire Hills

Boston

Brockton

Cambridge

Chicopee

Fall River

Fitchburg

Lawrence Family Devt. CS

Lowell Community CS

New Bedford

North Adams

Pittsfield

Ralph C Mahar Charter

Seven Hills CS

Somerville

Webster

Winchendon

Worcester

0

0

0

0

0

0

1

0

1

2

5

0

9 (39%) 18 (33%)

21 (21%) 16 (19%)

N/A

6

1

2

0

0

1

32

5

14 (100%)

0

1

20

32

18 16

51 (93%) 69 (86%)

6

32

17

31

38 (100%) 48 (100%)

17 (46%) 11 (61%)

2

0

2

0

0

0

0

2

0

0

0

4

0

0

4

0

0

1

0

0

0

0

0

0

0

0

0

4

1

1

8

7

1

0

1

1

2

0

1

0



Total to

Date

N/A

0

2

0

2

1

3

1

0

0

0

6

10

0

1

2

0

3

0

0

1

0

3 †

0

4

0

1

31 (39%)

1

3 †

1

6

5

1

56 (36%)

14 † (19%) 21 † (20%)

5

1

1

6

1

3

3 6

10 (100%) 16 (100%)

1

1

1

1

16

41

29

69

14 31

73 (79%) 131 (81%)

19

28

47 (96%)

30

64

94 (97%)

23 (53%)

1

1

11

20

2

2

2

2

1*

3

1

4

1

1

7

1

4

0

1

1





Partnership High Need Districts



Total to

Date


(continued)

Subtotal 15 (63%) 27 (41%) 32 (43%) 66 (46%)

MCLA – Math North Adams N/A N/A 1 (9%) 1 (9%)

University of Massachusetts

Amherst

Chicopee

Greenfield

Holyoke

Springfield

Westfield

Subtotal

N/A

0

2

5

7

2

16 (64%)

2

0

5

4

2

13 (37%)

2

2

8

9

4

25 (47%)

* One participant from Fitchburg took a course through WPI in Year 3 and also took a course through EduTron in a previous year.

†

In Year 3, one participant from Malden had taken one course through Harvard and another course through Lesley.

β

The Year 1 participant from Athol-Royalston does not appear in the “Total to Date” column because he/she took multiple courses and had changed school districts between Year 1 and time of completion of the last survey.

Reasons for Participation in MMSP Courses

For each course taken, respondents identified all of their reasons for participating. Unlike in the earlier portions of this report where data were presented for unique participants, data for this issue are presented for all course seats, since responses are uniquely relevant to each distinct course that a participant took. Table 7 presents findings for all participants for all courses taken for all three years .

Table 7: Reasons for Participation in the Course – All Seats

Reasons for Participation

(Multiple responses permitted) n

Year 1

%* n

Year 2

%* n

Year 3

%* n

Total

%*

To increase knowledge in content

To obtain graduate credit

To earn PDPs for recertification

To pursue a personal interest

To get an additional license (certification)

355

278

161

147

89

85%

66%

38%

35%

21%

573

456

309

245

157

80%

63%

43%

34%

22%

499

444

240

207

103

To prepare for the Massachusetts Test for Educator

Licensure (MTEL)

To earn PDPs for HOUSSE plan requirement

To follow an administrator’s suggestion

To obtain a first license (certification)

82

53

32

15

20%

13%

8%

4%

154

116

73

22

21%

16%

10%

3%

118

90

47

22

17%

13%

7%

3%

Other 25 6% 31 4% 34

TOTAL RESPONSES 1237 2136 1804

*Percentage of the total number of participants for the year indicating that this was a reason for participation

5%

72% 1427

64% 1178

35%

30%

15%

710

599

349

354

259

152

59

90

5177

78%

64%

39%

33%

19%

19%

14%

8%

3%

5%




Partnership-level Participant Background Data


Presented in Table 8 through Table 17, this section offers an overview of selected participant survey data for each partnership. The responses to the survey questions are presented as frequencies and percentages. In cases where not all participants responded to all of the items and in cases where multiple responses were permitted, percentages presented may not total 100%.

Each of the tables in this section contains information on “Grade of Licensure.” The categories are defined as follows:

 “Elementary school” refers to the survey options of “PreK-2,” “PreK-3,” “PreK-6,” “PreK-8,” “PreK-9,”

“1-6,” and “5-6.”

 “Middle school” refers to the survey options of “5-8” and “5-9.”

 “High School” refers to the survey options of “5-12,” “8-12,” and “9-12.”

 “All levels” refers to the survey option of “All levels.”




MMSP Partnership: EduTron

Table 8: EduTron Participant Background Information


Total Number of Participants

Participants Who Took Multiple Courses

Teach Regular Education

Teach Special Education (sole and inclusion)

Teach Elementary (all content areas)

Teach Elementary Math

Teach Mathematics Above Elementary

Teach Science or Technology

Teach in High Need District

Hold National Certification – Mathematics

Hold National Certification – General Science

Hold National Certification – Any Subject

Highly Qualified Yes

No

In some, but not all areas

Cannot be determined

Private school/Not applicable

Have a HOUSSE Plan Yes

No

Not Sure

Have Passed One or More MTEL Exams

Hold One or More Teaching Licenses

Total Number of Licenses Held

License Area Elementary


Mathematics

Grade of Licensure

Science

Elementary School

(see page 20 for category definitions) Middle School

High School

Licensed in Areas Taught

All Levels

Yes

No

Some Areas

Unknown and N/A

Year 1

Number of Participants

Year 2 Year 3 Total to Date

99

35

N/A

17

6

43

24

19

9

28

18

8

7

65 65

48 (74%) 44

12 (18%) 13

44 130

49

(68%) 30 (68%) 89 (68%)

(20%) 7 (16%) 23 (18%)

23 (35%) 33

N/A 1

(51%) 15 (34%) 48 (37%)

(2%) 1 (2%) 3 (2%)

35 (54%) 19 (29%) 1 (2%) 57 (44%)

12 (18%) 13 (20%) 24 (55%) 24 (18%)

51 (78%) 57 (88%) 37 (84%) 104 (80%)

N/A

17

N/A

(26%)

26

15

3

13

8

24

26

12

(40%)

(23%)

(5%)

(20%)

(12%)

(37%)

(40%)

(18%)

11 (17%) 20

1 (2%) 1 (2%)

34

3

3

21

4

32

6

24

1

N/A

(2%) 1

N/A

(2%)

(52%)

(5%)

(5%)

(32%)

(6%)

(49%)

(9%)

(30%)

30

1

0

10

3

25

4

13

(68%)

(2%)

(0%)

(23%)

(6%)

(57%)

(9%)

(30%)

(31%) 10 (23%)

2

28

(2%)

2

N/A

(2%)

69

12

2

33

14

56

28

38

(53%)

(9%)

(2%)

(25%)

(11%)

(43%)

(22%)

(29%)

(22%)

61 (94%) 65 (100%) 44 (100%) 126 (97%)

Number of Licenses

105

48

2

7

3

66

20

7

12

36

17

2

10

79

30

1

10

3

39

21

9

10

32

7

1

4

203

83

1

27

10

106

48

25

22

94

13

9

14
















No





No

Not Sure






Mathematics

Middle School Mathematics/Science

Science

Technology

Grade of Licensure Elementary School



High School

All Levels

Yes

No

Some Areas

Unknown and N/A


Research and Evaluation Group


MMSP Partnership: Harvard Graduate School of Education (HGSE)

Table 9: Harvard Participant Background Information

Year 1

23

17 (74%)

3 (13%)

1

N/A

(4%)

18 (78%)

5 (22%)

9 (39%)

N/A

N/A

2

11

5

0

5

2

1

9

12

(9%)

10 (43%)

(48%)

(22%)

(0%)

(22%)

(9%)

(4%)

(39%)

(52%)

21 (91%)

34

6

1

11

5

1

4

8

N/A

12

0

4

1

9

18


Year 2 Year 3

55

45 (82%)

5 (9%)

7 (13%)

4 (7%)

39 (71%)

5 (9%)

18 (33%)

5 (9%)

0 (0%)

29

8

1

11

6

15

22

18

N/A

19 (35%)

(53%)

(14%)

(2%)

(20%)

(11%)

(27%)

(39%)

(33%)

52 (95%) 74 (93%)


93

19

2

32

15

2

3

1

0

37

35

30

1

29

3

131

23

10

57

8

1

14

0

0

67

31

39

1

34

0

80

37 (46%)

38 (48%)

12 (15%)

6 (8%)

56 (70%)

4 (5%)

31 (39%)

2 (3%)

0 (0%)

46

13

0

15

6

32

26

18

N/A

25 (31%)

(58%)

(16%)

(0%)

(19%)

(8%)

(40%)

(33%)

(23%)

258

47

13

107

26

2

21

5

1

114

84

76

3

75

2

Total to Date

156

10

99 (63%)

45 (29%)

20 (13%)

10 (6%)

108 (69%)

14 (9%)

56 (36%)

7 (4%)

0 (0%)

87

24

1

31

13

49

57

45

N/A

53 (34%)

(56%)

(15%)

(1%)

(20%)

(8%)

(31%)

(37%)

(29%)

146 (94%)

22














No





No

Not Sure






Mathematics


Science

Grade of Licensure

Technology

Elementary School


High School


All Levels

Yes

No

Some Areas

Unknown and N/A




MMSP Partnership: Lesley University

Table 10: Lesley University Participant Background Information


Year 1

99

75 (75%)

14 (14%)

43 (43%)

N/A

54 (54%)

11 (11%)

21 (21%)

4

37

30

0

25

7

20

47

27

N/A

N/A

(4%)

(37%)

(30%)

(0%)

(25%)

(7%)

(20%)

(47%)

(27%)

27 (27%)

98 (99%)

176

36

18

19

62

26

4

6

74

N/A

20

0

6

0

103

140

27

16

16

57

20

2

3

6

0

19

2

2

0

79


Year 2 Year 3 Total to Date

16 (19%)

1

0

(1%)

(0%)

45

N/A

(53%)

19

3

17

1

(22%)

(4%)

(20%)

(1%)

32

32

19

(38%)

(38%)

(22%)

24 (28%)

85

64 (75%)

11 (13%)

38 (45%)

7 (8%)

37 (44%)

4 (5%)

82 (96%)

75

52 (69%)

10 (13%)

25 (33%)

16 (21%)

107

85

76 (71%)

13 (12%)

45 (42%)

22

(21%)

42 (56%)

3 (4%)

45 (42%)

6 (6%)

14 (19%)

1

0

(1%)

(0%)

53

N/A

(71%)

4

3

11

4

(5%)

(4%)

(15%)

(5%)

40

24

9

(53%)

(32%)

(12%)

23 (31%)

21 (20%)

2 (2%)

1 (1%)

71

N/A

(66%)

13

2

16

5

(12%)

(2%)

(15%)

(5%)

54

37

13

(50%)

(35%)

(12%)

32 (30%)

75 (100%) 106 (99%)


125

31

16

12

61

2

3

9

48

2

26

1

3

2

66

185

42

19

21

89

4

2

12

74

2

34

1

3

0

102

23


MMSP Partnership: Massachusetts College of Liberal Arts (MCLA) Science

Table 11: MCLA Science Participant Background Information













No





No

Not Sure






Mathematics

Grade of Licensure

Science

Elementary School



High School

All Levels

Yes

No

Some Areas

Unknown and N/A

Year 2


Year 3 Total to Date

24

2

0

2

8

2

2

3

11

11

9

0

1

14

12 (86%)

1 (7%)

1 (7%)

0 (0%)

7 (50%)

8 (57%)

14 (100%)

3

1

1

1

0

0

8

7

3

4

2 (14%)

(0%)

(0%)

(57%)

(21%)

(7%)

(7%)

(7%)

(50%)

(21%)

(29%)

14 (100%)

10

9

1

(90%)

(10%)

0

0

3

(0%)

(0%)

(30%)

7 (70%)

10 (100%)

0

1

1

0

0

0

8

8

1

1

3

(0%)

(0%)

(80%)

(0%)

(10%)

(10%)

(0%)

(80%)

(10%)

(10%)

(30%)

10 (100%)


14

4

2

7

2

1

0

4

0

0

2

4

4

2

0

6

9

16

0

0

11

1

1

2

1

9

3

4

2

16

16

9

13

2

28

5

2

9

5

1

1

4

10

11

9

0

1

(81%)

(13%)

(13%)

(0%)

(38%)

(56%)

(100%)

(0%)

(0%)

(69%)

(6%)

(6%)

(13%)

(6%)

(56%)

(19%)

(25%)

(13%)

(100%)
















No





No

Not Sure






Mathematics


Science

Grade of Licensure

Technology

Elementary School


High School


All Levels

Yes

No

Some Areas

Unknown and N/A




MMSP Partnership: Salem State College


Table 12: Salem State College Participant Background Information

Year 1

55

43 (78%)

11 (20%)

15 (27%)

N/A

39 (71%)

19 (35%)

51 (93%)

N/A

N/A

15 (27%)

0

18

19

16

22

14

3

16

(40%)

(25%)

(5%)

(29%)

(0%)

(33%)

(35%)

(29%)

12 (22%)

52 (95%)

88

24

8

4

23

24

4

1

35

N/A

11

2

4

1

50

130

49

23

3

43

25

7

3

39

0

46

0

8

0

54


Year 2 Year 3

80

67 (84%)

10 (13%)

10 (13%)

4 (5%)

64 (80%)

16 (20%)

69 (86%)

1 (1%)

0

N/A

(0%)

1

29

33

17

51

15

2

11

(64%)

(19%)

(3%)

(14%)

(1%)

(36%)

(41%)

(21%)

26 (33%)

78 (98%)


92

75 (82%)

7 (8%)

7 (8%)

2 (2%)

78 (85%)

11 (12%)

73 (79%)

4 (4%)

0

N/A

(0%)

2

31

36

20

74

8

6

2

(80%)

(9%)

(7%)

(2%)

(2%)

(34%)

(39%)

(22%)

47 (51%)

87 (95%)

Total to Date

162

72

128 (79%)

22 (14%)

20 (12%)

3 (2%)

129 (80%)

28 (17%)

131 (81%)

5 (3%)

0

N/A

(0%)

110

23

8

18

3

49

67

39

(68%)

(14%)

(5%)

(11%)

(2%)

(30%)

(41%)

(24%)

64 (40%)

157 (97%)

156

56

59

5

79

8

3

2

24

3

81

4

9

0

36

277

95

76

10

123

19

9

11

61

4

110

6

17

0

94

25


MMSP Partnership: Springfield/Holyoke Partnership


Table 13: Springfield/Holyoke Participant Background Information














No





No

Not Sure






Mathematics


Science

Grade of Licensure

Technology

Elementary School


High School


All Levels

Yes

No

Some Areas

Unknown and N/A

Year 1

38

34 (89%)

2 (5%)

0 (0%)

N/A

4 (11%)

36 (95%)

38 (100%)

N/A

N/A

7 (18%)

1

3

20

10

8

11

12

6

(21%)

(29%)

(32%)

(16%)

(3%)

(8%)

(53%)

(26%)

12 (32%)

34 (89%)

73

19

43

4

10

6

16

2

4

N/A

1

0

51

0

7

96

35

51

1

11

26

8

2

2

75

0

6

2

0

1


Year 2 Year 3

48

32 (67%)

5 (10%)

1

0

3

(2%)

(0%)

(6%)

41 (85%)

48 (100%)

0 (0%)

0

N/A

(0%)

1

19

6

22

30

3

7

7

(63%)

(6%)

(15%)

(15%)

(2%)

(40%)

(13%)

(46%)

16 (33%)

47 (98%)


49

37 (76%)

7 (14%)

4

0

4

(8%)

(0%)

(8%)

42 (86%)

47 (96%)

0 (0%)

2

N/A

(4%)

4

22

14

13

32

3

7

3

(65%)

(6%)

(14%)

(6%)

(8%)

(45%)

(29%)

(27%)

20 (41%)

46 (94%)

Total to Date

97

42

72 (74%)

11 (11%)

5

0

8

(5%)

(0%)

(8%)

82 (85%)

94 (97%)

0 (0%)

1

N/A

(1%)

4

33

26

34

50

13

16

14

(52%)

(13%)

(17%)

(14%)

(4%)

(34%)

(27%)

(35%)

30 (31%)

91 (94%)

79

21

43

4

30

8

8

3

4

51

3

11

3

0

0

166

40

95

7

47

24

20

6

3

107

3

21

8

0

1




MMSP Partnership: Wareham Public Schools

Table 14: Wareham Participant Background Information















No





No

Not Sure






Mathematics

Science




High School

All Levels

Yes

No

Some Areas

Unknown and N/A

57

32

N/A

8

0

39

9

3

5

25

9

2

0

37

32 (86%)

5 (14%)

20 (54%)

N/A

19 (51%)

4 (11%)

17 (46%)

N/A

N/A

6 (16%)

14 (38%)

11

0

12

0

(30%)

(0%)

(32%)

(0%)

5

15

16

(14%)

(41%)

(43%)

10 (27%)

Year 1



43

12

37 (86%)

5 (12%)

22 (51%)

1 (2%)

18 (42%)

2 (5%)

23 (53%)

1

0

(2%)

(0%)

22

N/A

(51%)

11

0

9

1

(26%)

(0%)

(21%)

(2%)

12

22

9

(28%)

(51%)

(21%)

10 (23%)

43 (100%)

34

19

7

6

2

13

0

8

1

13

4

0

1

18

13 (72%)

4 (22%)

5 (28%)

0 (0%)

12 (67%)

0 (0%)

11 (61%)

1

0

(6%)

(0%)

12

N/A

(67%)

3

0

2

1

(17%)

(0%)

(11%)

(6%)

6

11

1

(33%)

(61%)

(6%)

6 (33%)

36 (97%) 18 (100%)


68

47

10

6

4

38

0

13

1

41

2

0

0




MMSP Partnership: Worcester Polytechnic Institute


Table 15: WPI Participant Background Information














No





No

Not Sure






Mathematics


Science

Grade of Licensure

Technology

Elementary School


High School


All Levels

Yes

No

Some Areas

Unknown and N/A

Year 1

24

21 (88%)

0 (0%)

1 (4%)

N/A

21 (88%)

2 (8%)

15 (63%)

N/A

0

N/A

(0%)

3

4

13

5

13

6

1

1

(54%)

(25%)

(4%)

(4%)

(13%)

(17%)

(54%)

(21%)

13 (54%)

24 (100%)

42

12

8

2

17

4

1

2

15

N/A

19

0

1

0

18

114

50

25

10

39

20

2

2

25

0

39

4

8

0

29


Year 2 Year 3

66

55 (83%)

8 (12%)

8 (12%)

1 (2%)

56 (85%)

8 (12%)

27 (41%)

2 (3%)

0

N/A

(0%)

1

24

22

19

42

11

4

8

(64%)

(17%)

(6%)

(12%)

(2%)

(36%)

(33%)

(29%)

25 (38%)

63 (95%)


74

61 (82%)

6 (8%)

7 (9%)

2 (3%)

67 (91%)

9 (12%)

32 (43%)

5 (7%)

0

N/A

(0%)

2

22

26

25

62

6

4

0

(84%)

(8%)

(5%)

(0%)

(3%)

(30%)

(35%)

(34%)

48 (65%)

71 (96%)

Total to Date

145

47

121 (83%)

11 (8%)

16 (11%)

3 (2%)

121 (83%)

18 (12%)

66 (46%)

5 (3%)

1

N/A

(1%)

105

18

8

11

3

51

46

44

(72%)

(12%)

(6%)

(8%)

(2%)

(35%)

(32%)

(30%)

73 (50%)

138 (95%)

128

65

16

6

62

9

3

0

34

1

54

4

4

1

40

244

104

44

17

117

16

7

5

65

2

94

7

14

1

76




MMSP Partnership: Massachusetts College of Liberal Arts (MCLA) Math

Table 16: MCLA Math Participant Background Information


Year 3













No



Have a HOUSSE Plan


Yes

No

Not Sure


Hold One or More Teaching Licenses 11 (100%)


Total Number of Licenses Held 19

License Area


Elementary


Mathematics

Middle School Math/Science

Science



High School

All Levels

Yes

No

Some Areas

Unknown and N/A

3

1

5

2

2*

2

1

2

3

12

2

0

4

*One participant was licensed in some areas taught as of the first course taken in Year

3, but was not licensed in any areas taught as of the last course in Year 3 because content areas taught changed between the first and last course.

9

3

1

1

0

1

9

1

0

9

0

0

2

4

0

0

11

5

8

1

(73%)

(9%)

(0%)

(0%)

(82%)

(27%)

(9%)

(9%)

(0%)

(82%)

(0%)

(9%)

(0%)

(9%)

(82%)

(0%)

(18%)

(36%)




MMSP Partnership: University of Massachusetts Amherst

Table 17: University of Massachusetts Amherst Participant Background Information













No





No

Not Sure






Mathematics


Science




High School

All Levels

Yes

No

Some Areas

Unknown and N/A

36

5

6

11

7

2

1

11

1

7

0

2

16

8

(74%)

(14%)

(29%)

(3%)

(69%)

(6%)

(37%)

(0%)

(0%)

(71%)

(3%)

(3%)

(14%)

(9%)

(34%)

(31%)

(23%)

(43%)

10

1

24

2

35

26

5

13

0

0

25

1

1

5

3

12

11

8

15

Year 2

(76%)

(20%)

(16%)

(16%)

(68%)

(8%)

(64%)

(0%)

(0%)

(60%)

(16%)

(0%)

(16%)

(8%)

(52%)

(28%)

(16%)

(24%)

17

2

4

4

25

19

5

16

0

0

15

4

0

4

2

13

7

4

6

21



(84%) 33 (94%)


64

13

6

27

3

0

5

20

0

19

1

3

28

16

13

5

35

3

53

34

41

8

25

0

0

38

5

0

4

6

20

16

13

19

48

(77%)

(15%)

(25%)

(9%)

(66%)

(6%)

(47%)

(0%)

(0%)

(72%)

(9%)

(0%)

(8%)

(11%)

(38%)

(30%)

(25%)

(36%)

(91%)

83

16

9

38

9

1

5

29

1

21

1

4

39

18




Progress Toward Meeting MMSP Goals

MMSP partnerships strive to achieve five major goals. This section presents data collected by the

Donahue Institute at the state-level that illustrates the progress being made toward these goals.

Additional data collected at the partnership level, such as student achievement data, are not collected by the Donahue Institute and, therefore, are not included in this report.

Goal I.

Develop and implement an effective and sustained course of study for current teachers of grades 4-8 mathematics and/or grades 4-12 science and technology/engineering.

Over the span of Year 1 through Year 3, MMSP partnerships developed and implemented a total of 85 courses. Of those 85 courses, 51 (60%) were unique, and 34 (40%) were repeat offerings. Of the 85 courses, 76 (89%) of the courses offered mathematics content, eight (9%) offered science content, and one (1%) offered technology/engineering content.

Partnerships were very successful at encouraging and retaining repeat participants. All ten partnerships offered multiple courses, and all had participants who attended more than one course. In all, 354 participants attended multiple courses. Eleven participants took courses from multiple partnerships.

Table 18 provides details regarding repeat participation.

Table 18: Repeat Participants by Partnership

Partnership

Number of

Courses

Offered to

Date

7

Total Number of Unique*

Participants to

Date

130

Number

Taking

Multiple

Courses

49 EduTron

Harvard Graduate

School of Education

Lesley University

MCLA - Science

Salem State College

Springfield PS

Wareham PS

Worcester

Polytechnic Institute

MCLA - Math

7

18

2

26

7

3

6

156

107

16

162

97

43

145

10

85

9

72

42

12

47

2 11 5

UMass Amherst 7 53 34

* Unique refers to status within each individual partnership. Eleven participants took courses across partnerships.




In total, the partnerships served 909 participants. Course attrition rates were generally low and averaged under 4.5%. Of the 85 courses, 42 had an attrition rate of 0%, 22 had an attrition rate ranging between

2% and 10%, and 19 had an attrition rate of greater than 10%. (It was not possible to determine the attrition rate for two of the courses.) Table 19 provides a breakdown, by course, of enrollment and attrition rates.

Table 19: Enrollment and Attrition Information as Reported by Each MMSP Partnership

Partnership Year

1

Course Title

Building a Solid Bridge from

Arithmetic to Algebra

Linear Equations and Their Foundations

Number of

Participants

Enrolled First

Day

35

1

2

41

27

EduTron

2

2

3

3

Building a Rock Solid Math Foundation

An Odyssey to Algebra, Geometry and

Back

A Further Odyssey to Algebra,

Geometry and Back

Perspectives on Elementary

Mathematics

Perspectives on Middle School

Mathematics

35

35

21

41

Subtotal 235

1

2

Conceptually Challenging Topics in

Middle School Math (Making the Case )

Making the Case Ib

28

29

2 Making the Case Ic 24

31

Harvard

3 Making the Case I

3 Making the Case I

Summary – Making the Case I

2 Making the Case II

33

145

15

34

Lesley

University

3 Making the Case I (SPED)

Subtotal

1 Math as a Second Language 1

Math as a Second Language 2

1

1

Math as a Second Language 3

Summary – Math as a Second Language

1

Functions and Algebra – cohort 1

194

25

36

41

102

19

Number of

Participants

Completed

Course

35

40

27

32

33

20

40

227

23

170

25

34

40

99

19

32

123

15

32

27

24

17

Attrition Rate

0%

2%

0%

9%

6%

5%

2%

3%

18%

7%

0%

45%

12%

0%

6%

2%

3%

0%

3%

15%

0%

6%





Partnership

Lesley

University

(continued)

Year Course Title

2 Functions and Algebra – cohort 2

2 Functions and Algebra – cohort 3

Summary – Functions and Algebra

2

Trigonometry, Algebra, and Analytic

Geometry – cohort 1

2

Trigonometry, Algebra, and Analytic


Summary – Trigonometry, Algebra, & Geometry

2 Number Theory – cohort 1

2 Number Theory – cohort 2

Summary – Number Theory

2 Statistics and Data Analysis – cohort 1



Summary – Statistics and Data Analysis

3 Measurement and Probability – cohort 1

3 Measurement and Probability – cohort 2

Summary – Measurement and Probability

3 Calculus I – cohort 1

3 Calculus I – cohort 2

Summary – Calculus I

3 Calculus II

Subtotal

Number of

Participants

Enrolled First

Day

27

41

87

34

46

47

69

27

28

55

9

541

17

23

72

22

80

36

31

67

32

15 2 Conceptual Physical Science

MCLA

Science

3 Conceptual Biology

Salem State

College

Subtotal

1

2

Data, Probability, & Statistics for ES &

MS Teachers

Data, Probability, and Statistics for MS

Teachers

3

Data, Probability, and Statistics for MS

Teachers

Summary – Data, Probability, & Statistics

1 Number Systems for MS Teachers



11

26

23

13

19

55

13

Number of

Participants

Completed

Course

25

41

85

34

23

11

19

53

13

46

17

23

72

21

80

36

31

67

32

47

68

27

28

55

9

535

14

9

23

Attrition Rate

7%

0%

2%

0%

0%

15%

0%

4%

0%

0%

0%

1%

0%

0%

0%

0%

1%

0%

0%

0%

0%

0%

0%

0%

0%

5%

7%

18%

12%

33



Partnership

Salem State

College

(continued)

Year

2

2

Course Title

Number Systems for MS Teachers

Number Systems for MS Teachers

3 Number Systems for MS Teachers

Summary – Number Systems

1

History of Mathematics for MS

Teachers

2 History of Math for MS Teachers

3 History of Math for MS Teachers

Summary – History of Mathematics

Patterns, Relations & Algebra for MS

1

Teachers

2

Patterns, Relations, and Algebra for MS

Teachers

2


Teachers

3


Teachers

Summary – Patterns, Relations, & Algebra

Geometry & Measurement for MS

1

Teachers – Spring

1


Teachers – Summer

2


Teachers

3


Teachers

Summary – Geometry & Measurement

2

3

PreCalculus for MS Teachers

PreCalculus for MS Teachers

3 PreCalculus for MS Teachers

Summary – Precalculus for MS Teachers

2 Discrete Mathematics for MS Teachers

3 Discrete Mathematics for MS Teachers

Summary – Discrete Mathematics for MS Teachers

Number of

Participants

Enrolled First

Day

30

7

12

14

33

19

18

21

10

6

9

21

52

12

20

51

22

10

22

17

67

7

18

76

15

Number of

Participants

Completed

Course

30

7

8

21

49

12

17

47

20

9

21

17

67

15

14

31

17

17

67

7

10

16

17

9

6

Attrition Rate

0%

0%

0%

0%

0%

17%

0%

6%

11%

11%

19%

6%

12%

0%

10%

0%

15%

8%

9%

11%

0%

6%

0%

10%

5%





Partnership

Salem State

College

(continued)


2 Calculus for MS Teachers

3 Calculus for MS Teachers

Summary – Calculus for MS Teachers

3 Linear Systems for MS Teachers

Subtotal

1 Physics

Number of

Participants

Enrolled First

Day

16

14

30

21

407

24

Number of

Participants

Completed

Course

14

13

27

20

382

24

Attrition Rate

13%

7%

10%

5%

6%

0%

1 Chemistry 25 25 0%

2 Life Science

Springfield

PS

Wareham PS

2

3

3

Earth Science

Math for Science Teachers


3

Subtotal

Integrated Science

1

2

Improving Math Teaching,

Gr. 4-8

Topics in Mathematics for Teachers:

Improving Math Teaching Part II

2 Developing Teacher Portfolios

29

31

28

25

34

196

38

16

7

29

31

28

25

34

196

37

14

7

0%

0%

0%

0%

0%

0%

3%

13%

0%

61

Unknown

58

29

5%

Unknown

Subtotal

1 Geometrical Concepts

3

Geometry (same as Geometrical

Concepts course)

Summary – Geometry

1 Discrete Mathematics

WPI

2 Algebra for Middle School Teachers

2

3

Probability and Statistics

Middle School Math and MCAS Studies

Subtotal

MCLA-Math 3 Problem Solving and Numeration



37

Unknown

Unknown

68

31

80

Unknown

8

57

23

63

222

6

29

58

21

22%

Unknown

Unknown

16%

26%

21%

Unknown

25%

35



Partnership

MCLA-Math

(continued)

UMass

Amherst


3 Patterns, Relations, and Algebra

Subtotal

2 Algebra – cohort 1

3 Algebra – cohort 2

Summary - Algebra

2 Geometry – cohort 1

3 Geometry – cohort 2

Summary – Geometry

3 Algebra B (mini course)

3

3

Trigonometry – cohort 1

Discrete Mathematics – cohort 1

Subtotal

All

Partnerships

TOTAL

* Excluding data for WPI partnership, which were not available.

Number of

Participants

Enrolled First

Day

8

16

17

23

40

20

18

38

8

13

17

116

1792*

Number of

Participants

Completed

Course

8

14

16

21

37

20

15

35

5

13

17

107

1934

Attrition Rate

0%

13%

6%

9%

8%

0%

17%

8%

38%

0%

0%

8%

5%*




Goal II.

Increase the number of teachers currently employed in the partnership school districts who are licensed in the areas they teach and/or have completed their HOUSSE plans.

Goal II addresses the spirit of the federal NCLB legislation regarding teacher licensure, professional development, and competency in subject area taught. To comply with the federal NCLB legislation, public school teachers were required to meet the federal definition of highly qualified by the end of the

2005-2006 school year. One of the expectations of the MMSP is to provide high quality professional development that would allow teachers to attain federal highly qualified status.

Information regarding the following areas was used to determine participants’ highly qualified status: licensure, years in education, subject areas taught, HOUSSE plans held, Professional Development

Points (PDPs) held, MTELs passed, degrees held, undergraduate degree equivalents, and advanced or national certifications.

To be considered highly qualified, a teacher must be licensed and demonstrate subject matter competency in the areas of teaching. Demonstration of subject matter competency for elementary teachers was satisfied either by passing the appropriate MTEL (general curriculum MTEL if teaching multiple core subjects or elementary math MTEL if teaching mainly elementary math) or by having a

HOUSSE plan and having earned a particular number of PDPs. Demonstration of subject matter competency for middle and secondary school teachers was satisfied by one of the following means: passing the appropriate MTEL, completing an appropriate undergraduate major or graduate degree, completing appropriate coursework comparable to an undergraduate major, holding advanced or national certification in the appropriate subject area, or by having a HOUSSE plan and having earned a particular number of PDPs. For all teachers with HOUSSE plans, the minimum numbers of PDPs needed varied in relation to the date of June 30, 2006: Prior to June 30, 2006, the minimum was 48, and following June 30, 2006, the minimum was 96. Appendix H outlines options available for demonstrating subject matter competency.

A participant was identified as highly qualified if the criteria for meeting highly qualified status were met for all subjects that a participant taught. If a participant taught multiple subject areas and only met the highly qualified criteria for some of the subjects taught, he or she was determined to be “highly qualified in some, but not all” content areas.

In Year 1, the Participant Background Survey did not adequately capture information about teachers that could be used to determine highly qualified status. The survey was re-designed for Year 2 to capture this information. Year 2 survey responses indicate, though, that participants misunderstood the meaning of HOUSSE plans, and some who completed more than one survey reported inconsistent data across surveys.

The Participant Background Survey permitted determination of the impact of MMSP courses on highly qualified status only for 1) those participants who held HOUSSE plans and 2) those participants who took more than one course (i.e., completed more than one survey). Because surveys were administered after participants had completed MSP courses, if a participant completed only one course and did not have a HOUSSE plan it was not possible to determine whether that participant became highly qualified prior to or as a result of MMSP course participation.

Table 20 presents the highly qualified status of participants across all years of the program. This table provides an unduplicated count of participants. Because the number of courses a participant took was relevant to the process used to determine how many participants attained highly qualified status while



Annual Report of the MMSP State-level Evaluation MMSP Participant Background Data participating in MMSP, the findings are primarily organized according to number of courses taken

(only one vs. multiple). In Appendix I, a more detailed version of the table is presented in which the data are further broken down by year of participation and cohort.

Table 20: Highly Qualified Status of Unique Participants

Status

Took only one course

n = 555

Took multiple courses

n = 354

TOTAL

N = 909

Became Highly Qualified

Highly Qualified in some content areas but not all

Highly Qualified but unable to determine when became HQ

Not Highly Qualified

Private school or not teaching

Unknown

19

17

321

83

34

92

19

166

23

17

111

36

487

106

51

81 37 118

For the first three years of the program, at least 111 participants attained highly qualified status. (This was 13% of all participants who were public school teachers.) It is likely that more participants gained highly qualified status between the time they took their first course and completed their most recently completed course, but the survey approach did not permit capturing the time when all participants became highly qualified. Of the 111 who attained highly qualified status, 70 did so by completing a sufficient number of PDPs on their HOUSSE plans, 16 did so by passing the appropriate MTEL, five did so by obtaining a degree in content areas, three did so by obtaining undergraduate equivalents in content areas, and three did so by earning a teaching license. In addition, 14 participants did so by meeting two or more criteria simultaneously. An identification of the criteria by which teachers who gained highly qualified status demonstrated competency in their subject matter may be found in

Appendix J.

Table 21 shows the minimum number of participants who gained highly qualified status within each partnership. Again, it is likely that more participants gained highly qualified status than are indicated.




Table 21: Number of Unique Participants Gaining Highly

Qualified Status, by Partnership

Partnership

EduTron

Number of Unique

Participants Gaining

Highly Qualified

Status*

13

Harvard Graduate School of Education 14

Lesley University

MCLA – Science

Salem State College

Springfield Public Schools



MCLA – Mathematics

36

3

17

13

4

9

1

University of Massachusetts Amherst 4

TOTAL 111**

* Because it was not possible to determine when highly qualified status was attained for a large number of participants, these figures are probably lower than actuality.

** 3 participants are each counted twice as they took courses in two partnerships

MTEL Information

One method by which teachers may demonstrate subject-matter competency is to pass the

Massachusetts Tests for Educator Licensure (MTEL) in the content areas that they teach. Table 22 identifies the tests taken by public school teachers in each year of the study along with passage rates. Of the 354 participants taking multiple courses, 13% passed an MTEL test and 11% earned new licenses by the end of the third year of MMSP.

As shown in Table 22, based on data from the last survey completed by each participant, of the 92 participants who had taken the Mathematics MTEL, 70 (76%) reported passing the test, and two (2%) had not yet received their scores at the time of survey completion. Of the 190 respondents who had taken the Middle School Mathematics MTEL, 161 (85%) passed and nine (5%) had not yet received their scores. Of the 25 participants who completed the Middle School Mathematics/Science MTEL, fifteen (60%) passed and four (16%) had not yet received their scores. Of the 35 participants completing the General Science MTEL, 31 (89%) passed. Three participants both took and passed the

Chemistry MTEL, and two both took and passed the Earth Science MTEL. Eight respondents attempted the Biology MTEL, and five (63%) passed. One individual completed the Technology/Engineering

MTEL, but did not indicate whether or not it was passed. Of the 80 participants who reported taking

General Curriculum (formerly elementary) MTEL, 73 (91%) reported passing.




Table 22: MTEL Tests Taken by MMSP Participants – Total to Date

Based on each participant’s last survey

Taking Test Passing Test n n %

Failing Test

n %

Scores

Unknown n %

No Response*

n %

General Curriculum

(formerly Elementary)


Mathematics

Middle School Mathematics

Middle School

Mathematics/Science

General Science

Biology

Chemistry

Physics

Earth Science


80

15

92

190

25

35

8

3

0

2

1

73

12

70

161

15

31

5

3

0

2

0

91%

80%

76%

85%

60%

89%

63%

100%

0%

100%

0%

5

0

17

15

4

1

1

0

0

0

0

6%

0%

18%

8%

16%

3%

13%

0%

0%

0%

0%

0

3

2

9

4

1

0

0

0

0

0

0%

20%

2%

5%

16%

3%

0%

0%

0%

0%

0%

2

0

3

5

2

2

2

0

0

0

1

3%

0%

3%

3%

8%

6%

25%

0%

0%

0%

100%

TOTAL in STE Areas 74 56 76% 6 8% 5 7% 7

HOUSSE Professional Development Plan Status

Teachers may demonstrate subject-matter competency through a HOUSSE plan (see Appendix H). As shown in Table 23, at the end of the third year of the program, many public school participants (27%) were unsure whether or not they had a HOUSSE plan, and an additional 4% did not answer that survey question. Only 44% of the 354 participants who took more than one course were consistent across surveys in their responses to this item, indicating that many were confused about the issue. (All survey respondents (not just public school teachers) had been asked to respond to the item on HOUSSE plan status.) In all, only 38% of respondents working in public schools indicated that they had HOUSSE plans.

According to the guidelines established by the MADOE, in the 2003-2004 academic year, teachers needed 48 or more PDP hours in the content areas they teach to be considered highly qualified. For the

2005-2006 academic year, teachers were required to complete at least 96 content hours of their

HOUSSE plan to attain highly qualified status. Information regarding PDPs was requested from only those participants who reported that they had a HOUSSE plan. At the end of the third year of the program, of the 332 public school teachers indicating they had a HOUSSE plan, 76% reported having earned 48 or more PDP hours, 10% reported having fewer than 48 PDP hours, and 14% did not respond to the question.

9%




Table 23: HOUSSE Plan Status of Unique Public School Teachers

Year 1

N % N

Year 2

%

Year 3

N %

Total to Date

N %

Do you have a High Objective Uniform State Yes

Standard of Evaluation (HOUSSE) plan?

No

Not Sure

No Response

75 23%

142 43%

98 30%

17 5%

177 40%

133 30%

127 28%

9 2%

196

137

106

16

43%

30%

23%

4%

332 38%

285 32%

237 27%

29 3%

If you do have a HOUSSE Fewer than 48 PDP hours

Plan, how many PDP hours do 48 to 96 PDP hours you have in your content area(s)?

97 or more PDP hours

No Response

9 12%

6 8%

46 61%

14 19%

11

19

6%

11%

115 65%

32 18%

17

13

136

30

9%

7%

69%

15%

32 10%

22 7%

231 70%

47 14%

Mathematics, Science and Technology Licensure in Content Area Taught

By the end of Year 3, 147 regular education teachers and 22 special education teachers reported teaching in science or technology areas. For regular education teachers, the licensing reported by approximately 35% appeared to be appropriate for the level taught. For special education teachers, the licensing reported by approximately 3% appeared to be appropriate for the level taught. Tables 24a and

24b show how many teachers taught in each science and technology area for Year 1, Year 2, Year 3,and over the course of all years, and the tables show how many teachers reported being licensed in the area in which they taught. Table 24a provides information for regular education teachers and Table 24b provides information for special education teachers. (The numbers presented in Tables 24a and 24b exceed the number of teachers who reported teaching in these areas because some teachers taught in more than one area.)

Table 24a: MMSP Science and Technology Teaching Areas —Regular Education

Content Area

Teach in

Area n

General Science 49

Biology 17

Earth Science

Physics

17

11

Chemistry

Technology

11

9

Year 1

Licensed in

Area Taught

Teach in Area n n %*

20 40.8% 58

12 70.6% 17

0 0.0% 11

2 18.2% 11

3 27.3% 9

0 0.0% 1 n

28

12

0

0

1

1

Year 2

Licensed in

Area Taught

Teach in Area n %*

48.3% 46

70.6% 13

0.0% 10

0.0% 9

11.1% 10

100.0% 4 n

20

11

1

2

5

3

Year 3

Licensed in

Area Taught

Teach in Area n %*

43.5% 103

84.6% 29

10.0% 22

22.2% 15

50.0% 17

75.0% 8

*Of the number of regular education participants teaching in this area for this year

Total

Licensed in Area

Taught n

32

21

1

2

8

3

%*

31.1%

72.4%

4.5%

13.3%

47.1%

37.5%




Table 24b: MMSP Science and Technology Teaching Areas —Special Education

Content

Area

Teach in Area n

Year 1

Licensed in

Area Taught n %*

Teach in Area n

Year 2

Licensed in

Area Taught n %*

Teach in Area n

Year 3

Licensed in

Area Taught n %*

Teach in Area n

Total

Licensed in

Area Taught n %*

Gen. Science 10 1 10.0% 8 1 12.5% 7 0 0.0% 16 0 0.0%

Biology 4 0 0.0% 2 0 0.0% 3 0 0.0% 7 0 0.0%

Earth Science 4 0 0.0% 1 0 0.0% 4 0 0.0% 5 0 0.0%

Physics 0 0 0.0% 1 0 0.0% 1 0 0.0% 1 0 0.0%

Chemistry

Technology

1

0

0 0.0% 1 0 0.0%

0 0.0% 0 0 0.0%

1

0

0 0.0% 2 1 50.0%

0 0.0% 0 0 0.0%

* Of the number of special education participants teaching in this area for this year

** The “Total” number may be smaller than the sum of Years 1 through 3 because the Total is calculated from each participant’s last completed survey, and individuals may have changed teaching areas or not reported licensure in a given area on that survey.

By the end of Year 3, 426 regular education teachers and 69 special education teachers reported teaching mathematics. For regular education teachers, the licensing reported by approximately 67% appeared to be appropriate for the level taught. For special education teachers, the licensing reported by approximately 18% appeared to be appropriate for the level taught. Tables 25a and 25b provide a breakdown of each math level for Year 1, Year 2, Year 3, and for the total over the course of all years, with Table 25a including information for regular education teachers and Table 25b including information for special education teachers.

Table 25a: MMSP Mathematics Teacher Levels —Regular Education

Content Area

Teach in Area n

Year 1

Licensed in Area

Taught n %*

Teach in Area n

Year 2

Licensed in

Area Taught n %*

Teach in Area n

Year 3

Licensed in Area

Taught n %*

Teach in Area n

Total

Licensed in Area

Taught n %*

Middle School

High School

MS & HS grades

122 57 46.7% 187 103 55.1% 178 131 73.6% 354 223 63.0%

12

1

9 75.0% 17 13 76.5% 39 36 92.3% 59 52 88.1%

1 100.0% 3 2 66.7% 1 0 0.0% 4 3 75.0%

TOTAL Math 135 67 49.6% 207 118 57.0% 218 167 76.6% 417 278 66.7%

*Of the number of regular education participants teaching in this area for this year




Table 25b: MMSP Mathematics Teacher Levels —Special Education


Content Area

Teach in Area n

Middle School

High School

20

3

MS & HS grades 0

TOTAL Math 23

Year 1

Licensed in Area

Taught n %*

2

1

0

3

Teach in Area n

10.0% 23

33.3% 4

0.0% 3

13.0% 30

Year 2

Licensed in

Area Taught n %*

4

3

1

8

Teach in Area n

17.4% 32

75.0% 3

33.3% 1

26.7% 36

*Of the number of special education participants teaching in this area for this year

Year 3

Licensed in Area n

Taught

%*

Teach in Area n

5 15.6% 53

1 33.3% 7

0 0.0% 2

6 16.7% 62

Total

Licensed in Area

Taught n %*

7

3

1

13.2%

42.9%

50.0%

11 17.7%




Goal III.


Increase the number of highly qualified teachers in mathematics, science, and/or technology/engineering by integrating the courses of study into schools of education and/or arts and sciences at institutes of higher education.

For systemic change to occur at the higher education institutions, Departments of Arts and Sciences and

Education Departments are encouraged to work together through MMSP to support stronger content courses in mathematics and science for teacher preparation, undergraduate and graduate degree requirements, and for in-service teachers pursuing graduate-level content courses for recertification.

Integration of Title IIB courses into graduate programs at Institutes of Higher Education will ensure sustainability over time. The intent behind encouraging the partnerships is that the faculty from the Arts and Sciences Departments will bring strong content expertise to the partnership table. This integration will create greater opportunities for participants to complete coursework leading to a content-area degree and/or to licensure along with the highly qualified designation.

While this report focuses on Year 3 of the MMSP, institutional shift of the type the MMSP intends to promote requires time to occur and build upon itself over time. For Year 3, because partnerships were asked to report on integration activities only for Year 3 (and not for the entire duration of their projects) in order to convey as thorough of a sense as possible of the impact the MMSP has had on institutionalization, information related to it that has been gathered over the course of all three years will be presented here, chronologically, for each individual year.

At the end of Year 1, partnerships reported very general information about integration, or they reported specific percentages of integration into higher education institutions that were unclear. All partnerships reported progress toward meeting this goal, though. At the end of Year 1, the following five partnerships reported that they had integrated at least one course into their higher education institute:

Salem, Lesley, Wareham, Springfield, and WPI.

For Year 2, partnerships were asked to report more detailed information regarding how integration of the Title IIB courses was occurring, and they were provided with more guidance about what to report.

Specifically, they were asked to complete the following five items in the context of an Annual Report

Addendum:

1.

Which course(s), if any, have been embedded into existing undergraduate or graduate programs at your MMSP higher education partner? Please specify the department (education, math, science) into which the course(s) have been embedded.

2.

Which course(s), if any, have been integrated into the Continuing education program at your

MMSP higher education partner? Please describe the types of course credit and/or PDPs available to those taking the course through the continuing education program.

3.

Has any part of your MMSP Course curricula (i.e., case studies, assessments, materials) been integrated into other existing programs or courses at your MMSP higher education partner? If yes, please describe.

4.

Has your MMSP higher education partner created a new degree program based on the courses developed for your MMSP program? If yes, please describe.

5.

Describe any other steps you have made to integrate your MMSP course of study at your

MMSP higher education partner.

Integration or plans for integration in the future were evident from the nine partnerships that were actively providing courses during Year 2. As would be expected in a program involving partnerships with diverse structures and styles, the extent and type of integration varied across partnerships. To



Annual Report of the MMSP State-level Evaluation MMSP Participant Background Data convey a sense of how integration occurred, following are summaries for each partnership that was actively providing courses.



All five MMSP courses that EduTron created were offered through the Center for Professional

Studies at Fitchburg State College. All courses were offered to in-service teachers for graduate credit and PDPs.



Harvard used some of the case studies that were created through MMSP with the non-MMSP math and science teacher education students at the Harvard Graduate School of Education.



Prior to MMSP funding, Lesley University did not have a Math or Science Department, but as a result of the availability of Title IIB funds, they brought Dr. Ken Gross from the Vermont

Mathematics Initiative to Lesley to create a brand new Lesley University Center for

Mathematics Achievement, and, therefore, built Lesley's capacity by expanding their math faculty. This also caused institutional change by creating several new degree programs including the following: two new master's degrees with a specialization in elementary or middle mathematics education; a CAGS program for mathematics education to accommodate those teachers already possessing master’s degrees; and three new faculty positions.



The physics course for middle school teachers developed by Massachusetts College of Liberal

Arts (MCLA) Science was offered as part of the undergraduate teacher education curriculum.

 During Year 1 of the program, Salem State College developed a new master’s level teaching program in middle school mathematics. All courses developed by Salem State College through

MMSP can be applied towards earning a degree through that program.



The four science courses offered by the Springfield partnership through the MMSP program have been offered for graduate credit through University of Massachusetts Amherst (UMass

Amherst) Continuing Education. In addition, the chemistry course offered through the partnership also was offered for graduate level credit through the Chemistry Department and both the chemistry and earth science courses were offered for graduate level credit in the master’s level Science Education Program at the School of Education at UMass Amherst.

Materials, activities, and assessments have been used in UMass Amherst Science Education

Online courses and in a course at Hampshire College.



The two math courses developed through the UMass Amherst partnership can be used to fulfill requirements for the UMass Amherst Masters of Education degree and for the CAGS. In addition, they help to fulfill the subject matter requirements for initial licensure in middle school mathematics for the State and National Council for Accreditation of Teacher Education approved UMass Secondary Teacher Education Program.



All three math courses developed through the Wareham partnership have been embedded into existing programs at UMass Dartmouth. They are part of the mathematics graduate and undergraduate programs and also part of the master’s level teaching program. Courses are available to teachers working toward certification and to certified teachers seeking advanced degrees and recertification.

 The four courses created by WPI were embedded into a master’s level graduate program in mathematics education offered by WPI’s Mathematics Department. MMSP funding also aided

WPI in expanding their master’s level graduate program in mathematics education to include middle school level courses when it previously had only included high school level courses.




In addition, to the integration activities that are already occurring as a result of MMSP efforts, four partnerships articulated specific plans for additional future integration activities.

For Year 3, partnerships were asked to describe the activities of their partnerships during the Year 3 funding period that spoke to the “institutionalization” of their courses, the extent to which their courses have been integrated into activities of their higher education partners. The extent and type of integration varied across partnerships. Following are summaries for partnerships that indicated that additional integration was occurring in Year 3:



In Year 3, all MMSP courses that EduTron created were offered through the Center for

Professional Studies at Fitchburg State College. All courses were offered to in-service teachers for graduate credit. In addition, the partnership united Mathematics faculty and Education faculty at Fitchburg State College in a fundamental way: All parties are now working together to improve teacher preparation in mathematics through more stringent requirements in math courses and by improving mathematics offerings. The education department is in the process of hosting a math education summit to articulate the problems and present potential solutions that are arrived at through the joint efforts of math and education faculty.



In addition to continuing with the programs that were created in prior MMSP years, Lesley

University has begun developing an online mathematics program that will offer a master’s degree in elementary or middle school mathematics education. Through needs that have arisen through the classroom implementation component of their participation in MMSP, Lesley also has developed a relationship with Cisco Corporation who will be providing the online platform for the online mathematics program.



Massachusetts College of Liberal Arts (MCLA) Science now offers two new variable credit science for educators courses for in-service teachers.



Massachusetts College of Liberal Arts (MCLA) Math now offers one new variable credit math for educators courses for in-service teachers.

 Salem State College continues to offer courses developed through MMSP as part of a master’s level teaching program in middle school mathematics. All courses developed by Salem State

College through MMSP can be applied towards earning a degree through that program.



The three science courses offered by the Springfield partnership through the MMSP program in

Year 3 were offered for graduate credit through UMass Amherst Continuing Education. A goal of the partnership is to have these courses institutionalized in the UMass Amherst School of

Continuing Education through the School of Education.



All courses developed through the UMass Amherst partnership can be taken for graduate level credits that can be transferred to a variety of graduate level programs at UMass Amherst. In addition, the two courses are being reviewed by the School of Education Academic matters

Committee for permanent course approval.



The one new course created by WPI in Year 3 was embedded into the master’s level graduate program in mathematics education offered by WPI’s Mathematics Department.




Goal IV.


Increase the number of MSTE teachers currently employed in the partner school districts who participate in content-based professional development activities and substantially increase their content knowledge in order to be able to teach effectively the state learning standards.

Content Knowledge Gains

As a grant condition, MMSP partnerships were required to create a pre-course test and post-course test to assess participants’ knowledge of the content for each MMSP course. In most cases, the faculty who developed the course also developed the assessment. For each course, the same instrument served as both the pre-course test and the post-course test. Due to time and resource constraints, partnerships were not required to test their assessments for validity or reliability. Also, partnerships were not encouraged to locate a previously existing standardized instrument that had demonstrated validity and reliability because a priority was placed on developing assessments that would reflect the precise content that would be taught in each of their courses.

Eighty-four of the 85 courses offered across all partnerships to date showed score gains in average percentage of items correct between pre- and post-course test administrations. (Data were analyzed only for participants who completed both pre- and post-course assessments.) To determine if the scores showed statistically significantly changes between pre- and post-course test administrations, a paired samples t -test was used for each course for which ten or more participants completed both pre- and post-course assessments, and a Wilcoxon matched pairs test was used for courses for which fewer than ten participants completed both pre- and post-course assessments.

Statistically significant improvements in scores on content knowledge assessments occurred in 93% of all courses offered through MMSP courses. Table 26 provides information on pre- and post-course average scores, pre-topost changes in average scores, and statistical significance of differences between pre-course and post-course scores.

Three courses offered through one partnership had both treatment (i.e., MMSP) and comparison groups complete the content knowledge assessments. In each course, the comparison group was tested only once. For each course, the treatment group post-course scores were statistically significantly higher than those of the comparison group, although in one case, the treatment group pre-course scores also were statistically significantly higher than those of the comparison group.

Table 26: Mean Percentage Scores for Pre-course & Post-course Tests, Including Gains in Mean Scores

Year

Offered

1

1

Institute Name – Course

EduTron – Building a Solid Bridge

EduTron – Linear Equations

N

37

39

Mean

Pre-test

Arit: 72

Alg: 26

Conc: 12

Com: 46

Mean

Post-test

Arit: 85

Alg: 56

Conc: 60

Com: 66

Change on

Mean

Arit: 13

Alg: 30

Conc: 48

Com: 20 p <.05

Arit: Yes

Alg: Yes

Conc: Yes

Com: Yes

2

2

2

EduTron – Building a Rock Solid Math

Foundation

EduTron – An Odyssey to Algebra,

Geometry and Back

EduTron – A Further Odyssey to

Algebra, Geometry and Back

26

31

31

46

61

54

59

78

80

13

17

25

Yes

Yes

Yes





Year

Offered

Institute Name – Course N

Mean

Pre-test

Mean

Post-test

Change on

Mean p <.05

3

EduTron – Perspectives on Elementary

Mathematics

20 51 73 22 Yes

3

EduTron – Perspectives on Middle

School Mathematics

38 60 84 23 Yes

1 Harvard – Mathematics Case Study 23 54 72 18 Yes

2 Harvard – Making the Case 1b 27 52 63 12 Yes

2

2

Harvard – Making the Case 1c

Harvard – Making the Case II

24

15

65

74

75

81

10

7

Yes

Yes

3

Harvard – Making the Case I

17 50 65 15 Yes

3

Harvard – Making the Case I (SPED)

27 32 47 15 Yes

3

1

1

1

1

2

2

2

2

2

Harvard – Making the Case I

Lesley – Math as a Second Language,

Cohort 1


Cohort 2


Cohort 3

Lesley – Functions and Algebra,

Cohort 1

Lesley – Functions and Algebra – cohort 2

Lesley – Functions and Algebra – cohort 3

Lesley – Trigonometry, Algebra, and

Analytic Geometry – cohort 1

Lesley – Trigonometry, Algebra, and

Analytic Geometry – cohort 2

Lesley – Number Theory – cohort 1

32

22

26

37

16

18

39

31

44

36

57

82

69

73

64

55

53

17

12

29

64

87

76

83

81

79

75

58

59

79

7

5

7

10

17

24

22

41

48

50

No

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

2

2

2

Lesley – Number Theory – cohort 2

Lesley – Statistics and Data Analysis – cohort 1


29

32

15

39

39

29

75

84

79

36

45

51

Yes

Yes

Yes





Year

Offered


Mean

Pre-test

Mean

Post-test

Change on

Mean p <.05

3

3

3

Lesley – Measurement and Probability

– cohort 1

Lesley – Measurement and Probability

– cohort 2


21

44

21

36

42

28

74

71

59

38

29

31

Yes

Yes

Yes

3

Lesley – Calculus – cohort 1

25 27 61 34

Yes

3

Lesley – Calculus – cohort 2

27 35 75 40

Yes

3

Lesley – Calculus II – cohort 1

9 35 65 29

Yes

2

MCLA Science - Conceptual Physical

Science

14 38 55 17

Yes

3

MCLA Science – Conceptual Biology

10 46 55 9 Yes

1

1

1

Salem State – Statistics

Salem State – Geometry (spring session)

Salem State – Number Theory

23

16

13

60

20

65

79

82

71

19

62

6

Yes

Yes

No

1

1

1

2

2

2

2

2

Salem State – History of Math (June session)

Salem State – Geometry (summer session)

Salem State – History of Math (July session)

Salem State – Patterns, Relations, and

Algebra for Middle School Teachers

Salem State – Data, Probability, and

Statistics for Middle School Teachers

Salem State – Number Systems for

Middle School Teachers



Salem State – PreCalculus for Middle

School Teachers

7

9

17

15

11

29

16

20

21

39

59

61

55

39

78

49

81

84

82

80

89

76

86

79

61

45

23

19

34

36

8

29

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes





Year

Offered


Mean

Pre-test

Mean

Post-test

Change on

Mean p <.05

2

2

Salem State – History of Math for




11

7

24

73

89

94

66

21

Yes

Yes

2

2

2

3

3

3

3

3

3

3

3

3

3

Salem State – Discrete Mathematics for


Salem State – Geometry and

Measurement for Middle School

Teachers

Salem State – Calculus for Middle

School Teachers



Salem State – Linear Systems for


Salem State – Precalculus for Middle

School Teachers

Salem State – Precalculus for Middle

School Teachers

Salem State – Calculus for Middle

School Teachers

Salem State – Discrete Mathematics for


Salem State – Geometry and

Measurement for Middle School

Teachers



Salem State – History of Math for


Salem State – Probability and Statistics for Middle School Teachers

12

6

13

17

20

8

21

12

9

17

17

14

19

13

36

22

72

8

60

34

37

13

22

20

24

15

83

81

66

86

87

85

89

69

82

68

76

90

88

71

45

44

14

79

25

55

32

69

45

56

65

72

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

1

Springfield Public Schools – Physics

20 23 38 15 Yes

1

2

Springfield Public Schools – Chemistry

Springfield Public Schools – Life

Science

24

25

53

48

87

71

34

23

Yes

Yes





Year

Offered


Mean

Pre-test

Mean

Post-test

Change on

Mean p <.05

2

3

3

3

1

2

2

1

1

2

2

3

3

3

3

2

2

Springfield Public Schools – Earth

Science

Springfield Public Schools –

Mathematics for Science Teachers

Springfield Public Schools –


Springfield Public Schools – Integrated

Science

Wareham Public Schools – Improving

Math Teaching

Wareham Public Schools – Topics in

Mathematics for Teachers: Improving

Math Teaching Part II

Wareham Public Schools – Developing

Teacher Portfolios

Worcester Polytechnic Institute –

Geometry


Discrete Mathematics




Probability and Statistics


Middle School Mathematics and

MCAS Studies


Geometry

MCLA Math – Problem Solving and

Numeration

MCLA Math – Patterns, Relations, and

Algebra

University of Massachusetts Amherst -

Algebra – cohort 1



31

26

23

34

37

8

7

12

6

57

22

62

26

6

8

14

20

53

40

38

33

67

65

74

52

26

55

41

50

36

59

17

28

52

64

79

74

71

67

76

80

78

47

77

65

58

58

75

57

94

77

11

39

36

38

0

11

6

26

20

23

24

8

22

16

40

66

26

Yes

Yes

Yes

Yes

No

No

No

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes





Year

Offered


Mean

Pre-test

Mean

Post-test

Change on

Mean p <.05

3

3

3

3

3


Algebra – cohort 2



University of Massachusetts Amherst –

Algebra B (mini course)


Trigonometry – cohort 1

University of Massachusetts Amherst –

Discrete Mathematics – cohort 1

20

15

4

13

17

41

60

41

51

30

72

84

94

93

90

32

24

54

42

61

Yes

Yes

No

Yes

Yes




Goal V.

Improve student academic achievement as measured by Massachusetts Comprehensive Assessment

System (MCAS) mathematics and science and technology/engineering assessments and other assessments.

When possible, partnerships collected student achievement data for their local evaluations. There are two major challenges to collecting MCAS data for students of participating teachers. The first is that there are legal and logistical barriers to collecting student-level data from school districts that make it very difficult, if not impossible, to collect data. The other is that, especially in science, MCAS is not administered each year, so partnerships are not able to use the MCAS data consistently across years.

Number of Students Taught

By the end of Year 3 of the MMSP, those who participated in MMSP courses had been teaching over 89,000 students. At the end of Year 3, of the 873 participants who responded to a survey question regarding the number of students taught, 26% reported teaching between 1 and 40 students, 68% reported teaching between 41 and 150 students, and 5% taught over 151 students annually.

Participants also reported the number of Title I, academically advanced, special education, and English language learners they taught annually. During survey administration, some participants reported to their local evaluators that they had difficulty estimating the number of these students they taught annually. Therefore, limited confidence should be placed in this data. Despite concerns with the accuracy of this data, these figures are reported in the tables of all Participant Background Survey results in Appendix F.



Annual Report of the MMSP State-level Evaluation Discussion

Discussion and Recommendations

The MMSP partnership activities described in this report occurred between February 2, 2004 and August 31,

2006. The progress partnerships made at delivering programs is due in large part to the efforts of the partnerships and their local evaluators, and to the work done at the MADOE to support these programs.

Progress has been made towards meeting both the formal and informal goals of the program. The following sections highlight areas where progress has been made and also areas and issues that would benefit from some attention and effort. Specific recommendations are provided for follow-up activities.

Data Quality

While in many instances for this project the data are sound, there are some areas for which this was not the case.

The Donahue Institute became aware that problems existed with some of the data when 1) surveys received from participants taking multiple courses revealed inconsistencies in reporting across surveys for individual participants, especially for items used in determining highly qualified status, and 2) surveys indicated that many participants were confused about whether or not they had a HOUSSE plan.

The Participant Background Survey was modified after Year 1 to improve the likelihood of obtaining good quality data, but similar problems with the data arose again in Year 2. The survey was modified again in Year 3, but some problems still existed, and totals to date will be affected by problems from Year 1 and Year 2; therefore, information regarding the highly qualified status of teachers must be viewed with caution.

Reaching Targeted Participants

As a whole, the partnerships exceeded the MADOE goal of course enrollment consisting of at least 50% of participants from high need districts, and of the ten partnerships providing courses by the end of Year 3, five of them had met that goal as individual partnerships.

The goal to reach the participants who need professional development most was emphasized in summer 2005 by the MADOE in meetings with partnerships. The intended outcome for doing this was to increase the level of participation from high need districts. Two obstacles existed for partnerships when trying to raise the high need district participation. The first was that some participating districts that had been designated as high need districts at the start, when partnerships were formed, were no longer designated as such in Year 2. It was clear that the commitment to the district was a priority to both MADOE and the partnerships, so there was no way to remediate this issue. The other obstacle that existed involved one partnership, in particular, that had committed to one cohort going through multiple courses over the life of the project. For this partnership, it was not possible to change their participants mid-program.



MADOE should continue to remind partnerships that enrollment must include a high percentage of participants from high needs districts at the start of the program.



MADOE should encourage partnerships to recruit additional participants from their high need partners and/or identify additional high need partners, even if they meet the minimum participant rate of 50%.



MADOE has already begun revisiting the procedure by which districts are identified as high need districts. MADOE should continue efforts in this direction and should consider developing criteria to identify individual schools as high need.




Local Evaluation Designs

The Minimum Evaluation Requirements document (see Appendix B) has been the guiding document for providing partnerships with information regarding what they need to do for local and state-level evaluation. This has been the resource most used when providing technical assistance to partnerships.

It was determined early in the initial meetings with partnerships that implementing a rigorous evaluation design – using randomized-controlled trials or strong comparison groups – would be difficult, if not impossible. The limitations that arose were limited resources, the lack of sufficient participant numbers to form reasonable treatment and control groups, and insufficient time to recruit participants into a control group before the start of the program. At the end of Year 2, only one partnership had implemented a quasi-experimental evaluation design.

The remaining partnerships implemented basic evaluation models utilizing pre-post comparisons. To address the difficulties that partnerships have had in creating programs based on a rigorous evaluation design, the RFP for

MMSP programs for 2006-2007 included information on how partnerships might obtain additional support through MADOE and the Donahue Institute to utilize a rigorous approach.



MADOE and the Donahue Institute should continue to work closely with partnerships to provide technical support and resources to conduct rigorous evaluations in the continuation process.

Collecting Student Data

Perhaps the most pervasively problematic issue regarding evaluation of MMSP projects has been difficulty accessing student data. This has been both a logistical and legal challenge for evaluators. Logistically, many school districts do not routinely code their student data by teacher; thus, it is difficult to obtain data aggregated at the participant level. Furthermore, many partnerships anticipate that they will have trouble enlisting the necessary cooperation from participating districts, particularly those with few teachers enrolled in MMSP courses. To date, it appears as if only two partnerships, EduTron and Lesley, have made a concerted effort to obtain student outcome data at the participant (as opposed to school) level.

The expectation to collect student data was both unclear to and unattainable for most partnerships. This led to some difficult conversations between the Donahue Institute staff and partnership staff when reviewing these expectations. Partnerships also encountered problems for meeting the federal reporting requirements regarding student data.



MADOE should give partnerships guidance on forging agreements with their partner districts to have access to student outcome data for participating teachers.

Integrating Courses into Higher Education Institutions

In the first year of MMSP, efforts to determine the extent to which partnerships were able to integrate courses into their partner higher education institutions were hampered by difficulties in obtaining relevant data. In the second and third years of the project, efforts made to define integration and provide partnerships with structured guidance on reporting progress at integration yielded meaningful data. The data provided evidence that integration is occurring across most partnerships. While the extent and types of integration that have occurred varied across partnerships, integration has happened primarily in the following ways that encourage sustainability beyond the duration of the program: embedding MMSP courses into pre-existing programs, integrating courses into preexisting continuing education programs, and creating new degree programs.



MADOE should encourage partnerships to continue efforts to integrate MMSP courses into degree programs of partner higher education institutions.




Steering Committee

The Steering Committee has continued to provide guidance and feedback to the MADOE on issues regarding the

MMSP partnerships, on broader issues of professional development and math, science, technology, and engineering. The MADOE is fully aware how fortunate it is to have such an engaged and invested Steering

Committee for this initiative.



MADOE should continue the role of the Steering Committee providing advice and guidance and make all efforts to follow-up on committee suggestions and requests.



Annual Report of the MMSP State-level Evaluation Appendix A

Appendix A: Timeline for State-level Evaluation and TA Activities

MMSP State-level Evaluation and Technical Assistance Activities Timeline

The following is a summary timeline of state-level evaluation and technical assistance activities carried out between February, 2004, and end of Year 3 of the MMSP.

February 2004 Held Kick-off Meeting for all partnerships and their evaluators at the Department of Education

Spring 2004 Conducted individual partnership meetings with local evaluators and partnership leaders to:

Review the Minimum Expectations document along with the local evaluation and data collection plan

And

Explore potential modifications to implementation plans to create opportunities for experimental or quasi experimental design

Spring 2004 Developed common measures for state-level data collection

June 2004 Attended federal meeting held for MSP projects across the country

Summer 2004 Disseminated and collected end-of-course documents designed to collect course-level data for the statewide evaluation

Fall 2004 Conducted individual partnership meetings with local evaluators and partnership leaders to:

Review the Minimum Expectations document along with the local evaluation and data collection plan

And

Review the Federal Reporting document to ensure the partnerships were collecting the data needed to complete that report

Winter 2005 Conducted partnership meetings with the two new partnerships funded in the second round that constitutes Cohort 2 to:

Introduce the Minimum Expectations document along with the local evaluation and data collection plan

And



Annual Report of the MMSP State-level Evaluation Appendix A

Introduce the Federal Reporting document to ensure the partnerships were collecting the data needed to complete that report

June 2005 Held Technical Assistance Meeting for all partnerships regarding the requirements of the Federal

DOE report

The following activities were on going throughout the life of the project:

Disseminated and collected end-of-course documents designed to collect course-level data for the statewide evaluation

Managed data collected from partnerships at the end of each course

Provided technical assistance to partnerships in support of local partnership evaluation efforts

Monitored local evaluation plans to see they include both formative and summative research questions and corresponding activities

Monitored data collection and analysis around the basic logic model of professional development

Served as liaison to the U.S. Department of Education for evaluation and research issues including participation in national meetings and periodic conference calls

Met with MADOE MSP Team as needed to support integration of evaluation efforts with program goals

Attended MMSP Steering Committee meetings in role of state level evaluator and technical assistance provider.



Annual Report of the MMSP State-level Evaluation Appendix B

Appendix B: Minimum Expectations for Evaluation

The Massachusetts Department of Education (MADOE) has contracted with the UMass Donahue Institute to conduct the statewide evaluation of the Massachusetts Mathematics and Science Partnerships (MSSP) program. In addition, MADOE has required that each of the funded partnerships allocate $5,000 to the Donahue Institute to cover costs associated with providing evaluation-related technical assistance and common data collection instruments to their partnerships. Accordingly, MADOE has the following minimum expectations for the individual partnership evaluations as well as partnership cooperation with the statewide evaluation efforts.



Strong partnership evaluation plans – In contrast to many other grant programs funded through MADOE, it is expected that each partnership will develop its own plan for partnership evaluation. Each of these evaluations should include both formative and summative research questions with data collection and analysis plans that are adequate to address those questions.

The summative aspects of the evaluation should be designed to address the following outcomes for program participants: 1) changes in teacher knowledge and skills; 2) changes in the number of “highly qualified” teachers; 3) changes in classroom instruction; and 4) changes in student achievement. For more detail on expected measures see the Annual Report bullet beginning on page 3 of this document.

Although it is generally encouraged, there is no requirement that partnerships engage an external evaluator for this effort. However, partnerships that plan to use their own staff for the evaluation should have a clear written plan outlining how the evaluation activities will be conducted and who (preferably an individual, not an organization) will be responsible for each.



Prescribed anonymous teacher code – In some cases identified below (course participant knowledge assessment; course participant background surveys; annual participant survey), partnerships will be expected to provide the Donahue Institute with data on individual participants. Partnerships will be instructed to collect such data using a prescribed individual coding system provided by the Donahue Institute. This coding system will allow data to be anonymously linked across various instruments and program years, thus enabling the statewide evaluation to provide a more complete picture of MMSP impacts.



Systematic tracking of program participants – It is expected that each partnership will maintain a database (or other electronic tracking system) of all program participants including a record of the courses each participant completed and when those courses were completed. The specific structure of this database may be determined by each individual partnership, but it should be designed to allow for follow-up data collection from participants regardless of when they completed their last course in the program. (It is not expected that partnerships will be able to follow-up with participants who are no longer employed in their district.) Data from this database is primarily for the partnerships’ own program management and evaluation purposes – any data requested by the Donahue Institute or MADOE will be in aggregate and/or anonymous formats.

 End of course summary reports – At the end of each course, partnerships are required to submit a basic course summary report using a template to be provided by the Donahue Institute. This report will include course enrollment and completion rates, a template for submitting the pre/post results of the participant content knowledge assessment (described below), and a coversheet for the course participant background surveys

(described below), which should be returned as part of this packet. o Course participant content knowledge assessment – Partnerships must identify or develop an objective tool to assess course participants’ content knowledge gains for each course offered. At a



Annual Report of the MMSP State-level Evaluation Appendix B minimum these assessments should be administered as a pre/post test with participants completing the pre-test before receiving any course instruction and the post-test at the end of the course. Partnerships are encouraged to consider adding a third administration to measure the impact of any course-related follow-up activities (e.g., at the end of the school year). These assessments must yield quantifiable results that can be manipulated mathematically to yield mean scores and measurement of change over time. Data from these assessments must be collected in a manner that will allow individual participant results to be reported to the Donahue Institute using a prescribed anonymous individual coding system (described above) that will allow linkage of pre-test, post-test and follow-up scores.

This code will also allow linkage of the knowledge assessment results to other required data. o Course participant background surveys – Partnerships are required to administer a standard participant background survey at the end of each course offered. Similar to the survey that has been used to gather information on Summer Content Institutes’ participants, this survey asks about their qualifications and professional experience. Because we are interested in tracking changes (and/or the absence of changes) over time, this survey must be administered to all participants each time they complete a course, regardless of whether they have previously completed the survey.

The Donahue Institute will provide each partnership with machine-readable surveys that should be administered, collected and returned to the Institute for processing. These surveys will include the same anonymous individual coding system required for the content knowledge assessment described above. Partnerships may also choose to use the information collected through these surveys for their own program management and evaluation efforts. If so, they may either make copies to keep for their records or request to receive an electronic file of results from the Donahue Institute. It is important to note that any electronic file returned by the Donahue Institute will only identify teachers through the anonymous codes. Partnerships that wish to maintain this data in records with teachers’ names should retain copies for themselves or keep their own master list associating each teacher’s name with his/her code. The Donahue Institute is happy to help partnerships think through their specific data needs and to plan for related survey administration logistics.



Course participant feedback surveys – Although not required, partnerships are encouraged to administer and collect formative feedback from course participants. The Donahue Institute is happy to provide examples of such instruments from other professional development programs and/or offer suggestions for improving those developed by individual partnerships.



Annual survey of all program participants – Each partnership must conduct an annual survey of all participants who have completed at least one course during the program. It is expected that partnerships will be able to track and obtain cooperation from most individuals who are no longer active participants in the program, provided that they are still employed in their district. The Donahue Institute will provide each partnership with a bank of common required items that will be used to track changes in participants’ qualifications. A standard reporting template for those common items will also be provided. This data must be collected using the prescribed individual coding system to allow anonymous linkage with other data sources. Beyond the common required items, partnerships are welcome to add their own questions to these annual surveys. The Donahue Institute is happy to provide feedback and offer suggestions for improving any additional items added to the surveys. In particular, we encourage partnerships to add questions that will generate self-report data on any changes in teaching practices.

Please note, there may be cases where annual participant surveys and end of course surveys are administered at around the same time. By design, these two surveys are likely to ask some of the same questions. In those cases, the Donahue Institute can work with individual partnerships to minimize any duplication in the instruments.



Annual Report of the MMSP State-level Evaluation Appendix B

 Annual report – Each partnership must submit an annual report to the Massachusetts Department of

Education. To facilitate meta-analysis of all partnership reports it will be organized around a basic template provided by the Donahue Institute, while providing opportunities for partnerships to share additional information about their programs. The basic framework of the annual report template will be structured around the measures outlined in Form F of MMSP proposal. As such, partnerships should be designing data collection and management systems that will allow them to report annually on the following:

Process measures o Number of partnership meetings and level of partner participation o Number of courses developed o Number of courses delivered and total number of professional development hours available through those courses o Describe the extent to which the course of study has been integrated into higher education programs. o Number of participants completing each course (may be measured by the number of teachers completing the end of course survey) o Number of in-service program participants (taking one or more courses) during the given year.

Please note: For this particular measure each participant should only be counted once, regardless of the number of courses in which s/he participated.

Outcomes measures – All partnerships should be able to report this data for each program year as well as pre-program baseline data, where available. For programs with experimental or quasi-experimental designs, these measures should generally be reported for both treatment and control groups. Any exceptions should be discussed with the Donahue Institute well in advance. o The following data should be available through the required questions of the annual participant survey. Please note: For these particular measures the number of participants should be the number of unique individuals meeting the criteria in the given year. Each participant should only be counted once, regardless of the number of courses in which s/he participated.

 Number of participants taking the MTEL

 Number of participants passing the MTEL



Number of participants becoming licensed in the appropriate content area

 Number of participants with HOUSSE plans



Number of participants making progress on their HOUSSE plans

 Number of participants completing their house plans o For each course delivered in the given year, the number of participants demonstrating content knowledge gain on the pre/post content knowledge assessment and average gain of those participants o Percentage of the students of participating teachers scoring in each of the four MCAS performance categories (warning, needs improvement, proficient, advanced) – at applicable grade-levels. (Please note: It is not sufficient to report changes for the district as a whole, unless all teachers are participating in the professional development offered by the partnership.) o Quantifying the level of achievement among students of participating teachers using other assessments for grade levels where content-relevant MCAS is not available. (Please note, it is not sufficient to report changes for the district as a whole, unless all teachers are participating in the professional development offered by the partnership.)




Appendix C: Participant Background Survey – Year 1

Appendix C



Annual Report of the MMSP State-level Evaluation Appendix C










Appendix D: Participant Background Survey – Year 2

Appendix D



Annual Report of the MMSP State-level Evaluation Appendix D



















Appendix E: Participant Background Survey – Year 3

Appendix E



Annual Report of the MMSP State-level Evaluation Appendix E





















Annual Report of the MMSP State-level Evaluation Appendix F

Appendix F: Results of the Participant Background Survey

Item

How do you describe yourself?

American Indian or Alaskan native

Asian or Pacific Islander

Black – not Hispanic

Hispanic

White

Mixed Race

Other

No Response

What best describes your current position?

Teacher (Regular Education)

Special Education Teacher (Sole Instructor)

Special Education Inclusion Teacher

Other

Department Head or Curriculum Coordinator

Principal/Asst. Principal/Headmaster

Support Specialist (counselor, librarian, etc.)

Long-term Substitute

Paraprofessional

Superintendent or Asst. Superintendent

No Response

What grades do you currently teach?

Elementary and K-8

Middle School (Grades 6-8)

High School (Grades 9-12)

Substitute Teacher

Special Education (Alternative grades only)

Middle and High School grades

None at this time

No Response

How many years have you been employed in education?

1 st year

2-3 years

4-5 years

6-10 years

11-20 years

Over 20 years

0 or No Response

Year 1 Year 2

n (%) n (%)

0 (0%)

5 (2%)

0 (0%)

6 (1%)

5 (2%)

6 (2%)

10 (2%)

12 (3%)

307 (90%) 402 (88%)

2 (1%) 6 (1%)

6 (2%)

10 (3%)

6 (1%)

14 (3%)

270 (79%) 351 (77%)

25 (7%)

22 (7%)

13 (4%)

31 (7%)

31 (7%)

23 (5%)

5 (2%)

3 (1%)

2 (1%)

11 (2%)

3 (1%)

3 (1%)

N/A

0 (0%)

1 (<1%)

0 (0%)

1 (<1%)

1 (<1%)

0 (0%)

1 (<1%)

103 (30%) 110 (24%)

178 (52%) 270 (59%)

29 (9%) 40 (9%)

1 (<1%)

3 (1%)

4 (1%)

10 (3%)

13 (4%)

1 (<1%)

1 (<1%)

6 (1%)

10 (2%)

18 (4%)

14 (4%)

28 (8%)

39 (11%)

23 (5%)

51 (11%)

62 (14%)

97 (28%) 106 (23%)

96 (28%) 119 (26%)

61 (18%)

6 (2%)

84 (18%)

11 (2%)

n

Year 3

(%)

Total to Date

n (%)

Note: Percentages may not total 100% due to rounding error or items in which respondents may respond to all that apply.

2 (<1%) 2 (<1%)

7 (2%)

15 (3%)

8 (2%)

15 (2%)

25 (3%)

20 (2%)

404 (87%) 793 (87%)

6 (1%)

8 (2%)

14 (3%)

11

13

30

(1%)

(1%)

(3%)

331 (71%) 674 (74%)

43 (9%) 75 (8%)

38 (8%) 65 (7%)

22 (5%) 43 (5%)

10 (2%) 22 (2%)

5 (1%) 9 (1%)

5 (1%)

3 (1%)

8 (1%)

2 (<1%)

3

0

(1%)

(0%)

4 (1%)

4 (<1%)

1 (<1%)

5 (1%)

85 (18%) 197 (22%)

265 (57%) 515 (57%)

69 (15%) 108 (12%)

2 (<1%) 2 (<1%)

4 (1%) 7 (1%)

5 (1%) 11 (1%)

14 (3%) 27 (3%)

20 (4%) 42 (5%)

19 (4%) 37 (4%)

73 (16%) 109 (12%)

59 (13%) 115 (13%)

111 (24%) 226 (25%)

126 (27%) 247 (27%)

73 (16%) 161 (18%)

3 (1%) 14 (2%)




Item

Which of the following content areas are you currently teaching?

Mathematics

Elementary (all subjects)


General Science

Biology

Physics

Earth Science

Chemistry

Any science area*


Other

Computer Science

Do not teach currently

In which type of school do you currently work?

Public School

Public Charter School

Private School

No Response

Currently hold certification through the National

Board for Professional Teaching Standards.

In Any Subject

In Mathematics

In General Science

Appendix F

Year 1

n (%)

Year 2

n (%) n

Year 3

(%)

Total to Date

n (%)

190 (56%) 252 (55%) 288 (62%) 535 (59%)

103 (30%) 107 (23%) 79 (17%) 189 (21%)

N/A

65 (19%)

31 (7%)

71 (16%)

26

56

(6%)

(12%)

45

131

(5%)

(14%)

22 (6%)

11 (3%)

21 (6%)

12 (4%)

24 (5%)

16 (4%)

16 (4%)

14 (3%)

16

10

14

11

(3%)

(2%)

(3%)

(2%)

41

20

30

22

(5%)

(2%)

(3%)

(2%)

88 (26%)

10 (3%)

61 (18%)

N/A

13 (4%)

96 (21%)

3 (1%)

25 (5%)

2 (<1%)

10 (2%)

83 (18%) 183 (20%)

4 (1%) 10 (1%)

22

3

18

(5%)

(1%)

(4%)

59

4

32

(6%)

(<1%)

(4%)

327 (96%) 434 (95%) 445 (96%) 857 (94%)

5 (2%) 14 (3%) 9 (2%) 26 (3%)

8 (2%) 7 (2%) 6 (1%) 20 (2%)

1 (<1%) 1 (<1%) 4 (1%) 6 (1%)

51 (15%)

N/A

N/A

N/A

11 (2%) 14

1 (<1%) 3

N/A

(3%) 23

(1%) 5

N/A

(3%)

(1%)




Item

Appendix F


Approximately how many students do you teach annually who are Special Education students?

0 students

1-10 students

11-40 students

41-150 students

151+ students

No Response

n (%) n (%) n (%) n

Approximately how many students do you teach annually?

0 students

1-10 students

11-40 students

41-150 students

151+ students

No Response

Approximately how many students do you teach annually who are Title I students?

0 students

1-10 students

11-40 students

41-150 students

151+ students

No Response

Approximately how many students do you teach annually who are academically advanced students?

0 students

1-10 students

11-40 students

41-150 students

151+ students

No Response

2 (1%)

6 (2%)

3 (1%)

4 (1%)

3

8

(1%)

(2%)

4

16

(<1%)

(2%)

98 (29%) 118 (26%) 101 (22%) 214 (24%)

203 (60%) 295 (65%) 310 (67%) 596 (66%)

21 (6%)

11 (3%)

28 (6%) 20 (4%) 47

8 (2%) 22 (5%) 32

(5%)

(4%)

66 (19%) 117 (26%) 131 (28%) 222 (24%)

73 (21%) 69 (15%) 66 (14%) 137 (15%)

46 (13%)

39 (11%)

6 (2%)

58 (13%)

63 (14%)

7 (2%)

49 (11%)

43

7

(9%)

(2%)

107

107

14

(12%)

(12%)

(2%)

111 (33%) 142 (31%) 168 (36%) 322 (35%)

92 (27%) 154 (34%) 134 (29%) 262 (29%)

61 (18%) 66 (15%) 65 (14%) 129 (14%)

53 (16%)

7 (2%)

65 (14%)

14 (3%)

80 (17%)

18 (4%)

154

31

(17%)

(3%)

1 (<1%) 1 (<1%) 0 (0%) 2 (<1%)

127 (37%) 156 (34%) 167 (36%) 331 (36%)

19 (6%)

18 (5%)

1 (<1%)

51 (15%)

32 (7%)

(%)

129 (38%) 144 (32%) 143 (31%) 301 (33%)

123 (36%) 200 (44%) 216 (47%) 398 (44%)

23 (5%)

0 (0%)

57 (13%)

26 (6%) 49

17 (4%) 44

5 (1%)

(5%)

57 (12%) 112 (12%)

5

(5%)

(1%)

Approximately how many students do you teach annually who are English Language Learners?

0 students

1-10 students

11-40 students

41-150 students

151+ students

No Response

62 (18%)

47 (14%)

26 (8%)

88 (19%)

112 (33%) 149 (33%) 135 (29%) 262 (29%)

62 (14%)

36 (8%)

89 (19%) 176 (19%)

78 (17%)

31 (7%)

137

64

(15%)

(7%)

4 (1%) 5 (1%) 4 (1%) 12 (1%)

90 (26%) 116 (25%) 127 (27%) 258 (28%)




Item

Why did you participate in this course? *

To increase knowledge in content

To obtain graduate credit

To earn PDPs for recertification

To pursue a personal interest

To get an additional license (certification)

To prepare for the Massachusetts Test for

Educator Licensure (MTEL)

To earn PDPs for your HOUSSE plan requirement

To follow an administrator’s suggestion

To obtain a first license (certification)

Other

No Response

High Need District

Yes

No

Private School (not included)

No Response

Year 1 Year 2

n

Year 3

(%)

Total to Date

n (%) n (%) n (%)

355 (85%) 573 (80%) 499 (72%) 1427 (78%)

278 (66%) 456 (63%) 444 (64%) 1178 (64%)

161 (38%) 309 (43%) 240 (35%) 710 (39%)

147 (35%) 245 (34%) 207 (30%) 599 (33%)

89 (21%) 157 (22%) 103 (15%) 349 (19%)

82 (20%) 154 (21%) 118 (17%) 354 (19%)

53 (13%) 116 (16%)

32 (8%)

15 (4%)

25 (6%)

2 (1%)

73 (10%)

22 (3%)

31 (4%)

2 (<1%)

90 (13%) 259 (14%)

47 (7%) 152

22 (3%) 59

(8%)

(3%)

34 (5%)

0 (0%)

90

0

(5%)

(0%)

202 (59%) 276 (61%) 254 (55%) 530 (58%)

129(38%) 161 (35%) 199 (43%) 342 (38%)

8 (2%)

2 (1%)

7 (2%)

12 (3%)

6

6

(1%)

(1%)

20

17

(2%)

(2%)

Highly Qualified

Yes

No

In some, but not all areas taught

Not enough information to determine

Private School (not included)

Not applicable (not currently teaching)

131 (38%) 267 (59%) 333 (72%) 561 (62%)

92 (27%) 69 (15%) 36 (8%) 120 (13%)

19 (6%)

78 (23%)

8 (2%)

13 (4%)

21 (5%)

82 (18%)

7 (2%)

10 (2%)

22

18

(5%)

(4%)

38

32

(4%)

49 (11%) 139 (15%)

6 (1%) 19 (2%)

(4%)

Do you have a High Objective Uniform State Standard of

Evaluation (HOUSSE) plan? (Public School Teachers Only)

Yes

No

Not Sure

No Response

75 (23%)

142 (43%)

98 (30%)

17 (5%)

177(40%)

135 (30%)

127(28%)

9 (2%)

196

137

106

16

(43%)

(30%)

(23%)

(4%)

332

285

237

29

(38%)

(32%)

(27%)

(3%)

If you do have a HOUSSE plan, how many PDP hours do you have in your content area(s)?

Less than 48 PDP hours

48 to 100 PDP hours

101 to 250 PDP hours

251+ PDP hours

No Response

9 (12%)

8 (11%)

36 (48%)

8 (11%)

14 (19%)

11 (6%)

29 (16%)

58 (33%)

47 (27%)

32 (18%)

17 (9%)

19 (10%)

32 (10%)

34 (10%)

61 (31%) 112 (34%)

69 (35%) 107 (32%)

30 (15%) 47 (14%)

*

*Data for this item represents the number of seats filled from all courses, rather than from unique participants.




Item – Year 1

A degree currently held for each major.

Education

Math

General Science

Biology

Chemistry

Earth Science

Physics

All science/technology combined

Other

Bachelors

A degree currently being pursued for each major.

Education

Math

General Science

Biology

Chemistry

Earth Science

Physics


Other

Item – Year 2


Education*

Math

Math Education*

General Science

Biology

Chemistry

Earth Science

Physics


Other Science


Bachelors


Education*

Math

Math Education*

General Science

Biology

Chemistry

Earth Science

Physics


Other Science


*This question only asked of secondary level mathematics teachers.

48

32

10

2

29

3

3

1

0

13

51

0

0

0

0

0

0

1

0

0

0

1

0

0

0

0

1

0

0

0

0



160

27

8

22

7

3

1

41

106

Masters

182

5

4

3

3

1

0

11

48

34

55

4

0

1

0

0

5

9

Doctoral

2

0

0

1

0

0

0

1

1

0

0

0

0

3

0

0

0

0

CAGS

10

0

0

0

0

0

0

0

4

0

0

0

1

6

5

0

1

4

Masters

6

0

0

0

18

22

55

0

0

0

10

65

4

11

1

3

1

3

3

2

12

25

CAGS

1

1

0

0

3

1

3

0

0

0

2

0

0

0

0

3

0

0

0

0

0

0

Doctoral

0

0

0

0

1

0

2

1

0

0

1

0

0

1

0

0

0

0

0

0

2

3

Appendix F

85


Item – Year 3 Bachelors


Education

Math

Math Education

Science Education

General Science

Biology

Chemistry

Earth Science

Physics


Other


Education

Math

Math Education

Science Education

General Science

Biology

Chemistry

Earth Science

Physics


Other

162

46

15

3

10

19

9

6

2

46

166

3

0

0

0

3

2

3

0

0

0

2

CAGS

0

0

0

0

9

1

14

1

0

1

2

0

0

0

3

0

0

0

0

0

0

4

Masters

179

9

19

9

3

3

1

1

1

9

83

8

0

2

0

54

15

84

0

0

2

11

Doctoral

0

0

0

0

1

0

0

0

0

0

1

0

1

0

2

0

0

2

0

0

1

5

Appendix F




Item – Total to Date *


Education

Math

Math Education

Science Education

General Science

Biology

Chemistry

Earth Science

Physics

Technology

Other Science


Other

Bachelors

163

47

14

4

10

22

7

7

2

Unknown

Unknown

48

165

Masters


Education

Math

Math Education

Science Education

General Science

Biology

Chemistry

Earth Science

Physics

Technology

Other Science


Other

2

0

0

0

0

3

3

2

0

Unknown

Unknown

0

3

*Based on the last survey completed by each individual subject.

52

12

89

9

0

1

0

0

0

Unknown

Unknown

1

12

174

11

19

10

3

1

1

2

1

Unknown

Unknown

8

88

CAGS Doctoral

1

0

0

0

0

0

2

0

0

1

0

0

0

0

0

0

0

0

Unknown Unknown

Unknown Unknown

0 0

1 7

9

2

11

0

0

0

0

1

0

Unknown Unknown

Unknown Unknown

0

0

0

0

0

2

0

0

0

1 0

4 1

Appendix F




MTEL tests taken – Year 1

General Curriculum (formerly

Elementary)

Mathematics


Middle School Mathematics/

Science

General Science

Biology

Chemistry

Physics

Earth Science




Elementary)


Mathematics



Science

General Science

Biology

Chemistry

Physics

Earth Science




Elementary)


Mathematics



Science

General Science

Biology

Chemistry

Physics

Earth Science


MTEL

Taken

MTEL

Passed

Scores

Unknown

51 15% 46 90% 0 0%

23 7% 13 57% 5 22%

43 13% 34 79% 5 12%

7 2% 3 43% 2 29%

14 4% 12 86% 0 0%

7 2%

2 1%

0 0%

0 0%

4

0

0

57%

2 100%

0%

0%

1

0

0

0

14%

0%

0%

0%

2 1% 1 50% 0 0%

MTEL

Taken

MTEL

Passed

Scores

Unknown

42 9% 40 95% 0 0%

4 1% 2 50% 2 50%

41 9% 29 71% 0 0%

82 18% 66 80% 8 10%

11 2% 5 45% 0 0%

23 5% 22 96% 0 0%

4 1% 4 100% 0 0%

0 0%

0 0%

1 <1%

0

0

0%

0%

1 100%

0

0

0

0%

0%

0%

0 0% 0 0% 0 0%

MTEL Taken MTEL Passed

30 6% 27 90%

9 3% 8 89%

56 12% 49 88%

116 25% 102 88%

15 3% 12 80%

18

8

4%

2%

2 <1%

17

6

2

94%

75%

100%

0 0%

1 <1%

0 0%

1 100%

0 0% 0 0%

Scores

Unknown

1

1

0

0

0

0

0

0 0%

1 11%

1 2%

2 2%

7%

6%

0%

0%

0%

0%

0%



Appendix F

88


MTEL tests taken – Total to Date


Elementary)


Mathematics



Science

General Science

Biology

Chemistry

Physics

Earth Science


MTEL Taken MTEL Passed

80 9% 73 91%

15 2% 12 80%

92 10% 70 76%

190 21% 161 85%

25 3% 15 60%

35

8

4% 31 89%

1% 5 63%

3 <1%

0 0%

2 <1%

3

0

2

100%

0%

100%

1 <1% 0 0%

Scores

Unknown

0 0%

3 20%

2 2%

9 5%

4 16%

1

0

0

0

0

0

3%

0%

0%

0%

0%

0%

Appendix F




License Areas - Titles as of 10/01

23

24

25

26

19

20

21

22

27

28

29

30

15

16

17

18

11

12

13

14

05

06

07

08

09

10

01

02

03

04

Academically Advanced

Biology

Business

Chemistry

Dance

Early Childhood

Earth Science

Elementary

English

ESL

Foreign Language

General Science

Health/Family & Consumer Science

History

Instructional Technology

Latin and Classic Humanities

Library

Mathematics

Middle School

Middle School Humanities


Middle School Math/Science

Music

Physical Education

Physics

Political Science/Political Philosophy

Reading

Speech

Students w/ Moderate Disability

Students w/ Severe Disability

31

32

Deaf and Hard of Hearing

Visually Impaired

33

34


Theater

35

36

Transitional Bilingual Education

Visual Arts

37/38 Other

39 Elementary Mathematics


n (%) n (%) n (%) n (%)

0 (0%) 2 ( <1 %) 0 (0%) 1 (<1%)

22 (6%) 34 (7%) 23 (5%) 56

5 (1%) 3 (1%) 7 (2%) 11

(6%)

(1%)

9 (3%)

0 (0%)

6 (1%)

0 (0%)

8 (2%) 15

0 (0%) 0

17 (5%) 12 (3%) 13 (3%) 24

3 (1%) 4 (1%) 5 (1%) 10

(2%)

(0%)

(3%)

(1%)

203 (60%) 237 (52%) 201 (43%) 439 (48%)

6 (2%) 14 (3%) 6 (1%) 19 (2%)

5 (1%)

2 (1%)

6 (1%)

1 ( <1 %)

6 (1%) 10

3 (1%) 3

(1%)

(<1%)

34 (10%) 57 (13%) 38 (8%) 89 (10%)

3 (1%) 1 ( <1 %) 3 (1%) 5 (1%)

9 (3%) 15 (3%) 14 (3%) 23

2 (1%) 4 (1%) 4 (1%) 8

(3%)

(1%)

0 (0%)

0 (0%)

0 (0%)

0 (0%)

0 (0%)

3 (1%)

0 (0%)

3 (<1%)

45 (13%) 83 (18%) 114 (25%) 192 (21%)

45 (13%) 60 (13%) 48 (10%) 102 (11%)

0 (0%) 2 ( <1 %) 2 (<1%) 4 (<1%)

43 (13%) 74 (16%) 113 (24%) 183 (20%)

2 (1%) 11 (2%) 16 (3%) 22 (2%)

0 (0%)

3 (1%)

0 (0%)

2 ( <1 %)

1

3

(<1%)

(1%)

1

7

(<1%)

(1%)

4 (1%)

0 (0%)

3 (1%)

0 (0%)

2

2

(<1%)

(<1%)

5

2

(1%)

(<1%)

17 (5%) 19 (4%) 14 (3%) 25

2 (1%) 0 (0%) 0 (0%) 1

(3%)

(<1%)

53 (16%) 51 (11%) 82 (18%) 139 (15%)

0 (0%)

0 (0%)

0 (0%)

1 ( <1 %)

0 (0%)

0 (0%)

0 (0%)

0 (0%)

6 (1%)

1 (<1%)

0 (0%)

6 (1%)

6 (1%)

2 (<1%)

0 (0%)

4 (<1%)

0 (0%)

1 ( <1 %)

1 ( <1 %)

5 (1%)

0

2

(0%)

(<1%)

0

7

(0%)

(1%)

4 (1%) 2 ( <1 %) 0 (0%) 4 (<1%)

29 (9%) 59 (13%) 29 (6%) 76 (8%)

N/A 4 (1%) 9 (2%) 13 (1%)




Year 1

Grade of Certification

n (%)

F

G

H

I

J

K

L

A

B

C

D

E

Pre-K – 2

Pre-K – 3

Pre-K – 6

Pre-K – 8

Pre-K – 9

1 – 6

5 – 8

5 – 9

5 – 12

8 – 12

9 – 12

All Levels

Year 2

n (%)

2 (1%)

16 (5%)

1 (<1%)

14 (3%)

36 (10%) 19 (4%)

66 (19%) 102 (22%)

32 (9%) 37 (8%)

117 (34%) 144 (32%)

73 (22%) 130 (29%)

68 (20%) 112 (25%)

37 (11%) 51 (11%)

16 (5%) 35 (8%)

52 (16%)

45 (13%)

68 (15%)

52 (11%)

Year 1 Year 2

Level of Certification

n (%) n (%)

Appendix F

Year 3

n

92

30

120

175

78

2

12

34

55

39

91

54

(%)

(<1%)

(3%)

(7%)

(20%)

(6%)

(26%)

(38%)

(17%)

(12%)

(8%)

(20%)

(12%)

Total to Date

n

188

69

253

280

174

3

23

68

112

74

148

106

(%)

(<1%)

(3%)

(7%)

(21%)

(8%)

(28%)

(31%)

(19%)

(12%)

(8%)

(16%)

(12%)

n

Year 3

(%)

Total to Date

n (%)

P

I

S

T

Provisional Certification

Initial (Provisional with

Advanced Standing)

Standard**

Temporary

76 (22%) 86 (19%) 116 (25%) 206 (23%)

117 (34%) 181 (40%)

198 (43%) 357 (39%)

364 (107%) 500 (110%) 460 (99%) 925 (102%)

4 (1%) 2 (<1%) 0 (0%) 3 (<1%)

* This reflects the number of teachers teaching science and/or technology courses. The total is not a sum of the individual content areas, as many of these teachers teach multiple science courses.

** Percentage over 100% due to some participants holding multiple standard licenses.

Note: Percentages may not total 100% due to rounding error or items in which respondents may respond to all that apply



Annual Report of the MMSP State-level Evaluation Appendix G

Appendix G: High Need District Eligibility

High Need Districts (See list below.):

1.

For proposals with a mathematics content focus: A district is considered to be a high need district if it has a mathematics proficiency index for grades 4-8 that is below the state target for Cycle II for MMSP Year 1projects or below the state target for Cycle III for MMSP Year 2 projects.. Priority will be given to high need districts with two or more schools identified for improvement in mathematics.

2.

For proposals with a science and/or technology/engineering content focus: A district is considered to be a high need district if it has a science proficiency index for grades 5-8 in 2003 that is at or below the 20 th percentile for the state.

In addition, a high need district must demonstrate that there is a high number or percentage of teachers in the district who are teaching in the academic subject or grade level for which they have not demonstrated subject matter competency through licensure or completion of the professional development activities in their HOUSE plans.

An interested district that is not identified as high need is encouraged to contact a high need district to explore becoming a partner in the proposed program (e.g., vocational technical schools are encouraged to contact feeder school districts).

Year 1 High Need Districts

DISTRICT

AVON

BARNSTABLE

BOSTON

BROCKTON

CAMBRIDGE

CHELSEA

CHICOPEE

CLARKSBURG

EASTHAMPTON

EVERETT

FAIRHAVEN

FALL RIVER

FITCHBURG

FLORIDA

GARDNER

GREENFIELD

HAVERHILL

HOLBROOK

HOLYOKE

HULL

LAWRENCE

LOWELL

LYNN

MALDEN

MEDFORD

METHUEN

NEW BEDFORD

NORTH ADAMS

PITTSFIELD

PROVINCETOWN

RANDOLPH

REVERE

SALEM

SOMERVILLE

SOUTHBRIDGE

SPRINGFIELD









































MATH SCIENCE

T/E



















































































DISTRICT

TAUNTON

WALTHAM

WARE

WAREHAM

WEBSTER

WEST SPRINGFIELD

WESTFIELD

WINCHENDON

WINTHROP

WORCESTER

ABBY KELLEY FOSTER CS

ATLANTIS CS

BENJAMIN BANNEKER CS

BOSTON RENAISSANCE CS

CONSERVATORY LAB CS

EDWARD BROOKE CS

FREDERICK DOUGLASS CS

LAWRENCE FAMILY DEV CS

LOWELL COMMUNITY CS

NEIGHBORHOOD HOUSE CS

NEW BEDFORD GLOBAL CS

NEW LEADERSHIP HMCS

NORTH CENTRAL REG CS

ROBERT M. HUGHES CS

SABIS INTERNATIONAL CS

SEVEN HILLS CS

SOMERVILLE CS

UPHAMS CORNER CS

ATHOL-ROYALSTON RSD

BERKSHIRE HILLS RSD

FRONTIER RSD

GILL-MONTAGUE RSD

HAMPSHIRE RSD

HAWLEMONT RSD

MOUNT GREYLOCK RSD

RALPH C MAHAR RSD

MATH SCIENCE

T/E

 















































































































92

Attleboro

Avon

Boston

Bourne

Brockton

Cambridge

Chelsea

Chicopee

Clarksburg

Dedham

Dracut

East Bridgewater

Easthampton

Everett

Fairhaven

Fall River

Fitchburg

Florida

Gardner

Greenfield

Haverhill

Holbrook

Holyoke

Hull

Lawrence

Leominster

Lowell

Lynn

Malden

Medford

Methuen

New Bedford

North Adams

Orange

Oxford

Quincy

Pittsfield

Randolph

Revere

Rockland

Salem

Somerville



DISTRICT Grade 5

SCI

Grade 8

SCI

Grades 4-

8 Math

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√



Appendix G


SCI

Grade 8

SCI

Grades 4-

8 Math

Southbridge

Springfield

Taunton

Waltham

Ware

Wareham

Webster

Westfield

Winchendon

Winthrop

Worcester

Academy Of Strategic CS

Benjamin Banneker CS

Frederick Douglass CS

Lawrence Family Dev. CS

Lowell Community CS

Edward Brooke CS

Conservatory Lab CS

Community Day CS

Sabis International CS

Neighborhood House CS

Abby Kelley Foster Reg CS

So.Boston Harbor Academy CS

√

√

Robert M. Hughes Academy CS

√

√

√

√

√

√

√

√

√

√

√

New Leadership HMCS

New Bedford Global HMCS

North Central CS

Boston Renaissance CS

Seven Hills CS

Somerville CS

Prospect Hill Academy CS

Uphams Corner CS

Atlantis CS

Adams-Cheshire

Athol-Royalston

Berkshire

Freetown-Lakeville

Gateway

Gill-Montague

Hampshire

√

√

√

√

√

√

√

√

√

√

Hawlemont

New Salem-Wendell

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

93

Holyoke

Hull

Lawrence

Leominster

Lowell

Lynn

Malden

Medford

Methuen

New Bedford

North Adams

Orange

Oxford

Quincy

Pittsfield

Randolph

Revere

Rockland

Salem

Somerville

Southbridge

Springfield

Taunton

Attleboro

Avon

Boston

Bourne

Brockton

Cambridge

Chelsea

Chicopee

Clarksburg

Dedham

Dracut

East Bridgewater

Easthampton

Everett

Fairhaven

Fall River

Fitchburg

Florida

Gardner

Greenfield

Haverhill

Holbrook




SCIENCE

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

Grade 8

SCIENCE

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

Grades 4-8

MATH

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√



Appendix G

DISTRICT

Waltham

Ware

Wareham

Webster

Westfield

Winchendon

Winthrop

Worcester

Academy Of Strategic CS

Benjamin Banneker CS

Frederick Douglass CS

Edward Brooke CS

Conservatory Lab CS

Community Day CS

Sabis International CS

Neighborhood House CS

Abby Kelley Foster Regional

CS

South Boston Harbor

Grade 5

SCIENCE

√

√

√

√

√

√

√

√

√

√

√

√

CS

Lawrence Family

New Leadership HMCS

New Bedford Global HMCS

North Central CS

Boston Renaissance CS

Seven Hills CS

Somerville CS

Prospect Hill Academy CS

Uphams Corner CS

Atlantis CS

Adams-Cheshire RSD

Athol-Royalston RSD

Berkshire Hills RSD

Freetown-Lakeville RSD

Gateway RSD

Gill-Montague RSD

Hampshire RSD

Hawlemont RSD

New Salem-Wendell RSD

√

√

√

√

√

√

√

√

√

√

√

√

√

Grade 8

SCIENCE

√

√

√

√

√

√

√

√

Grades 4-8

MATH

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

94

Annual Report of the MMSP State-level Evaluation Appendix H

Appendix H: Subject Matter Competency Demonstration Options

How do teachers demonstrate subject matter competency in Massachusetts?

Elementary teachers may demonstrate competence in reading, writing, mathematics, and other areas of the basic elementary school curriculum through one of the following:

Middle and secondary school teachers may demonstrate subject matter competence in each of the areas they are teaching through one of the following:

Passing the Massachusetts Test for Educator

Licensure (MTEL) Elementary Subject Matter Test:

General Curriculum and the

Foundations of Reading

Passing the Massachusetts Test for Educator

Licensure (MTEL) appropriate Subject Matter Test;

Middle School Humanities



Subject Title (e.g., History, English, Physics)

Making sufficient progress* on Massachusetts High

Objective Uniform State Standard of Evaluation

(HOUSSE)

Completion of an appropriate academic major

`Making sufficient progress* on Massachusetts High

Objective Uniform State Standard of Evaluation

(HOUSSE)

Completion of an appropriate graduate degree

Completion of comparable coursework equivalent to an undergraduate academic major

Advanced certification or credentialing

*It is the Department's expectation that a teacher will have … completed at least half [48] of the content PDPs [96 total] needed to meet

HOUSSE requirements before being considered highly qualified.

Charter School Teachers who teach core academic subjects do not need a Massachusetts license, but must hold a Bachelor's degree and demonstrate competence in the subject area in which they teach. Charter school teachers may demonstrate subject matter competence through any one of the options available to elementary and middle/secondary teachers.

Teachers in Vocational Schools who teach core academic courses are required to meet the definition of a highly qualified teacher. A vocational school teacher who teaches a core academic subject must hold a Bachelor's degree, be licensed or certified by the state, and demonstrate subject matter competence in order to be considered highly qualified.

(information obtained from MADOE, 2004)


Research and Evaluation Group 95

Appendix I: Highly Qualified Status of Unique Participants

Highly Qualified Status of Unique Participants

STATUS

Became Highly

Qualified

Highly Qualified in some content areas but not all

Highly Qualified but unable to determine when became HQ

Not Highly

Qualified

Private school or not teaching n

Y1

=136

5

4

52

34

11

6

6

107

27

8

Took only one course

n = 555

Y2 Y3

C1 n =185 n = 196

C2 n =11

0

0

8

2

1

C1 n =209

8

7

145

20

11 n = 223

C2 n =14

0

0

9

0

3

19

17

321

83

34

Y1 only n =21

1

2

9

4

2

C1 n =25

2

2

14

2

2

Y2 only n = 31

C2 n =6

Took multiple courses

n = 354

Y3 only Multiple Years* n = 54

C1 n =35

C2 n =19

C1 n =241 n = 248

C2 n =7

1

0

2

0

1

4

1

21

3

0

3

0

13

0

2

81

14

104

13

9

0

0

3

1

1

92

19

166

23

17

TOTAL

N=909

111

36

487

106

51

Unknown 30 31 0 18 2 81 3 3 2 6 1 20 2 37 118

*Participants who participated in courses from partnerships in both cohorts 1 and 2 are included in the cohort 1 column as they initially began their participation in MMSP in cohort 1.



Appendix J: Criteria that Account for Gain in Highly Qualified Status

Criteria that Account for Gain in Highly Qualified Status

All unique participants to date

Criteria Accounting for

Gains in Highly

Qualified Status

MTEL only

National Certification only

Degree in Content Area only

Undergrad Equivalent only

HOUSSE only

Earned a Teaching License

MTEL/Undergrad Equiv combined

MTEL/Degree combined

MTEL/HOUSSE combined

HOUSSE/Undergrad Equiv

Combined

MTEL/HOUSSE/Undergrad

Equiv combined

TOTAL

Gained

HQ status during

Year 1

1

NA

1

NA

10

0

NA

0

0

0

NA

12

Gained HQ status during Year 2

Cohort 1

9

0

0

0

43

1

1

0

5

0

0

59

Cohort 2

0

0

0

1

0

0

0

0

0

0

0

1

Gained HQ status during Year 3

Cohort 1

5

0

3

2

17

2

3

2

0

1

1

36

Cohort 2

1

0

1

0

0

0

0

0

0

1

0

3

TOTAL became HQ over course of project

16

0

5

3

70

3

4

2

5

2

1

111



Download MS WORD Document size: 3.4 MB

Annual State-level Evaluation Report

Reporting Period:

February 2, 2004 through August 31, 2006

Contents

Tables Index

Executive Summary

Program Description

Evaluation Plan and Activities

State-level Evaluation

Local Evaluation and Related Technical Assistance

MMSP Participant Background Data

State-level Participant Background Data

Partnership-level Participant Background Data

MMSP Partnership: EduTron

MMSP Partnership: Harvard Graduate School of Education (HGSE)

MMSP Partnership: Lesley University

MMSP Partnership: Massachusetts College of Liberal Arts (MCLA) Science

MMSP Partnership: Salem State College

MMSP Partnership: Springfield/Holyoke Partnership

MMSP Partnership: Wareham Public Schools

MMSP Partnership: Worcester Polytechnic Institute

MMSP Partnership: Massachusetts College of Liberal Arts (MCLA) Math

MMSP Partnership: University of Massachusetts Amherst

Progress Toward Meeting MMSP Goals

Goal II.

Goal III.

Goal IV.

Goal V.

Discussion and Recommendations

Appendix A: Timeline for State-level Evaluation and TA Activities

Appendix B: Minimum Expectations for Evaluation

Appendix C: Participant Background Survey – Year 1

Appendix D: Participant Background Survey – Year 2

Appendix E: Participant Background Survey – Year 3

Appendix F: Results of the Participant Background Survey

Appendix G: High Need District Eligibility

Appendix H: Subject Matter Competency Demonstration Options

How do teachers demonstrate subject matter competency in Massachusetts?

Appendix I: Highly Qualified Status of Unique Participants

Appendix J: Criteria that Account for Gain in Highly Qualified Status

Related documents

Add this document to collection(s)

Add this document to saved

Suggest us how to improve StudyLib