advertisement

Mathematics Achievement Academies 2011 Evaluation Report to Chevron U.S.A., Inc. January, 2013 Mathematics Achievement Academies – 2011 Evaluation Report Prepared for Dr. Julia Olkin, Principal Investigator Co-Director, Center for Math Education and Research by Dr. Beth Yeager, Lead Researcher Institute for STEM Education CA State University, East Bay Dr. Eric A. Suess Department of Statistics and Biostatistics CA State University, East Bay With contributions from student assistants Hao Le Tony Tran Aida Yazdanparast Peter Cummings Qiongsi Dong 2 Abstract In Summer, 2011, the Mathematics Achievement Academies (MAA) program began Year 2 of a three year program, primarily funded through the generosity of Chevron, U.S.A., Inc., to support middle and high school students in enhancing their capacity to successfully complete Algebra I, Geometry and Algebra II (Summer, 2012). MAA offered a total of 39 academies in 2011, 22 in algebra and 17 in geometry, with a total of 470 students completing the academy. The program had a 68% retention rate in Algebra and a 72% retention rate in Geometry. Findings from the evaluation of the 2011 MAAs, support the conclusion that the program is having a positive impact on average student growth in content knowledge in both algebra and geometry. Based on analyses of assessments for all students enrolled in the program, raw percentages show a growth in algebra content knowledge from a mean of 40% for all students taking a pre-test to 52% for those students taking a post test. Findings also indicate growth in geometry academies from a mean of 37% for all students taking a pre-test to 50% for those students taking a post test. When findings are disaggregated by district and class, they show that 16 out of 22 classes in algebra showed statistically significant improvement, and in geometry, 15 out of 17 classes showed statistically significant improvement. Even if students attend only one year of summer academy, algebra results support the summer academy approach that emphasizes preparation rather than remediation. Additionally significant are the differences in growth in geometry academies. The findings from this analysis of the introduction of the geometry academies in 2011 supports the positive potential and consequences for student learning of a long-term approach to student preparation for entering gatekeeper math courses. 3 Table of Contents Executive Summary 5 Introduction 7 Program Design Overview 8 Project Impact 9 Project Participants Findings – Pre/Post Math Content Tests 9 13 Algebra - Pre/Post Test Findings by District and Class 17 Geometry – Pre/Post Test Findings by District and Class 22 Findings – Pre and Post Student Self-Assessment of Item Difficulty – Algebra and Geometry Concluding Remarks Appendices 27 32 33 Appendix I - Algebra - Student Content Knowledge Test Analyses by Class 34 Appendix II - Geometry - Student Content Knowledge Test Analyses by Class 57 4 Executive Summary In Summer, 2011, the Mathematics Achievement Academies (MAA) program began Year 2 of a three year program to support middle and high school students in enhancing their capacity to successfully complete Algebra I, Geometry and Algebra II (Summer, 2012) courses. Primarily funded through the generosity of Chevron, U.S.A., Inc., through its California Partnership, MAA is administered by the Center for Mathematics Education and Research (CMER) at California State University, East Bay, directed by Dr. Julia Olkin and Dr. Phil Duren. In 2011 CSUEB collaborated with Alameda County Office of Education (ACOE). Designed to assist in preparing students, who have previously struggled with math classes, for college-preparatory mathematics, MAA is a four-week (sixty hours) math-intensive summer program. The interactive classes and activities emphasize rigorous content, collaboration, creative thinking, and logical reasoning. Instructors for the academies were primarily drawn from a pool of teachers who had previously participated in professional development courses offered through Project ACCLAIM (the Alameda County Collaborative for Learning and Instruction in Mathematics). In 2011, MAA offered 22 academies (sections) in Algebra to students who would be entering 9th grade and enrolled in Algebra I in Fall, 2011. The 22 academies for this second cohort of students reflect an increase of 2 over the previous year’s (2010) offerings in algebra. In addition, 17 academies (sections) were added and offered in Geometry for students entering that discipline as 10th graders in fall, 2011, making a total of 39 academy sections in both algebra and geometry offered to a total of 470 students who completed the academy. The 39 academies were hosted across 13 different middle and/or high school sites for Algebra and 10 different sites for Geometry in the participating districts. Participating students were drawn from 11 districts in both Alameda and Contra Costa counties (with one district in San Benito County). A high number of these districts include a significant percentage of students who receive free and/or reduced lunch under Federal guidelines. Free and/or reduced lunch percentage is an indicator used by schools to identify socioeconomic status. At the same time, many of these districts have significant numbers of often underrepresented students in STEM pathways (e.g., Latino, African American). Findings from analysis of student enrollment show that the 2011 program had a 68% retention rate in Algebra and a 72% retention rate in Geometry. Students completed a pre and/or post assessment of math content knowledge at the beginning and end of the four-week summer session. The assessments in both algebra and geometry (one for each discipline) consisted of 35 multiple choice items. Findings from analyses of pre/post student assessments showed overall growth in math content knowledge in both algebra and geometry academies. Based on analyses of assessments for all students enrolled in the program, taking a pre and/or post-test, raw percentages show a growth in algebra content knowledge from a mean of 40% for all students taking a pre-test to 52% for those students taking a post test. Findings also indicate growth in geometry academies from a mean of 37% for all students taking a pre-test to 50% for those students taking a post test. 5 It is also possible to isolate those students (N=295) taking both the pre and the post-test in algebra. This analysis is able to show growth, for these students, of 3.83 points (between averages). There was a 6.12 difference between the pre-test average and the post-test averages for students taking both a pre and a post test in geometry, an increase twice as great as that for algebra. Finally, while all classes, examined broadly, show growth in mean student content knowledge, when findings are disaggregated by district and class, they show that 16 out of 22 classes (academies) in algebra showed statistically significant improvement. In addition, when findings are disaggregated by district and class in geometry, 15 out of 17 classes showed statistically significant improvement. As students took pre and post-tests of content knowledge, they were also asked to assess the degree of difficulty (simple, challenging, difficult) of each of the 35 items as they came to it (self-assessment, opinion). Findings from analyses in algebra and geometry of student assessment of difficulty showed the overall average reported difficulty for each item decreased between the pre and the post-test. Students increasingly indicated items as simple by the post-test and decreasingly indicated items as challenging and/or difficult. In addition, when further analyzed, the findings make visible a progressive shift between pre and post-tests in both Algebra and Geometry Academies in students self-reporting from high levels of difficulty of test items to increasingly perceiving items as ‘simple’, or less ‘challenging’ or ‘difficult’. Findings from the evaluation of the 2011 Math Achievement Academies, support the conclusion that the program is having a positive impact on average student growth in content knowledge, as evidenced by the overall growth in algebra. Even if students attend only one year of summer academy, algebra results support the summer academy approach that emphasizes preparation rather than remediation. Additionally significant are the differences in growth in geometry academies. It is recommended that additional attention be paid to this finding in Year 3, in order to assess student participation, opportunities for learning, and the impact of the program on those students who attended academies for multiple years. The findings from this analysis of the introduction of the geometry academies in 2011 supports the positive potential and consequences for student learning of a long-term approach to student preparation for entering gatekeeper math courses. 6 Mathematics Achievement Academies – 2011 Evaluation Report Introduction In Summer, 2011, the Mathematics Achievement Academies (MAA) program began Year 2 of a three year program to support middle and high school students in enhancing their capacity to successfully complete Algebra I, Geometry and Algebra II (Summer, 2012) courses. These classes are often considered ‘gatekeeper’ courses. Having the capacity to complete them, without requiring remedial coursework, can help to ensure that these students, most from underrepresented groups, will be prepared to enter college pathways (including California State University, East Bay) as freshmen. Primarily funded through the generosity of Chevron, U.S.A., Inc., through its California Partnership, MAA is administered by the Center for Mathematics Education and Research (CMER) at California State University, East Bay, directed by Dr. Julia Olkin and Dr. Phil Duren. In 2011 CSUEB collaborated with Alameda County Office of Education (ACOE), and since 2012, that collaboration is with West Contra Costa USD, with Phil Gonsalves. The project serves students throughout Alameda and Contra Costa Counties. Additional financial support from the Dean and Margaret Lesher Foundation supported further expansion into Contra Costa County, starting in 2012. Designed to assist in preparing students, who have previously struggled with math classes, for college-preparatory mathematics, MAA is a four-week (sixty hours) mathintensive summer program. The interactive classes and activities emphasize rigorous content, collaboration, creative thinking, and logical reasoning. The program is not meant as a substitute for courses offered during the school year, nor as remedial support. Rather the summer academies can be seen as a way of enhancing student capacity in order to successfully access and make meaning from complex content as they enter these ‘gatekeeper’ courses in high school. According to documents produced by or for the program, Year 2 was intended to both build on and expand what occurred in Year 1 (O’Driscoll, 2010; Olkin, Nov., 2011). To that end, MAA offered 22 academies (sections) in Algebra to students who would be entering 9th grade and enrolled in Algebra I in Fall, 2011. The 22 academies for this second cohort of students reflect an increase of 2 over the previous year’s (2010) offerings in algebra. In addition, consistent with the long term approach of the 3-year program, 17 academies (sections) were added and offered in Geometry for students entering that discipline as 10th graders in fall, 2011, making a total of 39 academy sections in both algebra and geometry offered to a total of 470 students who completed the academy, taking both pre and post assessments (see further discussion of participants below). This final evaluation report for 2011 will present an overview of the program design and participants, as well as a discussion of findings from analyses of pre and post-test measures of student content knowledge and from analyses of student attitudes toward degree of difficulty of test items. The report will also provide recommendations for future directions. 7 Program Design Overview Designed to prepare students entering Algebra I or Geometry in 9th or 10th grade, the intent of the MAA approach in 2011 was to support students who have previously struggled in math classes by providing them with an intensive preview of/ introduction to material they would later see in these algebra and/or geometry courses. Doing so through approaches that emphasized active learning, collaboration, opportunities to engage in logical reasoning and creative thinking in order to solve problems, and rigorous content instruction, was intended to increase student capacity and confidence in that capacity to succeed in these courses in 9th and/or 10th grade. The instructional curriculum was originally developed by Dr. Phil Duren, and updated and significantly expanded by Phil Gonsalves, to include Geometry and Algebra II as well as new middle school curriculum. Mr. Gonsalves updates the curriculum annually by getting feedback from teachers in the field. The MAA program was designed as a three year cycle, with students progressing through Algebra I preparation, Geometry I, and then Algebra II preparation across each of the three years. For that reason, students who participated in Algebra I preparation in 2010, could continue in Year 2 (2011), participating in a Geometry I preparation academy. In addition, the approach was seen as occurring in overlapping cycles, because a new cohort was added in 2011 in Algebra I, thus beginning a second cycle of three years for those students. The actual summer session occurred over four weeks, five days per week, for three hours per day (60 hours total). Instructors were primarily drawn from a pool of teachers who had previously participated in professional development courses offered through Project ACCLAIM (the Alameda County Collaborative for Learning and Instruction in Mathematics). These teachers participated in activities that enhanced their capacity to use such pedagogical approaches as showing multiple algorithms side-by-side, decomposition, developing visual representations such as bar models, appropriate use of mathematical syntax and academic language, and other content-focused student engagement strategies. MAA teachers were supported by 1-2 student mentors, most often CSUEB mathematics students, at each host site. These mentors attended academies daily, assisted instructors as needed, and worked with students in small groups. Finally, the MAA program is designed to support students throughout the school year. Study sessions are held prior to key assessments (‘study jams’). Students also have opportunities for tutoring with student mentors throughout the year. One additional goal of MAA is to foster relationships between students’ families and community as well as institutions such as CSU East Bay, and to support families’ capacity to assist their students in pursuing academic and career pathways. To that end, students and their families attend University Nights prior to the start of summer 8 academies, an evening of information about what is required for going to college, how to start planning, both academically and financially, and exposure to potential and possible pathways. The following discussions of district/sites, student participants, and findings from analyses of pre/post test results will assess project impact in the context of the summer academies component of the MAA program. Project Impact Project impact can first be assessed in terms of participating districts and students and will be discussed in the first part of this section of the report. Project Participants Twenty-two Algebra academies were offered in 2011 to students from eleven participating districts in both Alameda and Contra Costa Counties (with one school from San Benito County). Five of these districts were those originally considered Chevron districts (Dublin USD, Livermore Valley JUSD, West Contra Costa USD, Mt. Diablo USD, and Oakland USD). Additional districts included Alameda USD, Fremont USD, John Swett USD, Hayward USD, Antioch USD, and San Benito High School District. Seventeen Geometry academies were offered to students from nine of these same districts (minus Antioch and San Benito districts). The addition of the 17 Geometry academies represented an expansion of the MAA program, adding additional capacity to impact students entering ‘gatekeeper’ mathematics courses in high school. MAA is designed to be a district-based program. Therefore the 39 academies were hosted across 13 different middle and/or high school sites for Algebra and 10 different sites for Geometry in the participating districts. Students were recruited by participating districts (as well as through outreach efforts by CSU East Bay). A variety of local criteria were used for recruitment, including prior student performance in mathematics, with the common criterion that students needed to be entering Algebra 1 in 9 th grade or Geometry in 10th grade in 2011-12. In addition, a priority for the Geometry academies was placed on encouraging students from Year 1 to continue in the program in Year 2. Initial analysis of 2011 Geometry enrollment data indicated that over half of the students enrolled were students continuing from Year 1 (Olkin, 2011 Preliminary Report, November, 2011). Table 1 represents the demographic pool from which students were drawn for both the Algebra and Geometry academies in 2011. Two districts (Antioch, San Benito HS) are those not offering Geometry academies. As can be seen, while there are variations, a high number of these districts include a significant percentage of students who receive free and/or reduced lunch under Federal guidelines, with only two (Dublin, Fremont) below 20% and the other districts ranging between 25.5% (Livermore VJUSD) and 70.3% (Oakland USD). Free and/or reduced lunch percentage is an indicator used by schools to identify socioeconomic status. The same can be said in the area of diversity 9 for those districts with significant numbers of often underrepresented students in STEM pathways (e.g., Latino, African American). Table 1 Demographics of Districts (2010-2011) with Students Enrolled in Algebra and/or Geometry Academies (Summer, 2011) District Total Dist Enr % Free & Red. Lunch - Dist % Ethnic/Racial Enrolllment – District* Latino Hispanic Af. Amer . White, Not Hispanic Asian ELL Filipino. Alameda 10,494 34.3 13.3 12 30.4 32 8.3 USD Antioch USD 19,081 57.8 35 22.4 23.3 5 5 Dublin USD 6257 11.2 14.3 6.6 39.7 27.5 8 Fremont 32,607 19.5 15 4.3 19 51 6 USD Hayward 21,744 62.5 57.3 14.5 8 9 7 USD John Swett 1,735 45.2 31 20 21 9.5 8.2 USD Livermore 12,771 25.5 26.3 2.4 56 6 2.8 VJUSD Mt. Diablo 34,116 39.6 36 5 43 7 4 USD Oakland 46,584 70.3 40 32 8 13 1 USD San Benito HS School 3,072 37.1 63 1 32 1 1.3 Dist West Contra 29.978 68.7 48 14.4 8.6 8 4 Costa USD *Note: Racial and/or Ethnic groups with less than 4% across all represented districts are not represented in this table, primarily for space reasons. 22.3 18.3 9.2 17.4 32.6 16 14 21.4 26 10 33.4 Tables 2 (Algebra academies) and 3 (Geometry academies) below provide a summary of participants in the 2011 academies. Most districts, with the exception of Fremont, Antioch, and Oakland (Geometry academy only), hosted two or more academy ‘sections’ (Column 2), taught by one teacher per section (Column 4). These tables offer a perspective on the fluid nature of enrollment in many of the academies. Column 5 in each table indicates the total enrollment in a given academy section, by district and individual teacher/class. Column 6 (Matching Pre/Post Test) represents the number of students completing the entire academy, taking both a pre and a post math content assessment. Column 7, on the other hand represents the number of students taking the pre-test, including those students who enrolled but did not complete the academy, dropping out of the section at some point before the post-test, while Column 7 shows the number of students taking the post-test, including those students who enrolled in the academy sometime after the pre assessment had been given. 10 Table 2 Summary of Participants – Test Data Source - Algebra Algebra No. 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 District 2 Alameda USD Alameda USD Antioch USD Dublin USD Dublin USD Fremont USD Hayward USD Hayward USD John Swett USD John Swett USD Livermore VJUSD Livermore VJUSD Mt. Diablo USD Mt. Diablo USD Sites Hosting Classes 3 Alameda HS Alameda HS Fremont Elementary School Wells MS Wells MS Washington HS Brenkwitz HS F1, F2, F3, F4 Brenkwitz HS F1, F2, F3, F4 Rodeo Hills ES Rodeo Hills ES Livermore HS Livermore HS Riverview MS Riverview MS 15 16 Mt. Diablo USD Mt. Diablo USD 17 18 19 20 21 22 Oakland USD Oakland USD San Benito High School District San Benito High School District West Contra Costa USD West Contra Costa USD Pleasant Hill MS Pleasant Hill MS Colesium College Prep Academy Fremon HS San Benito HS San Benito HS Crespi HS Crespi HS Total Teacher Sending Students 4 A06 A15 A11 A20 A13 A03 A10 A16 A12 A01 A18 A02 A08 A19 Total Enrollment 5 34 33 29 36 36 29 24 26 11 11 14 7 16 17 Matching (Pre&Post) 6 30 23 24 8 9 17 19 18 8 5 8 5 13 13 Pre Test 7 34 32 29 26 28 28 24 26 11 11 14 7 16 17 Post Test 8 30 24 24 18 15 18 19 18 8 5 8 5 13 13 Dropout Rate 9 0.12 0.28 0.17 0.69 0.68 0.39 0.21 0.31 0.27 0.55 0.43 0.29 0.19 0.24 A22 A05 18 26 9 18 14 22 13 22 0.36 0.18 A07 A21 A14 A04 A09 A17 25 18 17 11 14 19 17 12 17 7 9 6 24 18 17 10 13 17 18 12 17 8 10 8 0.29 0.33 0.00 0.30 0.31 0.65 22 471 295 438 326 0.33 11 Table 3 Summary of Participants – Test Data Source - Geometry Geometry No. 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 District 2 Alameda USD Alameda USD Dublin USD Dublin USD Fremont USD Hayward USD Hayward USD John Swett USD John Swett USD Livermore VJUSD Livermore VJUSD Mt. Diablo USD Mt. Diablo USD Mt. Diablo USD Oakland USD West Contra Costa USD West Contra Costa USD Total Sites Hosting Classes 3 Alameda HS Alameda HS Wells MS Wells MS Washington HS Brenkwitz HS F1, F2, F3, F4 Brenkwitz HS F1, F2, F3, F4 Rodeo Hills ES Rodeo Hills ES Livermore HS Livermore HS Riverview MS Riverview MS Pleasant Hill MS Colesium College Prep Academy Crespi HS Crespi HS Teacher Sending Students 4 G13 G02 G17 G09 G12 G06 G15 G16 G05 G03 G10 G08 G11 G01 G07 G04 G14 Total Enrollment 5 19 22 15 16 7 16 15 14 16 14 14 18 13 23 14 5 10 Matching (Pre&Post) 6 14 15 15 7 4 14 12 9 8 10 12 11 7 18 8 4 7 Pre Test 7 19 22 15 14 7 15 15 14 16 13 14 17 12 19 14 5 10 Post Test 8 14 15 15 9 4 15 12 9 8 11 12 12 8 22 8 4 7 Drop-out Rate 0.26 0.32 0.00 0.50 0.43 0.07 0.20 0.36 0.50 0.23 0.14 0.35 0.42 0.05 0.43 0.20 0.30 17 251 175 241 185 0.28 12 As shown in Table 2 above, over the course of the Algebra academies, 471 students were enrolled and served in some way. Students enrolled and completing the pre-test numbered 438, while 326 completed only the post-test, having enrolled some time during the four-week academy. Two hundred and ninety-five students, then, completed all of one four-week Algebra academy, taking both a pre and a post test (with 31 additional students completing the academy but having missed taking the pre-test). No data is available disaggregating students based on the number of actual days they were enrolled, so that the number of students taking only the post-test might represent students who were enrolled for almost the entirety of the academy as well as some who enrolled much later. The same is true of those who dropped out prior to completing the post assessment. Table 3 makes visible that 251 students were enrolled in the Geometry academies, with 175 completing both the pre and post assessments (and ten additional students completing the academy but having missed taking the pre-test). Both Tables 2 and 3 show, in Column 8, drop-out rates for the academies (an average of .33 in Algebra and of .28 in Geometry). It is important to note, however, that examining these rates from the opposite perspective indicates that the 2011 program had a 68% retention rate in Algebra and a 72% retention rate in Geometry. Findings – Pre/Post Math Content Tests Overview of Overall Pre/Post Math Content Assessment Data in Algebra and Geometry Students completed a pre and/or post assessment of math content knowledge at the beginning and end of the four-week summer session. The assessments in both algebra and geometry (one for each discipline) consisted of 35 multiple choice items. The algebra assessment, given to students enrolled in the Algebra Academy, included content such as absolute value, simplifying algebraic expressions, solving algebraic equations, area, slope, quadratic equations, graphs, and functions. Items were the same for both pre and post-tests. Figure 1 shows, broadly, the overall growth in student content knowledge in the MAA academies across all sections, based on comparison of mean scores (of # correct out of 35 items). In this case, however, data was analyzed for all students taking a pre-test, a post test, or both a pre and a post test, so results in this analysis are somewhat skewed. However, raw percentages from these findings indicate growth in algebra academies from a mean of 40% for all students taking a pre-test to 52% for those students taking a post test (numbers broken down in Table 2). In addition, findings also indicate growth in geometry academies from a mean of 37% for all students taking a pre-test to 50% for those students taking a post test. While this particular figure does not accurately reflect pre/post findings since it is not disaggregated for those students taking both assessments, it does point to the possibility of significant overall growth in math content knowledge in both algebra and geometry academies. 13 Figure 1 MAA 2011 Student Content Knowledge Data: Mean Growth Across All Classes/Teachers (All Students Taking Pre and/or Post Tests) 20 15 Pre-Test 10 Post-Test 5 0 Algebra Geometry Further understanding of the assessments can be gained by examining Figures 2 and 3 below. As can be seen initially in Figure 2, like Figure 1, the N (sample) for pre-test is 438, while the N (sample) for the post test is 326. The figure shows the distribution of scores for pre and post-tests, using mean or average number of items answered correctly, out of 35, based on 438 students who took the pre-test (14.043 average items correct) and 326 students who took the post-test (18.230 average items correct). The distribution curve does make visible a general picture of a shift in student performance on the assessments. As the figure shows, by the post-test, more students, as a group, were answering more items correctly, with more students getting almost all items correct than in the pre-test. By allowing for the average sample size for students who completed the algebra academy, and using these scores, it is also possible to isolate those students (N=295) taking both the pre and the post-test. This analysis is able to show growth, for these students, of 3.831 points (between averages). This analysis supports the finding that there was, on average, a statistically significant increase in student knowledge of algebra content. Figure 3 shows that overall results for the geometry academies are consistent with findings for the algebra academies. The distribution is more clearly visible in the geometry data and is supported when the data is adjusted for the 175 students taking 14 Figure 2 Overall Analysis of Pre/Post Test Data – Algebra Academies Teacher Name: All Teachers Class: Algebra 1. Overall Analysis 1.1. Participating Students Table 1: Descriptive Statistics Type of Test N Mean Median Pre-Test Post-Test 13.000 17.000 438 326 14.043 18.230 St. Deviation Min (out of Max (out of 35) 35) 5.492 6.604 0.000 5.000 32.000 34.000 1.2. Class Distribution Figure 1: Normal Curve Histogram of pre-test score, post-test score Normal 0.08 Variable pre-test score post-test score 0.07 Mean StDev N 14.04 5.492 438 18.23 6.604 326 Density 0.06 0.05 0.04 0.03 0.02 0.01 0.00 0 6 12 18 Data 24 30 1.3. Hypothesis Test Test Statistic: 14.70 Difference of Averages: 3.831 Sample Size: 295 P-value: <0.0005 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 15 Figure 3 Overall Analysis of Pre/Post Test Data – Geometry Academies Teacher Name: All Teachers Class: Geometry 1. Overall Analysis 1.1. Participating Students Table 1: Descriptive Statistics Type of Test N Mean Median Pre-Test Post-Test 11.000 17.000 241 185 10.975 17.541 St. Deviation Min (out of Max (out of 35 35) 4.555 5.678 0.000 1.000 24.000 33.000 1.2. Class Distribution Figure 1: Normal Curve Histogram of pre-test score, post-test score Normal 0.09 Variable pre-test score post-test score 0.08 Mean StDev N 10.98 4.555 241 17.54 5.678 185 0.07 Density 0.06 0.05 0.04 0.03 0.02 0.01 0.00 0 6 12 18 Data 24 30 1.3. Hypothesis Test Test Statistic: 15.19 Difference of Averages: 6.120 Sample Size: 175 P-value: <0.0005 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 16 both the pre and post-test. In this case, there was a 6.120 difference between the pretest average and the post-test averages, an increase twice as great as that for algebra. In order to further understand pre and post-test findings in terms of project impact, it is necessary to examine test data by district and then by class, which will be the focus of the following sub-section. Algebra – Pre/Post Test Findings by District and Class As discussed above, when examined broadly, it appears that all districts represented by students in the Algebra Academies in 2011 showed growth in mean student content knowledge. Table 4 presents an analysis of mean pre-test scores and mean post-test scores for all students, without adjustment for students taking only the pre and post-test. Table 4 Algebra - Growth in Mean Student Knowledge by District (All Students Taking Pre and/or Post Test) School District Pre-Test – Mean % Correct Alameda USD Mt. Diablo USD Fremont USD John Swett USD San Benito High School District Oakland USD Antioch USD Hayward USD Dublin USD Livermore VJUSD West Contra Costa USD All (Mean) 39% 44% 49% 35% 36% 33% 41% 39% 46% 50% 31% 40% Post-Test – Mean % Correct 55% 58% 60% 46% 47% 43% 50% 48% 54% 57% 37% 47% % Point Increase 16% 14% 11% 11% 11% 10% 9% 9% 8% 7% 6% 10% Findings presented in Table 4 represent mean numbers converted to percent of total items correct (out of 35). When analyzed in this way, it is possible to see that Alameda USD’s students’, for example, mean number of correct answers increased 16%, while students from West Contra Costa USD, on average, increased by 6% (the difference between mean percent of items correct). Table 5 presents a further analysis of districts, disaggregated by class, and adjusted only for students taking both pre and post-tests (by allowing for sample size for this group as a test statistic and adjusting differences in mean scores based on sample size and further identifying whether these differences between averages are statistically significant enough to show improvement). This analysis (Table 5) makes visible, for example, that while one class (A20) representing Dublin USD made only 0.63 average point growth, the second class showed growth of 4.56 points between averages. The 0.63 differences between 17 averages, while not a decrease, is not considered statistically significant as improvement over time, but, as the analysis of broad mean scores in Table 4 shows, Dublin USD students, as a whole, grew 8% overall in pre and post mean scores. Table 5 Differences in Pre and Post Averages by District and by Class, with Enrollment Retention Rates by Class - Algebra District Teacher N=# Taking Pre and Post 30 23 Enrollment Retention Rate Diff Between Mean Scores Statistically Significant to Indicate Improvement? A06 A15 Diff. Betw. Pre & Post Mean 7.18 4.83 Alameda USD .88 .72 Yes Yes Antioch USD A11 2.046 24 .83 Yes A20 A13 0.63 4.56 8 9 .31 .32 Yes Fremont USD A03 3.882 17 .61 Yes Hayward USD A10 A16 1.53 4.22 19 18 .79 .69 Yes John Swett USD A12 A01 2.75 1.40 8 5 .73 .45 Yes Livermore VJUSD A18 A02 5.75 2.80 8 5 .57 .80 Yes Mt. Diablo USD A08 A19 A22 A05 3.31 2.923 7.22 5.61 13 13 9 18 .81 .76 .64 .82 Yes Yes Yes Yes Oakland USD A07 A21 2.471 3.08 17 12 .71 .67 Yes Yes San Benito HS District A14 A04 4.790 0.71 17 7 100 .70 Yes West Contra Costa USD A09 1.13 9 .69 A17 2.83 6 .35 Yes .68 16 Yes, 6 No Dublin USD Total Average 3.831 No No No No No No 18 Figures 4 and 5 present a visual representation of two classes within the same district and analyses of students’ average growth in algebra content knowledge over time within each of these classes (academy sections). As shown in Table 2, Oakland Unified School District (represented in Figures 4 and 5 below) fell exactly in the middle (10%) of mean percentage growth between pre and post assessments, the same percentage as that of the average growth across all districts and classes. At the same time, Oakland students were drawn from a pool with the highest percentage of those receiving free and reduced lunch (70.3 %), with its percentage of underrepresented students somewhere near the top of the range represented by all participating districts (e.g., 40% Latino, 32% African American). In addition, with a total enrollment across both classes of 43 students, Oakland USD classes had retention rates of 71% and 67% respectively. Oakland’s retention rates, particularly that of Teacher A21, are consistent with the average retention rate across all classes (68%). Finally, as shown in Table 5, both classes showed improvement in average scores when the sample was adjusted for students taking both the pre and the post assessment, with students of Teacher A21 (3.08 average point increase) showing a slightly higher difference between averages than those of Teacher A07 (2.471 average points). Again, the differences between averages for each class are between .75 and 1.36 points of the average point difference (3.81) across all classes. For these reasons, Oakland USD has been chosen to serve as an example of further analyses of each individual class (see Appendix I). The class distribution of both Oakland USD classes shown in Figures 4 and 5 below, then, show a shift in the ‘normal curve’ that is consistent with that representing the distribution of student scores overall in algebra (all districts and classes) across pre and post assessments, as represented in Figure 2. At the same time, Oakland’s shift in distribution is representative visually and statistically of shifts in student content knowledge in most classes when they are analyzed individually. That is, as shown in Figure 4, for example, more students in Teacher A07’s class were getting more test items correct in the post assessment than they were in the pre-test and there was a shift ‘to the right’ of the normal curve distribution. In other words, where students started in the pre-test (minimum number of items correct within the range of correct items) was lower than the minimum number of items correct received by the lowest scorer on the post-test. In Teacher A07’s class, that lowest (minimum) number correct was 5 in the pre-test and 8 in the post-test, while the lowest (minimum) number in Teacher A21’s class (Figure 5) for the pre-test was 7 and for the post-test, 10. And, as noted, the maximum score (the highest number correct within the range) was higher for both classes in the post-test than in the pre-test (i.e., students scoring highest on the test received more correct answers in the post test than did the highest scorers in the pretest). Similar individual analyses of all algebra academy classes can be found in Appendix I and further support the overall findings for the 2011 Algebra Academies discussed in this section. 19 Figure 4 Teacher: Class: School: District: A07 Algebra Colesium College Prep Academy Oakland USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N Mean 24 18 11.000 14.560 Median 11.000 12.000 St. Deviation 3.648 5.390 Min (out of Max (out of 35) 35) 5.000 8.000 17.000 27.000 2. Class Distribution Figure 1: Normal Curve Histogram of pre-test score, post-test score Normal 0.12 Variable pre-test score post-test score 0.10 Mean StDev N 11 3.648 24 14.56 5.393 18 Density 0.08 0.06 0.04 0.02 0.00 5 10 15 Data 20 25 3. Hypothesis Test Test Statistic: 3.30 Difference of Averages: 2.471 Sample Size: 17 P-value: 0.002 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 20 Teacher: Class: School: District: A21 Algebra Fremont HS Oakland USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N Mean 18 12 12.000 15.750 Median 12.000 14.000 St. Deviation 4.201 5.360 Min (out of Max (out of 35) 35) 7.000 10.00 23.000 28.000 2. Class Distribution Figure 1: Normal Curve Histogram of pre-test score, post-test score Normal 0.10 Variable pre-test score post-test score Density 0.08 Mean StDev N 12 4.201 18 15.75 5.362 12 0.06 0.04 0.02 0.00 5 10 15 Data 20 25 3. Hypothesis Test Test Statistic: 3.01 Difference of Averages: 3.08 Sample Size: 12 P-value: 0.006 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 21 Geometry – Pre/Post Test Findings by District and Class Findings for pre and post assessments of content knowledge for students enrolled in the Geometry Academies, when the nine participating districts are analyzed individually, are similar to those presented for the Algebra Academies, as shown in Table 6. When broad mean averages are analyzed (of all students taking a pre and/or a post-test) by district, all districts showed a significant increase in mean scores, ranging from 8% (Oakland USD) to 30% (JohnSwett USD) increase in mean scores (# of items correct out of 35 total items). Table 6 Geometry - Growth in Mean Student Knowledge by District (All Students Taking Pre and/or Post Test) School District Pre-Test – Mean % Correct John Swett USD Livermore VJUSD West Contra Costa USD Alameda USD Fremont USD Hayward USD Dublin USD Mt. Diablo USD Oakland USD All (Mean) 22% 36% 25% 28% 34% 33% 26% 33% 27% 29% Post-Test – Mean % Correct 52% 61% 50% 51% 57% 54% 45% 46% 35% 50% % Point Increase 30% 25% 25% 23% 23% 21% 19% 13% 8% 21% Findings, when analyzed by district, support initial overall findings in that these broad percentage point increases between pre and post-assessments are significantly higher for districts with students enrolled in geometry than for those enrolled in algebra, although all participating districts in the algebra academies also showed increases in mean scores between pre and post-tests. This again raises questions about the potential positive impact of the program, including its long-term approach which encourages students to continue each year for three years. While no cause/effect correlation can be claimed, we can at least posit that greater increases in content knowledge between pre and post assessments in geometry might be potentially linked to the fact that many of those students enrolled in the geometry academies were continuing students. Those students potentially had past experience to draw on as resource in order to understand what counted as participation in the academies. They had past experience in algebra preparation and a year in this gatekeeper class (Algebra I). Experience in Algebra I might also be the case for other students who were not continuing from the previous year. Again, these findings raise new questions about potential program impact, supporting the concept that continuity is important. Analyses of districts led, again, to the need to examine individual class (academy section) and district findings. Table 7 below presents analyses by district and by class of the differences between pre and post-test averages when sample size is adjusted for 22 only those students taking both the pre and the post-assessment. In this case, while all classes showed some difference between averages, only two classes had results that could not be considered statistically significant to indicate improvement. Table 7 Differences in Pre and Post Averages by District and by Class, with Enrollment Retention Rates by Class - Geometry District Teacher N=# Taking Pre and Post 14 15 Enrollment Retention Rate Diff Between Mean Scores Statistically Significant to Indicate Improvement? G13 G02 Diff. Betw. Pre & Post Mean 3.36 6.02 Alameda USD .74 .68 Yes Yes Dublin USD G17 G09 2.83 5.51 6 7 1.00 .50 Yes Fremont USD G12 1.08 4 .57 Hayward USD G06 G15 4.82 4.89 15 12 .93 .80 Yes Yes John Swett USD G16 G05 2.23 1.92 9 8 .64 .50 Yes Yes Livermore VJUSD G03 G10 5.60 9.417 10 12 .77 .86 Yes Yes Mt. Diablo USD G08 G11 G01 4.09 7.00 4.72 11 7 18 .65 .58 .95 Yes Yes Yes Oakland USD G07 2.875 8 .57 Yes West Contra Costa USD G04 G14 8.75 7.86 4 7 .80 .70 Yes Yes .72 15 Yes, 2 No Total Average 6.120 No No When analyzed in this way, the average difference between mean scores (pre and posttest) is 6.12 (3.81 for Algebra Academies). At the same time, the Geometry Academies, on average, had a 72% retention rate (68% retention rate in algebra). This set of analyses of individual classes adjusted for those students taking both the pre and the post test (N=175) again leads to potential questions about the impact of encouraging students to participate in the Math Achievement Academies across multiple years. 23 The two classes drawing students from Alameda USD (Teacher G13 and Teacher G02) have been selected to make visible a more micro analytic layer of individual classes within districts. In this case, the district selected again fell somewhere in the middle of the nine districts in terms of broad mean percentage score increase between pre and post assessments of geometry content knowledge, as shown in Table 6 (23%). Alameda USD falls somewhere in the mid-range of Federal free and/or reduced lunch rates (34%), and in the lower range, compared to other districts, in its percentage of students identified as Latino or African American (13% and 12% respectively). In addition, Alameda classes had retention rates of 74% (Teacher G13) and 68% (Teacher G02), each within 2 and 4 percentage points of the overall average retention rates across all classes (68%), as shown in Table 7. Finally, while Teacher G02’s class had approximately twice as large a point difference between pre/post averages (3.36) as Teacher G13 (6.02), both classes showed significant improvement. As shown in Figures 6 and 7 below, both classes from Alameda USD show a significant shift in class distribution between pre and post assessments. Class G13 shows, for example, a significant shift in the numbers of students falling within a narrower range of scores (8 points between the lowest and the highest score on the post test, in contrast to 14 points between lowest and highest on the pre-test). In addition, the ‘starting point’ or minimum score in the range rose from 1 to 11 on the post-test. While students on average in class G13 made this growth, the distribution is narrower in that the range was narrower than that of Class G02. As Figure 7 shows, Class G02 had more students receiving more correct answers in the post test. At the same time, their ‘starting point’ (the lowest number of correct answers – minimum) on the post test was higher than that of Class G13. That minimum number of correct items on the post-test (13) for Class G02 was significantly higher, too, than its lowest score on the pre-test (0). The findings from Alameda USD classes are consistent with those shown in the overall distribution for all classes, all teachers, shown in Figure 2. Similar individual analyses of all geometry academy classes can be found in Appendix II and further support the overall findings for the 2011 Geometry Academies discussed in this section. 24 Figure 6 Teacher: Class: School: District: G13 Geometry Alameda High School Alameda Unified School District 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N Mean 19 14 9.789 15.286 Median 10.000 15.500 St. Deviation 3.473 2.555 Min (out of Max (out of 35) 35) 1 11 15 19 2. Class Distribution Figure 1: Normal Curve Class Distribution Normal 0.16 Test Ty pe PostTest PreTest 0.14 Mean StDev N 15.29 2.555 14 9.789 3.473 19 Density 0.12 0.10 0.08 0.06 0.04 0.02 0.00 4 8 12 Test Scores 16 20 Geometry 3. Hypothesis Test Average Difference: 3.36 T-statistic: 4.66 Sample Size: 14 P-value: <0.0005 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 25 Figure 7 Teacher: G02 Class: Geometry School: Alameda High School District: Alameda Unified School District 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N Mean 22 15 9.82 20.07 Median 11.00 19.00 St. Deviation 5.88 5.50 Min (out of Max (out of 35) 35) 0 13 22 33 2. Class Distribution Figure 1: Normal Curve Class Distribution Normal 0.08 Test Ty pe Post Test Pre Test 0.07 Mean StDev N 20.07 5.496 15 9.818 5.885 22 Density 0.06 0.05 0.04 0.03 0.02 0.01 0.00 0 8 16 Test Scores 24 32 Geometry 3. Hypothesis Test Average Difference: 6.02 T-statistic: 7.29 Sample Size: 15 P-value: <0.0005 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 26 Findings – Pre and Post Student Self-Assessment of Item Difficulty – Algebra and Geometry As students took pre and post-tests of content knowledge, they were also asked to assess the degree of difficulty of each of the 35 items as they came to it (selfassessment, opinion). Students were asked to make these assessments in both algebra (Algebra Academy pre and post-test) and geometry (Geometry Academy pre and post-test). Findings from analyses of pre and post student self-assessment of item difficulty in both algebra and geometry are discussed in this section. Students were asked, on both the pre and the post test, to rate each item, as they completed it, according to their assessment of its degree of difficulty. Students were asked to give their opinion as to whether they considered the item to be simple, challenging, or difficult. It was hypothesized that the number of items rated as challenging or difficult should decrease between the pre and the post test and the number rated simple should increase, thus serving as an indicator of increased student confidence in their capacity as well as increase in that capacity itself to solve algebraic or geometric problems. A sample item similar to those of the Geometry assessment is shown below. In addition, the place for the student to give his/her opinion about the degree of difficulty of the item is also shown. Items on the Algebra assessment are similarly formatted. 7 What is the distance between points A (3, -7) and B (9, -2)? A) 61 B) 117 C) 61 D) 25 Simple Challenging Difficult 27 In Analysis I, the average degree of difficulty indicated by students on both the pre and the post-test was calculated and contrasted for all 35 items for both Algebra and Geometry academies. Whether or not students answered items correctly is not correlated with their indication of difficulty in this analysis. However, as shown in both Figures 8 (Algebra) and 9 (Geometry), below, while there is some variation among items, the overall average reported difficulty for each item decreased between the pre and the post-test. Figure 8 Average Student-Reported Difficulty across 35 Questions in Algebra – Pre-Test vs. Post-Test Average Difficulty for Algebra 3.00 2.50 2.00 1.50 1.00 0.50 0.00 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 PRE POST Figure 9 Average Student-Reported Difficulty across 35 Questions in Geometry – Pre-Test vs. Post-Test Average Difficulty for Geometry 3.00 2.50 2.00 1.50 1.00 0.50 0.00 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 PRE POST 28 A second layer of analysis, presented in Figures 10 (Algebra) and 11 (Geometry), makes visible even more clearly, on an item by item basis, the shifts in student perception of difficulty between the pre and the post test. In this case, each item is broken down according to percentage of students reporting that item as simple (blue), challenging (green), or difficult (red). Figures 10 and 11, first, clearly support the initial hypothesis that students would increasingly indicate items as simple by the post-test and decreasingly indicate items as challenging and/or difficult. As indicated by the blue portions of the lines for each item in Figure 10, the percentage of students indicating that they saw an item as ‘simple’ on the Algebra pre-test, doubled, and often tripled, if not more, by the post test. In Geometry (Figure 11), the same is true for most items. At the same time, the numbers of students indicating those same items as challenging decreased significantly, as indicated by the shortened portions of each line colored green on the post-test, in both algebra and geometry. In addition, as both figures show, some items increased significantly in the average number of students reporting them as simple, such that they also decreased significantly in numbers identifying those same items as either challenging or difficult (e.g., items 1 and 35 in algebra; items 1 and 29 in geometry). For many items, however, analysis showed, as represented in both Figure 10 and Figure 11, that the average number of items marked as ‘difficult’ (red) in contrast to ‘challenging’ (green), did not decrease significantly. This might be explained in several ways. Since, together, items marked challenging and difficult decreased in the post-test for algebra and geometry, it might be argued that some students had difficulty distinguishing between the two choices. It also might be argued that more students taking only the post-test (see Tables 2 and 3 above), selecting level of difficulty for the first time, may have perceived more items as difficult. The findings discussed in this section make visible a progressive shift between pre and post-tests in both Algebra and Geometry Academies in students self-reporting from high levels of difficulty of test items to increasingly perceiving items as ‘simple’, or less ‘challenging’ or ‘difficult’. 29 Figure 10 Shifts in Student Assessment of Difficulty Between Pre and Post Tests - Algebra Pre Post 34 34 32 32 30 30 28 28 26 26 24 24 22 22 20 20 18 18 16 16 13 13 11 11 9 9 7 7 5 5 3 3 1 1 0% 20% Simple 40% 60% Challenging 80% Difficult 100% 0% 20% Simple 40% 60% Challenging 80% 100% Difficult 30 Figure 11 Shifts in Student Assessment of Difficulty Between Pre and Post Tests – Geometry Pre Post 34 34 32 32 30 30 28 28 26 26 24 24 22 22 20 20 18 18 16 16 13 13 11 11 9 9 7 7 5 5 3 3 1 1 0% 20% Simple 40% 60% Challenging 80% Difficult 100% 0% 20% Simple 40% 60% Challenging 80% 100% Difficult 31 Concluding Remarks Overall findings from analyses of assessments of student content knowledge following participation in the summer Math Achievement Academies indicates a significant level of success in terms of student growth. Findings support the conclusion that the program is having a positive impact on average student growth in content knowledge, as evidenced by the overall growth in algebra. Even if students attend only one year of summer academy, algebra results support the summer academy approach that emphasizes preparation rather than remediation. Additionally significant are the differences in growth in geometry academies. It is recommended that additional attention be paid to this finding in Year 3, in order to assess student participation, opportunities for learning, and the impact of the program on those students who attended academies for multiple years. The findings from this analysis of the introduction of the geometry academies in 2011 supports the positive potential and consequences for student learning of a long-term approach to student preparation for entering gatekeeper math courses. In addition, analyses of student self-assessment of item degree of difficulty indicated a strong shift in how students perceived test items in both algebra and geometry. The percentage of students indicating that an item was simple significantly increased, while indications that an item was challenging or difficult significantly decreased between the pre and post-tests. It is recommended that correlation between self-reported degree of difficulty and whether an item is solved correctly be analyzed. It would also be helpful to have test items assessed by test makers as to their perceptions of problem level of difficulty in order to contrast with student perceptions and student actual answers (correct or incorrect). An added recommendation is related to analyses contrasting individual classes. This information might be more useful in future if it were accompanied by some analyses of classes in action – that is, what was made available to students and how students took up opportunities for learning afforded them. This may be a direction to be explored in future. 32 Appendices 33 Appendix I Algebra - Student Content Knowledge Test Analyses by Class (Listed Alphabetically by District) 34 Teacher: Class: School: District: A06 Algebra Alameda HS Alameda USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 34 30 Median 13.500 21.000 Mean 14.353 20.170 St. Deviation 5.493 6.260 Min 3.000 10.000 Max 25.000 31.000 2. Class Distribution Figure 1: Normal Curve Histogram of Pre-Score, Post-Score Normal 0.08 Variable Pre-Score Post-Score 0.07 Mean StDev N 14.35 5.493 34 20.17 6.265 30 Density 0.06 0.05 0.04 0.03 0.02 0.01 0.00 4 8 12 16 20 Data 24 28 32 3. Hypothesis Test Test Statistic: 5.333 Difference of Averages: 7.18 Sample Size: 30 P-value: <0.0005 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 35 Teacher: Class: School: District: A15 Algebra Alameda HS Alameda USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 32 24 Median 12.500 20.00 Mean 12.813 18.580 St. Deviation 4.987 5.580 Min 0.000 8.000 Max 23.000 30.000 2. Class Distribution Figure 1: Normal Curve Histogram of Pre-Score, Post-Score Normal 0.09 Variable Pre-Score Post-Score 0.08 Mean StDev N 12.81 4.987 32 18.58 5.579 24 0.07 Density 0.06 0.05 0.04 0.03 0.02 0.01 0.00 0 8 16 Data 24 32 3. Hypothesis Test Test Statistic: 4.91 Difference of Averages: 4.83 Sample Size: 23 P-value: <0.0005 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 36 Teacher: Class: School: District: A11 Algebra Fremont Elementary Antioch USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 29 24 Mean 14.241 17.625 Median 15.000 17.000 St. Deviation 3.757 4.126 Min 6 10 Max 20 23 2. Class Distribution Figure 1: Normal Curve Class Distribution Normal 0.12 Test Ty pe Post Test Pre Test 0.10 Mean StDev N 17.63 4.126 24 14.24 3.757 29 Density 0.08 0.06 0.04 0.02 0.00 8 12 16 Test Scores 20 24 Algebra 3. Hypothesis Test Average Difference: 2.046 T-statistic: 4.44 Sample Size: 24 P-value: <0.0005 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 37 Teacher: Class: School: District: A20 Algebra Wells MS Dublin USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 26 18 Median 17.000 19.500 Mean 17.654 18.940 St. Deviation 5.012 6.810 Min 10.000 6.000 Max 26.000 29.000 2. Class Distribution Figure 1: Normal Curve Histogram of Pre-Score, Post-Score Normal Variable Pre-Score Post-Score 0.08 0.07 Mean StDev N 17.65 5.012 26 18.94 6.812 18 Density 0.06 0.05 0.04 0.03 0.02 0.01 0.00 5 10 15 20 Data 25 30 35 3. Hypothesis Test Test Statistic: 0.41 Difference of Averages: 0.63 Sample Size: 8 P-value: 0.347 Output summary: We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 38 Teacher: Class: School: District: A13 Algebra Wells MS Dublin USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 28 15 Median 12.50 20.00 Mean 14.61 18.80 St. Deviation 7.10 6.43 Min 0.00 9.00 Max 29.00 26.00 2. Class Distribution Figure 1: Normal Curve Histogram of Pre-Score, Post-Score Normal 0.07 Variable Pre-Score Post-Score 0.06 Mean StDev N 14.61 7.099 28 18.8 6.428 15 Density 0.05 0.04 0.03 0.02 0.01 0.00 0 8 16 Data 24 32 3. Hypothesis Test Average Difference: 4.56 T-statistic: 2.00 Sample Size: 9 P-value: 0.040 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 39 Teacher: Class: School: District: A03 Algebra Washington HS Fremont USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 28 18 Median 16.50 20.50 Mean 17.00 21.11 St. Deviation 6.54 6.96 Min 6.00 5.00 Max 29.00 31.00 2. Class Distribution Figure 1: Normal Curve Histogram of Pre-Score, Post-Score Normal Variable Pre-Score Post-Score 0.06 0.05 Mean StDev N 17 6.543 28 21.11 6.961 18 Density 0.04 0.03 0.02 0.01 0.00 5 10 15 20 Data 25 30 35 3. Hypothesis Test Average Difference: 3.882 T-statistic: 5.62 Sample Size: 17 P-value: <0.0005 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 40 Teacher: Class: School: District: A10 Algebra Brenkwitz HS F1, F2, F3, F4 Hayward USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 24 19 Mean 15.750 17.950 Median 15.000 18.00 St. Deviation 4.542 7.830 Min 7.000 7.000 Max 27.000 32.000 2. Class Distribution Figure 1: Normal Curve Histogram of Pre-Score, Post-Score Normal 0.09 Variable Pre-Score Post-Score 0.08 Mean StDev N 15.75 4.542 24 17.95 7.835 19 0.07 Density 0.06 0.05 0.04 0.03 0.02 0.01 0.00 0 8 16 Data 24 32 3. Hypothesis Test Average Difference: 1.53 T-statistic: 1.28 Sample Size: 19 P-value: 0.109 Output summary: We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 41 Teacher: Class: School: District: A16 Algebra Brenkwitz HS F1, F2, F3, F4 Hayward USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 26 18 Median 10.500 14.50 Mean 11.423 15.940 St. Deviation 4.941 7.620 Min 1.000 7.000 Max 23.000 32.000 2. Class Distribution Figure 1: Normal Curve Histogram of Pre-Score, Post-Score Normal 0.09 Variable Pre-Score Post-Score 0.08 Mean StDev N 11.42 4.941 26 15.94 7.619 18 0.07 Density 0.06 0.05 0.04 0.03 0.02 0.01 0.00 0 5 10 15 Data 20 25 30 3. Hypothesis Test Average Difference: 4.22 T-statistic: 3.64 Sample Size: 18 P-value: 0.001 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 42 Teacher: Class: School: District: A12 Algebra Rodeo Hills ES John Swett USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 11 8 Median 12.000 16.000 Mean 12.000 15.130 St. Deviation 2.098 3.910 Min 8.000 9.000 Max 15.000 21.000 2. Class Distribution Figure 1: Normal Curve Histogram of Pre-Score, Post-Score Normal 0.20 Variable Pre-Score Post-Score Mean StDev N 12 2.098 11 15.13 3.907 8 Density 0.15 0.10 0.05 0.00 8 12 16 Data 20 24 3. Hypothesis Test Average Difference: 2.75 T-statistic: 2.62 Sample Size: 8 P-value: 0.017 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 43 Teacher: Class: School: District: A01 Algebra Rodeo Hills ES John Swett USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 11 5 Mean 12.27 17.40 Median 10.00 17.00 St. Deviation 4.41 2.61 Min 8 15 Max 19 21 2. Class Distribution Figure 1: Normal Curve Histogram of Pre-Score, Post-Score Normal 0.16 Variable Pre-Score Post-Score 0.14 Mean StDev N 12.27 4.407 11 17.4 2.608 5 Density 0.12 0.10 0.08 0.06 0.04 0.02 0.00 4 8 12 Data 16 20 3. Hypothesis Test Average Difference: 1.40 T-statistic: 1.03 Sample Size: 5 P-value: 0.181 Output summary: We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 44 Teacher: Class: School: District: A18 Algebra Livermore HS Livermore VJUSD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 14 8 Median 14.50 18.50 Mean 15.86 19.63 St. Deviation 6.42 7.69 Min 7.00 7.00 Max 25.00 30.00 2. Class Distribution Figure 1: Normal Curve Histogram of Pre-Score, Post-Score Normal 0.07 Variable Pre-Score Post-Score 0.06 Mean StDev N 15.86 6.419 14 19.63 7.689 8 Density 0.05 0.04 0.03 0.02 0.01 0.00 5 10 15 20 Data 25 30 35 3. Hypothesis Test Average Difference: 5.75 T-statistic: 2.77 Sample Size: 8 P-value: 0.014 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 45 Teacher: Class: School: District: A02 Algebra Livermore HS Livermore VJUSD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 7 5 Median 16.00 21.00 Mean 18.86 20.40 St. Deviation 6.77 8.41 Min 12.00 7.00 Max 32.00 28.00 2. Class Distribution Figure 1: Normal Curve Histogram of Pre-Score, Post-Score Normal 0.06 Variable Pre-Score Post-Score 0.05 Mean StDev N 18.86 6.768 7 20.4 8.414 5 Density 0.04 0.03 0.02 0.01 0.00 5 10 15 20 Data 25 30 35 40 3. Hypothesis Test Average Difference: 2.80 T-statistic: 0.98 Sample Size: 5 P-value: 0.191 Output summary: We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 46 Teacher: Class: School: District: A08 Algebra Riverview MS Mt. Diablo USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 16 13 Median 12.50 15.00 Mean 12.94 16.77 St. Deviation 4.60 6.60 Min 5.00 9.00 Max 25.00 32.00 2. Class Distribution Figure 1: Normal Curve Histogram of Pre-Score, Post-Score Normal 0.09 Variable Pre-Score Post-Score 0.08 Mean StDev N 12.94 4.597 16 16.77 6.597 13 0.07 Density 0.06 0.05 0.04 0.03 0.02 0.01 0.00 5 10 15 20 25 30 Data 3. Hypothesis Test Average Difference: 3.31 T-statistic: 2.64 Sample Size: 13 P-value: 0.011 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 47 Teacher: Class: School: District: A19 Algebra Riverview MS Mt. Diablo USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 17 13 Median 12.00 17.00 Mean 13.18 16.46 St. Deviation 4.61 6.51 Min 5.00 8.00 Max 22.00 27.00 2. Class Distribution Figure 1: Normal Curve Histogram of Pre-Score, Post-Score Normal 0.09 Variable Pre-Score Post-Score 0.08 Mean StDev N 13.18 4.613 17 16.46 6.514 13 0.07 Density 0.06 0.05 0.04 0.03 0.02 0.01 0.00 5 10 15 Data 20 25 30 3. Hypothesis Test Average Difference: 2.923 T-statistic: 3.56 Sample Size: 13 P-value: 0.002 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 48 Teacher: Class: School: District: A22 Algebra Pleasant Hill MS Mt. Diablo USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 14 13 Median 20.00 30.00 Mean 18.64 26.62 St. Deviation 7.43 7.09 Min 5.00 11.00 Max 30.00 34.00 2. Class Distribution Figure 1: Normal Curve Histogram of Pre-Score, Post-Score Normal 0.06 Variable Pre-Score Post-Score 0.05 Mean StDev N 18.64 7.428 14 26.62 7.089 13 Density 0.04 0.03 0.02 0.01 0.00 5 10 15 20 25 Data 30 35 40 3. Hypothesis Test Average Difference: 7.22 T-statistic: 4.40 Sample Size: 9 P-value: 0.001 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 49 Teacher: A05 Class: Algebra School: Pleasant Hill MS District: Mt. Diablo USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 22 22 Median 17.00 21.00 Mean 16.27 21.73 St. Deviation 5.41 6.81 Min 10.00 8.00 Max 28.00 31.00 2. Class Distribution Figure 1: Normal Curve Histogram of Pre-Score, Post-Score Normal 0.08 Variable Pre-Score Post-Score 0.07 Mean StDev N 16.27 5.409 22 21.73 6.812 22 Density 0.06 0.05 0.04 0.03 0.02 0.01 0.00 5 10 15 20 Data 25 30 35 3. Hypothesis Test Average Difference: 5.61 T-statistic: 4.93 Sample Size: 18 P-value: 0.0005 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 50 Teacher: Class: School: District: A07 Algebra Colesium College Prep Academy Oakland USD 4. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 24 18 Median 11.000 12.000 Mean 11.000 14.560 St. Deviation 3.648 5.390 Min 5.000 8.000 Max 17.000 27.000 5. Class Distribution Figure 1: Normal Curve Histogram of pre-test score, post-test score Normal 0.12 Variable pre-test score post-test score 0.10 Mean StDev N 11 3.648 24 14.56 5.393 18 Density 0.08 0.06 0.04 0.02 0.00 5 10 15 Data 20 25 6. Hypothesis Test Test Statistic: 3.30 Difference of Averages: 2.471 Sample Size: 17 P-value: 0.002 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 51 Teacher: Class: School: District: A21 Algebra Fremont HS Oakland USD 4. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 18 12 Median 12.000 14.000 Mean 12.000 15.750 St. Deviation 4.201 5.360 Min 7.000 10.00 Max 23.000 28.000 5. Class Distribution Figure 1: Normal Curve Histogram of pre-test score, post-test score Normal 0.10 Variable pre-test score post-test score Density 0.08 Mean StDev N 12 4.201 18 15.75 5.362 12 0.06 0.04 0.02 0.00 5 10 15 Data 20 25 6. Hypothesis Test Test Statistic: 3.01 Difference of Averages: 3.08 Sample Size: 12 P-value: 0.006 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 52 Teacher: Class: School: District: A14 Algebra San Benito High School San Benito High School District 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 17 17 Mean 12.53 18.88 Median 12.00 17.00 St. Deviation 5.68 4.24 Min 4 12 Max 24 26 2. Class Distribution Figure 1: Normal Curve Class Distribution Normal Test Ty pe PostTest PreTest 0.09 0.08 Mean StDev N 18.88 4.241 17 12.53 5.680 17 Density 0.07 0.06 0.05 0.04 0.03 0.02 0.01 0.00 0 5 10 15 Test Scores 20 25 Algebra 3. Hypothesis Test Average Difference: 4.790 T-statistic: 7.10 Sample Size: 17 P-value: <0.0005 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 53 Teacher: Class: School: District: A04 Algebra San Benito High School San Benito High School District 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 10 8 Median 13.00 13.50 Mean 13.10 14.13 St. Deviation 3.143 3.310 Min 9 9 Max 18 19 2. Class Distribution Figure 1: Normal Curve Class Distribution Normal 0.14 Test Ty pe Post Test Pre Test 0.12 Mean StDev N 14.13 3.314 8 13.1 3.143 10 Density 0.10 0.08 0.06 0.04 0.02 0.00 8 12 16 Test Scores 20 Algebra 3. Hypothesis Test Average Difference: 0.71 T-statistic: 0.55 Sample Size: 7 P-value: 0.302 Output summary: We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 54 Teacher: A09 Class: Algebra School: Crespi MS District: West Contra Costa USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 13 10 Median 10.000 11.500 Mean 10.308 11.900 St. Deviation 3.401 4.040 Min 5.000 7.000 Max 17.000 18.000 2. Class Distribution Figure 1: Normal Curve Histogram of pre-test score, post-test score Normal 0.12 Variable pre-test score post-test score 0.10 Mean StDev N 10.31 3.401 13 11.9 4.040 10 Density 0.08 0.06 0.04 0.02 0.00 2.5 5.0 7.5 10.0 12.5 Data 15.0 17.5 20.0 3. Hypothesis Test Test Statistic: 1.13 Difference of Averages: 1.67 Sample Size: 9 P-value: 0.145 Output summary: We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 55 Teacher: Class: School: District: A17 Algebra Crespi MS West Contra Costa USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 17 8 Median 12.000 13.500 Mean 11.294 13.880 St. Deviation 2.823 4.120 Min 7.00 9.00 Max 17.00 20.00 2. Class Distribution Figure 1: Normal Curve Histogram of pre-test score, post-test score Normal 0.16 Variable pre-test score post-test score 0.14 Mean StDev N 11.29 2.823 17 13.88 4.121 8 Density 0.12 0.10 0.08 0.06 0.04 0.02 0.00 4 8 12 16 20 Data 3. Hypothesis Test Test Statistic: 1.99 Difference of Averages: 2.83 Sample Size: 6 P-value: 0.052 Output summary: We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 56 Appendix II Geometry - Student Content Knowledge Test Analyses by Class (Listed Alphabetically by District) 57 Teacher: Class: School: District: G13 Geometry Alameda High School Alameda Unified School District 4. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 19 14 Median 10.000 15.500 Mean 9.789 15.286 St. Deviation 3.473 2.555 Min 1 11 Max 15 19 5. Class Distribution Figure 1: Normal Curve Class Distribution Normal 0.16 Test Ty pe PostTest PreTest 0.14 Mean StDev N 15.29 2.555 14 9.789 3.473 19 Density 0.12 0.10 0.08 0.06 0.04 0.02 0.00 4 8 12 Test Scores 16 20 Geometry 6. Hypothesis Test Average Difference: 3.36 T-statistic: 4.66 Sample Size: 14 P-value: <0.0005 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 58 Teacher: G02 Class: Geometry School: Alameda High School District: Alameda Unified School District 4. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 22 15 Median 11.00 19.00 Mean 9.82 20.07 St. Deviation 5.88 5.50 Min 0 13 Max 22 33 5. Class Distribution Figure 1: Normal Curve Class Distribution Normal 0.08 Test Ty pe Post Test Pre Test 0.07 Mean StDev N 20.07 5.496 15 9.818 5.885 22 Density 0.06 0.05 0.04 0.03 0.02 0.01 0.00 0 8 16 Test Scores 24 32 Geometry 6. Hypothesis Test Average Difference: 6.02 T-statistic: 7.29 Sample Size: 15 P-value: <0.0005 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 59 Teacher: Class: School: District: G17 Geometry Wells Middle School Dublin Unified School District 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 15 15 Mean 13.933 16.670 Median 14.000 16.000 St. Deviation 3.127 5.000 Min 9 7 Max 21 27 2. Class Distribution Figure 1: Normal Curve Class Distribution Normal 0.14 Test Ty pe PostTest PreTest 0.12 Mean StDev N 16.67 4.995 15 13.93 3.127 15 Density 0.10 0.08 0.06 0.04 0.02 0.00 5 10 15 20 Test Scores 25 Geometry 3. Hypothesis Test Average Difference: -0.14 T-statistic: 1.68 Sample Size: 15 P-value: 0.058 Output summary: We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 60 Teacher: Class: School: District: G09 Geometry Wells Middle School Dublin Unified School District 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 14 9 Mean 4.500 14.556 Median 5.000 15.000 St. Deviation 3.276 2.603 Min 0 11 Max 11 18 2. Class Distribution Figure 1: Normal Curve Class Distribution Normal 0.16 Test Ty pe PostTest PreTest 0.14 Mean StDev N 14.56 2.603 9 4.5 3.276 14 Density 0.12 0.10 0.08 0.06 0.04 0.02 0.00 0 4 8 12 Test Scores 16 20 Geometry 3. Hypothesis Test Average Difference: 5.51 T-statistic: 4.89 Sample Size: 7 P-value: 0.001 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 61 Teacher: Class: School: District: G12 Geometry Washington High School Fremont Unified School District 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 7 4 Median 12 20 Mean 12 20 St. Deviation 2.236 6.580 Min 8 13 Max 15 27 2. Class Distribution Figure 1: Normal Curve Class Distribution Normal 0.20 Test Ty pe PostTest PreTest Mean StDev N 20 6.583 4 12 2.236 7 Density 0.15 0.10 0.05 0.00 8 12 16 20 24 Test Scores 28 32 Geometry 3. Hypothesis Test Average Difference: -1.08 T-statistic: 2.03 Sample Size: 4 P-value: 0.068 Output summary: We fail to reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 62 Teacher: Class: School: District: G06 Geometry Brenkwitz High School F1, F2, F3, F4 Hayward Unified School District 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 15 15 Mean 10.867 18.070 Median 11.000 19.000 St. Deviation 3.523 4.990 Min 5 8 Max 17 26 2. Class Distribution Figure 1: Normal Curve Class Distribution Normal 0.12 Test Ty pe PostTest PreTest 0.10 Mean StDev N 18.07 4.992 15 10.87 3.523 15 Density 0.08 0.06 0.04 0.02 0.00 5 10 15 20 Test Scores 25 30 Geometry 3. Hypothesis Test Average Difference: 4.82 T-statistic: 5.32 Sample Size: 15 P-value: <0.0005 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 63 Teacher: Class: School: District: G15 Geometry Brenkwitz High School F1, F2, F3, F4 Hayward Unified School District 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 15 12 Median 12.00 20.00 Mean 12.20 19.83 St. Deviation 3.299 6.090 Min 12 20 Max 6 10 2. Class Distribution Figure 1: Normal Curve Class Distribution Normal Test Ty pe PostTest PreTest 0.12 0.10 Mean StDev N 19.83 6.088 12 12.2 3.299 15 Density 0.08 0.06 0.04 0.02 0.00 5 10 15 20 Test Scores 25 30 Geometry 3. Hypothesis Test Average Difference: 4.89 T-statistic: 4.79 Sample Size: 12 P-value: <0.0005 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 64 Teacher: Class: School: District: G16 Geometry Rodeo Hills Elementary School John Swett Unified School District 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 14 9 Median 14.50 20.00 Mean 13.50 19.22 St. Deviation 4.69 5.17 Min 8.75 13.5 Max 19 25 2. Class Distribution Figure 1: Normal Curve Class Distribution Normal 0.09 Test Ty pe PostTest PreTest 0.08 0.07 Mean StDev N 19.22 5.167 9 13.5 4.686 14 Density 0.06 0.05 0.04 0.03 0.02 0.01 0.00 5 10 15 20 Test Scores 25 30 Geometry 3. Hypothesis Test Average Difference: 2.23 T-statistic: 3.83 Sample Size: 9 P-value: 0.002 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 65 Teacher: Class: School: District: G05 Geometry Rodeo Hills Elementary School John Swett Unified School District 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 16 8 Mean 11.625 17.000 Median 11.000 16.500 St. Deviation 2.918 4.630 Min 7 12 Max 16 27 2. Class Distribution Figure 1: Normal Curve Class Distribution Normal 0.14 Test Ty pe PostTest PreTest 0.12 Mean StDev N 17 4.629 8 11.63 2.918 16 Density 0.10 0.08 0.06 0.04 0.02 0.00 5 10 15 20 Test Scores 25 Geometry 3. Hypothesis Test Average Difference: 1.92 T-statistic: 3.24 Sample Size: 8 P-value: 0.007 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 66 Teacher: Class: School: District: G03 Geometry Livermore HS Livermore VJUSD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 13 11 Median 11.00 20.00 Mean 13.54 20.64 St. Deviation 6.12 5.57 Min 6.00 14.00 Max 24.00 31.00 2. Class Distribution Figure 1: Normal Curve Class Distributions Normal 0.08 Test Ty pe Post Test Pre Test 0.07 Mean StDev N 20.64 5.573 11 13.54 6.118 13 Density 0.06 0.05 0.04 0.03 0.02 0.01 0.00 0 5 10 15 20 Stack Score 25 30 3. Hypothesis Test Average Difference: 5.60 T-Value : 2.31 Sample Size: 10 P-Value : 0.023 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 67 Teacher: Class: School: District: G10 Geometry Livermore HS Livermore VJUSD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 14 12 Median 11.500 20.00 Mean 11.929 21.830 St. Deviation 3.668 4.760 Min 7.000 16.000 Max 19.000 28.000 2. Class Distribution Figure 1: Normal Curve Class Distributions Normal 0.12 Test Ty pe Post Test Pre Test 0.10 Mean StDev N 21.83 4.764 12 11.93 3.668 14 Density 0.08 0.06 0.04 0.02 0.00 5 10 15 20 Stack Score 25 30 3. Hypothesis Test Average Difference: 9.417 T-Value : 10.27 Sample Size: 12 P-Value : <0.0005 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 68 Teacher: Class: School: District: G08 Geometry Riverview HS Mt. Diablo USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 17 12 Median 13.00 17.00 Mean 12.71 16.42 St. Deviation 5.32 7.89 Min 0.00 1.00 Max 21.00 27.00 2. Class Distribution Figure 1: Normal Curve Class Distributions Normal 0.08 Test Ty pe Post Test Pre Test 0.07 Mean StDev N 16.42 7.891 12 12.71 5.324 17 Density 0.06 0.05 0.04 0.03 0.02 0.01 0.00 0 8 16 Stack Score 24 32 3. Hypothesis Test Average Difference: 4.09 T-Value: 2.44 Sample Size: 11 P-Value: 0.017 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 69 Teacher: Class: School: District: G11 Geometry Riverview HS Mt. Diablo USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 12 8 Median 11.00 17.00 Mean 10.92 16.50 St. Deviation 4.21 7.33 Min 6.00 7.00 Max 17.00 30.00 2. Class Distribution Figure 1: Normal Curve Class Distributions Normal 0.10 Test Ty pe Post Test Pre Test Density 0.08 Mean StDev N 16.5 7.329 8 10.92 4.209 12 0.06 0.04 0.02 0.00 0 5 10 15 20 Stack Score 25 30 3. Hypothesis Test Average Difference: 7.00 T-Value: 3.92 Sample Size: 7 P-Value: 0.004 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 70 Teacher: G01 Class: Geometry School: Pleasant Hill MS District: Mt. Diablo USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 19 22 Mean 10.895 15.68 Median 12.00 14.50 St. Deviation 3.526 6.03 Min 2.00 6.00 Max 17.000 27.00 2. Class Distribution Figure 1: Normal Curve Class Distributions Normal 0.12 Test Ty pe Post Test Pre Test 0.10 Mean StDev N 15.68 6.027 22 10.89 3.526 19 Density 0.08 0.06 0.04 0.02 0.00 6 12 18 Stack Score 24 30 3. Hypothesis Test Average Difference: 4.72 T-Value: 3.44 Sample Size: 18 P-Value: 0.002 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 71 Teacher: Class: School: District: G07 Geometry Colesium College Prep Academy Oakland USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 14 8 Mean 9.57 12.25 Median 9.00 11.50 St. Deviation 4.11 3.01 Min 1.00 7.00 Max 19.00 16.00 2. Class Distribution Figure 1: Normal Curve Class Distributions Normal 0.14 Test Ty pe Post Test Pre Test 0.12 Mean StDev N 12.25 3.012 8 9.571 4.108 14 Density 0.10 0.08 0.06 0.04 0.02 0.00 0 4 8 12 Stack Score 16 3. Hypothesis Test Average Difference: 2.875 T-Value: 3.37 Sample Size: 8 P-Value: 0.006 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 72 Teacher: Class: School: District: G04 Geometry Crespi MS West Contra Costa USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 5 4 Median 10.00 18.00 Mean 9.60 18.25 St. Deviation 2.70 3.30 Min 5.00 15.00 Max 12.00 22.00 2. Class Distribution Figure 1: Normal Curve Class Distributions Normal 0.16 Test Ty pe Post Test Pre Test 0.14 Mean StDev N 18.25 3.304 4 9.6 2.702 5 Density 0.12 0.10 0.08 0.06 0.04 0.02 0.00 5 10 15 Stack Score 20 25 3. Hypothesis Test Average Difference: 8.75 T-Value: 2.83 Sample Size: 4 P-Value: 0.033 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 73 Teacher: Class: School: District: G14 Geometry Crespi MS West Contra Costa USD 1. Participating Students Table 1: Descriptive Statistics Type of Test Pre-Test Post-Test N 10 7 Median 9.00 18.00 Mean 8.70 16.71 St. Deviation 3.43 6.52 Min 3.00 6.00 Max 13.00 25.00 2. Class Distribution Figure 1: Normal Curve Class Distributions Normal 0.12 Test Ty pe Post Test Pre Test 0.10 Mean StDev N 16.71 6.525 7 8.7 3.433 10 Density 0.08 0.06 0.04 0.02 0.00 4 8 12 16 20 Stack Score 24 28 32 3. Hypothesis Test Average Difference: 7.86 T-Value: 4.67 Sample Size: 7 P-Value: 0.002 Output summary: We reject the null hypothesis that the Post-Test scores are equal to or below the Pre-Test Scores. 74